How do I give permission to publish websites to Azure?

Using the new Azure Resource Manager (ARM) you can use Role Based Access Control (RBAC) to grant a user a certain access on a certain Azure resource that you own.

There are 3 default RBAC roles: Owner, Contributor and Reader. If you want a user to be able to publish to a site without giving them your personal creds you should be able to add then as a Contributor to that particular site and they will be able to see that site in portal and use their own publishing credentials.

This is a much more granular approach than co-admin where the co-admin will have access to your entire subscription.

Read more about how to setup RBAC in Azure Websites and Publishing here