This is one of the most irritating build errors a developer might see, especially while setting up an existing .NET application on a fresh machine.
I have found one solution that works always. Go to "C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files" ( Modify the framework version, root directory etc according to your environment. ) Now, right-click and open up the Security tabs for this folder. Grant "Full control" to the 'users' group. (Modify will also work, thought I didn't try it out) However, it this is a security problem for your environment to grant full control to all users in Users group, another Solution could be to put the concerned assemblies lo GAC which I avoided as it was not logical to put the assemblies into GAC in my scenario.
IF the above solution doesn't help, you can try another way: adding your impersonate user to the IIS_WPG on the local machine should solve the problem. It grants your impersonate user the correct permissions on the server file system without having to modify the folder security and open up control to large numbers of users.
Hope this helps.