Backup uses non-buffer pool memory, also known as the MemToLeave memory. Se some detail at http://www.sqlservercentral.com/articles/Administration/67192/ and http://support.microsoft.com/kb/271624

>
>Ok, so in our case, the absence of hits during the overnight was probably in cause of the caching being dropped from the related table. Then, it became fast again on the upcoming hits. Also mentioned, the backup that happened two hours before might have been a factor. I assume that it required significant usage of SQL Server engine and might have caused the same situation as well.