You should be able to configure a minimum and maximum amount of memory for the sql server.
http://support.microsoft.com/KB/321363Jon
>We have experienced some slow downtimes on SQL Server last night and there was practically no one on the application. When we looked at the task manager, the task sqlservr.exe was taking 1,028,660 of memory usage. And, during that time, the response time was really slow. This has happened at a few occasions during the night. And, as I said, nothing was really running on that server. Is this normal?
