When you said your server freezes I was making the assumption that SSMS was hanging and that maybe there was a really long timeout configured on the server to wait for database connections to close prior to restoring the backup.

If by "freeze" you mean the server is totally completely locked up - no keyboard input is accepted - database and file access is denied to workstations - the only way to get a response is a hard reset by physically pushing the reset button - then, no, I don't think switching to single user mode will fix your issue.

Have you checked hard drive contraints?

Have you checked the Windows Event Viewer logs? I'll bet your answer lies in there.