Check this link:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/releasemutex.asp

It says:
- the handle must have the MUTEX_MODIFY_STATE access right
- the function fails if the calling thread does not own the mutex object

There is also example code on that page. Hope it helps.