I found the right MSDN reference

BUG: Visual Basic Application Using HtmlHelp API Crashes
Q244140

before issuing QUIT, I do the following

HH_CLOSE_ALL=18
HtmlHelp(0, "", HH_CLOSE_ALL, 0)

and problem goes away.