Shaishav,
A formset is an integrated set of forms that are treated as one by VFP. You cannot make a form in a formset modal as to the other forms in that formset. You can make it modal related to anyother forms or formsets though.
It sounds like FormA, FormB, and FormC should really be separate forms and not part of a formset. That way FormB and FormC can be modal and when they are running FormA will not be able to be given focus.