>>>Use the code given in the link:

 <system.webServer>
>>>    <staticContent>
>>>      <remove fileExtension=".woff" /> <!-- In case IIS already has this mime type -->
>>>      <mimeMap fileExtension=".woff" mimeType="application/x-font-woff" />
>>>    </staticContent>    
>>>  </system.webServer>

>>>
>>
>>That code was already in web config. I just need the same for extension woff2.
>
>Then you just need to duplicate it and add some '2''s ?

That's what I did. The only question is what should I use for mimeType. I originally tried using woff2 there too and got a different message. So trying without 2 in the mimeType now.

UPDATE. Seems to be working now.