Ok, I got it to work.

When I create a new Web site, there is an additional step that has to be performed. Not only Authentication Basic Settings has to be set to Connect as Application pool but so is the Open Feature from the upper right panel of that option, which is very not easy to find BTW, so to set the Anonymous Authentication in there as well to Application pool identity.

I never understand this double setting we have to do. This should be in only one place.