I played around with it and got it to work. Not very user friendly at all.
I went to page 4 and added a section break and told it was "continuous" (Note, I don't see how what word calls it fitting what it means).

Then I went to page 1. I made a header and just typed "Header 1" so to have one. I went back to page 4 and edited the header there. I clicked on the "Same as previous" icon on the header tool bar. I'm not sure why you do that, but that's what help said to do.

I then edited it to include a page number. It was 1 because I had put a section break there. Then I went to page 1 and deleted that header and it went away on pages 1 to 3.

Let me know if you have problems or it looks like I left out something.