So, next time, the second time they ask, I should just say, "I already told you. Now go away". Is that what you're saying?

If you think about it, repeating to an adult what you just said is kind of futile, isn't it?
Now and then, saying nothing at all is the most eloquent choice of words.