Dave S wrote:
Who says he has no authority in class E or G airspace?

Who says he DOES?

91.123(b) does. If he's ATC, and he issues an instruction, this
regulation obliges you to comply. There's no indication here (or
anywhere that I'm aware of) that you have to be in "his airspace" in
order for his instruction to be authoritative.