I think this cartoon is very relevant here.
Yes, I do mind if you distract me, while I’m doing something else!
Attribution: I couldn’t find the orginal post on Instagram, but the cartoon is by SteinbergDrawsCartoons, and his website is here, and his cartoons are prety good IMHO.
How to establish this boundary without pissing people off?
Trick question! It is not possible because James fully understands and respects this but he wants to ask you something. It is not urgent but he doesn’t want to go to his desk to type it in a mail or message and patiently wait for you to see it. Just asking you is much easier, see.