Of course I can't speak for every crew and set. . .
In my experience as an IA camera operator, we all work for the Director. The Director would not be 'asking' permission in any case in regard to using the camera. As long as the company has followed union rules and hired the required crew for each camera on the job then it would be unlikely that a complaint would taken very seriously. The Director can simply say "I am operating the camera" and that is that. Imagine telling Mr. Speilberg not to touch the camera:)
I have worked with Directors who like to operate the camera at times and in that situation I try my best to support that Director in any way I can - such as making sure a handheld camera is set up in a user freindly way or clearing trip hazards from the path. I would also watch the monitor if possible to let the Director know if there was any kind of problem (c-stand in shot?) or maybe a possible improvement.
I have seen a Director take over operating a camera when the operator has not gotten the shot like the Director wants - and that is an ego crushing scene for sure. But we are here at the discretion of the Director and DP so a thick skin can be an important asset as a camera operator!
The more common situation I have seen is that a DP will hire an operator to 'stand by' while the DP does the operating. In some cases this might be a pretty good deal - I would jump at the chance to watch Bob Richardson work and get paid for it to boot.