I believe RCAGENT.EXE is the driving engine of the remote control piece, so if you set a Software Restriction policy for those computers to disallow RCAGENT.EXE from running, that should disallow SCCM from emote control connections.
Disabling the remote control component at the server level is an all-or- nothing proposition unforunately.