Access to Custom Groups for Remote Office Admins

I have a situation where we have administrators for certain Remote Offices, they only manage that remote office and nothing else. These remote offices need access to certain custom groups, when they log in they can see the Remote Office and machines for that RO (as expected) but do not have access to the custom groups in the drop down list (see picture)

I have created an account to test with and assigned it admin privileges to a remote office. This seems to be some sort of permission issue. The only way to get that admin to see the custom group option in the "Select column" drop down is to make them an admin of All Computers under "Computers to be managed"

I need a way to limit access to their remote office ONLY but be able to have them manage custom groups as well. Makes no sense that they can create and modify custom groups but then are not able to select them in the drop down.

I know you can also make them admins of Static Unique Groups. But this doesn't really work for us as we need them to manage an entire remote office and not just a group.

Thanks.