Slow Web Access from Remote PC to OpManager

We are having an issue with the web interface for OpManager being very slow when the computer you are using is not on the same network as the OpManager server. It takes the login page a long time to appear and access within the program itself is pretty slow. Everything works, it is just slow.

Dashboards that are embedded in other pages take a while to load as well when remote to the office, but load almost instantly at an office computer. We have checked our firewall and load balancer settings, moved the OpManager VM to a different host at a different site, but nothing seems to make a difference. We are at a loss as to why it is so much slower when remote to the office. If we VPN to the office, it is fast once again, so that eliminates any sort of ISP bottleneck in between my house and the office (for example).

Any ideas what else we could check?