I see this feature advertised on the ManageEngine site they even host a video tutorial for it, but I don't have the option in my instance. It's supposed to be under Automation > Custom Actions > Custom Functions, but only that tab is absent.

Was this feature removed or is there a setting somewhere so I can enable it? Does it require a special license?

The tab is referenced in the documentation here: https://help.sdpondemand.com/custom_actions