It's not possible to run a Dataflow Gen2 as Service Principal.
This is documented here: Background Jobs - Run On Demand Execute - REST API (Dataflow) | Microsoft Learn
It would...
Thanks bernhard0614 for linking my issue - it's a daily thing for me right now - whenever I deploy using git refresh or fabric cicd using the service principal, I get this pipeline SPN issue, so having to go in and manually toggle it off and on is just so disruptive to my workflow especially when doing a lot of deployments
