Dataflow refresh job failed: Job instance failed without detail error

Hi all, 

I just wanted to add a few things. I've seen a number of different 20302 errors on dataflow refreshes. Some I have figured out, some I haven't. 

I just saw this specific version today: 
"Dataflow refresh job failed with status: Failed. Error Info: { errorCode: JobInstanceStatusNotFound, message: Job instance not found, requestId: 1702ea03-eada-4865-96c2-73c4f9c6ee9b }"

I looped through all the dataflows (Gen 2 CICD) ```````````````````````````````````````````````````````````````````````````` in a workspace (all created by me, with my own connections, no one else has touched them, and 2 out of 8 failed with that message. I checked out the recent runs of those dataflows. interestingly, these 2 dataflows were still actively running. I thought maybe it was some sort of resource error or timeout error so I was looking at the durations. One of the two that caused that error in the pipeline happened after 6m 10s. The other after 5m 14 seconds. However one of the successful runs had a duration of 5m 14 seconds. I checked the job instance status later for these dataflows, and both show failed, though the logs of the runs themselves show everything succeeded. 

A couple of the other errors I've seen, one with a somewhat helpful error message, one very much not. 

The first has an error something like notpublishederror..I can't remember exactly. With Gen2 CICD, some folks might not yet realize that when you deploy them to a new workspace you have to 1: Manually update the data destination. There is no functionality yet to use deploymen t parameters for datasources or data destinations for Gen 2 CICD. And 2: You must Save and Run (the new "publish") the CICD once deployed. All of this, every time. Kind of annoying but it works. There just doesn't seem to be a good way to handle deployments with varying data sources and destinations by environment yet, despite these being CICD versions. 

The other error will simply say "Invalid Request, Unexpected Dataflow Error". What I've learned so far is that the dataflow activity will work fine with Gen 2 CICD dataflows if you select them from the dropdown statically. Which, of course, isn't a very realistic use case. When running the dataflows dynamically and supplying the dataflow Id at runtime, the activity doesn't know that it's a CICD version, and it's missing a specific dataflow type property and value that it needs in order to process these. I have found a kind of work around where you can manually edit the pipeline json and add this property to the activity object code. And it works. But, if you open that dataflow activity and work with it and then close it, it has no overwritten your code change and defaulted back to removing that property and it will fail again. There is some good information in this Reddit thread. https://www.reddit.com/r/MicrosoftFabric/comments/1khpav5/issue_refreshing_gen_2_cicd_dataflow_in_pipeline/