Forum Discussion
Error using data imported through DataFlow when using pyspark
Since Sunday (13-11-2023) I have been getting this error message on data imported through DataFlows. Previously they have been working. Has anyone else experienced this issue? SparkRuntimeExcep...
Still seeing issues with this....not connected to gateway and using DFG2 to connect to REST API and copy the data to a table in the lakehouse.
Any updates on this issue? it makes DFG2 completely unusable.
I think i got the issue resolved. I'm not sure what exactly fixed it, but I removed a previous connection to an on prem gateway in my DFG2 even though the current flow wasn't using a gateway at all. Then, I re created the dataflow in a new DFG2. Something about the connection to the gateway was still being written to the files even though I wasn't using the on prem gateway connection. Weird.