Forum Discussion
Error using data imported through DataFlow when using pyspark
So, I found the root of the bug and also a temporary workaround.
The bug is that Dataflows gen2 writes this in the json delta log files :
- {"protocol":{"minReaderVersion":1,"minWriterVersion":2,"readerFeatures":[],"writerFeatures":[]}}
That's the reason for this error message : requirement failed: Mismatched minReaderVersion and readerFeatures
A temporary fix is to change this string in all the _delta_log json files for the table to:
- {"protocol":{"minReaderVersion":1,"minWriterVersion":2}}
You can change the log files within the onelake in your files browser:
Thansk for the workarround.. I had the same issue.. How can I edit these files?
You can change the files with the Onelake plugin that you can install in your windows file explorer :https://www.microsoft.com/en-us/download/details.aspx?id=105222
Keep in mind that every time you execute the dataflow, you'll have to modify the new log file that has been created for this workaround. Hopefully they'll quickly fix this bug.
Thanks very much... I suppose that Microsoft is working on this.. it doesn't make sense every day doing this process (if the DF runs each day)
