Forum Discussion

RobertCressey's avatar
RobertCressey
Frequent Visitor
2 years ago
Solved

Data Pipeline Error while decoding

Data that is written to my Datalake through a data pipeline can not be read using Spark. (even the autogenerated code when clicking open in notebook from the datalake)   Tables that have been writt...
pipelines
  • RobertCressey's avatar
    RobertCressey
    2 years ago

    From MS support ticket:

     

    Issue fixed - 

     updated to data gateway 3000.218.7 or newer.

     

    Some tables issues persisted after a refresh however we assumed this was due to the way that checkpoints are saved in the json , so we dropped all the tables and re-imported and this fixed the issues.

RobertCressey's avatar
RobertCressey
Frequent Visitor
2 years ago

From MS support ticket:

 

Issue fixed - 

 updated to data gateway 3000.218.7 or newer.

 

Some tables issues persisted after a refresh however we assumed this was due to the way that checkpoints are saved in the json , so we dropped all the tables and re-imported and this fixed the issues.

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi RobertCressey ,

    It was great to know that you were able to get to a resolution . We expect you to keep using this forum and also motivate others to do that same . You can always help other community members by answering to their queries