Forum Discussion

trutz's avatar
trutz
Advocate IV
2 years ago
Solved

MS Fabric Notebook: Error when using dataframe.to_json() with Fabric Lakehouse storage

Hi all, I'm getting the error below when using pandas dataframe method to_json() with dataframe.size() "4263942". Dataframes with lower size can be saved without issue. The filepath and name are j...
data engineering
Data Integration
one lake
  • trutz's avatar
    trutz
    2 years ago

    Hi,
    I switched from JSON to Parquet and had no issue with storing the data. So for the moment I circumvented the issue.

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi ,


Apologies for the issue you are facing. To eliminate the data issue, you can try validating the json to see if data is in proper json format. This might help us in identifying if the issue is from data or from ADLFS package.

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi trutz ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet . In case if you have any resolution please do share that same with the community as it can be helpful to others . If you have any question relating to the current thread, please do let us know and we will try out best to help you.

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi trutz ,
    We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
    In case if you have any resolution please do share that same with the community as it can be helpful to others .
    If you have any question relating to the current thread, please do let us know and we will try out best to help you.

    • trutz's avatar
      trutz
      Advocate IV
      2 years ago

      Hi,
      I switched from JSON to Parquet and had no issue with storing the data. So for the moment I circumvented the issue.