Forum Discussion

pavarayaaa's avatar
pavarayaaa
Regular Visitor
2 years ago
Solved

Error when writing data into schema enabled lakehouse using notebooks

I was try using PySpark and SparkSQL to write/create data/table into my lakehouse and it gave me an error. I was try using both .save and .saveAsTable options, but it didn't work. When I was using .s...
Lakehouse
  • Anonymous's avatar
    Anonymous
    1 year ago

    Resolution: The issue is caused by the use of internal APIs. In some cases, the internal APIs are returning a 403 when routed to a datacenter where the user is not present. The fix is to move to the public APIs. The code is already in there, but we need to provide the switch from the workload code.

     

    %%pyspark

    !echo "spark.trident.pbiApiVersion=v1">>/home/trusted-service-user/.trident-context


    The fix would be deployed within 3 to 4 weeks in all production regions.
     

arlindTrystar's avatar
arlindTrystar
Advocate I
1 year ago

This does not address the problem. Because the user is getting bad request error and forbidden error, not AnalysisException error.

 

Anonymous's avatar
Anonymous
Not applicable
1 year ago

Hi pavarayaaa ,

 

Thanks for the reply from arlindTrystar .

 

If none of the workaround we provided works, perhaps creating lakehouse without schema enabled is your last option.

 

Best Regards,
Yang
Community Support Team

 

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!