Forum Discussion

pavarayaaa's avatar
pavarayaaa
Regular Visitor
2 years ago
Solved

Error when writing data into schema enabled lakehouse using notebooks

I was try using PySpark and SparkSQL to write/create data/table into my lakehouse and it gave me an error. I was try using both .save and .saveAsTable options, but it didn't work. When I was using .s...
Lakehouse
  • Anonymous's avatar
    Anonymous
    1 year ago

    Resolution: The issue is caused by the use of internal APIs. In some cases, the internal APIs are returning a 403 when routed to a datacenter where the user is not present. The fix is to move to the public APIs. The code is already in there, but we need to provide the switch from the workload code.

     

    %%pyspark

    !echo "spark.trident.pbiApiVersion=v1">>/home/trusted-service-user/.trident-context


    The fix would be deployed within 3 to 4 weeks in all production regions.
     

Anonymous's avatar
Anonymous
Not applicable
1 year ago

Resolution: The issue is caused by the use of internal APIs. In some cases, the internal APIs are returning a 403 when routed to a datacenter where the user is not present. The fix is to move to the public APIs. The code is already in there, but we need to provide the switch from the workload code.

 

%%pyspark

!echo "spark.trident.pbiApiVersion=v1">>/home/trusted-service-user/.trident-context


The fix would be deployed within 3 to 4 weeks in all production regions.