Lakehouse SQL Point shows different Data from Notebook SQL

[ALHU]

Hello,

My problem is then when i'm overwriting an existing Lakehouse Table with a Datapipeline. The correct content is shown when i querying the sql endpoint. However when i want to query the content with a notebook using "Select * From" it shows me the old Data. It used to work until about 2 weeks ago. Does anybody has some suggestion for me?

Thank you for your Help!

[GilbertQ]

Hi @ALHU 

Can you make sure that you are using SparkSQL or PySpark to ensure that you are reading the data from the latest version of the Parquet file?

[ALHU]

To provide more Details i have a job that copies Table A from a sql database to my BronzeLakehouse.A in MS Fabric. When i use SELECT * FROM BronzeLakehouse.A in my SQL Endpoint view get the expected results, however when i connect my Notebook to BronzeLakehouse and try to use spark.sql("SELECT * FROM BronzeLakehouse.A") it does not work as it used to be 2 weeks ago

[ALHU · Accepted solution]

Hi Guys,

Thank you for your answers!

They issue was when not using a schema for the copy job, which i adjusted in the json editor.

[v-saisrao-msft]

Hi @ALHU,

Thanks for sharing the solution! Please consider marking your reply as the accepted solution this will help other community members who might encounter the same issue.

Thank you.

[v-saisrao-msft]

Hi @ALHU,
I wanted to check if you had the opportunity to review the information provided by @GilbertQ . Please feel free to contact us if you have any further questions. If the response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.
Thank you.