Forum Discussion

SFDucati's avatar
SFDucati
Helper I
3 years ago
Solved

Dataflow Gen2 - Online Power Query Experience Works / Loading to Lakehouse Fails

Hi everyone...   I will try to be as specific as possible.  I followed the tutorial provided by Microsoft, creating a Fabric Workspace, creating a new Lakehouse, and then creating various Dataflows...
data engineering
  • SFDucati's avatar
    SFDucati
    3 years ago

    Thanks for responding.  I received a text from support indicating there was an updated Gateway available to download that would correct the bug.  I downloaded the new Gateway (ver 3000.178.9) and then refreshed my Dataflows Gen2 that were not working.   They published to the Lakehouse!  So that seemed to solve the problem.  

     

    Except, they did not bring the date columns in my Dataflows Gen2 into the tables in the Lakehouse.  So, after some trial and error, I discovered that I needed to delete the published tables in the Lakehouse, then re-publish them.  

     

    So if anyone else has had my issue, the newest release of the Gateway resolves the issue.  But, you will need to delete existing tables in your Lakehouse and re-publish the Dataflows as a new table in the Lakehouse.  

     

    Hope that helps.

SFDucati's avatar
SFDucati
Helper I
3 years ago

Thanks for responding.  I received a text from support indicating there was an updated Gateway available to download that would correct the bug.  I downloaded the new Gateway (ver 3000.178.9) and then refreshed my Dataflows Gen2 that were not working.   They published to the Lakehouse!  So that seemed to solve the problem.  

 

Except, they did not bring the date columns in my Dataflows Gen2 into the tables in the Lakehouse.  So, after some trial and error, I discovered that I needed to delete the published tables in the Lakehouse, then re-publish them.  

 

So if anyone else has had my issue, the newest release of the Gateway resolves the issue.  But, you will need to delete existing tables in your Lakehouse and re-publish the Dataflows as a new table in the Lakehouse.  

 

Hope that helps.

SidJay's avatar
SidJay
Microsoft Employee
3 years ago

When the schema changes, you have to edit the settings for your Output Destination (which allows you to edit the mappings). You can also delete the table and start afresh as you suggested.

 

There isn't a mode that drops the table to automatically create a new schema. Even if such a mode is added, it could be disruptive to downstream artifacts. However, we are considering such a mode for convenience.

 

Thanks