Forum Discussion
Dataflow Gen2 not updating Lakehouse
Hi ina_dab
Thank you very much SudhavaniKolla3 for your prompt reply.
Make sure you have sufficient permissions. There may be a synchronization delay between the dataflow and the lakehouse, resulting in updates in the dataflow not being reflected in the lakehouse in a timely manner.
Here are some suggestions:
Ensure that the output of dataflow is set correctly to write data to lakehouse in a timely manner.
If possible, consider increasing the refresh frequency of the dataflow to minimize the delay between data updates.
Examine the logic in the dataflow to ensure that there are no redundant steps or complex calculations causing delays.
Looking forward to you sharing more.
Regards,
Nono Chen
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Hello, I am admin of my fabric environment so there is no permissions issue.
The issue really is that this specific table from the dataflow is not writing into the lakehouse. Let's call this Table A. Table A is a transformation of two other tables, one of them being Table B. This date column is in table B, it doesn't even get transformed in Table A. Table B is showing the correct date in my lakehouse, just Table A is not.
I have deleted the lakehouse table before and made sure the mapping is again correct, but I still don't get it to write correctly.
frithjof_vCommunity Champion
How are you checking what data you have in the Lakehouse?
Are you checking it in the SQL Analytics Endpoint (there may be a delay between the actual Lakehouse and the SQL Analytics Endpoint)?
How is your Power BI semantic model connected to the Lakehouse? Via Direct Lake or Import mode?
Kudos to frithjof_v ! I have a similar problem where a column shouldn't contain NULL values after the second run of the dataflow. But the column still contains NULL values when I query it through the SQL endpoint but it is filled correctly when I query the table with a Spark SQL notebook.
