Forum Discussion
Microsoft EmployeeLakehouse table schema not updating at dataflow refresh
Solution: As at 13th Dec 2023 this fix works: When your query schema changes (e.g. add or delete new column ) before clicking 'Publish' to lakehouse, go into data destination settings (cog bottom right), click 'Next' into 'Choose Destination Target' - and this is the critical bit - ensure 'New table' is selected (even though you know that this table already exists in your Lakehouse! I know, so intuitive right?!). Ensure your destination lakehouse and table name are unchanged, click 'Next' which should display a message to the effect of 'your schema has changed', (you might need to check the box next to your new column(s) to include them in the schema) then save and publish. Please note this method does not work on my historical dataflows/tables - only new ones. To get this to work on your older tables you will need to delete them from the lakehouse, re-publish them from your dataflow, then from that point on you should be good for future schema changes.
Super UserHave you opened the proxy to the required destinations?
Dataflow fails to read from the lakehouse: Solution:
The firewall rules on the gateway server and/or customer's proxy servers need to be updated to allow outbound traffic from the gateway server to the following:
Protocol: TCP
Endpoints: *.datawarehouse.pbidedicated.windows.net, *.datawarehouse.fabric.microsoft.com, *.dfs.fabric.microsoft.com
Port: 1433
Advocate IIII'm not sure I can do that as that would be managed by my IT department but I'm not sure that would determine if I can have the "next" button enabled for the solution stated above. There doesn't seem to be an issue of "dataflow fails to read from lakehouse" here, it's just the part of having the new column being displayed in the destinatoin editor.