Forum Discussion
Helper IVCan't create a Lakehouse destination
I get closer to the solution, if I find some time I'll try to replicate... It looks that the problem appears when you don't have any meaningful transformation in the Dataflow. Meaning: just add a source and try to add a destination and then the problem appeared for me. As soon as I added a transformation I could add the destination successfully. But as I said, I need to replicate that again... And in my case the Dataflow ran for more than 1 hour for the WWI fact table (about 50mio records, just reading the records without transformation but with staging enabled, no destination) which I think is REALLY slow...
Microsoft EmployeeWould you mind collecting some diagnostics so we can look into this further
1. in the Options -> Diagnostics window, copy the session ID and share it here
2. If possible, repro the problem and export the browser network logs. Share it with me via DM or email (pqian at microsoft). The network log contains active tokens so take care of not sending it publicly.
TePeI get closer to the solution, if I find some time I'll try to replicate... It looks that the problem appears when you don't have any meaningful transformation in the Dataflow. Meaning: just add a source and try to add a destination and then the problem appeared for me. As soon as I added a transformation I could add the destination successfully. But as I said, I need to replicate that again... And in my case the Dataflow ran for more than 1 hour for the WWI fact table (about 50mio records, just reading the records without transformation but with staging enabled, no destination) which I think is REALLY slow...
- pqian_MSFT
Interesting observation, that is definitely not expected - if you manage to reproduce the issue again, make sure you share the session IDs and/or network traces
- pqian_MSFT
BTW, if you do not need staging, please turn it off, it'll help in speeding up the overall ingestion speed since we do not need to copy the data twice.