Forum Discussion
Power BI Embedded using Databricks sql warehouse or data lake?
We are currently looking into moving from using a SQL Server datawarehouse to something new. We have large amount of data, currently in the millions of rows used by hundreds of clients. The current...
millions of rows should not take several hours. Are you doing a lot of transforms in Power Query? Are you attempting merges?
Consider using dataflows, or flat out Parquet files. Let the data model do the work for you.
Currently the data model is connecting to a view in SQL Server to pull the information. Hence the long load times and huge memory hogging.
I am looking into enhancing it to utilize Databricks Warehouse or Direct Lake parquet files. Either one is probably a huge improvement over what we have today.
Currently the data model is connecting to a view in SQL Server to pull the information. Hence the long load times and huge memory hogging.Optimize the query behind the view. Add indexes as needed and make sure the statistics are up to date.