Forum Discussion

anikah123's avatar
anikah123
Regular Visitor
2 years ago
Solved

ADLS vs Databricks for Power BI reporting, seeking advice and best practices.

Hi everyone,
 
I'm a business analyst currently facing a decision on how best to develop a Power BI report. I need to choose between using Databricks or Azure Data Lake Storage (ADLS) as the data source on the transformed data in csv or excel file format from databricks into ADLS. My main concern is about the capability to write SQL queries, especially since I've heard that it might not be possible with ADLS (file system). Our IT team is suggesting we use ADLS for this task.
 
Could anyone share insights on:
The pros and cons of using Databricks compared to ADLS for report generation?
 If using Databricks, does the cluster need to be up all the time for Power BI report refreshing?
 What are the best practices for setting this up?
 
Any advice or experiences shared would be greatly appreciated!
 
Thanks in Advance
Interesting Links
Need Help
Tutorial Requests
  • vanessafvg's avatar
    vanessafvg
    2 years ago

    personally i wouldn't use adls unless the data is small volumes and perfect  but it really depends on what you need to do and how much cleaning and transforming you need to do to the data, and the depth and breadth of your data.

     

    databricks will allow you to write sql but also do transformation up stream which is often more efficient and faster with spark technology rather than using power bi compute for some tasks which might be memory and cpu hungry, plus doing it in databricks there is always the opportunity to reuse common data.    you should be able to trigger the databricks cluster even if its off.  Databricks will also be faster no doubt

     

     

1 Reply

  • vanessafvg's avatar
    vanessafvg
    Icon for Community Champion rankCommunity Champion
    2 years ago

    personally i wouldn't use adls unless the data is small volumes and perfect  but it really depends on what you need to do and how much cleaning and transforming you need to do to the data, and the depth and breadth of your data.

     

    databricks will allow you to write sql but also do transformation up stream which is often more efficient and faster with spark technology rather than using power bi compute for some tasks which might be memory and cpu hungry, plus doing it in databricks there is always the opportunity to reuse common data.    you should be able to trigger the databricks cluster even if its off.  Databricks will also be faster no doubt