Incremental refresh on Azure Databricks datasource

[Victormar]

Hello community,

I am trying to work around the icnremental refresh using Azure Databricks as a source. The thing is that up until this point, we are working as well with parameters, to select different databases inside the clusters.

More specifically, I have 2 clusters, and I have 2 databases in one cluster, and another one in the other one.

We have a function that points to the specific database based on the selected parameter. That allows us to run a sql query to the database:

I saw a video from "Guy in a cube" where they use Value.NativeQuery to be able to perform a native query, they essentially point to the database and modify the applied step. I assume I cannot follow this path because I don't point straight to a database, but rather a cluster, and then a database inside, but maybe I am wrong.

Video: https://www.youtube.com/watch?v=8hjdOCni_ZY

Has anybody faced this issue? Thanks for any input!

[Anonymous]

Hi @Victormar ,

Based on your description, I found some possible approaches to this issue, you can try to use Databricks.Query to load the table or still use Value.NativeQuery to get it. Of course you can also find the gear button next to source on the right side of the power query interface, you can click on it and go to the navigation interface, he will show all the tables and you can import the tables you need here.

Regarding Incremental Refresh in Power BI with dat... - Microsoft Fabric Community
Connect Power BI to Azure Databricks - Azure Databricks | Microsoft Learn
Databricks Power BI Connector Native Query | Databricks Blog

Best regards,
Albert He

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly

[Victormar]

Hi @Anonymous ,

Thanks for your answer. I have been reading through the links, and tried to implement some of them. I had tried a similar approach before.

I think my "problem" is that I want to use a parameter to select which database I want to point to. I have DEV, QA & PROD, and it cannot be translated to a native query I believe. 

Best Regards,