topic Re: Possible to Query Keywords in SQL Statement Datasource Using API? in Developer

Possible to Query Keywords in SQL Statement Datasource Using API?

BrandedSaiyan — Thu, 28 Jul 2022 20:11:37 GMT

Hello Power BI Community,

I recently gained access to the Power BI REST API. I was hoping there would be a way to do a keyword search on the SQL Statement that is part of part of a SQL Server datasource for a dataset for a report. See screen shot below for where I am refering to.

Our issue is that instead of using views or stored procedures in our SQL statements, 90% of our reports are using inline SQL code. It's almost impossible to do a dependecy check or impact analysis with our reports written this way and I was hoping the REST API would allow us to do an organization wide search of the SQL statement in all datasources across all datasets so we could search for specific tables and objects.

Using the Datasets - Get Datasources endpoint only gets me the datasource type and connectiondetails (being the server and database) for a given datasource, but not the sql statement.

Is there any way that I can query for the SQL statement?

Edit: Adding @Greg_Deckler since his expertise has helped me in the past 🙂

Thank you

Re: Possible to Query Keywords in SQL Statement Datasource Using API?

lbendlin — Sat, 30 Jul 2022 00:55:58 GMT

You're in for a treat. You will want to learn about XMLA endpoints, and DMX queries. You can run these against all your datasets and extract the complete M code for all connections (including the embedded SQL).

NOTE: This is only valid for Import Mode connections. For Direct Query you have no control over, or visibility of the queries that are generated on the fly for each visual.

Re: Possible to Query Keywords in SQL Statement Datasource Using API?

BrandedSaiyan — Mon, 01 Aug 2022 12:51:45 GMT

Thank you, do you have any helpful links or resources you'd reccomend?

Re: Possible to Query Keywords in SQL Statement Datasource Using API?

lbendlin — Mon, 01 Aug 2022 13:23:11 GMT

There's DAX Studio , but beyond that the air is pretty thin. Mostly trial and error.

Re: Possible to Query Keywords in SQL Statement Datasource Using API?

BrandedSaiyan — Mon, 01 Aug 2022 14:06:07 GMT

I have DAX studio, but I'm unfamilar with any function or code that extracts the M code. I'll do some digging, was just hoping to be pointed in the right direction. Thanks again.

Re: Possible to Query Keywords in SQL Statement Datasource Using API?

lbendlin — Mon, 01 Aug 2022 15:08:12 GMT

In DAX Studio connect to your dataset (local or in service)

Run two queries (individually)

select * from $SYSTEM.TMSCHEMA_PARTITIONS

select * from $SYSTEM.TMSCHEMA_EXPRESSIONS

Grab the QueryDefinition column from the first and the Expression column from the second

Rinse and repeat for all your datasets.

Re: Possible to Query Keywords in SQL Statement Datasource Using API?

BrandedSaiyan — Mon, 01 Aug 2022 21:04:38 GMT

Thank you! Do you know if I'm able to run this against a shared power bi dataset? Or if it can be ran more in bulk rather than foiir just one dataset at a time? Much appreciated.

Re: Possible to Query Keywords in SQL Statement Datasource Using API?

lbendlin — Mon, 01 Aug 2022 22:15:56 GMT

You have to run it against each dataset (that you have access to) separately. However you can sort of automate that by running the queries from PowerShell.

Re: Possible to Query Keywords in SQL Statement Datasource Using API?

BrandedSaiyan — Wed, 07 Sep 2022 16:16:09 GMT

Circling back here. I've been unable to find a way to run these queries from powershell. Any help would be greatly appreciated.

Thanks!

Re: Possible to Query Keywords in SQL Statement Datasource Using API?

lbendlin — Wed, 07 Sep 2022 16:26:44 GMT

Invoke-ASCmd -Server "powerbi://api.powerbi.com/v1.0/myorg/<workspace name here>" -Database "<dataset name here>" -Query "select * from `$SYSTEM.TMSCHEMA_PARTITIONS" | Out-File -FilePath c:\users\xxx\Downloads\dmv.xml

Re: Possible to Query Keywords in SQL Statement Datasource Using API?

BrandedSaiyan — Wed, 07 Sep 2022 16:54:18 GMT

I really appreciate all your help, going to give this a try and get back to you! Thank you!

Re: Possible to Query Keywords in SQL Statement Datasource Using API?

BrandedSaiyan — Tue, 02 May 2023 20:34:51 GMT

This ended up working great!

Re: Possible to Query Keywords in SQL Statement Datasource Using API?

lbendlin — Tue, 27 Sep 2022 20:17:56 GMT

Note that there is a subtle difference between "PARTITIONS" and "EXPRESSIONS".

Re: Possible to Query Keywords in SQL Statement Datasource Using API?

BrandedSaiyan — Tue, 27 Sep 2022 21:23:18 GMT

Thanks for pointing this out. I'm noticing for a report I'm currently working on I have 3 SQL Server datasources showing under PARTITIONS and 1 SQL Server source showing under EXPRESSIONS. I'm not able to determine the difference or why one shows under expressions. Any further insight is appreciated.

Re: Possible to Query Keywords in SQL Statement Datasource Using API?

lbendlin — Tue, 27 Sep 2022 21:31:23 GMT

Both are queries. Partitions are loaded into the Vertipaq engine. Expressions are not loaded into the engine but may participate in the data pull, especially when other queries depend on them (even when you explicitly state you don't want to refresh them !).

You can tell by the italicization of the query name in Power Query.