Forum Discussion
Way to query data using SQL and csv file
Hi NancySharma
I wrote a blog explaining one technique you can use in Power Query to generate dynamic SQL
http://radacad.com/dynamic-sql-using-power-query
Hi Phil,
I am new to Power BI,
What i see is that you made changes to data on Power BI. What i want is that i get prefiltered data from sql based on filter in CSV file so that only needed data gets loaded in power BI.
Reason is when i try to form joins and filter data in power BI, it takes a lot of time when i refresh my CSV file, it has like million records. So what i am looking for is i get prefiltered data uploaded to BI based on CSV filer that i can put in query.
Hi NancySharma
All the work should be done in Power Query (which sits before Power BI). You can query your CSV file and you will have to read all of it. Then in Power Query, you can generate a list values from the CSV file that you are interested in and finally use that list inside your T-SQL query statement in the WHERE clause.
Yes i tried this approach but apparently when i refresh my csv file this query stops working. Is there a workaround for that or any other solution.
Yes i tried this approach but apparently when i refresh my csv file this query stops working. Is there a workaround for that or any other solution.
Hi NancySharma
It shouldn't stop. Power Query should be smart enough to know that it needs to fully load the CSV table to generate the list you need for your SQL data source. If you click the view dependances button on the PQ ribbon you should see a diagram showing the flow of the logic.
I think something else might be causing an issue.