Forum Discussion
Restrict rows retrieved by query?
I am evaluating Power BI for our user community, and it looks great. However, there's a problem I can't seem to get around: How to easily restrict the rows returned by a query? Can anyone point me to an easy, end-user-friendly solution?
The SQL Server table that contains the data of most interest to users contains over 123,000,000 rows, and each row contains 140 columns - a couple of KB per row in CSV format. So it is not possible or practical to retrieve the entire table in Power BI and do the filtering there. What I need to do is allow the user to easily create parameters (several, both date ranges and text "Like" clauses), set their values, and send the query with parameters to SQL Server to retrieve the several hundred rows they are interested in.
I could easily do this by modifying the SQL query, but most of our users can't. And they need to be able to create a query with the parameters they need - there are several dozen columns they might want use, alone or in combination, to restrict the rows retrieved. We've given them other tools that require using SQL to get the data, and it has not proven useful or productive.
I can do this fairly easily using a Microsoft Query connection in Excel and our users probably could, too, but that has some issues. I'm hoping Power BI would be a good end-user tool for this data retrieval and analysis need, and I'd like to give them the capabilities of Power BI as well. Is this possible?
Thanks for your help.
Sam
The single-qoutes is needed as part of my SQL query to filter a string column in the SQL database
The 1st double-qoute stop the "free text query" and allows to insert a parameter
Ampersand is needed to merge the parameter with the SQL code around paramater
The 2nd double-qoute continue the "free text query"
I hope this explains the code.
Now back to your problem... 19 foreign keys "scary face". I assume that you users will need to be able to filter several parameters at once? This will force you to type SQL code from now until you retire. As I see it you will have to create 19*18*17*16*15*14*13*12*11*10*9*8*7*6*5*4*3*2 combinations of your code + create 19 parameters lists with a complete set of values for your user - and this will only give them the option to select a single value per parameter. I don't think this is the right path for you. You will need to many different variations of your code because consider that your user only select a parameter in a few of your lists, then the rest will be blank but if you query your SQL with WHERE a.Country = '' then you will only get the customer with no country value (blank), but what you really want is not to filter the country column in your SQL table, so you will somehow have to handle all the combinations of posible where clauses you need.
With single value per parameter I mean they will not be able to select both US and UK customers but only one of them - unless you make you parameters free text values. Then you will be able to chance the SQL to use IN instead of =, but your users will only get the wanted result if they learn to write the right text - using my example from before this would be 'US', 'UK' and your where clause should be "WHERE a.Country IN (" & CountryParameter & ")"
15 Replies
As far as I understand, you would want something like, "picking the columns before loading rows"? It does seem possible in the future, but we don't have a direct way of doing this yet. I couldn't find a request for it here, so you might need to create a new one.
In the query editor, we don't load more than 1000 rows, so that you can preview the query and do your operations easier. But if your table has so many columns that even 1000 rows would be too much, you can cancel (on the bottom right side) and restrict the number of rows to something much lower, like 10. (Home tab of Query Editor > Keep Rows > Keep Top Rows)
I'm sorry, I don't understand your response. What approach is "better than 'here is all the data, ask yourself'"?
What I am looking for is the capability to create custom queries in Power BI as the basis for reports and visualizations - not limiting it to generic queries or pre-defined queries built by a guru, but letting end users build their own queries, reports and visualizations without having to be an expert in SQL, DAX, M or some other specialized language or feature set in order to limit the data they look at. SQL Server should do the work of retrieving the appropriate rows (which it can do quickly and efficiently), the tool should not require the user to bring in all of the data and create complex mechanisms to filter it down to the data they want. Can you imagine refreshing a query on a regular basis, if it retrieves all rows and then filters them? Ugh!
But it seems that's just not possible with this tool. Is that correct? Too bad, the capabilities of Power BI are great for end-user analysts to define and publish useful management and staff visualizations, but the learning curve is too steep (or the tool is not appropriate) if there's no simple way to do such things.
Am I missing something? I'm new to the product, so that's very possible - and the purpose of this question. I'd like to use Power BI if it has the needed capabilities.
Thanks for your response and help.
SamTrexler Is there some reason that you haven't gone down the modeling route? Using SSAS Tabular for instance? Everything you describe as it relates to an end user filtering/slicing and dicing without the need for complex queries is the purpose of developing a BI layer to your relational data. The end result is a model that contains the pertenant business information, and gives the end users the ability to search anything they want in the model.
