Forum Discussion
Postgres DirectQuery "Requires Data Gateway"
- 6 years ago
Yes Anonymous - if you have data that requires the gateway, like the CSV files, then it can make the issue more complex.
Can you get it working if you remove everything but the Postgre data? Just whack all of that other stuff and save as a new file. Then add data sources back until you hit the one causing the issue. You didn't say where your CSVs were. A gateway is needed for local/network files or SharePoint On Prem. Should not be needed for SharePoint online, Data Lake, and other online resources.
But mixing and matching can still cause issues that are frustrating to trouble shoot. That is why I suggest do 1 data source, then add until it breaks and fix the breaks one at a time.
Given the error you have, something in your report thinks it needs a gateway to access the "data source file"
Did you enable query folding in the Advanced View? See this configuration article for details.
- Anonymous6 years agoNot applicable
Nope, I've never heard of query folding.
Thank you for the article, but would you mind elaborating on it? I don't understand where to supply the the Value.NativeQuery statement in the advanced editor (is it line 1? is it in the source step? do I need to add it for each separate query?) and what args it needs. I've never needed the advanced editor til now.
Thank you very much for the reply, though!
- edhans6 years agoCommunity Champion
Query folding is the process Power Query uses to create a SQL statement and sends it to the server. So as an example, I imported the Purchasing table from the Microsoft World Wide Importders sample database on SQL Server using basic transformations in Power Query - removing columns, adding a conditional column. the M code is:
let Source = Sql.Databases("localhost"), WideWorldImportersDW = Source{[Name="WideWorldImportersDW"]}[Data], Dimension_Customer = WideWorldImportersDW{[Schema="Dimension",Item="Customer"]}[Data], #"Removed Other Columns" = Table.SelectColumns(Dimension_Customer,{"Customer Key", "WWI Customer ID", "Customer", "Bill To Customer", "Category", "Buying Group", "Primary Contact", "Postal Code", "Valid From", "Valid To"}), #"Added Purchasing Power" = Table.AddColumn(#"Removed Other Columns", "Purchasing Power", each if [Buying Group] = "N/A" then "Standard" else if [Buying Group] = "Tailspin Toys" then "Standard" else "Discount", type text) in #"Added Purchasing Power"Power Query though "folds" all of that. It doesn't acutally do any of that in the PQ mashup engine. It analyzes it, and send the following SQL statement to the server:
select [_].[Customer Key] as [Customer Key], [_].[WWI Customer ID] as [WWI Customer ID], [_].[Customer] as [Customer], [_].[Bill To Customer] as [Bill To Customer], [_].[Category] as [Category], [_].[Buying Group] as [Buying Group], [_].[Primary Contact] as [Primary Contact], [_].[Postal Code] as [Postal Code], [_].[Valid From] as [Valid From], [_].[Valid To] as [Valid To], case when [_].[Buying Group] = 'N/A' then 'Standard' when [_].[Buying Group] = 'Tailspin Toys' then 'Standard' else 'Discount' end as [Purchasing Power] from ( select [Customer Key], [WWI Customer ID], [Customer], [Bill To Customer], [Category], [Buying Group], [Primary Contact], [Postal Code], [Valid From], [Valid To] from [Dimension].[Customer] as [$Table] ) as [_]The server does all of the work and simply sends back the relevant data. You can do pivots, grouping, merges, appends, etc. Tons of stuff will fold.
For Direct Query to work, Query Folding must be allowed to happen. Direct Query sends SQL statements to the server and returns immediately those results. There is no other work done or allowed in the Power Query mashup engine. SQL to SQL statements, that is it. As soon as you "break folding" Direct Query breaks. Import Queryies will still work. They will do folding to the point the folding breaks, then do the rest internally in the gateway/mashup engine. Direct Query though doesn't allow it. 100% folding, or you have to switch to Import for that query.
I don't have access to a Postgre database to test with, but I would assume in the M code I have posted above for SQL Server, the PostGre connection would have its own code, and you'd need to modify it to enable query folding per that article link. I'm not sure why - SQL Server it just works - there is no flag to set. But there must be something with Postgre that it need to be able to be turned off and on as needed.
And you would need to add it to each query to your server. It is not a global setting but a per-query setting.
- Anonymous6 years agoNot applicable
Thank you for the thorough answer and clarification edhans!
I got the native query working last night in desktop, enabled the query_folding flag, and it returns exactly what I want.
I checked the native query step and it doesn't appear anything like yours does; it's just a single in-line SQL statement.
However, checking in on the webapp this morning I still see the same gateway error.
What's even more confusing is that a coworker created a small status-view report of some aggregated database values with DirectQuery and his just worked. And he's touched Power BI a total of 2 days. This is why I'd rather not go the data gateway route yet (also because it's another level of hassle).
A difference between my report from his is that I have some csv fact tables related to the direct queries. Would Mixed-Storage mode/Composite-Modeling be the problem?