Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
6 years ago
Solved

Postgres DirectQuery "Requires Data Gateway"

I have recently moved from importing a ton of CSV's to DirectQuery on a Postgres SQL DB hosted on an Azure Server with Doctrine managing the mapping and OpenVPN restricting access (this is most likely the issue, I don't mind getting rid of it though). To be clear, I picked Postgres specifically because the data gateway is optional.

 

Everything works fine in desktop w/ OpenVPN client running, but the published report's visuals contain no data. There is a banner error that says, "There is no data gateway access to the source file". I have found posts with similar errors to mine, but they mostly deal with troubleshooting an installed data gateway. 

 

What do I have to do to get this working? Does OpenVPN need to go or do I somehow need to tell Azure that requests from Power BI are OK?

 

Thanks!

  • edhans's avatar
    edhans
    6 years ago

    Yes Anonymous - if you have data that requires the gateway, like the CSV files, then it can make the issue more complex. 

    Can you get it working if you remove everything but the Postgre data? Just whack all of that other stuff and save as a new file. Then add data sources back until you hit the one causing the issue. You didn't say where your CSVs were. A gateway is needed for local/network files or SharePoint On Prem. Should not be needed for SharePoint online, Data Lake, and other online resources.

     

    But mixing and matching can still cause issues that are frustrating to trouble shoot. That is why I suggest do 1 data source, then add until it breaks and fix the breaks one at a time. 

    Given the error you have, something in your report thinks it needs a gateway to access the "data source file"

8 Replies

    • Anonymous's avatar
      Anonymous
      Not applicable

      Nope, I've never heard of query folding.

       

      Thank you for the article, but would you mind elaborating on it? I don't understand where to supply the the Value.NativeQuery statement in the advanced editor (is it line 1? is it in the source step? do I need to add it for each separate query?) and what args it needs. I've never needed the advanced editor til now.

       

      Thank you very much for the reply, though!

      • edhans's avatar
        edhans
        Community Champion

         Query folding is the process Power Query uses to create a SQL statement and sends it to the server. So as an example, I imported the Purchasing table from the Microsoft World Wide Importders sample database on SQL Server using basic transformations in Power Query - removing columns, adding a conditional column. the M code is:

        let
            Source = Sql.Databases("localhost"),
            WideWorldImportersDW = Source{[Name="WideWorldImportersDW"]}[Data],
            Dimension_Customer = WideWorldImportersDW{[Schema="Dimension",Item="Customer"]}[Data],
            #"Removed Other Columns" = Table.SelectColumns(Dimension_Customer,{"Customer Key", "WWI Customer ID", "Customer", "Bill To Customer", "Category", "Buying Group", "Primary Contact", "Postal Code", "Valid From", "Valid To"}),
            #"Added Purchasing Power" = Table.AddColumn(#"Removed Other Columns", "Purchasing Power", each if [Buying Group] = "N/A" then "Standard"
        else if [Buying Group] = "Tailspin Toys" then "Standard"
        else "Discount", type text)
        in
            #"Added Purchasing Power"

        Power Query though "folds" all of that. It doesn't acutally do any of that in the PQ mashup engine. It analyzes it, and send the following SQL statement to the server:

        select [_].[Customer Key] as [Customer Key],
            [_].[WWI Customer ID] as [WWI Customer ID],
            [_].[Customer] as [Customer],
            [_].[Bill To Customer] as [Bill To Customer],
            [_].[Category] as [Category],
            [_].[Buying Group] as [Buying Group],
            [_].[Primary Contact] as [Primary Contact],
            [_].[Postal Code] as [Postal Code],
            [_].[Valid From] as [Valid From],
            [_].[Valid To] as [Valid To],
            case
                when [_].[Buying Group] = 'N/A'
                then 'Standard'
                when [_].[Buying Group] = 'Tailspin Toys'
                then 'Standard'
                else 'Discount'
            end as [Purchasing Power]
        from 
        (
            select [Customer Key],
                [WWI Customer ID],
                [Customer],
                [Bill To Customer],
                [Category],
                [Buying Group],
                [Primary Contact],
                [Postal Code],
                [Valid From],
                [Valid To]
            from [Dimension].[Customer] as [$Table]
        ) as [_]

        The server does all of the work and simply sends back the relevant data. You can do pivots, grouping, merges, appends, etc. Tons of stuff will fold.

        For Direct Query to work, Query Folding must be allowed to happen. Direct Query sends SQL statements to the server and returns immediately those results. There is no other work done or allowed in the Power Query mashup engine. SQL to SQL statements, that is it. As soon as you "break folding" Direct Query breaks. Import Queryies will still work. They will do folding to the point the folding breaks, then do the rest internally in the gateway/mashup engine. Direct Query though doesn't allow it. 100% folding, or you have to switch to Import for that query.

         

        I don't have access to a Postgre database to test with, but I would assume in the M code I have posted above for SQL Server, the PostGre connection would have its own code, and you'd need to modify it to enable query folding per that article link. I'm not sure why - SQL Server it just works - there is no flag to set. But there must be something with Postgre that it need to be able to be turned off and on as needed.

         

        And you would need to add it to each query to your server. It is not a global setting but a per-query setting.