Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago

POST request works in Power Query Editor in Excel, but fails (400) in PQE in Power Bi

Good Afternoon all

 

I am currently trying to improve my organisation's data handling workflows, and as part of this I'm trying to set up a query to the Statistics Scotland API in Power Bi.

 

Just as a foreword, I'm a real beginner when it comes to data/computer science and I don't really know how to code in Python/Java etc - and I've just been learning to use DAX and M code as I go; mainly to make queries and visuals in Excel and PBI. 

 

I've successfully written a SPARQL query to search the Statistics Scotland database and, using ChatGPT, have written M code to send a POST request to the Statistics Scotland API - with the SPARQL query in the body - in order to pull the data into Power Query Editor in excel. Using this code (attached at the bottom), I've successfully made sevral queries in Excel to the Statistics Scotland API.

 

However, when I copy the code from the advanced editor in PQE in Excel and try to create a new query in PBI, I get a Data Source Error (400). 

 

I've opened a new workbook in Excel and made a new query with the exact same code, and it works! And double checking the Statistics Scotland website, it seems everything is operatign fine on that end. So I can't understand why the request won't work in Power Bi - as it and Excel both use PQE to make and edit queries.

 

Any help, hints or suggestions would be massively appreciated!

Thanks

Marc

 

 

let
    // Define the SPARQL query
    sparqlQuery = "
        PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
        SELECT ?industryname ?periodname ?value
        WHERE {  
            ?obs <http://purl.org/linked-data/cube#dataSet> <http://statistics.gov.scot/data/gross-domestic-product-quarterly-output-by-industry> .
            ?obs <http://purl.org/linked-data/sdmx/2009/dimension#refArea> ?areauri .
            ?obs <http://statistics.gov.scot/def/dimension/industrySector(sic07)> ?industryuri .
            ?obs <http://purl.org/linked-data/sdmx/2009/dimension#refPeriod> ?perioduri .
            ?obs <http://statistics.gov.scot/def/measure-properties/index> ?value .
            ?areauri rdfs:label ?areaname .
            ?perioduri rdfs:label ?periodname .
            ?industryuri rdfs:label ?industryname .
            FILTER (?areaname = 'Scotland')
        }
        ORDER BY DESC(?periodname)",

    // Define the URL
    url = "http://statistics.gov.scot/sparql",

    // Define the headers
    headers = [
        #"Accept" = "application/sparql-results+json",
        #"Content-Type" = "application/x-www-form-urlencoded"
    ],

    // Encode the SPARQL query
    body = "query=" & Uri.EscapeDataString(sparqlQuery),

    // Make the POST request
    Source = Json.Document(Web.Contents(url, [
        Headers = headers,
        Content = Text.ToBinary(body, TextEncoding.Utf8)
    ])),

    // Extract the "results" section from the JSON response
    results = Source[results][bindings],

    // Create a list of records from the JSON data
    records = List.Transform(results, each [
        industryname = _[industryname][value],
        periodname = _[periodname][value],
        value = _[value][value]
    ]),

    // Convert the list of records to a table
    finalTable = Table.FromRecords(records),
    #"Reordered Columns" = Table.ReorderColumns(finalTable,{"periodname", "industryname", "value"}),
    #"Renamed Columns" = Table.RenameColumns(#"Reordered Columns",{{"periodname", "Period"}}),
    #"Added Custom" = Table.AddColumn(#"Renamed Columns", "Date", each let
    YearText = Text.Middle([Period], 0, 4),
    QuarterText = Text.Middle([Period], 6, 1),
    Year = Number.FromText(YearText),
    Quarter = Number.FromText(QuarterText),
    Month = if Quarter = 1 then 3 else if Quarter = 2 then 6 else if Quarter = 3 then 9 else 12,
    Date = #date(Year, Month, 1)
in
    Date),
    #"Reordered Columns1" = Table.ReorderColumns(#"Added Custom",{"Period", "Date", "industryname", "value"}),
    #"Renamed Columns1" = Table.RenameColumns(#"Reordered Columns1",{{"industryname", "Industry"}}),
    #"Pivoted Column" = Table.Pivot(#"Renamed Columns1", List.Distinct(#"Renamed Columns1"[Industry]), "Industry", "value"),
    #"Changed Type" = Table.TransformColumnTypes(#"Pivoted Column",{{"Date", type number}, {"Water Supply and Sewerage (Section E)", type number}, {"Construction (Section F)", type number}, {"Government and Other Services (Section O-T)", type number}, {"Business Services and Finance (Section K-N)", type number}, {"Services (Section G-T)", type number}, {"Distribution, Hotels and Restaurants (Section G,I)", type number}, {"Production (Section B-E)", type number}, {"Mining & Quarrying (Section B)", type number}, {"Agriculture, Forestry and Fishing (Section A)", type number}, {"Total Gross Value Added (GVA) (Section A-T)", type number}, {"Manufacturing (Section C)", type number}, {"Transport, Storage and Communication (Section H,J)", type number}, {"Electricity, Gas, Steam and Air (Section D)", type number}}),
    #"Changed Type1" = Table.TransformColumnTypes(#"Changed Type",{{"Date", type date}}),
    #"Reordered Columns2" = Table.ReorderColumns(#"Changed Type1",{"Period", "Date", "Total Gross Value Added (GVA) (Section A-T)", "Water Supply and Sewerage (Section E)", "Construction (Section F)", "Government and Other Services (Section O-T)", "Business Services and Finance (Section K-N)", "Services (Section G-T)", "Distribution, Hotels and Restaurants (Section G,I)", "Production (Section B-E)", "Mining & Quarrying (Section B)", "Agriculture, Forestry and Fishing (Section A)", "Manufacturing (Section C)", "Transport, Storage and Communication (Section H,J)", "Electricity, Gas, Steam and Air (Section D)"})
in
    #"Reordered Columns2"

 

 

1 Reply

  • I tried on Excel and hit the same 400 issue.  Which version of Excel are you using?