Getting data from API with JSON body

[keribeal]

I'm trying to get data out of my ElasticSearch and into PowerBI.
I can do it with Postman Fine but am having issues with PowerBI.

If i remove the body I do not get an error.

My Query for PowerQuery Editor is

let
    url = "http://MYSITE/search",
    body = "
            {
            ""query"": {
                ""bool"": {
                    ""must"": [
                        {
                            ""range"": {
                                ""createdDate"": {
                                    ""gt"": ""2020-01-01T00:00:00"",
                                    ""lt"": ""2099-12-31T23:59:59""
                                }
                            }
                        },
                        {
                            ""terms"": {
                                ""interestFlags.systemName"": [""SalesApplicant"",""LettingsApplicant"",""Vendor"",""Company"",""Landlord""]
                            }
                        }
                    ],
                    ""must_not"": [],
                    ""should"": []
                }
            },
            ""from"": 0,
            ""size"": 0,
            ""sort"": [],
            ""aggs"": {
                ""className"": {
                    ""terms"": {
                        ""field"": ""interestFlags.systemName"",
                        ""size"": 200
                    }
                }
            }
        }
    ",
    Source = Json.Document(Web.Contents(url),[
        Headers = [#"Content-Type"="application/json"],
        Content=Text.ToBinary(body)
    ])
in
    Source

I get the following error when I pass in the JSON body

Expression.Error: We cannot convert a value of type Record to type Number.
Details:
Value=[Record]
Type=[Type]

Hope someone can point me in the right direction.

Thanks,

Keri.

[dax]

Hi @keribeal , 

I am not sure whether this is caused by the code, you could try below to see whether it work or not

Source = Json.Document(Web.Contents(url,[
        Headers = [#"Content-Type"="application/json"],
        Content=Text.ToBinary(body)
    ]))

You could refer to this post   for details. By the way, you also could @ super users in this forum for more suggestions.

Best Regards,
Zoe Zhi

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[keribeal]

@dax Thanks for the reply, you are right I had a ) after url which shouldn't have been there.

That gives a new error for me

DataSource.Error: Web.Contents with the Content option is only supported when connecting anonymously.
Details:
DataSourceKind=Web
DataSourcePath=http://XXXXX

So that's a new error to get searching for as the BASIC AUTH credentials were entered at the start when prompted.

Sorry i'm new to these forums, with your comment about @ super users, do you mean i just look for the names of super users and just add them in here?

Thanks,

Keri.

[keribeal]

Having read what feels like half the internet i've now learnt that you can pass the BASIC AUTH U&P as an Authorization header, thus allowing the save it in PowerBI as anonymous and therefore allowing to post the body.

I've tested this in postman and it works fine, however in powerBI i get the following error

DataSource.Error: Web.Contents failed to get contents from 'http://mysite/_search' (400): Bad Request
Details:
DataSourceKind=Web
DataSourcePath=http://mysite/_search
Url=http://mysite/_search

let
    url = "http://mysite/_search",
    body = "
            {
            ""query"": {
                ""bool"": {
                    ""must"": [
                        {
                            ""range"": {
                                ""createdDate"": {
                                    ""gt"": ""2020-01-01T00:00:00"",
                                    ""lt"": ""2099-12-31T23:59:59""
                                }
                            }
                        },
                        {
                            ""terms"": {
                                ""interestFlags.systemName"": [""SalesApplicant"",""LettingsApplicant"",""Vendor"",""Company"",""Landlord""]
                            }
                        }
                    ],
                    ""must_not"": [],
                    ""should"": []
                }
            },
            ""from"": 0,
            ""size"": 0,
            ""sort"": [],
            ""aggs"": {
                ""className"": {
                    ""terms"": {
                        ""field"": ""interestFlags.systemName"",
                        ""size"": 200
                    }
                }
            }
        }
    ",
    //body="",
    header = [  #"Authorization"="Basic BASE64-USER:PWD",
                #"Content-Type"= "application/json"],


    response = Web.Contents(url,[Content=Json.FromValue(body),Headers=header]),
    Source = Json.Document(response,1252) 
in
    Source

 Is there anyone that can help shine some light on this issue for me, help point me in the right direction?

Thanks,

Keri

[keribeal]

Good thing is I now know that the auth is definitely working, if I make the call without a body i get data back on one of my endpoints, i get 5m records instead of the 4 totals i was hoping for, but it's a start.

Looking at the fiddler data for both Postman and PowerBI i think i can see what is probably the issue.

For the Postman call, i can see the JSON all neatly formatted in the request

The original PowerBI with neatly formatted JSON came out like this

So i put it all on a single line and had this

"{\"query\": {\"bool\": {\"must\": [{\"range\": {\"createdDate\": {\"gt\": \"2020-01-01T00:00:00\", \"lt\": \"2099-12-31T23:59:59\"}}},{\"terms\": {\"interestFlags.systemName\": [\"SalesApplicant\",\"LettingsApplicant\",\"Vendor\",\"Company\",\"Landlord\"]}}],\"must_not\": [],\"should\": []}},\"from\": 0,\"size\": 0,\"sort\": [],\"aggs\": {\"className\": {\"terms\": {\"field\": \"interestFlags.systemName\",\"size\": 200}}}}"

I'm guessing the quotes either end aren't helping nor is the escaped characters throughout, anyone got any tips how to get this working?

Thanks,

Keri

[mattmasson · Accepted solution]

The problem is the use of Json.FromValue. body is a text value, so it is giving you a json encoded string rather than the structured object the API is expecting.

If you want to keep body as text, then you'd use Text.ToBinary to convert it to POST content. 

let

    url = "xxxxxxx",

    body = "{""query"": {""bool"": {""must"": [{""range"": {""createdDate"": {""gt"": ""2020-01-01T00:00:00"", ""lt"": ""2099-12-31T23:59:59""}}},{""terms"": {""interestFlags.systemName"": [""SalesApplicant"",""LettingsApplicant"",""Vendor"",""Company"",""Landlord""]}}],""must_not"": [],""should"": []}},""from"": 0,""size"": 0,""sort"": [],""aggs"": {""className"": {""terms"": {""field"": ""interestFlags.systemName"",""size"": 200}}}}",

    header = [  #"Authorization"="xxxxxx",

                #"Content-Type"= "application/json"],  

    response = Web.Contents(url,[Content=Text.ToBinary(body),Headers=header]),

    Source = Json.Document(response,1252) 

in

    Source

Alternatively, you could build up body as a record, and then use Json.FromValue. The main thing to keep in mind here is that the {} and [] are reversed between M and JavaScript. This would look something like:

let

    url = "xxxxxx",

    body = [

        query = [

                bool = [

                    must = {[ 

                        range = [ 

                            createdDate = [ gt = "2020-01-01T00:00:00", lt = "2099-12-31T23:59:59" ]

                        ]

                    ]},

                    must_not = {},

                    should = {}

                ]

            ],

        from = 0,

        size = 0,

        sort = {},

        aggs = [

            className = [

                terms = [

                    field = "interestFlags.systemName",

                    size = 200

                ]

            ]

        ]

    ],

    header = [  #"Authorization"="xxxxxxxx",

                #"Content-Type"= "application/json"],  

    response = Web.Contents(url,[Content=Json.FromValue(body),Headers=header]),

    Source = Json.Document(response,1252) 

in

    Source

[keribeal]

Fantastic, many thanks to PowerBI support for this help.