Forum Discussion

Andras89's avatar
Andras89
Regular Visitor
1 year ago
Solved

Refreshing data fails because of dynamic script - nextlink

I have a specific problems with our data flow because I cannot make the whole script static.
We want to download the data  of all of our devices and then compare it which ones are cloud only from Microsoft Online - Microsoft Graph. 
There are different parts of the script which already makes the code dynamic, I could overcome that obstacle, my problem is that beacause of the size we need to use a response with nextlink, which I was not able make to do static. My best guess was to make it relative path, but it didn't work.

The first main part of the code which could be interesting for us:

let
GetAccessToken = () =>
let
tokenUrl = "https://login.microsoftonline.com/" & "token",
body = "client_id=" & "client_id" & "microsoftgraphscode" & "lettersandnumbers" & "&grant_type=client_credentials",
tokenResponse = Json.Document(Web.Contents(tokenUrl, [Content=Text.ToBinary(body), Headers=[#"Content-Type"="application/x-www-form-urlencoded"]])),
accessToken = tokenResponse[access_token]
in
accessToken
in
GetAccessToken
(We have real values in the script of course.)

And the second:

let
accessToken = GetAccessToken(),
getAllMembers = (url) =>
let
response = Json.Document(Web.Contents(url, [Headers=[Authorization="Bearer " & accessToken]])),
members = response[value],
nextLink = try response[#"@odata.nextLink"] otherwise null,
moreMembers = if nextLink <> null then GetMembersOfGroups(nextLink) else {}
in
List.Combine({members, moreMembers})
in
getAllMembers

Then we use something like this:

let
accessToken = GetAccessToken(),
initialUrl = "https://graph.microsoft.com/v1.0/devices?$count=true",
allDevices = GetMembersOfGroups(initialUrl),
#"Converted to Table" = Table.FromList(allDevices, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
Rest of the script

I can combine these to one script and until I don't use pagination, and create a gateway, but in that case I don't have access to all data.

Does anyone have an idea how to solve it within this Power BI framework?

Thanks in advance!

Andras





  • Andras89 

    try this

    let
    // Constants
    accessToken = GetAccessToken(),
    baseUrl = "https://graph.microsoft.com/v1.0/devices?$top=100",

    // Function to get one page
    GetPage = (url as text) as record =>
    let
    response = Json.Document(Web.Contents(
    "https://graph.microsoft.com",
    [
    RelativePath = Text.AfterDelimiter(url, "https://graph.microsoft.com/"),
    Headers = [Authorization = "Bearer " & accessToken]
    ])),
    data = response[value],
    nextLink = try response[#"@odata.nextLink"] otherwise null
    in
    [Data = data, Next = nextLink],

    // Use List.Generate to loop through all pages
    AllPages = List.Generate(
    () => [Result = GetPage(baseUrl)],
    each [Result][Next] <> null,
    each [Result = GetPage([Result][Next])],
    each [Result][Data]
    ),

    // Flatten all pages into a single list
    AllDevices = List.Combine(AllPages),

    // Convert to table
    DevicesTable = Table.FromList(AllDevices, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Expanded Records" = Table.ExpandRecordColumn(DevicesTable, "Column1", {"id", "displayName", "operatingSystem", "isCompliant", "deviceId", "isManaged"})

    Why This Works:

    • It uses List.Generate (loop-based, not recursion).

    • It constructs URLs with RelativePath only — static from Power BI’s point of view.

    • It supports @odata.nextLink within the gateway environment.

    It avoids anonymous queries, dynamic hosts, or runtime-injected full URLs.

    Additional Notes:

    • Make sure your Graph API App Registration has the proper permissions (e.g., Device.Read.All).

    • If you get a 403 Forbidden or 401 Unauthorized, double-check token scopes or that you're querying the correct endpoint (/devices, /users, etc.).

    If @odata.nextLink includes a full path (like https://graph.microsoft.com/...), make sure to strip it down using Text.AfterDelimiter() as shown.

    Did I answer your question? Mark my post as a solution! Appreciate your Kudos !!

  • Hi Andras89,

    Thank you for the update, and I’m sorry to hear the solution didn’t resolve the issue. To help get this sorted out, could you share a bit more detail about what’s happening? For example:

    • What specific error message do you see in Power BI Service when the refresh fails?
    • Does the script work in Power BI Desktop, or are you seeing issues there too?
    • Are you using a gateway, and if so, have the data sources been configured in Power BI Service?

    These details will help us pinpoint the root cause. In the meantime, I’ve reviewed johnbasha33’s script and suggest a few tweaks to enhance error handling, ensure robust pagination for @odata.nextLink, and support your cloud-only device filtering (trustType = "AzureAD"). The updated approach keeps the base URL static using RelativePath, adds diagnostics to verify data retrieval, and includes checks for API or authentication errors.

    Please share the error message or behavior you’re seeing, and we’ll work together to resolve this. Thank you for your patience.

8 Replies

  • Andras89 

    try this

    let
    // Constants
    accessToken = GetAccessToken(),
    baseUrl = "https://graph.microsoft.com/v1.0/devices?$top=100",

    // Function to get one page
    GetPage = (url as text) as record =>
    let
    response = Json.Document(Web.Contents(
    "https://graph.microsoft.com",
    [
    RelativePath = Text.AfterDelimiter(url, "https://graph.microsoft.com/"),
    Headers = [Authorization = "Bearer " & accessToken]
    ])),
    data = response[value],
    nextLink = try response[#"@odata.nextLink"] otherwise null
    in
    [Data = data, Next = nextLink],

    // Use List.Generate to loop through all pages
    AllPages = List.Generate(
    () => [Result = GetPage(baseUrl)],
    each [Result][Next] <> null,
    each [Result = GetPage([Result][Next])],
    each [Result][Data]
    ),

    // Flatten all pages into a single list
    AllDevices = List.Combine(AllPages),

    // Convert to table
    DevicesTable = Table.FromList(AllDevices, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Expanded Records" = Table.ExpandRecordColumn(DevicesTable, "Column1", {"id", "displayName", "operatingSystem", "isCompliant", "deviceId", "isManaged"})

    Why This Works:

    • It uses List.Generate (loop-based, not recursion).

    • It constructs URLs with RelativePath only — static from Power BI’s point of view.

    • It supports @odata.nextLink within the gateway environment.

    It avoids anonymous queries, dynamic hosts, or runtime-injected full URLs.

    Additional Notes:

    • Make sure your Graph API App Registration has the proper permissions (e.g., Device.Read.All).

    • If you get a 403 Forbidden or 401 Unauthorized, double-check token scopes or that you're querying the correct endpoint (/devices, /users, etc.).

    If @odata.nextLink includes a full path (like https://graph.microsoft.com/...), make sure to strip it down using Text.AfterDelimiter() as shown.

    Did I answer your question? Mark my post as a solution! Appreciate your Kudos !!

  • v-ssriganesh's avatar
    v-ssriganesh
    Icon for Community Support rankCommunity Support

    Hi Andras89,

    Thank you for posting your query in the Microsoft Fabric Community Forum, and thanks to johnbasha33 for sharing valuable insights.

     

    Could you please confirm if your query has been resolved by the provided solution? If so, please mark it as the solution. This will help other community members solve similar problems faster.

    Thank you.

    • Andras89's avatar
      Andras89
      Regular Visitor

      As for now we didn't have time to implement the offered solution, so I cannot confirm if it helps, or which part helps.

      • v-ssriganesh's avatar
        v-ssriganesh
        Icon for Community Support rankCommunity Support

        Hi Andras89,
        Could you please confirm if your query have been resolved by the solution provided. If so, please mark it as the solution. This will be helpful for other community members who have similar problems to solve it faster. 

        Thank you.

  • v-ssriganesh's avatar
    v-ssriganesh
    Icon for Community Support rankCommunity Support

    Hi Andras89,
    I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.
    Thank you.