Forum Discussion
BI007
1 year agoHelper I
Dynamic data source issue
Hello. I have this M script in Power Query, which is getting data using api (using pagination): let // Function to retrieve a page of data GetPage = (url as text) => let RawData = Json.Doc...
- Anonymous1 year ago
Hi BI007 ,
Thanks for ZhangKun reply.
One way to do this is to make the URL parameters static in the query folding contextlet GetPage = (url as text) => let RawData = Json.Document(Web.Contents(url, [Headers=[Authorization="Bearer " & GetToken()]])), Users = RawData[value], NextLink = try RawData[#"@odata.nextLink"] otherwise null, DataTable = if List.IsEmpty(Users) then null else Table.FromList(Users, Splitter.SplitByNothing(), null, null, ExtraValues.Error) in [Table = DataTable, NextLink = NextLink], InitialUrl = "https://graph.microsoft.com/v1.0/groups", Initial = GetPage(InitialUrl), AllPages = List.Generate( () => [Page = Initial, Link = Initial[NextLink]], each [Link] <> null, each [Page = GetPage([Link]), Link = [Page][NextLink]], each [Page][Table] ), CombinedData = Table.Combine(List.RemoveNulls(AllPages)), #"Expanded Column1" = Table.ExpandRecordColumn(CombinedData, "Column1", {"id", "description"}, {"id", "description"}) in #"Expanded Column1"You can also use the RelativePath and Query options to build more flexible and maintainable URLs.
let GetPage = (relativePath as text, query as record) => let RawData = Json.Document(Web.Contents("https://graph.microsoft.com", [RelativePath = relativePath, Query = query, Headers = [Authorization = "Bearer " & GetToken()]])), Users = RawData[value], NextLink = try RawData[#"@odata.nextLink"] otherwise null, DataTable = if List.IsEmpty(Users) then null else Table.FromList(Users, Splitter.SplitByNothing(), null, null, ExtraValues.Error) in [Table = DataTable, NextLink = NextLink], InitialRelativePath = "v1.0/groups", InitialQuery = [], Initial = GetPage(InitialRelativePath, InitialQuery), AllPages = List.Generate( () => [Page = Initial, Link = Initial[NextLink]], each [Link] <> null, each [Page = GetPage(Text.Middle([Link], Text.Length("https://graph.microsoft.com/")), []), Link = [Page][NextLink]], each [Page][Table] ), CombinedData = Table.Combine(List.RemoveNulls(AllPages)), #"Expanded Column1" = Table.ExpandRecordColumn(CombinedData, "Column1", {"id", "description"}, {"id", "description"}) in #"Expanded Column1"Data refresh in Power BI - Power BI | Microsoft Learn
Best regards,
Albert HeIf this post helps, then please consider Accept it as the solution to help the other members find it more quickly
BI007
1 year agoHelper I
in my case, first parameter is dynamic, not second of Web.Contents.
Can you show me on the script provided what is the correct way?