Forum Discussion
API JSON URL 10,000 row limitation
Hi Anonymous
Here's an example of pagination, this query gets the blog post titles from the first 5 pages of the Microsoft Excel blog
let
url= "https://www.microsoft.com/en-us/microsoft-365/blog/excel/page/",
GetPage = (Page) =>
let PageNum = Text.From(Page),
Source = Web.BrowserContents(url & PageNum & "/"),
HTML_Table = Html.Table(Source, {{"Column1", ".entry-date"}, {"Column2", ".card-title"}, {"Column3", ".my-3 *"}, {"Column4", ".bg-gray-100 .cta"}}, [RowSelector=".col"])
in HTML_Table[Column2],
PageIndices = { 1 .. 5 },
Pages = List.Combine(List.Transform(PageIndices, each GetPage(_)))
in
Pages
I know that I can access pages on the blog by using /page/x at the end of the main url. I use List.Transform to call the Getpage function 5 times as specified in PageIndices.
Exactly how your API deals with pagination shoudl eb set out in the API documentation. Check it out and if you need more helpjusy post back here.
regards
Phil
If I answered your question please mark my post as the solution.
If my answer helped solve your problem, give it a kudos by clicking on the Thumbs Up.
- Anonymous5 years agoNot applicable
Unfortunately the API Documentation didnt have any stuff related to pages.
The only work around I made is to pull by day.
Source = "source"& Today
Source = "source"& Today -1
Source = "source"& Today -2
and so on and then append it.
But not sure if theres a neat way or efficienct way to do this.