Help writing a function to avoid pagination limits in Web.Contents query from Wrike

[Seward12533]

Hi, I'm tring to pull data from Wrike via an API call and they have a limit on 1000 records on any query and provided me this feedback.

"However, you can use pagination to work around this limit (https://developers.wrike.com/documentation/api/methods/query-tasks). To request pagination in your call, please, specify the pageSize parameter and use the nextPageToken from the response to query for additional pages until you get a page with no next page token, which is your last page in the data set. This way you should be able to get the full task list if it's longer than 1000 tasks."

Doing some research I found methhods for handling it https://medium.com/@marktiedemann/how-to-do-pagination-in-power-query-430460c17c78 but this requries parameters that the API does not support.  Instead they include a token as the last record than can be used to fetch the next record and I'm uncertain how to handle that.

Any help in allwoing me to make recursive calls to get around this 1000 record limit would be appreciated. 

Here is a copy of my base query

let
    Source = Web.Contents("https://www.wrike.com/api/v3/tasks",
                [
                Query=[#"fields"="[briefDescription,parentIds,superParentIds,sharedIds,dependencyIds,
                        subTaskIds,responsibleIds,description,recurrent,authorIds,attachmentCount,hasAttachments,
                        customFields,superTaskIds,metadata]"],
                Headers=[#"Authorization"="bearer m5...IUK"]
                ]),
    convertToJson = Json.Document(Source),
    data = convertToJson[data],
    #"Converted to Table" = Table.FromList(data, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"id", "accountId", "title", "description", "briefDescription", "parentIds", "superParentIds", "responsibleIds", "status", "importance", "createdDate", "updatedDate", "completedDate", "dates", "scope", "authorIds", "customStatusId", "hasAttachments", "attachmentCount", "permalink", "priority", "superTaskIds", "subTaskIds", "metadata", "customFields", "sharedIds", "dependencyIds"}, {"id", "accountId", "title", "description", "briefDescription", "parentIds", "superParentIds", "responsibleIds", "status", "importance", "createdDate", "updatedDate", "completedDate", "dates", "scope", "authorIds", "customStatusId", "hasAttachments", "attachmentCount", "permalink", "priority", "superTaskIds", "subTaskIds", "metadata", "customFields", "sharedIds", "dependencyIds"}),
    #"Fix Created Dates" = Table.TransformColumns(#"Expanded Column1",{{"createdDate", each Date.From(DateTimeZone.From(_)), type date}, {"updatedDate", each Date.From(DateTimeZone.From(_)), type date}}),
    #"Fix Complted Dates" = Table.TransformColumns(#"Fix Created Dates",{{"completedDate", each Date.From(DateTimeZone.From(_)), type date}}),
    #"Expanded dates" = Table.ExpandRecordColumn(#"Fix Complted Dates", "dates", {"type", "duration", "start", "due", "workOnWeekends"}, {"type", "duration", "start", "due", "workOnWeekends"}),
    #"Fix Date formatting" = Table.TransformColumns(#"Expanded dates",{{"start", each Date.From(DateTimeZone.From(_)), type date}, {"due", each Date.From(DateTimeZone.From(_)), type date}}),
    #"Removed Columns" = Table.RemoveColumns(#"Fix Date formatting",{"accountId", "metadata"})
in
    #"Removed Columns"

[Anonymous]

Hi @Seward12533 and @difg 

One way of getting around this is by using a third party tool – check this out: https://www.synchub.io/connectors/wrike

It's a simple platform that does all the syncing for you.

– Bryce

[difg]

Hi

Did you find the solution with nextPageToken of wrike?

I'm looking for a solution... I've made a lot os proves, but... it doesn´t work!

Thanks in advance.

[v-juanli-msft]

Hi @Seward12533

Could you check this thread and discussion under it?

It may give you some ideas.

Best Regards

Maggie