full load of workitemsnapshot table of Azure Devops to Fabric lakehouse

[shivanshi1311]

I want to do a full load of  workitemsnapshot table of Azure Devops to Fabric lakehouse. I am trying this using odata queries but got stuck for 10000 rows limit. I already applied filters conditions to odata url to minimize the result. And I can't apply aggregations to this table in odata url as per our business needs and reqiurements.

I want to know a way to do a full load of workitemsnapshot table of Azure Devops to Fabric lakehouse without 10000 rows limit.

[v-pbandela-msft]

Hi @shivanshi1311,

I wanted to follow up since we haven't heard back from you regarding our last response. We hope your issue has been resolved.
If the community member's answer your query, please mark it as "Accept as Solution" and select "Yes" if it was helpful.
If you need any further assistance, feel free to reach out.

Thank you,
Pavan.

[v-pbandela-msft]

Hi @shivanshi1311,

I wanted to follow up since we haven't heard back from you regarding our last response. We hope your issue has been resolved.
If the community member's answer your query, please mark it as "Accept as Solution" and select "Yes" if it was helpful.
If you need any further assistance, feel free to reach out.

Thank you,
Pavan.

[v-pbandela-msft]

Hi @shivanshi1311,

I wanted to follow up since we haven't heard back from you regarding our last response. We hope your issue has been resolved.
If the community member's answer your query, please mark it as "Accept as Solution" and select "Yes" if it was helpful.
If you need any further assistance, feel free to reach out.

Thank you,
Pavan.

[v-pbandela-msft · Accepted solution]

Hi @shivanshi1311,

Thank for reaching out in Microsoft Community Forum.

Here we're providing some possible approaches to To load the full WorkItemSnapshot table from Azure DevOps to Fabric Lakehouse without hitting the 10,000 row limit in OData

1. Please retrieve data in chunks by setting up $top=10000 and incrementing $skip.

2. To manage data more efficiently, please filter by date ranges or Work Item IDs to load it in smaller batches.

3. The Azure DevOps REST API provides enhanced control and supports pagination for managing larger data sets efficiently.

4. Please use Azure DevOps Data Warehouse or third-party tools for bulk data export.

5. Please retrieve only the new or modified data since the last update.

If you found this post helpful, please consider marking it as "Accept as Solution" and select "Yes" if it was helpful. help other members find it more easily.

Thank you,
Pavan.

[i-m-7uraj]

$Skip is not acceptable with WorkItemSnapshot Table.