Forum Discussion

anusha_2023's avatar
anusha_2023
Helper IV
2 years ago
Solved

how to read multiple objects json data using multiple relative url path in fabric.

For example I have projects data in one relative path and resources data in another relative path and I am trying to read these two dimensional data from hubplanner rest api into fabric. I have fact...
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi anusha_2023 
    Thanks for using Fabric Community.
    As I understand you want to perform incremental ETL on your data from hubplanner rest api. 

    Here are few points that might help you:

    1. REST API supports pagination . You can copy data from REST API which sends response in Pages using Azure data factory. Please refer to the following links for more details: 
      Link1
      Link2
    2. You can use copy activity and use REST API dataset as the source and Lakehouse table as the sink dataset and load all the data into a dummy table using a FULL LOAD technique and then use that dummy table to join to the main table to perform incremental load.

    You can also refer to this link for additional help: Incremental Data 

    Hope this helps. Please let me know if you have any further queries.