Forum Discussion

Darwood's avatar
Darwood
Frequent Visitor
1 year ago
Solved

Create Wallboard from API REST calls

How can I get the output of a Web Activity into a Lakehouse in Microsoft Fabric please?   A bit of background: I am trying to create a wallboard showing the status of our IT Infrastructure. One pa...
data hub
one lake
real-time hub
workspaces
  • nilendraFabric's avatar
    nilendraFabric
    1 year ago

    hello Darwood 

     

    I think you have done most of the hard work to get the authentication and connection done

     

    Now for Web Activity Output Handling
    • Use a Set Variable activity to capture the Web Activity response
    • Configure the variable to store the complete JSON output


    Create Copy Data Activity Setup
    • Create a dummy source query using a Fabric Datawarehouse connection
    • Use the “Additional Columns” section in the source to define your output schema
    • Configure the destination as the Lakehouse Files section
    • Map the fields from your variable to the destination columns

     


    check this discussion it will be helpful 

     

    https://community.fabric.microsoft.com/t5/Data-Pipeline/Writing-Pipeline-Output-to-JSON-file-in-Lakehouse/m-p/4117453

     

     

    If this helps. Please accept the solution 

     

    Thanks

nilendraFabric's avatar
nilendraFabric
Icon for Super User rankSuper User
1 year ago

hello Darwood 

 

I think you have done most of the hard work to get the authentication and connection done

 

Now for Web Activity Output Handling
• Use a Set Variable activity to capture the Web Activity response
• Configure the variable to store the complete JSON output


Create Copy Data Activity Setup
• Create a dummy source query using a Fabric Datawarehouse connection
• Use the “Additional Columns” section in the source to define your output schema
• Configure the destination as the Lakehouse Files section
• Map the fields from your variable to the destination columns

 


check this discussion it will be helpful 

 

https://community.fabric.microsoft.com/t5/Data-Pipeline/Writing-Pipeline-Output-to-JSON-file-in-Lakehouse/m-p/4117453

 

 

If this helps. Please accept the solution 

 

Thanks

  • Darwood's avatar
    Darwood
    Frequent Visitor
    1 year ago

    Thanks for the suggestion nilendraFabric 

    I am getting a data type error on the Set variable activity though when I try and run the Data Pipeline. I have tried using a string or array data type and I get the same sort of problem.

     

    The settings for the Set variable activity are:

    • Variable type: Pipeline variable
    • Name: devicesJson
    • Value: @activity('Devices').output (this is the name of the previous Web activity API call)

     

    "The variable 'devicesJson' of type 'Array' cannot be initialized or updated with value of type 'Object'. The variable 'devicesJson' only supports values of types 'Array'."

  • Darwood's avatar
    Darwood
    Frequent Visitor
    1 year ago

    Ignore that last message. I didn't realise the output wrapped the result in a JSON object with a "response" property. I have updated the variable expression to 

    @json(activity('Devices').output.response) and the array variable seems to be correctly populated now. Still struggling to get the Additional Columns working in the Copy Data activity though...
  • Darwood's avatar
    Darwood
    Frequent Visitor
    1 year ago

    ok so I'm almost there....

    I have created a dummy query with additional columns and dynamic data and I can get the id value for the first record using a value of

    @variables('devicesJson')[0].id

    However I want to bring across the values for every single record in the output. I can obviously change the 0 to a different number to get a particular record but I want them all. How would I do this please?