Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago
Solved

Dataflow (gen1) fails after Join operation

Hi everyone,

 

I pull 7 tables from our Postgres DB into a dataflow. I was able to refresh it just fine, but after resuming my ETL process, and merging 'Orders' and 'Orderline' table to get a flat fact table, and although I am able to see the data in the dataflow data view, upon saving and exiting the dataflow fails to refresh, showing this error:

'Error: On-Prem execution not supported for entity '<ccon>9ac68d65-62fe-4f31-975c-63fcca146f32/123695af-7b5c-44a6-a79c-060514043792/Final_Spf orders</ccon>' since it refers to another entity.. Param1 = <ccon>9ac68d65-62fe-4f31-975c-63fcca146f32/123695af-7b5c-44a6-a79c-060514043792/Final_Spf orders</ccon> Request ID: 53969960-39c0-6b27-9983-735323ac30dd.'

 

Can anyone please advise what am I doing wrong with such a simple ETL process?

 

Thanks,

 

EDIT: confirming that removing the merge step fixes the issue.

 

I'll keep troubleshooting, but any ideas are welcome 🙂

  • Anonymous's avatar
    Anonymous
    2 years ago

    In case anyone stumbles on this post, I got around by having another admin of the workspace reconfigure the credentials. Apparently the connection to a google sheet was the culprit.

     

    It has been working correctly for the last two days, fingers cross y does not randomly break.

3 Replies

  • mohitkumawat's avatar
    mohitkumawat
    Icon for Impactful Individual rankImpactful Individual

    Hi Anonymous ,

    This error is triggered due to the dependencies with the source “Orders”  tables, where the latter is the one that necessitates the use of on-premises gateway, from the merged entity.

     

    you can check below things if this is enabled 

    First, navigate to the dataflow settings and ensure the “Enable load” for the linked entities’ “Orders”  checkbox is not ticked .

     

    By refreshing the data in the cloud first, and then using the reloaded data for merging he can thus bypass the on-premises gateway.

     

    Instead of merging the tables directly, create a reference with the original tables in the dataflow. Consequently, this will enable him to use the cloud-based source of the dataflow entities during the refresh rather than relying on on-premises gateway.

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi mohitkumawat ,

     

    Thanks for your assistance.

     

    When you say dataflow settings, I cannot really see any load options here:

     

    If you are referring to this 'enable load' then I can confirm it is now off on all tables but now the dataflow won't validate

     

     

     

     

    Am I missing something?

     

    Also are you suggesting that before every merge I create a reference table and merge those instead?

    Thanks,

    • Anonymous's avatar
      Anonymous
      Not applicable

      In case anyone stumbles on this post, I got around by having another admin of the workspace reconfigure the credentials. Apparently the connection to a google sheet was the culprit.

       

      It has been working correctly for the last two days, fingers cross y does not randomly break.