Forum Discussion

DennesTorres's avatar
DennesTorres
Impactful Individual
2 years ago
Solved

Debugging a complex dataflow

Hi, My dataflow is a bit complex, because I need to call an API, retrieve a JSON, work on the JSON to extract two different tables and materialize them in the lakehouse.   "->" means "calls" ...
Dataflows
  • DennesTorres's avatar
    DennesTorres
    2 years ago

    Hi,

    The problem is there, but became very low priority.

    I realized the dataflow was calling the API and in this way it would not reach the desired result. 

    I created a 2nd version of the dataflow, which instead of calling the API, received the JSON (API result) as parameter. This 2nd version works without the failure.

    I can't explain that. But I also can't use the 2nd version because I can't pass parameters from pipelines to dataflows. 

    Kind Regards,

     

    Dennes

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi DennesTorres 
Apologies for the delay in response. 
Are you still facing this issue? If yes please let us know. 
We will revert back with more details and try to help.

DennesTorres's avatar
DennesTorres
Impactful Individual
2 years ago

Hi,

The problem is there, but became very low priority.

I realized the dataflow was calling the API and in this way it would not reach the desired result. 

I created a 2nd version of the dataflow, which instead of calling the API, received the JSON (API result) as parameter. This 2nd version works without the failure.

I can't explain that. But I also can't use the 2nd version because I can't pass parameters from pipelines to dataflows. 

Kind Regards,

 

Dennes