Forum Discussion

DennesTorres's avatar
DennesTorres
Impactful Individual
2 years ago
Solved

Debugging a complex dataflow

Hi, My dataflow is a bit complex, because I need to call an API, retrieve a JSON, work on the JSON to extract two different tables and materialize them in the lakehouse.   "->" means "calls" ...
Dataflows
  • DennesTorres's avatar
    DennesTorres
    2 years ago

    Hi,

    The problem is there, but became very low priority.

    I realized the dataflow was calling the API and in this way it would not reach the desired result. 

    I created a 2nd version of the dataflow, which instead of calling the API, received the JSON (API result) as parameter. This 2nd version works without the failure.

    I can't explain that. But I also can't use the 2nd version because I can't pass parameters from pipelines to dataflows. 

    Kind Regards,

     

    Dennes

jwelch's avatar
jwelch
Microsoft Employee
2 years ago

Please disable staging on the functions and see if that changes the behavior. Generally, you don't want to stage functions. I'd need more detail on what you are doing in the functions to determine if it's appropriate here.

  • DennesTorres's avatar
    DennesTorres
    Impactful Individual
    2 years ago

    Hi,
    I tried to disable staging on the functions, but nothing changed. It was like if the function was still using staging, even when disabled.

     

    Kind Regards,

     

    Dennes

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      Hi DennesTorres 
      Apologies for the delay in response. 
      Are you still facing this issue? If yes please let us know. 
      We will revert back with more details and try to help.

      • DennesTorres's avatar
        DennesTorres
        Impactful Individual
        2 years ago

        Hi,

        The problem is there, but became very low priority.

        I realized the dataflow was calling the API and in this way it would not reach the desired result. 

        I created a 2nd version of the dataflow, which instead of calling the API, received the JSON (API result) as parameter. This 2nd version works without the failure.

        I can't explain that. But I also can't use the 2nd version because I can't pass parameters from pipelines to dataflows. 

        Kind Regards,

         

        Dennes