Forum Discussion
Dataflow - Append
- 1 year ago
when I create a new table, there is no Append feature? meaning that I cannot have a range of dates. So, I need to create a new table for one date of data first? then run the same data pipeline with a range of dates for the append to the table created?
- 1 year ago
me and tan_thiamhuat had an offline sync.
The issue was related to how the data destination settings were set. After manually changing the creation of a new table to go into the manual settings (and avoid the automatic settings) things worked as expected.
Marking this thread as resolved.
- 1 year ago
It was great and fantastic that Miguel help me solve this issue within 15mins when I took more than 2 days to look into what is wrong 😁. Thanks Miguel for your proactive effort to do it offline to get this solved. Appreciate it a lot.
Dataflow refresh activity inside of your pipeline -- can you guide the steps to get this?
I just saw the screenshots that you shared around the "Recent runs" of your Dataflow, so we can rule out that a value is not being passed to the Dataflow.
The best thing that you can do is to modify your data destination settings and point the solution to a new table.
If you can repro the behavior even when changing the data destination settings that create a new table, please share that repro steps with us. You can also use the simplified mashup code that I shared to compare it with your own. Anything that would help us reproduce this behavior on our environments will help us pinpoint what could be happening.