Forum Discussion

michaelsh's avatar
michaelsh
Kudo Kingpin
5 years ago
Solved

Using Table.Buffer

I am connecting to a Folder that has a bunch of JSON files. I use a custom function to combine the files. Then I do all kinds of stuff, like dynamycally extracting the column names from nested json...
Need Help
  • PhilipTreacy's avatar
    PhilipTreacy
    5 years ago

    Hi michaelsh 

    If you buffer a table in a step, then transform that table in some way, then this new transformed table isn't the one that is buffered. 

    So in subsequent steps you can't continue to refer to the buffered table from your 1st step.

    To be honest the workings of the Buffer functions aren't cleary undestood by anyone other than the Microsoft dev team.  I've never read a definitive explanation of how or when to use them, hence Pat's suggestion to 'just try them out'.

    Cheers

    Phil

mahoneypat's avatar
mahoneypat
Microsoft Employee
5 years ago

The best way is to just try it out and see if it makes things significantly faster.  Hard to predict if it will have an impact as sometimes is does that type of optimization on its own.  The key thing is to use it within the same query where it is referenced multiple times (which your proposal does).

Regards,
Pat