Forum Discussion

Expresso's avatar
Expresso
Frequent Visitor
4 years ago

Web content data extraction functions

Isn't it better/optimized to call the function in a new table for every new request made instead adding it to the main table and having it repeat all the previous data inside the table just to relate to the new data inserted? If it is definitly better, please let me know how to do it.


Power Query – Building functions to fetch web data

 

Example:

 

main_table

A | B | C |

C | A | B |

B | C | A |

 

>function to made web crawling with data of main_table as illustred in the link before.

 

main_table

A | B | C | 1

A | B | C | 2

C | A | B | 1

C | A | B | 2

B | C | A | 1

B | C | A | 2

 

3 Replies

  • A new table for each new request? That sounds to me like you'd end up with way too many tables.

    • Expresso's avatar
      Expresso
      Frequent Visitor

      Depends on the requested data, but it isn't better than having a lot of duplicated data?

      • AlexisOlson's avatar
        AlexisOlson
        Super User

        I think it's usually easier to de-duplicate data than handle lots of tables but I can't speak to your specific use case.