Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago
Solved

read from azure storage container folder

hello, I have managed to get the .csv file from the web page and place it into the azure storage container using the copy activity in fabric data factory. Now I would like to read that .csv file, f...
pipelines
  • v-cboorla-msft's avatar
    v-cboorla-msft
    2 years ago

    Hi Anonymous 

     

    I would to like to understand what are you trying to achieve with the help of pre-script? If you are looking to adding the additional column value to warehouse, then you simply continue with mapping option. 

    If your query is different, please share more details of your query for better understanding.

     

    Thank you.

v-cboorla-msft's avatar
v-cboorla-msft
Icon for Microsoft Employee rankMicrosoft Employee
2 years ago

Hi Anonymous 

 

Thanks for using Microsoft Fabric Community.

You can read the .csv file from azure storage container and append it to Fabric Warehouse by using Copy activity in Fabric Pipeline.

I hope this information helps. Please do let us know if you have any further queries.

 

Thank you.

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    I see...

    In that case, question please:

    1- in the destination tab - pre-script - before data is inserted into a warehouse table, get the filename (I know I can get the filename at source copy activity using $$FILENAME but not sure how to pass it into the pre-script box in destination tab.

    thanks

    • v-cboorla-msft's avatar
      v-cboorla-msft
      Icon for Microsoft Employee rankMicrosoft Employee
      2 years ago

      Hi Anonymous 

       

      I would to like to understand what are you trying to achieve with the help of pre-script? If you are looking to adding the additional column value to warehouse, then you simply continue with mapping option. 

      If your query is different, please share more details of your query for better understanding.

       

      Thank you.

      • Anonymous's avatar
        Anonymous
        Not applicable
        2 years ago

         basically there is a table in the warehouse I created called tblwh The ADF pipeline reads the csv and places the data including the additional columnsi I added in th ecopy activity source nto this table tblwh. All good

        Now if I re-run the pipeline I do not want the data to be duplicated into the tblwh. So I want to make sure it is deleted if it is already inserted in the previous pipeline run for the same filename.

        and so I would like to have in the pre-scrip box to have something like:

        delete from tblwh where filename = 'xyz'