Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago
Solved

copy .csv data to warehouse

hello,

using the copy activity in fabric I can take data from .csv file (Which is sitting in azure storage account) and place the data into the table in the warehouse (tblwh) - I am using mapping in the copy activity. all good so far.

Now I have added a few more columns into the tblwh. These new columns are not in the source .csv file.

For example, I have added a new column called presentdate, filepath, fileextension, fullfilepath

Question:

How is it possible I populate these extra columns in the tblwh because the mapping tab in copy activity does not obviously have any source columns for these ?

thanks

pipelines
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi Anonymous ,

    I can also find the same result -


    By default FileName and FilePath are designed like that.
    As a workaround (assume file path won't change dynamically), you can use custom value -

     

    Hope this is helpful. Please let me know incase of further queries.

7 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi Anonymous ,

    Thanks for using Fabric Community.
    You can add your additional columns in Copy Activity from here - 

     

    Eg:

     



    After adding additional columns in source can you please click on import schema in mapping?

    Eg:

     


    After importing you can run the pipeline

    Hope this helps.

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      this is good. thanks

      in source copy tab, I use

      filename --> $$FILENAME which gives 'some-03-May-2024.csv' --> this is correct

       

      directory_path --> $$FILEPATH but this gives the fullpath such as 'Data/some-new/some-03-May-2024.csv'

      whereas I would like to see the directory such as 'Data/some-new'

      thanks

      • Anonymous's avatar
        Anonymous
        Not applicable
        2 years ago

        Hi Anonymous ,

        I can also find the same result -


        By default FileName and FilePath are designed like that.
        As a workaround (assume file path won't change dynamically), you can use custom value -

         

        Hope this is helpful. Please let me know incase of further queries.