Forum Discussion

coolie's avatar
coolie
Helper I
2 years ago

Create additional column for partition id

I am reading Rest data to create a lakehouse table, but I'd now like the table to be partitioned by date of the daily run. I eventually managed to add the additional source column (derived from pipel...
pipelines
v-cboorla-msft's avatar
v-cboorla-msft
Microsoft Employee
2 years ago

Hi coolie 

 

Thanks for your reponse.

 

I have tried to repro the same scenario but couldn't find any error. So could you please confirm if you are missing something or if I am missing any steps in the process.

 

Please find the screenshots and the steps I followed for my repro.

To do the repro I took source as Lakehouse and selected the sampledata table, destination as Lakeshouse named the output as DemoTable.

Source:

Destination:

Mapping:

Result:

 

I hope this helps. If you have any further questions please do let us know.

 

coolie's avatar
coolie
Helper I
2 years ago

Hi thanks for taking the time to do that. I also reproduced what you did successfully. The difference is my source is a REST (external) connection. If you keep everything the same just change the source you will find you get the same error, so it must be a bug. I got round this by using a intermediate temp table before writing to the partitioned one. Unfortunately you cant append to a partitioned table so going to raise BUG #2. Not looking too good for Fabric so far is it.

  • v-cboorla-msft's avatar
    v-cboorla-msft
    Microsoft Employee
    2 years ago

    Hi coolie 

     

    Apologies for the inconvenience that you are facing here.

    This might require a deeper investigation from our engineering team about your workspace, the pipeline details and the logic behind it to properly understand what might be happening. 

    Please go ahead and raise a support ticket to reach our support team:

    https://support.fabric.microsoft.com/support
    Please provide the ticket number here as we can keep an eye on it.

     

    Thank you.

    • v-cboorla-msft's avatar
      v-cboorla-msft
      Microsoft Employee
      2 years ago

      Hi coolie 

       

      We haven’t heard from you on the last response and was just checking back to see if you got a chance to create a support ticket. If yes please share the ticket number.
      Otherwise, will respond back with the more details and we will try to help.

       

      Thanks

      • coolie's avatar
        coolie
        Helper I
        2 years ago

        Support request is 2312040050001409