Forum Discussion

junminn30's avatar
junminn30
Regular Visitor
1 year ago
Solved

How does Dataflow gen2 handles flat (txt) files

My txt file has blank rows, headers and footers. I do see these rows when load as txt file into dataflow gen2 but after publishing it to lakehouse, I don't see those rows anymore. I didn't perform an...
one lake
  • suparnababu8's avatar
    suparnababu8
    1 year ago

    Hi junminn30 

     

    In Lakehouse, you will not see blank rows becuase it's has limitation (show upto 1000 rows). I replicated your scenario. 
    Lets's say I have a txt file(this i uploaded from my machine into dataflow gen2) called Sales_Fabric with column called SalesOrderLinenumber having blanks

    Later I published this into my Sample_LH lakehouse. At that I'm able to see like this. Assume that I'm not able to see blank rows.

    But if i query the table in SQL analytics end point, It clearly showing blanks in SalesOrderLinenumber column

    I think it's sorting in differet way in the lakehouse. 

     

    Hope you are clear now. Let me know if it works

     

    Thank you!!

     

    Did I answer your question? Mark my post as a solution!

    Proud to be a Super User!

     

     

     

     

suparnababu8's avatar
suparnababu8
Icon for Super User rankSuper User
1 year ago

Hi junminn30 

 

In Lakehouse, you will not see blank rows becuase it's has limitation (show upto 1000 rows). I replicated your scenario. 
Lets's say I have a txt file(this i uploaded from my machine into dataflow gen2) called Sales_Fabric with column called SalesOrderLinenumber having blanks

Later I published this into my Sample_LH lakehouse. At that I'm able to see like this. Assume that I'm not able to see blank rows.

But if i query the table in SQL analytics end point, It clearly showing blanks in SalesOrderLinenumber column

I think it's sorting in differet way in the lakehouse. 

 

Hope you are clear now. Let me know if it works

 

Thank you!!

 

Did I answer your question? Mark my post as a solution!

Proud to be a Super User!

 

 

 

 

  • junminn30's avatar
    junminn30
    Regular Visitor
    1 year ago

    Thank you so much. I've been quite busy with other projects and didn't have the time to respond. I will try and get back to this message if I face any problems.