Forum Discussion

Shawn_Eary's avatar
Shawn_Eary
Advocate V
2 years ago
Solved

Version Control for Lakehouse File Uploads

When I upload files into an MS Fabric backed lakehouse, I don't think any version control is used: (175) Manually Upload Large CSV files to a Microsoft Fabric Lakehouse - YouTube https://www.youtub...
one lake
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi Shawn_Eary ,
    The internal team has updated me regarding version control in Fabric.

    Git integration is for doing version control on code, not data files. OneLake, isn't connected to Git Integration. Instead it is basically ADLS gen2. 
    You can go through this link for reference : Solved: Delta lake time travel in Fabric SQL endpoints? - Microsoft Fabric Community
    Instead if you want file versioning you can follow the below steps:

    1)  You can connect to Blob Storage using the Dataflows or Data Factory connectors, or in a python notebook using the Storage APIs directly.

     

    2) You can use GIT to version your files . Just stick them in the repo and then again connect to the repo using a python notebook. 

    Hope this helps . Please let us know if you have any further queries.

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi Shawn_Eary ,

Thanks for using the Fabric community and reporting this . 

I have reached the internal team for help on this. I will update you once I hear from them.

Appreciate your patience.

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi Shawn_Eary ,
    The internal team has updated me regarding version control in Fabric.

    Git integration is for doing version control on code, not data files. OneLake, isn't connected to Git Integration. Instead it is basically ADLS gen2. 
    You can go through this link for reference : Solved: Delta lake time travel in Fabric SQL endpoints? - Microsoft Fabric Community
    Instead if you want file versioning you can follow the below steps:

    1)  You can connect to Blob Storage using the Dataflows or Data Factory connectors, or in a python notebook using the Storage APIs directly.

     

    2) You can use GIT to version your files . Just stick them in the repo and then again connect to the repo using a python notebook. 

    Hope this helps . Please let us know if you have any further queries.

    • maxkent's avatar
      maxkent
      Frequent Visitor
      1 year ago

      Hi,

       

      do you have any steps/example in order to implement what you suggested?

       

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      Hi Shawn_Eary ,
      It was great to know that you were able to get to a resolution. We expect you to keep using this forum and also motivate others to do that same. 

      Thanks
      Nikhila N