Forum Discussion

waltheed's avatar
waltheed
Icon for Impactful Individual rankImpactful Individual
3 years ago

Landing zone in OneLake

We'd like to load data from source systems into a landing zone first, before merging it with our Bronze layer. All in OneLake of course.    If we try to do that, choosing overwrite as load option, ...
pipelines
ajarora's avatar
ajarora
Icon for Microsoft Employee rankMicrosoft Employee
3 years ago

Walther, Are you using pipelines copy activity to load data ?

If so, can you please explain further what do you mean by duplicate parquet files left around ? Are you compacting/vacuuming the destination table in lakehouse for the unused files ? If so let me know we may have something for you.