The ultimate Fabric, Power BI, SQL, and AI community-led learning event. Save €200 with code FABCOMM.
Get registeredEnhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.
I'm choosing Onelake to store operational temp files (e.g temp data, manifest files, and some temporary logs). Back then, I was a fan of ADLS, for a container, where there had 3 zones raw, curated and processed layer with operational data stored in a single folder, thus, I can check all temporary, or operational output for activity related to data processing between tier.
But now, in MS Fabric, It's said that for each Lakehouse, there is a single tier (Bronze, Silver and Gold). Based on practices of experted in MS Fabric, where should I store the temp file when ther're all separated in each lakehouses now.?
p/s: I'm having 2 options: a temp folder for each Lakehouse or a single lakehouse just for storing operational data
Solved! Go to Solution.
Hi @nduc4nh,
In fact the Tier only consider when you design the lakehouse with medallion structures. They are three standalone lakehouses instead of one lakehouse with three Tiers.
The Bronze lakehouse used to data ingestion, the Silver lakehouse used to data transform, the Gold lakehouse used to consume and display the result. (Each level lakehouse ingest data from previous level)
Based on the medallion structure design, I suppose the temporary data should be stored in Bronze or Silver Tier.
These two Tiers all include transform operations. (the Bronze Tier getting data and apply simple cleanup on ingested data, the complex transform operation will processed on the Silver Tier)
Regards,
Xiaoxin Sheng
Hi @nduc4nh,
In fact the Tier only consider when you design the lakehouse with medallion structures. They are three standalone lakehouses instead of one lakehouse with three Tiers.
The Bronze lakehouse used to data ingestion, the Silver lakehouse used to data transform, the Gold lakehouse used to consume and display the result. (Each level lakehouse ingest data from previous level)
Based on the medallion structure design, I suppose the temporary data should be stored in Bronze or Silver Tier.
These two Tiers all include transform operations. (the Bronze Tier getting data and apply simple cleanup on ingested data, the complex transform operation will processed on the Silver Tier)
Regards,
Xiaoxin Sheng