Forum Discussion
pmscorca
Post Prodigy
Post ProdigyUsing MLV to write data from a lakehouse into a warehouse
Hi, for a Fabric solution I've a lakehouse to save data read from some sources and a warehouse having a staging area and a dwh area. I think to use some MLVs to perform some transformation on raw d...
I stand corrected, you can use a shortcut, but you dont have to, you can query the MLV directly and insert it in a staging table in the warehouse, and from there ETL to your gold layer, using a pipeline
Some sample code:TRUNCATE TABLE stg.factSales_stg;INSERT INTO stg.factSales_stg(sale_id, sale_date, customer_id, product_id,quantity, amount, region_name, last_update_ts)SELECTsale_id, sale_date, customer_id, product_id,quantity, amount, region_name, SYSUTCDATETIME()FROM lh_sales.sales.v_fact_sales_mlvMLV to staging table, staging table ETL to gold layer is the considered best practice.
pmscorcaPost Prodigy
Post ProdigyHi, do you say that the best pratice to put MLV data into a warehouse is to use shortcuts without implementing any ETLs, obtaining optimal performances?
smeetsh
Continued Contributor
Continued ContributorI stand corrected, you can use a shortcut, but you dont have to, you can query the MLV directly and insert it in a staging table in the warehouse, and from there ETL to your gold layer, using a pipeline
Some sample code:
TRUNCATE TABLE stg.factSales_stg;
INSERT INTO stg.factSales_stg
(
sale_id, sale_date, customer_id, product_id,
quantity, amount, region_name, last_update_ts
)
SELECT
sale_id, sale_date, customer_id, product_id,
quantity, amount, region_name, SYSUTCDATETIME()
FROM lh_sales.sales.v_fact_sales_mlv
MLV to staging table, staging table ETL to gold layer is the considered best practice.