Forum Discussion
Spark SQL query data in one datalake and store transformed data in another data lake
Hi SergioTorrinha , Hope your doing well.
In this scenario i suggest you to raise a support ticket here. so, that they can assit you in addressing the issue you are facing. please follow below link on how to raise a support ticket:
How to create a Fabric and Power BI Support ticket - Power BI | Microsoft Learn
Thanks,
Prashanth Are
MS Fabric community support
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and give Kudos if helped you resolve your query
Hello SergioTorrinha
Assume you’ve created a shortcut named bronze_tableA in your Silver workspace that points to the original tableA in Bronze. Then your notebook cell could look like this
%%sql
CREATE OR REPLACE TABLE tableA_silver AS
SELECT
col1,
col2,
col4,
col5
FROM bronze_tableA
Hi nilendraFabric !
Thanks for you'r input.
Do I really need to create shortcuts, for this kind of task, even if these lakehouses are within the same tenant/domain/workspace?
Imagine I have multiple tables in bronze_lh, for which I would like to do the same sort of operations. Would I need to create those shortcuts in the silver_lh? How is one automating such task?
Thank you.Hello SergioTorrinha
In your notebook, you need access to the Bronze table. If both lakehouses share the same workspace, you can reference the table directly (for example, using its fully qualified name). Otherwise, create a shortcut in your Silver lakehouse that points to the Bronze table
-- Using fully qualified name (catalog.schema.table)
CREATE OR REPLACE TABLE silver_lh.tableA_silver AS
SELECT
Cola
FROM bronze_lh.tableAOk, now this is making more sense, I'll have to try this one out.
Thank you for your help!
