When to use T-SQL notebook

[Srisakthi]

Hello All,

Can you please suggest on what scenario i should consider using T-SQL notebooks for transformations.

1. I have to use only If I have warehouse tables and i need to do transformations on those data?

2. Can I use T-SQL notebooks on lakehouse tables for transformation and push the transformed data to warehouse?

Regards,

Srisakthi

[Srisakthi]

Hi @Anonymous ,

Thanks for your response. What I'm looking for is in which Scenario I should be using T-SQL notebooks. Because in data pipeline there is no option for T-SQL notebooks. T-SQL notebook to be used for manual analysis and has to be ran manually, no automation for this ?

Regards,

Srisakthi

[AndyDDC · Accepted solution]

Hi @Srisakthi IMHO T-SQL support in Notebooks is really for adhoc SELECT analysis over the Lakehouse SQL Endpoint.  I would not look do do any ETL/ELT code in Notebooks.

So for me, I use it for analysis of data, not for loading/engineering (Lakehouse SQL Endpoint is read only, cannot write data using T-SQL).

Hope that helps.

[Anonymous]

Hi @Srisakthi,

As you said, current it seems not support directly use t-sql model notebook.
For this scenario, I'd like to suggest create a new common notebook to invoke in pipeline and enter the notebook and manually switch its mode to 't-sql'. (I test with this and it works normally)

Regards,

Xiaoxin Sheng

[Anonymous]

Hi @Srisakthi,

#1, I think you can also use it to transform and show the temporary reuslt or save queries as view.

#2, I think dataflow or pipeline features should more suitable to push data from the Lakehouse. If you only use notebook, you may need to use some libraries and pyspark scripts to acheive these. (e.g. use mssparkutils and jdbc driver to write data from dataframe)

Solved: Re: Dataframe write to Warehouse through Notebook.... - Microsoft Fabric Community

Regards,

Xiaoxin Sheng