Currenly the only method to drop / truncate tables in a Lakehouse during a pipeline run is to start a spark session and initiate spark.sql to execute a sql statment.
In Data Factory the script activity does not support lakehouse as source and the sql endpoint of a lakehouse does not support drop.
Can an activity be created or the script activity updated in Data Factory pipelines to execute a sql statement on a lakehouse without having to start a notebook and spark session to do it.
Starting a spark session can take up to 5 mins so takes too long if we simply want drop/truncate a few lakehouse tables.
on:
DO NOT CONVERT my Workspace to Fabric Free Trial W...
on:
MS Fabric: Manage Connections and gateways with Fo...
on:
CU% over time: Filter by workspace (Capacity Metri...
VChimbili
on:
Add/Enhance CustomCode Transformation Node (JavaSc...