REST API and python SDK for Spark jobs

The management of Spark jobs is done via the GUI Create an Apache Spark job definition - Microsoft Fabric | Microsoft Learn

This is fine for ad-hoc development, but not suitable for serious operationalising/automation of Spark pipelines. Please support a REST API and/or python SDK for developers to create/execute/monitor Spark jobs programmatically

data science

2 Comments

nishalit
New Member
3 years ago
RestAPI and SDK support for Spark Job is planned and we already activtly working on this feature..
fbcideas_migusr
New Member
1 year ago
Status added:
Planned

Recent ideas

Provide a Read-Only / Source-Safe Mode for Schema Shortcuts in Lakehouse
Schema Shortcuts provide a much better management experience than individual Table Shortcuts because they preserve the source schema structure and allow multiple tables to be onboarded at once. Howev...
Junghyoen
Microsoft Employee
2 hours ago
New
20Views
8likes
0Comments
Access Variable Library in Semantic Models
Enable the Variable Library as a centralized location for storing all environment‑specific variables, allowing us to adjust them for promotion scenarios (e.g., from dev to prod) without relying on de...
FreddyH
Advocate II
13 hours ago
New
920Views
31likes
2Comments
Enhance Fabric Pipeline Monitoring with Parent-Child Pipeline Lineage and Parameter Visibility
Currently, Microsoft Fabric Pipeline monitoring lacks several capabilities that are available in Azure Data Factory, making troubleshooting and operational support challenging in enterprise environme...
dwramreddy
Regular Visitor
13 hours ago
New
8Views
0likes
0Comments
Dynamic ADLS-Gen2 path input for Spark Jobs Main definition file
I would like the ability to add a dynamic input box on a spark job definition's "Main Definition File" "ADLS-Gen2 path". this would be useful to set base and variable paths across all spark jobs...
mfink_db
New Member
18 hours ago
New
238Views
2likes
2Comments
Reintroduce Tenant/Capacity Switch to Control "Users can create Plan items" Post-GA
During the Preview phase of Fabric Plan items, administrators had access to a dedicated tenant/capacity setting: "Users can create Plan items". With General Availability (GA), this granular administr...
Sri-Surendra_Ku
New Member
20 hours ago
New
106Views
14likes
1Comment

REST API and python SDK for Spark jobs

Provide a Read-Only / Source-Safe Mode for Schema Shortcuts in Lakehouse

Access Variable Library in Semantic Models

Enhance Fabric Pipeline Monitoring with Parent-Child Pipeline Lineage and Parameter Visibility

Dynamic ADLS-Gen2 path input for Spark Jobs Main definition file

Reintroduce Tenant/Capacity Switch to Control "Users can create Plan items" Post-GA