Forum Discussion
Helper IISelect the fabric kernel to debug spark job definition in VSCode
Currently, Spark Job Definitions cannot be debugged using the Fabric kernel in VSCode. The Fabric kernel is mainly designed for running notebooks, whereas Spark Job Definitions require execution on the remote Spark Runtime. If you try to debug a Spark Job Definition using the Fabric kernel, it won’t function as expected since it lacks the necessary Spark execution environment.
To properly develop and debug Spark jobs, you need to select the remote Spark Runtime, which provides the required Spark components and dependencies. This ensures that your job runs in an environment similar to the one it will be deployed in, allowing for accurate testing and debugging.
If you were expecting this functionality within the Fabric kernel itself, it would be worth checking Microsoft’s official roadmap or documentation for any future support. Having the ability to debug Spark Job Definitions directly in the Fabric kernel would be a valuable enhancement, as it would streamline development and testing workflows without requiring an external runtime.
Are there any specific issues you’re facing while debugging your Spark Job Definition? Let me know how I can assist further!
Community SupportHi Broeks,
We sincerely appreciate your update.
We are reaching out to confirm whether the support ticket you raised has successfully resolved the issue. If so, we would appreciate it if you could share the solution with the community and mark it as the accepted solution. This will help others encountering similar challenges and contribute to the broader community’s benefit.
Thank you.
Helper IIHello,
I've raised a ticket and I'm still waiting for a response. Once I've got the solution I'll share it here.
greetings,
Martijn