Forum Discussion
Access Fabric Spark Context Remotely
I suspect I know the answer to this but I'll ask anyway to see if anyone has a good idea.
I want to use Fabric to do my data processing, then I want to store my data and give access to external applications.
I know that I can use the SQL Server ODBC driver to query the data, but sometimes the data comes in formats thats supported in Spark but not in the ODBC driver, is it possible to get access to the Spark context to run queries on the Cluster as you would from a notebook from outside of Fabric for application development?
2 Replies
Hi Anonymous ,
In Fabric, you can do this with the Apache Livy service.
The installation, configuration, and usage of Livy is described in detail here, which you can refer to:
- Start by installing and running Livy on your Fabric Spark cluster.
- Set up Livy to support remote access and authentication (e.g. with Microsoft Entra).
- Submit Spark jobs, monitor status, and get results via Livy's REST API.
Best Regards,
Yang
Community Support TeamIf there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!Hi Anonymous ,
Is my follow-up just to ask if the problem has been solved?
If so, can you accept the correct answer as a solution or share your solution to help other members find it faster?
Thank you very much for your cooperation!
Best Regards,
Yang
Community Support TeamIf there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!