Installing Packages

[versydney]

Has anyone been able to successfully install the spark-xml library on the spark pool in Fabric?

I added this in the workspace settings:

After a few minutes, it was 'successful', but the notebook still complains:

Am I missing something? Thanks in advance Fabric community! 🙂

[Joshrodgers123]

@PRADEEPCH-MSFT Can you let us know if there is any feedback from the product team as to why this does not work and if there is a planned ETA for when this will work? This works perfectly fine in Synapse notebooks.

[Joshrodgers123]

I opened a support ticket for this, but they never got it working and said it was not supported now. They didn't give any timeline or any good reason as to why it doesn't work.

As a workaround, I am reading the XML with scala, which does work with this package. Then just doing the rest of what I need to do in Pysypark.

[tpenrose]

Thanks @Joshrodgers123, that's a great suggestion.

With a little help from ChatGPT I was able to rewrite parts of my notebook to use Scala, and then drop back into Pyspark to do the rest of the work.

It's a shame that Microsoft haven't got this working properly, especially as it works in Synapse Analytics, but this workaround will do for now.

[koenverbeeck]

I'm having the same issue with spark-xml. Have there been any updates on this?

[Joshrodgers123]

@PRADEEPCH-MSFT Was there any update on this? I'm running into the same issue on my end. 

[wahmed20]

You can install packages using the 

[versydney]

There is no feed library for spark-xml hence I used custom library.

[PRADEEPCH-MSFT]

@versydney - As per the repro, I was experiencing the same issue from my end and reached out to the internal team to get more information related to your query and will get back to you as soon as we have an update.

Stay Tuned!

[versydney]

Thanks @PRADEEPCH-MSFT , looking forward to hearing from MS.

[tpenrose]

Has there been any update from Microsoft on this?

I'm running into the same problem as @versydney .

Spark-XML package uploaded via Library Management on the Fabric Workspace:

In my notebook, I have the following code:

And this returns the error:

This is the third iteration of this particular notebook, having first written it a couple of years ago to work on Databricks, then last year it was converted to a Synapse Spark Pool notebook and now it's been converted to a Microsoft Fabric notebook.

In Databricks and Synapse, simply importing the spark-xml package was enough to get it working, but for some reason the same approach isn't working in Fabric.