Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
versydney
New Member

Installing Packages

Has anyone been able to successfully install the spark-xml library on the spark pool in Fabric?

I added this in the workspace settings:

pd4voomnti2b1.png

After a few minutes, it was 'successful', but the notebook still complains:

8ddncj4zti2b1.png

00fe3e81ui2b1.png

Am I missing something? Thanks in advance Fabric community! 🙂

10 REPLIES 10
Joshrodgers123
Advocate V
Advocate V

@PRADEEPCH-MSFT Can you let us know if there is any feedback from the product team as to why this does not work and if there is a planned ETA for when this will work? This works perfectly fine in Synapse notebooks.

Joshrodgers123
Advocate V
Advocate V

I opened a support ticket for this, but they never got it working and said it was not supported now. They didn't give any timeline or any good reason as to why it doesn't work.

 

As a workaround, I am reading the XML with scala, which does work with this package. Then just doing the rest of what I need to do in Pysypark.

Thanks @Joshrodgers123, that's a great suggestion.

 

With a little help from ChatGPT I was able to rewrite parts of my notebook to use Scala, and then drop back into Pyspark to do the rest of the work.

 

It's a shame that Microsoft haven't got this working properly, especially as it works in Synapse Analytics, but this workaround will do for now.

 

I'm having the same issue with spark-xml. Have there been any updates on this?

Blog: sqlkover.com
Joshrodgers123
Advocate V
Advocate V

@PRADEEPCH-MSFT Was there any update on this? I'm running into the same issue on my end. 

wahmed20
Resolver I
Resolver I

You can install packages using the 

wahmed20_0-1686057127423.png

 

There is no feed library for spark-xml hence I used custom library.

PRADEEPCH-MSFT
Microsoft Employee
Microsoft Employee

@versydney - As per the repro, I was experiencing the same issue from my end and reached out to the internal team to get more information related to your query and will get back to you as soon as we have an update.

Stay Tuned!

Thanks @PRADEEPCH-MSFT , looking forward to hearing from MS.

Has there been any update from Microsoft on this?

I'm running into the same problem as @versydney .

 

Spark-XML package uploaded via Library Management on the Fabric Workspace:

 

tpenrose_0-1689924981965.png

 

In my notebook, I have the following code:

 

tpenrose_1-1689925126034.png

 

And this returns the error:
tpenrose_2-1689925174022.png

 

This is the third iteration of this particular notebook, having first written it a couple of years ago to work on Databricks, then last year it was converted to a Synapse Spark Pool notebook and now it's been converted to a Microsoft Fabric notebook.

 

In Databricks and Synapse, simply importing the spark-xml package was enough to get it working, but for some reason the same approach isn't working in Fabric.

 

 

Helpful resources

Announcements
Sept Fabric Carousel

Fabric Monthly Update - September 2024

Check out the September 2024 Fabric update to learn about new features.

September Hackathon Carousel

Microsoft Fabric & AI Learning Hackathon

Learn from experts, get hands-on experience, and win awesome prizes.

Sept NL Carousel

Fabric Community Update - September 2024

Find out what's new and trending in the Fabric Community.