Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Reply
versydney
New Member

Installing Packages

Has anyone been able to successfully install the spark-xml library on the spark pool in Fabric?

I added this in the workspace settings:

pd4voomnti2b1.png

After a few minutes, it was 'successful', but the notebook still complains:

8ddncj4zti2b1.png

00fe3e81ui2b1.png

Am I missing something? Thanks in advance Fabric community! 🙂

10 REPLIES 10
Joshrodgers123
Advocate IV
Advocate IV

@PRADEEPCH-MSFT Can you let us know if there is any feedback from the product team as to why this does not work and if there is a planned ETA for when this will work? This works perfectly fine in Synapse notebooks.

Joshrodgers123
Advocate IV
Advocate IV

I opened a support ticket for this, but they never got it working and said it was not supported now. They didn't give any timeline or any good reason as to why it doesn't work.

 

As a workaround, I am reading the XML with scala, which does work with this package. Then just doing the rest of what I need to do in Pysypark.

Thanks @Joshrodgers123, that's a great suggestion.

 

With a little help from ChatGPT I was able to rewrite parts of my notebook to use Scala, and then drop back into Pyspark to do the rest of the work.

 

It's a shame that Microsoft haven't got this working properly, especially as it works in Synapse Analytics, but this workaround will do for now.

 

I'm having the same issue with spark-xml. Have there been any updates on this?

Blog: sqlkover.com
Joshrodgers123
Advocate IV
Advocate IV

@PRADEEPCH-MSFT Was there any update on this? I'm running into the same issue on my end. 

wahmed20
Resolver I
Resolver I

You can install packages using the 

wahmed20_0-1686057127423.png

 

There is no feed library for spark-xml hence I used custom library.

@versydney - As per the repro, I was experiencing the same issue from my end and reached out to the internal team to get more information related to your query and will get back to you as soon as we have an update.

Stay Tuned!

Thanks @PRADEEPCH-MSFT , looking forward to hearing from MS.

Has there been any update from Microsoft on this?

I'm running into the same problem as @versydney .

 

Spark-XML package uploaded via Library Management on the Fabric Workspace:

 

tpenrose_0-1689924981965.png

 

In my notebook, I have the following code:

 

tpenrose_1-1689925126034.png

 

And this returns the error:
tpenrose_2-1689925174022.png

 

This is the third iteration of this particular notebook, having first written it a couple of years ago to work on Databricks, then last year it was converted to a Synapse Spark Pool notebook and now it's been converted to a Microsoft Fabric notebook.

 

In Databricks and Synapse, simply importing the spark-xml package was enough to get it working, but for some reason the same approach isn't working in Fabric.

 

 

Helpful resources

Announcements
April AMA free

Microsoft Fabric AMA Livestream

Join us Tuesday, April 09, 9:00 – 10:00 AM PST for a live, expert-led Q&A session on all things Microsoft Fabric!

March Fabric Community Update

Fabric Community Update - March 2024

Find out what's new and trending in the Fabric Community.