Starting December 3, join live sessions with database experts and the Microsoft product team to learn just how easy it is to get started
Learn moreGet certified in Microsoft Fabric—for free! For a limited time, get a free DP-600 exam voucher to use by the end of 2024. Register now
Hi everyone
I am trying to use Selenium python library in a notebook
I created an Environment and add Selenium and all library that seemed requiered to make it work but I can't make it work in my notebook
when using the library, my requierement is to use the function driver, in the library webdriver
For that I need to stecify the .exe driver of the webbrowser of my choice, but I can't uoload file other than :
Someone has try it and made it work ?
Thank you in advance
PS: my need is to scrap dynamic data from a web page, if you have another solution other than Selenium I would take it gladly
Solved! Go to Solution.
Hi Thomas,
I have been dealing with selenium installation in a notebook environment without success. In my case, I'm trying to scrap content from a website that involves authentication so I cannot use the provided solution of using Beautifulsoup. So I have to stick to selenium.
In that direction, I managed to provide the chromedriver, and make it executable but its execution failed due to the security constraints of the environment. So I found the following solution:
SOLUTION
- Using a remote selenium service via LambdaTest
I have the same issue. My script runs perfectly in jupyter notebook but now when I try to migrate it to Fabric I am getting errors/WebDriverExeptions. I have tried Beautiful Soup but it doesn't have the capabilities that are needed in this instance. If anyone has a solution on how to get selenium/webdriver installed and working in fabric it would be MUCH appreciated!
Hi Thomas,
I have been dealing with selenium installation in a notebook environment without success. In my case, I'm trying to scrap content from a website that involves authentication so I cannot use the provided solution of using Beautifulsoup. So I have to stick to selenium.
In that direction, I managed to provide the chromedriver, and make it executable but its execution failed due to the security constraints of the environment. So I found the following solution:
SOLUTION
- Using a remote selenium service via LambdaTest
I'm hoping to do something similar but at present my 'options' varaible is coming from this import:
Hello Aminael,
thank you for your response. It confort me with all the test I did.
My last resort was to try a report server and I am glad that you manage to did it.
I will try it myself anbd hope it work. thank you for the sample code.
It interesting to know that we can't handle authentication with Beautifulsoup. I am still a rookie in this field so at least I won't try this without Selenium.
Regards,
Thomas
Hello @ThomasDealFind ,
You can package an exe inside a JAR file : https://stackoverflow.com/questions/1383536/including-an-exe-file-to-jar
Thanks
Himanshu
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others.
Otherwise, will respond back with the more details and we will try to help.
Thanks.
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others.
If you have any question relating to the current thread, please do let us know and we will try out best to help you.
In case if you have any other question on a different issue, we request you to open a new thread.
Thanks.
Hello, thank you for your response, I already use beautifull soup but I need to retrieve data not accessible by it, in a javascript content. that is why I completed my script with selenium.
Thanks for using Microsoft Fabric Community.
As I understand that you want to scrap dynamic data from a web page within the Microsoft Fabric. Certainly! Scraping dynamic data from web pages within the Microsoft Fabric environment can be achieved using various approaches.
One approach is to use Beautiful Soup, a Python library for web scraping. You can create a BeautifulSoup object to parse the HTML content of a web page and extract relevant data.
Consider watching this YouTube video that demonstrates an end-to-end project in Microsoft Fabric involving web scraping and data storage in a lakehouse.
Link : Web Scraping in Microsoft Fabric.
I hope this information helps. Please do let us know if you have any further questions.
Thanks.
Starting December 3, join live sessions with database experts and the Fabric product team to learn just how easy it is to get started.
Check out the November 2024 Fabric update to learn about new features.
User | Count |
---|---|
7 | |
7 | |
5 | |
4 | |
1 |
User | Count |
---|---|
17 | |
13 | |
12 | |
8 | |
6 |