Forum Discussion

ThomasDealFind's avatar
ThomasDealFind
Regular Visitor
2 years ago
Solved

Notebook Environment, Selenium installation

Hi everyone   I am trying to use Selenium python library in a notebook   I created an Environment and add Selenium and all library that seemed requiered to make it work but I can't make it work i...
data engineering
  • asanchez2's avatar
    asanchez2
    2 years ago

    Hi Thomas,

     

    I have been dealing with selenium installation in a notebook environment without success. In my case, I'm trying to scrap content from a website that involves authentication so I cannot use the provided solution of using Beautifulsoup. So I have to stick to selenium.

     

    In that direction, I managed to provide the chromedriver, and make it executable but its execution failed due to the security constraints of the environment. So I found the following solution:

     

    SOLUTION

    - Using a remote selenium service via LambdaTest

     # Sample code snippet:
        driver = webdriver.Remote(
            command_executor=f'https://{lambda_test_username}:{lambda_test_access_key}@hub.lambdatest.com/wd/hub',
            options=options
        )
     
    I hope this helps everyone whose solution has to be approached with selenium. 
    Aminael
v-cboorla-msft's avatar
v-cboorla-msft
Icon for Microsoft Employee rankMicrosoft Employee
2 years ago

Hi ThomasDealFind 

 

Thanks for using Microsoft Fabric Community.

As I understand that you want to scrap dynamic data from a web page within the Microsoft Fabric. Certainly! Scraping dynamic data from web pages within the Microsoft Fabric environment can be achieved using various approaches.

One approach is to use Beautiful Soup, a Python library for web scraping. You can create a BeautifulSoup object to parse the HTML content of a web page and extract relevant data.

Consider watching this YouTube video that demonstrates an end-to-end project in Microsoft Fabric involving web scraping and data storage in a lakehouse. 

Link : Web Scraping in Microsoft Fabric.

 

I hope this information helps. Please do let us know if you have any further questions.

 

Thanks.