Forum Discussion

ThomasDealFind's avatar
ThomasDealFind
Regular Visitor
2 years ago
Solved

Notebook Environment, Selenium installation

Hi everyone   I am trying to use Selenium python library in a notebook   I created an Environment and add Selenium and all library that seemed requiered to make it work but I can't make it work i...
data engineering
  • asanchez2's avatar
    asanchez2
    2 years ago

    Hi Thomas,

     

    I have been dealing with selenium installation in a notebook environment without success. In my case, I'm trying to scrap content from a website that involves authentication so I cannot use the provided solution of using Beautifulsoup. So I have to stick to selenium.

     

    In that direction, I managed to provide the chromedriver, and make it executable but its execution failed due to the security constraints of the environment. So I found the following solution:

     

    SOLUTION

    - Using a remote selenium service via LambdaTest

     # Sample code snippet:
        driver = webdriver.Remote(
            command_executor=f'https://{lambda_test_username}:{lambda_test_access_key}@hub.lambdatest.com/wd/hub',
            options=options
        )
     
    I hope this helps everyone whose solution has to be approached with selenium. 
    Aminael
Anonymous's avatar
Anonymous
Not applicable
2 years ago

I have the same issue. My script runs perfectly in jupyter notebook but now when I try to migrate it to Fabric I am getting errors/WebDriverExeptions. I have tried Beautiful Soup but it doesn't have the capabilities that are needed in this instance. If anyone has a solution on how to get selenium/webdriver installed and working in fabric it would be MUCH appreciated!