Forum Discussion

ThomasDealFind's avatar
ThomasDealFind
Regular Visitor
2 years ago
Solved

Notebook Environment, Selenium installation

Hi everyone   I am trying to use Selenium python library in a notebook   I created an Environment and add Selenium and all library that seemed requiered to make it work but I can't make it work i...
data engineering
  • asanchez2's avatar
    asanchez2
    2 years ago

    Hi Thomas,

     

    I have been dealing with selenium installation in a notebook environment without success. In my case, I'm trying to scrap content from a website that involves authentication so I cannot use the provided solution of using Beautifulsoup. So I have to stick to selenium.

     

    In that direction, I managed to provide the chromedriver, and make it executable but its execution failed due to the security constraints of the environment. So I found the following solution:

     

    SOLUTION

    - Using a remote selenium service via LambdaTest

     # Sample code snippet:
        driver = webdriver.Remote(
            command_executor=f'https://{lambda_test_username}:{lambda_test_access_key}@hub.lambdatest.com/wd/hub',
            options=options
        )
     
    I hope this helps everyone whose solution has to be approached with selenium. 
    Aminael
asanchez2's avatar
asanchez2
New Member
2 years ago

Hi Thomas,

 

I have been dealing with selenium installation in a notebook environment without success. In my case, I'm trying to scrap content from a website that involves authentication so I cannot use the provided solution of using Beautifulsoup. So I have to stick to selenium.

 

In that direction, I managed to provide the chromedriver, and make it executable but its execution failed due to the security constraints of the environment. So I found the following solution:

 

SOLUTION

- Using a remote selenium service via LambdaTest

 # Sample code snippet:
    driver = webdriver.Remote(
        command_executor=f'https://{lambda_test_username}:{lambda_test_access_key}@hub.lambdatest.com/wd/hub',
        options=options
    )
 
I hope this helps everyone whose solution has to be approached with selenium. 
Aminael
  • ThomasDealFind's avatar
    ThomasDealFind
    Regular Visitor
    2 years ago

    Hello Aminael,

     

    thank you for your response. It confort me with all the test I did.

    My last resort was to try a report server and I am glad that you manage to did it.

     

    I will try it myself anbd hope it work. thank you for the sample code.

     

    It interesting to know that we can't handle authentication with Beautifulsoup. I am still a rookie in this field so at least I won't try this without Selenium.

     

    Regards,

    Thomas

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    I'm hoping to do something similar but at present my 'options' varaible is coming from this import:

     
    from selenium.webdriver.chrome.options import Options
     
    which of course, fabric doesnt like.
     
    What are you using for options = options?