Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
5 years ago

How to bypass cookie popup when web scraping?

Hi there

 

Many websites have popups asking what cookies to allow, how do I get past these popups and load the content directly?

Best Regardes
Søren

8 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi Anonymous ,

     

    Based on my test, the data scraped by the webpage does not seem to be affected by the cookie.

    let
        Source = Web.BrowserContents("https://www.betriheim.fo/solulisti/"),
        #"Extracted Table From Html" = Html.Table(Source, {{"Column1", ".section-inner BR + A"}, {"Column2", ".suggestedPrice"}, {"Column3", ".latestBid"}, {"Column4", ".icon-buildingSize + *"}, {"Column5", ".icon-buildingBedrooms + *"}, {"Column6", ".icon-plotSize + *"}, {"Column7", ".icon-buildingFloors + *"}, {"Column8", ".icon-buildingYearBuilt + *"}, {"Column9", ".suggestedPrice *"}}, [RowSelector=".section-inner BR + A"]),
        #"Changed Type" = Table.TransformColumnTypes(#"Extracted Table From Html",{{"Column1", type text}, {"Column2", type text}, {"Column3", type text}, {"Column4", type text}, {"Column5", type text}, {"Column6", type text}, {"Column7", type text}, {"Column8", type text}, {"Column9", type text}})
    in
        #"Changed Type"

     

    I used to get data from the web, and the credentials were obtained anonymously.

     

     

    Best Regards,

    Stephen Tao

     

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

    • Anonymous's avatar
      Anonymous
      Not applicable

      Hi Stephen
      Thanks for your effort.

      Your proposed script does not provide usable data, I get a table with the email adress [email protected] in column1 as the only data.

      Am I doing something wrong?

      Best regards
      Søren

      • Anonymous's avatar
        Anonymous
        Not applicable

        Hi Anonymous ,

         

        My fault, sorry. Because copying the code directly to the advanced editor seems to make an error. Please try to follow the steps in my gif to connect to the web.

         

         

        Best Regards,

        Stephen Tao

         

        If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

         

    • Syndicate_Admin's avatar
      Syndicate_Admin
      Administrator

      Hi Stephen
      Thanks for your effort.

      Your proposed script does not provide usable data, I get a table with the email adress [email protected] in column1 as the only data.

      sosje_0-1629442407825.png

      Am I doing something wrong?

      Best regards
      Søren