Forum Discussion

Jammaz's avatar
Jammaz
New Member
3 years ago

Web Scraping extended time per webpage

Hi I'll, I've set up a web scape to pull information from a flight search website. The website itself updates each time a user visits each page. Each page can take approx. 30 second to update when visited. 

 

The web scape is returning nulls where pages haven't been visited. I suspect this is due to the scrape visiting for a very brief time so only returning the available info. Is there a way of extending the time the scrape visits each page? 

 

Thanks, J

No RepliesBe the first to reply