The ultimate Fabric, Power BI, SQL, and AI community-led learning event. Save €200 with code FABCOMM.
Get registeredCompete to become Power BI Data Viz World Champion! First round ends August 18th. Get started.
I have a list of companies that I need to seach for individually on a website. Once each company has been found, I need to return certain data items, such as date of registration and director names (all publically available info). How can I automate the input of each company name into the search field on the website and import the data for each company's resulting page?
Hi @Everyone ,
I second @PhilipTreacy idea.
Other than that this might be helpful.
16 Tools to Extract Data from Website (softr.io)
Best Regards,
Neeko Tang
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Hi @Everyone
You can't do this with PQ - it can't interact with a web page in this way.
You could try entering the company name on the web page, clicking the Search/Submit button and see if the URL chnages to show the search parameters. For example you might search for Microsoft and after clicking the Search button the URL changes to:
https://somewebsite.com/?company=Microsoft
You can use this URL structure to make multiple calls to the website for the data on each company. Create a column with the company names, then create a column that calls (for example) Web.Contents for each company.
Alternatively you can try Power Automate or a solution using VBA with Selenium, or there are multiple ways to scrape data using JavaScript or Python.
Regards
Phil
Proud to be a Super User!
Hi, thank you for your reply. That gives me a few things to try!
It looks like each company has a unique URL - they are not predictable but I can hopefully use VBA to find out what each one is.
thanks