Web scraping with search tool

[Pedro_Lourenco]

Hi,

I'm trying to get some data automatically from a website with the Power BI. Unfortunately, I'm not very familiar with this mechanic so if possible explain to me how to do it I will be very grateful.

I have a list with some codes that are the National Register of Legal Entities (as a sample the Table below) 

National Register Legal Entities
3883571000100
33913620000170
31964019000181
30509213000104
34172417000153
29298535000172
17329708000193
4299584000107
831328000150
4323964000121

First problem is that some of these entries need to insert a zero in front of the code in order to work in the website search tool and others will still not be localized by the website.

So, going to the mechanic... the website that I want to scrap the data is this one: https://magnetis.com.br/fundos-de-investimento/

Inside the field highlighted I insert the code from the list, then it returns me an option.

When I click in that box I'm redirected to the page with the information that I want.

*Notice that the URL changed.

There are basically four pieces of information that I want to scrap, three of them can be found in the blue box at the beginning of the page. Following our example:

Name: 051 Brucutu Fundo de Investimento Multimercado Crédito Privado

Gestora: Zero Cinco Um C API Tal Gestao de Recursos

Categoria: Multimercado

The last is found in a chart below the page and it is the TOTAL appointed in the far right dot of the chart.

In the example, the latest data is from January 9th, so it`s that one that I want it

Total PL: R$10,507,667.87

with these four information, I want to build a table like the one below with a National Register in every row. If the site don't find return the fields blank()

National Register Legal Entities	Name	Gestora	Categoria	Total PL
33913620000170	051 Brucutu Fundo de Investimento Multimercado Crédito Privado	Zero Cinco Um C API Tal Gestao de Recursos	Multimercado	R$10,507,667.87

Few... that was a lot haha

Can anyone help me?

Thanks

[Anonymous]

Hi @Pedro_Lourenco,

Unfortunate, you can't directly get data from web site(power query not able to invoke script and request redirect on web page, it can only get the HTML elements).
You need to find out the source API that web site invoked, then use power query web connector to get data from that API.

Regards,

Xioaxin Sheng