Forum Discussion

NuTek's avatar
NuTek
Frequent Visitor
9 years ago

Web scraping a site that has dates in URL?

I would like to scrape the contents of a table on a web page.

The URL that is in the format:

     https://www.abc.org/research/stats/units_06_17

Where the 06 is the month and 17 is the year.

 

Is there a way to specify this for the  previous month using Power BI?

It look slike tere is a section for "HTTP request header parameters" but nothing 

striks me as someting I can use to format the above URL.

 

 

 

Need Help

3 Replies

  • v-jiascu-msft's avatar
    v-jiascu-msft
    Icon for Microsoft Employee rankMicrosoft Employee
    9 years ago

    Hi NuTek,

     

    Maybe you could try parameters.

    1. Add a parameter. It's called "URL" in my test. There are three values. (Yours may be "units_04_17", "units_05_17", "units_06_17")

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

    2. There will be a query when step 1 is done. Then connect to web source, select "Advanced". In the second part of URL, select parameter "URL".

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

    3. Every time I changed the parameter, the table "table0" would retrieve data from the website.

     

    Best Regards!

    Dale

     

  • v-jiascu-msft's avatar
    v-jiascu-msft
    Icon for Microsoft Employee rankMicrosoft Employee
    9 years ago

    Hi NuTek,

     

    Could you please tell me if your problem was resolved? Could you please share the answer or mark the proper answer as solution if it's convenient for you? That will be a help to the others.

    Best Regards!
    Dale

    • NuTek's avatar
      NuTek
      Frequent Visitor
      9 years ago

      It has not been resolved.  Closest answer seems to be to define report parameters, but that would prompt when the report is run and I want to automate the refresh using the Gateway.