Webscraping a Webpage with a Login and Password

[Anonymous]

I need some help working through an issue with trying to pull into both desktop and dataflows of a table of data behind a login and password.

The data is coming from https://fcafuel.org/historical-fuel-index/.

The public version includes historical data, yet not the current records.

The entire historical table is only available behind the pay wall.

I want to pull this data into a dataset for a report.

[v-diye-msft]

Hi @Anonymous 

Let me know if you'd like to get below data:

You might refer to this article:

https://www.poweredsolutions.co/2018/05/14/new-web-scraping-experience-in-power-bi-power-query-using-css-selectors/ 

[Anonymous]

@v-diye-msft  Thanks for the suggestion.

I've already tried similar methods and run into the issues of not being able to pass/use login and password to access current values.

Instead, I run into the "Table highlighting is disabled..." error like in the thread below.

Get error message as Table Highlighting is disable... 

[PaulDBrown]

@Anonymous 

I tried to extract data form a website which required credentials and failed dismally. I eventually succeeded with a long workaround using Excel.

If you're interested, you can see how in this thread:

Get data from web with credentials 

[Anonymous]

@PaulDBrown 

Thanks! I'll look into the link you sent and try it out.

[PaulDBrown]

@Anonymous 

Good luck! (btw, you still need the credentials to get into the dataset- the solution in the thread only solves the problem of getting access to the dataset going through the login requirements; it does not provide a back-entry avoiding login...)

EDIT: I just did a search for a solution using Python and came across this:

Scraping data from web which requires login using Python