Forum Discussion
Yahoo Finance Power Query Link Not Working
Hello,
I have a spreadsheet which downloads daily currency rates from Yahoo Finance using PowerQuery, for example...
= Csv.Document(Web.Contents("https://query1.finance.yahoo.com/v7/finance/download/USDGBP=X?period1=1609459200&period2=1924905600&interval=1d&events=history&includeAdjustedClose=true"),[Delimiter=",", Columns=7, Encoding=65001, QuoteStyle=QuoteStyle.None])
This has been working fine for months, and then today, when I go to refresh, I'm met with a message saying, " We couldn't authenticate with the credentials provided. Please try again".
This information is readily available and doesn't require an account. So why is this happening now? Please can anyone help this has broken many months of work (and I've not changed a thing since setting it up and getting it working).
30 Replies
- G05DVDFrequent Visitor
Hi,
Just to be clear I have a FREE Yahoo Finance account and have 2x queries in a Power BI report to scrape Web data.
Seems the endpoint has changed plus it now returns JSON as I think someone else has stated.
So changes to one of my original queries:
- Updated the URL to use the v8/finance/chart endpoint; I can confirm the new endpoint doesn't require authentication for basic historical data requests hence works with a free Yahoo Finance account
- Changed from Csv.Document to Json.Document as the new endpoint returns JSON
- Added steps to parse the JSON response and reconstruct the table
Obviously different members with differing scenarios but hopefully this will help members.
(StickerSymbol as text) as table => let EpochReferenceDate = #datetime(1970,1,1,0,0,0), DefaultStartDate = DateTime.From(Date.AddYears(Date.From(DateTime.LocalNow()),-5)), DefaultEndDate = DateTime.From(Date.From(DateTime.LocalNow())), StartDate = if Date.DayOfWeek(DefaultStartDate, Day.Monday) = 5 then Date.AddDays(DefaultStartDate,-1) else /* Saturday */ if Date.DayOfWeek(DefaultStartDate, Day.Monday) = 6 then Date.AddDays(DefaultStartDate,-2) else /* Sunday */ if Date.DayOfWeek(DefaultStartDate, Day.Monday) = 0 then Date.AddDays(DefaultStartDate,-3) else DefaultStartDate, /* Monday */ EndDate = if Date.DayOfWeek(DefaultEndDate, Day.Monday) = 5 then Date.AddDays(DefaultEndDate,-1) else /* Saturday */ if Date.DayOfWeek(DefaultEndDate, Day.Monday) = 6 then Date.AddDays(DefaultEndDate,-2) else /* Sunday */ if Date.DayOfWeek(DefaultEndDate, Day.Monday) = 0 then Date.AddDays(DefaultEndDate,-3) else DefaultEndDate, /* Monday*/ StartOfPeriod = Text.Start(Number.ToText(Duration.TotalSeconds(StartDate - EpochReferenceDate)),10), EndOfPeriod = Text.Start(Number.ToText(Duration.TotalSeconds(EndDate - EpochReferenceDate)),10), Source = Json.Document(Web.Contents("https://query1.finance.yahoo.com/v8/finance/chart/" & StickerSymbol & "?period1=" & StartOfPeriod & "&period2=" & EndOfPeriod & "&interval=1d&events=history&includeAdjustedClose=true")), Result = Source[chart][result]{0}, Timestamps = Result[timestamp], QuoteData = Result[indicators][quote]{0}, AdjCloseData = Result[indicators][adjclose]{0}, CombinedData = Table.FromColumns({ List.Transform(Timestamps, each Date.From(DateTimeZone.FromSecondsSinceEpoch(_))), QuoteData[open], QuoteData[high], QuoteData[low], QuoteData[close], AdjCloseData[adjclose], QuoteData[volume] }, {"Date", "Open", "High", "Low", "Close", "Adj Close", "Volume"}), #"Changed Type" = Table.TransformColumnTypes(CombinedData,{ {"Date", type date}, {"Open", type number}, {"High", type number}, {"Low", type number}, {"Close", type number}, {"Adj Close", type number}, {"Volume", Int64.Type} }) in #"Changed Type"
- A_V_BNew Member
This works. Thanks. The JSON chart is a text document, so use whatever you wish to fit the historical data into your existing scripts. I used awk to load and transpose the chart to the old CSV format. The only reason is that I've been using awk since the 1970s and it is second nature. Anyone younger won't do that. I automatically load hundreds of tables daily, Unix bash scripts.
- Paul_ClearCutITRegular Visitor
Nice.... I slightly predate the PC as we know it starting out on Sun Micrososytems SunOS & Silicon Graphics but AWK is new to me!
- Tomek1982Helper I
I found the problem. For me it was necessary to replace function:
Date.From(DateTimeZone.FromSecondsSinceEpoch([_]))
with
#datetime(1970, 1, 1, 0, 0, 0) + #duration(0,0,0,_)
- gecko407New Member
Same thing happening to me. Looks like they either intentionally or inadvertantly put this behind a login today.
- TheLittleWarrenRegular Visitor
Hi Guys, Yahoo is on the case, I have been having this issue via Power BI and Power Shell since Friday (possibly Thursday) myself, so I reached out to them since I have a model which took me 1.5 years double full time to build and I was ready to pay for the data; apparently no need to, finger crossed since the market is hot at the moment...
Hi Marco,
Thanks for reaching out to Yahoo Finance.
I apologize for the inconvenience. I understand that you're experiencing issues with the feature to download historical data.
Please know our Engineering team is continuing to work to fix this known issue for you. We know it’s frustrating when things don't work and appreciate your patience while we correct the problem.
If you have any other questions or can provide further information regarding this issue, let us know by replying to this message.Best,
Lucas
Yahoo Customer Care- NutFlushFrequent Visitor
Hey that's really GREAT!!! 😀👍
I tried contacting them myself, but found out that they make it really hard to get in touch with their support (and it appeared as if you need to be a paying customer as well, but I might be mistaken).
Anyway, thanks to you we at least know that they are aware of the issue. Please keep us posted!!!
- gecko407New Member
This URL will give the same data, but the response is formatted in JSON:
- Dash_Riprock_UKFrequent Visitor
Hello,
Many thanks for your responses, it's much appreciated.
I've never used JSON... I tried the url in PowerQuery, but it's not clear what to do with it. What are the steps to get this into just a data table?
Also, my end date is (was) dynamic, i.e. previously, my query would search Yahoo Finance for today's date minus 1 day and return the data. How does one incorporate that into the query?
- NutFlushFrequent Visitor
I have the exact same problem. I believe it started on Wednesday or Thursday this week (Sep. 04 or 05 2024).
Please post here, if somebody finds a way to resolve this, because I need the data download on a daily basis! Maybe somebody with a paid YAHOO subscription can let us know if only the free version has this problem? - nickboddingtonNew Member
Hi
I had exactly the same problem as you describe. I have found the solution by using the STOCKHISTORY function in Excel:
HTH
Nick
- NutFlushFrequent Visitor
Thanks for your reply, Nick!
Unfortunately this only works with Office 365. I have a standalone version of EXCEL 2021 that doesn't support this function.
- Jase68Helper I
I am getting the same problem since 7th September 2024.
- G05DVDFrequent Visitor
Hi,
I have the same problem with a long standing Power BI report that scrapes stock data.
Does appear to be change in authentication on the Yahoo Finance end. Member has already commented would be good to hear from someone who has a paid subscription which I do not.
Many thanks...
- Vampi77New Member
I'm having same issue with queries to get stocks data. Have someone found a solution to avoid editing the queries that were working for months?
- kar2022Frequent Visitor
Can someone please help as I followed the above and I am unable to resolve. I changed to "Json.Documents" and I do not know what else to change.
(StockQuote as text) as table =>
let
today = Duration.TotalSeconds(DateTime.LocalNow() - #datetime(1970,1,1,0,0,0)),
lastyear = Duration.TotalSeconds(Date.AddYears(DateTime.LocalNow(),-5) - #datetime(1970,1,1,0,0,0)),
Source = Json.Document(Web.Contents("https://query1.finance.yahoo.com",[RelativePath="v7/finance/download/"&StockQuote&"?period1="&Number.ToText(Number.Round(lastyear))&"&period2="&Number.ToText(Number.Round(today))&"&interval=1d&events=history&includeAdjustedClose=true"]),[Delimiter=",", Columns=7, Encoding=65001, QuoteStyle=QuoteStyle.None]),
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"Date", type date}, {"Open", type number}, {"High", type number}, {"Low", type number}, {"Close", type number}, {"Adj Close", type number}, {"Volume", Int64.Type}})
in
#"Changed Type" - Dash_Riprock_UKFrequent Visitor
Thanks to all for their help and suggestions. Sadly, I couldn't get the JSON links to work meaningfully. Either it involved playing around with many steps (at which point it was difficult to feel confident about data quality) or I couldn't achieve normal dates.
After much faffing I gave up with Yahoo (hopefully they fix it in the future).
For now, I found this website... https://eodhd.com/
Free option... easy to use API... 20 free requests per day.
- G05DVDFrequent Visitor
Thanks for this
One issue I have with Yahoo Finance is I scrape data for a small number of ETF investments and said data doesn't include trade volumes. This could provide an alternative data source
- Mathf18Regular Visitor
(Symbol as text,min_date as text, max_date as text) as table =>
let
Source = Json.Document(Web.Contents("https://query2.finance.yahoo.com", [RelativePath = "/v8/finance/chart/" & Symbol & "?period1=" & min_date & "&period2=" & max_date & "&interval=1d&events=history&includeAdjustedClose=true"])),
// Extract the relevant parts of the JSON response
ChartData = Source[chart][result]{0},
Timestamps = ChartData[timestamp],
Quotes = ChartData[indicators][quote]{0},
// Convert the JSON into a table
CombinedData = Table.FromColumns({
List.Transform(Timestamps, each Text.From(_)),
Quotes[open],
Quotes[high],
Quotes[low],
Quotes[close],
Quotes[volume]
}, {"DateUnix", "Open", "High", "Low", "Close", "Volume"}),
#"Changed Type" = Table.TransformColumnTypes(CombinedData,{{"DateUnix", Int64.Type}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Date.1", each DateTimeZone.SwitchZone(
#datetimezone(1970, 1, 1, 0, 0, 0, 0, 0) + #duration(0, 0, 0, [DateUnix] ),
2, 0
)),
#"Renamed Columns" = Table.RenameColumns(#"Added Custom",{{"Date.1", "Date"}}),
#"Removed Columns" = Table.RemoveColumns(#"Renamed Columns",{"DateUnix"})
in
#"Removed Columns"
this work - Jase68Helper I
Update: I never managed to resolve this issue but I am not very technically minded so some of the suggestions may have worked but were beyond me. To resolve this I switched my data source from the Yahoo web source to a Google Sheets file which used the GOOGLEFINANCE function to pull the shareprice, making sure the file was "public" and then connected my Power BI to that source.
- kar2022Frequent Visitor
Hi Jase68,
The Google Finance will not work with me the way data is collected. Thanks