Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Don't miss out! 2025 Microsoft Fabric Community Conference, March 31 - April 2, Las Vegas, Nevada. Use code MSCUST for a $150 discount. Prices go up February 11th. Register now.

Reply
Dash_Riprock_UK
Frequent Visitor

Yahoo Finance Power Query Link Not Working

Hello,

I have a spreadsheet which downloads daily currency rates from Yahoo Finance using PowerQuery, for example...

 

= Csv.Document(Web.Contents("https://query1.finance.yahoo.com/v7/finance/download/USDGBP=X?period1=1609459200&period2=1924905600&..."),[Delimiter=",", Columns=7, Encoding=65001, QuoteStyle=QuoteStyle.None])

 

This has been working fine for months, and then today, when I go to refresh, I'm met with a message saying, " We couldn't authenticate with the credentials provided.  Please try again".

 

This information is readily available and doesn't require an account.  So why is this happening now?  Please can anyone help this has broken many months of work (and I've not changed a thing since setting it up and getting it working).

30 REPLIES 30
TheLittleWarren
Regular Visitor

here is the M language translation of the same M language code that before was working with the connection https://query1.finance.yahoo.com/v7/finance/download

this maintanes the exact same format. ChatGPT did it for me in a single passage after I prepared a well formatted request, simply unbelivable. This code deals with a day only, the current, since the historical data I load into sql server and combine with the live ones via DAX, but it can easily be adapted to process a period, chatgpt will do for you in no time. Only relevant parameter is Symbol, Cur and CountrySE are currency and stock exchange country but probably not relevant in your code. This is a function that cycles multiple symbols one by one, I load around 20k symbols and it does not work in a multiple-symbol string. Hope this helps.

let
Source = (Symbol as text, Cur as text, CountrySE as text) => let

// Calculate the date differences to generate period1 and period2
DateDiff = Duration.Days(Date.From(DateTime.LocalNow()) - #date(1970, 1, 1)),
DateDiffSecL = DateDiff * 24 * 60 * 60,
DateDiffSecU = (DateDiff + 1) * 24 * 60 * 60,
DateDiff2SecLT = Number.ToText(DateDiffSecL),
DateDiff2SecUT = Number.ToText(DateDiffSecU),

// Get JSON data from Yahoo Finance API
Source = Web.Contents(
"https://query2.finance.yahoo.com/v8/finance/chart/" & Symbol,
[
Query = [
period1 = DateDiff2SecLT,
period2 = DateDiff2SecUT,
interval = "1d"
],
ManualStatusHandling = {404}
]
),

// Parse JSON response
JsonResponse = Json.Document(Source),

// Check for errors in the JSON response
Chart = JsonResponse[chart],
Result = Chart[result]{0},
Meta = Result[meta],
Indicators = Result[indicators],
Quote = Indicators[quote]{0},
Timestamps = Result[timestamp],
AdjClose = Indicators[adjclose]{0}[adjclose],

// Convert timestamps (in seconds) to dates
Dates = List.Transform(Timestamps, each DateTime.From(#datetime(1970, 1, 1, 0, 0, 0)) + #duration(0, 0, 0, _)),

// Combine the data into a table
DataTable = Table.FromColumns({
Dates,
Quote[open],
Quote[high],
Quote[low],
Quote[close],
AdjClose,
Quote[volume]
}, type table [Date=datetime, Open=number, High=number, Low=number, Close=number, #"Adj Close"=number, Volume=Int64.Type]),

// Change data types (if necessary, although columns should already be typed correctly)
#"Changed Type" = Table.TransformColumnTypes(DataTable, {
{"Date", type date},
{"Open", type number},
{"High", type number},
{"Low", type number},
{"Close", type number},
{"Adj Close", type number},
{"Volume", Int64.Type}
}),

// Add Symbol column
#"Added Symbol Column" = Table.AddColumn(#"Changed Type", "Symbol", each Symbol),

// Filter out any erroneous data (e.g., dates or close prices that are null or empty)
#"Filtered Rows" = Table.SelectRows(#"Added Symbol Column", each [Date] <> #date(1900, 1, 1) and [Close] <> null and [Close] <> "")

in
#"Filtered Rows"
in
Source




Jase68
Helper I
Helper I

Update: I never managed to resolve this issue but I am not very technically minded so some of the suggestions may have worked but were beyond me.  To resolve this I switched my data source from the Yahoo web source to a Google Sheets file which used the GOOGLEFINANCE function to pull the shareprice, making sure the file was "public" and then connected my Power BI to that source.

Hi Jase68,

The Google Finance will not work with me the way data is collected.  Thanks

 

Mathf18
Regular Visitor

(Symbol as text,min_date as text, max_date as text) as table =>

let
Source = Json.Document(Web.Contents("https://query2.finance.yahoo.com", [RelativePath = "/v8/finance/chart/" & Symbol & "?period1=" & min_date & "&period2=" & max_date & "&interval=1d&events=history&includeAdjustedClose=true"])),
// Extract the relevant parts of the JSON response
ChartData = Source[chart][result]{0},
Timestamps = ChartData[timestamp],
Quotes = ChartData[indicators][quote]{0},
// Convert the JSON into a table
CombinedData = Table.FromColumns({
List.Transform(Timestamps, each Text.From(_)),
Quotes[open],
Quotes[high],
Quotes[low],
Quotes[close],
Quotes[volume]
}, {"DateUnix", "Open", "High", "Low", "Close", "Volume"}),
#"Changed Type" = Table.TransformColumnTypes(CombinedData,{{"DateUnix", Int64.Type}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Date.1", each DateTimeZone.SwitchZone(
#datetimezone(1970, 1, 1, 0, 0, 0, 0, 0) + #duration(0, 0, 0, [DateUnix] ),
2, 0
)),
#"Renamed Columns" = Table.RenameColumns(#"Added Custom",{{"Date.1", "Date"}}),
#"Removed Columns" = Table.RemoveColumns(#"Renamed Columns",{"DateUnix"})
in
#"Removed Columns"

this work

Dash_Riprock_UK
Frequent Visitor

Thanks to all for their help and suggestions.  Sadly, I couldn't get the JSON links to work meaningfully.  Either it involved playing around with many steps (at which point it was difficult to feel confident about data quality) or I couldn't achieve normal dates.  

 

After much faffing I gave up with Yahoo (hopefully they fix it in the future).

 

For now, I found this website... https://eodhd.com/

 

Free option... easy to use API... 20 free requests per day.  

Thanks for this

 

One issue I have with Yahoo Finance is I scrape data for a small number of ETF investments and said data doesn't include trade volumes. This could provide an alternative data source

kar2022
Frequent Visitor

Can someone please help as I followed the above and I am unable to resolve.  I changed to "Json.Documents" and I do not know what else to change.

 

(StockQuote as text) as table =>
let
today = Duration.TotalSeconds(DateTime.LocalNow() - #datetime(1970,1,1,0,0,0)),
lastyear = Duration.TotalSeconds(Date.AddYears(DateTime.LocalNow(),-5) - #datetime(1970,1,1,0,0,0)),
Source = Json.Document(Web.Contents("https://query1.finance.yahoo.com",[RelativePath="v7/finance/download/"&StockQuote&"?period1="&Number.ToText(Number.Round(lastyear))&"&period2="&Number.ToText(Number.Round(today))&"&interval=1d&events=history&includeAdjustedClose=true"]),[Delimiter=",", Columns=7, Encoding=65001, QuoteStyle=QuoteStyle.None]),
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"Date", type date}, {"Open", type number}, {"High", type number}, {"Low", type number}, {"Close", type number}, {"Adj Close", type number}, {"Volume", Int64.Type}})
in
#"Changed Type"

G05DVD
Frequent Visitor

Hi,

 

Just to be clear I have a FREE Yahoo Finance account and have 2x queries in a Power BI report to scrape Web data.

 

Seems the endpoint has changed plus it now returns JSON as I think someone else has stated.

 

So changes to one of my original queries:

  1. Updated the URL to use the v8/finance/chart endpoint; I can confirm the new endpoint doesn't require authentication for basic historical data requests hence works with a free Yahoo Finance account
  2. Changed from Csv.Document to Json.Document as the new endpoint returns JSON
  3. Added steps to parse the JSON response and reconstruct the table

Obviously different members with differing scenarios but hopefully this will help members.

 

(StickerSymbol as text) as table => let EpochReferenceDate = #datetime(1970,1,1,0,0,0), DefaultStartDate = DateTime.From(Date.AddYears(Date.From(DateTime.LocalNow()),-5)), DefaultEndDate = DateTime.From(Date.From(DateTime.LocalNow())), StartDate = if Date.DayOfWeek(DefaultStartDate, Day.Monday) = 5 then Date.AddDays(DefaultStartDate,-1) else /* Saturday */ if Date.DayOfWeek(DefaultStartDate, Day.Monday) = 6 then Date.AddDays(DefaultStartDate,-2) else /* Sunday */ if Date.DayOfWeek(DefaultStartDate, Day.Monday) = 0 then Date.AddDays(DefaultStartDate,-3) else DefaultStartDate, /* Monday */ EndDate = if Date.DayOfWeek(DefaultEndDate, Day.Monday) = 5 then Date.AddDays(DefaultEndDate,-1) else /* Saturday */ if Date.DayOfWeek(DefaultEndDate, Day.Monday) = 6 then Date.AddDays(DefaultEndDate,-2) else /* Sunday */ if Date.DayOfWeek(DefaultEndDate, Day.Monday) = 0 then Date.AddDays(DefaultEndDate,-3) else DefaultEndDate, /* Monday*/ StartOfPeriod = Text.Start(Number.ToText(Duration.TotalSeconds(StartDate - EpochReferenceDate)),10), EndOfPeriod = Text.Start(Number.ToText(Duration.TotalSeconds(EndDate - EpochReferenceDate)),10), Source = Json.Document(Web.Contents("https://query1.finance.yahoo.com/v8/finance/chart/" & StickerSymbol & "?period1=" & StartOfPeriod & "&period2=" & EndOfPeriod & "&interval=1d&events=history&includeAdjustedClose=true")), Result = Source[chart][result]{0}, Timestamps = Result[timestamp], QuoteData = Result[indicators][quote]{0}, AdjCloseData = Result[indicators][adjclose]{0}, CombinedData = Table.FromColumns({ List.Transform(Timestamps, each Date.From(DateTimeZone.FromSecondsSinceEpoch(_))), QuoteData[open], QuoteData[high], QuoteData[low], QuoteData[close], AdjCloseData[adjclose], QuoteData[volume] }, {"Date", "Open", "High", "Low", "Close", "Adj Close", "Volume"}), #"Changed Type" = Table.TransformColumnTypes(CombinedData,{ {"Date", type date}, {"Open", type number}, {"High", type number}, {"Low", type number}, {"Close", type number}, {"Adj Close", type number}, {"Volume", Int64.Type} }) in #"Changed Type"

Hi @G05DVD ,
May I ask for your help?
I tried to follow your way of doing that, but unfortunatelly not fully successful.

When I run it for any sticker, in this example ISAC.L there is following result

Tomek1982_0-1726122379802.png

when checking the error:

Tomek1982_1-1726122428599.png

How can I deal with that?

I found the problem. For me it was necessary to replace function:
Date.From(DateTimeZone.FromSecondsSinceEpoch([_]))
with
#datetime(1970, 1, 1, 0, 0, 0) + #duration(0,0,0,_)

This works. Thanks. The JSON chart is a text document, so use whatever you wish to fit the historical data into your existing scripts. I used awk to load and transpose the chart to the old CSV format. The only reason is that I've been using awk since the 1970s and it is second nature. Anyone younger won't do that. I automatically load hundreds of tables daily, Unix bash scripts.

Nice.... I slightly predate the PC as we know it starting out on Sun Micrososytems SunOS & Silicon Graphics but AWK is new to me!

I am facing exactly the same issue. Mine loads a csv file into a blob storage container to be used in a data warehouse pipeline. I wonder how to get this going as I am not sure of how to change the json to csv before it hits the blob storage. Any ideas?

G05DVD
Frequent Visitor

Hi,

 

I have the same problem with a long standing Power BI report that scrapes stock data.

 

Does appear to be  change in authentication on the Yahoo Finance end. Member has already commented would be good to hear from someone who has a paid subscription which I do not.

 

Many thanks...

I'm having same issue with queries to get stocks data. Have someone found a solution to avoid editing the queries that were working for months?

Jase68
Helper I
Helper I

I am getting the same problem since 7th September 2024.

nickboddington
New Member

Hi

 

I had exactly the same problem as you describe. I have found the solution by using the STOCKHISTORY function in Excel:

 

https://support.microsoft.com/en-gb/office/stockhistory-function-1ac8b5b3-5f62-4d94-8ab8-7504ec7239a...

 

HTH

 

Nick

Thanks for your reply, Nick!

Unfortunately this only works with Office 365. I have a standalone version of EXCEL 2021 that doesn't support this function.

NutFlush
Frequent Visitor

I have the exact same problem. I believe it started on Wednesday or Thursday this week (Sep. 04 or 05 2024).

Please post here, if somebody finds a way to resolve this, because I need the data download on a daily basis! Maybe somebody with a paid YAHOO subscription can let us know if only the free version has this problem?

gecko407
New Member

Same thing happening to me.  Looks like they either intentionally or inadvertantly put this behind a login today.

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!

Jan25PBI_Carousel

Power BI Monthly Update - January 2025

Check out the January 2025 Power BI update to learn about new features in Reporting, Modeling, and Data Connectivity.

December 2024

A Year in Review - December 2024

Find out what content was popular in the Fabric community during 2024.