Check your eligibility for this 50% exam voucher offer and join us for free live learning sessions to get prepared for Exam DP-700.
Get StartedDon't miss out! 2025 Microsoft Fabric Community Conference, March 31 - April 2, Las Vegas, Nevada. Use code MSCUST for a $150 discount. Prices go up February 11th. Register now.
Hello,
I have a spreadsheet which downloads daily currency rates from Yahoo Finance using PowerQuery, for example...
= Csv.Document(Web.Contents("https://query1.finance.yahoo.com/v7/finance/download/USDGBP=X?period1=1609459200&period2=1924905600&..."),[Delimiter=",", Columns=7, Encoding=65001, QuoteStyle=QuoteStyle.None])
This has been working fine for months, and then today, when I go to refresh, I'm met with a message saying, " We couldn't authenticate with the credentials provided. Please try again".
This information is readily available and doesn't require an account. So why is this happening now? Please can anyone help this has broken many months of work (and I've not changed a thing since setting it up and getting it working).
here is the M language translation of the same M language code that before was working with the connection https://query1.finance.yahoo.com/v7/finance/download
this maintanes the exact same format. ChatGPT did it for me in a single passage after I prepared a well formatted request, simply unbelivable. This code deals with a day only, the current, since the historical data I load into sql server and combine with the live ones via DAX, but it can easily be adapted to process a period, chatgpt will do for you in no time. Only relevant parameter is Symbol, Cur and CountrySE are currency and stock exchange country but probably not relevant in your code. This is a function that cycles multiple symbols one by one, I load around 20k symbols and it does not work in a multiple-symbol string. Hope this helps.
let
Source = (Symbol as text, Cur as text, CountrySE as text) => let
// Calculate the date differences to generate period1 and period2
DateDiff = Duration.Days(Date.From(DateTime.LocalNow()) - #date(1970, 1, 1)),
DateDiffSecL = DateDiff * 24 * 60 * 60,
DateDiffSecU = (DateDiff + 1) * 24 * 60 * 60,
DateDiff2SecLT = Number.ToText(DateDiffSecL),
DateDiff2SecUT = Number.ToText(DateDiffSecU),
// Get JSON data from Yahoo Finance API
Source = Web.Contents(
"https://query2.finance.yahoo.com/v8/finance/chart/" & Symbol,
[
Query = [
period1 = DateDiff2SecLT,
period2 = DateDiff2SecUT,
interval = "1d"
],
ManualStatusHandling = {404}
]
),
// Parse JSON response
JsonResponse = Json.Document(Source),
// Check for errors in the JSON response
Chart = JsonResponse[chart],
Result = Chart[result]{0},
Meta = Result[meta],
Indicators = Result[indicators],
Quote = Indicators[quote]{0},
Timestamps = Result[timestamp],
AdjClose = Indicators[adjclose]{0}[adjclose],
// Convert timestamps (in seconds) to dates
Dates = List.Transform(Timestamps, each DateTime.From(#datetime(1970, 1, 1, 0, 0, 0)) + #duration(0, 0, 0, _)),
// Combine the data into a table
DataTable = Table.FromColumns({
Dates,
Quote[open],
Quote[high],
Quote[low],
Quote[close],
AdjClose,
Quote[volume]
}, type table [Date=datetime, Open=number, High=number, Low=number, Close=number, #"Adj Close"=number, Volume=Int64.Type]),
// Change data types (if necessary, although columns should already be typed correctly)
#"Changed Type" = Table.TransformColumnTypes(DataTable, {
{"Date", type date},
{"Open", type number},
{"High", type number},
{"Low", type number},
{"Close", type number},
{"Adj Close", type number},
{"Volume", Int64.Type}
}),
// Add Symbol column
#"Added Symbol Column" = Table.AddColumn(#"Changed Type", "Symbol", each Symbol),
// Filter out any erroneous data (e.g., dates or close prices that are null or empty)
#"Filtered Rows" = Table.SelectRows(#"Added Symbol Column", each [Date] <> #date(1900, 1, 1) and [Close] <> null and [Close] <> "")
in
#"Filtered Rows"
in
Source
Update: I never managed to resolve this issue but I am not very technically minded so some of the suggestions may have worked but were beyond me. To resolve this I switched my data source from the Yahoo web source to a Google Sheets file which used the GOOGLEFINANCE function to pull the shareprice, making sure the file was "public" and then connected my Power BI to that source.
Hi Jase68,
The Google Finance will not work with me the way data is collected. Thanks
(Symbol as text,min_date as text, max_date as text) as table =>
let
Source = Json.Document(Web.Contents("https://query2.finance.yahoo.com", [RelativePath = "/v8/finance/chart/" & Symbol & "?period1=" & min_date & "&period2=" & max_date & "&interval=1d&events=history&includeAdjustedClose=true"])),
// Extract the relevant parts of the JSON response
ChartData = Source[chart][result]{0},
Timestamps = ChartData[timestamp],
Quotes = ChartData[indicators][quote]{0},
// Convert the JSON into a table
CombinedData = Table.FromColumns({
List.Transform(Timestamps, each Text.From(_)),
Quotes[open],
Quotes[high],
Quotes[low],
Quotes[close],
Quotes[volume]
}, {"DateUnix", "Open", "High", "Low", "Close", "Volume"}),
#"Changed Type" = Table.TransformColumnTypes(CombinedData,{{"DateUnix", Int64.Type}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Date.1", each DateTimeZone.SwitchZone(
#datetimezone(1970, 1, 1, 0, 0, 0, 0, 0) + #duration(0, 0, 0, [DateUnix] ),
2, 0
)),
#"Renamed Columns" = Table.RenameColumns(#"Added Custom",{{"Date.1", "Date"}}),
#"Removed Columns" = Table.RemoveColumns(#"Renamed Columns",{"DateUnix"})
in
#"Removed Columns"
this work
Thanks to all for their help and suggestions. Sadly, I couldn't get the JSON links to work meaningfully. Either it involved playing around with many steps (at which point it was difficult to feel confident about data quality) or I couldn't achieve normal dates.
After much faffing I gave up with Yahoo (hopefully they fix it in the future).
For now, I found this website... https://eodhd.com/
Free option... easy to use API... 20 free requests per day.
Thanks for this
One issue I have with Yahoo Finance is I scrape data for a small number of ETF investments and said data doesn't include trade volumes. This could provide an alternative data source
Can someone please help as I followed the above and I am unable to resolve. I changed to "Json.Documents" and I do not know what else to change.
(StockQuote as text) as table =>
let
today = Duration.TotalSeconds(DateTime.LocalNow() - #datetime(1970,1,1,0,0,0)),
lastyear = Duration.TotalSeconds(Date.AddYears(DateTime.LocalNow(),-5) - #datetime(1970,1,1,0,0,0)),
Source = Json.Document(Web.Contents("https://query1.finance.yahoo.com",[RelativePath="v7/finance/download/"&StockQuote&"?period1="&Number.ToText(Number.Round(lastyear))&"&period2="&Number.ToText(Number.Round(today))&"&interval=1d&events=history&includeAdjustedClose=true"]),[Delimiter=",", Columns=7, Encoding=65001, QuoteStyle=QuoteStyle.None]),
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"Date", type date}, {"Open", type number}, {"High", type number}, {"Low", type number}, {"Close", type number}, {"Adj Close", type number}, {"Volume", Int64.Type}})
in
#"Changed Type"
Hi,
Just to be clear I have a FREE Yahoo Finance account and have 2x queries in a Power BI report to scrape Web data.
Seems the endpoint has changed plus it now returns JSON as I think someone else has stated.
So changes to one of my original queries:
Obviously different members with differing scenarios but hopefully this will help members.
(StickerSymbol as text) as table => let EpochReferenceDate = #datetime(1970,1,1,0,0,0), DefaultStartDate = DateTime.From(Date.AddYears(Date.From(DateTime.LocalNow()),-5)), DefaultEndDate = DateTime.From(Date.From(DateTime.LocalNow())), StartDate = if Date.DayOfWeek(DefaultStartDate, Day.Monday) = 5 then Date.AddDays(DefaultStartDate,-1) else /* Saturday */ if Date.DayOfWeek(DefaultStartDate, Day.Monday) = 6 then Date.AddDays(DefaultStartDate,-2) else /* Sunday */ if Date.DayOfWeek(DefaultStartDate, Day.Monday) = 0 then Date.AddDays(DefaultStartDate,-3) else DefaultStartDate, /* Monday */ EndDate = if Date.DayOfWeek(DefaultEndDate, Day.Monday) = 5 then Date.AddDays(DefaultEndDate,-1) else /* Saturday */ if Date.DayOfWeek(DefaultEndDate, Day.Monday) = 6 then Date.AddDays(DefaultEndDate,-2) else /* Sunday */ if Date.DayOfWeek(DefaultEndDate, Day.Monday) = 0 then Date.AddDays(DefaultEndDate,-3) else DefaultEndDate, /* Monday*/ StartOfPeriod = Text.Start(Number.ToText(Duration.TotalSeconds(StartDate - EpochReferenceDate)),10), EndOfPeriod = Text.Start(Number.ToText(Duration.TotalSeconds(EndDate - EpochReferenceDate)),10), Source = Json.Document(Web.Contents("https://query1.finance.yahoo.com/v8/finance/chart/" & StickerSymbol & "?period1=" & StartOfPeriod & "&period2=" & EndOfPeriod & "&interval=1d&events=history&includeAdjustedClose=true")), Result = Source[chart][result]{0}, Timestamps = Result[timestamp], QuoteData = Result[indicators][quote]{0}, AdjCloseData = Result[indicators][adjclose]{0}, CombinedData = Table.FromColumns({ List.Transform(Timestamps, each Date.From(DateTimeZone.FromSecondsSinceEpoch(_))), QuoteData[open], QuoteData[high], QuoteData[low], QuoteData[close], AdjCloseData[adjclose], QuoteData[volume] }, {"Date", "Open", "High", "Low", "Close", "Adj Close", "Volume"}), #"Changed Type" = Table.TransformColumnTypes(CombinedData,{ {"Date", type date}, {"Open", type number}, {"High", type number}, {"Low", type number}, {"Close", type number}, {"Adj Close", type number}, {"Volume", Int64.Type} }) in #"Changed Type"
Hi @G05DVD ,
May I ask for your help?
I tried to follow your way of doing that, but unfortunatelly not fully successful.
When I run it for any sticker, in this example ISAC.L there is following result
when checking the error:
How can I deal with that?
I found the problem. For me it was necessary to replace function:
Date.From(DateTimeZone.FromSecondsSinceEpoch([_]))
with
#datetime(1970, 1, 1, 0, 0, 0) + #duration(0,0,0,_)
This works. Thanks. The JSON chart is a text document, so use whatever you wish to fit the historical data into your existing scripts. I used awk to load and transpose the chart to the old CSV format. The only reason is that I've been using awk since the 1970s and it is second nature. Anyone younger won't do that. I automatically load hundreds of tables daily, Unix bash scripts.
Nice.... I slightly predate the PC as we know it starting out on Sun Micrososytems SunOS & Silicon Graphics but AWK is new to me!
I am facing exactly the same issue. Mine loads a csv file into a blob storage container to be used in a data warehouse pipeline. I wonder how to get this going as I am not sure of how to change the json to csv before it hits the blob storage. Any ideas?
Hi,
I have the same problem with a long standing Power BI report that scrapes stock data.
Does appear to be change in authentication on the Yahoo Finance end. Member has already commented would be good to hear from someone who has a paid subscription which I do not.
Many thanks...
I'm having same issue with queries to get stocks data. Have someone found a solution to avoid editing the queries that were working for months?
I am getting the same problem since 7th September 2024.
Hi
I had exactly the same problem as you describe. I have found the solution by using the STOCKHISTORY function in Excel:
HTH
Nick
Thanks for your reply, Nick!
Unfortunately this only works with Office 365. I have a standalone version of EXCEL 2021 that doesn't support this function.
I have the exact same problem. I believe it started on Wednesday or Thursday this week (Sep. 04 or 05 2024).
Please post here, if somebody finds a way to resolve this, because I need the data download on a daily basis! Maybe somebody with a paid YAHOO subscription can let us know if only the free version has this problem?
Same thing happening to me. Looks like they either intentionally or inadvertantly put this behind a login today.
March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!
Check out the January 2025 Power BI update to learn about new features in Reporting, Modeling, and Data Connectivity.
User | Count |
---|---|
23 | |
20 | |
12 | |
10 | |
10 |