Forum Discussion
Yahoo Finance Power Query Link Not Working
Hi,
Just to be clear I have a FREE Yahoo Finance account and have 2x queries in a Power BI report to scrape Web data.
Seems the endpoint has changed plus it now returns JSON as I think someone else has stated.
So changes to one of my original queries:
- Updated the URL to use the v8/finance/chart endpoint; I can confirm the new endpoint doesn't require authentication for basic historical data requests hence works with a free Yahoo Finance account
- Changed from Csv.Document to Json.Document as the new endpoint returns JSON
- Added steps to parse the JSON response and reconstruct the table
Obviously different members with differing scenarios but hopefully this will help members.
(StickerSymbol as text) as table => let EpochReferenceDate = #datetime(1970,1,1,0,0,0), DefaultStartDate = DateTime.From(Date.AddYears(Date.From(DateTime.LocalNow()),-5)), DefaultEndDate = DateTime.From(Date.From(DateTime.LocalNow())), StartDate = if Date.DayOfWeek(DefaultStartDate, Day.Monday) = 5 then Date.AddDays(DefaultStartDate,-1) else /* Saturday */ if Date.DayOfWeek(DefaultStartDate, Day.Monday) = 6 then Date.AddDays(DefaultStartDate,-2) else /* Sunday */ if Date.DayOfWeek(DefaultStartDate, Day.Monday) = 0 then Date.AddDays(DefaultStartDate,-3) else DefaultStartDate, /* Monday */ EndDate = if Date.DayOfWeek(DefaultEndDate, Day.Monday) = 5 then Date.AddDays(DefaultEndDate,-1) else /* Saturday */ if Date.DayOfWeek(DefaultEndDate, Day.Monday) = 6 then Date.AddDays(DefaultEndDate,-2) else /* Sunday */ if Date.DayOfWeek(DefaultEndDate, Day.Monday) = 0 then Date.AddDays(DefaultEndDate,-3) else DefaultEndDate, /* Monday*/ StartOfPeriod = Text.Start(Number.ToText(Duration.TotalSeconds(StartDate - EpochReferenceDate)),10), EndOfPeriod = Text.Start(Number.ToText(Duration.TotalSeconds(EndDate - EpochReferenceDate)),10), Source = Json.Document(Web.Contents("https://query1.finance.yahoo.com/v8/finance/chart/" & StickerSymbol & "?period1=" & StartOfPeriod & "&period2=" & EndOfPeriod & "&interval=1d&events=history&includeAdjustedClose=true")), Result = Source[chart][result]{0}, Timestamps = Result[timestamp], QuoteData = Result[indicators][quote]{0}, AdjCloseData = Result[indicators][adjclose]{0}, CombinedData = Table.FromColumns({ List.Transform(Timestamps, each Date.From(DateTimeZone.FromSecondsSinceEpoch(_))), QuoteData[open], QuoteData[high], QuoteData[low], QuoteData[close], AdjCloseData[adjclose], QuoteData[volume] }, {"Date", "Open", "High", "Low", "Close", "Adj Close", "Volume"}), #"Changed Type" = Table.TransformColumnTypes(CombinedData,{ {"Date", type date}, {"Open", type number}, {"High", type number}, {"Low", type number}, {"Close", type number}, {"Adj Close", type number}, {"Volume", Int64.Type} }) in #"Changed Type"
This works. Thanks. The JSON chart is a text document, so use whatever you wish to fit the historical data into your existing scripts. I used awk to load and transpose the chart to the old CSV format. The only reason is that I've been using awk since the 1970s and it is second nature. Anyone younger won't do that. I automatically load hundreds of tables daily, Unix bash scripts.
- Paul_ClearCutIT1 year agoRegular Visitor
Nice.... I slightly predate the PC as we know it starting out on Sun Micrososytems SunOS & Silicon Graphics but AWK is new to me!
- Ston5571 year agoNew Member
I am facing exactly the same issue. Mine loads a csv file into a blob storage container to be used in a data warehouse pipeline. I wonder how to get this going as I am not sure of how to change the json to csv before it hits the blob storage. Any ideas?