Forum Discussion

Ben_Nash's avatar
Ben_Nash
Frequent Visitor
8 years ago

Automatically refresh JSON feed

Hi

 

I’m looking to automatically refresh a webhose.io JSON feed in Power BI.  

 

Webhose.io gives you an ‘endpoint’ and I’ve managed to feed this into Power BI successfully via the JSON connector, but this only gives you the first 100 articles:

 

http://webhose.io/filterWebContent?token=cea99d95-447a-4aa9-955b-05a55c1a9f32&format=json&sort=crawled&q=%22donald%20trump%22%20language%3Aenglish

 

You need to keep adding the ‘next’ code in order to get the next 100 articles:

 

"totalResults": 38544,

"moreResultsAvailable": 38444,

 "next": "/filterWebContent?token=cea99d95-447a-4aa9-955b-05a55c1a9f32&format=json&ts=1510365182000&q=%22donald+trump%22+language%3Aenglish&sort=crawled",

 

Is it possible to write a script in Power Query that allows you to automatically 'cycle' through all articles?

 

Any assistance appreciated!

 

Thanks

 

Ben

3 Replies