Hello everyone,
I am currently making a personal SPARQL connector in Power BI. It is mostly basic stuff. I have an interface to enter an Endpoint, a SPARQL Query, and Timeout. I am supposed to display the result in a table in the power bi desktop.
The problem I am having is that the timeout doesn't trigger. If I don't put a LIMIT on the query itself, the program ultimately fails after a few minutes due to a Buffer Overflow. For large datasets, I need to timeout to trigger and the program execution to stop so that I can display whatever results I get. I have checked out previous queries regarding timeouts and I can't seem to find an answer. M script is a completely different field for me so I would really appreciate any help.
Here is my implementation so far:
DefaultResponseFormat = "json";
DefaultTimeout = 100;
DefaultRetries = 5;
DefaultDelayAPICallRetry = #duration(0,0,0,1/2);
SparqlImpl = (Endpoint as text, Query as text, optional ResponseFormat as text, optional Timeout as number) as table =>
let
//ResponseFormat = if ResponseFormat is null then DefaultResponseFormat else ResponseFormat,
ResponseFormat = DefaultResponseFormat,
Timeout = if Timeout is null then DefaultTimeout else Timeout,
Url = Endpoint,
Table =
let
response = Value.WaitFor(
(iteration) =>
let
isRetry = if iteration > 0 then true else false,
response = Web.Contents(Url,
[
Query = [
query = Query,
format = ResponseFormat
],
Timeout = #duration(0, 0, 0, Timeout),
IsRetry = isRetry,
ManualStatusHandling={429}
]
),
buffered = Binary.Buffer(response),
status = if buffered = null then 0 else Value.Metadata(response)[Response.Status],
actualResult = if status = 429 then null else buffered
in
actualResult,
(iteration) => DefaultDelayAPICallRetry,
DefaultRetries
),
// Bunch of parsing Code
in
table
in
Table;
Value.WaitFor = (producer as function, interval as function, optional count as number) as any =>
let
list = List.Generate(
() => {0, null},
(state) => state{0} <> null and (count = null or state{0} < count),
(state) => if state{1} <> null then {null, state{1}} else {1 + state{0}, Function.InvokeAfter(() => producer(state{0}), interval(state{0}))},
(state) => state{1})
in
List.Last(list);
