Forum Discussion

aandrade1125's avatar
aandrade1125
Regular Visitor
3 years ago

Error: The preview for Section1/Query1/Source doesn't exist

Hello!

 

I am using Python Scripting to set up a connection to an endpoint from an API. For each call I can retrieve 100 rows of data, but for this specific endpoint I am looking for 17,000 rows. The amount of calls I need is then 170. I have written Python code to loop each call and place into a data frame via append. I tested low amounts of calls first. I tried 5 and 8 calls to establish that the code runs how I want it to. But when I scale up to the 170 calls I need Power Bi errors out saying "Unable to Connect. We encountered an error while trying to connect. Details: "The preview for Section1/Query1/Source doesn't exist." I know that the number of calls is well within the limit that I am allowed from this API but I am still getting this error code. 

 

Below is my code. I have replaced all the sensitive inputs for capital letters matching them where they show up more than once. 

 

##############################

import re
import requests
import json
from IPython.display import display
import pandas as pd

 

results_to_pull_in = 17000


url = 'A'
payload = json.dumps({'data': {'type': 'B'}})
headers = {
'Content-Type': 'C',
'Accept': 'D',
'Authorization': 'E'
}


response = requests.request("GET", url, headers=headers, data=payload)
response = response.json()
df = pd.json_normalize(response['data'])
df.columns = df.columns.str.replace('attributes.', '')

results = pd.DataFrame()
results = results.append(df)

next_cursor = pd.json_normalize(response['links'])
next_cursor = next_cursor.iloc[0,1]

count = 1

while count < (results_to_pull_in // 100):

url = f"{next_cursor}"

payload2 = json.dumps({'data': {'type': 'B'}})
headers = {
'Content-Type': 'C',
'Accept': 'D',
'Authorization': 'E'
}

response = requests.request("GET",url,headers=headers,data=payload2)
response = response.json()
df2 = pd.json_normalize(response['data'])
df2.columns = df2.columns.str.replace('attributes.','')
results = results.append(df2)

try:
next_cursor = pd.json_normalize(response['links'])
next_cursor = next_cursor.iloc[0,1]

except:
continue

count += 1

display(results)

#################################################

 

Again, this code works with small iterations 5 or even 8 but it is giving me this error when I scale up. I understand you wont be able to run this code without the information hidden but I was wondering if anyone had any ideas or thoughts that might help. 

 

Thank you, 

 

3 Replies