Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Did you hear? There's a new SQL AI Developer certification (DP-800). Start preparing now and be one of the first to get certified. Register now

Reply
VKravc1
Frequent Visitor

Script running looping URL from Microsoft Dynamics 365 API

Hi eveyone!

Details of issue: i need to get some data from the CRM based on MS Dynamics 365 through the API (to Power BI or Power Query). The database has millions of rows and I don’t need all of them (anyway these is lack of memory on the PC, when I’m trying to get all the data). There are several problems: 

1)  A single way to find only rows which I need – IDs of transactions from another table. There are about 500 unique IDs and every ID has from couple of hundred to 25 thousand rows (i know, it's bad approach, but still don't find more sofisticated columns to filter it). 

2) For all request I need to use paging (through @odata.nextLink) because most of the data in IDs is over 5000 rows (API limitation).

3) And the main issue – creating of script running looping URL, which iterate over the all data and returns only the ones I need. I'm tried to find solution with this link  and this video. So finally, it works when i make checking of invoked function, but it still doesn't work as a final solution and returns error. 

 

How it looks:

- firstly, i created function (Query3cb), which includes pagination:

 

= (param2 as text) as table =>
let
initReq = Json.Document(Web.Contents("https://site.api.crm4.dynamics.com/",
[RelativePath="api/data/v9.2/some_payments"&(param2)]
)),
nextUrl = initReq[#"@odata.nextLink"],
initValue= initReq[value],
gather=(data as list, url)=>
let
newReq=Json.Document(Web.Contents(url)),
newNextUrl = newReq[#"@odata.nextLink"],
newData= newReq[value],
data=List.Combine({data,newData}),
Converttotable = Record.ToTable(newReq),
Pivot_Columns = Table.Pivot(Converttotable, List.Distinct(Converttotable[Name]), "Name", "Value", List.Count),
Column_Names=Table.ColumnNames(Pivot_Columns),
Contains_Column=List.Contains(Column_Names,"@odata.nextLink"),
check = if Contains_Column = true then @gather(data, newNextUrl) else data
in
check,
Converttotable = Record.ToTable(initReq),
Pivot_Columns = Table.Pivot(Converttotable, List.Distinct(Converttotable[Name]), "Name", "Value", List.Count),
Column_Names=Table.ColumnNames(Pivot_Columns),
Constain_Column=List.Contains(Column_Names,"@odata.nextLink"),
outputList= if Constain_Column= true then @gather(initValue,nextUrl) else initValue,
expand=Table.FromRecords(outputList)

in
expand

 

 - then i have the list of unique IDs and trying to use invoke custom function:

VKravc1_0-1675090447068.png

- and as result it returns error: 

VKravc1_1-1675090564961.png

(i can send all error code if it could help)

 

- BUT, when i'm trying to apply this function directrly:

VKravc1_2-1675090757642.png

everything works fine (invoked function):

VKravc1_3-1675090949448.png

 

So i have no idea, what's wrong. May be error not from my side but from server? 

Thanks in advance for any help and tips!

 

0 REPLIES 0

Helpful resources

Announcements
April Power BI Update Carousel

Power BI Monthly Update - April 2026

Check out the April 2026 Power BI update to learn about new features.

Fabric SQL PBI Data Days

Data Days 2026 coming soon!

Sign up to receive a private message when registration opens and key events begin.

New to Fabric survey Carousel

New to Fabric Survey

If you have recently started exploring Fabric, we'd love to hear how it's going. Your feedback can help with product improvements.

Power BI DataViz World Championships carousel

Power BI DataViz World Championships - June 2026

A new Power BI DataViz World Championship is coming this June! Don't miss out on submitting your entry.