Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Try your skills in the Power BI Dataviz World Championship! Round one ends June 26. Join now

Reply
VKravc1
Frequent Visitor

Script running looping URL from Microsoft Dynamics 365 API

Hi eveyone!

Details of issue: i need to get some data from the CRM based on MS Dynamics 365 through the API (to Power BI or Power Query). The database has millions of rows and I don’t need all of them (anyway these is lack of memory on the PC, when I’m trying to get all the data). There are several problems: 

1)  A single way to find only rows which I need – IDs of transactions from another table. There are about 500 unique IDs and every ID has from couple of hundred to 25 thousand rows (i know, it's bad approach, but still don't find more sofisticated columns to filter it). 

2) For all request I need to use paging (through @odata.nextLink) because most of the data in IDs is over 5000 rows (API limitation).

3) And the main issue – creating of script running looping URL, which iterate over the all data and returns only the ones I need. I'm tried to find solution with this link  and this video. So finally, it works when i make checking of invoked function, but it still doesn't work as a final solution and returns error. 

 

How it looks:

- firstly, i created function (Query3cb), which includes pagination:

 

= (param2 as text) as table =>
let
initReq = Json.Document(Web.Contents("https://site.api.crm4.dynamics.com/",
[RelativePath="api/data/v9.2/some_payments"&(param2)]
)),
nextUrl = initReq[#"@odata.nextLink"],
initValue= initReq[value],
gather=(data as list, url)=>
let
newReq=Json.Document(Web.Contents(url)),
newNextUrl = newReq[#"@odata.nextLink"],
newData= newReq[value],
data=List.Combine({data,newData}),
Converttotable = Record.ToTable(newReq),
Pivot_Columns = Table.Pivot(Converttotable, List.Distinct(Converttotable[Name]), "Name", "Value", List.Count),
Column_Names=Table.ColumnNames(Pivot_Columns),
Contains_Column=List.Contains(Column_Names,"@odata.nextLink"),
check = if Contains_Column = true then @gather(data, newNextUrl) else data
in
check,
Converttotable = Record.ToTable(initReq),
Pivot_Columns = Table.Pivot(Converttotable, List.Distinct(Converttotable[Name]), "Name", "Value", List.Count),
Column_Names=Table.ColumnNames(Pivot_Columns),
Constain_Column=List.Contains(Column_Names,"@odata.nextLink"),
outputList= if Constain_Column= true then @gather(initValue,nextUrl) else initValue,
expand=Table.FromRecords(outputList)

in
expand

 

 - then i have the list of unique IDs and trying to use invoke custom function:

VKravc1_0-1675090447068.png

- and as result it returns error: 

VKravc1_1-1675090564961.png

(i can send all error code if it could help)

 

- BUT, when i'm trying to apply this function directrly:

VKravc1_2-1675090757642.png

everything works fine (invoked function):

VKravc1_3-1675090949448.png

 

So i have no idea, what's wrong. May be error not from my side but from server? 

Thanks in advance for any help and tips!

 

0 REPLIES 0

Helpful resources

Announcements
Fabric Data Days is here Carousel

Fabric Data Days 2026

Don't miss out on Data Days, June 15 through August 7. Learn Fabric, Power BI, SQL, AI and more.

May Power BI Update Carousel

Power BI Monthly Update - May 2026

Check out the May 2026 Power BI update to learn about new features.

Power BI DataViz World Championships carousel

Power BI DataViz World Championships - June 2026

A new Power BI DataViz World Championship is coming this June! Don't miss out on submitting your entry.