Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
VKravc1
Frequent Visitor

Script running looping URL from Microsoft Dynamics 365 API

Hi eveyone!

Details of issue: i need to get some data from the CRM based on MS Dynamics 365 through the API (to Power BI or Power Query). The database has millions of rows and I don’t need all of them (anyway these is lack of memory on the PC, when I’m trying to get all the data). There are several problems: 

1)  A single way to find only rows which I need – IDs of transactions from another table. There are about 500 unique IDs and every ID has from couple of hundred to 25 thousand rows (i know, it's bad approach, but still don't find more sofisticated columns to filter it). 

2) For all request I need to use paging (through @odata.nextLink) because most of the data in IDs is over 5000 rows (API limitation).

3) And the main issue – creating of script running looping URL, which iterate over the all data and returns only the ones I need. I'm tried to find solution with this link  and this video. So finally, it works when i make checking of invoked function, but it still doesn't work as a final solution and returns error. 

 

How it looks:

- firstly, i created function (Query3cb), which includes pagination:

 

= (param2 as text) as table =>
let
initReq = Json.Document(Web.Contents("https://site.api.crm4.dynamics.com/",
[RelativePath="api/data/v9.2/some_payments"&(param2)]
)),
nextUrl = initReq[#"@odata.nextLink"],
initValue= initReq[value],
gather=(data as list, url)=>
let
newReq=Json.Document(Web.Contents(url)),
newNextUrl = newReq[#"@odata.nextLink"],
newData= newReq[value],
data=List.Combine({data,newData}),
Converttotable = Record.ToTable(newReq),
Pivot_Columns = Table.Pivot(Converttotable, List.Distinct(Converttotable[Name]), "Name", "Value", List.Count),
Column_Names=Table.ColumnNames(Pivot_Columns),
Contains_Column=List.Contains(Column_Names,"@odata.nextLink"),
check = if Contains_Column = true then @gather(data, newNextUrl) else data
in
check,
Converttotable = Record.ToTable(initReq),
Pivot_Columns = Table.Pivot(Converttotable, List.Distinct(Converttotable[Name]), "Name", "Value", List.Count),
Column_Names=Table.ColumnNames(Pivot_Columns),
Constain_Column=List.Contains(Column_Names,"@odata.nextLink"),
outputList= if Constain_Column= true then @gather(initValue,nextUrl) else initValue,
expand=Table.FromRecords(outputList)

in
expand

 

 - then i have the list of unique IDs and trying to use invoke custom function:

VKravc1_0-1675090447068.png

- and as result it returns error: 

VKravc1_1-1675090564961.png

(i can send all error code if it could help)

 

- BUT, when i'm trying to apply this function directrly:

VKravc1_2-1675090757642.png

everything works fine (invoked function):

VKravc1_3-1675090949448.png

 

So i have no idea, what's wrong. May be error not from my side but from server? 

Thanks in advance for any help and tips!

 

0 REPLIES 0

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.