Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Grow your Fabric skills and prepare for the DP-600 certification exam by completing the latest Microsoft Fabric challenge.

Azure Data Explorer Set notruncation not working.

I'm working on creating a dataflow that pulls  data from our Azure Log Analytics using the Azure Data Explorer connection. I have set notrunction written in the query, and it will only pull data until it hits the 500,000 row limit. 

That said, the article says that set notruncation is supposed to allow all the records to pull, in this case it should be over 1 million rows being pulled if not more.

Copy and transform data in Azure Data Explorer - Azure Data Factory & Azure Synapse | Microsoft Lear...

Wilsonjr_0-1683209011605.png

 

Example of query for one month, Note hostname is subbed with actual name instead of ******

 

 

set notruncation;
Syslog
| where HostName == "*******" and SeverityLevel == "info" and EventTime >=ago(30d)
| extend url = extract("GET\\s.*HTTP", 0, SyslogMessage)
| extend client = extract("/Client:\\w+", 0, url)
| extend IPCountry = extract("IPCountry: \\w+", 0, SyslogMessage)
| extend isLoggedOut = extract("isLoggedOut=\\w", 0, SyslogMessage)
| extend IpAddress = extract("(([0-9]{1,3})\\.([0-9]{1,3})\\.([0-9]{1,3})\\.(([0-9]{1,3})))", 1, SyslogMessage)
| project
    url,
    client,
    IPCountry,
    isLoggedOut,
    IpAddress,
    EventTime;

 

 

Let me know if you need any more information,

Thanks,

James

 

Status: Investigating

Hi @Wilsonjr 

Maybe you can refer to the limits below . Hope it will help you !

Query limits - Azure Data Explorer | Microsoft Learn

 

Best Regards,
Community Support Team _ Ailsa Tao

Comments
v-yetao1-msft
Community Support
Status changed to: Needs Info

Hi @Wilsonjr 

By default, Azure Data Explorer source has a size limit of 500,000 records or 64 MB . So what is your question? Or what kind of results do you want to get ?

 

Best Regards,
Community Support Team _ Ailsa Tao

Wilsonjr
Frequent Visitor

@v-yetao1-msft , yes by default, but if you use the set notruncation before the query it should allow more then 500,000 rows. But when I set it, it does not bring in more then 500,000 rows. Is there some setting I am missing that is capping it at 500,000?

v-yetao1-msft
Community Support
Status changed to: Investigating

Hi @Wilsonjr 

Maybe you can refer to the limits below . Hope it will help you !

Query limits - Azure Data Explorer | Microsoft Learn

 

Best Regards,
Community Support Team _ Ailsa Tao