Azure Data Explorer Set notruncation not working.

[Wilsonjr]

I'm working on creating a dataflow that pulls  data from our Azure Log Analytics using the Azure Data Explorer connection. I have set notrunction written in the query, and it will only pull data until it hits the 500,000 row limit. 

That said, the article says that set notruncation is supposed to allow all the records to pull, in this case it should be over 1 million rows being pulled if not more.

Copy and transform data in Azure Data Explorer - Azure Data Factory & Azure Synapse | Microsoft Lear...

Example of query for one month, Note hostname is subbed with actual name instead of ******

set notruncation;

Syslog

| where HostName == "*******" and SeverityLevel == "info" and EventTime >=ago(30d)

| extend url = extract("GET\\s.*HTTP", 0, SyslogMessage)

| extend client = extract("/Client:\\w+", 0, url)

| extend IPCountry = extract("IPCountry: \\w+", 0, SyslogMessage)

| extend isLoggedOut = extract("isLoggedOut=\\w", 0, SyslogMessage)

| extend IpAddress = extract("(([0-9]{1,3})\\.([0-9]{1,3})\\.([0-9]{1,3})\\.(([0-9]{1,3})))", 1, SyslogMessage)

| project

    url,

    client,

    IPCountry,

    isLoggedOut,

    IpAddress,

    EventTime;

Let me know if you need any more information,

Thanks,

James

[Anonymous]

Hi @Wilsonjr 

By default, Azure Data Explorer source has a size limit of 500,000 records or 64 MB . So what is your question? Or what kind of results do you want to get ?

Best Regards,
Community Support Team _ Ailsa Tao

[Wilsonjr]

@Anonymous , yes by default, but if you use the set notruncation before the query it should allow more then 500,000 rows. But when I set it, it does not bring in more then 500,000 rows. Is there some setting I am missing that is capping it at 500,000?

[Anonymous]

Hi @Wilsonjr 

Maybe you can refer to the limits below . Hope it will help you !

Query limits - Azure Data Explorer | Microsoft Learn

Best Regards,
Community Support Team _ Ailsa Tao