Datalake shortcut is not showing the same data (i.e. row count) as the source datawarehouse table

jon_clemens — Tue, 19 Sep 2023 17:50:11 GMT

This might not be the right forum. Sorry about that.

I have a shortcut in a lakehouse pointing to a table in a warehouse. The record count I get when querying the lakehouse table is the same as when the shortcut was created. The warehouse table has had additional rows appended since the shortcut was created.

When I drill into the default data set >> Settings>>Refresh history, I see the error below. When I look at the files that underlie the shortcut in the lakehouse, I see additional parquet files that have created times that correspond with the refresh times.

An error has occurred while checking the latest version of the direct lake table Transfer, error System.InvalidOperationException: The requested operation requires an element of type 'Object', but the target element has type 'Null'. at System.Text.Json.JsonElement.EnumerateObject() at Azure.Core.Pipeline.StorageRequestFailedDetailsParser.TryParse(Response response, ResponseError& error, IDictionary`2& data) at Azure.RequestFailedException.GetRequestFailedExceptionContent(Response response, RequestFailedDetailsParser parser) at Azure.RequestFailedException..ctor(Response response, Exception innerException, RequestFailedDetailsParser detailsParser) at Azure.Storage.Files.DataLake.FileSystemRestClient.<ListPathsAsync>d__21.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Azure.Storage.Files.DataLake.DataLakeFileSystemClient.<GetPathsInternal>d__65.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Azure.Storage.Files.DataLake.Models.GetPathsAsyncCollection.<GetNextPageAsync>d__6.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Threading.Tasks.ValueTask`1.get_Result() at Azure.Storage.StorageCollectionEnumerator`1.StorageAsyncPageable.<GetAsyncEnumerator>d__5.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1.GetResult(Int16 token) at Microsoft.ASWL.Service.Engine.SeethruAutoSync.DirectLakeTable.<FetchTableVersionAsync>d__30.MoveNext() in C:\__w\1\s\ASWL.Service\Engine\SeethruAutoSync\DirectLakeTable.cs:line 107 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.ASWL.Service.Engine.SeethruAutoSync.DirectLakeTable.<FetchTableVersionAsync>d__30.MoveNext() in C:\__w\1\s\ASWL.Service\Engine\SeethruAutoSync\DirectLakeTable.cs:line 107.

Re: Datalake shortcut is not showing the same data (i.e. row count) as the source datawarehouse table

GraceGu — Wed, 20 Sep 2023 08:08:18 GMT

Can you move this shortcut topic to General Discussion - Microsoft Fabric Community to get help? Thank you!

Re: Datalake shortcut is not showing the same data (i.e. row count) as the source datawarehouse table

jon_clemens — Wed, 20 Sep 2023 17:24:06 GMT

I reposted to the suggested forum. Thx.

topic Re: Datalake shortcut is not showing the same data (i.e. row count) as the source datawarehouse table in Pipelines

Datalake shortcut is not showing the same data (i.e. row count) as the source datawarehouse table

Re: Datalake shortcut is not showing the same data (i.e. row count) as the source datawarehouse table

Re: Datalake shortcut is not showing the same data (i.e. row count) as the source datawarehouse table