"Unexpected error" when reading sharepoint folder full of large text files using Dataflows

[Accokeekian]

Hi there,

Excited for the new features of dataflows from the Nov 2019 update which allows the same "combine" files experience as in power query on desktop. Unfourtunately, when I attempt to load a folder of delimited text files (.log), I recieve the generic error of "Unexpected Error" and a session ID when I attempt to execute the "combine files" command. 

I get this same error when accessing the folder using the sharepoint online connector as well as a local folder connection using my gateway.

Has anyone else encountered this issue?

[fabo]

Hi everyone.

I'm facing the same issue here.  I'm trying to combine Excel files from a Sharepoint folder, but when I try to expand the content I get the "unexpected error" message.  Has this been solved?  Is there a workaround?

Greetings!

@fabo 

[Accokeekian]

No resolution just yet. I have a support ticket in and we're working through the issue.

Initially msft concluded that it was a file type issues but after testing that was deemed to not be the case. Your issue with Excel files contributes to that view.

I'll link your comment to my case and keep you updated.

[fabo]

Thank you for replying @Accokeekian .

My Excel files have .xlsm extension so it doesn't seem to be the file type.  I leave you the mashup text, it might be useful somehow (translated from spanish):

section Section1;
shared data_oferta_prog = let
  Source = SharePoint.Files("https://xxxxxxxxxxx.sharepoint.com/sites/xxxxxxxxxxx", [ApiVersion = 15]),
  #"Filtered rows" = Table.SelectRows(Source, each ([Extension] = ".xlsm")),
  #"Keep content" = Table.SelectColumns(#"Filtered rows", {"Content"}),
  #"Filtered hidden files" = Table.SelectRows(#"Keep content", each [Attributes]?[Hidden]? <> true)
in
  #"Filtered hidden files";
shared #"Example file" = let
  Source = SharePoint.Files("https://xxxxxxxxxxx.sharepoint.com/sites/xxxxxxxxxxx", [ApiVersion = 15]),
  #"Filtered rows" = Table.SelectRows(Source , each ([Extension] = ".xlsm")),
  #"Keep content" = Table.SelectColumns(#"Filtered rows", {"Content"}),
  #"Filtered hidden files" = Table.SelectRows(#"Keep content", each [Attributes]?[Hidden]? <> true),
  Navegation = #"Filtered hidden files"{0}[Content]
in
  Navegation;
shared Parameter = let
  Parameter = #"Example file" meta [IsParameterQuery = true, IsParameterQueryRequired = true, Type = type binary]
in
  Parameter;

Thanks!

[jmdh]

Hope this helps: I have faced the same issue many times.
Now i prefer to use SharePoint.Contents instead of SharePoint.Files which is both easier to reach the files i want ie (their folder) and seems to work well.

This unexpected error has started in may or June i believe and no one seems to have acknoledged it.

I would love an extension to the API allowing to directly access a given folder.

[fabo]

Hi @jmdh , thank you for your suggestion.

Unfortunately, it didn't workout for me.  I can summarize the steps for both methods as followed:

Using SharePoint.Files

1. Source = SharePoint.Files("https://xxxxx.sharepoint.com/sites/xxxxx", [ApiVersion = 15])

2. #"Keep Content" = Table.SelectColumns(Source ,{"Content"})

Using SharePoint.Contents

1. Source = SharePoint.Contents("https://xxxxx.sharepoint.com/sites/xxxxx", [ApiVersion = 15])

2. #"Shared Documents" = Source{[Name="Shared Documents"]}[Content]
3. Data = #"Shared Documents"{[Name="Data"]}[Content]

4. #"Keep Content" = Table.SelectColumns(Data ,{"Content"})

With both approaches I come to the same next step: combine files (expand the content), selecting the two down arrows next to the field name Content.

That's the point where I get this unexpected error, no matter which of the previous functions I use.

[jmdh]

May sound dumb but does it work on PowerBI Desktop ? 

[fabo]

Yes, it does work on PowerBI Desktop.  No problem there.

I was trying to replicate the same over dataflows.

[jmdh]

I will look further tonight : however the same code does not yield the same results in Desktop and Dataflow.
In short: Desktop creates a Content column which contains the Binary of each excel file, while Dataflow creates a Content column which contains a Table value for each file...

[v-shex-msft]

Hi @Accokeekian ,

Any advanced operations you add in your query tables? If these operations work on the desktop but not able to execute on dataflow side, it means some of the custom steps are invalid on the dataflow side.

You can try to assign these queries to computed entities to confirm if they work in high permissions. (computed entity allow more operations)

Using computed entities on Power BI Premium 

BTW, if you transform your '.log' file to '.txt', did these queries work on dataflow side? (for security reasons, some of the functions only work with common file types when you work on service side)
Regards,

Xiaoxin Sheng

[Accokeekian]

Thanks for your response. At this point in testing I am not using any complex operations built into my queries, I am just using the basic functionality of combining a series of files within a sharepoint folder using the UI.

I had read that filetypes could be an issue and have experiemted with renaming the files from .log to .txt, .csv and other more common types with no sucess.

FYI - This is all taking place in a premium workspace

[v-shex-msft]

Hi @Accokeekian ,

I can't find out the reason of your scenario from your description. Maybe you can open a support ticket to get further support from power bi team.

submit a support ticket

Regards,

Xiaoxin Sheng

[jmdh]

Hi is suspect this has to do with the fact that basically you can't use all the new commands in dataflow that require a Premium licence. 

[venal]

@Accokeekian 

Please refer the support page for known issue related to Data Flow.

If the issue is different, please submit a ticket by clicking on "Create Support Ticket".

If this post helps, then please consider Accept it as the solution to help the other members find it more

If this post was helpful may I ask you to mark it as solution and give it some kudos?