topic Re: MIME type for excel file when row count is greater than ~50K in Developer

MIME type for excel file when row count is greater than ~50K

ibmbaranski — Wed, 10 Aug 2022 13:56:31 GMT

I have a daily file I receive from a vendor, and 6 days a week it works fine in the (closed source) tool I'm sending it to. These days the row count is < 10000.

One day a week I get the same file with over 60K rows and my tool will not load it. It gives me an error that says:

MIME type mismatch for file: x.xlsx. Expected type:application/vnd.openxmlformats-officedocument.spreadsheetml.sheet, Actual type: application/x-tika-ooxml

Does anyone know why I would be getting this error? I'm assuming that it has to do with the number of rows, but I'm not positive. Everything else (I am assured by the vendor) is the same when generating the file - it's the same job. I am copying the file using the same method (downloading and using scp) and loading the file into the tool the same way.

I've used `file --mime-type` and the python magic libs and the files look to be the same from those tools. The utility that's giving me the error is closed source and I don't know how they check the MIME type.

Does anyone have any ideas? I'm stuck.

Re: MIME type for excel file when row count is greater than ~50K

lbendlin — Sun, 14 Aug 2022 03:30:28 GMT

wait, are you reading these Excel files with Python? What made you do that? Why not use the native Excel connector in Power Query ?

Re: MIME type for excel file when row count is greater than ~50K

ibmbaranski — Mon, 15 Aug 2022 02:33:43 GMT

I'm not reading the files with Python. They are being delivered to me from PowerBI and I'm passing them on to the next step (via SCP) with Python.

The MIME type is what I'm trying to get to the bottom of.

Re: MIME type for excel file when row count is greater than ~50K

lbendlin — Mon, 15 Aug 2022 10:38:45 GMT

They sent you OpenOffice files, not Excel files.

Re: MIME type for excel file when row count is greater than ~50K

ibmbaranski — Mon, 15 Aug 2022 13:25:01 GMT

So now we are back to the original question, which is:

They are adamant that the job is the exact same every day, and that the large file is generated by the same job that generates the smaller files.

What would cause the MIME type to be different, the size is the obvious answer...

Re: MIME type for excel file when row count is greater than ~50K

lbendlin — Mon, 15 Aug 2022 13:31:42 GMT

Are they at least admitting that they are producing OpenOffice files? Can you ask them to send CSV instead?

Re: MIME type for excel file when row count is greater than ~50K

ibmbaranski — Mon, 15 Aug 2022 13:50:18 GMT

I might ask for CSV - I need to see if the other team can ingest that properly.

The answer has been "We generate all the files the same way so it is not on our end"