Binary.Decompress possible error?

Question

Binary.Length(Web.Contents("https://epss.cyentia.com/epss_scores-current.csv.gz"))

Returns 1391340

Binary.Length(Binary.Decompress(Web.Contents("https://epss.cyentia.com/epss_scores-current.csv.gz"), Compression.GZip))

Returns 63

Why? Please help me.

jennratten · Answer

Sometimes the data will be chunked when compressed or encoded so you may be experiencing data loss if it's not handled accordingly.&nbsp; Try querying the headers to get more info like below. The content length should be included in the response.
let
 searchText = "Power Query"
in
 Web.Headers(
 "https://www.bing.com",
 [
 RelativePath = "search",
 Query = [q = searchText]
 ]
 )

&nbsp;

elitlogik · Answer

Hi, thank you for your response.&nbsp;The Gzipped CSV from cyentia.com is what I want to fetch and analyze in Power BI.Downloading it manually, Unzip with 7zip and open in Excel works fine.Using the code in the original post results in a table with only one row.&nbsp;Isn't it strange that a Compressed file is 1.4 MB compressed and 63 characters Decompressed?

jennratten · Answer

That particular compression method is very old and is limited in bits.&nbsp;&nbsp;

elitlogik · Answer

I understand. You mean that the Binary.Decompress with the Compression.GZip parameter is old code? Because GZip as a standard is still vewry widely used on the web.&nbsp;Strangely enough it decompresses another 8 MB file from another source just fine.

jkba · Answer

I found your post experiencing exactly the same error. It seems to be a problem with the compression format: If I use this code;&nbsp;let
    web_path = "https://epss.cyentia.com/epss_scores-current.csv.gz",
    file = Binary.Decompress(Web.Contents(web_path), Compression.GZip),
    src=Lines.FromBinary(file)
...&nbsp;&nbsp;...then I only get the topmost line from the file;#model_version:v2023.03.01,score_date:2024-08-09T00:00:00+0000&nbsp;But if I decompress the gzip file from cyentia.com and re-compress it in 7-zip using the default parameters for gzip and then use this code:&nbsp;let
    compressed_file_path = "C:	emp\epss_scores-2024-08-09.csv.gz",
    file = Binary.Decompress(File.Contents(compressed_file_path), Compression.GZip),
    src=Lines.FromBinary(file),
...&nbsp;&nbsp;...then I get the entire file:#model_version:v2023.03.01,score_date:2024-08-09T00:00:00+0000cve,epss,percentileCVE-1999-0001,0.00383,0.73334CVE-1999-0002,0.02080,0.89267CVE-1999-0003,0.04409,0.92508...&nbsp;I have therefore filed a service request with support.first.org asking...&nbsp;...if any special compression settings have been used (and if so, it is possible just to use the default parameters so that Power BI’s Binary.Decompress can decompress it)?If the compression format cannot be changed into the default one, if is then possible to know which compression parameters that have been used so that I can file a feature request/bug report to Microsoft regarding the Binary.Decompress function?Did you find a workaround yourself?&nbsp;BR,&nbsp;--&nbsp;Jakob&nbsp;&nbsp;&nbsp;

Forum Discussion

Binary.Decompress possible error?

5 Replies

Related content

What is Json.Document(Binary.Decompress ?

Possible Localization issue

error message

Table.FromRows(Json.Document(Binary.Decompress(Binary) - how to find this source?

data integrity possibility?