Forum Discussion
Changing a data source in powerBI
I have a .csv file that was generated from a .parquet file. I want to change the source for the powerBI file from .parquet to the .csv.
The columns for both files are exactly the same names and the length of data is the same. So I went into the power query Advanced editor to edit the source file from:
let
Source = Parquet.Document(File.Contents("C:\...\OneDrive\Shared Data\production.parquet"), [Compression=null, LegacyColumnNameEncoding=false, MaxDepth=null]),
#"Removed Other Columns" = Table.SelectColumns(Source,{"Point Name", "Point ID"})
in
#"Removed Other Columns"
I change it to:
let
Source = Table.PromoteHeaders(Csv.Document(File.Contents("C:\...\OneDrive\Shared Data\production.csv"),
[Delimiter=",", Columns=35, Encoding=1252, QuoteStyle=QuoteStyle.None]), [PromoteAllScalars=true]),
#"Removed Other Columns" = Table.SelectColumns(Source,{"Point Name", "Point ID"})
in
#"Removed Other Columns"
The data loads without error but the bar chart is not the same, even though the data in the table is exactly the same. It has the appearance of less data when that is not true. How would I fix this?
So the problem is that the .parquet treats the null values differently than the .csv file. If anybody else does this samething look at how the nulls appear in each.
2 Replies
Hi Ssibbett , I am concerned with this comment: "a .csv file that was generated from a .parquet file".
If you have successfully imported both files and they have different results, it suggests that the conversion from parquet to csv is not working properly.
In this situation I would the following options:- Keep the parquet format because Power BI accepts this format
- Try to do a row by row, cell by cell reconcilation between the csv and parquet file.
With the latter, you need to compare to the nth decimal place because if the two files have the same number of rows, it is likely that value columns are being truncated during the conversion process.
Many thanks
Daryl
So the problem is that the .parquet treats the null values differently than the .csv file. If anybody else does this samething look at how the nulls appear in each.
