Forum Discussion
Python script: Error tokenizing data
Hard to say without more info on the error.
I would try, Highlight all columns, go to Transform > Format > Clean. Then doTransform > Format > Trim.
If this does not work still, try renaming each of the column headers, maybe just test with A / B / C etc to see if that works.
If still erroring, it may be useful to see the M code, if you copy the code from Home > Advanced Editor.
Love hearing about Power BI tips, jobs and news?
I love to share about these - connect with me!Stay up to date on
Read my blogs onRemember to spread knowledge in the community when you can!
The code fail with something as simple as
df = dataset
The error that Power BI return is
DataSource.Error: ADO.NET: Python script error...
pandas.errors.ParserError: Error tokenizing data. C error: Expected 5 fields in line 3, saw 6
I created an environment in anaconda with Python 3.5 since I was having troubles using python 3.6 or 3.7
About the data input, I transformed and combined several Excel documents (using power query) that have the following structure:
Hope this gives a bit more information
If you can post a screenshot of the table before the python step, that could help.
Python actually uses pandas.read_csv() to import the data into "dataset", although you cannot see this directly:
What I would guess is happening, is that you have commas in your field. As pandas is using read_csv it is detecting this as a delimiter, and incorrectly splitting your column.
dbarrera wrote:About the data input, I transformed and combined several Excel documents (using power query) that have the following structure:
You could try adding a step of Find and Replace, and replace all commas with something else (such as ; ). Do this as a first step to your files in Power Query, before you comine them.
Thanks for your reply. I did as you said
SteveCampbell wrote:You could try adding a step of Find and Replace, and replace all commas with something else (such as ; ). Do this as a first step to your files in Power Query, before you comine them.
I tried changing (,) for (;) and also just for blank, but is not working as well.
Following, a screenshot of the table before the python step: