Forum Discussion
Duplicate Employee ID and Employee Names in data set - How can I clean this data?
- 1 year ago
Hi thanks for reaching out i suppose you wnt to clean the data and load so for that
- Load Data: Import the Employee Table into Power Query Editor.
- Remove Exact Duplicates: Select Employee ID and Employee Name columns, and remove exact duplicates.
- Group by Columns: Group by Employee ID and Employee Name to review and reconcile duplicate entries.
- Standardize Data: Clean inconsistencies (e.g., trim spaces, standardize cases) for Employee Name.
Finally load the cleaned the data into Power BI and model it and start building the report
if the suggestion helped please accept this as the solution and give a kudos
- 1 year ago
Hi AJ2025
I would sujjest that remove duplicates based on the entire table ( all columns ) ,
after that if there are duplicates in the employee id column and employee name for the same id is different then you should check from the source and respective ( data engineers )
or at last ideal pratice would be remove duplicates from the customer id column ( unique = distinct ) as it is the primary key column of the table ( and all values must be unique ) for that.
Regards
Govind Sapkade ( Data Analyst , Power BI PL 300 Certified , MS Fabric Enthusiast )
Linkdin : www.linkedin.com/in/govind-sapkade-845104225
Youtube : http://www.youtube.com/@govind_dataanalyst
Hi thanks for reaching out i suppose you wnt to clean the data and load so for that
- Load Data: Import the Employee Table into Power Query Editor.
- Remove Exact Duplicates: Select Employee ID and Employee Name columns, and remove exact duplicates.
- Group by Columns: Group by Employee ID and Employee Name to review and reconcile duplicate entries.
- Standardize Data: Clean inconsistencies (e.g., trim spaces, standardize cases) for Employee Name.
Finally load the cleaned the data into Power BI and model it and start building the report
if the suggestion helped please accept this as the solution and give a kudos
- AJ20251 year agoNew Member
HI All
Thank yu for all of your suggestions. I have not had chance to try any of these out yet. I will let you know when I do 😉