topic Re: Notebook: AttributeError: 'NoneType' object has no attribute 'alias' in Data Engineering

Notebook: AttributeError: 'NoneType' object has no attribute 'alias'

MaryJo — Fri, 27 Dec 2024 16:51:29 GMT

Question on running a notebook using pyspark.sql. I am loading multiple files into a new table that is specified in the notebook. The notebook loads the first file successfully then all other files it throws out the error such as the below. The code fails around line 79 when processing the sebsequent files. I will also paste in the notebook code at line 79 for reference. Trying to figure out how to fix the attribute error stating 'Nonetype' object has no attribute 'alias'.

Notebook line 79 code.

If anyone has any suggestions that would be great. Thank you for your time.

Re: Notebook: AttributeError: 'NoneType' object has no attribute 'alias'

Anonymous — Tue, 31 Dec 2024 05:15:26 GMT

Hi @MaryJo
Thank you for reaching out Microsoft fabric community forum.

The error AttributeError: 'NoneType' object has no attribute 'alias' suggests that the delta_table object isn’t being set up correctly when moving on to the next files.

Double-check that the Delta table exists at the specified path. If it doesn't, you'll encounter this error.
Make sure the path_to_delta_table is correct and accessible. path issues can cause problems.
Adding some print statements can help you determine if delta_table is None before you use it. This way, you can see where things might be going wrong.
delta_table = DeltaTable.forPath(spark, path_to_delta_table)
if delta_table is None:
print("Delta table not found or created successfully.")
else:
keyfield_lines = raw_df.rdd.map(lambda row: row[0]).collect()[0]
keyfields = keyfield_lines.replace("KeyFields=", "").split(",")
condition = " AND ".join([f"old_data.{field} = new_data.{field}" for field in keyfields])
delta_table.alias("old_data").merge(df.alias("new_data"), condition) \
.whenMatchedUpdateAll() \
.whenNotMatchedInsertAll() \
.execute()
Ensure the Delta table is set up correctly during the first write operation. If the initial write isn't successful, the table won't exist for subsequent operations.

By following these steps, you will be able to fix the issue causing the NoneType error.

If you need any further assistance or have any questions, please feel free to contact us.

If this solution helps, please consider giving us Kudos and accepting it as the solution so that it may assist other members in the community.
Best regards,
shamili.v

Re: Notebook: AttributeError: 'NoneType' object has no attribute 'alias'

MaryJo — Thu, 02 Jan 2025 15:22:11 GMT

Thank you very much for your detailed explanation and this resolved our issue.