topic Re: Notebooks writing tables with names lowercased - how to fix? in Data Science

Notebooks writing tables with names lowercased - how to fix?

Scott_Powell — Sun, 10 Mar 2024 02:08:49 GMT

Hi, I have a spark notebook that's applying changes to a table in a lakehouse. Everything works fine until I try to run this last line of code:

final_df.write.format("delta").mode("overwrite").saveAsTable(target_table)

Here's the issue - my original table is called ThisIsASample, and I've verified that the target_table variable holds "ThisIsASample". But when the above line of code runs, it recreates the table as "thisisasample". This mucks up everything downstream - I need it to write the table name respecting the case I specified.

Any ideas on why the notebook is doing this, and what I can do to correct it?

Thanks,

Scott

Re: Notebooks writing tables with names lowercased - how to fix?

StrategicSavvy — Sun, 10 Mar 2024 22:05:19 GMT

hi Scott_Powell

As for now spark enforce lowercase on write and there is no way to change it.

Re: Notebooks writing tables with names lowercased - how to fix?

Anonymous — Mon, 11 Mar 2024 04:19:43 GMT

Hi Scott_Powell ,

Thanks for using Fabric Community.

Yes, this is a limitation of the HIVE metastore. It stores the schema of a table in all lowercase.

A table name can contain only lowercase alphanumeric characters and underscores and must start with a lowercase letter or underscore.

Hope this will help. Please let us know if any further queries.

Re: Notebooks writing tables with names lowercased - how to fix?

Scott_Powell — Mon, 11 Mar 2024 15:07:23 GMT

The very odd thing though is that I have other lakehouses where this is working fine. See image below. It's using the exact same code, but you can see the output table is properly named IP_Addresses, not ip_addresses.

The lakehouse where I'm seeing this error is very old - it was created 3 or 4 months ago at least. The one shown below where case is being properly respected is new. I wonder if something changed?

Thanks,

Scott

Re: Notebooks writing tables with names lowercased - how to fix?

Anonymous — Mon, 11 Mar 2024 17:17:37 GMT

Hi Scott_Powell ,

Spark is not case sensitive by default.

There is a way to handle this issue by adding spark config , using a SparkSession object named spark:

spark.conf.set('spark.sql.caseSensitive', True)

By default it is False.

Inorder to check the configuration status -

print(spark.conf.get('spark.sql.caseSensitive'))

Can you please try and let me know if it resolves your issue?

Hope this is helpful. Please let me know incase of further queries.

Re: Notebooks writing tables with names lowercased - how to fix?

Scott_Powell — Mon, 11 Mar 2024 18:24:28 GMT

Anonymous this seems to work perfectly - thank you! I'm not very comfortable with Spark stuff yet - is there a way to set this option "globally" either across all of Fabric, or maybe at the workspace level, so that we don't have to remember to put this code into every notebook?

This helped me a ton - thank you!

Scott

Re: Notebooks writing tables with names lowercased - how to fix?

Anonymous — Tue, 12 Mar 2024 05:42:06 GMT

Hi Scott_Powell ,

Unfortunately we don't have any option to do this "globally.

Glad to know your query got resolved. Please continue using Fabric Community for your further queries.

Re: Notebooks writing tables with names lowercased - how to fix?

XianZheng — Tue, 29 Apr 2025 16:03:41 GMT

Thanks this works!

Re: Notebooks writing tables with names lowercased - how to fix?

murat-to — Sun, 24 May 2026 23:13:23 GMT

Table Naming Inconsistency Causing Friction in Data Integration and Query Sharing

Even if we accept the case-insensitive + force-lowercase behavior (e.g., due to Hive metastore constraints),

there is still a consistency issue within Fabric itself.

As far as discussed here, there seems to be no way to define this behavior as a default at the workspace or artifact level.
This creates practical issues when integrating with existing data assets and sharing queries, as naming inconsistencies become a persistent and avoidable source of friction that users have to handle explicitly.

From a platform design perspective, this feels like a lack of cohesion:
a platform-level constraint is effectively being pushed onto users as an implementation burden.

At minimum, having a workspace-level or global configuration for naming behavior would significantly improve consistency and reduce confusion.