Fixed length decimal data type in Databricks

[MattKarriker2]

Good Day Forum

I am in the process of moving our Power BI reporting data sources from Azure Synapse to Azure Dtabricks.  Our models are composite model meaning we have direct query and and in-memory aggregation tables to help support our used experience.  To produce the models we followed Microsofts guidance to convert floating point decimals to fixed length decimals.  This is to reduce the oversize of the model on the in-memory tables.  However to make aggregation awareness work the data types of the direct query columns and the imported aggregate table need to match.  In Synapse this is no issue.  As the query engine can easily convert decimals to money in Synapse.  However there is no money/currency type available in databricks.  Converting the direct query column from floating point decimal to fixed length decimal on databricks sources fail reporing an error with query folding.  The real issue is Power BI is sending a Cast(Field as Money) to databricks and engine does not know how to handel it.  If I leave the direct query fields as floating and change the imported tables to fixed aggregation awareness does not work due to mismatched data types.  
Does anyone have experience converting floating point decimals to fixed length decimals using databricks as the source?  

[Gabry]

Hey @MattKarriker2 ,

I’m not very familiar with Databricks, so I can’t help much with that. However, are you sure that using the currency data type reduces memory footprint? I recall reading articles on SQLBI stating that currency  and decimal  both use 8 bytes.
Here I found the article https://www.sqlbi.com/articles/choosing-numeric-data-types-in-dax/

Have you tested the performance without converting the numbers?

EDIT: Oh, I just noticed this thread is from 2023, never mind! 😅