Pandas in Fabric - Data size limit

[frithjof_v]

Hi,

What is the data size limit for working with Pandas dataframes in a Fabric notebook?

Does it depend on the F SKU size?

(E.g. F4 vs F64)

What happens if we go above the limit? 

Thanks in advance!

[frithjof_v · Accepted solution]

I got some answers here: https://www.reddit.com/r/MicrosoftFabric/s/90fY07XK3r

[Anonymous]

Hi @frithjof_v ,

As I understand it, the data size limit for using Pandas data frames in a Fabric notebook is largely dependent on the memory available in the environment. the F64 has more memory and can also handle larger data frames more efficiently.

If the memory limit of the environment is exceeded, the notebook may become very slow or unresponsive. It may even encounter memory errors that cause the operation to fail.

I don't see any official documentation mentioning or describing the limitations in detail, so maybe you could consider proposing an idea and waiting for users with similar needs to vote for your idea.

Best Regards,

Ada Wang

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[frithjof_v]

Thanks @Anonymous ,

However, this is a question, not an idea suggestion. 

I will wait to see if some users can help to answer my question here ☺️