Forum Discussion

smpa01's avatar
smpa01
Icon for Community Champion rankCommunity Champion
2 years ago

Reading limitation spark vs sempy

Does the reading limitation (same limitation as Execute Queries)  of semantic tables through spark apply to reading through sempy?   I am building a production element and whether I choose spark ...
notebook
smpa01's avatar
smpa01
Icon for Community Champion rankCommunity Champion
2 years ago

I am more interested in number of rows read for which spark has clear limiations. Can you please confirm if sempy has any limitations in respect to that? Anonymous I am not interested in any other limiations sempy might have ATM.  Cause I need to handle that, if it does have any such limitations.

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi smpa01 ,

 

I have not seen documentation that indicates sempy is limit on the number of rows that can be read.

 

The limit I see in the documentation for sempy is:

 

The amount of data that can be retrieved is limited by the per-query maximum memory of the capacity SKU hosting the semantic model and the Spark driver node running the notebook (see Node Size).

 

Please see this official documentation for information:

Read data from semantic models and write data that semantic models can consume using python - Microsoft Fabric | Microsoft Learn

 

If you have any other questions please feel free to contact me.

 

Best Regards,
Yang
Community Support Team

 

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!