Forum Discussion
Fabric Notebook too big
Hi Anonymous ,
Thanks for using Fabric Community and reporting this issue.
While going through your scenario team is in plan to extend size to 32M, same as the notebook size limit. This will increase output storage size and help users to work productively.
Unfortunately 32M may be not enough in your case, since notebook content will have a limit size and team can't extend this size to infinite.
In your case loading large data is fine, the problem is to display large dataframe in notebook output.
Note: Display in notebook = storage a large notebook content in our service system storage
You should try to display large output in files and not directly displaying in console.
We can observe display dataframe code cell in your notebook.When trying to run through pipeline or schedule, Notebook service will store all code cell outputs. But in manually run, it will show directly without saving outputs. That's why you are only facing this error in pipeline or schedule. The correct thing to do is check all code cells and aviod display or print large dataframe. Incase if requires to display the dataframe then you should display this output in files.!
Hope this is helpful. Please let me know incase of any queries.
Hi Anonymous ,
Thanks for using Fabric Community.
As I understand you were facing an issue - [Error: Size of notebook content and exit value must be less than 8MB. Please try to write large outputs to files (Parameter 'resultSnapshot') Actual value was Microsoft.Analytics.SynapseNotebookExecService.Models.RunNotebookSnapshot.] while retrieving data from an API.
Can you please share your workspace id, artifact id and activity id? We'd like to check if it's our issue or hit the error by design. It will be great if you can also share the code snippet along with it. We would like to understand why notebook requires snapshot more than 8M?
You can send us this information through email to AzCommunity[at]Microsoft[dot]com with the below details,
Email subject: <Attn - v-gchenna-msft : Fabric Notebook too big>
Thread URL: Fabric Notebook too big - Microsoft Fabric Community
Thanks for the quick response. I've sent everything over.
Hi Anonymous ,
Thank you for sharing the necessary details.
Team is looking into this issue and we will get back to you as soon as we have an update.Hi Anonymous ,
Thanks for using Fabric Community and reporting this issue.
While going through your scenario team is in plan to extend size to 32M, same as the notebook size limit. This will increase output storage size and help users to work productively.
Unfortunately 32M may be not enough in your case, since notebook content will have a limit size and team can't extend this size to infinite.
In your case loading large data is fine, the problem is to display large dataframe in notebook output.
Note: Display in notebook = storage a large notebook content in our service system storage
You should try to display large output in files and not directly displaying in console.
We can observe display dataframe code cell in your notebook.When trying to run through pipeline or schedule, Notebook service will store all code cell outputs. But in manually run, it will show directly without saving outputs. That's why you are only facing this error in pipeline or schedule. The correct thing to do is check all code cells and aviod display or print large dataframe. Incase if requires to display the dataframe then you should display this output in files.!
Hope this is helpful. Please let me know incase of any queries.