This behavior is not something that can be fixed by increasing a configurable file size limit. It is a limitation of how Microsoft Fabric handles file writes in OneLake, especially for append-style operations on text files.
In Fabric, continuously appending to a single .txt file is not a supported or reliable pattern. While small append operations may work initially, once the file grows beyond a certain size (around 100 KB in this case), further writes can silently fail without throwing an error. This is expected behavior due to the underlying storage and write semantics rather than a user-side configuration issue.
There is no setting in Fabric to increase this limit.
Recommended workarounds are:
Avoid a single growing log file
Instead of appending to one .txt file, generate multiple smaller log files (for example, per pipeline run, per batch, or per timestamp). This aligns with Fabric and OneLake design principles.
Use overwrite instead of append
If a single file is required, read the existing content, append new log entries in memory, and overwrite the file entirely. Overwrite operations are more stable than append operations in Fabric.
Use a table-based logging approach (best practice)
Store logs in a Lakehouse or Delta table with columns such as:
Timestamp
Pipeline / Process name
Step name
Log level
Message
Run ID
This approach removes file size limitations, supports querying and monitoring, and is the recommended enterprise logging pattern in Fabric.
In summary, Fabric is not designed for application-style, continuously growing text log files. The most stable and scalable solution is to switch to partitioned log files or table-based logging rather than trying to increase the file size of a single .txt log.
