Forum Discussion
Fabric having issues importing or ingesting json files and responses
Hi smeetsh , Yeah, I think the issue is with the JSON complexity rather than the payload size too. Fabric generally handles large JSON files fine, but deeply nested and irregular schemas with optional arrays/objects can make the automatic schema import struggle or hang. The fact that even the smaller endpoint failed during schema import points more toward a schema inference limitation than a pure performance problem.
Your notebook -> Lakehouse -> ETL/Warehouse approach is probably the better fit here anyway. Spark gives you much more control over flattening arrays, handling nullable branches and managing evolving schemas compared to relying on automatic inference in the import UI.
All,
It appears indeed that the complexity of the json is the issue. I am trying to unravel it using a notebook and multiple table in our lakehouse. Powerquery is not a solution, but a notebook to ingest into lakehouse and using ETL to a warehouse may be.
Hi smeetsh , Yeah, I think the issue is with the JSON complexity rather than the payload size too. Fabric generally handles large JSON files fine, but deeply nested and irregular schemas with optional arrays/objects can make the automatic schema import struggle or hang. The fact that even the smaller endpoint failed during schema import points more toward a schema inference limitation than a pure performance problem.
Your notebook -> Lakehouse -> ETL/Warehouse approach is probably the better fit here anyway. Spark gives you much more control over flattening arrays, handling nullable branches and managing evolving schemas compared to relying on automatic inference in the import UI.