Forum Discussion
Solved
Where Do I Start With Consuming Source Files That can be Queries in Fabric Data Warehouse
Hello, I know how to do this in AWS, but want to make sure I start on the correct path with MicroSoft Fabric Warehouse. We receive 520 files from our core system everyday. We want to create a...
What BI tool will you use? (Power BI?)
Will you query the files directly from your BI tool? Or will you load the data from the files into delta tables first, and then query the delta tables from your BI tool?
Could you write the contents of your files into a Delta table (by using Notebook), and use Notebook to add the Business Effective Date as a column in the Delta table?
frithjof_v
Community Champion
Community ChampionI've never done something like that before, but if you want to use files and directories I think you need to use the Lakehouse (for storing and working with the directories and files) instead of the Warehouse.
Sounds like a Notebook (or a combination of Notebook and Data pipeline) could be relevant tools.
Or Spark job (I never tried Spark job).