Power BI is turning 10! Tune in for a special live episode on July 24 with behind-the-scenes stories, product evolution highlights, and a sneak peek at what’s in store for the future.
Save the dateEnhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.
Hi all,
Greatly appreciate any thoughts or advice. I'm new to power query and created a simple query to pull some data from the web and am hoping to use this to perform some analysis. The problem is, the dataset is massive. It took about 8 hours to pull everything I needed. And well, that's fine if that's the time it takes to pull the data. The problem is, if I want to perform subsequent operations still using power query, it appears that any query operations referring back to the table I've created with the web data rerun the pull requests because the table is pointing to those pull formulas.
What I'm trying to get at here is if there's some way to simply treat this data as static going forward so that subsequent operations are simply performed on the data that was pulled itself, as opposed to a table pointing to a formula, which requires the pull request to rerun?
If my dataset were simpler, as a workaround I would've just copied the output data into a new table, created a new query and gone from there. The problem is, my dataset has nested tables, and (to my knowledge from poking around so far) I can't really see a way to copy a dataset that includes nested tables, create a new query with that dataset, and preserve the nested tables. I.e., in excel outside of the power query setting, this is just plain text entries of "[Table]".
Maybe I'm missing something, but it seems silly if, e.g., excel has pulled and loaded all the data I need from the web, and I want to add a new column in the nested tables with very simple data and I can't do this without completely rerunning the pull request. As a test, I ran an example on ~15,000 rows and added a new column to each of the nested tables that was simply the value "1", and the operation took just as long to complete as the first query that pulled the data from the web source. Again, maybe this is just my ignorance speaking, but this really just doesn't make sense to me since this seems like something that could be done in seconds if the data was static.
Any advice greatly appreciated!
Hi @webquery029 - you consider using a Power BI Dataflow. Creating a dataflow - Power BI | Microsoft Docs
This will allow you to GET the data from the Web into a staged CSV table. Using a second dataflow to read the first you can do transformations and joins.
Hi @webquery029
Can you please untick this feature and check. Please confirm
https://pbivisuals.com/2021/02/18/enable-load-include-in-report-refresh-in-query-editor/
If solve your requirement, please mark this answer as SOLUTION
If this comment helps you, please LIKE this comment/Kudos
Proud to be a Super User! | |
Hi, thank you for the comment! Unfortunately this doesn't resolve, but I appreciate the advice.