Forum Discussion
Appending 419 pages
I have a single pdf report (consisting of 419 pages) that has been imported into PQ and the page appended. 6600 rows and 15 columns. Using Windows 10. Fast processor and 64 mb or ram. Even so, it takes a LONG time to update. Is there a way I can speed this up? Do I need to break this up into multiple appends and then append those?
Any help is greatly appreciated.
Hi Robjr Try these please
- Filter pages and columns early in the query to limit data.
- Split the PDF into smaller files, process separately, and append later.
- Use Table.Buffer to reduce repeated data loads:BufferedTable = Table.Buffer(YourTable)
- Simplify the query by removing unnecessary steps and filtering early.
- You could also Pre - Process the pdf using Python or any other compatable tool
If this post helped please do give a kudos and accept this as a solution
Thanks In Advance
3 Replies
Hi Robjr Try these please
- Filter pages and columns early in the query to limit data.
- Split the PDF into smaller files, process separately, and append later.
- Use Table.Buffer to reduce repeated data loads:BufferedTable = Table.Buffer(YourTable)
- Simplify the query by removing unnecessary steps and filtering early.
- You could also Pre - Process the pdf using Python or any other compatable tool
If this post helped please do give a kudos and accept this as a solution
Thanks In Advance
Thank you for your sugggestions. I had already incorporated several of them but the Table.Buffer approach was unknown to me. I found a couple of good instructional videos on YouTube (e.g., from APQ16) that guided me through the process.
To speed up your Power Query process with a large PDF (419 pages, 6600 rows, 15 columns), consider these optimizations: 1) Load only necessary columns to reduce data size. 2) Use Table.Buffer() after key transformation steps to store intermediate results in memory. 3) Disable "Enable Data Preview" under Query Options. 4) If merging queries, ensure they're optimized with indexed keys. 5) Load the PDF in smaller batches, process them separately, and then append them. 6) Use a folder-based approach instead of a single PDF file if possible. These steps should improve performance significantly.
