Forum Discussion
Helper IIIPower BI Dataflow - response too large to return
Can't use direct connection to BQ for access reasonsChallenge that assumption. Tell them that 650 GB is too much.
If they don't budge, use Incremental Refresh. Start VERY small, with a day or so. See how many rows you get. Recommended partition size is around 8 to 20 million rows.
Please explain "import does not work" - are you getting an error message?
Direct Query on a dataflow = datamart = lipstick on a pig (Azure SQL db being the lipstick and the dataflow being the pig). Possible, sure, but a travesty nevertheless.
You can consider switching the incremental refresh to the dataset instead and to completely remove the dataflow from the equation.
You can consider dataflow Gen2 and store the results as Delta Lake in Fabric.
Super Useryes, looks like your day partitions processed fine. Do not add your own filters for the range, and do not use any transforms that break query folding.
Helper IIIAm sorry was that a yes for getting rid of the date filter which i had initially ? (last 30 days)
Because right now when connected to the dataflow, it's empty - shows nothing on there.
Just want to make sure.
The only tansformations i have on there is that date filter and one where i remove those 20 odd columns.
Oh actually one more i had to change the data type on the date column from date to date/time in order to use it for incremental refresh.