Hi,
I have been having issues with dataflows for over 2 months now and I have tried everything I can think of, but still keep getting errors on my refresh. A handful of times the refresh has randomly worked, but then the next day - without making any changes - it refreshes and fails again, and then typically just never works again. I have tried:
Creating the dataflow in a new workspace
Creating a fresh dataflow (multiple times)
Creating a new entity within the dataflow
Reducing the time period being queried (though it had already worked with the original time period)
Removing special characters from entity names
Changing the logic of my m code a hundred different ways
Waiting for a new day to allow any limits to reset
Disabling all other dataflows and reports querying our GA4 properties for a day
Conducting the refresh at all hours of the day and all days of the week
I'm sure there are other things I've tried and am forgetting, but I receive these errors each time and I have no idea what they refer to. Please can someone assist me?
Here is a small sample list of some of my failed refreshes, with one refresh that was successful in the middle:
| Requested on | Dataflow name | Dataflow refresh status | Table name | Partition name | Refresh status | Start time | End time | Duration | Rows processed | Bytes processed (KB) | Max commit (KB) | Processor Time | Wait time | Compute engine | Error |
| 23/11/2024 01:16 | GA4 - Overall - Monthly | Failed | Fact_GA4_Overall_Monthly | NA | Failed | 23/11/2024 01:16 | ######## | 58:39.0 | NA | NA | NA | NA | NA | NA | Error: Request ID: 877c243d-5db4-4940-bf2c-c14487a027cf Activity ID: dbc591b1-597a-441d-8c0e-eae750d70167 |
| 45615.68793 | GA4 - Overall - Monthly - Last 14 | Failed | GA4_Overall_Old | NA | Failed | 45615.68793 | 45615.74 | 0.050053 | NA | NA | NA | NA | NA | NA | Error: DataFormat.Error: We found an unexpected character in the JSON input.. RootActivityId = 6eaf859f-cfea-492f-90cd-cfc8234fec3b.Param1 = DataFormat.Error: We found an unexpected character in the JSON input. Request ID: 18086ea1-ee5a-c92a-a2f4-a41d97f8fb1c. |
| 45615.92657 | GA4 - Overall - Monthly - Last 14 | Failed | GA4_Overall_Old | NA | Failed | 45615.92657 | 45615.96 | 0.033101 | NA | NA | NA | NA | NA | NA | Error: DataFormat.Error: We found an unexpected character in the JSON input.. RootActivityId = 0bdcaea4-21fe-40f4-851c-ebeb3bd35b29.Param1 = DataFormat.Error: We found an unexpected character in the JSON input. Request ID: c56a8f04-d2c4-9437-76eb-6e57962c75d0. |
| 19/11/2024 23:56 | GA4 - Overall - Monthly - Last 14 | Failed | GA4_Overall_Old | NA | Failed | 19/11/2024 23:56 | ######## | 49:13.3 | NA | NA | NA | NA | NA | NA | Error: DataFormat.Error: We found an unexpected character in the JSON input.. RootActivityId = c7e330fe-e520-436d-b616-daddf0d671cc.Param1 = DataFormat.Error: We found an unexpected character in the JSON input. Request ID: 9b3b675b-270b-b413-aed4-5020c66678ee. |
| 20/11/2024 01:11 | GA4 - Overall - Monthly - Last 14 | Completed | GA4_Overall_Old | FullRefreshPolicyPartition | Completed | 20/11/2024 01:11 | ######## | 30:29.9 | NA | 3896183 | 324620 | 29:50.2 | 00:00.1 | Not used | NA |
| 20/11/2024 18:08 | GA4 - Overall - Monthly - Last 14 | Failed | GA4_Overall_Monthly | NA | Failed | 20/11/2024 18:08 | ######## | 43:34.3 | NA | NA | NA | NA | NA | NA | Error: Request ID: 7acf2320-1e70-4d1e-af81-a4b388c1056c Activity ID: 055230b3-5a53-4370-a5a4-28f1afb7ebb2 |
| 21/11/2024 18:08 | GA4 - Overall - Monthly - Last 14 | Failed | GA4_Overall_Monthly | NA | Failed | 21/11/2024 18:08 | ######## | 45:05.3 | NA | NA | NA | NA | NA | NA | Error: Request ID: 59ea8659-c6dd-4cbb-b98d-c42801169f61 Activity ID: e3ae6555-2027-4857-b0c4-12580b1b6b1f |
| 22/11/2024 11:39 | GA4 - Overall - Monthly - Last 14 | Failed | GA4_Overall_Monthly | NA | Failed | 22/11/2024 11:39 | ######## | 48:03.7 | NA | NA | NA | NA | NA | NA | Error: Request ID: 023b7ba7-8ee0-016c-ca82-ae572c78ec7f Activity ID: 3b50a173-efdd-4ffc-a622-dc003ca34d8f |
| 22/11/2024 13:41 | GA4 - Overall - Monthly - Last 14 | Failed | GA4_Overall_Monthly | NA | Failed | 22/11/2024 13:41 | ######## | 44:03.4 | NA | NA | NA | NA | NA | NA | Error: Request ID: 48e41299-7bc1-1d3d-db85-ae5b629ab0f0 Activity ID: 913e2e6a-5d15-44b7-9f69-ac9d866fcbd3 |
| 45618.75556 | GA4 - Overall - Monthly - Last 14 | Failed | GA4_Overall_Monthly | NA | Failed | 45618.75556 | 45618.79 | 0.036161 | NA | NA | NA | NA | NA | NA | Error: Request ID: a8b4ef38-d2c8-4dbd-8b61-d18478fab9cb Activity ID: 9f76192e-4ed0-4240-9079-dcc8f8ce5b99 |
| 22/11/2024 19:54 | GA4 - Overall - Monthly - Last 14 | Failed | GA4_Overall_Monthly | NA | Failed | 22/11/2024 19:54 | ######## | 46:03.5 | NA | NA | NA | NA | NA | NA | Error: Request ID: 38d356ba-34ed-b2d9-4e88-c7c0d958f8f0 Activity ID: eef3c9c2-7662-4436-8051-3e7dd23a8862 |
| 23/11/2024 10:21 | Monthly | Failed | GA4 Overall Monthly | NA | Failed | 23/11/2024 10:21 | ######## | 44:03.6 | NA | NA | NA | NA | NA | NA | Error: Request ID: 5537387a-1945-ad71-09f1-1eed2051137a Activity ID: 16e25a19-d6fe-448a-b5fb-9600eca336fb |
| 23/11/2024 11:56 | GA4 - Overall - Monthly | Failed | GA4_Overall_Monthly | NA | Failed | 23/11/2024 11:56 | ######## | 03:00.8 | NA | NA | NA | NA | NA | NA | Error: Request ID: 875a700a-4dfb-251f-423d-6d212486e08d Activity ID: 746ff818-83f7-4aca-8cda-10cf3e8aa219 |
As you can see, the errors do not explain anything much. Sometimes I get an error 'DataFormat.Error: We found an unexpected character in the JSON input..', but other times, once the refresh has worked again, it just randomly begins giving me 'Error: Request ID: 7acf2320-1e70-4d1e-af81-a4b388c1056c Activity ID: 055230b3-5a53-4370-a5a4-28f1afb7ebb2' (with a different Request ID and Activity ID each time).
I have tried searching for similar issues, but none of them have resolved my issue.
Any help would be greatly appreciated.
Thank you
