Forum Discussion
issue with enhanced compute engine settings for dataflow - workaround is to DISABLE
Hi frano72,
I think it probably related to premium workload settings.
You can refer to the following link to know the resource setting of premium capacity and ask the admin to check if your resource exceeds the limit/settings.
Configure workloads in a Premium capacity
Regards,
Xiaoxin Sheng
Anonymous - Hi, I think there is still an issue with the Service but I realise now what is going on.
The deeper understanding came from Mattew Roches excellent blog site.
https://ssbipolar.com/2020/09/16/dataflows-enhanced-compute-engine-will-it-fold/
basically I was thinking about it wrong (but it still should not have affected it).
The compute engine should be used for the ingestion dataflows. There is no need to use the compute engine for the transformation dataflows. It’s the transformation onces that are accessing the sql cache (ie compute engine) of the ingestion ones.
So this explains why it can be DISABLED for the transformation dataflows (what i originally thought was a workaround)
However – when the setting is set to OTPIMISE – it should realise that those entities are NOT linked to any further downstream dataflows and therefore should be equivalent to an DISABLED state.
So this is where I do think something has changed in the way this aspect works. Before Sep 24 an OPTIMISE setting was correctly determining these dataflows aren't linked anywhere else therefore treat as DISABLED. Since Sep 24, that logic isn't working, and workaround is to MANUALLY set them to DISABLED.