Connecting SQL Operators Back-to Back
Scenario: I want to build complex SQL transformations on the data in my evenstream. Currently, I am using Confluent and building Flink SQL on a Kafka topic. I can create views to encapsulate complex logic which is then computed on the fly. This enables me to syntactically split up complex transformations in more managable, multi-step operations (SQL CREATE VIEW Statement in Confluent Cloud for Apache Flink | Confluent Documentation).
Problem: In Fabric Eventstreams, I can't chain SQL operators to achieve the same behaviour. This is specifically listed in limitations here: Process Events Using a SQL Operator - Microsoft Fabric | Microsoft Learn. I am forced to materilize the query result as a derived stream, incurring additional cost and cluttering the UI.
1 Comment
- arindamcMicrosoft EmployeeStatus changed:NewtoNeeds Votes
Recent ideas
Data Pipelines - Run only selected activities
For debugging and testing pipeline activities during development, allow us to select one or multiple activities and run only the selected pipeline activities. For example, I'm working on editing ...frithjof_v22 hours agoCommunity ChampionNew618Views11likes2CommentsSemantic model connection bindings should be in source control (Git)
Semantic model data source connection bindings should be source controlled. A semantic model can contain multiple data source references, each of which can be mapped to a separate Fabric data connec...frithjof_v1 day agoCommunity ChampionNew21Views1like0CommentsBulk changing column names in Visualizations Pane
We often use raw/api column names or measures with a set nomenclature to be consistent and to keep track of them but we do not want to display these names in the visuals. Currently we have to change ...vishal1401971 day agoFrequent VisitorNew6Views0likes0Comments