Forum Discussion
Microsoft EmployeeShare your thoughts on streaming dataflows aka SDF (preview feature)
Microsoft EmployeeHi, you can pull it by using Manage Fields operator
Helper IEklavya still does not work. When I use Manage fields to pull f.e. 1st level items (not nested), it works fine. When I try to do it with 2nd level nested items - it does not. I'm pulling all from the same event hub, same streaming dataflow, use manage fields, filters, output tables and the only output is for the items that are not nested.
Works here:
Does not work here:
Streaming Dataflow:
- Eklavya
I see. Let's do some quick checks here:
1. You've verified that env field does have data coming in from the source?2. Do you see the data if you remove the filter?
3. I am assuming you are using preview to check presence of data - but can you save and run the streaming dataflow and then check if you see the output in the report?
Is it possible to share the schema/sample event of the data you are using? (You can dm me this if you'd like).- MrKrukauskas
Eklavya MiguelMartinez here are the answers:
I see. Let's do some quick checks here:
1. You've verified that env field does have data coming in from the source?- Yes
2. Do you see the data if you remove the filter?
- I see data with and without filter in the Preview.
3. I am assuming you are using preview to check presence of data - but can you save and run the streaming dataflow and then check if you see the output in the report?
- Yes, in each step of the dataflow in the preview, i can see streaming and static data, even in the output tables.
- If I save and run, I only see the output for not nested data.
The Data preview in streaming dataflows: green - able to pull, not nested, red - not able to pull, nested two levels.
Thank You for checking this Eklavya
- MrKrukauskas
Just to add, in some cases empty rows are coming through with and without Filter feature.