Forum Discussion
Community ChampionSource option missing
Hi smpa01,
Thank you for reaching out to the Microsoft Fabric Community Forum. The solutions provided by the datacoffee, are correct according to the thread.
And, I have identified few alternative workarounds that may help resolve the issue. Please follow these steps:
You are correct. When you create a KQL database directly from the Event house or Fabric UI, the "Source" step does not appear in the "New Table" wizard. This behaviour is intentional.
Navigate to the workspace settings by clicking the workspace name and selecting "Settings." Under "Workspace type," you should see “Real-Time Intelligence” listed. If this is not visible, you are not in an RTI workspace. Only RTI workspaces support streaming scenarios with the full table wizard, including the Source step.
Tables created manually from One Lake files will not automatically refresh when the files are updated. This information is crucial for setting expectations and directing them towards the appropriate methods, such as Event Streams or Notebooks.
Option A: Use Event Streams- Create a table from Event Streams > Add Destination > New Table.
- This enables the full wizard with Source, supports auto-refreshing data when new events arrive.
Option B: Use a Notebook to Manage Refresh
- Use a notebook to drop and recreate/update the table regularly.
- This is more manual but gives you control if Event Streams are not part of your current setup.
Kindly refer to the below documentation links:
Get data from Event stream - Microsoft Fabric | Microsoft Learn
Create an event stream in Microsoft Fabric - Microsoft Fabric | Microsoft Learn
Real-Time Intelligence in Microsoft Fabric documentation - Microsoft Fabric | Microsoft Learn
If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.
Thank you for using Microsoft Community Forum.
Community SupportHi smpa01,
Thank you for reaching out to the Microsoft fabric community forum. Also thanks for datacoffee, his inputs.
You're already using the correct documentation. I can confirm that the Source step is missing from your new table wizard. This typically happens when the table is being created outside of a Real-Time Intelligence (RTI) enabled workspace or not through an Event stream-connected flow. To resolve this, please try the following:
Verify Your Workspace Type: Ensure the workspace you're working in is a Real-Time Intelligence workspace. You can confirm this in the workspace settings. If not, you'll need to create a new workspace and select Real-Time Intelligence as the type.
Create the Table from the Event stream Interface: Instead of creating the table directly from the KQL database view. Go to Data Factory > Event stream. From there, add a destination and choose New Table. This will open the full table creation wizard with the “Source” step included.
Ensure Event stream is Connected to the KQL Database: The KQL database must be registered as a destination in an Event stream to support streaming ingestion.
Note: The Source tab only appears if a source is being used, such as an Event stream or another streaming ingestion method. If you're manually creating a table without using a streaming source, the Source step is intentionally skipped, as it’s not required.
If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.
Thank you for using Microsoft Community Forum.
Community ChampionVerify Your Workspace Type: Ensure the workspace you're working in is a Real-Time Intelligence workspace. You can confirm this in the workspace settings. - what am I looking for?
What amI currently doing? Creating Eventhouse -> creating KQL DB -> Was expecting same as 27:49
Insetead I see this after - Creating Eventhouse -> creating KQL DB
But I don't get the same option if I do this
I need to work with what I have; so circling back to this, if I stick to this; where I am connecting to One Lake file from this UI to build a db table; can I expect this to be auto refreshed upon the same onelake like update? I will be updating the files in onelake with notebook.
- v-kpoloju-msft
Hi smpa01,
Thank you for reaching out to the Microsoft Fabric Community Forum. The solutions provided by the datacoffee, are correct according to the thread.
And, I have identified few alternative workarounds that may help resolve the issue. Please follow these steps:
You are correct. When you create a KQL database directly from the Event house or Fabric UI, the "Source" step does not appear in the "New Table" wizard. This behaviour is intentional.
Navigate to the workspace settings by clicking the workspace name and selecting "Settings." Under "Workspace type," you should see “Real-Time Intelligence” listed. If this is not visible, you are not in an RTI workspace. Only RTI workspaces support streaming scenarios with the full table wizard, including the Source step.
Tables created manually from One Lake files will not automatically refresh when the files are updated. This information is crucial for setting expectations and directing them towards the appropriate methods, such as Event Streams or Notebooks.
Option A: Use Event Streams- Create a table from Event Streams > Add Destination > New Table.
- This enables the full wizard with Source, supports auto-refreshing data when new events arrive.
Option B: Use a Notebook to Manage Refresh
- Use a notebook to drop and recreate/update the table regularly.
- This is more manual but gives you control if Event Streams are not part of your current setup.
Kindly refer to the below documentation links:
Get data from Event stream - Microsoft Fabric | Microsoft Learn
Create an event stream in Microsoft Fabric - Microsoft Fabric | Microsoft Learn
Real-Time Intelligence in Microsoft Fabric documentation - Microsoft Fabric | Microsoft Learn
If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.
Thank you for using Microsoft Community Forum.- v-kpoloju-msft
Hi smpa01,
May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.
Thank you.
- v-kpoloju-msft
Hi smpa01,
I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.
Thank you.
- datacoffee
Most Valuable Professional
Hi
the easiest way to build a KQL table from a OneLake file is to create a shortcut.
select new in the menu and then find the shortcut option
here you can choose to have an accelerated shortcut or a "normal" shortcut. The accelerated option gives you KQL speed of the data when querying whereas the second option relies on the speed of the storage read in OneLake...
- smpa01
If I do what you are suggesting, will the KQL table always reflect up-to-date data?
- datacoffee
Yes - the KQL engine will "listen" to the underlying parquet files and update accordingly 😊
