topic Re: Fabric - semantic model - refresh in Data Warehouse

Fabric - semantic model - refresh

joakimfenno — Thu, 05 Sep 2024 12:15:02 GMT

I just started with semantic models in fabric

What is the best way to resync a table a table from the source (i.e add a new column to a table in the model that has been added to the source table).

Currently I am using the edit tables button but it not really what I need in the case since it refresh the whole model.

A part from that it seems be a bit buggy - adding new tables works but using it to refresh current tables does not seem work every time.

How is this supposed to be done?

Re: Fabric - semantic model - refresh

AndyDDC — Thu, 05 Sep 2024 14:26:56 GMT

Hi joakimfenno for adding new columns to a semantic model, in the Fabric UI you'll need to use the Edit Tables button, but you can filter what tables will be included in the refresh. E.G in the image below I have added a new column to my Warehouse employees tables, but to get that column to appear in a custom semantic model, I need to open the model and click Edit Tables. I can then enable/disable which tables I want the metadata to be refreshed for.

Re: Fabric - semantic model - refresh

Anonymous — Fri, 06 Sep 2024 06:50:14 GMT

Hi joakimfenno ,

Thanks for the reply from AndyDDC .

You can refresh individual tables using the Enhanced Refresh API. This method allows you to specify which tables to refresh.

This is a simple example code:

import sempy.fabric as fabric # Define the dataset and workspace dataset = "YourDatasetName" workspace = "YourWorkspaceName" # Objects to refresh objects_to_refresh = [ {"table": "YourTableName"} ] # Refresh the dataset fabric.refresh_dataset(workspace=workspace, dataset=dataset, objects=objects_to_refresh) # List the refresh requests fabric.list_refresh_requests(dataset=dataset, workspace=workspace)

For more information please see:

Refreshing Individual Tables and Partitions With Semantic Link (fabric.guru)

If you have any other questions please feel free to contact me.

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

Re: Fabric - semantic model - refresh

joakimfenno — Fri, 06 Sep 2024 07:14:17 GMT

I tried that and selected the table with thenew column only but that made all other tables dissapeared
and not only that, it seems like there is no optioon to reverse the change?

In this case my model (datatypes, relations, measures etc.) was huge

Can I reverse the change somehow?

Also, if I have two new columns in a table, how do I add one of them to the model?

and actually, I tried with another model now, edit table did not pick up the new columns (source is a view in warehouse sourced from lakehouse)

Re: Fabric - semantic model - refresh

AndyDDC — Fri, 06 Sep 2024 08:04:50 GMT

I should have also stated that you're existing tables will need to be selected, my apologies for not being more clear on that.

As for column-by-column I don't believe you can do that.

Re: Fabric - semantic model - refresh

frithjof_v — Fri, 06 Sep 2024 17:47:38 GMT

If you are using a view as data source, the queries will not run as Direct Lake instead they will run as DirectQuery.

Have you considered instead to use an Import mode model, created in Power BI desktop, to connect to your Fabric Warehouse?

Then you get more customization options, e.g. option to select specific columns.

Re: Fabric - semantic model - refresh

joakimfenno — Sat, 07 Sep 2024 04:30:29 GMT

thanks the plan is to do it in the next phase

Re: Fabric - semantic model - refresh

joakimfenno — Sat, 07 Sep 2024 04:31:31 GMT

unfortunatley I had ro recreate the whole model

seems like there is no way to version control semantic models yet?

Re: Fabric - semantic model - refresh

AndyDDC — Sat, 07 Sep 2024 08:36:02 GMT

You can link the Workspace to Azure DevOps or GitHub and source control the Semantic Models there:

Re: Fabric - semantic model - refresh

joakimfenno — Mon, 09 Sep 2024 07:11:47 GMT

Hi again
I still have issues with the GUI refresh
when trying Edit tables - refresh I receive this (screenshot below)

Re: Fabric - semantic model - refresh

frithjof_v — Mon, 09 Sep 2024 07:51:14 GMT

Are you using default or custom semantic model?

Did you delete a table column?

You could try to deselect a table from the semantic model, apply, and then reselect it again. If you do this, you could perhaps lose any measures which belongs to that table (I'm not sure) and relationships. But perhaps deselecting a table, apply, and then reselect the table will work.

I believe alternatives to the web GUI is Tabular Editor or Power BI Desktop (Power BI Desktop can't edit direct lake models yet, but it can create and edit Import mode models which offer more customization options).

Re: Fabric - semantic model - refresh

joakimfenno — Mon, 09 Sep 2024 07:52:28 GMT

custom model

I changed a source object from view to tabel (same name)

I tried deslect - select but that did not work

I will try the API for refreshing ythe model but this is frsutrating since some of our users will use the GUI

Re: Fabric - semantic model - refresh

joakimfenno — Mon, 09 Sep 2024 08:17:58 GMT

can this be used to add new columns (i.e update the model)?

what is dataset? is that the name of the semantic model?

do you run this within fabric (notebook) or outside?

Re: Fabric - semantic model - refresh

Anonymous — Mon, 09 Sep 2024 08:36:58 GMT

Hi joakimfenno ,

Thanks to all for their interest in this case.

1. Can this be used to add new columns (i.e. update the model)?

It can be used to refresh a single table, not to update.

2. What is Dataset and is this the name of the semantic model?

Yes.

3. Are you running it inside the fabric (Notebook) or outside?

Run it in a Fabric notebook, like the one shown below.

If you have any other questions please feel free to contact me.

Best Regards,
Yang
Community Support Team

Re: Fabric - semantic model - refresh

joakimfenno — Tue, 10 Sep 2024 04:18:58 GMT

this is urgent and frustrating

shall I create a new forum topic for this issue specifically?

Re: Fabric - semantic model - refresh

frithjof_v — Tue, 10 Sep 2024 06:20:57 GMT

I think I will recommend to create a support ticket https://support.fabric.microsoft.com

Also, I think there are two separate concepts here:

Edit the schema of the model ("Edit Tables"). https://learn.microsoft.com/en-us/fabric/get-started/direct-lake-edit-tables
Refresh the data content. https://learn.microsoft.com/en-us/fabric/get-started/direct-lake-overview#refresh

I believe the first one is the cause of your issues?

The latter can be done automatically, in that case there would be no need for your users to refresh.

Perhaps you will find your issue listed as a Known issue here: https://learn.microsoft.com/en-us/fabric/get-started/fabric-known-issues#currently-active-known-issues

Re: Fabric - semantic model - refresh

joakimfenno — Tue, 10 Sep 2024 06:36:10 GMT

Thanks for getting back

My biggest issue rigth now is the first one ("Edit tables")

I use Warehouse as a source but there is a mix of tables (warehouse) and views (sourced from delta tables in lakehouse). I am aware about the direct lake functionality and I will soon only have tables in Warehouse as source. However, the semantic edit table refresh shoudl still work (pick up the updated model from warehouse).

I will create a support ticket.

Can you please guide me on how I do that or send me a more specific link? What permission do I need?

Re: Fabric - semantic model - refresh

frithjof_v — Tue, 10 Sep 2024 07:01:51 GMT

I don't have experience with it, but I think scroll down to Power BI -> Get customer support https://support.fabric.microsoft.com

I think you should also check with IT / Microsoft 365 Admin in your organization to see if you have some special considerations regarding support tickets, how much data you can share with the MS support personnel, etc.

I think the issue you are encountering is a known issue and current workarounds seem to be to use Tabular Editor or Power BI Desktop (import mode).

Re: Fabric - semantic model - refresh

joakimfenno — Tue, 10 Sep 2024 07:45:10 GMT

thanks

Import mode works but I need to create a general model that van be used for mutiplöe reports and be consumed directly from excel. Can you acheive this with Power BI Desktop?
regarding tabular editor - can ypou publish a semantic model or a general model from that tool?

Re: Fabric - semantic model - refresh

frithjof_v — Tue, 10 Sep 2024 08:26:29 GMT

I think any semantic model is a general model.

You can create multiple reports with live connection to any semantic model.

I also think you can connect Excel to any semantic model.

This is possible for Power BI Desktop (import mode) semantic models, but the semantic model needs to be published to the web service before you can create new reports connected to the model or connect to the model from Excel.

Here's how to do it: you create a model (actually you need to create a report) in Power BI desktop, then you publish this report with it's model to the web service. This way the model becomes available to create other reports connected to this model, or connect to the model from Excel.

Obviously, there are sharing and governance questions regarding this, so be sure to read up on and understand topics like sharing of power bi models/reports, power bi access control and permissions, analyze in excel.

With tabular editor you can also create, edit and publish a semantic model. It can not be used to create a report.

Power BI desktop is used for creating report+model. It's not possible to create a model without report in Power BI desktop. So you will need to create a report (it could be empty) and it means you create a model at the same time. This is a bit unintuitive. However in order to create a model in Power BI desktop, you will need to create a report (it could be blank). When you publish the Power BI Desktop report+model to the web service, other reports and Excel can connect to the model which is hosted in the web service. I think Power BI desktop is a quite easy tool to use.