Fabric - semantic model - refresh

[joakimfenno]

I just started with semantic models in fabric

What is the best way to resync a table a table from the source (i.e add a new column to a table in the model that has been added to the source table).

Currently I am using the edit tables button but it not really what I need in the case since it refresh the whole model.

A part from that it seems be a bit buggy - adding new tables works but using it to refresh current tables does not seem work every time.

How is this supposed to be done? 

[Anonymous]

Hi @joakimfenno ,

Is my follow-up just to ask if the problem has been solved?

If so, can you accept the correct answer as a solution or share your solution to help other members find it faster?

Thank you very much for your cooperation!

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

[Anonymous]

Hi @joakimfenno ,

Thank you for your interest in this case.

Glad to hear you found out what was causing the problem.

frithjof_v's suggestion is a good one, I think you can give feedback on Home (microsoft.com) and attach the link to the reply and I'll give you kudos, as I'm sure others who are concerned about this issue will do the same.

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

[joakimfenno]

Hi again
I still have issues with the GUI refresh
when trying Edit tables  - refresh I receive this (screenshot below)

[frithjof_v]

Are you using default or custom semantic model?

Did you delete a table column?

You could try to deselect a table from the semantic model, apply, and then reselect it again. If you do this, you could perhaps lose any measures which belongs to that table (I'm not sure) and relationships. But perhaps deselecting a table, apply, and then reselect the table will work.

I believe alternatives to the web GUI is Tabular Editor or Power BI Desktop (Power BI Desktop can't edit direct lake models yet, but it can create and edit Import mode models which offer more customization options).

[joakimfenno]

Hi

this is urgent and frustrating

shall I create a new forum topic for this issue specifically?

[frithjof_v]

I think I will recommend to create a support ticket https://support.fabric.microsoft.com

Also, I think there are two separate concepts here:

• Edit the schema of the model ("Edit Tables"). https://learn.microsoft.com/en-us/fabric/get-started/direct-lake-edit-tables
• Refresh the data content. https://learn.microsoft.com/en-us/fabric/get-started/direct-lake-overview#refresh

I believe the first one is the cause of your issues?

The latter can be done automatically, in that case there would be no need for your users to refresh.

Perhaps you will find your issue listed as a Known issue here: https://learn.microsoft.com/en-us/fabric/get-started/fabric-known-issues#currently-active-known-issu...

Similar topics:

https://community.fabric.microsoft.com/t5/General-Discussion/Edit-Tables-in-a-custom-Semantic-Model-...

https://community.fabric.microsoft.com/t5/Data-Engineering/Edit-tables-of-semantic-model-does-not-ad...

Using Tabular Editor to edit semantic models may be an alternative.

Or using Power BI desktop (import mode).

Are you using Lakehouse or Warehouse?

Anyway, using views is bad for performance in direct lake semantic models. Views make the report queries fall back to DirectQuery (which has lower performance than Direct Lake). So the recommendation is to use tables, not views, if you wish to use Direct Lake mode.

[joakimfenno · Accepted solution]

I found the issue

I had one view in the warehouse with two columns having identical names where one of them had a capital letter

I tried to refresh the warehouse from fabric gui and received the duplicate column name error message

It is fully possible (at least from sql endpoint) to create a view with two columns names aa and Aa for example. However its not supported in semantic models and semabtic model will not give you an indication of this issue 

[frithjof_v]

That's interesting! Thanks for sharing. So parts of Fabric is case sensitive while other parts of Fabric is not case sensitive. That's good to know. I think there should be a warning indicator or error message for these kind of issues.

[joakimfenno]

Thanks for getting back

My biggest issue rigth now is the first one ("Edit tables")

I use Warehouse as a source but there is a mix of tables (warehouse) and views (sourced from delta tables in lakehouse). I am aware about the direct lake functionality and I will soon only have tables in Warehouse as source. However, the semantic edit table refresh shoudl still work (pick up the updated model from warehouse).

I will create a support ticket.

Can you please guide me on how I do that or send me a more specific link? What permission do I need?

[frithjof_v]

I don't have experience with it, but I think scroll down to Power BI -> Get customer support https://support.fabric.microsoft.com

I think you should also check with IT / Microsoft 365 Admin in your organization to see if you have some special considerations regarding support tickets, how much data you can share with the MS support personnel, etc.

I think the issue you are encountering is a known issue and current workarounds seem to be to use Tabular Editor or Power BI Desktop (import mode).

[joakimfenno]

thanks

Import mode works but I need to create a general model that van be used for mutiplöe reports and be consumed directly from excel. Can you acheive this with Power BI Desktop?
regarding tabular editor - can ypou publish a semantic model or a general model from that tool?

[frithjof_v]

I think any semantic model is a general model.

You can create multiple reports with live connection to any semantic model.

I also think you can connect Excel to any semantic model.

This is possible for Power BI Desktop (import mode) semantic models, but the semantic model needs to be published to the web service before you can create new reports connected to the model or connect to the model from Excel.

Here's how to do it: you create a model (actually you need to create a report) in Power BI desktop, then you publish this report with it's model to the web service. This way the model becomes available to create other reports connected to this model, or connect to the model from Excel.

Obviously, there are sharing and governance questions regarding this, so be sure to read up on and understand topics like sharing of power bi models/reports, power bi access control and permissions, analyze in excel.

With tabular editor you can also create, edit and publish a semantic model. It can not be used to create a report.

Power BI desktop is used for creating report+model. It's not possible to create a model without report in Power BI desktop. So you will need to create a report (it could be empty) and it means you create a model at the same time. This is a bit unintuitive. However in order to create a model in Power BI desktop, you will need to create a report (it could be blank). When you publish the Power BI Desktop report+model to the web service, other reports and Excel can connect to the model which is hosted in the web service. I think Power BI desktop is a quite easy tool to use.

[joakimfenno]

custom model

I changed a source object from view to tabel (same name)

I tried deslect - select but that did not work

I will try the API for refreshing ythe model but this is frsutrating since some of our users will use the GUI

[Anonymous]

Hi @joakimfenno ,

Thanks for the reply from AndyDDC .

You can refresh individual tables using the Enhanced Refresh API. This method allows you to specify which tables to refresh.

This is a simple example code:

import sempy.fabric as fabric

# Define the dataset and workspace
dataset = "YourDatasetName"
workspace = "YourWorkspaceName"

# Objects to refresh
objects_to_refresh = [
    {"table": "YourTableName"}
]

# Refresh the dataset
fabric.refresh_dataset(workspace=workspace, dataset=dataset, objects=objects_to_refresh)

# List the refresh requests
fabric.list_refresh_requests(dataset=dataset, workspace=workspace)

For more information please see:

Refreshing Individual Tables and Partitions With Semantic Link (fabric.guru)

If you have any other questions please feel free to contact me.

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

[joakimfenno]

Hi

can this be used to add new columns (i.e update the model)?

what is dataset? is that the name of the semantic model?

do you run this within fabric (notebook) or outside?

[Anonymous]

Hi @joakimfenno ,

Thanks to all for their interest in this case.

1. Can this be used to add new columns (i.e. update the model)?

It can be used to refresh a single table, not to update.

2. What is Dataset and is this the name of the semantic model?

Yes.

3. Are you running it inside the fabric (Notebook) or outside?

Run it in a Fabric notebook, like the one shown below.

If you have any other questions please feel free to contact me.

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

[AndyDDC]

Hi @joakimfenno for adding new columns to a semantic model, in the Fabric UI you'll need to use the Edit Tables button, but you can filter what tables will be included in the refresh.  E.G in the image below I have added a new column to my Warehouse employees tables, but to get that column to appear in a custom semantic model, I need to open the model and click Edit Tables.  I can then enable/disable which tables I want the metadata to be refreshed for.

[joakimfenno]

I tried that and selected the table with thenew column only but that made all other tables dissapeared
and not only that, it seems like there is no optioon to reverse the change?

In this case my model (datatypes, relations, measures etc.) was huge

Can I reverse the change somehow?

Also, if I have two new columns in a table, how do I add one of them to the model? 

and actually, I tried with another model now, edit table did not pick up the new columns (source is a view in warehouse sourced from lakehouse)

[frithjof_v]

If you are using a view as data source, the queries will not run as Direct Lake instead they will run as DirectQuery.

Have you considered instead to use an Import mode model, created in Power BI desktop, to connect to your Fabric Warehouse?

Then you get more customization options, e.g. option to select specific columns.

[joakimfenno]

thanks the plan is to do it in the next phase