Forcing the refresh of the sql endpoint of a lakehouse using

[smeetsh]

Hi All,

I came across this site:
Code Examples · microsoft/semantic-link-labs Wiki · GitHub
I am trying to implement the semanticlink, but it doesnt seem to work. My notebook is as follows:

----------

#%pip install semantic-link-labs 

import sempy_labs as labs

item = 'xxx' # Enter the name or ID of the Fabric item

type = 'Lakehouse' # Enter the item type

workspace = 'yyy' # Enter the name or ID of the workspace

# Example 1: Refresh the metadata of all tables

tables = None

x = labs.refresh_sql_endpoint_metadata(item=item, type=type, workspace=workspace, tables=tables)

display(x)

--------

When i run it I get error: TypeError: got an unexpected keyword argument 'tables'

The language I have chosen in pyspark(python).

What am I missing here?
Cheers

Hans

[smeetsh]

I have created an issue on the git, lets see what comes back

[R1k91]

the bug is in this file I think: _sql_endpoints.py 

the parameter "tables" doesn't exist for that function.

def refresh_sql_endpoint_metadata(
    item: str | UUID,
    type: Literal["Lakehouse", "MirroredDatabase"],
    workspace: Optional[str | UUID] = None,
    timeout_unit: Literal["Seconds", "Minutes", "Hours", "Days"] = "Minutes",
    timeout_value: int = 15,
) -> pd.DataFrame:
    """
    Refreshes the metadata of a SQL endpoint.

    This is a wrapper function for the following API: `Items - Refresh Sql Endpoint Metadata <https://learn.microsoft.com/rest/api/fabric/sqlendpoint/items/refresh-sql-endpoint-metadata>`_.

    Service Principal Authentication is supported (see `here <https://github.com/microsoft/semantic-link-labs/blob/main/notebooks/Service%20Principal.ipynb>`_ for examples).

    Parameters
    ----------
    item : str | uuid.UUID
        The name or ID of the item (Lakehouse or MirroredDatabase).
    type : Literal['Lakehouse', 'MirroredDatabase']
        The type of the item. Must be 'Lakehouse' or 'MirroredDatabase'.
    workspace : str | uuid.UUID, default=None
        The Fabric workspace name or ID.
        Defaults to None which resolves to the workspace of the attached lakehouse
        or if no lakehouse attached, resolves to the workspace of the notebook.
    timeout_unit : Literal['Seconds', 'Minutes', 'Hours', 'Days'], default='Minutes'
        The unit of time for the request duration before timing out. Additional duration types may be added over time.
    timeout_value : int, default=15
        The number of time units in the request duration.

anyway as clearly stated in the function definition this is just a wrapper of the api that refresh metadata for all tables it seems there's no way to do it just for one table (as of now).

[R1k91 · Accepted solution]

Kowalsky confirmed on the issue's thread that the example is wrong since tables parameters is not exposed by the api.
as of now you can't choose which table to refresh, it's all or nothing.

[tayloramy]

Hi @smeetsh, 

I have not used semantic link labs to do this, so I'm not sure what error you may be running into there. Seeing as you're using the sample code, maybe this is a bug that you should report? 

What I have done is refresh directly with the API. IT is described here: 

Refresh SQL analytics endpoint Metadata REST API (Generally Available) | Microsoft Fabric Blog | Mic... 

and further described here: Items - Refresh Sql Endpoint Metadata - REST API (SQLEndpoint) | Microsoft Learn

If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.

[smeetsh]

I am aware of that article, but it requires some work in the background (clientID's secrets, api keys etc), Also it seems to refresh the whole lakehouse and we have a lot of pipelines that are scheduled at various times, which means i will be calling that notebook dozens of times per day, which seems a waste of CU's