After a little bit of investigation, it looks like the latest scan got detailed metatdata for some datasets but not others in the same workspace.
I'm wondering if I'm getting caught in the limitations spelled out here, but that doesn't make sense because the metadata is blank on datasets that have been refreshing daily. Run metadata scanning - Microsoft Fabric | Microsoft Learn
From the article:
- semantic models that haven't been refreshed or republished will be returned in API responses but without their subartifact information and expressions. For example, semantic model name and lineage are included in the response, but not the semantic model's table and column names.
- semantic models containing only DirectQuery tables will return subartifact metadata only if some sort of action has been taken on the semantic model, such as someone building a report on top of it, someone viewing a report based on it, etc."
How recently does the dataset need to be republished to get the metadata? This limitation seems like it would make it very difficult to get the initial full scan of metadata if the dataset has to be republished very recently. And in this case, I'm getting metadata for datasets last published a year ago, and not getting metadata for a dataset re-published in March.
The workspaces I'm using are in premium workspaces, so shouldn't be subject to the size limits
