[Issue] Power BI REST API Import Dataflow CreateOrOverwrite Error

[Anonymous]

Hi team,

It seems there is an issue with the Power BI REST API when attempting to Import a dataflow with an ImportConflictHandlerMode of "CreateOrOverwrite".

Steps:

Create the following request.

curl -X POST \
  'https://api.powerbi.com/v1.0/myorg/groups/{{workspaceId}}/imports?datasetDisplayName=model.json&nameConflict=CreateOrOverwrite' \
  -H 'Accept: */*' \
  -H 'Accept-Encoding: gzip, deflate' \
  -H 'Authorization: Bearer {{authorizationToken}}' \
  -H 'Cache-Control: no-cache' \
  -H 'Connection: keep-alive' \
  -H 'Content-Length: 1522' \
  -H 'Content-Type: application/json' \
  -H 'Host: api.powerbi.com' \
  -H 'Postman-Token: 98346372-e9c3-4c82-afdd-50050023c55f,ad5ca53d-35bf-4258-b487-b9e328e0baa6' \
  -H 'User-Agent: PostmanRuntime/7.15.2' \
  -H 'cache-control: no-cache' \
  -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
  -F =@/{{pathToDataflowJson}}

Expected behavior is for the request to successfully:

- create the dataflow if a dataflow with the same name does not exist

- overwrite the dataflow if a dataflow with the same name already exists

However, the received response is 404 Not Found with a body of:

{
    "error": {
        "code": "DuplicatePackageNotFoundError",
        "pbi.error": {
            "code": "DuplicatePackageNotFoundError",
            "parameters": {},
            "details": []
        }
    }
}

Other observed behavior:

1. Creating the dataflow first by hitting 

POST https://api.powerbi.com/v1.0/myorg/groups/{{workspaceId}}/imports?datasetDisplayName=model.json

first and then executing 

POST https://api.powerbi.com/v1.0/myorg/groups/{{workspaceId}}/imports?datasetDisplayName=model.json&nameConflict=CreateOrOverwrite

works as expected i.e. the dataflow is overwritten.

2. Hitting 

POST https://api.powerbi.com/v1.0/myorg/groups/{{workspaceId}}/imports?datasetDisplayName=model.json&nameConflict=Overwrite

without creating the dataflow prior to the request works as expected i.e. the system returns the following error

{
    "error": {
        "code": "DuplicatePackageNotFoundError",
        "pbi.error": {
            "code": "DuplicatePackageNotFoundError",
            "parameters": {},
            "details": []
        }
    }
}

3. Hitting 

POST https://api.powerbi.com/v1.0/myorg/groups/{{workspaceId}}/imports?datasetDisplayName=model.json

while the dataflow exists works as expected i.e. returns:

{
    "error": {
        "code": "Conflict",
        "message": "Dataflow 'OdataDataflow' already exists"
    }
}

Could you please look into this?

Thanks,

Laurentiu

[v-qiuyu-msft]

Hi @Anonymous, 

I would suggest you create a support ticket to let engineers look into the issue on your side. 

Best Regards,
Qiuyun Yu 

[dbeavon3]

It has already been three years. But there is now a bug reported to the PG!

The title of the bug is "REST API - imports with parameter CreateOrOverwrite throws PackagenotfoundError" and it was created on 5th Sept 2022.

Here is the Bug ID: 875909

Maybe the PG will fix this before another three years have passed.  (I was not the one who first reported that bug to them, but contacted them in December 2022 when I noticed the same problem.)

[dbeavon3]

Another clue from me... there appears to be a very loose relationship between that error and the chain of previous owners of the dataflow.  (Maybe there is some file in an internal Azure storage account with a bad ACL or something like that.) 

Even "taking over" the dataflow doesn't make the error go away.

I had to export the json, delete the dataflow, then re-import.  After that, the normal name-conflict modes would start to work (Overwrite/ CreateOrOverwrite).

Hope someone can double check me on this....

[ABI_Rufino]

As described on the documentation here, for the 'nameConflict' parameter, dataflows only support as options 'Abort' or 'GenerateUniqueName'.

The 'CreateOrOverwrite' option is only for datasets.

[dbeavon3]

@ABI_Rufino The docs for Power BI are truly confusing, and are not internally consistent.  I think the problem is that the PG keeps changing their minds about how their platform is supposed to work.  The docs suffer as well, since these docs can't be of higher quality than the underlying software itself.

I would suggest you explore the behavior yourself, and open CSS cases to talk to Satya and Jeroen, rather than just going by what the documentation says.

At the end of my most recent support case, they had made a few rounds of changes to the docs and they may have overlooked the part you shared.  There are plenty of github repos and blogs that contradict the information in those docs.  Here is the part of the docs which I was involved with, when I was last working with CSS :

See:
https://learn.microsoft.com/en-us/power-bi/transform-model/dataflows/dataflows-features-limitations#...

Dataflows can be overwritten with the CreateOrOverwrite parameter, if they have initially been created using the import API.

I think there are lots of other caveats however, and you must explore them for yourself.  For example, the platform may not be happy about two administrators of the same workspace who are overwriting/redeploying each other's dataflow.  In general Power BI features aren't very well tested to ensure the proper behaviors when multiple users are managing a single workspace.  The product seems to be designed to assume single-user administration of a given workspace.  Just as one example, an admin of a workspace does not even have read-only access to all the dataset configuration details until they "take over" the dataset.  It is pretty silly, IMHO.