Forum Discussion

jnixon's avatar
jnixon
Advocate II
4 years ago

Get Power BI Dataflow csv from Azure Blob Storage

Greetings experts!  I'm trying to retrieve Power BI Dataflow connected to Gen2 Blob storage location.  I can retrieve the list of blobs (Lists  blob (V2)) that represent the entities from each Dataflow refresh from the 'model.json.snapshots/ folder.  Now I need to select the name of the blob with the most recent data in it, e.g. the third blob in this set:  

 

model.json@snapshot=2021-12-29T13:43:11.8221557Z
model.json@snapshot=2021-12-29T15:42:47.6820754Z
model.json@snapshot=2022-01-02T13:57:44.4572616Z 

 

Then I plan to set that file name to a variable and use as a parameter in a Get Blob Content action. 

 

I should also note that the output from the Lists Blob V2 action is a JSON object, which I then pass to a Create CSV table that generates the lines pasted above.  I need to get the JSON member for the most recent datetime (in red below), so maybe it is not necessary to make the csv first.  How can I extract/query the member with the recent datetime?  Here is the JSON output from the List blobs action:  

[
  {
    "Id": "xxxxx==",
    "Name": "model.json@snapshot=2021-12-29T13:43:11.8221557Z",
    "DisplayName": "model.json@snapshot=2021-12-29T13:43:11.8221557Z",
    "Path": "/powerbi/Azure test/Excel Datasource via Gateway/model.json.snapshots/model.json@snapshot=2021-12-29T13:43:11.8221557Z",
    "LastModified": "2021-12-29T13:43:11Z",
    "Size": 1267,
    "MediaType": "application/octet-stream",
    "IsFolder": false,
    "ETag": "\"xxxxx\"",
    "FileLocator": "xxxxx=="
  },
  {
    "Id": "xxxxx==",
    "Name": "model.json@snapshot=2021-12-29T15:42:47.6820754Z",
    "DisplayName": "model.json@snapshot=2021-12-29T15:42:47.6820754Z",
    "Path": "/powerbi/Azure test/Excel Datasource via Gateway/model.json.snapshots/model.json@snapshot=2021-12-29T15:42:47.6820754Z",
    "LastModified": "2021-12-29T15:42:47Z",
    "Size": 1587,
    "MediaType": "application/octet-stream",
    "IsFolder": false,
    "ETag": "\"xxxxx\"",
    "FileLocator": "xxxxx=="
  },
  {
    "Id": "xxxxx==",
    "Name": "model.json@snapshot=2022-01-02T13:57:44.4572616Z",
    "DisplayName": "model.json@snapshot=2022-01-02T13:57:44.4572616Z",
    "Path": "/powerbi/Azure test/Excel Datasource via Gateway/model.json.snapshots/model.json@snapshot=2022-01-02T13:57:44.4572616Z",
    "LastModified": "2022-01-02T13:57:44Z",
    "Size": 1587,
    "MediaType": "application/octet-stream",
    "IsFolder": false,
    "ETag": "\"xxxxx\"",
    "FileLocator": "xxxxx=="
  }
]
Need Help

2 Replies

  • V-lianl-msft's avatar
    V-lianl-msft
    Community Support
    4 years ago

    Hi jnixon ,

     

    To do this we first need to build a complete URL for the file. Build a formula that uses the URL to the container, the filename, and the querystring to create a complete URL, and name the field "FileURL"

    Then you can define a parameter to get the specified content. Please Refer to:

    https://www.sqlservercentral.com/blogs/connecting-to-azure-blobs-in-power-bi 

    https://powerbi.microsoft.com/de-de/blog/deep-dive-into-query-parameters-and-power-bi-templates/ 


    Best Regards,
    Liang
    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

  • jnixon's avatar
    jnixon
    Advocate II
    4 years ago

    Thanks Liang, I appreciate the reply.  My question was not well-structured and clear, I'm sorry. I was trying to get the csv files from a dataflow entity stored in Azure Blob Storage and copy them to a OneDrive cloud location.  I found a way to do this using Power Automate.  There is also a way to do this with Power BI, but it involves embedding a R or Python script into the Power Query to write these files to a local drive and then synching to the cloud.  Not reliable enough.

    Thanks again for you thoughts on it.