Forum Discussion

bloigge's avatar
bloigge
Advocate IV
11 years ago
Solved

Refreshing queries with functions doens't work

Does anyone else experience that refreshing on-premise data via a gateway (for example amount of files in a folder) works if you hardcode the path to the folder, but when you create a function where the path is the input parameter powerbi says that you are not able to refresh this data source?

19 Replies

  • Greg_Deckler's avatar
    Greg_Deckler
    Community Champion

    Have not tried that one, can you publish your function here and I'll play around with it and see if I can get it to function. (pun intended)

  • miguel's avatar
    miguel
    Community Admin

    I've tried using a function from an Azure Machine Learning Web Service and it works perfectly. I've tried others that work over a set of tables from Excel Files and also from a SQL Database table and they work great as well.

     

    Is there any chance that you can paste the M code here so we can take a look?

    • bloigge's avatar
      bloigge
      Advocate IV

      Hardcoded Folder Path Query is working - Gateway options are shown:

       

       

      let
          Source = Folder.Files("P:\MyFolder\MySubFolder"),
          #"Sorted Rows" = Table.Sort(Source,{{"Date created", Order.Descending}}),
          #"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 0, 1),
          #"Filtered Rows" = Table.SelectRows(#"Added Index", each ([Index] = 0)),
          tempPath = #"Filtered Rows"{0}[Folder Path],
          folderPath = Text.Range(Text.Range(tempPath,0,Text.Length(tempPath)-1),Text.PositionOfAny(Text.Range(tempPath,0,Text.Length(tempPath)-1),{"\"},Occurrence.Last) + 1,(Text.Length(tempPath)-1) - (Text.PositionOfAny(Text.Range(tempPath,0,Text.Length(tempPath)-1),{"\"},Occurrence.Last) + 1)),
          #"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"Content", "Name", "Extension", "Date accessed", "Date modified", "Attributes", "Folder Path", "Index"}),
          #"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Date created", folderPath}})
      in
          #"Renamed Columns"

       

       

      Creating a function which I then recall multiple times is not working:

       

       

      let
          Abfrage = (path as text) =>
      let
          Source = Folder.Files(path),
          #"Sorted Rows" = Table.Sort(Source,{{"Date created", Order.Descending}}),
          #"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 0, 1),
          #"Filtered Rows" = Table.SelectRows(#"Added Index", each ([Index] = 0)),
          tempPath = #"Filtered Rows"{0}[Folder Path],
          folderPath = Text.Range(Text.Range(tempPath,0,Text.Length(tempPath)-1),Text.PositionOfAny(Text.Range(tempPath,0,Text.Length(tempPath)-1),{"\"},Occurrence.Last) + 1,(Text.Length(tempPath)-1) - (Text.PositionOfAny(Text.Range(tempPath,0,Text.Length(tempPath)-1),{"\"},Occurrence.Last) + 1)),
          #"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"Content", "Name", "Extension", "Date accessed", "Date modified", "Attributes", "Folder Path", "Index"}),
          #"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Date created", folderPath}})
      in
          #"Renamed Columns"
      in
          Abfrage

       

       

      Calling the function:

       

      let
          Source = GetLatestFile("P:\MyFolder\MySubFolder")
      in
          Source

       

      The notification I get when using a function:

      Capture.PNG

       

  • I experience exactly the same problem. When I use hardcoded path (local or OneDrive for Business), it allows me to configure refresh. However, when I use function which returns me file according to path specified dynamically, the refresh cannot be configured.

     

    Function:

    let GetSheet = (path as text, SourceType as text) =>
    let
    Source = if SourceType = "web" then Excel.Workbook(Web.Contents(path)) else Excel.Workbook(File.Contents(path)),
  • Yep, same issue here.  source wrapped in a function appears not to be refreshable from the portal.

    • curth's avatar
      curth
      Power BI Team

      This is currently correct; queries where data access happens inside a function and where the data source is dependent on parameters to the function can't currently be refreshed. That's because we're doing static analysis of the query to discover the data source, and our static analysis can't yet handle this scenario.

      • bloigge's avatar
        bloigge
        Advocate IV

        any timetable on that? depending on the size of my reports without functions code gets a mess really fast... 

  • nakia's avatar
    nakia
    Frequent Visitor

    Hi,

     

    Seems I am running into the same hurdle.

     

    Any advice on how I could modify my query?

    let
        // Calculate the Start of the Month, Today then format the date parameter string as YYYYMMDD.
        MonthStart = Date.StartOfMonth(DateTime.LocalNow()),
        MonthStart_YYYYMMDD = Date.ToText(#date(Date.Year(MonthStart), Date.Month(MonthStart), Date.Day(MonthStart)), "yyyyMMdd"),  
        Today_YYYYMMDD = Date.ToText(#date(Date.Year(DateTime.LocalNow()), Date.Month(DateTime.LocalNow()), Date.Day(DateTime.LocalNow())),"yyyyMMdd"),
        // Store a list of the User ID's from Harvest (Only include the ID's of the people you need visalised on this data set)
        User_IDS = {"1219708","294662","337056"},
        // Intiate the query
        Data = List.Generate(
        ()=> [Counter = 0],
        each [Counter]<List.Count(User_IDS),
        each [Counter=[Counter] + 1],
        each Json.Document(Binary.Buffer(Web.Contents("https://equipmentmanagementsolutions.harvestapp.com/people/" & Text.From(User_IDS{[Counter]}) & "/entries?from=" & MonthStart_YYYYMMDD & "&to=" & Today_YYYYMMDD  ,[Headers=[#"Authorization" = Harvest_Credentials,#"Content-Type"="application/json",#"Accept"="application/json"]])))
        ),
        #"Converted to Table" = Table.FromList(Data, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
        #"Expanded Column1" = Table.ExpandListColumn(#"Converted to Table", "Column1"),
        #"Expanded Column2" = Table.ExpandRecordColumn(#"Expanded Column1", "Column1", {"day_entry"}, {"Column1.day_entry"}),
        #"Expanded Column1.day_entry" = Table.ExpandRecordColumn(#"Expanded Column2", "Column1.day_entry", {"id", "notes", "spent_at", "hours", "user_id", "project_id", "task_id", "created_at", "updated_at", "adjustment_record", "timer_started_at", "is_closed", "is_billed", "started-at", "ended-at"}, {"Column1.day_entry.id", "Column1.day_entry.notes", "Column1.day_entry.spent_at", "Column1.day_entry.hours", "Column1.day_entry.user_id", "Column1.day_entry.project_id", "Column1.day_entry.task_id", "Column1.day_entry.created_at", "Column1.day_entry.updated_at", "Column1.day_entry.adjustment_record", "Column1.day_entry.timer_started_at", "Column1.day_entry.is_closed", "Column1.day_entry.is_billed", "Column1.day_entry.started-at", "Column1.day_entry.ended-at"}),
        #"Renamed Columns" = Table.RenameColumns(#"Expanded Column1.day_entry",{{"Column1.day_entry.id", "Entry.ID"}, {"Column1.day_entry.notes", "Entry.Notes"}, {"Column1.day_entry.spent_at", "Entry.Date"}, {"Column1.day_entry.hours", "Hours.OnTask"}, {"Column1.day_entry.user_id", "User.ID"}, {"Column1.day_entry.project_id", "Project.ID"}, {"Column1.day_entry.task_id", "Task.ID"}, {"Column1.day_entry.created_at", "Entry.Created.DateTime"}, {"Column1.day_entry.updated_at", "Entry.Updated.DateTime"}, {"Column1.day_entry.started-at", "Entry.StartTime"}, {"Column1.day_entry.ended-at", "Entry.EndTime"}}),
        #"Removed Columns" = Table.RemoveColumns(#"Renamed Columns",{"Column1.day_entry.timer_started_at"}),
        #"Renamed Columns1" = Table.RenameColumns(#"Removed Columns",{{"Column1.day_entry.is_billed", "Is.Billable"}, {"Column1.day_entry.is_closed", "Entry.IsClosed"}}),
        #"Removed Columns1" = Table.RemoveColumns(#"Renamed Columns1",{"Column1.day_entry.adjustment_record"}),
        #"Merged Queries" = Table.NestedJoin(#"Removed Columns1",{"User.ID"},Harvest_Users,{"Column1.user.id"},"NewColumn",JoinKind.LeftOuter),
        #"Expanded NewColumn" = Table.ExpandTableColumn(#"Merged Queries", "NewColumn", {"Column1.user.first_name", "Column1.user.last_name", "Column1.user.department"}, {"NewColumn.Column1.user.first_name", "NewColumn.Column1.user.last_name", "NewColumn.Column1.user.department"}),
        #"Merged Queries1" = Table.NestedJoin(#"Expanded NewColumn",{"Project.ID"},Harvest_Projects,{"Column1.project.id"},"NewColumn",JoinKind.LeftOuter),
        #"Expanded NewColumn1" = Table.ExpandTableColumn(#"Merged Queries1", "NewColumn", {"Column1.project.client_id", "Column1.project.name", "Column1.project.billable"}, {"NewColumn.Column1.project.client_id", "NewColumn.Column1.project.name", "NewColumn.Column1.project.billable"}),
        #"Merged Queries2" = Table.NestedJoin(#"Expanded NewColumn1",{"NewColumn.Column1.project.client_id"},Harvest_Clients,{"Column1.client.id"},"NewColumn",JoinKind.LeftOuter),
        #"Expanded NewColumn2" = Table.ExpandTableColumn(#"Merged Queries2", "NewColumn", {"Column1.client.name"}, {"NewColumn.Column1.client.name"}),
        #"Merged Queries3" = Table.NestedJoin(#"Expanded NewColumn2",{"Task.ID"},Harvest_Tasks,{"Column1.task.id"},"NewColumn",JoinKind.LeftOuter),
        #"Expanded NewColumn3" = Table.ExpandTableColumn(#"Merged Queries3", "NewColumn", {"Column1.task.name", "Column1.task.billable_by_default"}, {"NewColumn.Column1.task.name", "NewColumn.Column1.task.billable_by_default"}),
        #"Removed Columns2" = Table.RemoveColumns(#"Expanded NewColumn3",{"NewColumn.Column1.project.billable", "Is.Billable", "Entry.IsClosed"})
    in
        #"Removed Columns2"

     

     

  • nakia's avatar
    nakia
    Frequent Visitor

    Anyone got any suggestions for how to get around this for web queries. My code is above.

    This is really going to limit the use I can get out of Power BI if it can't do this!