Forum Discussion

Evogelpohl's avatar
Evogelpohl
Helper V
11 years ago
Solved

Azure AD Graph Refresh

Created a PowerBI Desktop solution that gets data from Azure AD via the Graph API.  Worked flawlessly on the desktop.

 

Here's the first line of the query:

Source = Json.Document(Web.Contents("https://graph.windows.net/domain.com/users?api-version=1.5")

Posted to PowerBI.com - won't refresh.  Says source is not supported.

 

What's wrong with a simple REST query?  Why doesn't PowerBI support this call?  Doing something wrong?

 

-Eric.

  • curth's avatar
    curth
    11 years ago

    Yeah :(.

     

    When you run this inside Power Query or PBI Desktop, you either have to enable "Fast Combine" or you need to set privacy levels for the two sources. Currently, all sources inside refresh are marked as "Private" with no ability to change this configuration or set "Fast Combine". This prevents the query from working correctly. This was picked as a default to maximize security without asking the user hard-to-explain questions, but it's obviously quite limiting in the kinds of queries that can be performed -- so we need to figure out how best to enable this scenario for refresh.

  • curth's avatar
    curth
    11 years ago

    This is probably my fault for pasting colorized text into the editor. %E2%80%8B is the UTF-8 for an unbreakable space. Can you just retype the line instead of copying and pasting it?

15 Replies

  • Thanks Curt. I finally got back to this today. I retyped your example and got it to work. I posted to PowerBI.com and it is successfully refreshing. Thanks again.
    • Anonymous's avatar
      Anonymous
      Not applicable

      Hi Evogelpohl 

       

      Do you have the URL for your post in Powerbi_com ?

      That would be handy 

      Many thanks

       

      Jude

  • curth's avatar
    curth
    Power BI Team

    While I think this isn't officially supported, I know of no reason why it wouldn't work. What's the rest of the query doing?

    • Evogelpohl's avatar
      Evogelpohl
      Helper V

      Understand the list of supported item, but - for some reason - i thought generic WEB source calls worked.

       

      Here's what I'm doing:

       

      1) Get the SaleForce.Com Object = User Profile (a list of all our SF.com users)

       

      2) Using the 1.5 version of the Azure AD Graph API - Connect to my company's AD-Directory (I want to get metadata for each user)

       

      3) Using the Add Custom Column ability of Power Query - For Each SalesForce.com user's email address, lookup in AD Graph their Company, Dept & Location (Using this: Add Custom Column, 'try FnADUserLookup([Email])'

       

      This BTW works flawlessly in Excel & the Designer.  Just won't refresh on PowerBI.com :mansad:

       

      Here's the M-code of the Function i wrote. 

      let
          FNLookupAD = (EmailAddress as text) =>
      
      let
          Source = Json.Document(Web.Contents("https://graph.windows.net/<mydomain>.com/users?api-version=1.5&$filter=proxyAddresses/any(c:c eq 'smtp:" & EmailAddress & "')")),
          value = Source[value],
          #"Converted to Table" = Table.FromList(value, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
          #"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"odata.type", "objectType", "objectId", "deletionTimestamp", "accountEnabled", "assignedLicenses", "assignedPlans", "city", "companyName", "country", "creationType", "department", "dirSyncEnabled", "displayName", "facsimileTelephoneNumber", "givenName", "immutableId", "jobTitle", "lastDirSyncTime", "mail", "mailNickname", "mobile", "onPremisesSecurityIdentifier", "otherMails", "passwordPolicies", "passwordProfile", "physicalDeliveryOfficeName", "postalCode", "preferredLanguage", "provisionedPlans", "provisioningErrors", "proxyAddresses", "sipProxyAddress", "state", "streetAddress", "surname", "telephoneNumber", "[email protected]", "usageLocation", "userPrincipalName", "userType"}, {"Column1.odata.type", "Column1.objectType", "Column1.objectId", "Column1.deletionTimestamp", "Column1.accountEnabled", "Column1.assignedLicenses", "Column1.assignedPlans", "Column1.city", "Column1.companyName", "Column1.country", "Column1.creationType", "Column1.department", "Column1.dirSyncEnabled", "Column1.displayName", "Column1.facsimileTelephoneNumber", "Column1.givenName", "Column1.immutableId", "Column1.jobTitle", "Column1.lastDirSyncTime", "Column1.mail", "Column1.mailNickname", "Column1.mobile", "Column1.onPremisesSecurityIdentifier", "Column1.otherMails", "Column1.passwordPolicies", "Column1.passwordProfile", "Column1.physicalDeliveryOfficeName", "Column1.postalCode", "Column1.preferredLanguage", "Column1.provisionedPlans", "Column1.provisioningErrors", "Column1.proxyAddresses", "Column1.sipProxyAddress", "Column1.state", "Column1.streetAddress", "Column1.surname", "Column1.telephoneNumber", "[email protected]", "Column1.usageLocation", "Column1.userPrincipalName", "Column1.userType"}),
          #"Removed Columns" = Table.RemoveColumns(#"Expanded Column1",{"Column1.odata.type", "Column1.objectType", "Column1.objectId", "Column1.deletionTimestamp"}),
          #"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Column1.accountEnabled", "Account Enabled"}}),
          #"Removed Columns1" = Table.RemoveColumns(#"Renamed Columns",{"Column1.assignedLicenses", "Column1.assignedPlans"}),
          #"Renamed Columns1" = Table.RenameColumns(#"Removed Columns1",{{"Column1.city", "City"}, {"Column1.companyName", "OA-FA"}, {"Column1.country", "Country"}}),
          #"Removed Columns2" = Table.RemoveColumns(#"Renamed Columns1",{"Column1.creationType"}),
          #"Renamed Columns2" = Table.RenameColumns(#"Removed Columns2",{{"Column1.department", "Department"}}),
          #"Removed Columns3" = Table.RemoveColumns(#"Renamed Columns2",{"Column1.dirSyncEnabled"}),
          #"Renamed Columns3" = Table.RenameColumns(#"Removed Columns3",{{"Column1.displayName", "Name"}}),
          #"Removed Columns4" = Table.RemoveColumns(#"Renamed Columns3",{"Column1.facsimileTelephoneNumber", "Column1.givenName", "Column1.immutableId"}),
          #"Renamed Columns4" = Table.RenameColumns(#"Removed Columns4",{{"Column1.jobTitle", "Title"}}),
          #"Removed Columns5" = Table.RemoveColumns(#"Renamed Columns4",{"Column1.lastDirSyncTime"}),
          #"Renamed Columns5" = Table.RenameColumns(#"Removed Columns5",{{"Column1.mail", "Email"}}),
          #"Removed Columns6" = Table.RemoveColumns(#"Renamed Columns5",{"Column1.mailNickname", "Column1.mobile", "Column1.onPremisesSecurityIdentifier"}),
          #"Expanded Column1.otherMails" = Table.ExpandListColumn(#"Removed Columns6", "Column1.otherMails"),
          #"Removed Columns7" = Table.RemoveColumns(#"Expanded Column1.otherMails",{"Column1.otherMails", "Column1.passwordPolicies", "Column1.passwordProfile", "Column1.physicalDeliveryOfficeName"}),
          #"Renamed Columns6" = Table.RenameColumns(#"Removed Columns7",{{"Column1.postalCode", "Postal Code"}}),
          #"Removed Columns8" = Table.RemoveColumns(#"Renamed Columns6",{"Column1.preferredLanguage", "Column1.provisionedPlans", "Column1.provisioningErrors"}),
          #"Expanded Column1.proxyAddresses" = Table.ExpandListColumn(#"Removed Columns8", "Column1.proxyAddresses"),
          #"Removed Columns9" = Table.RemoveColumns(#"Expanded Column1.proxyAddresses",{"Column1.sipProxyAddress"}),
          #"Renamed Columns7" = Table.RenameColumns(#"Removed Columns9",{{"Column1.state", "State"}, {"Column1.streetAddress", "Street Address"}}),
          #"Removed Columns10" = Table.RemoveColumns(#"Renamed Columns7",{"Column1.surname", "Column1.telephoneNumber", "[email protected]", "Column1.usageLocation", "Column1.userPrincipalName", "Column1.userType"}),
          #"Removed Duplicates" = Table.Distinct(#"Removed Columns10", {"Name"})
      in
          #"Removed Duplicates"
      in
          FNLookupAD

       

       
      • curth's avatar
        curth
        Power BI Team

        Yeah :(.

         

        When you run this inside Power Query or PBI Desktop, you either have to enable "Fast Combine" or you need to set privacy levels for the two sources. Currently, all sources inside refresh are marked as "Private" with no ability to change this configuration or set "Fast Combine". This prevents the query from working correctly. This was picked as a default to maximize security without asking the user hard-to-explain questions, but it's obviously quite limiting in the kinds of queries that can be performed -- so we need to figure out how best to enable this scenario for refresh.

  • Good afternoon!!!
    I saw many websites, chats and posts, but few are objective and each person values his method as if it were unique !!!
    Each has its logic, so there is no single method !!
    Objectively, for me ...
    It worked this way:

    Enjoy: ğŸ˜˜

    ===========================
    let
    tenantId = "12345678-9012-3456-7890-123456789012",
    clientId = "abcdefgh-9012-3456-7890-123456789012",
    clientSecret = "1234567890qwertyuiopasdf",
    grantType = "client_credentials",
    resource = "https://graph.windows.net/",
    endpointUsers = "/users",
    apiVersion = "?api-version=1.6",

    baseURL="https://login.microsoftonline.com",
    relativePath = "/"&tenantId&"/oauth2/token",
    urlToken = baseURL & relativePath,

    body = [client_id=clientId,
    grant_type=grantType,
    client_secret=clientSecret,
    resource=resource],

    //Access
    Source = Json.Document(Web.Contents(urlToken, [Headers=[#"Accept" = "application/json", #"Content-Type"="application/x-www-form-urlencoded;charset=UTF-8"],Content = Text.ToBinary(Uri.BuildQueryString(body))])),
    accessToken = Source[access_token],
    tokenType = Source[token_type],
    fullAccessToken = tokenType&" "&accessToken,

    //Get Users
    SourceUsers = OData.Feed(resource & tenantId & endpointUsers, [Authorization=fullAccessToken], [Implementation="2.0", Query=[#"api-version"="1.6"]]),

    Step1 = Table.RemoveColumns(SourceUsers,{"createdObjects", "department", "employeeId", "facsimileTelephoneNumber", "legalAgeGroupClassification", "jobTitle", "telephoneNumber", "mobile", "givenName", "physicalDeliveryOfficeName", "consentProvidedForMinor", "sipProxyAddress", "streetAddress", "city", "state", "country", "postalCode", "ageGroup", "companyName", "preferredLanguage", "manager", "directReports", "members", "transitiveMembers", "owners", "ownedObjects", "appRoleAssignments", "licenseDetails", "oauth2PermissionGrants", "ownedDevices", "registeredDevices", "assignedLicenses", "assignedPlans", "deletionTimestamp", "immutableId", "isCompromised", "lastDirSyncTime", "passwordProfile", "provisionedPlans", "provisioningErrors", "proxyAddresses", "signInNames", "usageLocation", "userIdentities", "memberOf", "transitiveMemberOf", "thumbnailPhoto", "createdOnBehalfOf", "dirSyncEnabled", "onPremisesDistinguishedName", "onPremisesSecurityIdentifier", "showInAddressList"}),
    Step2 = Table.TransformColumnTypes(Step1,{{"createdDateTime", type datetime},{"refreshTokensValidFromDateTime", type datetime}}),
    Step3 = Table.TransformColumns(Step2, {"otherMails", each Text.Combine(List.Transform(_, Text.From)), type text})
    in
    Step3