Forum Discussion

jaime_blackwell's avatar
5 years ago

Building Custom Connector for Xero API

Hi Community,

I am aiming to get data from the Xero API into Power BI for my Organisation. To this end I need to create a custom connector as PBI doesn't have a certified connector for Xero (the exisiting apps are too limited for my use case). I have some experience in Power BI but this is my first attempt at building a custom connector. The connector is written in M language.

As the Xero API requires OAuth2 authentication this was the first hurdle. I used the below resource to create a custom connector in Visual Studio using the Power Query SDK template. I believe I have set up authentication successfully as the connector allows me to log into my Xero account and 'allow access' to the requested resource. I know the connection is successful because in Xero my app appears in Settings>Connected Apps.

https://github.com/jussiroine/OuraCloudConnector/blob/master/OuraCloudConnector/OuraCloudConnector.pq

However. Once authenticated I am receiving an error; Details: "Access to the resource is forbidden."

Possible Causes I can think of:
- I'm using The Code Flow not PKCE Flow (I assumed this was correct as I'm connecting from Power BI Desktop)
- Error in my code (I'm inexperienced in writing M language)
- The API url I'm sending is incomplete or incorrect (I'm currently trying: https://api.xero.com/api.xro/2.0/Reports/AgedReceivablesByContact?fromDate=2020-01-01&toDate=2021-01-01 )

I have checked that my user permissions are high enough to access the data I'm requesting. I have also checked that the scope includes the required data. Can anyone advise if they've had success building a custom connector for Xero in M or offer some advice to resolve this error?

Thank you!

// This file contains your Data Connector logic
section JaimesXeroConnector;

//JaimesXeroConnector OAuth2 values;

client_id = Text.FromBinary(Extension.Contents("client_id.txt"));
client_secret = Text.FromBinary(Extension.Contents("client_secret.txt"));
redirect_uri = "https://oauth.powerbi.com/views/oauthredirect.html";
token_uri = "https://identity.xero.com/connect/token";
authorize_uri = "https://login.xero.com/identity/connect/authorize";
logout_uri = "https://login.microsoftonline.com/logout.srf";
scopes = {"offline_access openid profile email accounting.transactions accounting.reports.read accounting.transactions.read accounting.reports.read accounting.journals.read accounting.settings accounting.settings.read accounting.contacts accounting.contacts.read accounting.attachments accounting.attachments.read assets projects"
};
System.Net.ServicePointManager.SecurityProtocol= "SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12";

// Login modal window dimensions
windowWidth = 720;
windowHeight = 1024;

[DataSource.Kind="JaimesXeroConnector", Publish="JaimesXeroConnector.Publish"]
shared JaimesXeroConnector.Contents = (url as text) =>
    let
        source = Json.Document(Web.Contents(url))
    in
        source;


// Data Source Kind description
JaimesXeroConnector= [
    TestConnection = (dataSourcePath) => { "JaimesXeroConnector.Contents", dataSourcePath },
    Authentication = [
        OAuth = [
            StartLogin=StartLogin,
            FinishLogin=FinishLogin,
            Refresh=Refresh,
            Logout=Logout
        ]
    ],
    Label = Extension.LoadString("DataSourceLabel")
];

// Data Source UI publishing description
JaimesXeroConnector.Publish = [
    Beta = true,
    Category = "Other",
    ButtonText = { Extension.LoadString("ButtonTitle"), Extension.LoadString("ButtonHelp") },
    LearnMoreUrl = "https://powerbi.microsoft.com/",
    SourceImage = JaimesXeroConnector.Icons,
    SourceTypeImage = JaimesXeroConnector.Icons
];

// Helper functions for OAuth2: StartLogin, FinishLogin, Refresh, Logout
StartLogin = (resourceUrl, state, display) =>
    let
        authorizeUrl = authorize_uri & "?" & Uri.BuildQueryString([
            response_type = "code",
            client_id = client_id,  
            redirect_uri = redirect_uri,
            state = state,
            scope = scopes
        ])
    in
        [
            LoginUri = authorizeUrl,
            CallbackUri = redirect_uri,
            WindowHeight = 720,
            WindowWidth = 1024,
            Context = null
        ];

FinishLogin = (context, callbackUri, state) =>
    let
        // parse the full callbackUri, and extract the Query string
        parts = Uri.Parts(callbackUri)[Query],
        // if the query string contains an "error" field, raise an error
        // otherwise call TokenMethod to exchange our code for an access_token
        result = if (Record.HasFields(parts, {"error", "error_description"})) then 
                    error Error.Record(parts[error], parts[error_description], parts)
                 else
                    TokenMethod("authorization_code", "code", parts[code])
    in
        result;

Refresh = (resourceUrl, refresh_token) => TokenMethod("refresh_token", "refresh_token", refresh_token);

Logout = (token) => logout_uri;

// see "Exchange code for access token: POST /oauth/token" for details
TokenMethod = (grantType, tokenField, code) =>
    let
        queryString = [
            grant_type = "authorization_code",
            redirect_uri = redirect_uri,
            client_id = client_id,
            client_secret = client_secret
        ],
        queryWithCode = Record.AddField(queryString, tokenField, code),

        tokenResponse = Web.Contents(token_uri, [
            Content = Text.ToBinary(Uri.BuildQueryString(queryWithCode)),
            Headers = [
                #"Content-type" = "application/x-www-form-urlencoded",
                #"Accept" = "application/json"
            ],
            ManualStatusHandling = {400} 
        ]),
        body = Json.Document(tokenResponse),
        result = if (Record.HasFields(body, {"error", "error_description"})) then 
                    error Error.Record(body[error], body[error_description], body)
                 else
                    body
    in
        result;

Value.IfNull = (a, b) => if a <> null then a else b;

GetScopeString = (scopes as list, optional scopePrefix as text) as text =>
    let
        prefix = Value.IfNull(scopePrefix, ""),
        addPrefix = List.Transform(scopes, each prefix & _),
        asText = Text.Combine(addPrefix, " ")
    in
        asText;
        

JaimesXeroConnector.Icons = [
    Icon16 = { Extension.Contents("JaimesXeroConnector16.png"), Extension.Contents("JaimesXeroConnector20.png"), Extension.Contents("JaimesXeroConnector24.png"), Extension.Contents("JaimesXeroConnector32.png") },
    Icon32 = { Extension.Contents("JaimesXeroConnector32.png"), Extension.Contents("JaimesXeroConnector40.png"), Extension.Contents("JaimesXeroConnector48.png"), Extension.Contents("JaimesXeroConnector64.png") }
];

8 Replies

  • CalebGuthrie's avatar
    CalebGuthrie
    Frequent Visitor

    jaime_blackwell did you ever get this up and running? I've followed your posts across Google and stumbled on your code in github. My company is needing to build reporting off of our Xero account which is going to require setting up a custom connector to handle the OAuth2 protocol.

    I've replicated your code in github but getting an error after login "The field 'access_token' of the record wasn't found". 

    Did you ever find a fix to this? I feel like I'm so close!!! (thanks to you sharing your work)


    Code below for reference:

    // This file contains your Data Connector logic
    section EleganceXeroConnector;

    //EleganceXeroConnector OAuth2 values;

    client_id = "xxxxxxxxxxxxxxxxx";
    client_secret = "xxxxxxxxxxxxxxxxx";
    redirect_uri = "https://oauth.powerbi.com/views/oauthredirect.html";
    token_uri = "https://identity.xero.com/connect/token?=";
    authorize_uri = "https://login.xero.com/identity/connect/authorize";
    logout_uri = "https://login.microsoftonline.com/logout.srf";
    connection_uri = "https://api.xero.com/connections";
    tenantid = Text.From(GetTenantId(connection_uri));

    // Login modal window dimensions
    windowWidth = 720;
    windowHeight = 1024;

    //OAuth2 Scopes
    scope_prefix = "";
    scopes = {
    "offline_access",
    "openid",
    "profile",
    "email",
    "accounting.transactions.read",
    "accounting.reports.read",
    "accounting.journals.read",
    "accounting.contacts.read",
    "assets.read"
    };

    GetTenantId = (url as text) =>
    let
    Source = Web.Contents(url),
    ImportedJSON = Json.Document(Source,1252),
    ConvertedtoTable = Table.FromList(ImportedJSON, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    ExpandedColumn1 = Table.ExpandRecordColumn(ConvertedtoTable, "Column1", {"id", "authEventId", "tenantId", "tenantType", "tenantName", "createdDateUtc", "updatedDateUtc"}, {"id", "authEventId", "tenantId", "tenantType", "tenantName", "createdDateUtc", "updatedDateUtc"}),
    ChangedType = Table.TransformColumnTypes(ExpandedColumn1,{{"id", type text}, {"authEventId", type text}, {"tenantId", type text}, {"tenantType", type text}, {"tenantName", type text}, {"createdDateUtc", type datetime}, {"updatedDateUtc", type datetime}}),
    tenantId = Text.From(ChangedType{0}[tenantId])
    in
    tenantId;

    [DataSource.Kind="EleganceXeroConnector", Publish="EleganceXeroConnector.Publish"]
    shared EleganceXeroConnector.Contents = (url as text) =>
    let
    source = Json.Document(Web.Contents(url,[Headers = [#"xero-tenant-id"=tenantid,#"Accept" = "application/json"]]))
    in
    source;


    // Data Source Kind description
    EleganceXeroConnector= [
    TestConnection = (DataSourcePath) =>
    let
    json = Json.Document(DataSourcePath),
    server = json[server],
    database = json[database]
    in
    try
    { "EleganceXeroConnector.Contents", server, database }
    otherwise
    let
    message = Text.Format("Couldn't find entity.")
    in
    Diagnostics.Trace(TraceLevel.Error, message, () => error message, true )
    ,

    Authentication = [
    OAuth = [
    StartLogin=StartLogin,
    FinishLogin=FinishLogin,
    Refresh=Refresh,
    Logout=Logout
    ]
    ],
    Label = Extension.LoadString("DataSourceLabel")
    ];

    // Data Source UI publishing description
    EleganceXeroConnector.Publish = [
    Beta = true,
    Category = "Other",
    ButtonText = { Extension.LoadString("ButtonTitle"), Extension.LoadString("ButtonHelp") },
    LearnMoreUrl = "https://powerbi.microsoft.com/",
    SourceImage = EleganceXeroConnector.Icons,
    SourceTypeImage = EleganceXeroConnector.Icons
    ];

    // Helper functions for OAuth2: StartLogin, FinishLogin, Refresh, Logout
    StartLogin = (resourceUrl, state, display) =>
    let
    authorizeUrl = authorize_uri & "?" & Uri.BuildQueryString([
    response_type = "code",
    client_id = client_id,
    redirect_uri = redirect_uri,
    state = state,
    scope = GetScopeString(scopes, scope_prefix)
    ])
    in
    [
    LoginUri = authorizeUrl,
    CallbackUri = redirect_uri,
    WindowHeight = 720,
    WindowWidth = 1024,
    Context = null
    ];

    FinishLogin = (context, callbackUri, state) =>
    let
    // parse the full callbackUri, and extract the Query string
    parts = Uri.Parts(callbackUri)[Query],
    // if the query string contains an "error" field, raise an error
    // otherwise call TokenMethod to exchange our code for an access_token
    result = if (Record.HasFields(parts, {"error", "error_description"})) then
    error Error.Record(parts[error], parts[error_description], parts)
    else
    TokenMethod("authorization_code", "code", parts[code])
    in
    result;

    Refresh = (resourceUrl, refresh_token) => TokenMethod("refresh_token", "refresh_token", refresh_token);

     

    Logout = (token) => logout_uri;



    // see "Exchange code for access token: POST /oauth/token" for details
    TokenMethod = (grantType, tokenField, code) =>
    let
    queryString = [
    grant_type = "authorization_code",
    redirect_uri = redirect_uri,
    client_id = client_id,
    client_secret = client_secret
    ],
    queryWithCode = Record.AddField(queryString, tokenField, code),

    tokenResponse = Web.Contents(token_uri, [
    Content = Text.ToBinary(Uri.BuildQueryString(queryWithCode)),
    Headers = [
    #"authorization" = Text.Combine({"Basic ", "base64encode(", client_id, ":", client_secret, ")"} ),
    // #"Xero-tenant-id" = "xxxxxxxxxxxxxxxxx",
    #"Content-type" = "application/x-www-form-urlencoded",
    #"Accept" = "application/json"
    ],
    ManualStatusHandling = {400}
    ]),
    body = Json.Document(tokenResponse),
    result = if (Record.HasFields(body, {"error", "error_description"})) then
    error Error.Record(body[error], body[error_description], body)
    else
    body
    in
    result;

    Value.IfNull = (a, b) => if a <> null then a else b;

    GetScopeString = (scopes as list, optional scopePrefix as text) as text =>
    let
    prefix = Value.IfNull(scopePrefix, ""),
    addPrefix = List.Transform(scopes, each prefix & _),
    asText = Text.Combine(addPrefix, " ")
    in
    asText;


    EleganceXeroConnector.Icons = [
    Icon16 = { Extension.Contents("EleganceXeroConnector16.png"), Extension.Contents("EleganceXeroConnector20.png"), Extension.Contents("EleganceXeroConnector24.png"), Extension.Contents("EleganceXeroConnector32.png") },
    Icon32 = { Extension.Contents("EleganceXeroConnector32.png"), Extension.Contents("EleganceXeroConnector40.png"), Extension.Contents("EleganceXeroConnector48.png"), Extension.Contents("EleganceXeroConnector64.png") }
    ];

    • jaime_blackwell's avatar
      jaime_blackwell
      Helper I

      Hi CalebGuthrie 
      I did manage to get the connector working, I think the below is the updated code. I'll also update in Github shortly. 
      Since then, I switched to using OData feeds for Xero data as my connector could only retrieve one payslip at a time. The provider of these feeds is ODataLink. https://odatalink.com/

      // This file contains your Data Connector logic
      section JaimesXeroConnector;
      
      //JaimesXeroConnector OAuth2 values;
      client_id = "3";
      client_secret = "v";
      redirect_uri = "https://oauth.powerbi.com/views/oauthredirect.html";
      token_uri = "https://identity.xero.com/connect/token?=";
      authorize_uri = "https://login.xero.com/identity/connect/authorize";
      logout_uri = "https://login.microsoftonline.com/logout.srf";
      connection_uri = "https://api.xero.com/connections"; //Jaime added 6.1.21
      tenantid = Text.From(GetTenantId(connection_uri));
      
      // Login modal window dimensions
      windowWidth = 720;
      windowHeight = 1024;
      
      //OAuth2 Scopes
      scope_prefix = "";
      scopes = {
      "offline_access",
      "openid",
      "profile",
      "email",
      "accounting.transactions.read",
      "accounting.settings.read",
      "accounting.reports.read",
      "accounting.journals.read",
      "accounting.contacts.read",
      "assets.read",
      "payroll.employees.read",
      "payroll.payruns.read",
      "payroll.payslip.read",
      "payroll.settings.read",
      "payroll.timesheets.read"
      };
      
      
      GetTenantId = (url as text) =>
      let
      Source = Web.Contents(url),
      ImportedJSON = Json.Document(Source,1252),
      ConvertedtoTable = Table.FromList(ImportedJSON, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
      ExpandedColumn1 = Table.ExpandRecordColumn(ConvertedtoTable, "Column1", {"id", "authEventId", "tenantId", "tenantType", "tenantName", "createdDateUtc", "updatedDateUtc"}, {"id", "authEventId", "tenantId", "tenantType", "tenantName", "createdDateUtc", "updatedDateUtc"}),
      ChangedType = Table.TransformColumnTypes(ExpandedColumn1,{{"id", type text}, {"authEventId", type text}, {"tenantId", type text}, {"tenantType", type text}, {"tenantName", type text}, {"createdDateUtc", type datetime}, {"updatedDateUtc", type datetime}}),
      tenantId = Text.From(ChangedType{0}[tenantId])
      in
      tenantId;
      
      [DataSource.Kind="JaimesXeroConnector", Publish="JaimesXeroConnector.Publish"] 
      shared JaimesXeroConnector.Contents = (url as text) => 
          let 
              source = Json.Document(Web.Contents(url,[Headers = [#"xero-tenant-id"=tenantid,#"Accept" = "application/json"]])) 
          in 
              source;
      
      // Data Source Kind description
      JaimesXeroConnector= [
          TestConnection = (DataSourcePath) => 
              let
                  json = Json.Document(DataSourcePath),
                  server = json[server],
                  database = json[database]
              in   
                  try
                      { "JaimesXeroConnector.Contents", server, database }
                  otherwise
                      let
                          message = Text.Format("Couldn't find entity.")
                      in 
                          Diagnostics.Trace(TraceLevel.Error,  message, () => error message, true )
                  ,
      
          Authentication = [
              OAuth = [
                  StartLogin=StartLogin,
                  FinishLogin=FinishLogin,
                  Refresh=Refresh,
                  Logout=Logout
              ]
          ],
          Label = Extension.LoadString("DataSourceLabel")
      ];
      
      // Data Source UI publishing description
      JaimesXeroConnector.Publish = [
          Beta = true,
          Category = "Other",
          ButtonText = { Extension.LoadString("ButtonTitle"), Extension.LoadString("ButtonHelp") },
          LearnMoreUrl = "https://powerbi.microsoft.com/",
          SourceImage = JaimesXeroConnector.Icons,
          SourceTypeImage = JaimesXeroConnector.Icons
      ];
      
      // Helper functions for OAuth2: StartLogin, FinishLogin, Refresh, Logout
      StartLogin = (resourceUrl, state, display) =>
          let
              authorizeUrl = authorize_uri & "?" & Uri.BuildQueryString([
                  response_type = "code",
                  client_id = client_id,  
                  redirect_uri = redirect_uri,
                  state = state,
                  scope = GetScopeString(scopes, scope_prefix)
              ])
          in
              [
                  LoginUri = authorizeUrl,
                  CallbackUri = redirect_uri,
                  WindowHeight = 720,
                  WindowWidth = 1024,
                  Context = null
              ];
      
      FinishLogin = (context, callbackUri, state) =>
          let
              // parse the full callbackUri, and extract the Query string
              parts = Uri.Parts(callbackUri)[Query],
              // if the query string contains an "error" field, raise an error
              // otherwise call TokenMethod to exchange our code for an access_token
              result = if (Record.HasFields(parts, {"error", "error_description"})) then 
                          error Error.Record(parts[error], parts[error_description], parts)
                       else
                          TokenMethod("authorization_code", parts[code])
          in
              result;
      
      Refresh = (resourceUrl, refresh_token) => TokenMethod("refresh_token", "refresh_token", refresh_token);
      
             
      
      Logout = (token) => logout_uri;
      
      
              
      // see "Exchange code for access token: POST /oauth/token" for details
      TokenMethod = (grantType, code) =>
      //TokenMethod = (grantType, tokenField, code) =>
          let
              query = [
      //        queryString = [
                  grant_type = grantType,
      //            grant_type = "authorization_code",
                  redirect_uri = redirect_uri,
                  client_id = client_id,
                  client_secret = client_secret
              ],
              queryWithCode = if(grantType = "refresh_token") then [refresh_token = code] else [code = code],
      //        queryWithCode = Record.AddField(queryString, tokenField, code),
      
              tokenResponse = Web.Contents(token_uri, [
                  Content = Text.ToBinary(Uri.BuildQueryString(query & queryWithCode)),
      //            Content = Text.ToBinary(Uri.BuildQueryString(queryWithCode)),
      //            Headers = [
      //                #"authorization" = Text.Combine({"Basic ", "base64encode(", client_id, ":", client_secret, ")"} ), // JB
      //                #"Content-type" = "application/x-www-form-urlencoded",
      //                #"Accept" = "application/json"
      //            ],
                  Headers= [#"Content-type" = "application/x-www-form-urlencoded",#"Accept" = "application/json"],
                  ManualStatusHandling = {400} 
              ]),
              body = Json.Document(tokenResponse),
                       result = if (Record.HasFields(body, {"error", "error_description"})) then 
                       error Error.Record(body[error], body[error_description], body) 
                               else
                          body
          in
              result;
      
      Value.IfNull = (a, b) => if a <> null then a else b;
      
      GetScopeString = (scopes as list, optional scopePrefix as text) as text =>
          let
              prefix = Value.IfNull(scopePrefix, ""),
              addPrefix = List.Transform(scopes, each prefix & _),
              asText = Text.Combine(addPrefix, " ")
          in
              asText;
              
              
      
      JaimesXeroConnector.Icons = [
          Icon16 = { Extension.Contents("JaimesXeroConnector16.png"), Extension.Contents("JaimesXeroConnector20.png"), Extension.Contents("JaimesXeroConnector24.png"), Extension.Contents("JaimesXeroConnector32.png") },
          Icon32 = { Extension.Contents("JaimesXeroConnector32.png"), Extension.Contents("JaimesXeroConnector40.png"), Extension.Contents("JaimesXeroConnector48.png"), Extension.Contents("JaimesXeroConnector64.png") }
      ];



      • Anonymous's avatar
        Anonymous
        Not applicable

        jaime_blackwell I tried your code but getting an error in the login

         

        can you help me out.

  • Looks good Jamie, would it be possible for you to talk me through how you got this to work? I'm rather new to PowerBI and so my knowledge doesn't extend this far, but I know that I want an API between Xero and PowerBI that would import all tables from Xero.