Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
Alina12
Frequent Visitor

How to connect Jira REST API as data source? Power BI Service friendly and with pagination??

Hi to all

I've spent hours and hours trying to load the Jira data into Power BI via its REST API. I have now managed to do it, but with the well-known problem of the error message of a dynamic data source in Power BI Service. Here my working code so far:

 

    BaseUrl = "https://XXX.XXX.net/jira/rest/api/2/search?jql=filter=525545&maxResults=1000&fields=issuetype,key,summary,customfield_10401,labels,customfield_10000,status,resolution,customfield_11200",
  
    JiraIDPerPage = 1000,

    GetJson = (Url) =>
        let 
            RawData = Web.Contents(Url,[Headers=[Authorization="Bearer XXX"]]),
            Json    = Json.Document(RawData)
        in  Json,

    GetJiraIDCount = () =>
        let Url   = BaseUrl & "&maxResults=0",
            Json  = GetJson(Url),
            Count = Json[#"total"]
        in  Count,

    GetPage = (Index) =>
        let Skip  = "&startAt=" & Text.From(Index * JiraIDPerPage),
            Top   = "&maxResults=" & Text.From(JiraIDPerPage),
            Url   = BaseUrl & Skip & Top,
            Json  = GetJson(Url),
            Value = Json[#"issues"]
        in  Value,

    JiraIDCount = List.Max({ JiraIDPerPage, GetJiraIDCount() }),

    PageCount   = Number.RoundUp(JiraIDCount / JiraIDPerPage),

    PageIndices = { 0 .. PageCount - 1 },

    Pages       = List.Transform(PageIndices, each GetPage(_)),

    JiraID    = List.Union(Pages),

    Table       = Table.FromList(JiraID, Splitter.SplitByNothing(), null, null, ExtraValues.Error),

...

 

 

Now that Power BI Desktop is working, I need to address the dynamic data source issue and I kept coming across this post while browsing the community:
https://blog.crossjoin.co.uk/2016/08/16/using-the-relativepath-and-query-options-with-web-contents-i... 
https://blog.crossjoin.co.uk/2016/08/23/web-contents-m-functions-and-dataset-refresh-errors-in-power... 


My modified code so far:

 

let 
    BaseUrl = 
    Web.Contents(
        "https://XXX.XXX.net/jira/rest/api",
        [
            RelativePath="2/search",
            Query=
            [
                jql="filter=525545",
                maxResults="1000",
                fields="issuetype,key,summary,customfield_10401,labels,customfield_10000,status,resolution,customfield_11200"
            ],
            Headers=
            [
                Authorization="Bearer XXX"
            ]
        ]
        
    )  ,

    JiraIDPerPage = 1000,

    GetJson = (Url) =>
        let 
            RawData = Web.Contents(Url,[Headers=[Authorization="Bearer XXX"]]),
            Json    = Json.Document(RawData)
        in  Json,

    GetJiraIDCount = () =>
        let Url   = BaseUrl & "&maxResults=1000",
            Json  = GetJson(Url),
            Count = Json[#"total"]
        in  Count,

    GetPage = (Index) =>
        let Skip  = "&startAt=" & Text.From(Index * JiraIDPerPage),
            Top   = "&maxResults=" & Text.From(JiraIDPerPage),
            Url   = BaseUrl & Skip & Top,
            Json  = GetJson(Url),
            Value = Json[#"issues"]
        in  Value,

    JiraIDCount = List.Max({ JiraIDPerPage, GetJiraIDCount() }),

    PageCount   = Number.RoundUp(JiraIDCount / JiraIDPerPage),

    PageIndices = { 0 .. PageCount - 1 },

    Pages       = List.Transform(PageIndices, each GetPage(_)),

    JiraID    = List.Union(Pages),

    Table       = Table.FromList(JiraID, Splitter.SplitByNothing(), null, null, ExtraValues.Error)
    
...

 


What is not described in the blog post, however, is how I can manage the pagination. Now I may have managed the query, but my current error message refers specifically to GetJiraIDCount=...

Expression.Error: We cannot apply operator & to types Binary and Text.
Details:
Operator=&
Left=[Binary]
Right=&maxResults=1000

I'm a complete beginner in this field and I'm getting really lost. Somehow it has to work???? I really would appreciate any help!!

1 ACCEPTED SOLUTION
v-huijiey-msft
Community Support
Community Support

Hi @Alina12 ,

 

Hope all is going well.

 

Please follow these steps:

 

1. The error you are encountering in the function is due to attempting to concatenate a binary () with a string. Since is used with , it is treated as a binary.

 

You should modify your approach to use and options within for pagination as well.GetJiraIDCountBaseUrlBaseUrlWeb.ContentsRelativePathQueryWeb.Contents

 

2. Modify the GetPage function:

GetPage = (Index) =>
    let
        Skip = Text.From(Index * JiraIDPerPage),
        Top = Text.From(JiraIDPerPage),
        PageData = Web.Contents(
            https://XXX.XXX.net/jira/rest/api,
            [
                RelativePath="2/search",
                Query=
                [
                    jql="filter=525545",
                    startAt=Skip,
                    maxResults=Top,
                    fields="issuetype,key,summary,customfield_10401,labels,customfield_10000,status,resolution,customfield_11200"
                ],
                Headers=[Authorization="Bearer XXX"]
            ]
        ),
        Json = Json.Document(PageData),
        Value = Json[#"issues"]
    in Value,

 

If you have other questions, please contact me at any time.

 

Best Regards,
Yang
Community Support Team

 

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

View solution in original post

4 REPLIES 4
v-huijiey-msft
Community Support
Community Support

Hi @Alina12 ,

 

Hope all is going well.

 

Please follow these steps:

 

1. The error you are encountering in the function is due to attempting to concatenate a binary () with a string. Since is used with , it is treated as a binary.

 

You should modify your approach to use and options within for pagination as well.GetJiraIDCountBaseUrlBaseUrlWeb.ContentsRelativePathQueryWeb.Contents

 

2. Modify the GetPage function:

GetPage = (Index) =>
    let
        Skip = Text.From(Index * JiraIDPerPage),
        Top = Text.From(JiraIDPerPage),
        PageData = Web.Contents(
            https://XXX.XXX.net/jira/rest/api,
            [
                RelativePath="2/search",
                Query=
                [
                    jql="filter=525545",
                    startAt=Skip,
                    maxResults=Top,
                    fields="issuetype,key,summary,customfield_10401,labels,customfield_10000,status,resolution,customfield_11200"
                ],
                Headers=[Authorization="Bearer XXX"]
            ]
        ),
        Json = Json.Document(PageData),
        Value = Json[#"issues"]
    in Value,

 

If you have other questions, please contact me at any time.

 

Best Regards,
Yang
Community Support Team

 

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

hi @v-huijiey-msft ,

 

thanks a lot for your reply! 

I tried your recommendation and fortunately it works fine now in power bi desktop and I get all hits, but unfortunately I still get the error message with the dynamic data sources in Power BI Services. Do I have to use Web.Contents, Relative Path and Query everywhere instead of BaseUrl and not only in the PageData? Can the problem also be caused by the pagination?

My code so far:

 

let 
    BaseUrl = 
    Web.Contents(
        "https://XXX.XXX.net/jira/rest/api",
        [
            RelativePath="2/search",
            Query=
            [
                jql="filter=525545",
                maxResults="1000",
                fields="issuetype,key,summary,customfield_10401,labels,customfield_10000,status,resolution,customfield_11200"
            ],
            Headers=
            [
                Authorization="Bearer XXX"
            ]
        ]
        
    )  ,
 
    JiraIDPerPage = 1000,

    GetJson = (Url) =>
        let 
            RawData = BaseUrl,
            Json    = Json.Document(RawData)
        in  Json,

    GetJiraIDCount = () =>
        let Url   = BaseUrl,
            Json  = GetJson(Url),
            Count = Json[#"total"]
        in  Count,

    GetPage = (Index) =>
    let
        Skip = Text.From(Index * JiraIDPerPage),
        Top = Text.From(JiraIDPerPage),
        PageData = Web.Contents(
            "https://XXX.XXX.net/jira/rest/api",
            [
                RelativePath="2/search",
                Query=
                [
                    jql="filter=525545",
                    startAt=Skip,
                    maxResults=Top,
                    fields="issuetype,key,summary,customfield_10401,labels,customfield_10000,status,resolution,customfield_11200"
                ],
                Headers=[Authorization="Bearer XXX"]
            ]
        ),
        Json = Json.Document(PageData),
        Value = Json[#"issues"]
    in Value,

    JiraIDCount = List.Max({ JiraIDPerPage, GetJiraIDCount() }),

    PageCount   = Number.RoundUp(JiraIDCount / JiraIDPerPage),

    PageIndices = { 0 .. PageCount - 1 },

    Pages       = List.Transform(PageIndices, each GetPage(_)),

    JiraID    = List.Union(Pages),

    Table       = Table.FromList(JiraID, Splitter.SplitByNothing(), null, null, ExtraValues.Error),

    ....

 

Hi @Alina12 ,

 

You mentioned that it works fine in Power BI Desktop, but there is a problem in Service. One possible influencing factor is:

 

Please check that the authentication token is still valid and is passed correctly with every request. Token expiration or authentication issues can sometimes manifest as errors with dynamic data sources.

 

Here is a revised approach for the pagination part:

GetJiraIDCount = () =>
    let
        Json = Web.Contents(
            https://XXX.XXX.net/jira/rest/api,
            [
                RelativePath="2/search",
                Query=[jql="filter=525545", maxResults="0", fields=""],
                Headers=[Authorization="Bearer XXX"]
            ]
        ),
        Data = Json.Document(Json),
        Count = Data[total]
    in
        Count,

GetPage = (Index) =>
    let
        Skip = Text.From(Index * JiraIDPerPage),
        PageData = Web.Contents(
            https://XXX.XXX.net/jira/rest/api,
            [
                RelativePath="2/search",
                Query=[
                    jql="filter=525545",
                    startAt=Skip,
                    maxResults="1000",
                    fields="issuetype,key,summary,customfield_10401,labels,customfield_10000,status,resolution,customfield_11200"
                ],
                Headers=[Authorization="Bearer XXX"]
            ]
        ),
        Json = Json.Document(PageData),
        Value = Json[issues]
    in
        Value,

 

For more information on using in Power BI, I recommend reviewing the following documentation:

Web.Contents - PowerQuery M | Microsoft Learn

 

If you have any further questions please feel free to contact me.

 

Best Regards,
Yang
Community Support Team

 

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

Alina12
Frequent Visitor

UPDATE

I have managed to get the code to work with Web.Contents, Relative Path and Query. The problem now is that a) I only get the first 1000 hits instead of approx. 5,500 hits (although JiraIDCount shows the correct number of total hits) and b) it still shows me a dynamic data source and the data refresh is not possible.

So it seems that my problem might be with the pagination, because the whole www says that the dynamic data refresh should work with Web.Contents, Relative Path and Query. Can anyone help me?

let 
    BaseUrl = 
    Web.Contents(
        "https://XXX.XXX.net/jira/rest/api",
        [
            RelativePath="2/search",
            Query=
            [
                jql="filter=525545",
                maxResults="1000",
                fields="issuetype,key,summary,customfield_10401,labels,customfield_10000,status,resolution,customfield_11200"
            ],
            Headers=
            [
                Authorization="Bearer XXX"
            ]
        ]
        
    )  ,

    JiraIDPerPage = 1000,

    GetJson = (Url) =>
        let 
            RawData = BaseUrl,
            Json    = Json.Document(RawData)
        in  Json,

    GetJiraIDCount = () =>
        let Url   = BaseUrl,
            Json  = GetJson(Url),
            Count = Json[#"total"]
        in  Count,

    GetPage = (Index) =>
        let Skip  = "&startAt=" & Text.From(Index * JiraIDPerPage),
            Top   = "&maxResults=" & Text.From(JiraIDPerPage),
            Url   = Text.From(BaseUrl) & Skip & Top,
            Json  = GetJson(Url),
            Value = Json[#"issues"]
        in  Value,

    JiraIDCount = List.Max({ JiraIDPerPage, GetJiraIDCount() }),

    PageCount   = Number.RoundUp(JiraIDCount / JiraIDPerPage),

    PageIndices = { 0 .. PageCount - 1 },

    Pages       = List.Transform(PageIndices, each GetPage(_)),

    JiraID    = List.Union(Pages),

    Table       = Table.FromList(JiraID, Splitter.SplitByNothing(), null, null, ExtraValues.Error),

...

Helpful resources

Announcements
Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

Power BI Carousel June 2024

Power BI Monthly Update - June 2024

Check out the June 2024 Power BI update to learn about new features.

PBI_Carousel_NL_June

Fabric Community Update - June 2024

Get the latest Fabric updates from Build 2024, key Skills Challenge voucher deadlines, top blogs, forum posts, and product ideas.

Top Solution Authors