ArcGIS Maps for Power BI (Preview) Discussion

[lukaszp]

Since we announced ArcGIS Maps for Power BI (Preview), we've been getting a lot of great questions and positive feedback.

Let's use this thread for a discussion of ArcGIS Maps for Power BI, including any issues you might encounter when using the feature.

You can now use the ArcGIS Maps for Power BI feature in the September release of Power BI Desktop. Remeber to enable it in the options menu.

You can find documentation here:

• Power BI documentation (How to enable, How to manage, etc.)
• Esri's ArcGIS Maps for Power BI documentation
• Esri's ArcGIS Maps for Power BI product page

Some have asked for features:

A) Sign in to ArcGIS Online; it's on the road map ... vote for this idea to keep updated

B) Shapefile support ... use the ShapeMap visual

Known Issues:

Starting Monday 2/14/2017 you may receive an error when opening Power BI desktop when you have enabled ArcGIS Maps for Power BI (preview).  Please review this thread for further details. The workaround for this issue is documented here.

[lukaszp]

When you bind data to ArcGIS Maps for Power BI, any data you bind to it is handled by code written by Esri. So you should assume the data may be transmitted across the network to Esri's servers. Typically, data in the Location Field is sent to be geocoded (a network API call) so it's location on the map can be determined. Your client computer (where you browser or Power BI desktop is running) will make a network request directly to Esri's services which may store information like IP address as part of their normal diganostics/logging.

The data you bind to the visual is covered under the terms and privacy policy you agree to when start using the visuals. You can review them here: https://powerbi.microsoft.com/en-us/documentation/powerbi-service-tutorial-arcgis/#user-consent

[DuncanP]

Hi

Ok, so here's another issue I've looked into further. There are some non-geographic postcode districts which get mapped to geographic areas. Not necessarily a problem per se, but the way it manifests itself is confusing. In the image below, I've mapped postcode districts and excluded BS29. This shows BS99 (non-geographic) mapped to the BS29 area:

You can see that the colour of the area shows there are significantly more bookings than in the surrounding postcode districts. Now if I remove the filter to show BS29, we get the following:

This now shows the correct postcode district and booking count for that district, but the colour still seems to be driven by the underlying BS99 postcode which is now not visible on the map at all.

Obviously now that I know about this, I can go back into my data and mark all the non-geographic postcodes so they can be excluded from mapping, but until I worked this out, I thought there was something seriously wrong with the sizing algorithm! I don't know what a good solution would be from the mapping point of view, but I thought I'd flag this up anyway.

Cheers,

Duncan

[scottball]

Hi @DuncanP, thanks for the great screen shots and write-ups of the issue.

We were already aware of the currency formatting issue and are working on a fix. The faulty matching of postcodes without geographies should be handled by our backend, but for some reason it's not being handled correctly. We are investigating what we can do to resolve the issue. Bear with us as we develop and roll out a fix.

[DuncanP]

Hi @scottball, thanks for looking into this. You're really going to get fed up with me though, as I've now started to notice geographical postcode districts that are incorrectly mapped as well.

so,

TS13 is mapped to BS13

TS15 is mapped to BS15

DN55 is mapped to DE55

DN21 is mapped to BN21

SY13 is mapped to DY13

OX17 is mapped to EX17

SE3 is mapped to SS3

SE9 is mapped to SS9

All the IM districts are mapped to CM districts

There are also some minor issues with some of the shape data, for example:

A part of NR21 has split off and ended up inside PE31

There is also a small gap between NW1 and N1

PO22 has been split by BN18

Part of BS22 is inside BS24

Part of NE30 is in NE29

There are quite a few more of the bisected districts, which is understandable given the weird shapes some of them have, so this is probably nit-picking really.

I'm really enjoying using this though, and I'm very impressed by the speed of it, so thank you.

Best,

Duncan

[scottball]

Hey @DuncanP - 

No worries. I think it's all related (regarding the mismatches). Our underlying service used to give error codes for unmatched addresses, which client applications could filter out or handle differently at their discretion. Now the backend service attempts to find a match no matter what - resulting in some undesireable matches. We're working with the team responsible for this behavior to get it modified, and then we'll update our code in turn. It won't get fixed overnight, but rest assured we're working on it.

Regarding the bisected districts and weird shapes you might be seeing - in order to keep the speed high, we are 'generalizing' the shapes - that's to say, we're making the shapes a lot simpler than they really are in order to transmit them across the wires quickly. It's a balancing act - we want them to be simple enough that they are fast, but we don't want them so simple that they lose the ability to represent the thing they're supposed to represent. Please let me know if it's erring too much towards the latter and we can take a look at tweaking it a bit. 

I'm really happy to hear that despite some hiccups you're enjoying the visual! 

[dcresp]

Hi @scottball As the Australian Burea of Statistics is starting to release the 2016 census data soon will you be updating the Esri visual to also support ASGS 2016 as well as 2011. Hopefully with some more usable pollygons. We love the visual and want to make it part of our Power BI Embeded roll out but need a few things to work a little better and support for Embeded and it will be a great tool.

[scottball]

Hi @dcresp - the free visual includes demographics from the US only. However, if the Australian Bureau of Statistics (or some other agency / organization) publishes the census data as a publicly shared feature service on ArcGIS, you would have access to them as an ArcGIS reference layer inside the ArcGIS map visual.

Later in the year we plan to provide a way to access Esri global demographic content, so keep an eye out for that as well.  

[otravers]

---

@scottball wrote:

Later in the year we plan to provide a way to access Esri global demographic content, so keep an eye out for that as well.  

---

Any idea when that might happen? I'm specifically interested in Latin American countries, for which I've seen ESRI has some demo data. It would be great to have reference layers, infographics, and driving times, like is currently available for US locations. Something like the below but for Sao Paulo or Buenos Aires instead of Austin, TX.

Austin demo

[dcresp]

@scottball sorry I am refering to the ability to represent areas as boundaries when put into Location and the country is selcted. Then choose that you are looking at Australian SA2 areas. There are two issues with this being that the polygons are so bad they make it unusable in Australia and the ASGS boundaries are for 2011 not 2016 so will not work properly for the upcoming census data.See my previous post of the issues with the polygons. We are very keen to be able to use this visula for census data as we think it has great potential.

[scottball]

@dcresp I think I see what you mean. The boundaries that are returned when you select the boundary type using the 'Location Type' tool in edit mode are wrong - right? I'll need to follow up with our content team to find out the refresh cycle for boundary data and when the Australian boundaries will be due for a refresh.  

The boundary generalization issue is in the backlog and we plan to address it in an upcoming release. 

We're very keen to get you using the visual! We are making good progress rounding out rough edges and we have a lot more to show you throughout 2017. Stay tuned!

[dcresp]

@scottball any update on when the Australian ASGS boundaries are going to get looked at. Also any idea on when the visual will be avaliable in Power BI Embeded. We want to use the Esri Visual and Power BI Embeded for a 2016 Census data project in July but these two issues are major obsticals at this stage. Thanks

[scottball]

@dcresp - I'm working with our content team to see what we can do regarding ASGS boundary data. To give some background, we have data content providers that provide some of our content (including the ASGS boundaries). The data provider that gives us the ASGS info (along with other boundary data as well) provides annual updates to Esri and due to the publish date of the ASGS data, it will not be a part of the 2016 data update. At this point we will not receive the updated ASGS data until our 2017 data update. I'm pushing internally to see what we can do to get more timely updates, or incorporate content 'off cycle'.  

[dcresp]

@scottball thank you for the update. Waiting till 2017 would certainly be a major issue for us. Your data provider has done a horrible job of normalising the ASGS boundaries to an extent that they are not usable (see my previous picture). If it helps we have a full GIS team using Esri I can get them to provide the ASGS polygons in what ever format you want at no cost. The base information comes from the ABS at no cost in an Esri format. We can re-format this if this would help. The new Power BI Premium means that we can now use the Esri app in external user content instead of using PBI Embeded but we really need to have access to working ASGS boundaries for the census data release in July. Please let me know if there is anything we can do to assist with this. 

[mike_honey]

@scottball - I second the sentiments of@dcresp , although I dont have any active project for ArcGIS maps. That ASGS boundary issue emphatically killed the ArcGIS visual as a serious contender for all the mapping projects I've been working on. 

Fortunately thanks to @dcresp sharing his findings on the issue, (I think) I was able to avoid any of my clients actually seeing it in all it's horror.  The lengthy delays you describe in accessing freely available data only add to my concerns. 

I suggest you should be looking for a new data provider.  Do you really want to be reliant on an organisation that:

a) is happy to provide Microsoft and it's customers with data of such poor quality, presumably untested

b) takes so long to fix such an obvious issue, where complete & accurate data is freely available

[mwschepers]

Hi,

First of all let me say I am very enthousiastic about the added functionality of ArcGIS in PowerBI! It greatly enhances the value of PBI for me.

I am using it intensively for supply chain planning and have some suggestions for improvement. My wishlist for future improvements in order of priority:

1. In a size and categorical color mode, when multiple catagories share one location it would be essential for me to show values in a pie chart (currently the circles are drawn on top of each other). The total size of the pie chart should match with the total volume of all overlapping categories, which is actually what the existing map funtion in PowerBI already has.
2. Adjust the size based on what is selected. Currently the size is only reset when going to edit mode and adjusting the size slider, so filtering values on a page can make the bubble size very small.
3. Option to permanently display tool tips/labels on the map 
4. Disable the auto zoom feature
5. Current size sttings (equal interval, natural breaks) do not seem to be proportional to the value behind it. Small values are still relatively large circles on the map. I would like to set the bubble size area or the bubble size diameter equal to the quantity.
6. Add a legend for color coding used (both categorical as well as continuous color scheme)
7. Expand the colors of the pin locations, to better match a company color scheme.
8. Ideally, I would like to map a network using from- and to- locations, with arrows/lines in between. But I can imagine this is a bridge too far.

I am looking forward to the formal release, please let me know which of these suggestions can be taken into account!

Best regards

Michiel

[scottball]

@mwschepers - thank you for using the visual and providing some really great feedback around what you'd like to see in the future. As I'm sure you've noticed, we iterate quickly and roll out new functionality and bug fixes nearly monthly. We will evaluate your enhancement requests and determine which can be incorporated into our near term and long term roadmap.  As we dig into these issues we may need to reach out to more clearly understand your use case.

One thing I did want to mention, regarding your highest priority - are you aware that you can get pie charts when using categorical data right now? It's a part of clustering. Take a look at this gif and let me know how close it is to what you're looking for: http://imgur.com/trpY69E

[mwschepers]

@scottball: Thanks for your message. Good to hear that you are seriously taking the suggestions into consideration!

Regarding your remark on the clustering function: This is a little bit what I'm aiming for, since it involves the pie charts with category colors. What I miss is:

- The clustering view radius has a minimum of 5 px, so it will cluster locations which are near to each other. It would be good to have the option to lower this radius so that it only clusters categories which overlap on exactly the same location.

- The pie segments should be proportional to the quantity of each category. Now they are divided equally.

- The size of the total pie should represent the total quantity. Preferrably be equal to the size of another location with the same (total) quantity.

Note that this functionality is exactly equal to what the built-in map visual of PBI provides. Also the custom visual "flow map" contains a similar feature. Because the Esri visual is much better in many other respects, I would really appreciate if it can be included.

Looking forward to your reply, feel free to contact me directly to exchange more details.

Michiel

[dcresp]

I woudl second the ideas that @mwschepers has put forward. Some of these are very much in line with some of our requests for a map legend and some of the other features. The pie charts working in line with the Microsoft mpping visual would certainly also be our preference. Also the drill down ability that the Microsoft visuals have would also be highly desirable. This should be able to be achieved with changes the Microsoft are making so hopefully this will feed through to the Esri visual.

[mike_honey]

Like @dcresp I'm keen to see this working with Australian boundaries - SA2 and also SA1. I hope the 2011 version of these remain available as I have historical data I need to map which is on the 2011 boundaries.

But anyway I cant get it to show anything at all with SA1 boundaries.  I couldnt find any specific info on how to set this up, so my Location field is Text with a SA1 7-digit code e.g. 2142529 (for a suburb of Colac, Victoria).  Then I set the Location Type tool in Edit mode to: Boundaries / One country / Australia / StatisticsAreasLevel1.

Should that work?  I dont see anything coloured on my map.  

Is there any documentation of what Location data is expected for each Location Type setting?

I've sent an example to Lukasz by email.

[dcresp]

@mike_honey SA1 work but you need to use the MainCode which is 11 digits not the 7 digit code for the SA1. It seems to work fine but the generalisation of the SA1 shapes that I have mentioend to @scottball means that the maps are fairly unusable as you end up with lots of triangles all over the map that cross over each other. If the polygon shapes worked and it included ASGS 2011 and 2016 worked it would be an really powerful tool. I am sure that this will be working before too long. Now all I need is for the Esri mapping to be included in Power BI. @scottball is there any time line on this?

[mike_honey]

Thanks @dcresp for clarifying that.  I'm curious how you came across that info (about the 11 digit vs 7 digit codes) ?  Is it published somewhere?

Anyway it sounds like for right now we are stuck with Shape Maps or Synoptic Panels for this requirement ...