Data Model with Multiple Companies

[jeffmasse]

I've spent a considerable amount of time building a complex data model for a single company. This company was recently acquired by another company with approximately 30 subsidiaries. Each company in the group (around 30 companies in total) all use the same ERP system. But each company runs its own instance of the ERP database - same table definitions, but separate DBs.

I'm trying to decide how to incorporate a "Company" table into my existing model and which table(s) to relate this Company table to. I've considered inactive relationships to lookup or fact tables. I've considered making the Company table "disconnected" and using FILTER in my measures to only include Companies that are selected in my slicer. I'm sure there are other options out there too. This is going to be a significant endeavor, so I'm trying to come up with a "tried and tested" approach so that I hopefully avoid some mistakes.

My existing model is made up of 50+ tables loaded to a model. At the core, I've got Sales, Purchases and Inventory. I've also got many look-ups, some side fact tables (e.g., inventory re-work, etc). And I'm able to link my Sales and Purchases tables together through my Inventory table.

For simplicity, assume my model looks like the following:

My general plan to get all data from all companies into a single model is as follows:

1. Create dataflows for each company. Each dataflow would be the same code, but would point to each individual company DB.
2. Append all the dataflows together into a single data model. e.g., the Customers table would get appended 30x. I would likely introduce a Company ID column into the table in case of duplicate Customer IDs.

But my issue is determining where/how to introduce a Company table. Again, considering relationships (active/inactive) to either/both of lookup/fact tables. Also, considering a disconnected Company table and using FILTER in my measures to filter Sales, etc. on the selected company(ies).

I would love to hear of an approach someone else has taken and has had success with. Particularly where there are multiple lookup and fact tables involved.

Thanks in advance!

Jeff

[AlexisOlson · Accepted solution]

In order to deal with potential duplicated IDs, you'll probably need to re-index the Customers, Salesperson, Buyers, and Vendor tables so that it's possible to have, for example, the same customer/buyer/vendor associated with multiple companies. Sales and Purchases should get a new Company ID column each. The other dimension tables could as well, but don't necessarily need active relationships with Company.

This is my basic idea:

Disclaimer: Since I haven't spent more than a few minutes considering this, consider it more of a possible approach to mull over rather than anything definitive. You know your own situation far better than I do.

[agangas]

Finally someone with the same problem that I have.  I just spent a lot of time looking for any suggestion to this and I can't find nothing

My data model is even simplier.  A sales fact table, Customers and Products dim tables, as always the Calendar table and now ... the country table, linked to both dim tables.  So, imagine a very simple report with only a table with products on columns and country/customers on rows, showing the total sale on each cell.  As in the data model I only can activate either the country->customer or country->product relation, the report is always show me the wrong total sales in each cell
The only way I can imagine to solve this is duplicating the country table and make the relations separated. But then, how can I handle if I want to add a slicer with the country in my report?

[jeffmasse]

Thanks for the quick response on this! I was anticipating the need to re-index my lookup tables due to duplicate values.

I like your suggestion to introduce Company where you did. However, won't this cause an issue where the Company table will be filtering the Inventory table via both Sales and Purchases? I could get around this by deactivating the relationships from Company to Sales/Purchases in the model and then using CROSSFILTER to activate in my measures as necessary. Is this along the lines of what you were suggesting?

[AlexisOlson]

However, won't this cause an issue where the Company table will be filtering the Inventory table via both Sales and Purchases?

You are correct. That does create an ambiguous filter path and one possible resolution would be to relate Company directly to Inventory and de-activate the other two relationships.

In general, relationships between fact tables are not a great idea, so you may want to consider relating Inventory directly to its relevant dimension tables rather than through Sales and Purchases.

[jeffmasse]

I agree with your comment that relating fact tables is not a good idea. I've committed a bit of a moral sin on this model in that respect. However, I can't come up with a way around it, given what I'm trying to achieve. A specific use case of this model is that we want to analyze how our sales are being fulfilled (e.g., we select a customer and can see which vendors supplied the inventory for those sales - we use a specific identification inventory system). This allows us to easily evaluate profitability by customer, vendor, salesperson, buyer, etc. Or if a specific Sales Order loses money, we can see the specific vendor that supplied the product for that order. We need high level analysis at the lookup table level as well as granular details for specific sales orders, etc.

I realize tyring to figure out how to make this model into a better star schema is not within the scope of my question here. But that's the context as to why I've build the model in this way and why I'm struggling to figure out the best way to implement this model for multiple companies.

[ravjay]

Hi,

Having a similar issue with my ERP model. I've 10 subsidiary companies and need to combine them into a single data model. I've tried creating similar dim & fact tables for each company and then combined them into one table ( consolidated table). But it creates issues in refreshing. I've also tried the composite data model. Then stuck in schedule refreshing. 

please see my posted question below,

https://community.powerbi.com/t5/Desktop/COMPOSITE-DATA-MODEL-WITH-NUMEROUS-APIS/m-p/2588804#M913394

So, did you get to succeed on your model? any suggestions for me?

[jeffmasse]

I was able to get my model to work the way I wanted.

For your situation, I recommend combining the same data (e.g., POs) from multiple companies into a single table. Can you explain what your refresh issues were? I may have suggestions to resolve your refresh issue if you provide details.

I think a composite model in this case likely won't be the best approach. I think you want to "stack" your data by company into the same tables.

[ravjay]

@jeffmasse 

The reasons for not stacking the data by the company to the same table and keeping each company separately are  -

  *  if any APIs failed at a company level it wouldn't block the loading of other companies' data. 

 * there are some companies with larger data sets, which create issues in loading 

 * if we need to add a new company, we can create a data model separately and add it to the composite table without changing every single table

When I tried to build this data model and refresh the composite data model I received the error 

"Microsoft.PowerBI.AS.ConnectionDetailsProvider, Single sign-on (SSO) credential is not provided which is required to execute the current request. Note that performing a refresh over a dataset with a calculated table or a calculated column which references a Direct Query data source is not supported within the Power BI service.."

[jeffmasse]

@ravjay

One approach to stacking, which may alleviate your concerns is by using dataflows to connect to each of your individual company datasets, then stack the dataflows. This would address the following concers:

1. If one dataflow failes, the stacking will still take place (it will rely on an earlier refresh).

2. Refresh speed should be better, because it is not unlike have individual datasets refresh against your companies.

3. Adding new companies is in fact easier under this method - you could achieve it without modifying the PBIX file at all. Whereas in your composite model, you would need to edit the composite file for the addition of each company.

I successfully use this approach in a few scenarios. It is very flexible and scalable. The approach you are taking with composite models is likely to cause frustrations in the future.