Forum Discussion

RRanks's avatar
RRanks
Frequent Visitor
2 years ago
Solved

Beginner - Data Modeling help.. 5 data sources

Hi, I am looking for help with some PowerBI Data modeling. I am working on building some visuals to compare data in tables and I am trying to avoid creating giganta tables.. and learn some Data modeling at the same time.

**This is my first post and just starting my PowerBI journey.. forgive me if I am going down he wrong track here**

 

Background on what I am doing:
I work on the Exchange/Messaging team and am working on building out Room Mailboxes for confrenece rooms etc.. based of some real estate data stored in DataLake in Parquet files. I have these imported into PowerBI as well as my own Azure Table Storage, Tables I use to store this RE Data + other information which is then used by Azure Functions to create the rooms in Exchange.

What I want to do:

 

As the Real Estate system changes... Rooms are added / Removed, Names changed, Capacity or the type of room changes.. I want to check the data against my own tables, so that we can update Exchange to match if they fall our of sync. 

I want to compare the data in the Real Estate DataLake Parquet files, to what is stored in my Azure Tables to make sure it still matches, to ensure we have the current information for each building, room etc.. applied in exchange.

 

5 Data Sources as follows

Real Estate Data is broken down like this:

Building .Parquet

  • Each Building has a BuildingID + City, State, CountryCode, Address PostCode etc..

Floors .Parquet

  • Each Floor is in a Building
  • Each Floor has FloorID and BuildingID + Level, CountryCode

Space. Parquet

  • Each Space is on a Floor
  • Each Space - has a SpaceID and FloorID + Capacity, DisplayName,Reservable(true/false),SpaceType,SecurityType etc..

 

My Exchange Azure Tables:

BuildingRoomList

  • Extracted Data from Building Parquet file + Exchange specific settings

Rooms

  • Extracted Data from Floor and Space Parquest files + Exchange Specific settings.
  • Each Room is a member of a BuildingRoomList

(For the purpose of this I am ingnoreing the Exchange Specific settings as these are not needed to be compared)

 

Data Modeling relationships:

 

Building .Par -> One to Many -> Floor .Par

Floor .Par -> One to Many -> Space .Par

 

BuildingRoomList.AzTable -> One to Many -> Rooms.AzTable

 

Space.Par -> One to One -> Rooms.AzTable

Building .Par -> One to Many -> BuildingRoomList.AzTable
(Some buildings have to be broken into multiple RoomLists due to number of rooms)

Help Questions: I have so many haha but I am will start where I am at... 

 

1. Comparing Building .Par / RoomList .Par:
So far I have just done a "MergeQueries to New" and joined the Building and BuildingRoomList then done a new Custom column for each value to compare. Set to true/false.

 

Then used that to build this visual.. I can now see where each building has different values in RE Data / Exchange.

 

Question: Is this the 'right' way to do this? Should I be using the data modeling somehow to check if they match on each table instead of doing a merge?

2. For Spaces / Rooms

Seeing as there is a an additional Floors .Par I need to get some data from should I be doing 2 merges? Merging Space/Floors.Par into a new table to join that data. Then doing another Merge between that table and Room.AzTable to check each value the same as I did above Buildings? OR... is there a way I can do all this without needing for merge all these new tables and just use the existing data modeling and filtering in the visuals?

 

Thanks in Advance!

 

 

  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi RRanks,

    Regarding your first question, if you want to determine if the data in two tables are the same. I would take your approach as well, I think it is intuitive and effective.

    Regarding your second question, Power query is used to get the data you need. dax is used to analyze the data. Stick to your point.Using Power query.

1 Reply

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi RRanks,

    Regarding your first question, if you want to determine if the data in two tables are the same. I would take your approach as well, I think it is intuitive and effective.

    Regarding your second question, Power query is used to get the data you need. dax is used to analyze the data. Stick to your point.Using Power query.