Forum Discussion

postvijay's avatar
postvijay
Regular Visitor
1 year ago
Solved

Filtering which includes Blanks on Bitemporal Tables

We use Bi-temporal tables to reduce volume of Data (having validity on each row as 'From Date' -'To Date') e.g as below

 

IDFrom DateTo DateLocationAttribute n
101-01-202415-05-2024Londonabc
116-05-202419-05-2024Londonxyz
120-05-202415-09-2024Manchesterxyz

 

I have similar many tables which are in above format. 

Use Case: User wants to select Date range and get the data as per the range validity. 

User uses metrics or table visuals mostly and the columns are mostly text hence there is limited need of aggregations

  • My current approach is to create a measure which uses the selected date from slicer and puts flag as 1/0 based on Date range against each row
  • Then I apply the filter on each visual that User needs. 

Challenge

  1. Its challenging to connect tables which has its individual Date range for validity and apply filters individually on each Visual (any miss in filter will throw wrong data) --> any way to create a page level filter or repor level filter (measures cant be used on page/report level filters)
  2. Showing blanks instead of Inner join kind of result: When some tables might have introduced later and hence doesnt match the start date of old table :  If someone queries the data for old time period, default inner join will return blank as there is no match in one of the table and we require to show atleast a blank or null instead of entire blank table as user is using matrcis visual. 
  • Hi postvijay 
    Sorry fo the late response.
    PLease do check the below detailed steps that might resolve your issue.

    To address performance and data consistency issues in your Power BI model with bitemporal and snowflake schema design, consider flattening related tables (like A1–A7, B1–B3, etc.) into their respective central tables (A, B, C) using Power Query or SQL views. This reduces the risk of data loss from inner joins and improves performance.

    Introduce a centralized Date table and map your entities using a valid-from/to structure in a separate mapping table. Use DAX functions like TREATAS() or USERELATIONSHIP() to apply custom date filters without expanding millions of rows at runtime.

    Avoid direct many-to-many joins by using bridge tables or by pre-processing the mappings into your model to reduce relationship complexity. When handling large historical data, split the model using composite mode: recent data can be imported for fast access, while older records can be accessed via DirectQuery or summarized tables.

    Wherever possible, push filtering and bitemporal logic into the data source to avoid expensive transformations in Power BI. Avoid using DAX-generated values in slicers; instead, build static slicer tables with predefined time ranges or categories.

    Finally, leverage incremental refresh for long-term datasets to prevent full refreshes and maintain efficient query performance.

    This combined modeling approach should provide a more maintainable, scalable, and performant solution. Let me know if you'd like help structuring this with sample schema or visuals.

    If all your tables are in import mode, you can use Power query to prepare your data.
    You can follow these steps-

    1. Import all your bi-temporal tables.
    2. Create all possible ID-Date combinations (through calendar table cross join).
    3. Merge tables one-by-one using Left Joins based on:
      ID match

      Date between From and To

    This results in a flattened structure where unmatched data from newly introduced tables will result in nulls, not blanks.


    If the above information is helpful, please give us Kudos and mark the response as Accepted as solution.
    Best Regards,
    Community Support Team _ C Srikanth.

15 Replies

  • Hello postvijay 

     

    Use calculated column

    IsInSelectedRange =

    VAR SelectedFrom = MIN('DateTable'[Date])

    VAR SelectedTo = MAX('DateTable'[Date])

    RETURN

    IF(

        AND(

            'Table'[From Date] <= SelectedTo,

            'Table'[To Date] >= SelectedFrom

        ),

        1, 0

    )

     

    Thanks,
     Pankaj Namekar | LinkedIn

    If this solution helps, please accept it and give a kudos (Like), it would be greatly appreciated.

    • postvijay's avatar
      postvijay
      Regular Visitor

      Thanks Pankaj, I am doing same thing currently but issue is if any date if missing in any of the fact tables, the entire table or.metrics shows blank. This is because we filter isinrange=1 

  • Hi postvijay ,

    I think you're working with a bitemporal model in Power BI and aiming to simplify user interactions without losing date-valid accuracy — a common challenge when handling time-bound records across multiple tables.

    For Issue 1:

    1. Create a 'Date Filter' table like this:
      • DateFilter = 
        CALENDAR ( DATE(2020, 1, 1), DATE(2030, 12, 31) )
    2. Then, in each of your bitemporal tables (e.g., TableA, TableB, etc.), create a calculated column like:
    3. Now, instead of filtering visuals one-by-one, create a composite table or model where you use a FILTER over only IsInRange = 1.
    4. To avoid writing filters on each visual, you can create calculated tables that only include valid rows:
      • ValidTableA = 
        FILTER (
        TableA,
        TableA[IsInRange] = 1
        )
      • Then use ValidTableA in your visuals instead of raw TableA.

    This way, the filter context is centralized via the slicer, and visuals stay clean.

    For Issue 2: 
    Use Left Joins + Default/Placeholder Records

    Power BI doesn’t have native SQL-like joins for relationships, but we can simulate this in DAX using LOOKUPVALUE, TREATAS, or LEFT JOIN-style merging in Power Query.

    We can do this with 2 Approaches:

     

    Option A: Use LOOKUPVALUE with a Fallback

    In your main table, when referencing other tables:

    Attribute_n_From_B = 
    LOOKUPVALUE (
    TableB[Attribute_n],
    TableB[ID], TableA[ID],
    TableB[IsInRange], 1
    )

    And use IF(ISBLANK(...), "No Match", ...) to prevent full row blanks.

     

    Option B: Power Query Merge with Placeholder Row

    In Power Query:

    1. Create a row in each table with ID = -1, FromDate = 1900-01-01, ToDate = 9999-12-31, and all columns set to "Unknown" or null.
    2. When merging tables, always do a Left Outer Join and fallback to this row if there’s no match:
      • This prevents matrix visuals from going fully blank when one side lacks data for a given date.

    Hope this solution helps you make the most of Power BI! If it did, click 'Mark as Solution' to help others find the right answers.
    💡Found it helpful? Show some love with kudos 👍 as your support keeps our community thriving!
    🚀Let’s keep building smarter, data-driven solutions together! 🚀 [Explore More]

    • postvijay's avatar
      postvijay
      Regular Visitor

      Thanks for reply, I will try this approach today. Just wondering, does.calculated table takes too much memory and is it dynamically.changes based on user date range selection?

    • postvijay's avatar
      postvijay
      Regular Visitor

      Hi GrowthNatives Calculated tables as suggested, I tried to use but found that my pbix file is getting doubled in size.

      Is there any alternate way to do this?

      May be by clubbing all filters and then applying at once

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi postvijay 
    In addition to the GrowthNatives pankajnamekar25 Please check the below point to resolve your issue.

    • Use ISBLANK() in DAX filters instead of = BLANK() or <> BLANK() to accurately detect blank values.

    • Create calculated columns to replace blank values with descriptive labels (e.g., "No Date" or "Not Applicable").

    • Format dates in calculated columns to make visuals clearer, such as using FORMAT([Date], "dd-mm-yyyy").

    • Enable the "Show items with no data" option in visuals to ensure all categories appear, even those without associated data.

    • Use ALLNOBLANKROW() in DAX instead of ALL() to prevent unintended blank rows caused by table relationships.

    • Include a complete date table in your model to cover all date ranges in ‘From Date’ and ‘To Date’ columns for consistent filtering.


    If the above information helps you, please give us a Kudos and marked the Accept as a solution.
    Best Regards,
    Community Support Team _ C Srikanth.

     

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi postvijay 
    Sorry for the late response.

    To handle filtering blanks on bitemporal tables in Power BI, especially when using date ranges across multiple tables with ValidFrom and ValidTo columns, you can follow this approach:

    1. Create a separate date filter table using the CALENDAR function, for example:
      DateFilter = CALENDAR(DATE(2020, 1, 1), DATE(2030, 12, 31))
    2. In each of your bitemporal tables (e.g., TableA, TableB), create a calculated column to check if a row is in the selected date range:
      IsInRange =
      VAR SelectedStartDate = MIN('DateFilter'[Date])
      VAR SelectedEndDate = MAX('DateFilter'[Date])
      RETURN
      IF(
      [ValidFrom] <= SelectedEndDate &&
      (ISBLANK([ValidTo]) || [ValidTo] >= SelectedStartDate),
      1,
      0
      )
    3. This logic includes rows where ValidTo is blank (interpreted as "still valid").

    In your visuals or filters, include only the rows where IsInRange = 1.

    This method allows you to control date-range filtering even when some of your validity periods are open-ended (i.e., have blank ValidTo values), without breaking the logic in visuals or relationships.

    If the above information is helpful, please give us Kudos and mark the response as Accepted as solution.
    Best Regards,
    Community Support Team _ C Srikanth.

     

    • postvijay's avatar
      postvijay
      Regular Visitor

      Hi Srikanth, 
      Sorry for late reply but i couldnt login to the portal. 
      The challenge I am facing is in modelling: snowflake schema created out of few star schema
      Table structure:

      • I have tables in  star schema having 7 tables  (A1 to A7) attached to central table A.
      • One more star schema having 3 tables (B1 to B3) attached to central table B
      • and one more star schema which connects 4 tables (C1 to C4) to its central table C
      • and some more similar strucutred tables. 

      Issues:

      • As powerbi by default connects using inner join, and in case when some relation doesnt exists in table B or C, my entire row in Table A also vanishes though table A has valid data. 
      • I am able to achieve the results
        • by calculating each attribute against date and by passingthrough hoops to get data from table c4 against attribute in table A1 
        • this appraoch makes powerbi too slow.
        • the dax cant be used as slicer hence its cant be used infull capacity
      • as all tables have from_date and To_date --> the key columns repeat and hence we have to join all these tables in many-many  --> which i feel is bad design. 
      • there are around 500K records already and I am trying to get data for last 10+ years hence expanding each table on each business date kills powerbi . 

      need any best approach to handle bitemporal tables and without loosing data and with best performance.

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi postvijay 

    I wanted to follow up since I haven't heard from you in a while. Have you had a chance to try the suggested solutions?
    If your issue is resolved, please consider marking the post as solved. However, if you're still facing challenges, feel free to share the details, and we'll be happy to assist you further.
    Looking forward to your response!


    Best Regards,
    Community Support Team _ C Srikanth.

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi postvijay 

    We haven't heard from you since last response and just wanted to check whether the solution provided has worked for you. If yes, please Accept as Solution to help others benefit in the community.
    Thank you.


    If the above information is helpful, please give us Kudos and mark the response as Accepted as solution.
    Best Regards,
    Community Support Team _ C Srikanth.

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi postvijay 

    We haven't heard from you since last response and just wanted to check whether the solution provided has worked for you. If yes, please Accept as Solution to help others benefit in the community.
    Thank you.


    If the above information is helpful, please give us Kudos and mark the response as Accepted as solution.
    Best Regards,
    Community Support Team _ C Srikanth.

    • postvijay's avatar
      postvijay
      Regular Visitor

      Hi Srikanth, 
      Sorry for late reply but i couldnt login to the portal. 
      The challenge I am facing is in modelling: snowflake schema created out of few star schema
      Table structure:

      • I have tables in  star schema having 7 tables  (A1 to A7) attached to central table A.
      • One more star schema having 3 tables (B1 to B3) attached to central table B
      • and one more star schema which connects 4 tables (C1 to C4) to its central table C
      • and some more similar strucutred tables. 

      Issues:

      • As powerbi by default connects using inner join, and in case when some relation doesnt exists in table B or C, my entire row in Table A also vanishes though table A has valid data. 
      • I am able to achieve the results
        • by calculating each attribute against date and by passingthrough hoops to get data from table c4 against attribute in table A1 
        • this appraoch makes powerbi too slow.
        • the dax cant be used as slicer hence its cant be used infull capacity
      • as all tables have from_date and To_date --> the key columns repeat and hence we have to join all these tables in many-many  --> which i feel is bad design. 
      • there are around 500K records already and I am trying to get data for last 10+ years hence expanding each table on each business date kills powerbi . 

      need any best approach to handle bitemporal tables and without loosing data and with best performance.

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi postvijay 
    Sorry fo the late response.
    PLease do check the below detailed steps that might resolve your issue.

    To address performance and data consistency issues in your Power BI model with bitemporal and snowflake schema design, consider flattening related tables (like A1–A7, B1–B3, etc.) into their respective central tables (A, B, C) using Power Query or SQL views. This reduces the risk of data loss from inner joins and improves performance.

    Introduce a centralized Date table and map your entities using a valid-from/to structure in a separate mapping table. Use DAX functions like TREATAS() or USERELATIONSHIP() to apply custom date filters without expanding millions of rows at runtime.

    Avoid direct many-to-many joins by using bridge tables or by pre-processing the mappings into your model to reduce relationship complexity. When handling large historical data, split the model using composite mode: recent data can be imported for fast access, while older records can be accessed via DirectQuery or summarized tables.

    Wherever possible, push filtering and bitemporal logic into the data source to avoid expensive transformations in Power BI. Avoid using DAX-generated values in slicers; instead, build static slicer tables with predefined time ranges or categories.

    Finally, leverage incremental refresh for long-term datasets to prevent full refreshes and maintain efficient query performance.

    This combined modeling approach should provide a more maintainable, scalable, and performant solution. Let me know if you'd like help structuring this with sample schema or visuals.

    If all your tables are in import mode, you can use Power query to prepare your data.
    You can follow these steps-

    1. Import all your bi-temporal tables.
    2. Create all possible ID-Date combinations (through calendar table cross join).
    3. Merge tables one-by-one using Left Joins based on:
      ID match

      Date between From and To

    This results in a flattened structure where unmatched data from newly introduced tables will result in nulls, not blanks.


    If the above information is helpful, please give us Kudos and mark the response as Accepted as solution.
    Best Regards,
    Community Support Team _ C Srikanth.

  • postvijay's avatar
    postvijay
    Regular Visitor

    Thanks Srikanth, 
    i will try the current and historic date combination as per suggeston using direct query and import mode.
    Falttening of data is not an option as all tables are having historic date range and each associated with atleast 60000 and few tables already have million rows. 
    e.g. 2009 to today for few rows like 20147 to today() - 

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi postvijay 
    Thanks for confirming that using the direct query and import mode will resolve your issue.

    If your issue is resolved, please consider marking the post as solved. However, if you're still facing challenges, feel free to share the details, and we'll be happy to assist you further.
    Looking forward to your response!


    Best Regards,
    Community Support Team _ C Srikanth.