Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Grow your Fabric skills and prepare for the DP-600 certification exam by completing the latest Microsoft Fabric challenge.

Reply
angelflight
Frequent Visitor

Developing a custom visual but not getting all the rows in the dataset

I'm building a custom visual. The dataset is a csv file with 18,100 rows and 6 columns. I'm loading the data using a table data row mapping as shown below. When I look at the options.dataViews[0].table.rows.length in the code, there are only 1,000 rows. I have already (I think) applied the dataReductionAlgorithm parameter. There are no segments listed in options.dataViews[0].metadata.segment so fetchMoreData has no effect. I can create a basic table in this report and I see all the rows in the table.

 

Any help would be appreciated...

 

"dataViewMappings": [
{
"table": {
"rows": {
"select": [
{
"for": {
"in": "column"
},
"dataReductionAlgorithm": {
"window": {"count": 100000}
}
},
{
"for": {
"in": "value"
},
"dataReductionAlgorithm": {
"window": { "count": 100000 }
}
}
]
}
}
}
]

Stephan

4 REPLIES 4
stefanmmarek
New Member

Hi,

i had a similar issue and found out that depending on the mapping you need to set the supported datalength:

    "dataViewMappings": [
        {
            "table": {
                "rows": {
                    "select": [
                        {
                            "for": {
                                "in": "parameter"
                            }
                        },
                        {
                            "for": {
                                "in": "value"
                            }
                        }
                    ],
                    "dataReductionAlgorithm": {
                        "top": { "count": 5000000 }
                    }
                }, "rowCount": { "supported": {"max": 5000000}}
            }
        }
 

 

now i always get all rows since it will never exceed 5 mio 😉

 

dm-p
Super User
Super User

Hi @angelflight,

The dataReductionAlgorithm has a max value of 30,000, even (if you want to use the window algorithm with fetchMoreData) and will likely be reverting to the default of 1,000 as the value is out of range.

I'm not near my dev machine to validate this suggestion for you but I would start by setting this to 30K or lower and seeing if this gets you over 1,000 rows. In the case of using window, the 30K limit still applies for this but the presence of the segment property in your dataView can then be used to load in the additional data.

If this doesn't work for you, let me know and I'll see if I can provide you with a valid configuration when I can get back to my dev machine (likely tomorrow), but I'm hoping that's all you need to get this working.

Good luck!

Daniel

EDIT: completely forgot I made this repo a while ago to demo this exact situation, even down to using the table dataViewMapping... you hopefully might find it useful





Did I answer your question? Mark my post as a solution!

Proud to be a Super User!


My course: Introduction to Developing Power BI Visuals


On how to ask a technical question, if you really want an answer (courtesy of SQLBI)




Thank you so much for your response. I had actually already found your repo and had integrated that code, but since there are no segments in the metadata object, it doesn't do the fetch. I tried removing the check to see if the segments exist:

 

if (options.dataViews[0].metadata.segment) {

 

When I do that, this.host.fetchMoreData() evaluates to true, but I still only get 1,000 rows.

 

What's weird is that I only have 18,100 rows. I have tried setting the dataReductionAlgorithm count to 20,000 and also to 100,000 and then, based on your suggestion, to 6,000 to see if it will force the creation of the segments, but it does not.

 

Any additional thoughts you may have would be appreciated.

Have you found the solution ? Because I'm facing the same issue

Helpful resources

Announcements
RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

MayPowerBICarousel

Power BI Monthly Update - May 2024

Check out the May 2024 Power BI update to learn about new features.

Top Kudoed Authors