Forum Discussion
Sampling of more than 30,000 data
I develop with custom visuals.
I have one question.
How is it sampled when more than 30,000 data is imported?
* In dataReductionAlgorithm of capabilities.json setting
sample is being set.
"categorical": {
"categories": {
"select": [{
"bind": {
"to": "xxxxxxxxx"
}
}],
"dataReductionAlgorithm": {
"sample": {
"count": 30000
}
}
},
* In the documentation
sample: Reduce the dataset by a simple sampling algorithm, limited to a count number of items.It means that the first and last items are included, and a count number of items have equal intervals between them.For example, if you have a dataset [0, 1, 2, ... 100] and a count of 9, you'll receive the values ​​[0, 10, 20 ... 100].
But I feels different.
*Shouldn't Power BI read more than 30,000 data?
that's all. Thank you
Hi Anonymous,
I have requested help from the Custom Visuals team in the past on how the sampling reduction method actually works, as I use it in one of my visuals. All I could get confirmed is that it is not the same as the one used in the core visuals. You might have better luck if you want to liaise with them at [email protected].
You can go beyond the 30,000 row limit in TypeScript custom visuals, but it is still limited by the amount of memory the development visual sandbox is consuming. I can consistently get 150,000 rows in a test visual I have set up.
If you want to explore this, you need to work with fetchMoreData. Note that you will need to change your dataReductionAlgorithm to window.
I have a couple of repositories that demonstrate this that you are welcome to refer to:
Hopefully some of this might be useful for you. Good luck!
Daniel
If my post solves your challenge, then please consider accepting as a solution to help other forum members find the answer more quickly 🙂
1 Reply
- dm-p
Super User
Hi Anonymous,
I have requested help from the Custom Visuals team in the past on how the sampling reduction method actually works, as I use it in one of my visuals. All I could get confirmed is that it is not the same as the one used in the core visuals. You might have better luck if you want to liaise with them at [email protected].
You can go beyond the 30,000 row limit in TypeScript custom visuals, but it is still limited by the amount of memory the development visual sandbox is consuming. I can consistently get 150,000 rows in a test visual I have set up.
If you want to explore this, you need to work with fetchMoreData. Note that you will need to change your dataReductionAlgorithm to window.
I have a couple of repositories that demonstrate this that you are welcome to refer to:
Hopefully some of this might be useful for you. Good luck!
Daniel
If my post solves your challenge, then please consider accepting as a solution to help other forum members find the answer more quickly 🙂