Pbiviz to fetch more than 30 000 row

[sa-]

Hello,

I want my dataview to contain more than 30 000 rows for a TS custom visual.

I looked on documentation and I found that I can use the API call host.fetchMorData() . The problem is that the use of this call resultats in the spinner on the top left of the visual turning infinitly, and the update() isn't called.

Do you have any idea where this might come from  ?

[dm-p]

Hi @sa-,

fetchMoreData has a number of quirks, as calling it triggers the update method and if you're not managing state between updates it can recursively call the update method forever even if it's not actually fetching more data.

I have written a very minimal example for the latest SDK that you are welcome to explore to see if anything you're currently doing is tripping you up.

The repo link is here, so you just need to clone and run npm i before firing up the visual for testing.

Hopefully this should provide you with some guidance. Good luck!

Daniel

[sa-]

I did exactly what you have in the example, and i am not getting recursive calls of the update().

Here is a snap of my code:

	public update(options: VisualUpdateOptions) {
		console.log(options.operationKind);
		console.log(options.type);
		if (options.dataViews[0].metadata.segment) {
			console.log('there are new data comming !');
			let canFetchMore = this.host.fetchMoreData();
			if (!canFetchMore) {
				console.log('this is the limit of data we can fetch !');
			}
		}
		// on second or subesquent segments:
		else if (options.type === 2 || options.type === 62) {
			console.log('Constructing visual');

and this is what I get in the console:

0
2
there are new data comming !

With the spinner spinning ..

[jppp]

Hi @sa- ,

I have implemented a bit different (see below) and I am not looking at the operationKind variable: I am just looking if the segment is available yes or no: if available more pages are avialable, and if the fetchMoreData is returning true that data is being loaded and is false the dataView limit of 100MB is exceeded. 

const firstCall = options.operationKind === VisualDataChangeOperationKind.Create,            
      lastCall = (dataView.metadata.segment) ? false : true,

if (!lastCall) {
    let moreData: boolean = this.hostServices.fetchMoreData();
    if ((moreData)) {
        // More data is being loaded
        return;
    } else {
        // Show warning e.g.
    }
}

If you still are not geting it working, can you share your code so we can take a look?

-JP

[sa-]

In fact I'm suspecting the dataView Mapper I'm using a categories, with the following conf:

      "categorical": {
        "categories": {
					"select": [
						{ "bind": { "to": "category" } },
						{ "bind": { "to": "tooltip" } },
						{ "bind": { "to": "cat" } },
						{ "bind": { "to": "subCat" } },
						{ "bind": { "to": "lat" } },
						{ "bind": { "to": "long" } },
						{ "bind": { "to": "icon" } }
					],
					"dataReductionAlgorithm": { "window": { "count": 30000 } }
        }
      }

The update look as shared previously:

	public update(options: VisualUpdateOptions) {
		console.log(options.operationKind);
		console.log(options.type);
		console.log(options.dataViews[0].metadata);
		if (options.dataViews[0].metadata.segment) {
			console.log('there are new data comming !');
			console.log(options.dataViews[0]);
			let canFetchMore = this.host.fetchMoreData();
			if (!canFetchMore) {
				console.log('this is the limit of data we can fetch !');
				console.log(options.dataViews[0]);
			} else {
				console.log('There is no data to fetch');
				console.log(options.dataViews[0]);
			}
		}

what I'm getting is: 

& spinner is spinning without calling back the update.

[jppp]

Hi @sa- ,

Can you try to put a return in the last else part? This is because Power BI is calling the update method after a page of data is ready.

And also you can better look at your data via the categories: 

console.log(options.dataViews[0].categories);

-JP 

[jppp]

Hi @sa- ,

My experience with fetchMoreData() is that is not correctly working in the developer visual: the data view is telling that there are more pages, but the fetchmoredata is not actually retrieving it or response false to state that there are no pages left.

For me the best way to test it, was to implement the logic, compile the visual and use that version to test if it was working correctly.

Don't know what the current status is, but there used to be an issue that the next page(s) are not always in the same structure / included empty values.

-JP

[sa-]

I have indeed compiled the code into a pbiviz. But still having the same issue, Isn't it related to the dataviewmapping ?