Why only 1000 rows of data uploaded to BI from Lakehouse

[NiteshS]

Dear Community,

I connected my data housed in Dataverse to Lakehouse in Fabric. Did some transformation in lakehouse using Notebook. After this I loaded the transformed table back to Table folder of Lakehouse. Now when i connect this transfomed table to PowerBI, I see only 1000 rows of data. 

How do I overcome this?

Thanks

Nitesh

[rubayatyasmin · Accepted solution]

Hi, @NiteshS 

It sounds like you're experiencing a common issue. Many data tools, including Power BI, have default row limitations for initial data view, which can limit the number of rows loaded or displayed. Here's what you can do:

1. Check Power BI Import Settings: Make sure to check your Power BI settings for data import. Sometimes, the settings may limit the number of rows being imported to Power BI. If possible, try to adjust the settings to accommodate the entire dataset.

2. Increase Load Limit: If you're pulling the data via a connector (e.g., ODBC, JDBC, REST API, etc.), check the settings or configuration files for that connector. Some connectors have a default limit on the number of rows they will retrieve in a single operation (for example, the first 1,000 rows), and this can usually be changed.

3. Pagination: If your data source supports pagination, you may need to make use of this feature. Some sources will only return a certain number of records per page and require you to retrieve additional records by requesting the next page.

4. Split your Query: You could consider splitting your query into multiple smaller queries, each retrieving a portion of the data, and then append the results together in Power BI.

this thread will be helpful.

https://community.fabric.microsoft.com/t5/Desktop/Limit-of-1000-value-reached/m-p/368850

[rubayatyasmin · Accepted solution]

Hi, @NiteshS 

It sounds like you're experiencing a common issue. Many data tools, including Power BI, have default row limitations for initial data view, which can limit the number of rows loaded or displayed. Here's what you can do:

1. Check Power BI Import Settings: Make sure to check your Power BI settings for data import. Sometimes, the settings may limit the number of rows being imported to Power BI. If possible, try to adjust the settings to accommodate the entire dataset.

2. Increase Load Limit: If you're pulling the data via a connector (e.g., ODBC, JDBC, REST API, etc.), check the settings or configuration files for that connector. Some connectors have a default limit on the number of rows they will retrieve in a single operation (for example, the first 1,000 rows), and this can usually be changed.

3. Pagination: If your data source supports pagination, you may need to make use of this feature. Some sources will only return a certain number of records per page and require you to retrieve additional records by requesting the next page.

4. Split your Query: You could consider splitting your query into multiple smaller queries, each retrieving a portion of the data, and then append the results together in Power BI.

this thread will be helpful.

https://community.fabric.microsoft.com/t5/Desktop/Limit-of-1000-value-reached/m-p/368850

[NiteshS · Accepted solution]

Thank you. Second option worked out for me. 🙂