We have updated Flexmonster Software License Agreement, effective as of September 30, 2024. Learn more about what’s changed.

how to improve connection resilience when loading large amount of records via json list?

Answered
Jeff Morsch asked on May 4, 2020

We currently load data from SQL into a list<dynamic> and json it and feed that to the FlexMonster Pivot Table via ajax
This works great onsite and from places with very solid internet.
But when an end user has poor internet that initial jsonresult download (which can be up to 100mb) often gets a ERR_CONNECTION_CLOSED, and the process never finishes.
is there a better way to handle this, or a way to implement a "resume" or "reconnect" when the connection is lost or closed?
Thanks
Technologies:
C#
MVC
SQL Server
 

8 answers

Public
Vera Didenko Vera Didenko Flexmonster May 6, 2020

Hello, Jeff,
 
Thank you for reaching out to us. 
 
Our team kindly recommends trying the recently released Flexmonster Data Server, which is a new tool for connecting to databases.
It allows handling large data sets from different data sources, including Microsoft SQL Server database.
Using the Flexmonster Data Server could also provide a more stable connection.
 
We would like to provide some additional details on the way the data is loaded to the component when using the Data Server and why it is recommended to use it for large data sets.
 
The main advantage of such an approach is the fact that only the required for the current display part of the data set is loaded instead of loading the whole data set before it is displayed on the grid.
 
For more information about the Flexmonster Data Server and its usage please see the following guides:
1. Introduction to the Flexmonster Data Server
2. Installation guide
3. Connecting to the SQL Server database with the Flexmonster Data Server.
 
Please let us know if this would work for you.
Looking forward to your response.
 
Kind regards,
Vera

Public
Vera Didenko Vera Didenko Flexmonster May 28, 2020

Hello, Jeff,
 
How are you?
 
We are checking in to ask whether you've found our previous response about using the Flexmonster Data Server helpful. Do you think the described approach would be suitable in your situation?
 
Please let us know what you think.
Looking forward to your response.
 
Kind regards,
Vera

Public
Jeff Morsch May 28, 2020

Hi,  
We have investigated the data server, but unfortunately it will not work for our application currently. 
We are using parameters in our queries for hundreds of reports.  We cannot cache the entire database.  Plus users can define new data queries dynamically in the application, which I'm not sure how we'd accomplish with the data server.
We are looking at some potential uses of the data server, however, and may make use of it in a different area of our web application.

Public
Vera Didenko Vera Didenko Flexmonster May 28, 2020

Hello, Jeff,
 
Thank you for your reply.
 
In case a more custom approach is needed, we highly recommend implementing the custom data source API, which is a protocol developed by our team that the Flexmonster Data Server is built on.
The custom data source API greatly improves the performance and provides full control over how the data is processed. With this approach, it is possible to implement your own server-side connector to the data source of your choice and feed the data into Flexmonster already in the aggregated form.
For more information about how this can be done, please see the following guides: 
1) How to implement your own implementation of the Custom Data Source API approach: https://www.flexmonster.com/doc/implement-custom-data-source-api/
2) A sample implementation of the Custom Data Source API approach with Node.js: https://www.flexmonster.com/doc/pivot-table-with-node-js-server/
3) A sample implementation of the Custom Data Source API approach with .Net Core: https://www.flexmonster.com/doc/pivot-table-with-dot-net-core-server/
 
As for applying the Flexmonster Data Server in a different area of your web application, if questions arise along the way, feel free to reach out.
 
Please let us know if this would work for you.
Looking forward to your response.
 
Kind regards, 
Vera

Public
Vera Didenko Vera Didenko Flexmonster June 17, 2020

Hello, Jeff,
 
How have you been?
 
Our team would like to kindly take an interest if you've found our previous response about implementing the custom data source API helpful. Did it work for you or perhaps you still have some questions that we could help with?
 
Please share your thoughts with us.
Looking forward to your response.
 
Kind regards,
Vera

Public
Vera Didenko Vera Didenko Flexmonster July 8, 2020

Hello, Jeff,
 
How have you been?
 
We are wondering if the custom data source API approach worked for your case. 
 
Please let us know what you think.
Looking forward to your feedback.
 
Kind regards,
Vera

Public
Jeff Morsch July 8, 2020

Sorry for the delayed feedback.  We have not had time to develop our own custom approach using the API.  We are currently functioning with the original deployment, which works well as long as the user has a very good internet connection.  (which the majority of our user base has.)  We will return to this project in the future, but have higher priorities currently.
 
Thank you for your diligence in attempting to address my questions.
Regards,
Jeff

Public
Vera Didenko Vera Didenko Flexmonster July 8, 2020

Hello, Jeff, 
 
Thank you for your kind feedback. 

You are welcome to write to us if further questions arise in the meantime.

Kind regards, 
Vera

Please login or Register to Submit Answer