asked on May 4, 2023
Hello,
in my company we implemented the custom data source api following the solution provided here (api-data-source/server-dotnetcore at master · flexmonster/api-data-source · GitHub ) to optimize the load and interaction with very large data sources, but we are encountering performance issues when users edit cell values and a full data source full reload is required updates.
Thus, my question is: do you have any out-of-the-box feature to handle partial updates in custom data source api? Also suggestions based on the above mentioned solution (or even better code examples) are really welcome.
Thank you
Massimo
⋅ May 4, 2023
Hello, Massimo!
Thank you for your question.
Kindly note that only the whole dataset can be reloaded with a custom data source API.
As for possible performance improvements, we would like to start off by asking some questions about the procedure of editing and updating the data to ensure that we are on the same page. To our understanding, you send the request to modify a single record to your server after each cell editing. After that request, the updateData API call is triggered, sending all the necessary requests to the server, which reloads and aggregates the full dataset from the original data source. Is our understanding of cell editing in your project correct? If so, we suggest checking which part of this logic takes the most time to complete.
Also, we would like to know how the editing is implemented on a client. Are users editing the data in the drill-through, on a flat grid layout, or using a custom input outside the component?
We are looking forwards to hearing your reply.
Best Regards,
Maksym
⋅ May 5, 2023
Hallo Maksym,
first of all thanks for your quick reply.
As for the question you asked, in our implementation of Flexmonster Pivot we changed the DOM of the Cell to allow the user to edit its value. This value can be either a leaf record or an aggregate.
Once the cell is edited, a request to our server starts, which updates the udenrlying dataset. Then data is refreshed also in the client. That's more or less the complete flow.
However, the problem here is that the implementation of the sample solution (the one from github) recalculates the aggregates on the entire dataset and not just the subset affected by the change determining the performance issues I previously mentioned.
Could you suggest us a way for doing such a work, on the server side, only for the affected piece of dataset?
Best regards,
Massimo
⋅ May 8, 2023
Hello, Massimo!
Thank you for sharing more details with us.
Please note that we do not have any similar examples, as the described use case is very specific. The custom API was developed with an emphasis on data visualization, not on editing. Still, it is theoretically possible to implement a solution to partially aggregate the values after editing the cell. We recommend using the cache to achieve this goal. Also, it would be useful to send additional parameters about the edited cell, which can be retrieved by getCell() API call.
Feel free to contact us if further assistance is required.
Best Regards,
Maksym
⋅ May 25, 2023
Hello, Massimo!
Hope you are doing well.
We are wondering if you were able to make some progress in implementing the partial aggregation after editing the cell.
Our team is ready to provide you with assistance.
Best Regards,
Maksym
⋅ May 29, 2023
Hi Maksym,
no we're still blocked on this point.
Unfortunately the amount of data we have to manage is so huge and our customers want to edit this data that for us the use case is not so uncommon.
Currently we still rely on an implementation that uses cache and that is based on this code (https://github.com/flexmonster/api-data-source/blob/master/server-dotnetcore/Controllers/CubeController.cs#L277) but without the desired results, as I said in my previous posts.
In the next days we'll try to understand if there is a way to split up further the SelectReponse calculation logic to achieve the goal of minimizing the aggregation sets and any suggestion will be really welcome.
We're open to share with you our code with a real example of the use case if this could help.
Regards
Massimo
⋅ May 31, 2023
Hello, Massimo,
Thank you for sharing your concerns and current ideas with our team.
We understand that the solution in the custom API implementation example you mentioned may not be the best approach for every case as it doesn't cover specific scenarios. The sample implementation on GitHub is mainly for demonstration purposes to provide an idea of how the protocol could be implemented.
At the same time, our team can't provide precise recommendations when it comes to implementing the server-side calculation logic as this part is specific to every use case and hence should be adapted to your project's needs individually.
However, we will be happy to address any questions you may have on the specifics of the custom data source API protocol.
Kind regards,
Vera
