We’re planning to use the Flexmonster Data Server DLL to process large datasets, deployed within an ECS container.
The AWS team has asked whether Flexmonster supports horizontal and/or vertical scaling. If scaling is supported, do we need to make any changes to our build or configuration, or will the server adapt automatically to additional resources or instances?
Additionally, what is the recommended memory allocation to efficiently handle a dataset of ~600MB, containing 40 columns and approximately 600,000 rows?
Hello Varun,
Thank you for reaching out to us.
Flexmonster Data Server DLL can be horizontally scaled across multiple containers with one instance of the Data Server per container. The requests are independent of the server state, allowing requests from a single pivot to be sent to different servers with the same index loaded.
As for the memory allocation, the Data Server typically requires from 2 to 3 times the amount of RAM compared to the original dataset size. However, this number may vary depending on the data structure, cardinality of values in columns, and volume of cached responses. Also, the memory usage may spike on index reload when the KeepDataOnRefresh option is enabled. We recommend testing the performance with your data to get more specific details on memory allocation.
Please let us know if more questions arise.
Best regards,
Maksym