The Art of BenchmarksPerformance benchmarks with GigaSpaces' In-Memory Data Grid mainly involve testing latency and throughput. Since GigaSpaces offers data access routines for remote data access as well as colocated data access, you should test latency and throughput in both cases (as appropriate for your architecture and deployment topology). In addition, you should also test transactional and non-transactional operations, single and batch operations, synchronous and aynchronous options, as all of these are basic building blocks for any application. Benchmarks should test simple and complex queries as GigaSpaces supports both simple and complex data models and provides indexes to speed up query execution time. Another dimension for these tests is concurrency - which measures how the system scales as a function of the amount of concurrent remote users or colocated threads accessing the system. One more dimension is the grid size (the topology). This measures how the system scales as a function of the number of data-grid nodes leveraging GigaSpaces In-Memory-Map-Reduce API. There is also a client side caching component that is viable in many scenarios that is important to test (read-mostly scenarios). As you see the performance benchmark matrix is extensive and requires the benchmark tool to support all these options/scenarios. Benchmark ResultsThe different benchmarks results can be found on the benchmarks page. We update these from time to time and post new benchmarks results on our benchmark blog. Benchmark Tools
Automated Benchmark FrameworkBeyond these tools, we have a powerful Java-based benchmark framework, provided to accounts that are interested in extending their benchmarks beyond the ones provided out of the box. This benchmark framework provides a fully automated environment that supports many of the options and scenarios described above. This framework is used by the GigaSpaces R&D team. To share this framework with you please approach our support team. You can find attached examples for the output this framework can produce. These examples are from testing throughput in remote and colocated (embedded) modes. There is a similar automated benchmark for latency tests. Large Scale BenchmarksNote that once you are running large scale benchmarks with a large number of concurrent accesses, or a large number of data grid partitions, you should consider our involvement to help tune the configuration of the environment and JVM, and in some cases we should help with the code, too, as some of the default behaviors for the basic API have special modifiers that may boost the performance. Many times developers sometime miss or misunderstand these advanced options, due to the vast amount of features the products offers. Please do not hesitate to approach our support team for advanced benchmarking and testing options. The information about tuning the product for such environments is not fully publicly available, since many larger systems have specialized requirements for which configurations are available but for which generalized solutions are not entirely appropriate. |
![]() |
GigaSpaces.com - Legal Notice - 3rd Party Licenses - Site Map - API Docs - Forum - Downloads - Blog - White Papers - Contact Tech Writing - Gen. by Atlassian Confluence |