Heterogeneity and characteristic testing – Powered by Ericsson
Cloud technology is continuously developing and we are clearly heading in the direction of increasing heterogeneity. ARM is carving out a larger and larger chunk from the cloud market, but the real change is happening in the accelerator segment. If we want to write energy-efficient applications, they need to use the relevant framework and be tested on every supported HW combination.
Cloud infrastructure is already very sensitive to configuration settings—one single modification can generate huge differences in the application performance. In addition, as cloud providers try to differentiate themselves, they provide different services on different infrastructures, powered by different HW accelerators. This means it is harder and harder to predict application performance in a production environment. The application must be prepared for more and more scenarios with different CPU architectures, different accelerators, different environment settings, and different external services.
Heterogeneity means exponentially increasing testing needs. Especially painful is to run the typically slow and expensive characteristic testing, but to sign SLA this is unavoidable. Similarly, target environments must be tested, too, so we know how our application will perform and how to dimension it. This usually means some kind of benchmarking, but it is not trivial whether the benchmark has the same footprint as the main application or not. As you can see, new technology comes with new challenges, and requires new ideas and new methods.