JUnit validation on cloud metrics during performance tests

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

JUnit validation on cloud metrics during performance tests

Andrew Burton
Hi list,

Does anyone have experience in incorporating cloud metrics (typically AWS
or Azure) as part of their post-test JUnit validation whether via lightning
plugin - https://github.com/automatictester/lightning or some other Maven

In addition to pass/fail tests on sampler response times or error rates, it
would be useful to add pass/fail conditions on such metrics as CPU
utilisation, queue/topic depth, etc. for the period of a test. In addition,
this means we could retain the relevant server stats in the results file
for retrospective analysis.

Most cloud providers provide a Java library to retrieve such stats - the
Azure one is here (
The approach I had in mind was to:
* Add a tearDown thread group with a sampler to retrieve the relevant stats
for the duration of the test (e.g. the past hour)
* For each metric returned, convert this to a SampleResult object and log
to the results file
* Add a relevant JUnit test for the metrics

(An alternate option would be to have this as a Java sampler invoked at
5-10 minute intervals during the test).

Want to make sure I'm not re-inventing the wheel before I start work, and
whether anyone had other ideas or suggestions.