JUnit validation on cloud metrics during performance tests
Does anyone have experience in incorporating cloud metrics (typically AWS
or Azure) as part of their post-test JUnit validation whether via lightning
plugin - https://github.com/automatictester/lightning or some other Maven
In addition to pass/fail tests on sampler response times or error rates, it
would be useful to add pass/fail conditions on such metrics as CPU
utilisation, queue/topic depth, etc. for the period of a test. In addition,
this means we could retain the relevant server stats in the results file
for retrospective analysis.
Most cloud providers provide a Java library to retrieve such stats - the
Azure one is here (
The approach I had in mind was to:
* Add a tearDown thread group with a sampler to retrieve the relevant stats
for the duration of the test (e.g. the past hour)
* For each metric returned, convert this to a SampleResult object and log
to the results file
* Add a relevant JUnit test for the metrics
(An alternate option would be to have this as a Java sampler invoked at
5-10 minute intervals during the test).
Want to make sure I'm not re-inventing the wheel before I start work, and
whether anyone had other ideas or suggestions.