Response assertion seems to not always set JMeterThread.last_sample_ok correctly

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Response assertion seems to not always set JMeterThread.last_sample_ok correctly

Krister Nilsson
Hello!

I have a testplan sending several different HTTP requests. After each I have a "Response assertion". Some of the HTTP requests turns up RED (i.e. failed) correctly in the View Results Tree due to the assertion but I can see, by logging that the variable JMeterThread.last_sample_ok is not set to false for some of them. That is, for some of the failing requests it is set to false as expected, but for some it is set to true, even though it should and has failed.

The "Response assertions" are set identically for all and the response code is also the same, 200, for all of them. The response content are not identical for them, but they have in common the they do no match the Contains pattern (.*from>\d{4}-\d{2}-\d{2}.*) .

Is there a common configuration mistake made by many for this, or is there a known fault or some other explanation?

Med vänlig hälsning
Krister Nilsson
____________________________________________________________

Krister Nilsson / översteprest, prestandatest

IT Fond konto och teknisk plattform
Telefon direkt 010-454 21 97, mobil 072-210 21 97
[hidden email]<mailto:[hidden email]>

Pensionsmyndigheten,
Telefon vxl 0771-771 771, fax 08-658 13 00, www.pensionsmyndigheten.se<http://www.pensionsmyndigheten.se/>


P Överväg miljöpåverkan innan du skriver ut detta e-brev.

Reply | Threaded
Open this post in threaded view
|

Re: Response assertion seems to not always set JMeterThread.last_sample_ok correctly

glinius@live.com
The questions is: how exactly you're "logging that the variable
JMeterThread.last_sample_ok"?

Be aware of JMeter test elements  execution order
<https://jmeter.apache.org/usermanual/test_plan.html#executionorder>  ,
after Assertions only Listeners are executed so it makes sense to "log" the
value for example using  JSR223 Listener
<https://www.blazemeter.com/blog/groovy-new-black>   and the following code:


> log.info('Previous sampler successful: ' +
> vars.get('JMeterThread.last_sample_ok'))

Also you can log the value using  __log()
<https://jmeter.apache.org/usermanual/functions.html#__log>  or  logn()
<https://jmeter.apache.org/usermanual/functions.html#__logn>   functions
which are evaluated immediately where they found





--
Sent from: http://www.jmeter-archive.org/JMeter-User-f512775.html

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

SV: Response assertion seems to not always set JMeterThread.last_sample_ok correctly

Krister Nilsson

Hello and thanks!

 

If I interpret you correctly I am doing exactly what you are suggesting.

 

 

Also the “Check if all is OK” indicates the same problem as I set a variable in the JS223 PostProcessor depending on the value of JMeterThread.last_sample_ok.

 

BR

/Krister

 

-----Ursprungligt meddelande-----
Från: [hidden email] <[hidden email]>
Skickat: den 1 april 2021 14:05
Till: [hidden email]
Ämne: Re: Response assertion seems to not always set JMeterThread.last_sample_ok correctly

 

The questions is: how exactly you're "logging that the variable JMeterThread.last_sample_ok"?

 

Be aware of JMeter test elements  execution order <https://jmeter.apache.org/usermanual/test_plan.html#executionorder>  , after Assertions only Listeners are executed so it makes sense to "log" the value for example using  JSR223 Listener

<https://www.blazemeter.com/blog/groovy-new-black>   and the following code:

 

 

> log.info('Previous sampler successful: ' +

> vars.get('JMeterThread.last_sample_ok'))

 

Also you can log the value using  __log() <https://jmeter.apache.org/usermanual/functions.html#__log>  or  logn()

<https://jmeter.apache.org/usermanual/functions.html#__logn>   functions

which are evaluated immediately where they found

 

 

 

 

 

--

Sent from: http://www.jmeter-archive.org/JMeter-User-f512775.html

 

---------------------------------------------------------------------

To unsubscribe, e-mail: [hidden email]

For additional commands, e-mail: [hidden email]

 

Reply | Threaded
Open this post in threaded view
|

Re: Response assertion seems to not always set JMeterThread.last_sample_ok correctly

Felix Schumacher
In reply to this post by Krister Nilsson

Am 01.04.21 um 13:43 schrieb Krister Nilsson:
> Hello!
>
> I have a testplan sending several different HTTP requests. After each I have a "Response assertion". Some of the HTTP requests turns up RED (i.e. failed) correctly in the View Results Tree due to the assertion but I can see, by logging that the variable JMeterThread.last_sample_ok is not set to false for some of them. That is, for some of the failing requests it is set to false as expected, but for some it is set to true, even though it should and has failed.
>
> The "Response assertions" are set identically for all and the response code is also the same, 200, for all of them. The response content are not identical for them, but they have in common the they do no match the Contains pattern (.*from>\d{4}-\d{2}-\d{2}.*) .
>
> Is there a common configuration mistake made by many for this, or is there a known fault or some other explanation?

Can you post a simple test plan that exhibits the described behaviour?

Are they always failing/succeeding or is it a bit more flaky?

Can you see other differences in a View Results Tree element, when you
compare the failed/succeeded results?

Felix

>
> Med vänlig hälsning
> Krister Nilsson
> ____________________________________________________________
>
> Krister Nilsson / översteprest, prestandatest
>
> IT Fond konto och teknisk plattform
> Telefon direkt 010-454 21 97, mobil 072-210 21 97
> [hidden email]<mailto:[hidden email]>
>
> Pensionsmyndigheten,
> Telefon vxl 0771-771 771, fax 08-658 13 00, www.pensionsmyndigheten.se<http://www.pensionsmyndigheten.se/>
>
>
> P Överväg miljöpåverkan innan du skriver ut detta e-brev.
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

SV: Response assertion seems to not always set JMeterThread.last_sample_ok correctly

Krister Nilsson

Hi Felix!

 

The requests/responses that fails, trigger the assertion and sets last_sample_ok = false, always do that. (Checked upto 1000 consecutive times.)

The requests/responses that fails, trigger the assertion and sets last_sample_ok = true, always do that. (Checked fewer times but I’m quite sure.)

 

The latter of these two is of course the problem.

 

Regarding differences between the ones that behaves ok and the ones that do not, I see none in the View Results Tree. Just differing characters like tx id, but no principal difference.

 

I cannot post a simple test plan, but perhaps the screen shot below shows you what is going on!?

The purpose is to find a "customer id" that gives a response which matches the regexp, and I want that to be the case for each request in one while loop. (This customer id will later on be used in another test, this is just a brute trial-and-error way of finding the test data for that test.) If each response is OK I will write the test data to a file and continue to the next group of requests in the test plan, i.e. to the next while loop.

 

 

BR

/Krister

 

-----Ursprungligt meddelande-----
Från: Felix Schumacher <[hidden email]>
Skickat: den 1 april 2021 15:21
Till: [hidden email]
Ämne: Re: Response assertion seems to not always set JMeterThread.last_sample_ok correctly

 

 

Am 01.04.21 um 13:43 schrieb Krister Nilsson:

> Hello!

> 

> I have a testplan sending several different HTTP requests. After each I have a "Response assertion". Some of the HTTP requests turns up RED (i.e. failed) correctly in the View Results Tree due to the assertion but I can see, by logging that the variable JMeterThread.last_sample_ok is not set to false for some of them. That is, for some of the failing requests it is set to false as expected, but for some it is set to true, even though it should and has failed.

> 

> The "Response assertions" are set identically for all and the response code is also the same, 200, for all of them. The response content are not identical for them, but they have in common the they do no match the Contains pattern (.*from>\d{4}-\d{2}-\d{2}.*) .

> 

> Is there a common configuration mistake made by many for this, or is there a known fault or some other explanation?

 

Can you post a simple test plan that exhibits the described behaviour?

 

Are they always failing/succeeding or is it a bit more flaky?

 

Can you see other differences in a View Results Tree element, when you compare the failed/succeeded results?

 

Felix

 

> 

> Med vänlig hälsning

> Krister Nilsson

> ____________________________________________________________

> 

> Krister Nilsson / översteprest, prestandatest

> 

> IT Fond konto och teknisk plattform

> Telefon direkt 010-454 21 97, mobil 072-210 21 97

> [hidden email]

> myndigheten.se>

> 

> Pensionsmyndigheten,

> Telefon vxl 0771-771 771, fax 08-658 13 00,

> www.pensionsmyndigheten.se<http://www.pensionsmyndigheten.se/>

> 

> 

> P Överväg miljöpåverkan innan du skriver ut detta e-brev.

> 

> 

 

---------------------------------------------------------------------

To unsubscribe, e-mail: [hidden email]

For additional commands, e-mail: [hidden email]

 

Reply | Threaded
Open this post in threaded view
|

SV: Response assertion seems to not always set JMeterThread.last_sample_ok correctly

Krister Nilsson

Well, forgot, there is the obvious difference that the requests have different end points which may take shorter or longer to respond.

Sorry for forgetting… 😊

 

Från: Krister Nilsson
Skickat: den 1 april 2021 17:33
Till: [hidden email]
Ämne: SV: Response assertion seems to not always set JMeterThread.last_sample_ok correctly

 

Hi Felix!

 

The requests/responses that fails, trigger the assertion and sets last_sample_ok = false, always do that. (Checked upto 1000 consecutive times.)

The requests/responses that fails, trigger the assertion and sets last_sample_ok = true, always do that. (Checked fewer times but I’m quite sure.)

 

The latter of these two is of course the problem.

 

Regarding differences between the ones that behaves ok and the ones that do not, I see none in the View Results Tree. Just differing characters like tx id, but no principal difference.

 

I cannot post a simple test plan, but perhaps the screen shot below shows you what is going on!?

The purpose is to find a "customer id" that gives a response which matches the regexp, and I want that to be the case for each request in one while loop. (This customer id will later on be used in another test, this is just a brute trial-and-error way of finding the test data for that test.) If each response is OK I will write the test data to a file and continue to the next group of requests in the test plan, i.e. to the next while loop.

 

 

BR

/Krister

 

-----Ursprungligt meddelande-----
Från: Felix Schumacher <[hidden email]>
Skickat: den 1 april 2021 15:21
Till: [hidden email]
Ämne: Re: Response assertion seems to not always set JMeterThread.last_sample_ok correctly

 

 

Am 01.04.21 um 13:43 schrieb Krister Nilsson:

> Hello!

> 

> I have a testplan sending several different HTTP requests. After each I have a "Response assertion". Some of the HTTP requests turns up RED (i.e. failed) correctly in the View Results Tree due to the assertion but I can see, by logging that the variable JMeterThread.last_sample_ok is not set to false for some of them. That is, for some of the failing requests it is set to false as expected, but for some it is set to true, even though it should and has failed.

> 

> The "Response assertions" are set identically for all and the response code is also the same, 200, for all of them. The response content are not identical for them, but they have in common the they do no match the Contains pattern (.*from>\d{4}-\d{2}-\d{2}.*) .

> 

> Is there a common configuration mistake made by many for this, or is there a known fault or some other explanation?

 

Can you post a simple test plan that exhibits the described behaviour?

 

Are they always failing/succeeding or is it a bit more flaky?

 

Can you see other differences in a View Results Tree element, when you compare the failed/succeeded results?

 

Felix

 

> 

> Med vänlig hälsning

> Krister Nilsson

> ____________________________________________________________

> 

> Krister Nilsson / översteprest, prestandatest

> 

> IT Fond konto och teknisk plattform

> Telefon direkt 010-454 21 97, mobil 072-210 21 97

> [hidden email]

> myndigheten.se>

> 

> Pensionsmyndigheten,

> Telefon vxl 0771-771 771, fax 08-658 13 00,

> www.pensionsmyndigheten.se<http://www.pensionsmyndigheten.se/>

> 

> 

> P Överväg miljöpåverkan innan du skriver ut detta e-brev.

> 

> 

 

---------------------------------------------------------------------

To unsubscribe, e-mail: [hidden email]

For additional commands, e-mail: [hidden email]

 

Reply | Threaded
Open this post in threaded view
|

Re: SV: Response assertion seems to not always set JMeterThread.last_sample_ok correctly

glinius@live.com
In reply to this post by Krister Nilsson
I might be suffering from a form of a mental fatigue but I pretty much sure I
posted the link to the JMeter test elements  execution order
<https://jmeter.apache.org/usermanual/test_plan.html#executionorder>  

Response assertion is executed *AFTER* the JSR223 PostProcessor so if the
assertion fails the request - you will get false positive result.

You need to change it to the  JSR223 Assertion
<https://www.blazemeter.com/blog/scripting-jmeter-assertions-in-groovy-a-tutorial>  
(but make sure to place it *below* the Response Assertion as they're
executed upside down) or switch to the  JSR223 Listener
<https://jmeter.apache.org/usermanual/component_reference.html#JSR223_Listener>
.



--
Sent from: http://www.jmeter-archive.org/JMeter-User-f512775.html

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SV: Response assertion seems to not always set JMeterThread.last_sample_ok correctly

Felix Schumacher
In reply to this post by Krister Nilsson


Am 1. April 2021 17:32:46 MESZ schrieb Krister Nilsson <[hidden email]>:

>Hi Felix!
>
>
>
>The requests/responses that fails, trigger the assertion and sets
>last_sample_ok = false, always do that. (Checked upto 1000 consecutive
>times.)
>
>The requests/responses that fails, trigger the assertion and sets
>last_sample_ok = true, always do that. (Checked fewer times but I’m
>quite sure.)
>
>
>
>The latter of these two is of course the problem.
>

Which is which in the picture?

One of the loops has three, while the other has two samplers. Could that be the difference?

>
>
>Regarding differences between the ones that behaves ok and the ones
>that do not, I see none in the View Results Tree. Just differing
>characters like tx id, but no principal difference.

How are the whole controllers configured?


Felix

>
>
>
>I cannot post a simple test plan, but perhaps the screen shot below
>shows you what is going on!?
>
>The purpose is to find a "customer id" that gives a response which
>matches the regexp, and I want that to be the case for each request in
>one while loop. (This customer id will later on be used in another
>test, this is just a brute trial-and-error way of finding the test data
>for that test.) If each response is OK I will write the test data to a
>file and continue to the next group of requests in the test plan, i.e.
>to the next while loop.
>
>
>
>[cid:image001.png@01D7271D.06CCF830]
>
>
>
>BR
>
>/Krister
>
>
>
>-----Ursprungligt meddelande-----
>Från: Felix Schumacher <[hidden email]>
>Skickat: den 1 april 2021 15:21
>Till: [hidden email]
>Ämne: Re: Response assertion seems to not always set
>JMeterThread.last_sample_ok correctly
>
>
>
>
>
>Am 01.04.21 um 13:43 schrieb Krister Nilsson:
>
>> Hello!
>
>>
>
>> I have a testplan sending several different HTTP requests. After each
>I have a "Response assertion". Some of the HTTP requests turns up RED
>(i.e. failed) correctly in the View Results Tree due to the assertion
>but I can see, by logging that the variable JMeterThread.last_sample_ok
>is not set to false for some of them. That is, for some of the failing
>requests it is set to false as expected, but for some it is set to
>true, even though it should and has failed.
>
>>
>
>> The "Response assertions" are set identically for all and the
>response code is also the same, 200, for all of them. The response
>content are not identical for them, but they have in common the they do
>no match the Contains pattern (.*from>\d{4}-\d{2}-\d{2}.*) .
>
>>
>
>> Is there a common configuration mistake made by many for this, or is
>there a known fault or some other explanation?
>
>
>
>Can you post a simple test plan that exhibits the described behaviour?
>
>
>
>Are they always failing/succeeding or is it a bit more flaky?
>
>
>
>Can you see other differences in a View Results Tree element, when you
>compare the failed/succeeded results?
>
>
>
>Felix
>
>
>
>>
>
>> Med vänlig hälsning
>
>> Krister Nilsson
>
>> ____________________________________________________________
>
>>
>
>> Krister Nilsson / översteprest, prestandatest
>
>>
>
>> IT Fond konto och teknisk plattform
>
>> Telefon direkt 010-454 21 97, mobil 072-210 21 97
>
>>
>[hidden email]<mailto:krister.nilsson@pensions<mailto:[hidden email]%3cmailto:krister.nilsson@pensions>
>
>> myndigheten.se>
>
>>
>
>> Pensionsmyndigheten,
>
>> Telefon vxl 0771-771 771, fax 08-658 13 00,
>
>>
>www.pensionsmyndigheten.se<http://www.pensionsmyndigheten.se/<http://www.pensionsmyndigheten.se%3chttp:/www.pensionsmyndigheten.se/>>
>
>>
>
>>
>
>> P Överväg miljöpåverkan innan du skriver ut detta e-brev.
>
>>
>
>>
>
>
>
>---------------------------------------------------------------------
>
>To unsubscribe, e-mail:
>[hidden email]<mailto:[hidden email]>
>
>For additional commands, e-mail:
>[hidden email]<mailto:[hidden email]>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: SV: Response assertion seems to not always set JMeterThread.last_sample_ok correctly

Felix Schumacher
In reply to this post by glinius@live.com


Am 1. April 2021 17:39:46 MESZ schrieb "[hidden email]" <[hidden email]>:
>I might be suffering from a form of a mental fatigue but I pretty much
>sure I
>posted the link to the JMeter test elements  execution order
><https://jmeter.apache.org/usermanual/test_plan.html#executionorder>  
>
>Response assertion is executed *AFTER* the JSR223 PostProcessor so if
>the
>assertion fails the request - you will get false positive result.

Right.

>
>You need to change it to the  JSR223 Assertion
><https://www.blazemeter.com/blog/scripting-jmeter-assertions-in-groovy-a-tutorial>
>
>(but make sure to place it *below* the Response Assertion as they're
>executed upside down) or switch to the  JSR223 Listener
><https://jmeter.apache.org/usermanual/component_reference.html#JSR223_Listener>

I remember, that the listener might have surprising results, when used in a transaction controller (which is hopefully not the case here). But it is be late eough in the game to have all information needed.

Thanks for your persistence
 Felix

>
>.
>
>
>
>--
>Sent from: http://www.jmeter-archive.org/JMeter-User-f512775.html
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: [hidden email]
>For additional commands, e-mail: [hidden email]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]