Hello!
I have a testplan sending several different HTTP requests. After each I have a "Response assertion". Some of the HTTP requests turns up RED (i.e. failed) correctly in the View Results Tree due to the assertion but I can see, by logging that the variable JMeterThread.last_sample_ok is not set to false for some of them. That is, for some of the failing requests it is set to false as expected, but for some it is set to true, even though it should and has failed. The "Response assertions" are set identically for all and the response code is also the same, 200, for all of them. The response content are not identical for them, but they have in common the they do no match the Contains pattern (.*from>\d{4}-\d{2}-\d{2}.*) . Is there a common configuration mistake made by many for this, or is there a known fault or some other explanation? Med vänlig hälsning Krister Nilsson ____________________________________________________________ Krister Nilsson / översteprest, prestandatest IT Fond konto och teknisk plattform Telefon direkt 010-454 21 97, mobil 072-210 21 97 [hidden email]<mailto:[hidden email]> Pensionsmyndigheten, Telefon vxl 0771-771 771, fax 08-658 13 00, www.pensionsmyndigheten.se<http://www.pensionsmyndigheten.se/> P Överväg miljöpåverkan innan du skriver ut detta e-brev. |
The questions is: how exactly you're "logging that the variable
JMeterThread.last_sample_ok"? Be aware of JMeter test elements execution order <https://jmeter.apache.org/usermanual/test_plan.html#executionorder> , after Assertions only Listeners are executed so it makes sense to "log" the value for example using JSR223 Listener <https://www.blazemeter.com/blog/groovy-new-black> and the following code: > log.info('Previous sampler successful: ' + > vars.get('JMeterThread.last_sample_ok')) Also you can log the value using __log() <https://jmeter.apache.org/usermanual/functions.html#__log> or logn() <https://jmeter.apache.org/usermanual/functions.html#__logn> functions which are evaluated immediately where they found -- Sent from: http://www.jmeter-archive.org/JMeter-User-f512775.html --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
Hello and thanks! If I interpret you correctly I am doing exactly what you are suggesting. Also the “Check if all is OK” indicates the same problem as I set a variable in the JS223 PostProcessor depending on the value of
JMeterThread.last_sample_ok. BR /Krister -----Ursprungligt meddelande----- The questions is: how exactly you're "logging that the variable JMeterThread.last_sample_ok"?
Be aware of JMeter test elements execution order <https://jmeter.apache.org/usermanual/test_plan.html#executionorder>
, after Assertions only Listeners are executed so it makes sense to "log" the value for example using JSR223 Listener <https://www.blazemeter.com/blog/groovy-new-black> and the following code: > log.info('Previous sampler successful: ' + > vars.get('JMeterThread.last_sample_ok')) Also you can log the value using __log() <https://jmeter.apache.org/usermanual/functions.html#__log>
or logn() <https://jmeter.apache.org/usermanual/functions.html#__logn> functions which are evaluated immediately where they found -- Sent from:
http://www.jmeter-archive.org/JMeter-User-f512775.html --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
In reply to this post by Krister Nilsson
Am 01.04.21 um 13:43 schrieb Krister Nilsson: > Hello! > > I have a testplan sending several different HTTP requests. After each I have a "Response assertion". Some of the HTTP requests turns up RED (i.e. failed) correctly in the View Results Tree due to the assertion but I can see, by logging that the variable JMeterThread.last_sample_ok is not set to false for some of them. That is, for some of the failing requests it is set to false as expected, but for some it is set to true, even though it should and has failed. > > The "Response assertions" are set identically for all and the response code is also the same, 200, for all of them. The response content are not identical for them, but they have in common the they do no match the Contains pattern (.*from>\d{4}-\d{2}-\d{2}.*) . > > Is there a common configuration mistake made by many for this, or is there a known fault or some other explanation? Can you post a simple test plan that exhibits the described behaviour? Are they always failing/succeeding or is it a bit more flaky? Can you see other differences in a View Results Tree element, when you compare the failed/succeeded results? Felix > > Med vänlig hälsning > Krister Nilsson > ____________________________________________________________ > > Krister Nilsson / översteprest, prestandatest > > IT Fond konto och teknisk plattform > Telefon direkt 010-454 21 97, mobil 072-210 21 97 > [hidden email]<mailto:[hidden email]> > > Pensionsmyndigheten, > Telefon vxl 0771-771 771, fax 08-658 13 00, www.pensionsmyndigheten.se<http://www.pensionsmyndigheten.se/> > > > P Överväg miljöpåverkan innan du skriver ut detta e-brev. > > --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
Hi Felix! The requests/responses that fails, trigger the assertion and sets last_sample_ok = false, always do that. (Checked upto 1000 consecutive times.) The requests/responses that fails, trigger the assertion and sets last_sample_ok = true, always do that. (Checked fewer times but I’m quite sure.) The latter of these two is of course the problem. Regarding differences between the ones that behaves ok and the ones that do not, I see none in the View Results Tree. Just differing characters like tx id, but no principal difference. I cannot post a simple test plan, but perhaps the screen shot below shows you what is going on!? The purpose is to find a "customer id" that gives a response which matches the regexp, and I want that to be the case for each request in one while loop. (This customer id will later on be used in another test, this
is just a brute trial-and-error way of finding the test data for that test.) If each response is OK I will write the test data to a file and continue to the next group of requests in the test plan, i.e. to the next while loop. BR /Krister -----Ursprungligt meddelande----- Am 01.04.21 um 13:43 schrieb Krister Nilsson: > Hello! > > I have a testplan sending several different HTTP requests. After each I have a "Response assertion". Some of the HTTP requests turns up RED (i.e. failed) correctly in the View Results Tree due to the assertion but I can see, by logging
that the variable JMeterThread.last_sample_ok is not set to false for some of them. That is, for some of the failing requests it is set to false as expected, but for some it is set to true, even though it should and has failed. > > The "Response assertions" are set identically for all and the response code is also the same, 200, for all of them. The response content are not identical for them, but they have in common the they do no match the Contains pattern
(.*from>\d{4}-\d{2}-\d{2}.*) . > > Is there a common configuration mistake made by many for this, or is there a known fault or some other explanation? Can you post a simple test plan that exhibits the described behaviour? Are they always failing/succeeding or is it a bit more flaky? Can you see other differences in a View Results Tree element, when you compare the failed/succeeded results? Felix > > Med vänlig hälsning > Krister Nilsson > ____________________________________________________________ > > Krister Nilsson / översteprest, prestandatest > > IT Fond konto och teknisk plattform > Telefon direkt 010-454 21 97, mobil 072-210 21 97 > myndigheten.se> > > Pensionsmyndigheten, > Telefon vxl 0771-771 771, fax 08-658 13 00, >
www.pensionsmyndigheten.se<http://www.pensionsmyndigheten.se/> > > > P Överväg miljöpåverkan innan du skriver ut detta e-brev. > > --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
Well, forgot, there is the obvious difference that the requests have different end points which may take shorter or longer to respond. Sorry for forgetting… 😊 Från: Krister Nilsson
Hi Felix! The requests/responses that fails, trigger the assertion and sets last_sample_ok = false, always do that. (Checked upto 1000 consecutive times.) The requests/responses that fails, trigger the assertion and sets last_sample_ok = true, always do that. (Checked fewer times but I’m quite sure.) The latter of these two is of course the problem. Regarding differences between the ones that behaves ok and the ones that do not, I see none in the View Results Tree. Just differing characters like tx id, but no principal difference. I cannot post a simple test plan, but perhaps the screen shot below shows you what is going on!? The purpose is to find a "customer id" that gives a response which matches the regexp, and I want that to be the case for each request in one while loop. (This customer id will later on be used in another test, this
is just a brute trial-and-error way of finding the test data for that test.) If each response is OK I will write the test data to a file and continue to the next group of requests in the test plan, i.e. to the next while loop. BR /Krister -----Ursprungligt meddelande----- Am 01.04.21 um 13:43 schrieb Krister Nilsson: > Hello! > > I have a testplan sending several different HTTP requests. After each I have a "Response assertion". Some of the HTTP requests turns up RED (i.e. failed) correctly in the View Results Tree due to the assertion but I can see, by logging
that the variable JMeterThread.last_sample_ok is not set to false for some of them. That is, for some of the failing requests it is set to false as expected, but for some it is set to true, even though it should and has failed. > > The "Response assertions" are set identically for all and the response code is also the same, 200, for all of them. The response content are not identical for them, but they have in common the they do no match the Contains pattern
(.*from>\d{4}-\d{2}-\d{2}.*) . > > Is there a common configuration mistake made by many for this, or is there a known fault or some other explanation? Can you post a simple test plan that exhibits the described behaviour? Are they always failing/succeeding or is it a bit more flaky? Can you see other differences in a View Results Tree element, when you compare the failed/succeeded results? Felix > > Med vänlig hälsning > Krister Nilsson > ____________________________________________________________ > > Krister Nilsson / översteprest, prestandatest > > IT Fond konto och teknisk plattform > Telefon direkt 010-454 21 97, mobil 072-210 21 97 > myndigheten.se> > > Pensionsmyndigheten, > Telefon vxl 0771-771 771, fax 08-658 13 00, >
www.pensionsmyndigheten.se<http://www.pensionsmyndigheten.se/> > > > P Överväg miljöpåverkan innan du skriver ut detta e-brev. > > --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
In reply to this post by Krister Nilsson
I might be suffering from a form of a mental fatigue but I pretty much sure I
posted the link to the JMeter test elements execution order <https://jmeter.apache.org/usermanual/test_plan.html#executionorder> Response assertion is executed *AFTER* the JSR223 PostProcessor so if the assertion fails the request - you will get false positive result. You need to change it to the JSR223 Assertion <https://www.blazemeter.com/blog/scripting-jmeter-assertions-in-groovy-a-tutorial> (but make sure to place it *below* the Response Assertion as they're executed upside down) or switch to the JSR223 Listener <https://jmeter.apache.org/usermanual/component_reference.html#JSR223_Listener> . -- Sent from: http://www.jmeter-archive.org/JMeter-User-f512775.html --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
In reply to this post by Krister Nilsson
Am 1. April 2021 17:32:46 MESZ schrieb Krister Nilsson <[hidden email]>: >Hi Felix! > > > >The requests/responses that fails, trigger the assertion and sets >last_sample_ok = false, always do that. (Checked upto 1000 consecutive >times.) > >The requests/responses that fails, trigger the assertion and sets >last_sample_ok = true, always do that. (Checked fewer times but I’m >quite sure.) > > > >The latter of these two is of course the problem. > Which is which in the picture? One of the loops has three, while the other has two samplers. Could that be the difference? > > >Regarding differences between the ones that behaves ok and the ones >that do not, I see none in the View Results Tree. Just differing >characters like tx id, but no principal difference. How are the whole controllers configured? Felix > > > >I cannot post a simple test plan, but perhaps the screen shot below >shows you what is going on!? > >The purpose is to find a "customer id" that gives a response which >matches the regexp, and I want that to be the case for each request in >one while loop. (This customer id will later on be used in another >test, this is just a brute trial-and-error way of finding the test data >for that test.) If each response is OK I will write the test data to a >file and continue to the next group of requests in the test plan, i.e. >to the next while loop. > > > >[cid:image001.png@01D7271D.06CCF830] > > > >BR > >/Krister > > > >-----Ursprungligt meddelande----- >Från: Felix Schumacher <[hidden email]> >Skickat: den 1 april 2021 15:21 >Till: [hidden email] >Ämne: Re: Response assertion seems to not always set >JMeterThread.last_sample_ok correctly > > > > > >Am 01.04.21 um 13:43 schrieb Krister Nilsson: > >> Hello! > >> > >> I have a testplan sending several different HTTP requests. After each >I have a "Response assertion". Some of the HTTP requests turns up RED >(i.e. failed) correctly in the View Results Tree due to the assertion >but I can see, by logging that the variable JMeterThread.last_sample_ok >is not set to false for some of them. That is, for some of the failing >requests it is set to false as expected, but for some it is set to >true, even though it should and has failed. > >> > >> The "Response assertions" are set identically for all and the >response code is also the same, 200, for all of them. The response >content are not identical for them, but they have in common the they do >no match the Contains pattern (.*from>\d{4}-\d{2}-\d{2}.*) . > >> > >> Is there a common configuration mistake made by many for this, or is >there a known fault or some other explanation? > > > >Can you post a simple test plan that exhibits the described behaviour? > > > >Are they always failing/succeeding or is it a bit more flaky? > > > >Can you see other differences in a View Results Tree element, when you >compare the failed/succeeded results? > > > >Felix > > > >> > >> Med vänlig hälsning > >> Krister Nilsson > >> ____________________________________________________________ > >> > >> Krister Nilsson / översteprest, prestandatest > >> > >> IT Fond konto och teknisk plattform > >> Telefon direkt 010-454 21 97, mobil 072-210 21 97 > >> >[hidden email]<mailto:krister.nilsson@pensions<mailto:[hidden email]%3cmailto:krister.nilsson@pensions> > >> myndigheten.se> > >> > >> Pensionsmyndigheten, > >> Telefon vxl 0771-771 771, fax 08-658 13 00, > >> >www.pensionsmyndigheten.se<http://www.pensionsmyndigheten.se/<http://www.pensionsmyndigheten.se%3chttp:/www.pensionsmyndigheten.se/>> > >> > >> > >> P Överväg miljöpåverkan innan du skriver ut detta e-brev. > >> > >> > > > >--------------------------------------------------------------------- > >To unsubscribe, e-mail: >[hidden email]<mailto:[hidden email]> > >For additional commands, e-mail: >[hidden email]<mailto:[hidden email]> --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
In reply to this post by glinius@live.com
Am 1. April 2021 17:39:46 MESZ schrieb "[hidden email]" <[hidden email]>: >I might be suffering from a form of a mental fatigue but I pretty much >sure I >posted the link to the JMeter test elements execution order ><https://jmeter.apache.org/usermanual/test_plan.html#executionorder> > >Response assertion is executed *AFTER* the JSR223 PostProcessor so if >the >assertion fails the request - you will get false positive result. Right. > >You need to change it to the JSR223 Assertion ><https://www.blazemeter.com/blog/scripting-jmeter-assertions-in-groovy-a-tutorial> > >(but make sure to place it *below* the Response Assertion as they're >executed upside down) or switch to the JSR223 Listener ><https://jmeter.apache.org/usermanual/component_reference.html#JSR223_Listener> I remember, that the listener might have surprising results, when used in a transaction controller (which is hopefully not the case here). But it is be late eough in the game to have all information needed. Thanks for your persistence Felix > >. > > > >-- >Sent from: http://www.jmeter-archive.org/JMeter-User-f512775.html > >--------------------------------------------------------------------- >To unsubscribe, e-mail: [hidden email] >For additional commands, e-mail: [hidden email] --------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] |
Free forum by Nabble | Edit this page |