How to diagnose NoHttpResponseException

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

How to diagnose NoHttpResponseException

Antony Bowesman
I am using JMeter 3.1 and HttpClient4
I have the following httpclient settings in user.properties

httpclient4.retrycount=3
httpclient4.validate_after_inactivity=20000
httpclient4.time_to_live=1800000

I have 5 load generators and each generator is running approx. 6,000 VU across a number of different samplers.

We are migrating away from a commercial product, so in addition to this load, I have 3 commercial product load generators running the same profile and the effective server load is 75% JMeter and 25% other

We do not have any problems with the commercial product, which we have been using for 3 years.

CPU and memory are fine on the Linux JMeter boxes, but I am seeing frequent NoHttpResponseException errors

I know that the retry logic is happening as in the test I can see these messages appearing and there are far more of these than actual errors and it's logging my retry count as 3

2017/08/07 14:36:09 INFO  - org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl$7: I/O exception (java.net.SocketException) caught when processing request to {s}-><a href="https://host:443:">https://host:443: Connection reset
2017/08/07 14:36:09 INFO  - org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl$7: Retrying request to {s}->https://host:443

So, I've simplified the test down to a single machine and enabled debug and run the test until I see the first error.  I get this in the log, but there is no retry logged

2017/08/07 16:06:32 DEBUG - jmeter.protocol.http.control.CacheManager: GET(OAH) https://host/experience/racing/upcomingracing null
2017/08/07 16:06:32 DEBUG - jmeter.protocol.http.control.HC4CookieHandler: Found 6 cookies for https:// host/experience/racing/upcomingracing
2017/08/07 16:06:32 DEBUG - jmeter.protocol.http.control.CacheManager: inCache https:// host/experience/racing/upcomingracing null
2017/08/07 16:06:32 DEBUG - jmeter.protocol.http.sampler.HTTPHC4Impl: IOException org.apache.http.NoHttpResponseException: host:443 failed to respond
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
        at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
        at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:281)
        at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:259)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:209)
        at org.apache.jmeter.protocol.http.sampler.MeasuringConnectionManager$MeasuredConnection.receiveResponseHeader(MeasuringConnectionManager.java:212)
        at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
        at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:686)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:488)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.executeRequest(HTTPHC4Impl.java:654)
        at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:413)
        at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:74)
        at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1166)
        at au.com.williamhill.perftest.endpoint.Browser.sendRequest(Browser.java:202)
        at au.com.williamhill.perftest.jmeter.samplers.Command.sendRequest(Command.java:462)
        at au.com.williamhill.perftest.jmeter.samplers.Command.sendUrls(Command.java:531)
        at au.com.williamhill.perftest.jmeter.samplers.Command.postUrls(Command.java:271)
        at au.com.williamhill.perftest.jmeter.samplers.ViewRace.viewRace(ViewRace.java:162)
        at au.com.williamhill.perftest.jmeter.samplers.ViewRace.runTest(ViewRace.java:113)
        at org.apache.jmeter.protocol.java.sampler.JavaSampler.sample(JavaSampler.java:196)
        at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:475)
        at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:418)
        at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:249)
        at java.lang.Thread.run(Thread.java:748)

The request sequence is for the virtual user is


1.       Request 1 starts at 16:06:07.982

2.       Requests 2...17 all successful with request 17 finishing at 16:06:32.134

3.       Request 18 starts at 16:06:32.135 and throws this exception after an elapsed time of 78ms, ending at 16:06:32.213

4.       Request 19 starts at 16:06:32.213 and is successful

5.       Requests continue to be successful

I don't see why this is not entering the retry logic.  I have debug logging on

log_level.http=DEBUG
log_level.jmeter.protocol=DEBUG

so, I don't see why I don't see the log message

this.log.debug("Closing the connection.");

in the IOException handling at line 690 in DefaultRequestDirector

I can easily create these errors. Any help on how to diagnose further.

Thanks
Antony






pmd
Reply | Threaded
Open this post in threaded view
|

Re: How to diagnose NoHttpResponseException

pmd
Hello,
Those settings are related to your server keep alive timeout.

But there are fixes in both JMeter 3.2 and nightly build.
Can you give nightly build a try:
- http://jmeter.apache.org/nightly.html

Regards

On Mon, Aug 7, 2017 at 8:43 AM, Antony Bowesman <
[hidden email]> wrote:

> I am using JMeter 3.1 and HttpClient4
> I have the following httpclient settings in user.properties
>
> httpclient4.retrycount=3
> httpclient4.validate_after_inactivity=20000
> httpclient4.time_to_live=1800000
>
> I have 5 load generators and each generator is running approx. 6,000 VU
> across a number of different samplers.
>
> We are migrating away from a commercial product, so in addition to this
> load, I have 3 commercial product load generators running the same profile
> and the effective server load is 75% JMeter and 25% other
>
> We do not have any problems with the commercial product, which we have
> been using for 3 years.
>
> CPU and memory are fine on the Linux JMeter boxes, but I am seeing
> frequent NoHttpResponseException errors
>
> I know that the retry logic is happening as in the test I can see these
> messages appearing and there are far more of these than actual errors and
> it's logging my retry count as 3
>
> 2017/08/07 14:36:09 INFO  - org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl$7:
> I/O exception (java.net.SocketException) caught when processing request to
> {s}-><a href="https://host:443:">https://host:443: Connection reset
> 2017/08/07 14:36:09 INFO  - org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl$7:
> Retrying request to {s}->https://host:443
>
> So, I've simplified the test down to a single machine and enabled debug
> and run the test until I see the first error.  I get this in the log, but
> there is no retry logged
>
> 2017/08/07 16:06:32 DEBUG - jmeter.protocol.http.control.CacheManager:
> GET(OAH) https://host/experience/racing/upcomingracing null
> 2017/08/07 16:06:32 DEBUG - jmeter.protocol.http.control.HC4CookieHandler:
> Found 6 cookies for https:// host/experience/racing/upcomingracing
> 2017/08/07 16:06:32 DEBUG - jmeter.protocol.http.control.CacheManager:
> inCache https:// host/experience/racing/upcomingracing null
> 2017/08/07 16:06:32 DEBUG - jmeter.protocol.http.sampler.HTTPHC4Impl:
> IOException org.apache.http.NoHttpResponseException: host:443 failed to
> respond
>         at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(
> DefaultHttpResponseParser.java:143)
>         at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(
> DefaultHttpResponseParser.java:57)
>         at org.apache.http.impl.io.AbstractMessageParser.parse(
> AbstractMessageParser.java:259)
>         at org.apache.http.impl.AbstractHttpClientConnection.
> receiveResponseHeader(AbstractHttpClientConnection.java:281)
>         at org.apache.http.impl.conn.DefaultClientConnection.
> receiveResponseHeader(DefaultClientConnection.java:259)
>         at org.apache.http.impl.conn.ManagedClientConnectionImpl.
> receiveResponseHeader(ManagedClientConnectionImpl.java:209)
>         at org.apache.jmeter.protocol.http.sampler.
> MeasuringConnectionManager$MeasuredConnection.receiveResponseHeader(
> MeasuringConnectionManager.java:212)
>         at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(
> HttpRequestExecutor.java:273)
>         at org.apache.http.protocol.HttpRequestExecutor.execute(
> HttpRequestExecutor.java:125)
>         at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(
> DefaultRequestDirector.java:686)
>         at org.apache.http.impl.client.DefaultRequestDirector.execute(
> DefaultRequestDirector.java:488)
>         at org.apache.http.impl.client.AbstractHttpClient.doExecute(
> AbstractHttpClient.java:884)
>         at org.apache.http.impl.client.CloseableHttpClient.execute(
> CloseableHttpClient.java:82)
>         at org.apache.http.impl.client.CloseableHttpClient.execute(
> CloseableHttpClient.java:55)
>         at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.
> executeRequest(HTTPHC4Impl.java:654)
>         at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.
> sample(HTTPHC4Impl.java:413)
>         at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.
> sample(HTTPSamplerProxy.java:74)
>         at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.
> sample(HTTPSamplerBase.java:1166)
>         at au.com.williamhill.perftest.endpoint.Browser.sendRequest(
> Browser.java:202)
>         at au.com.williamhill.perftest.jmeter.samplers.Command.
> sendRequest(Command.java:462)
>         at au.com.williamhill.perftest.jmeter.samplers.Command.
> sendUrls(Command.java:531)
>         at au.com.williamhill.perftest.jmeter.samplers.Command.
> postUrls(Command.java:271)
>         at au.com.williamhill.perftest.jmeter.samplers.ViewRace.
> viewRace(ViewRace.java:162)
>         at au.com.williamhill.perftest.jmeter.samplers.ViewRace.
> runTest(ViewRace.java:113)
>         at org.apache.jmeter.protocol.java.sampler.JavaSampler.
> sample(JavaSampler.java:196)
>         at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(
> JMeterThread.java:475)
>         at org.apache.jmeter.threads.JMeterThread.processSampler(
> JMeterThread.java:418)
>         at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.
> java:249)
>         at java.lang.Thread.run(Thread.java:748)
>
> The request sequence is for the virtual user is
>
>
> 1.       Request 1 starts at 16:06:07.982
>
> 2.       Requests 2...17 all successful with request 17 finishing at
> 16:06:32.134
>
> 3.       Request 18 starts at 16:06:32.135 and throws this exception after
> an elapsed time of 78ms, ending at 16:06:32.213
>
> 4.       Request 19 starts at 16:06:32.213 and is successful
>
> 5.       Requests continue to be successful
>
> I don't see why this is not entering the retry logic.  I have debug
> logging on
>
> log_level.http=DEBUG
> log_level.jmeter.protocol=DEBUG
>
> so, I don't see why I don't see the log message
>
> this.log.debug("Closing the connection.");
>
> in the IOException handling at line 690 in DefaultRequestDirector
>
> I can easily create these errors. Any help on how to diagnose further.
>
> Thanks
> Antony
>
>
>
>
>
>
>


--
Cordialement.
Philippe Mouawad.
Reply | Threaded
Open this post in threaded view
|

RE: How to diagnose NoHttpResponseException

Antony Bowesman
Hi Phillippe, we have tried the 3.2 build and that has dramatically reduced the errors, still have a small number, but I will do some digging into those and I'll see if we can get the nightly build into our test.
Thanks
Antony


> -----Original Message-----
> From: Philippe Mouawad [mailto:[hidden email]]
> Sent: Monday, 7 August 2017 5:31 PM
> To: JMeter Users List
> Subject: Re: How to diagnose NoHttpResponseException
>
> Hello,
> Those settings are related to your server keep alive timeout.
>
> But there are fixes in both JMeter 3.2 and nightly build.
> Can you give nightly build a try:
> -
> https://apac01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fjmet
> er.apache.org%2Fnightly.html&data=02%7C01%7CAntony.Bowesman%40wi
> lliamhill.com.au%7C9643cd0bfeab4966d99508d4dd66505a%7Cee4fd668f99f4
> f05a74b6405db44621a%7C0%7C0%7C636376878889185773&sdata=CD1iO23Q
> ySR3LVYYhUXNVhtg%2FM%2Forw4JAMuHJO6%2BQbk%3D&reserved=0
>
> Regards
>
> On Mon, Aug 7, 2017 at 8:43 AM, Antony Bowesman <
> [hidden email]> wrote:
>
> > I am using JMeter 3.1 and HttpClient4
> > I have the following httpclient settings in user.properties
> >
> > httpclient4.retrycount=3
> > httpclient4.validate_after_inactivity=20000
> > httpclient4.time_to_live=1800000
> >
> > I have 5 load generators and each generator is running approx. 6,000
> > VU across a number of different samplers.
> >
> > We are migrating away from a commercial product, so in addition to
> > this load, I have 3 commercial product load generators running the
> > same profile and the effective server load is 75% JMeter and 25% other
> >
> > We do not have any problems with the commercial product, which we
> have
> > been using for 3 years.
> >
> > CPU and memory are fine on the Linux JMeter boxes, but I am seeing
> > frequent NoHttpResponseException errors
> >
> > I know that the retry logic is happening as in the test I can see
> > these messages appearing and there are far more of these than actual
> > errors and it's logging my retry count as 3
> >
> > 2017/08/07 14:36:09 INFO  -
> org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl$7:
> > I/O exception (java.net.SocketException) caught when processing
> > request to
> > {s}-><a href="https://host:443:">https://host:443: Connection reset
> > 2017/08/07 14:36:09 INFO  -
> org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl$7:
> > Retrying request to {s}->https://host:443
> >
> > So, I've simplified the test down to a single machine and enabled
> > debug and run the test until I see the first error.  I get this in the
> > log, but there is no retry logged
> >
> > 2017/08/07 16:06:32 DEBUG - jmeter.protocol.http.control.CacheManager:
> > GET(OAH) https://host/experience/racing/upcomingracing null
> > 2017/08/07 16:06:32 DEBUG -
> jmeter.protocol.http.control.HC4CookieHandler:
> > Found 6 cookies for https:// host/experience/racing/upcomingracing
> > 2017/08/07 16:06:32 DEBUG - jmeter.protocol.http.control.CacheManager:
> > inCache https:// host/experience/racing/upcomingracing null
> > 2017/08/07 16:06:32 DEBUG - jmeter.protocol.http.sampler.HTTPHC4Impl:
> > IOException org.apache.http.NoHttpResponseException: host:443 failed
> > to respond
> >         at
> > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(
> > DefaultHttpResponseParser.java:143)
> >         at
> > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(
> > DefaultHttpResponseParser.java:57)
> >         at org.apache.http.impl.io.AbstractMessageParser.parse(
> > AbstractMessageParser.java:259)
> >         at org.apache.http.impl.AbstractHttpClientConnection.
> > receiveResponseHeader(AbstractHttpClientConnection.java:281)
> >         at org.apache.http.impl.conn.DefaultClientConnection.
> > receiveResponseHeader(DefaultClientConnection.java:259)
> >         at org.apache.http.impl.conn.ManagedClientConnectionImpl.
> > receiveResponseHeader(ManagedClientConnectionImpl.java:209)
> >         at org.apache.jmeter.protocol.http.sampler.
> >
> MeasuringConnectionManager$MeasuredConnection.receiveResponseHea
> der(
> > MeasuringConnectionManager.java:212)
> >         at
> > org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(
> > HttpRequestExecutor.java:273)
> >         at org.apache.http.protocol.HttpRequestExecutor.execute(
> > HttpRequestExecutor.java:125)
> >         at
> > org.apache.http.impl.client.DefaultRequestDirector.tryExecute(
> > DefaultRequestDirector.java:686)
> >         at org.apache.http.impl.client.DefaultRequestDirector.execute(
> > DefaultRequestDirector.java:488)
> >         at org.apache.http.impl.client.AbstractHttpClient.doExecute(
> > AbstractHttpClient.java:884)
> >         at org.apache.http.impl.client.CloseableHttpClient.execute(
> > CloseableHttpClient.java:82)
> >         at org.apache.http.impl.client.CloseableHttpClient.execute(
> > CloseableHttpClient.java:55)
> >         at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.
> > executeRequest(HTTPHC4Impl.java:654)
> >         at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.
> > sample(HTTPHC4Impl.java:413)
> >         at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.
> > sample(HTTPSamplerProxy.java:74)
> >         at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.
> > sample(HTTPSamplerBase.java:1166)
> >         at au.com.williamhill.perftest.endpoint.Browser.sendRequest(
> > Browser.java:202)
> >         at au.com.williamhill.perftest.jmeter.samplers.Command.
> > sendRequest(Command.java:462)
> >         at au.com.williamhill.perftest.jmeter.samplers.Command.
> > sendUrls(Command.java:531)
> >         at au.com.williamhill.perftest.jmeter.samplers.Command.
> > postUrls(Command.java:271)
> >         at au.com.williamhill.perftest.jmeter.samplers.ViewRace.
> > viewRace(ViewRace.java:162)
> >         at au.com.williamhill.perftest.jmeter.samplers.ViewRace.
> > runTest(ViewRace.java:113)
> >         at org.apache.jmeter.protocol.java.sampler.JavaSampler.
> > sample(JavaSampler.java:196)
> >         at
> > org.apache.jmeter.threads.JMeterThread.executeSamplePackage(
> > JMeterThread.java:475)
> >         at org.apache.jmeter.threads.JMeterThread.processSampler(
> > JMeterThread.java:418)
> >         at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.
> > java:249)
> >         at java.lang.Thread.run(Thread.java:748)
> >
> > The request sequence is for the virtual user is
> >
> >
> > 1.       Request 1 starts at 16:06:07.982
> >
> > 2.       Requests 2...17 all successful with request 17 finishing at
> > 16:06:32.134
> >
> > 3.       Request 18 starts at 16:06:32.135 and throws this exception after
> > an elapsed time of 78ms, ending at 16:06:32.213
> >
> > 4.       Request 19 starts at 16:06:32.213 and is successful
> >
> > 5.       Requests continue to be successful
> >
> > I don't see why this is not entering the retry logic.  I have debug
> > logging on
> >
> > log_level.http=DEBUG
> > log_level.jmeter.protocol=DEBUG
> >
> > so, I don't see why I don't see the log message
> >
> > this.log.debug("Closing the connection.");
> >
> > in the IOException handling at line 690 in DefaultRequestDirector
> >
> > I can easily create these errors. Any help on how to diagnose further.
> >
> > Thanks
> > Antony
> >
> >
> >
> >
> >
> >
> >
>
>
> --
> Cordialement.
> Philippe Mouawad.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]