CriticalSectionController: Lock global_lock not released

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

CriticalSectionController: Lock global_lock not released

Shmuel Krakower
Hello,
It has been a while since I've participated in the users' list..

I am running a stress test with multiple thread groups and I'm using the
Critical Section Controller to prevent a specific action from taking place
multiple times on the same time.

I notice that the results are much lower than the required throughput I
plan to achieve.
After looking into the jmeter logs I notice many of my threads were
actually "locked" waiting for the critical section and this is the reason I
am not reaching my target RPS.

The log show entries such as:
WARN  - jmeter.control.CriticalSectionController: Lock global_lock not
released in:Critical Section Controller, releasing in threadFinished

'global_lock' - is just the default text used in the controller. But it
clearly shows that at some point one of the threads keeps the lock busy
which in turn just block the others.

Some ideas/questions:
Maybe it would make sense to have a timeout on the lock?
Is it possible that an exception raised inside the critical section,
prevented it from being released?
The main suspect I have in my test plan is a Test Action element I use
which is set to "Go to next loop iteration" in some cases, maybe that's the
one which doesn't release the critical section?...

Would it help if I take a thread dump and share it here?
Should I open a defect in Bugzilla for that?
Has anyone faced such an issue before?

Best,
Shmuel Krakower.
pmd
Reply | Threaded
Open this post in threaded view
|

Re: CriticalSectionController: Lock global_lock not released

pmd
Hello,
Yes please open a bugzilla and provide:
- an excerpt of your test plan
- jmeter.log
- 3 thread dumps at 5s distance when issue occurs

Thank you

On Sunday, October 29, 2017, Shmuel Krakower <[hidden email]> wrote:

> Hello,
> It has been a while since I've participated in the users' list..
>
> I am running a stress test with multiple thread groups and I'm using the
> Critical Section Controller to prevent a specific action from taking place
> multiple times on the same time.
>
> I notice that the results are much lower than the required throughput I
> plan to achieve.
> After looking into the jmeter logs I notice many of my threads were
> actually "locked" waiting for the critical section and this is the reason I
> am not reaching my target RPS.
>
> The log show entries such as:
> WARN  - jmeter.control.CriticalSectionController: Lock global_lock not
> released in:Critical Section Controller, releasing in threadFinished
>
> 'global_lock' - is just the default text used in the controller. But it
> clearly shows that at some point one of the threads keeps the lock busy
> which in turn just block the others.
>
> Some ideas/questions:
> Maybe it would make sense to have a timeout on the lock?
> Is it possible that an exception raised inside the critical section,
> prevented it from being released?
> The main suspect I have in my test plan is a Test Action element I use
> which is set to "Go to next loop iteration" in some cases, maybe that's the
> one which doesn't release the critical section?...
>
> Would it help if I take a thread dump and share it here?
> Should I open a defect in Bugzilla for that?
> Has anyone faced such an issue before?
>
> Best,
> Shmuel Krakower.
>


--
Cordialement.
Philippe Mouawad.
Reply | Threaded
Open this post in threaded view
|

Re: CriticalSectionController: Lock global_lock not released

Felix Schumacher


Am 29. Oktober 2017 19:21:19 MEZ schrieb Philippe Mouawad <[hidden email]>:
>Hello,
>Yes please open a bugzilla and provide:
>- an excerpt of your test plan
>- jmeter.log
>- 3 thread dumps at 5s distance when issue occurs

I think the most likely cause is the premature end of an iteration - "go to next loop iteration". We probably need to add an iteration listener that unlocks the locks on iteration start.

Regards,
Felix

>
>Thank you
>
>On Sunday, October 29, 2017, Shmuel Krakower <[hidden email]>
>wrote:
>
>> Hello,
>> It has been a while since I've participated in the users' list..
>>
>> I am running a stress test with multiple thread groups and I'm using
>the
>> Critical Section Controller to prevent a specific action from taking
>place
>> multiple times on the same time.
>>
>> I notice that the results are much lower than the required throughput
>I
>> plan to achieve.
>> After looking into the jmeter logs I notice many of my threads were
>> actually "locked" waiting for the critical section and this is the
>reason I
>> am not reaching my target RPS.
>>
>> The log show entries such as:
>> WARN  - jmeter.control.CriticalSectionController: Lock global_lock
>not
>> released in:Critical Section Controller, releasing in threadFinished
>>
>> 'global_lock' - is just the default text used in the controller. But
>it
>> clearly shows that at some point one of the threads keeps the lock
>busy
>> which in turn just block the others.
>>
>> Some ideas/questions:
>> Maybe it would make sense to have a timeout on the lock?
>> Is it possible that an exception raised inside the critical section,
>> prevented it from being released?
>> The main suspect I have in my test plan is a Test Action element I
>use
>> which is set to "Go to next loop iteration" in some cases, maybe
>that's the
>> one which doesn't release the critical section?...
>>
>> Would it help if I take a thread dump and share it here?
>> Should I open a defect in Bugzilla for that?
>> Has anyone faced such an issue before?
>>
>> Best,
>> Shmuel Krakower.
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: CriticalSectionController: Lock global_lock not released

Shmuel Krakower
Hi
I just moved the test action element out of the critical section but the
problem persists. I will collect more data and try to provide the simplest
script to reproduce the issue in bugzilla.

Thanks

On Sun, Oct 29, 2017, 10:14 PM Felix Schumacher <
[hidden email]> wrote:

>
>
> Am 29. Oktober 2017 19:21:19 MEZ schrieb Philippe Mouawad <
> [hidden email]>:
> >Hello,
> >Yes please open a bugzilla and provide:
> >- an excerpt of your test plan
> >- jmeter.log
> >- 3 thread dumps at 5s distance when issue occurs
>
> I think the most likely cause is the premature end of an iteration - "go
> to next loop iteration". We probably need to add an iteration listener that
> unlocks the locks on iteration start.
>
> Regards,
> Felix
> >
> >Thank you
> >
> >On Sunday, October 29, 2017, Shmuel Krakower <[hidden email]>
> >wrote:
> >
> >> Hello,
> >> It has been a while since I've participated in the users' list..
> >>
> >> I am running a stress test with multiple thread groups and I'm using
> >the
> >> Critical Section Controller to prevent a specific action from taking
> >place
> >> multiple times on the same time.
> >>
> >> I notice that the results are much lower than the required throughput
> >I
> >> plan to achieve.
> >> After looking into the jmeter logs I notice many of my threads were
> >> actually "locked" waiting for the critical section and this is the
> >reason I
> >> am not reaching my target RPS.
> >>
> >> The log show entries such as:
> >> WARN  - jmeter.control.CriticalSectionController: Lock global_lock
> >not
> >> released in:Critical Section Controller, releasing in threadFinished
> >>
> >> 'global_lock' - is just the default text used in the controller. But
> >it
> >> clearly shows that at some point one of the threads keeps the lock
> >busy
> >> which in turn just block the others.
> >>
> >> Some ideas/questions:
> >> Maybe it would make sense to have a timeout on the lock?
> >> Is it possible that an exception raised inside the critical section,
> >> prevented it from being released?
> >> The main suspect I have in my test plan is a Test Action element I
> >use
> >> which is set to "Go to next loop iteration" in some cases, maybe
> >that's the
> >> one which doesn't release the critical section?...
> >>
> >> Would it help if I take a thread dump and share it here?
> >> Should I open a defect in Bugzilla for that?
> >> Has anyone faced such an issue before?
> >>
> >> Best,
> >> Shmuel Krakower.
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: CriticalSectionController: Lock global_lock not released

Shmuel Krakower
Hi,
Just to update on this.
I will not create a bugzilla as it wasn't the root cause of my original
issue as well as I am not sure it is an issue with JMeter core.
I can only reproduce those warning messages while using the Ultimate Thread
Group from JMeterPlugins.

For the record:
To reproduce - create a new testplan with single Ultimate Thread Group
element.
Set it to start 1 thread (or more) and let it run for 1 second (or more).
In that thread group create a Critical Section Controller.
In that Critical Section Controller create a Debug Sampler.
Run it and get this error:
WARN  - jmeter.control.CriticalSectionController: Lock global_lock not
released in:Critical Section Controller, releasing in threadFinished

-Test Plan
--Ultimate Thread Group
---Critical Section Controller
----Debug Sampler


Best,
Shmuel.

Shmuel Krakower.

2017-10-29 23:31 GMT+02:00 Shmuel Krakower <[hidden email]>:

> Hi
> I just moved the test action element out of the critical section but the
> problem persists. I will collect more data and try to provide the simplest
> script to reproduce the issue in bugzilla.
>
> Thanks
>
> On Sun, Oct 29, 2017, 10:14 PM Felix Schumacher <felix.schumacher@
> internetallee.de> wrote:
>
>>
>>
>> Am 29. Oktober 2017 19:21:19 MEZ schrieb Philippe Mouawad <
>> [hidden email]>:
>> >Hello,
>> >Yes please open a bugzilla and provide:
>> >- an excerpt of your test plan
>> >- jmeter.log
>> >- 3 thread dumps at 5s distance when issue occurs
>>
>> I think the most likely cause is the premature end of an iteration - "go
>> to next loop iteration". We probably need to add an iteration listener that
>> unlocks the locks on iteration start.
>>
>> Regards,
>> Felix
>> >
>> >Thank you
>> >
>> >On Sunday, October 29, 2017, Shmuel Krakower <[hidden email]>
>> >wrote:
>> >
>> >> Hello,
>> >> It has been a while since I've participated in the users' list..
>> >>
>> >> I am running a stress test with multiple thread groups and I'm using
>> >the
>> >> Critical Section Controller to prevent a specific action from taking
>> >place
>> >> multiple times on the same time.
>> >>
>> >> I notice that the results are much lower than the required throughput
>> >I
>> >> plan to achieve.
>> >> After looking into the jmeter logs I notice many of my threads were
>> >> actually "locked" waiting for the critical section and this is the
>> >reason I
>> >> am not reaching my target RPS.
>> >>
>> >> The log show entries such as:
>> >> WARN  - jmeter.control.CriticalSectionController: Lock global_lock
>> >not
>> >> released in:Critical Section Controller, releasing in threadFinished
>> >>
>> >> 'global_lock' - is just the default text used in the controller. But
>> >it
>> >> clearly shows that at some point one of the threads keeps the lock
>> >busy
>> >> which in turn just block the others.
>> >>
>> >> Some ideas/questions:
>> >> Maybe it would make sense to have a timeout on the lock?
>> >> Is it possible that an exception raised inside the critical section,
>> >> prevented it from being released?
>> >> The main suspect I have in my test plan is a Test Action element I
>> >use
>> >> which is set to "Go to next loop iteration" in some cases, maybe
>> >that's the
>> >> one which doesn't release the critical section?...
>> >>
>> >> Would it help if I take a thread dump and share it here?
>> >> Should I open a defect in Bugzilla for that?
>> >> Has anyone faced such an issue before?
>> >>
>> >> Best,
>> >> Shmuel Krakower.
>> >>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: CriticalSectionController: Lock global_lock not released

sebb-2-2
On 8 November 2017 at 09:27, Shmuel Krakower <[hidden email]> wrote:
> Hi,
> Just to update on this.
> I will not create a bugzilla as it wasn't the root cause of my original
> issue as well as I am not sure it is an issue with JMeter core.

Thanks for the followup.

> I can only reproduce those warning messages while using the Ultimate Thread
> Group from JMeterPlugins.

In which case, please report this to the providers of the Ultimate Thread Group.
This is not an Apache JMeter product so there's nothing the JMeter
developers can do.

> For the record:
> To reproduce - create a new testplan with single Ultimate Thread Group
> element.
> Set it to start 1 thread (or more) and let it run for 1 second (or more).
> In that thread group create a Critical Section Controller.
> In that Critical Section Controller create a Debug Sampler.
> Run it and get this error:
> WARN  - jmeter.control.CriticalSectionController: Lock global_lock not
> released in:Critical Section Controller, releasing in threadFinished
>
> -Test Plan
> --Ultimate Thread Group
> ---Critical Section Controller
> ----Debug Sampler
>
>
> Best,
> Shmuel.
>
> Shmuel Krakower.
>
> 2017-10-29 23:31 GMT+02:00 Shmuel Krakower <[hidden email]>:
>
>> Hi
>> I just moved the test action element out of the critical section but the
>> problem persists. I will collect more data and try to provide the simplest
>> script to reproduce the issue in bugzilla.
>>
>> Thanks
>>
>> On Sun, Oct 29, 2017, 10:14 PM Felix Schumacher <felix.schumacher@
>> internetallee.de> wrote:
>>
>>>
>>>
>>> Am 29. Oktober 2017 19:21:19 MEZ schrieb Philippe Mouawad <
>>> [hidden email]>:
>>> >Hello,
>>> >Yes please open a bugzilla and provide:
>>> >- an excerpt of your test plan
>>> >- jmeter.log
>>> >- 3 thread dumps at 5s distance when issue occurs
>>>
>>> I think the most likely cause is the premature end of an iteration - "go
>>> to next loop iteration". We probably need to add an iteration listener that
>>> unlocks the locks on iteration start.
>>>
>>> Regards,
>>> Felix
>>> >
>>> >Thank you
>>> >
>>> >On Sunday, October 29, 2017, Shmuel Krakower <[hidden email]>
>>> >wrote:
>>> >
>>> >> Hello,
>>> >> It has been a while since I've participated in the users' list..
>>> >>
>>> >> I am running a stress test with multiple thread groups and I'm using
>>> >the
>>> >> Critical Section Controller to prevent a specific action from taking
>>> >place
>>> >> multiple times on the same time.
>>> >>
>>> >> I notice that the results are much lower than the required throughput
>>> >I
>>> >> plan to achieve.
>>> >> After looking into the jmeter logs I notice many of my threads were
>>> >> actually "locked" waiting for the critical section and this is the
>>> >reason I
>>> >> am not reaching my target RPS.
>>> >>
>>> >> The log show entries such as:
>>> >> WARN  - jmeter.control.CriticalSectionController: Lock global_lock
>>> >not
>>> >> released in:Critical Section Controller, releasing in threadFinished
>>> >>
>>> >> 'global_lock' - is just the default text used in the controller. But
>>> >it
>>> >> clearly shows that at some point one of the threads keeps the lock
>>> >busy
>>> >> which in turn just block the others.
>>> >>
>>> >> Some ideas/questions:
>>> >> Maybe it would make sense to have a timeout on the lock?
>>> >> Is it possible that an exception raised inside the critical section,
>>> >> prevented it from being released?
>>> >> The main suspect I have in my test plan is a Test Action element I
>>> >use
>>> >> which is set to "Go to next loop iteration" in some cases, maybe
>>> >that's the
>>> >> one which doesn't release the critical section?...
>>> >>
>>> >> Would it help if I take a thread dump and share it here?
>>> >> Should I open a defect in Bugzilla for that?
>>> >> Has anyone faced such an issue before?
>>> >>
>>> >> Best,
>>> >> Shmuel Krakower.
>>> >>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]