tracker issue : CF-4203250

select a category, or use search below
(searches all categories and all time range)
Title:

Deadlock resulting POST calls to web service CFC

| View in Tracker

Status/Resolution/Reason: To Track//PRNeedInfo

Reporter/Name(from Bugbase): A. Bakia / ()

Created: 08/08/2018

Components: Web Services, Axis 2

Versions: 2016

Failure Type: Others

Found In Build/Fixed In Build: 2016.0.06.308055 /

Priority/Frequency: Normal /

Locale/System: / Win 2016

Vote Count: 0

Problem Description:
We migrated some weeks ago from ColdFusion 11 to ColdFusion 2016. In one of our applications, we publish web services by means of POST calls to CFCs. 

The web services had been running for years, without major incident, on ColdFusion 11. However, recently, on ColdFusion 2016, one of the calls has resulted in deadlock. 

The 3 threads involved are:

ajp-nio-8034-exec-3
ajp-nio-8034-exec-4
ajp-nio-8034-exec-8

See attached pictures. 

Apparently, thread 3 waits for an object of type coldfusion.xml.rpc.SkeletonClassLoader. The object is held by thread 8. Thread 4 is waiting for an object that is held by thread 3. In turn, thread 8 is waiting for an object that is held by thread 3. Hence the deadlock. See "FusionReactor_deadlock_log.txt"

The deadlock symptoms are:

Server 500 status
High CPU usage
Continually running request, with high request runtime (the 3 threads ran for over an hour, then we decided to restart the Coldfusion instance.

We use Fusionreactor to monitor the server, and found out that the threads failed to respond to suspend or kill commands from  FusionReactor. 

Our server uses JsessionID by default. We noticed that the deadlock was caused by the only web service call for which ColdFusion failed to generate a JsessionID. See the calls between 11:15 and 11:16 in the attached file "webservice calls - sm2_imports_1.txt".

Steps to Reproduce:
We are unable to reproduce this because the deadlock involves Java code within the ColdFusion engine.   

Actual Result:
Cyclical deadlock resulting from 3 threads within ColdFusion's web service engine.

Expected Result:
Web service POST call to a CFC, without any deadlock.

Any Workarounds:
None known, besides restarting the ColdFusion instance.

Attachments:

Comments:

Hi, Can you please provide  me following details : 1) ColdFusion 2016 update version which is causing this issue 2) A code snippet which is causing this issue. These details would be really helpful to debug this issue.   Thanks Poonam
Comment by Poonam Jain
29456 | August 10, 2018 08:39:17 AM GMT
Thanks, Poonam. I already stated the ColdFusion Build. It is 2016.0.06.308055 I would rather not publish the code snippet here, as it contains sensitive company information. Could you please give me an e-mail address to which I can send it?
Comment by A. Bakia
29496 | August 15, 2018 01:21:27 PM GMT
I have dropped a e-mail on your ID. Please provide the details there. -Poonam
Comment by Poonam Jain
29499 | August 16, 2018 04:17:35 AM GMT