tracker issue : CF-4203636

select a category, or use search below
(searches all categories and all time range)
Title:

Weird behaviour in ehcache

| View in Tracker

Status/Resolution/Reason: Closed/Fixed/Fixed

Reporter/Name(from Bugbase): Ben Ferdinandus / ()

Created: 12/04/2018

Components: Caching

Versions: 2018

Failure Type: Incorrectly functioning

Found In Build/Fixed In Build: adobe 2018.0.01+311402 / 313942

Priority/Frequency: Normal / Some users will encounter

Locale/System: ALL / Windows 10 64 bit

Vote Count: 5

Problem Description:
We have two applications which are using the same backend. We run those in 2 different coldfusion instances, and with a different application name.
In the \cfusion\lib\ehcache.xml file we configured our own cacheregions. We use a Terracotta server to serve the cache to the 2 applications.

This all works fine with CF2016, but with CF2018 we encountered a problem.

•	If we try to use cacheGet right after the server has started we get the error: Cache region with name userInfo does not exist.
•	Then if we put an item in that cacheregion, (cachePut) and then chacheGet it, the problem is gone.

To investigate what was happening I used a java object to get the cache names. (var joCacheManager=createObject("java", "net.sf.ehcache.CacheManager").getInstance();)
When the server is just started up it looks like this: (with the userInfo we defined in the ehcache.xml)
1	userInfo
2	authcache
3	customcache
4	authtokenmappingcache

If we then try to get an item from the region userInfo, we get the error: Cache region with name userInfo does not exist.
Then we put an item in the region userInfo. That gives no error. If I then take a look at the cache regions I see the following:
1	userInfo
2	authcache
3	myApplicationCF2018userInfo
4	customcache
5	authtokenmappingcache

There is an extra cacheregion consisting of the region name defined in ehcache.xml prefixed with the coldfusion application name.
Now we can use cacheget to get items from the region userInfo.

To take my investigation one step further I used cacheGetAllIds to see the items in the cacheregions.

•	writeDump(cacheGetAllIds("userInfo"));
This command gives me the item I just put in the region userInfo

•	writeDump(cacheGetAllIds("myApplicationCF2018userInfo"));
This however gives me the error:Cache region with name myApplicationCF2018userInfo does not exist.

So to me it appears that there is a bug that creates a mystery cache region. And if the mystery region does not exist, cacheget throws an error.

I have added a zipfile with my test code to reproduce the problem.

Steps to Reproduce:
Add a new region to \cfusion\lib\ehcache.xml
Create an index.cfm and then try to use cacheGet("object1", "newCacheRegion");

Actual Result:
Cache region with name newCacheRegion does not exist.

Expected Result:
The cacheGet command succeeds without errors.

Any Workarounds:
At the start of the application put some object in the newCacheRegion. Then try to get "object1".

Attachments:

Comments:

Hi Ben, Firstly, I tried to repro user case. In 2018, I dont see two region with name "UserInfo" and "myApplicationCF2018userInfo". Only one region exist, which is "myApplicationCF2018userInfo". The region name "myApplicationCF2018userInfo", will not exist in 2016. The reason for this is because, in 2018 there is slight change in design. We appended Application name with <regionName> (if it exists), so that if two applications are creating regions with same name then they can be distinguished on the basis of <application name>. However, user will address the region with given <region name> only, But application name will exists together with region name when created at the backend.    The behavior that you are seeing, when trying to get an object from an newly created region- > "Cache region with name userInfo does not exist." . it exist in 2016 as well. You need to put an object first and then you can get it. May be the error is incorrect here. This behavior is same across 2016, 2018.   Attached is the result from 2018, where I see only one region existing with the name "myApplicationCF2018userInfo" Thanks Poonam
Comment by Poonam J.
30004 | December 05, 2018 06:20:53 AM GMT
Hi Poonam, Whichever way you look at it, there is a problem with the new design. For a number of reasons: 1) A cache region may exist even if it contains no objects. A cache region is a collection of sorts, separate from its contents. Like an empty array, [], or an empty struct, {}, an empty cache region should have an independent existence. According to the principle of separation of concerns, you should not have to put an object into it for it to exist. 2) ColdFusion 2018 has no business transparently renaming any variable that the user is free to name. In the background ColdFusion may, of course, do whatever it likes with user-given names. But, according to the principle of encapsulation, the user shouldn't see what ColdFusion does privately to solve such a problem. When you dump cache regions, then you must only see the exact names of the default regions and of any other regions you yourself created. 3) Consider the following design once again: "if two applications are creating regions with same name then they can be distinguished on the basis of <application name>. However, user will address the region with given <region name> only," . It has another flaw besides the one discussed in 2). If you enable two or more applications to share cache regions having the same name, then you will be promoting coupling. The solution is obvious. Where a cache-region name already exists, ColdFusion should simply inform the second application to choose a different name.
Comment by A. B.
30006 | December 05, 2018 01:51:50 PM GMT
Here are the responses:   1) A cache region may exist even if it contains no objects. A cache region is a collection of sorts, separate from its contents. Like an empty array, [], or an empty struct, {}, an empty cache region should have an independent existence. According to the principle of separation of concerns, you should not have to put an object into it for it to exist.   --> I agree. Here, cache region do have an independent existence. Whats needs to be corrected is message. Because the object was not present, error message could be nonexistence of object rather than region. 2) ColdFusion 2018 has no business transparently renaming any variable that the user is free to name. In the background ColdFusion may, of course, do whatever it likes with user-given names. But, according to the principle of encapsulation, the user shouldn't see what ColdFusion does privately to solve such a problem. When you dump cache regions, then you must only see the exact names of the default regions and of any other regions you yourself created.   --> You are using java API yo get cache regions. This API is getting data from backend. Do you see cache region name appended with Application name in any of the caching functions? 3) Consider the following design once again: "if two applications are creating regions with same name then they can be distinguished on the basis of <application name>. However, user will address the region with given <region name> only," . It has another flaw besides the one discussed in 2). If you enable two or more applications to share cache regions having the same name, then you will be promoting coupling. The solution is obvious. Where a cache-region name already exists, ColdFusion should simply inform the second application to choose a different name.   -->Yes, user will address the region with <region name>only. As I said, there in no change in functions and user behavior/experience. For a user. region name will be referenced by <region name> only. Now coming to your next question:   if you enable two or more applications to share cache regions having the same name, then you will be promoting coupling. The solution is obvious. Where a cache-region name already exists, ColdFusion should simply inform the second application to choose a different name. -- -> If two applications will never share same cache regions because we promote applicationa name to be unique.  The solution you are proposing is not full proof. Applications should be independent of each other. We cannot ask an application to decide unique name for cache regions. This way another application will come to know that there is an existing  cache region with the a name and can or may disrupt the data of that region, Which CF will not promote. Appending <application name> itself makes the region unique and application independent of each other. also, this insures data security.
Comment by Poonam J.
30008 | December 06, 2018 05:33:27 AM GMT
Hello Poonam, Problem 1: When you tried to reproduce the case, did you edit the file ""\cfusion\lib\ehcache.xml" to have an extra cacheregion with the name "userInfo"? That is key to this problem. I included the ehcache.xml files I used for this problem in the zipfile It is for us important that the cache region name is exactly as we define it in the ehcache.xml. We have one Terracotta server to serve the ehcache. We have two applications, with different names, which use the same cacheregions from the ehcache. If the cacheregionnames now get prefixed with the application name we lose the advantage of a shared cache! Problem 2: I should be able to do a cacheGet on an object which does (not yet) exists in the specified cacheregion. cacheGet should return a null if the object is not there. It should not return an error saying the cacheregion does not exist Lastly I do not see your result attached to the case. With kind regards, Ben
Comment by Ben F.
30012 | December 06, 2018 09:30:20 AM GMT
+1
Vote by A. B.
30016 | December 06, 2018 02:26:28 PM GMT
Hi Poonam, There is not a 1-to-1 relationship between Application.cfc and THIS.name. Many sites (mine included) have dynamic THIS.name (ex: CGI.HTTP_HOST-based) in same Application.cfc file. This CF2018 change negatively impacts sites like mine whose Application.cfc shares same cache regions across its dynamic THIS.names. This should be a CF Admin setting like "[ ] Allow global cache regions". Checked by default, unless Secure Profile is enabled (Secure Profile would uncheck it). Thoughts? Thanks!, -Aaron P.S. I do appreciate the security focus for shared hosting environments. But it shouldn't break sites for customers that know what they're doing. :)
Comment by Aaron N.
30018 | December 07, 2018 04:35:17 AM GMT
+1 - Needs CF Admin setting
Vote by Aaron N.
30019 | December 07, 2018 04:45:16 AM GMT
Hi Aaron, Cache regions are created at two level 1) server level 2) Applications level. Server level : If an application is not having any name then regions are created at server level. Also, Regions such as  "Object", "Template","Query" are default regions created at the server level if your tries to cache an object, template or query. The server level regions be shared across applications(if they dont have any application name). Application level: two  Applications with different application name should not be allowed to access regions created by each other. This is clearly a security breach, which Adobe cannot promote. If they share same application name then they can  shared cache regions among each other. And thus sharing is mutual.   So to share data : Use  server level cache regions by not providing application name to the applications (wants to share the data) Or have a common application name which can be provided by keeping an application file at the common/top level of two applications.   -Poonam      
Comment by Poonam J.
30033 | December 10, 2018 04:38:27 AM GMT
Hi Poonam, Regarding: "The behavior that you are seeing, when trying to get an object from an newly created region- > "Cache region with name userInfo does not exist." . it exist in 2016 as well. You need to put an object first and then you can get it. May be the error is incorrect here. This behavior is same across 2016, 2018. I see differently. `cacheGet("myID", "myServerLevelCache")` does not throw any exception in CF11 or CF2016 when ID "myID" does not exist in region "myServerLevelCache". Confirmed in 11,0,07,296330 and 2016,0,01,298513. Can you please confirm that issue is new to CF2018? Thanks!, -Aaron
Comment by Aaron N.
30038 | December 11, 2018 09:46:49 AM GMT
Hi Poonam, Regarding: "in 2018 there is slight change in design. We appended Application name with <regionName> (if it exists)": 1) In CF11 and CF2016, CF also prepends Application name to <regionName>, specifically for the regions that CF creates (i.e. OBJECT), and regardless if THIS.cache.configfile is used or not. 2) In CF11 and CF2016, CF also prepends Application name to <regionName>, specifically for the regions that a user creates (i.e. myCustomApplicationSpecificRegion), and specifically only when THIS.cache.configfile is used. Confirmed in 11,0,07,296330 and 2016,0,01,298513. Please note that the CF2018-specific change is: CF also prepends Application name to <regionName>, for regions that a user creates (i.e. myCustomServerWideRegion) in \cfusion\lib\ehcache.xml, when THIS.cache.configfile is not used. Can you please confirm that is the exact change new to CF2018? Thanks!, -Aaron
Comment by Aaron N.
30039 | December 11, 2018 09:59:41 AM GMT
Hi Aaron, For 11 , 2016 behavior you reported is correct. 1) In CF11 and CF2016, CF also prepends Application name to <regionName>, specifically for the regions that CF creates (i.e. OBJECT), and regardless if THIS.cache.configfile is used or not.  --->-> correct 2) In CF11 and CF2016, CF also prepends Application name to <regionName>, specifically for the regions that a user creates (i.e. myCustomApplicationSpecificRegion), and specifically only when THIS.cache.configfile is used.  --->->correct 3) "Please note that the CF2018 specific change is: CF also prepends Application name to <regionName>, for regions that a user creates (i.e. myCustomServerWideRegion) in \cfusion\lib\ehcache.xml, when THIS.cache.configfile is not used. "  --->-> here, the change is that CF now prepends application name <region name> that a user creates (i.e. myCustomServerWideRegion), when THIS.cache.configfile is not provided in application cfc/cfm file. It will not consider the file present at the path \cfusion\lib\ehcache.xml. File  \cfusion\lib\ehcache.xml is considered for Server level cache regions which can be accessed if Application doesnot have any name. In 2016, and 11 , if config file is not present the applications fall back to server level caching regions. \cfusion\lib\ehcache.xml file is considered by Ehcache engine on server start up and it creates the regions at server level (which are define in this file). The problem with this approach was that two Applications can disrupt the common server level cache regions data if they will not specify "THIS.cache.configfile"   Now to your next comment: To your comment = "I see differently. `cacheGet("myID", "myServerLevelCache")` does not throw any exception in CF11 or CF2016 when ID "myID" does not exist in region "myServerLevelCache". Confirmed in 11,0,07,296330 and 2016,0,01,298513."  -> This is because they are accessing server level cache regions which got created on server startup. cacheput command creates the region if region is not created. Now to solve your problem : We can make an Enhancement , that is , if THIS.cache.configfile is not defined by the application file then it will consider the one defined at the \cfusion\lib\ehcache.xml. And will create the regions for an Application(which are defined in the \cfusion\lib\ehcache.xml file). But ofcourse, Application name will prepend the region name. So that regions formed will be unique to the Applications and two applications cannot common server level cache regions. Will this enhancement will be fine?   -Poonam    
Comment by Poonam J.
30044 | December 12, 2018 10:56:43 AM GMT
Hi Poonam, You said earlier: -------------------------------------------------------------- If two applications will never share same cache regions because we promote applicationa name to be unique. The solution you are proposing is not full proof. Applications should be independent of each other. We cannot ask an application to decide unique name for cache regions. This way another application will come to know that there is an existing cache region with the a name and can or may disrupt the data of that region, Which CF will not promote. Appending <application name> itself makes the region unique and application independent of each other. also, this insures data security. -------------------------------------------------------------- This is not a strong reason. Applications may share the same cache-region and yet be independent. There is a ubiquitous use-case for this. Namely, different applications sharing the same database.
Comment by A. B.
30049 | December 12, 2018 08:32:04 PM GMT
Hi Bakia In case applications wants to share cache region, then this is still possible. 1) Application doesn't have any name. They will be able to access server level cache regions. 2) if application name exists, then applications should have a common name which can be placed at the the common/higher level of the applications, inside application cfc file. An application should have independent regions , whether config file is provided or not. If an application does not specify config file , does not mean that it should fall back to server level regions.  -Poonam
Comment by Poonam J.
30050 | December 13, 2018 03:58:24 AM GMT
Hi Poonam, Thank you very much for your explanations and continued suggestions. Since CF10, differently-named applications have relied on the ability to share \cfusion\lib\ehcache.xml regions by omitting THIS.cache.configfile. And it's too late to remove the application name of those applications or have them share an application name, b/c they must have separate application scopes/config. I realize why CF2018 made the change that it did. However, the use case for server-vs-application cache regions is still as valid as server-vs-application scheduled tasks. Suggestion: CF Admin's "Caching" page could have a new disabled-by-default option "[ ] Allow server-wide cache regions. If enabled, the file \cfusion\lib\ehcache.xml is considered for server-wide cache regions". Thoughts? Thanks!, -Aaron
Comment by Aaron N.
30052 | December 14, 2018 03:45:14 AM GMT
Hi Poonam, Thank you for your reply. You will get an improved solution if you combine your second suggestion (to make it possible for an application to place a common server-wide cache name in Application.cfc) with Aaron's suggestion (to make it possible for an application to enable server-wide caching in CF Admin). The two settings should have the same function. If the application enabled a server-wide cache-name in Application.cfc, that should override the setting in CF Admin. That is after all the tradition in ColdFusion. Such a solution will have the major advantage of backward compatibility. Existing code in which distinct applications share a common cache-region wont break. A. Bakia
Comment by A. B.
30057 | December 15, 2018 12:20:10 PM GMT
Hi Aaron and Bakia, I understand your concern here, possibly I will discuss this with Development team and get back to you. -Poonam
Comment by Poonam J.
30061 | December 17, 2018 05:27:30 AM GMT
Hello, It is for us important that the cache region name is exactly as we define it in the /cfusion/libs/ehcache.xml, at server level. We have one Terracotta server to serve the ehcache. We have two applications, with different names, which use the same database and also cache regions from the ehcache. If the cache region names now get prefixed with the application name we lose the advantage of a shared cache! With regards, Ben Ferdinandus
Comment by Ben F.
30069 | December 18, 2018 09:26:44 AM GMT
Poonam (or anyone at Adobe), did you ever get a resolution for this? I have had someone today to me the problem that their code, which had run in CF11, breaked on CF2018, and it is because of this change. They had a site where two apps (two folders, with different cf application names) had previously been able to share a same-named region, and now as of CF2018 they cannot. As Aaron noted, while it may be admirable for you guys to have implemented this as a change for the sake of security, you really should have provided a way for us to control that--and an Admin level setting (to indicate whether same-named regions would be made app-specific) would seem the perfect solution. Defaulting it to "off" would be risky as it could break code, but you could at least have clarified to folks at launch that this had been done. Instead, we have code breaking, and there is NO solution in a case like the above. No, they cannot just remove the name of the app (as you proposed), as that may break other expected behavior that DOES rely on them being in separate apps. And I don't recall ever seeing ANYTHING talking about this. Can I ask where this may be documented? I didn't find it in many places I looked, before I finally dug here into the tracker. It's not mentioned on these related to caching: https://helpx.adobe.com/coldfusion/developing-applications/changes-in-coldfusion/caching-enhancements.html https://helpx.adobe.com/coldfusion/developing-applications/developing-cfml-applications/designing-and-optimizing-a-coldfusion-application/optimizing-coldfusion-applications.html And it's not mentioned in these pages related what was new or changed in CF2018: https://helpx.adobe.com/coldfusion/using/whats-new.html https://helpx.adobe.com/coldfusion/release-note/coldfusion-2018-release-notes.html (which includes also bugs fixed and known issues) If it is documented anywhere, I would love to know. And if it is not, it should be, of course!
Comment by Charlie A.
30342 | February 20, 2019 03:13:02 AM GMT
+1. This should not have been done without clarification that it was being done, nor without an Admin setting to control it. See my comments.
Vote by Charlie A.
30343 | February 20, 2019 03:20:51 AM GMT
+1 this needs to be addressed.
Vote by Cory H.
30352 | February 20, 2019 03:25:24 PM GMT
One other facet of this new "feature" (change in behavior) is that it could also lead to CF2018 implicitly using more heap than previous versions would have. Since two or more apps that DO share a cache region might have code that populates that cache if it's detected to be empty, that would therefore cause a corresponding increase in heap use to hold those duplicated (or "triplicated" and so on) caches. Of course, the original point being made by Ben above was that he was being thrown off BECAUSE they were not identical, as expected. That led to errors for him. I could see it leading to logic errors for others, if again they have code that populates the region "if empty" and then code may later modify the cache, but they're unaware that technically these are NO LONGER the same region/cache. I'm just adding that there is a potential (and for some potentially significant) impact on heap use, by their code perhaps unknowingly creation multiple similarly-sized caches, one for each application using the "region", which they used to all "share" as one.
Comment by Charlie A.
30367 | February 21, 2019 05:46:29 PM GMT
Adobe folks?
Comment by Charlie A.
30677 | May 01, 2019 02:43:52 PM GMT
Adobe folks, can anyone please reply to my last two notes comments, from Feb? Also, I heard from someone (privately) that they'd been given a fix by Adobe that addressed this. Can you tell us if anyone can request that? Would we ask for it by the tracker id? Does the change revert behavior or add some new control for this?
Comment by Charlie A.
30720 | May 13, 2019 09:37:36 PM GMT
Hi Adobe, Can you please communicate? Thanks!, -Aaron
Comment by Aaron N.
30723 | May 14, 2019 08:32:21 AM GMT