ColdboxCache

Error parsing plugin definition: An error occured while Parsing an XML document. Element type "Redirect" must be followed by either attribute specifications, ">" or "/>".

<< Back to Dashboard

ColdBox Cache Guide

Introduction

The ColdBox Cache is an in-memory cache that lives in Application Scope designed for handlers, plugins, events, views and any other objects or data you so desire. It has various tuning parameters such as default object timeout, default last access timeout, maximum objects to have in cache, a JVM memory threshold, a reaping frequency, eviction policies, an event model and so much more. The cache is based on java soft references (explained further below), several concurrency classes (if using JDK 5 and above) and is multi-threaded, which makes it an incredibly stable and fast caching engine. The cache is also tied to an event broadcast model, implemented via ColdBox interceptors, that can announce when objects are cached, removed or expired. In this guide we will explore the concepts, configurations and how to use the ColdBox Cache.

Useful Resources

• http://ecet.ecs.ru.acad.bg/cst05/Docs/cp/SII/II.20.pdf
• http://www.ibm.com/developerworks/java/library/j-jtp01246.html
• http://java.sun.com/j2se/1.5.0/docs/api/java/lang/ref/SoftReference.html
• http://jdj.sys-con.com/read/36434_p.htm

ColdBox Cache Features

• Fast and Simple
  • Fast ColdFusion and Java hybrid cache
  • Simple API and basic configuration parameters
  • Small Footprint
  • No need to create it, configure it or manage it if used within a ColdBox Application
• Solid Core
  • Multi-Threaded
  • Based on Java Concurrency Classes
  • Multiple Eviction Policies: LRU,LFU and FIFO
  • Memory Management & Memory Sensitive caching based on Java Soft References
  • Production Tested
  • Fully Documented
• Extensible & Flexible
  • Cache Listeners for broadcasting when objects are:
    • inserted
    • removed
    • expired
  • Create your own custom eviction policies
  • Extensive and granular purging mechanisms (regular expressions and key snippets)
  • Cache Statistics API for creating custom cache reports
• Highly Configurable
  • JVM Threshold Checks
  • Object Limits
  • Ability to time expire objects
  • Eternal (singletons) and time-lived objects
  • Object purging based on object usage (Access Timeouts)
  • Fully configurable at runtime via dynamic configurations and hot-updates
• Visually Appealing and Useful
  • Fully featured caching monitor and commands panel
  • Complete cache performance reports

Inner Workings

The ColdBox Cache in versions 2.5.0 and later, have cf8/Bluedragon7/Railo3 support, in which it takes advantage of using the cfthread tag to create new threads for reaping and maintenance. Not only does it take advantage of coldfusion improvements but also improvements of the new java JDKs. It uses several new data structures introduced in JDK 5 that are used for concurrency and caching, that gives the cache more stability under high traffic and faster performance.

The cache is a dual-performing cache in that it will store eternal (singleton) and non-eternal (life span) objects side by side. The non-eternal objects are stored as java soft references that provide you with a memory aware caching engine. This relieves the cache from purging non-used objects or purging when the JVM needs to recover RAM. All this is provided by the java soft reference implementations, which make this cache adapt as memory is needed or reaped by the JVM. However, it is also a fixed size & dynamic configuration cache engine in which it can be size limited and also limited by several configuration options. There have been several studies in the computer science world delineating why having a mixed cache will not only improve performance as load increases, but will also increase stability. The combination of having java soft references, fixed limits, and eviction policies is what makes this caching engine unique and powerful.

Java Soft References

"In the past, developers didn't have much control over garbage collection or memory management. Java2 has changed that by introducing the java.lang.ref package. The abstract base class Reference is inherited by classes SoftReference, WeakReference and PhantomReference. SoftReferences are well suited for use in applications needing to perform caching."

Why is this? Well, a soft reference is nothing but a wrapper class that wraps an object in memory. Whenever the JVM requires memory and runs its reference rules, it can detect these soft references and decide to purge them if it meets the garbage collector rules. If it purges them, the wrapped object inside of the soft reference is marked for collection, but the java soft reference itself still exists. Therefore, the programmer can determine if the object inside the reference exists or not. If it does not exist, it means the object was garbage collected and I have no more object. I won't go into the implementation semantics of the cache, just theory. Please visit the references to understand more of how the ColdBox cache was built. In summary, soft references are what determine if an object is still available or not. The ColdBox cache can then run maintenance on itself and clean out all of its references for you.

Configuring the cache

The ColdBox cache can be configured framework wide by tweaking the framework's settings xml file or via the ColdBox Dashboard. However, you can override all the values in your application via its configuration file (coldbox.xml.cfm). I will list below a table of the main settings and the framework defaults.

Sample application configuration xml for your application's configuration file:

<Cache>
	<ObjectDefaultTimeout>45</ObjectDefaultTimeout>
	<ObjectDefaultLastAccessTimeout>15</ObjectDefaultLastAccessTimeout>
	<UseLastAccessTimeouts>true</UseLastAccessTimeouts>
	<ReapFrequency>1</ReapFrequency>
	<MaxObjects>50</MaxObjects>
	<FreeMemoryPercentageThreshold>0</FreeMemoryPercentageThreshold>
	<EvictionPolicy>LRU</EvictionPolicy>
</Cache>

As you can see, the cache is extremely configurable. So let's analyze each setting one by one.

Object Default Timeout

This numerical setting denotes the lifespan of objects in minutes. However, this is the minimum an object should live because there is no background process reaping the objects, the framework relies on incoming requests to cleanup the cache storage. Therefore, this timeout cannot be guaranteed, but it is a minimum.

Last Access Timeout

This numerical setting denotes the amount of time in minutes that an object if not used, should live. The best way to describe this setting is to provide an example. Let's say object A has a timeout of 40 minutes and a last access timeout of 15 minutes. This means that once the object gets created, a timer starts. If that object is not used again in the following 15 minutes, then the cache sees that it has not been used and evicts it. This is a very useful setting as it can provide an even higher rotation for your objects and provide a more dynamic cache. However, tweaking is important and monitoring also. There has to be good enough separation or analysis on object life span’s in order to apply this setting correctly.

Use Last Access Timeouts

A Boolean variable that determines if you want to activate the last access timeout expiration evictions.

Reap Frequency

A numerical setting that denotes the minimum frequency, in minutes, that the cache should reap itself. Reaping is always costly, so you will have to analyze your application in order to determine a correct frequency. Again, this setting is not guaranteed as it relies on incoming requests. So be sure to monitor your application and determine a realistic reaping frequency. Internally, the JVM will garbage collect all the soft references it needs as it requires RAM or not. All of this is provided by the JVM at its discretion (better for us). So our cache reaping basically cleans up what the JVM already cleaned up. All the hard work is done by the JVM, in order to maintain a safe and stable system.

Max Objects

This determines the maximum number of objects the cache can hold. If the cache has reached its limit, the new objects will not be cached until space is available. Therefore, it will evict objects form the cache using the selected eviction policy.

Free Memory Percentage Threshold

This setting is a numerical setting denoting the free memory percentage in the JVM. This is a fail-safe setting that basically tells the cache NOT to cache anything more if the JVM is running out of memory, plus it will evict objects using the eviction policy, until the JVM threshold is clear. So it can potentially recover some needed RAM if needed. However, since the cache is based on java soft references, the JVM gives an additional boost by garbage collecting those objects too.

Eviction Policies

ColdBox ships with the following eviction policies:

• LRU : Least Recently Used
• LFU : Least Frequently Used
• FIFO : First In First Out

However, the framework provides you with an interface that you can use in order to build your very own eviction algorithm classes and use them. Please look at the following interface: coldbox.system.cache.policies.AbstractEvictionPolicy.

LRU

With this eviction policy, the cache discards the least recently used items first.

LFU

This policy counts how often an item has been accessed and it will discard the items that have been used the least.

FIFO

This policy works as a queue, first object in will be the first object out of the cache. So older staler objects are purged.

Dynamically Changing Policies

The ColdBox cache let's you dynamically change the eviction policy of the cache by using the method setevictionPolicy(policy:coldbox.system.cache.policies.AbstractEvictionPolicy). With this functionality you can create sophisticated eviction mechanisms that can be changed according to your application's criteria, maybe traffic, memory consumption, you name it. With the ability to dynamically influence the eviction policy, you have control on how aggressive or not your policy can be.

<cfset getColdboxOCM().setEvictionPolicy(MyPolicyObject)>

How do I use It?

Well, the main methods for using the cache can be found in the ColdBox API. However, the main method that you need to memorize to use get cache manager within your application is getColdBoxOCM(). This method is used to retrieve the ColdBox cache manager so you can interact with the cache from within your handlers, plugins, interceptors, proxy, etc.

Below are some of the most used caching methods:

Cache Monitor Screenshot

What good is a cache if I can't see what is inside or monitor it. Well, that is why we built the cache monitor that you can see below. The monitor will let you know all the necessary statistics and metrics about your cache. You can even dump the contents of individual items if needed.

Cache Panel Commands

The cache content panel has some nice commands that you can use on-demand! This will help you issue commands to the cache when you are in debug mode ONLY:

• Clear cache entries
• View cache entries
• Clear all cached events
• Clear all cached views
• Expire all objects

How to open the caching panel?

In order to open the cache debug panel you must first be in debug mode. Please look at the [wiki:cbURLActions URL Actions Guide]. Once the panel is open, you can see a cache overview report and a cache content report. You can now even click on the name of the cached element in order to see its content.

The debugpanel url action can ONLY BE used when in debug mode. If you are not in debug mode, then the panel will not be rendered. You can open the monitor by clicking on the Open Monitor button inside of the cache panel of the debugger or you can open it by going to the following URL:

index.cfm?debugpanel=cache

Techniques

There are various ways that you can use the ColdBox cache and it all depends on the data you want to store and how to retrieve it. One method that I have used for storing queries in the cache is to use the on application start method to place the queries on the cache indefinitely.

//On the application start
<cfscript>
var qStates = getPlugin("ioc").getBean("lookupsService").getStates();
var qCountries = getPlugin("ioc").getBean("lookupsService").getCountries();

//Place in cache indefinitely until application reinitializes.
getColdBoxOCM().set('qStates',qStates,0);
getColdBoxOCM().set('qCountries',qCountries,0);
</cfscript>

//Then on my handlers or views/layouts I can do the following
<cfset qStates = getColdboxOCM().get('qStates')>

This is a simple example of placing data on the cache that will persist throughout the entire application. Another example is you can place a query to persist for 10 minutes:

<cfset getColdboxOCM().set('myQuery', myQuery, 10)>

However, when you do a get the method can fail because the query might have expired. That is why you have the lookup() method which tells you if the key you sent in exists in the cache. This is useful to create facade methods where you can do the following:

<cffunction name="getMetadata" access="private" returntype="any" output="false">
  <cfargument name="lookup" type="string">
  <cfscript>
  var oMD = structnew();
  if ( getColdboxOCM().lookup(arguments.lookup) ){
    oMD = getColdboxOCM().get(arguments.lookup);
  }
  else{
   //Create md from Transfer
   oMD = getTransfer().getTransferMetaData(arguments.lookup);
   //set back in cache
   getColdboxOCM().set(arguments.lookup, oMD, 10);
  }
  return oMD;
  </cfscript>
</cffunction>

This is a great example of using the ColdBox cache for checking and verifying data. As you can see, there are several ways to tap into the power of the ColdBox cache. The following example is to create your own url action to expire objects that is part of a request start handler.

<cffunction name="onRequestStart" access="private" returntype="void" output="false">
  <cfargument name="event" type="any" required="true">
  <cfscript>
  //Request start handler.

  //check if you pass in a url var called: clear cache
  if ( event.valueExists("clearcache") ){
     getColdBoxOCM().expireAll();
     setNextEvent();
  }
  </cfscript>
</cffunction>

Plugin/Handler Caching

The framework can keep persistence for you for handlers and plugins. Plugin persistence is done automatically, all you need to do is set several caching parameters in your plugin's cfcomponent declaration. Handlers act the same way, in which you need to add the caching parameters to the cfcomponent tag, but also require a setting called: HandlerCaching. You need to activate this setting in order for handlers to be cached by the framework.

Enabling Handler Caching

To enable handler caching, you will need to set a setting in your configuration file called HandlerCaching which is a Boolean variable.

<Setting name="HandlerCaching" value="true" />

If you do not enable this setting, the framework will not cache your handlers. It would be a good idea to have this set to false in development, so your changes can be reflected.

Caching Parameters

Also, the framework detects if these parameters exist in the cfcomponent declaration. If they do not exist, it has some internal rules that will determine if the object is cached or not. Below is a table determining caching defaults:

As you can see from above, by default handlers WILL BE cached, unless you specifically use the cache meta data attributes to tell the framework NOT to cache it. Caching of handlers simulates persistence, so remember this if you are planning handlers that can maintain their own persistence. This is a true flexible and awesome feature. Persistence controlled by the framework for you. Same applies to plugins, but plugins are NOT cached by default. You need to specifically use the cache metadata attributes to tell the framework to cache them. Below are the cache metadata parameters you will add to the cfcomponent tag.

<cfcomponent name="mainHandler" output="false" cache="true" cacheTimeout="0">
</cfcomponent>

<cfcomponent name="myPlugin" output="false" cache="false">
</cfcomponent>

<cfcomponent name="captchaPlugin" output="false" cache="true" cacheTimeout="30" cacheLastAccessTimeout="10">
</cfcomponent>
<cfcomponent name="captchaPlugin" output="false" cache="true" cacheTimeout="5">
</cfcomponent>

<cfcomponent name="userHandler" output="false" cache="true" cacheTimeout="30" cacheLastAccessTimeout="10">
</cfcomponent>

Interceptor Caching

All interceptors declared in the configuration file will be cached automatically by the framework as singletons. In order, for changes to take effect in interceptors, you must re-initialize the application by using the fwreinit action url.

View Caching

In order for you to cache a view, you will have to use the event.setView() or the actual renderView() methods. You can read an in-depth guide about view caching in the Layouts & Views Guide. This guide will also show you how to purge views from the cache.

So in order to tell the framework to cache the view called vwHome for 10 minutes, you would do the following:

event.setView(view="vwHome",cache="true",cacheTimeout="10");

That easy! The framework reads the caching metadata, constructs a cache entry and then saves it for pickup by the renderer plugin. How can you tell if the view was rendered from cache or not? Well, if you are in debug mode, you can check out the debugging panel and you will see that if you are normally rendering a view it says: Rendering View [vwHome.cfm]. However, if rendering from cache, it will say: Rendering Cached View [vwHome.cfm]

Here is an example of rendering and caching at the same time:

<body>
	
	<---< Render and cache the header --->
	#renderView(name='tags/header',cache=true,cacheTimeout='10')#
	
	<div>
	<---< Main View --->
	#renderView()#
	</div>
</body>

Event Caching

Event caching is extremely useful and easy to use. All you need to do is add several metadata arguments to the cffunction tag of your event and the framework will cache the output of the event. In other words, the event executes and produces output that the framework then caches. So the subsequent calls do not do any processing, but just output the content. Almost all of the entire life cycle is skipped, the content is just delivered. Below you can see the life cycle of a cached event:

Cached Event Life cycle

1. Event is trapped
2. URL action fwCache is verified, if it exists, incoming event is purged from cache.
3. preProcess Interception is executed
4. RequestStartHandler is executed
5. Cached content is retrieved and outputted.

As from the warning above, when event caching is enabled for an event, the framework will hash the incoming request collection plus the incoming event to determine a cacheable key. So an event can have thousands of different combinations that will create multiple cache entries. So make sure that you have the Cache Max Objects setting turned on, so your cache can be more dynamic.

Enabling Event Caching

To enable event caching, you will need to set a new setting in your configuration file called EventCaching which is a Boolean variable.

<Setting name="EventCaching" value="true" />

If you do not enable this setting, the framework will not cache your events. It would be a good idea to have this set to false in development, so your changes can be reflected.

Setting up an event for caching

The way to set up an event for caching is on its cffunction declaration with the following extra arguments:

Here is an example of setting up an event for caching:

<cffunction name="dspBlog" access="public" returntype="void" output="false" cache="true" cacheTimeout="30">

That's it, whenever the framework detects the dspBlog event, it will cache it.

Event Cache Suffix

This property is great for adding your own dynamic suffixes when using event caching. All you need to do is create a public property called EVENT_CACHE_SUFFIX in your event handler and populate it with something you want; static or dynamic. A great use case for this, is multi-lingual applications. You can set the suffix to the user's locale. Then the event caching mechanisms will automatically append the suffix and thus create event caching according to locales now. How cool is that, a simple key and you have multi-lingual caching at your fingertips.

<---  Set it to the user's current locale --->
<cfset this.EVENT_CACHE_SUFFIX = getFWLocale()>

Purging Events

If you need to override the event so you can see the real deal, you can use an URL action called: fwCache. This URL action tells the framework to not cache the incoming request. If the event is cached, it will first purge it and re-execute and re-cache it. If you want to purge all events, well, just reinit the application using the fwReinit URL action or call the clearAllEvents() method on the ColdBox cache; this will clear all the events in the cache. However, there are other methods that can clear a family of cached events, or family of handler events and more.

Ways to purge events:

• URL Action fwCache
• getColdboxOCM().clearAllEvents([async=true])
• getColdboxOCM().clearEvent(eventSnippet,[queryString=""],[async=true])

Examples:

//Trigger to purge all Events
getColdBoxOCM().clearAllEvents();

//Trigger to purge all events synchronously
getColdboxOCM().clearAllEvents(async=false);

//Purge all events from the blog handler
getColdbocxOCM().clearEvent('blog');

//Purge all permutations of the blog.dspBlog event
getColdbocxOCM().clearEvent('blog.dspBlog');

//Purge the blog.dspBlog event with entry of 12345
getColdbocxOCM().clearEvent('blog.dspBlog','id=12345');

As you can see from the clearEvent() samples, you can send in snippets of event names and the cache will try to find keys that match and reap them. You can also do this asynchronously if cfthread is available in your engine.

Cache Monitor Alongside Application Screenshot