Going Through the Garbage – Memory Issues for ColdFusion and the JVM

Restart, Restart, Restart

We had an interesting problem the other day – one of our ColdFusion 9 servers was spontaneously stopping and restarting. I had been at it for a few days it seems.

I only actually noticed it when I got a ‘service not available’ message when running a .cfm page on the server, but on closer inspection I noticed lots of hs_err_pidXXXX files littering the {cf_install}\runtime\bin directory.

Looking at one of these files showed that a fatal error had been detected by the JRE, namely:

java.lang.OutOfMemoryError: requested 32756 bytes for ChunkPool::allocate. Out of swap space?

This is a pretty drastic situation the JRE found itself in, so it killed itself. There’s lots of info in this dump, but the interesting bit for me was the following section about the heap:

PSPermGen total 69248K, used 68830K [0x04800000, 0x08ba0000, 0x10800000)
object space 69248K, 99% used [0x04800000,0x08b37bb8,0x08ba0000)

99% used – sounds no good at all.

The Generation Game

I realized I’d need to figure out the whole generation business, so I did that by reading up on a few articles. I’ve put them here so others can find all the resources in one place.

* Tuning Java garbage collection as described by Oracle. A killer article that will make you dizzy. Try to pick up the basics from this.

* Know the ColdFusion angle, and know it’s limits, especially on a 32 bit machine. When you set your heap space too high in CF admin and you’re wondering why CF is not starting, you’ll need to fix it with this technote and there’s a helpful explanation why the number won’t go as high as you thought here.

* When you do start playing around with the generations (which JSP applications often need to, and ColdFusion is JSP) you might like to know what the Permanent Generation actually is. You can find a nice explanation here.

* If (or when) you need to really stinky and get into the garbage and see what’s going on there’s a great free tool to help you. It’s called jconsole, you use it to monitor JVM applications and it’s free. It’s described in detail here.

A few notes about jconsole – it’s part of the Java Development Kit (JDK) and not to be confused with the JRE (Java Runtime Environment). God knows I used always get them mixed up. If you install the latest JDK you see under the \bin directory that a number of executable jar files are included. If you run jconsole you’ll see a list of PIDs (Process IDs) you can monitor with this tool. One of these is ColdFusion.

Just remember you must have CF running as a console, i.e. NOT as a service. If you’re running CF as a Windows service you’ll need to disable your CF service and then run CF from the command line via the {cf_install}\bin\cfstart bat file. Restart jconsole and you should see a new PID to monitor – that’s ColdFusion.

Using jconsole you can see all sorts of good things such as the size of your generations, the duration of garbage collection for each generation and the amount of memory being used.

Every application is different so you might need to tweak away at JVM startup parameters and ColdFusion settings before finding the right settings for your server. Good luck!

Advertisements
This entry was posted in regular and tagged , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s