I have been investigating VisualVM as a profiling tool to help track down and optimize some performance problems on my current project. VisualVM comes bundled with newer versions of the Java 6 SDK so it’s free and already available to use. The thing I like most about VisualVM is the relatively simple and intuitive User Interface, unlike say JProbe. Here’s a simple tutorial to get started with profiling with VisualVM. However, it was a PITA to get VisualVM to profile an application running inside the JBoss container.
Make sure you download the latest version of the JDK to use its profiler. I had problems using 6.0.18 and running 6.0.30 seems to have fixed many issues. Your application can run another version of the JVM and still connect with the profiler.
I was able to attach VisualVM to JBoss and monitor the CPU, live threads, and memory, etc. That worked without a hitch. When it came to actually profiling the application, JBoss kept crashing in all sorts of wonderful ways. At first, I was even fooled into thinking things were working nicely after I saw the following messages in my server log.
Profiler Agent: JNI On Load Initializing... Profiler Agent: JNI OnLoad Initialized successfully Profiler Agent: Waiting for connection on port 5140 (Protocol version: 10) Profiler Agent: Established connection with the tool Profiler Agent: Local accelerated session ... Profiler Agent: 250 classes cached. ... Profiler Agent: Redefining 100 classes at idx 2200, out of total 2362
And then the pain began. I’ll outline the steps that I took to get things working and save you hours of frustration.
Modify the JBoss start up
Whether you start JBoss from within an IDE like Eclipse or running from the command line, you need to modify some of the start up parameters.
Out of PermGen Space
I imagine that VisualVM is dynamically creating proxies for all the classes and that’s how it measures method timings. PermGen space can quickly run out so you should increase that to at least 256M.
Out of Heap Memory
I had to increase the maximum heap size for JBoss, what the actual value should be is dependent on your application, mine was increased to 768M. There’s no harm in increasing the maximum, the JVM won’t allocate all of it right off the bat if it’s not needed. The theoretical maximum is dependent on your OS and whether the JVM is 32 or 64 bits.
ProfilerRuntimeCPUFullInstr Class Not Found
After getting JBoss to run for a while with the profiling, I ran into java.lang.NoClassDefFoundError: org/netbeans/lib/profiler/server/ProfilerRuntimeCPUFullInstr messages in my server log and the application actually fails to function properly. Thus, useless for profiling!
You *must* include the jfluid-server.jar in the JBoss bootstrap classpath. This jar *must* come from the same JDK version as the VisualVM that you’re using. It does not have to be the same as the version of JVM that runs JBoss. This jar can be found in the folder under $JAVA_HOMElibvisualvmprofilerlib. If you start JBoss on the command line, add something like the following to your start up script. Otherwise, add the jar to your IDE’s launch target for JBoss.
Restart JBoss if necessary, and you should now be able to profile your application(s) within the container.
- Keep the number of classes that you profile down to something reasonable by filtering out packages that are beyond your control unless you’re fixing something in the infrastructure, i.e. java.*, javax.*, sun.*, sunw.*, com.sun.*, org.jboss.*, etc.
- Because of the dynamic instrumentation generating nature of VisualVM, the first run during your profile is way off and the numbers can’t be trusted whatsoever. So if you need to profile first run performance to analyze and tune initialization costs, this is not the right tool to use.