HPjmeter 4.
© Copyright 2005-2013 Hewlett-Packard Development Company, L.P. Copy of Software License END USER LICENSE AGREEMENT FOR SOFTWARE PLEASE READ CAREFULLY: THE USE OF THE SOFTWARE IS SUBJECT TO THE TERMS AND CONDITIONS THAT FOLLOW (“AGREEMENT”), UNLESS THE SOFTWARE IS SUBJECT TO A SEPARATE LICENSE AGREEMENT BETWEEN YOU AND HP OR ITS SUPPLIERS.
provided by HP in writing. Nothing in this Agreement grants You any right to purchase or receive Software updates, upgrades, or support, and HP is under no obligation to make such support available to you. Updates, upgrades, enhancements, or other Support may only be available under separate HP support agreements. You may contact HP to learn more about any support offerings HP may make available.
installation by HP more than thirty (30) days after delivery, the warranty period begins on the 31st day after delivery. This limited warranty is subject to the terms, limitations, and exclusions contained in the limited warranty statement provide for Software in the country where the Software is located when the warranty claim is made. b. HP warrants that any physical media containing HP Branded Software will be shipped free of viruses. c.
. LIMITATION OF LIABILITY AND REMEDIES Notwithstanding any damages that You might incur, and except for damages for bodily injury (including death) and for the amounts in section 4.a, the entire aggregate liability of HP and any of its suppliers relating to the Software or this Agreement, and Your exclusive remedy for all of the foregoing, shall be limited to the greater of the amount actually paid by You separately for the Software or U.S. $5.00.
ASM: a very small and fast Java bytecode manipulation framework Copyright (c) 2000-2011 INRIA, France Telecom All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2.
For example, on rare occasions, there may be a special need to encourage the widest possible use of a certain library, so that it becomes a de-facto standard. To achieve this, non-free programs must be allowed to use the library. A more frequent case is that a free library does the same job as widely used non-free libraries. In this case, there is little to gain by limiting the free library to free software only, so we use the Lesser General Public License.
copy, so the ordinary GNU General Public License applies to all subsequent copies and derivative works made from that copy. This option is useful when you wish to copy part of the code of the Library into a program that is not a library. 4.
8. You may not copy, modify, sublicense, link with, or distribute the Library except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense, link with, or distribute the Library is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 9.
1. License. Oracle grants to Licensee a personal, non-exclusive, non-transferable license to use the Program only: (i) to provide an online help system for Licensee’s use with Licensee’s applications, and in accord with the terms set forth in this Agreement, and (ii) to distribute the Program as part of Licensee’s own products, provided that the Program is only used as an online help system with Licensee’s applications. Licensee shall not: a.
Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. Hewlett-Packard shall not be held liable for errors contained herein or direct, indirect, special, incidental, or consequential damages in connection with the furnishing, performance, or use of this material.
Contents About This Document...................................................................................19 Intended Audience..................................................................................................................19 Typographic Conventions.........................................................................................................19 Additional HPjmeter Documents................................................................................................
Setting Data Collection Preferences...........................................................................................46 Managing Node Agents.........................................................................................................47 Managing Node Agents On HP-UX......................................................................................47 Running Node Agent as a Daemon.................................................................................
Profiling from HPjmeter with Zero Preparation........................................................................74 Configuring Remote Operations window...............................................................................75 Profiling with -agentlib:hprof................................................................................................76 Naming Profile Data Files...................................................................................................
JVM Agent.................................................................................................................112 Open and Cached Sessions.........................................................................................113 Time Slice Entries........................................................................................................113 Saving Data...............................................................................................................
Percentage of Time Spent in Garbage Collection.............................................................143 Guidelines.............................................................................................................143 Details..................................................................................................................144 Unfinalized Objects.....................................................................................................144 Guidelines...................
Average Inclusive Method Clock Times*.........................................................................166 Starvation by Method*................................................................................................166 Starvation Ratio*........................................................................................................167 Methods with Loops*...................................................................................................167 Exclusive Class CPU Times*....
Tool Bar Buttons for Manipulating Graphical Data...............................................................202 Tool Bar Buttons for Manipulating Garbage Collection Data.............................................202 Special Button or Other Gadget Functions...........................................................................203 Mark an Item for Search..............................................................................................203 Using the Marked Object List on the Console........
About This Document This document provides updated information about product features, known problems, and workarounds for the 4.3 release of HPjmeter. Intended Audience This document is intended for application administrators and software developers responsible for monitoring Java application performance and operation on HP-UX. Users are expected to have some knowledge of concepts and/or commands for the HP-UX operating system, usage of the Java Virtual Machine on HP-UX, and Java application performance.
HP Encourages Your Comments We encourage your comments concerning this document. We are committed to providing documentation that meets your needs. Please send your comments via the Customer Feedback web page: http://www.hp.com/bizsupport/ feedback/ww/webfeedback.html. Include the document title, HP part number, and any comment, error found, or suggestion for improvement you have concerning this document. Also, please let us know what we did right so we can incorporate it into other documents.
1 Introducing HPjmeter 4.3.00.00 HPjmeter is a performance analysis tool for deployed Java applications. It will help you identify performance problems in your production environment as well as during development. NOTE: You cannot use HPjmeter to monitor Java applets in a production environment. HPjmeter helps you diagnose many types of Java application problems that occur only after a product is deployed.
• Visibility into user-defined MBeans, with the ability to modify attributes and trigger operations in real time, then monitor the resulting application behavior.
Node Agent Node agents manage communication between JVM agents and consoles. The preferred way of using node agents is to start the node agent as a daemon as root on each managed node.
2 Completing Installation of HPjmeter NOTE: This chapter assumes that you have installed the HPjmeter components on the appropriate server(s) and are ready to configure the agents. For information on initial installation, see HPjmeter Release Notes and Installation Guide. Platform Support and System Requirements Although the agents and console can execute on the same system, this document assumes they are installed on separate systems—the recommended configuration.
Table 2 Console Requirements Operating system and architecture The console is a pure Java application, so it should execute on any platform that supports Java. Java Java 6.x or 7.x versions. Patches and updates HPjconfig can help you determine which Java patches are recommended or required for best operation of the HPjmeter console on HP-UX. See http://h20392.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=HPJCONFIG . Processor and memory • Minimum 500 MHz processor is recommended.
To Take Advantage of Dynamic Attach: Check that JM_JAVA_HOME in $JMETER_HOME/config/hpjmeter.conf is set to a Java 6 (6.0.03 or later) or a Java 7 directory to be able to later dynamically attach to a running JVM from the HPjmeter console without first setting HPjmeter options on the command line. When HPjmeter is installed on a system that has Java 6 or 7 installed in the usual location, the installation procedure will automatically store the JDK location in hpjmeter.conf configuration file. If the Java 6.
$ /opt/java1.4/bin/java -Xms256m -Xmx512m -Xbootclasspath/a:/opt/hpjmeter/lib/agent.jar -Xrunjmeter myapp NOTE: With the addition of JVMTI in JDK 5.0, you should use the -agentlib switch to take advantage of improvements in JDK 5.0 and reduce the impact of data sampling on application performance. While -Xrunjmeter can still be used to specify the JVM agent with Java 5 versions, the impact on application performance is significantly greater than when using -agentlib. Starting with Java 5.
include=filter1[:filter2]..., exclude=filter1[:filter2]... Creates a colon-separated list of classes to specify which classes or packages are instrumented. Method-level filtering is not supported. • If a class is not instrumented, the JVM agent metrics that use bytecode instrumentation (BCI) do not provide any output related to the class methods. To see the list of filters in effect while the data was collected, including default agent filters, click the you see it in the console.
• The default filters always include the following packages. They cannot be overriden with an exclude option: sun.jdbc.odbc. weblogic.jdbc.informix4. org.apache.jsp. weblogic.jdbc.mssqlserver4. org.apache.jasper. weblogic.jdbc.oci. weblogic.jdbc.rowset. com.bea.p13n. com.bea.netuix. oracle.jdbc. oracle.sql. oracle.evermind.sql. com.ibm.samples. com.sun.j2ee.blueprints. com.ibm.websphere.samples • com.bea.medrec. com.sun.ebank. You can change the default behavior by using the include option.
-agentlib:jmeter=monitor_batch ◦ To specify a file name: -agentlib:jmeter=monitor_batch:file=mysaveddata.hpjmeter noalloc Reduces dormant overhead by skipping bytecode instrumentation that applies to object allocation metrics: Allocated Object Statistics by Class and Allocating Method Statistics. The noalloc option makes those metrics unavailable. Example usage: -agentlib:jmeter=noalloc nohotspots Reduces dormant overhead by skipping the bytecode instrumentation that supports Java Method HotSpots.
nothreads Reduces overhead by turning off the collection of data that support the Thread Histogram and Lock Contention metric. Example usage: -agentlib:jmeter=nothreads nolockcon Reduces overhead by turning off the collection of data that supports the Lock Contention metric.
$ $ $ $ -agentlib:jmeter=noalloc,appserver_port=7001,public,include=weblogic:com.bea -agentlib:jmeter=nohotspots,owner_private,verbose:file=bcifilters.txt -agentlib:jmeter=version -agentlib:jmeter Security Awareness Securing Communication Between the HPjmeter Node Agent and the Console IMPORTANT: The data stream between the HPjmeter console and agents is not protected from tampering by a network attacker.
IMPORTANT: Setting access for owner or group should not be considered a security solution because node agent to JVM communications are not secured by default—see below. Securing Communication Between the JVM and the HPjmeter Node Agent IMPORTANT: The data stream between the JVM and the node agent is not protected from tampering by a user logged into the system running the JVM.
3 Getting Started Are You Monitoring an Application or Analyzing Collected Data? HPjmeter helps you to monitor a live run of your application. You can also view saved data captured using HPjmeter during a live session, by using profiling or garbage collection options when you start your application, or if you are running HP JDK/JRE 5.0.04 (or higher), by starting and stopping profiling data collection from the command line.
1. Choose Connect from the File menu or click Connect to server [ 2. In the Connect to Server dialog box, type the complete host name (server.domain.com) on which a Java application and a corresponding node agent are running. Optionally, in the Optional Port box, specify the same port number that was used when the node agent was started. 3. 4. ] Click Connect.
1. Choose Connect from the File Menu or click Connect to server [ 2. In the Connect to Server dialog box, type the complete host name (server.domain.com) of the system that is the designated host for the SSH tunnel (the SSH delegate machine). Optionally, in the Optional Port box, specify a port number through which information can pass from the SSH delegate to the node agent. Check the box labeled Use SSH Options. 3. 4. ] The Connect to Server dialog expands to reveal required SSH attributes. 5. 6.
9. Repeat the previous steps if you want to connect to several node agents. Set Session Preferences 1. Double-click on the JVM icon in the data pane for the application that you want to monitor. This opens the Session Preferences dialog box. NOTE: When the Optional Port Box Is Not Used The console automatically attempts to use a port on the remote system between 9505 and 9515 when arranging a port for its server socket.
Some alerts require collection of specific metrics such that even if a metric is de-selected, it may be activated by selection of a related alert. 3. Once all selections are made, click OK. The preferences window will close and the new Open Session will be visible, marked by this icon: Though it is possible to begin viewing application activity immediately in the monitoring visualizers, you may want to wait for a short period of time before viewing some of the metrics.
The length of time needed to collect sufficient data depends on the application size, the load imposed on the application, and on the selected preferences. Typically, a wait of 5 to 30 minutes provides sufficient data to begin seeing significant clues to the application behavior and performance. A longer collection time gives you greater accuracy in the results. For details, see: Setting Monitoring Session Preferences (page 117) . Changing Session Preferences During a Session With HPjmeter 4.
NOTE: If you are running the HP JDK/JRE 5.0.04 or later, you can send a signal to the running JVM to start and stop a profiling data collection period with zero preparation and no interruption of your application. See Profiling with Zero Preparation (page 74). If you are running the HP JDK/JRE 7.0.04 or later, you can also use the HPjmeter console to start and stop a profiling data collection period with no interruption of your application. See Profiling from HPjmeter with Zero Preparation (page 74) 1.
6. Click among the tabs to view available metrics. Use the Metrics or Estimate menus to select additional metrics to view. Each metric that you select opens in a new tab. Mousing over each category in the cascading menu will reveal the relevant metrics for that category.
4. Click File→Open File to browse for and open the data file, or drag and drop the file onto the console. 5. A GC viewer window opens and displays a set of tabs containing summary and graphical metric data. Click among the tabs to view available metrics. 6.
$ cd $JMETER_HOME/demo $ ../bin/run_simple_jvmagent -jar sample_program Use the file name of the specific sample you want to run in place of sample_program. 4. 5. 6. In the console main window, click Connect and type in the host name of the machine that runs the sample application. If you specified a port number when starting the node agent, use the same port number. Otherwise, leave the port number box empty. An icon representing the host appears in the main window.
Figure 1 Heap Display Showing Possible Memory Leak Behavior See also Heap Usage Alert (page 129). The Pi sample program uses a simple Java program that allocates some arrays, thereby exhibiting the behavior of a memory-leaking application. The demo application is available from the HPjmeter installation directory: Source: $JMETER_HOME/demo/Pi.java Binary: $JMETER_HOME/demo/Pi.jar Use -jar Pi.jar with the run_simple_jvmagent script to start the sample.
Source: $JMETER_HOME/demo/DL1.java Binary: $JMETER_HOME/demo/DL1.jar Use —jar DL1.jar with the run_simple_jvmagent script to start the sample. Use the Thread Histogram display to see the thread activity. Deadlocked threads show a solid red bar.
4 Monitoring Applications In addition to using HPjmeter when you notice a problem, you can also use it to prevent problems. For example: • You can use HPjmeter to check the efficiency of your applications, even if you don't notice a problem. To make it easier to detect problems, you should periodically review key metrics for your application to establish a baseline profile. To save metrics information to a file that you can review later, see Saving Monitoring Metrics Information (page 48).
Managing Node Agents Each managed node requires an HPjmeter node agent that manages communication between the console and JVM agents. Once the Session Preferences window closes, and after waiting for a few moments, you should see the running JVM agents listed below the connected server host name. If no JVM appears, check that a node agent is running on that server. If no node agent is running, the console cannot show the JVM agents.
Start the node agent from the command line by typing: $ JMETER_HOME/bin/nodeagent where $JMETER_HOME has the default value of JMETER_HOME=/opt/hpjmeter You can run the node agent as a background process.
NOTE: Saved data files can be quite large. If you expect to keep the files or you plan to transfer the files, compress the files for speedier transfer. Compressing the files also reduces the chances that the file could be corrupted during a transfer. You can compress various profile data files; for example: .vgc, .hprof (ascii text and binary format), .eprof, and .hpjmeter. The preferred compression format is gzip. HPjmeter 4.1 and higher can read gzip compressed files, without having to expand them.
Related Topics • Java Method HotSpots (page 136) • Process CPU Usage Alert (page 131) • System CPU Usage Alert (page 132) Viewing the Application Load Use the Heap Monitor display. This metric shows how busy the application is. It checks to see whether it is doing lots of allocations, which typically corresponds to the load level, or whether it is idle.
You can verify this using a system tool like HP GlancePlus. The possible remedies for thrashing: • Decrease the maximum heap size, which corresponds to a decrease of the maximum load supported by the application. • Eliminate other load on the system. • Install more physical memory. Related Topics • Heap Monitor (page 140) • Garbage Collections (page 141) • Heap Usage Alert (page 129) Identifying Excessive Calls to System.gc() Use the Heap Monitor display.
Figure 3 Example Metric: Percentage of Time Spent in Garbage Collection When Application Behavior is Normal This next example shows an application with a potential memory leak. • A rising trend in the percentage of time spent in garbage collection.
• Array Leak Locations Alert (page 130) • Class Loaders (page 173) Checking for Proper Heap Sizing Efficiencies in program performance can be obtained by allocating the optimal amount of memory for the heap according to the needs of the program and the operation of the JVM garbage collection routine. Checking activity in the heap and comparing to garbage collection frequency and duration can help you determine optimal heap size needed for best performance from your application.
Selecting large view ports, one hour or longer, allows you to easily see the trend in the heap size after garbage collection and its increase rate, if any. You can visually assess the danger associated with the memory leak, and estimate, based on the knowledge of the total heap size, when the application would run out of memory, whether it is a matter of minutes, hours, or days.
Related Topics • Abnormal Thread Termination Alert (page 126) • Thread Histogram (page 152) Identifying Multiple Short-lived Threads Use the Thread Histogram. Threads are relatively costly to create and dispose; usually a more efficient solution is through thread pooling. Related Topics • Thread Histogram (page 152) • Identifying Abnormal Thread Termination (page 54) Identifying Excessive Lock Contention Use the Thread Histogram. There is no simple answer to how much lock contention is excessive.
Related Topics • Thread Deadlock Alert (page 133) • Thread Histogram (page 152) Identifying Excessive Thread Creation Use the Thread Histogram to identify excessive thread creation. The Thread Histogram shows the number of threads existing in the observed period of time. By selecting the smallest time interval, one minute, you can see an estimate of the number of simultaneously active threads. Most operating systems have a limited capacity for the number of threads a single process may create.
due to the stress on the JVM, such phenomena are typically much more costly than just running this method exclusively in interpreted mode. Repeated compilation problems may result in SLO violations and will affect some or all transactions. Read the JVM Release Notes to learn how to disable compilation of selected methods for the HP Java HotSpot Virtual Machine.
Figure 6 Main Console Showing an Expanded JMX Server Node in an Open Session To access this summary data, double-click on the summary JMX server node. The summary JMX Viewer opens. For information on this part of the JMX Viewer, see Understanding the JMX Summary View (page 58). To manipulate some JVM and MBean functions on this server, expand the summary node and double-click the entry labeled Modify JMX MBean data. The server JMX Viewer opens.
Figure 7 Appearance of the Summary JMX Viewer When First Opened JMX Memory Tab Fluctuations in memory usage in heap and non-heap areas are graphically displayed and periodically updated for the duration of the open session. Mouse over the features of the graph to learn what spaces are represented, as well as to learn what each of the markers designates. The following image demonstrates some of the features available on this tab.
Figure 8 JMX Viewer with Summary Memory Tab Selected Select a region of interest Click the color bar in the graph to select the memory space of interest. The bar will become outlined in blue and the Region Details will update in the text area. Consult the Region Details for updated information on memory usage in that region, total count of garbage collections, and cumulative duration of GC events.
Set a usage threshold Where usage threshold can be set, a “0” (zero) appears in the Usage Threshold box. To set a threshold, replace the current value with a desired, valid value in the box, and press Enter. To see what values are valid for a region, mouse over the Usage Threshold box. On pressing Enter, a red marker will appear on the region usage bar to mark the point at which the threshold is reached. The bar color turns red when the threshold is reached and a notification is generated.
Figure 9 JMX Viewer with Summary Threads Tab Open See the list of live threads Pull the slider in the data pane to the right to reveal the current list of live threads. See details for a selected thread Click a thread name in the left pane. Details on thread activity and current state appear in the right pane. Apply a filter This filtering text box is useful when the thread list is long. It allows you to reduce the list size according to the starting letters of the thread name.
Related Topic • Using Monitoring Displays (page 135) JMX Runtime Tab The Runtime tab summarizes important characteristics of the runtime environment, including data on the JVM version and uptime, options used to start the monitoring agent, and some aspects of the hardware and operating system such as memory assignment, swap space, and operating system type and version. JMX Notifications Tab Notifications triggered by changes that you make using the JMX viewer appear on the Notifications tab.
JMX server drop-down menu The server from which this JMX viewer was launched is shown in the “JMX Servers” drop-down menu. JMX MBean drop-down menu This drop-down menu lists the viewable MBeans for which data can be displayed in the viewer. This list can get quite long when viewing data for application servers. To reduce the list length in the drop-down menu, click the MBean filter button at .
Figure 11 MBean Attributes Tab Open for Display List of name/value pairs found in the selected MBean The names are listed with their value. Values may be dynamically updating; you may open them to reveal further details as in , or you may edit them as in . Clickable value Names or values appearing in boldface type can be opened to display details in the Detail Viewer area. Double-click the boldface text to open a tab in the Detail Viewer as in .
The MBean Operations Tab Not all MBeans have operations associated with them. This tab is grayed out when the selected MBean has no operation associated with it. The next figure shows an open Operations tab with details about a particular MBean. An explanation follows the image. Figure 12 MBean Operations Tab Open for Display Selected Operation Name The names are listed with their return type and number of arguments.
The MBean Notifications Tab Not all MBeans have notifications associated with them. This tab is grayed out when this information is not present. The next figure shows an open Notifications tab with details about the selected MBean. An explanation follows the image. Figure 13 MBean Notifications Tab Open for Display Enable or disable notification Click the check box to enable or disable the notification.
Figure 14 MBean Information Tab Open for Display View classification information Name/Value pairs are given that provide relational information for the selected MBean.
5 Profiling Applications HPjmeter allows you to process profile data from Java virtual machines. Separating the profile data collection step from the analysis step has the following advantages: • The data analysis can be done at a different time and on a different platform than was used to run the application. For example, it can be done on a desktop system or on a laptop. • A non-interactive profiling agent will often impose less overhead than an interactive one.
Generally, a metric is a mapping that associates numerical values with program static or dynamic elements such as functions, variables, classes, objects, types, or threads. The numerical values may represent various resources used by the program. For in-depth analysis of program performance, it is useful to analyze a call graph. Call graphs capture the “call” relationships between the methods.
There are two important rules to remember when modifying programs to improve performance. These might seem obvious, but in practice they are often forgotten. • Don't put performance above correctness. When you modify the code, and especially when you change some of the algorithms, always take care to preserve program correctness. After you change the code, you'll want to test its performance. Do not forget to test its correctness.
and optimization, and the application initializations (database connections, threads, and data structures). ◦ • If the benchmark runs for too long, it will be very time consuming to repeat the run, and you'll be tempted to make several changes to your application between the consecutive profiling runs, which is not recommended. As your application grows in terms of performance, you may need to scale up your benchmark as well.
Table 3 Supported -Xeprof options time_on=integer Specifies the time in seconds between the application start and the time when the profile data collection will start. If no time_on option is present, the data collection begins at the VM initialization. time_on=sigusr1|sigusr2 Specifies which signal will cause profiling to begin (profile data collection). • Be aware that the application or the VM may already be using the sigusr signals for their own purposes.
Table 3 Supported -Xeprof options (continued) The default value for this option is enable. Another way to disable inlining is to collect the profile data while running the VM in interpreted mode (-Xint). ie=yes|no Enable/disable the profiling intrusion estimation. ie=yes, the default value, specifies that the profiler estimates the profiling intrusion and writes the estimated values to the profile data file.
3. 4. 5. to open the Remote Operations window. Press the Configure button to set up your required configuration, if needed. Press the Profile button to start profiling When the profiling time period has ended, one of the following tasks are performed based on your configuration: • If the appropriate configuration options are set, HPjmeter will automatically transfer the profiling data file from the remote server to the local machine, and open the data file.
must also ensure that the host name specified here must match the name of the host used in the open session. Use known_hosts file Click the checkbox if the specified known_hosts file should be used to connect to the remote host. Use public key identity Click the checkbox if the specified public key file should be used for logging in to the remote host. If a public key is not used or a password is required, you will be prompted when the sftp connection is opened.
The following options are useful: • For performance profiling: cpu=samples,thread=y,depth=10,cutoff=0,format=a • If you must have method call counts for performance profiling: cpu=times,thread=y,cutoff=0,format=a • For analyzing object allocations: heap=sites,cpu=samples,thread=y,cutoff=0,format=a • For solving memory retention problems ("memory leaks"): heap=all,thread=y,cutoff=0,format=a,doe=n Then use kill -QUIT pid (on UNIX ) to get the heap dump. NOTE: Beginning with HPjmeter version 3.
Table 4 Supported -agentlib:hprof options (continued) Remember that a heap dump contains all objects, not just live objects cpu=samples|times cpu=samples uses sampling as the collecting technique, while cpu=times uses tracing with reduction. • Typically, the times collected by cpu=samples|times are CPU virtual times. However, HP-UX 11i versions use real CPU times rather than CPU virtual times unless sampling is used. • Some implementations of cpu=times report clock time.
Profiling with jmap Java 6.0 on HP-UX introduced a new tool, jmap, in the JDK. (Note that Sun Microsystems added jmap in Java 5, but it did not ship with HP-UX until Java 6.) The jmap tool can generate a Java heap dump file for later analysis by HPjmeter. To generate a heap dump, use a command like the following: jmap –dump:format=b,file=heapdump.hprof You can now open the heapdump.hprof file in the HPjmeter console. You can obtain the pid for a Java application using the jps command.
Table 6 Available Metrics or Features from -Xeprof and -agentlib:hprof Data (continued) Feature 80 -Xeprof -agentlib:hprof cpu=samples cpu=times heap=dump heap=sites Reference Graph Tree (page 169) No N/A N/A Yes No Reference Sub-Trees by Size (page 170) No N/A N/A Yes No Class Loaders (page 173) No No No Yes No Residual Objects (Count) (page 175) No N/A N/A Yes Yes Residual Objects (Bytes) (page 175) No N/A N/A Yes Yes Contested Lock Claims by Method (page 178) Yes No
Table 6 Available Metrics or Features from -Xeprof and -agentlib:hprof Data (continued) Feature -Xeprof -agentlib:hprof cpu=samples cpu=times heap=dump heap=sites Exclusive Class Clock Times* (page 168) Yes No No N/A N/A Exclusive Class Lock Delay* (page 179) Yes No No N/A N/A Inline Candidates (page 180) heuristics Yes No Yes N/A N/A Exceptions Thrown (page 180) heuristics Yes No Yes N/A N/A Memory Leaks (page 180) heuristics No N/A N/A Yes No N/A Yes Yes Yes Allocat
Looking at the Data from the Top Down An alternative, top-down approach to profile data analysis is to start from the call graph based on Clock or CPU time and continue expanding the topmost nodes until you find a method or multiple methods that implement high-level operations characteristic for your application (that is, are responsible for transaction processing). By expanding this node you can see how this high-level operation splits its work among the methods it calls.
CPU Versus Clock Time The clock time is the time as measured by an external independent clock, sometimes also called wall clock. Clock time includes the time that passes for any state that a thread may be in. That is, clock time includes the time the thread spends in sleeping, waiting, performing an I/O operation, waiting for garbage collection to complete, being preempted by the operating system, and in processing. CPU time accounts for the time spent by any of the CPUs (processors) on executing a thread.
2. 3. 4. 5. Click the Scope menu name to see the scope options. By default, Process scope is selected. Note that if you mark an item in a profiling viewer and then switch scope, the “find immediate” action is automatically executed. If you want to disable this default action before switching scope, uncheck the Automatic find immediately menu item. Select the scope that you want. The scope change is immediately applied to the data displayed in the visualizers appearing in the open viewer.
Comparing Profiling Data Files It can be useful to compare session data in order to understand the effect of differences in activity at different times of day, week, or month. Comparisons can be especially useful when running performance testing benchmarks. (See Preparing a Benchmark (page 71).) To compare data files, do the following: 1. From the main console, open two data files of the same type that you want to compare.
A viewer will open for each data representation (see Using HPjmeter to Analyze Profiling Data (page 39) for an example profiling viewer). 2. 3. Click on one of the viewers to make it active. Two ways exist to create a combined set of comparable data: • In one viewer, click File→Compare , or • click the Compare button . The Compare window opens, and you can now see the other file to compare your first file against. 4. Click the file name that you want to compare against. 5.
HPjmeter automatically recalculates the call count for each method in the peak data with the result that each value is normalized to the ratio of the call counts for the selected method. HPjmeter then sorts the data. By viewing the information that appears in the newly sorted list, you can compare differences in activity between the two data files. If you want to see how this works in detail, select View→Show Formula to display the calculations that HPjmeter uses.
Table 7 Key to Default Color Representation of Thread States in Histograms Lock Contention. Time wasted trying to acquire a Java lock while it was acquired by another thread. More precisely, it is the summary time measured from the moment the thread requests a Java lock until the lock is actually granted. CPU. Time spent running on any of the computer's processors. This includes Java code, the native code (if any), and the time spent in the virtual machine by this thread.
Interpreting the Histogram Presentation You can assess the scalability of the application from the spectrum of a typical “working” thread from your application. • A large amount of lock contention (red) indicates that scalability will be a problem. You need to change the inter-thread communication mechanism to reduce the lock contention. • A large amount of waiting (green) may indicate that the load on the application can be increased.
Interpreting Call Graph Data In the trees that are based on the call graph, the nodes represent methods and the arcs represent cumulative calls from the preceding node. • For CPU and Clock trees, the numerical values represent the time used by the arc. That is, the numerical values represent the inclusive time used by all invocations represented by the arc. The numbers in parentheses indicate what percentage of the total inclusive time used by the method was caused by the calls from the preceding node.
NOTE: Inclusive times include the time spent in called methods. Therefore, for recursive calls, the same time is accumulated many times, depending on the depth of the recursion. When this happens, a recursive method may show an inclusive time that is greater than the inclusive time shown for its parent method. TIP: To see the list of all callers of a given method, double-click on the method name. The line showing repaint is marked in red.
Searching the Trees The search functions work in trees basically the same way they work for other metrics, but there are a few special considerations for searching call graph trees. • • Occasionally, not all nodes found in a search will be accessible (visible). You'll be notified about such situations in the status bar at the bottom of the window.
the number of bytes that would be reclaimed. It is up to you to check if the presented objects are still needed by the application, and how the critical references should be removed.
6 Analyzing Garbage Collection Data HPjmeter allows you to process garbage collection (GC) data from Java virtual machines. Separating the GC data collection step from the analysis step has the following advantages: • The data analysis can be done at a different time and on a different platform than was used to run the application. For example, it can be done on a desktop system or on a laptop. • A non-interactive agent will often impose less overhead than an interactive one.
Table 9 Supported -Xverbosegc options for Java 1.5.0.04 (continued) filename writes output to the specified file. [n][h][d][u][t] n h u d t prevents appending the pid to the log filename. appends the hostname to the log file name. appends the username to the log file name. appends the date to the log file name. appends the time to the log file name. At every selected garbage collection, the following 20 fields are printed. These values hold true for Java 1.5.0.04. Other versions may differ.
Table 10 Fields Captured in GC Log Data When Using -Xverbosegc print options (continued) ◦ 13: Last ditch collection ◦ 14: GC Locker-Initiated GC ◦ 15: Heap Dump Triggered GC (HP implementation) ◦ 16: Heap Dump Triggered GC ◦ 17: Heap Inspection Initiated GC • When %1 is equal to 3, %2 indicates that a complete background, concurrent mark and sweep GC has occurred.
Table 10 Fields Captured in GC Log Data When Using -Xverbosegc print options (continued) • When %1 is equal to 5, %2 the G1 sub-type and whether there was a G1 initial mark. The form for this data set is given as n.
Table 10 Fields Captured in GC Log Data When Using -Xverbosegc print options (continued) %6 Tenuring threshold - memory allocation that determines how long newborn objects remain in the new generation. %7, Eden sub-space within the new generation, in bytes.These spaces are reported: %8, • Before (Occupied before garbage collection) %9 • After (Occupied after garbage collection) • Capacity (Current capacity ) %10, Survivor sub-space within the new generation, in bytes.
file. Collecting both the Glance data and the GC data at the same time is very useful because you can view all the data together in one HPjmeter console. Data from the opened file is visible in the Glance Data (page 194) visualizer, accessible from the Glance Adviser tab on the GC visualizer. Glance system call data is obtained from the PROC_SYSCALL loop in Glance Plus, and is visible in the Glance System Call Data (page 198) visualizer, accessible from the Glance System Call tab on the GC visualizer.
Figure 17 Appearance of GC Data Collected in Intervals During a Session NOTE: For the signal to be captured by the JVM, you must either be logged in as root, or you must be the user who started the JVM. Related Topics • Using Specialized Garbage Collection Displays (page 181) Data Collection with -Xloggc -Xloggc directs a smaller set of data to a file than what is available with -Xverbosegc.
$ java ... -XX:+PrintGCDetails -Xloggc:filename Use these additional options to obtain more data on GC activity in the eden space, the survivor space, the old generation, and the permanent generation. NOTE: Using -XX:+PrintGCDetails and -XX:+PrintHeapAtGC Together Some JVMs improperly mix the output from these two options, resulting in a data file that cannot be parsed properly by HPjmeter. We recommend using only one of the two options. When using the G1 collector, HPjmeter 4.
See also: • Data Collection with -Xverbosegc (page 94) • Data Collection with -Xloggc (page 100) Locating Summary Information for Saved Data Sets Summary data is displayed on the first tab that you encounter when you open a GC data set for viewing. For -Xverbosegc data, an extensive summary is given for garbage collection findings, as well as for the state of the system.
4. Click the file name that you want to compare against; or, if choosing more than one file, hold the Ctrl key while selecting each desired file. 5. When the files are highlighted, click the Compare button. A new viewer opens that presents the selected data sets in comparison to one another. NOTE: For alternate access when you have multiple viewers open, click the button to bring the console to the top. The console lists all open sessions and data sets.
Table 13 Common Garbage Collection Types Recognized by HPjmeter (continued) collection and the old generation collection occur independently and cannot overlap. Incomplete Concurrent Mark-Sweep Occurs when a background thread is performing a garbage collection in the old area, but is interrupted when the JVM determines that a full GC is needed. Garbage First (G1) The G1 collector is a server-style garbage collector, targeted for multi-processor machines with large memories.
For an explanation of Java memory terminology, see the Sun Developer Network publication Memory Management in the Java HotSpot™ Virtual Machine (http://java.sun.com/j2se/reference/whitepapers/memorymanagement_whitepaper.pdf) Understanding the Summary Presentation of GC Data To view the summary presentation of GC data, double-click on the data representation in the main panel. A window will open to display the data. The following image shows a summary of .
The highest percentage of actual use by the application of the heap at its final capacity configuration. A rule of thumb for optimal utilization would be to have the eden space at 100% with the survivor and old spaces showing some reserve capacity, depending on the application. GC Activity Summary In this example, a comparison of scavenges and full garbage collection of the old and permanent generations, showing number of GC occurrences, duration, and memory usage for each GC type displayed.
Indicates whether or not process swapping is occurring during the measured period. To determine the amount of swap memory occupied when processes are being swapped, click the System Details tab, and locate the swap data in the System and Memory Details section. Comparison of Percentages The color bars compare the percentage of time spent in Full GC or in System.gc calls as a percentage of total time spent doing garbage collection. However, if the total time spent in System.
Figure 19 Summary Panel Showing System Details for a Period of Xverbosegc Data Collection NOTE: The Number of Localities, Heap Pointer Mode, and UseNUMA fields only display if JDK/JRE 6.0.08 or later is being used. As with any field whose value is too long for its column width, if the JVM Arguments value shown at the bottom of the panel is extensive and runs beyond its column, you can mouse over the value to display a yellow pop-up box that shows all the JVM arguments.
Figure 20 System Details Tab with JVM Arguments Pop-up Box Related Topics • Using Specialized Garbage Collection Displays (page 181) • Basic Garbage Collection Concepts (page 103) • Tool Bar Buttons for Manipulating Garbage Collection Data (page 202) • Changing Time Interval in GC Data Visualizers (page 206) Basic Garbage Collection Concepts 109
7 Using the Console The console main window displays when you start HPjmeter. Starting the Console You can run the HPjmeter console on HP-UX, Linux, and Microsoft Windows systems. Starting the Console On HP-UX There are two ways to start the console on HP-UX. • From the command line, type /opt/hpjmeter/bin/hpjmeter The console will be ready to display performance data.
Figure 21 The Main Console Window Data Representation The console main window pane displays session information in a tree format. Figure 22 Main Pane with Several File Types Loaded and an Open Session When monitoring a running application, double-click on a JVM entry to open a session. You can display multiple node agents and sessions in the data tree.
When viewing activity in the JMX server, double-click on the Summary to see the data collection options used for this session, or while monitoring, change management bean attributes and observe resulting changes in the data. To quickly access alert visualizers or the Alert Controller, double-click on the alert entries in the tree. Status messages appear briefly in this area. To view console memory usage, mouse over the bar to see current percentage use of memory allocated to the console.
Table 15 JVM Agent Connection Icons (continued) The JVM agent is not ready to open a session. Another console has already opened a session with it, the JVM agent is running in batch mode, or the JVM agent is incompatible. The JVM agent is unresponsive, that is, the node agent lost contact with it. The JVM might be down, or it might be overloaded. Verify that the JVM is running and wait until it becomes responsive again before opening a monitoring session with it.
Data from a session is saved by using the appropriate options on the command line when you start your application. • Collecting Profile Data (page 72) • Setting Data Collection Preferences (page 46) • Saving Monitoring Metrics Information (page 48) Console Tool Bar Buttons Open File Select a previously saved monitor batch file, a snapshot file, or a profile data file to view. After opening a saved file, you can review data using the menu metrics appropriate for the data file type.
Console Menu Choices The console menu bar contains these choices: • File -- Contains selections corresponding to the buttons in the tool bar, plus Exit the HPjmeter application. • Edit -- Contains these commands: ◦ Look and Feel Preferences Graphical user interface look-and-feel selections ◦ Window Preferences – Remember Last Input File Directory on Exit Click the menu item to toggle whether HPjmeter remembers the directory of the last input file upon exit/restart of the console window.
• • Monitor — Contains these categorical submenus and their associated monitoring metrics: ◦ JVM Summary ◦ Code/CPU ◦ Memory/Heap ◦ Threads/Locks ◦ JVM/System Help — Provides links to the HPjmeter User's Guide, getting started material, and demonstration instructions. The Monitor Menu The Monitor menu provides a summary of JVM information plus four submenus that give access to the lists of specific available metrics.
Figure 23 Console Guide Location and First Screen To enable or disable the console guide, click Edit Window Preferences and toggle to the desired option.
Specifying Metrics to Collect for Monitoring You can open the Session Preferences window at any time by double-clicking on an Open Session, or single-clicking on it then clicking the Session Preferences toolbar button. In pre-deployment or development, it is typical to enable all items. In deployment mode, when you may be concerned about the impact on the application, you can enable a subset of the available items.
To reset the selections to the original settings, click Restore Defaults. Related Topics • Setting Data Collection Preferences (page 46) • Controlling Data Collection and Display (page 46) • Specifying Filters for Monitoring (page 120) • View Monitoring Metrics During Your Open Session (page 39) Enabling Monitoring Alerts If you turn off an alert in the Session Preferences dialog, it will not show in the alert controller window and no data will be generated for it.
Figure 25 Enable or Disable Alerts Window Related Topics • Using the Alert Controller (page 123) to deactivate or reactivate enabled alerts Specifying Filters for Monitoring You can control the amount of information that HPjmeter provides during a live session by filtering out classes using the Filters tab of the Session Preferences dialog box. Filters may be inclusive or exclusive, but not both: • Inclusive filters include only the specified classes in a filter set.
1. 2. In the Session Preferences dialog box, click on the Filters tab. Double-click the filter name in the list of filters on the Filters tab. The Filter Set Editor opens. 3. Specify the list of packages for the filter in the Filter Set field. See Specifying Filters for Monitoring (page 120) for details. Adding Filters To add a new filter: 1. In the Session Preferences dialog box, click on the Filters tab. 2. Click Add Filter Set. The Filter Set Editor opens. 3. 4.
by using the Alert Controller. In addition, you can set thresholds in the Alert Controller to vary the sensitivity of the console to enabled alert conditions, and you can edit e-mail notification attributes submitted when you set session preferences. When active alerts are triggered, a notification appears in the main pane of the console window, and the alert type and threshold value are displayed on all relevant metric visualizers.
See “Responding to Alerts” (page 126) for information on specific alert notifications. The console displays an alert while the alert condition exists, and stops displaying when the alert condition no longer exists. A displayed alert may also stop displaying when HPjmeter recalculates the alert based on real-time information. The console displays a warning icon in the lower right corner of the window showing the current number of alerts detected.
Figure 29 Alert Controller Window You can use the Alert Controller in these ways: Deactivate or reactivate enabled alerts. To deactivate or reactivate alerts once a session has started, use the check box next to the alert name. • When you deactivate an alert, the console stops reporting that alert to the console main pane. Note that a record of the alert activity up to that point has been written into a log file for reference.
Viewing a Log of the Alert History From the Alert Controller, select the Alert Log button to see a history of alerts noted during the monitoring session. Figure 30 Example Alert Log Change the alert log capacity. From the Alert Log window, click the “Select Log Capacity” icon to open the log capacity window. Change the value as desired for number of lines to include in the log.
Table 19 E-mail Notification Fields and Options for Receiving Alerts Fields/Options Expected Attribute Values SMTP E-mail Server: Correctly constructed host name of a To locate the desired mail server valid SMTP server that allows connection without authentication.
Double-clicking on the alert in the main window displays the uncaught exception window. The window shows: • Thread name • Exception • Source location where the exception was thrown Figure 32 Abnormal Thread Termination Alert Visualizer You can use this information to debug the cause of the exception. For example, each row in the Abnormal Thread Termination window represents a thread in the Thread Histogram window.
The Uncaught Exception Statistics display shows you: • Thread name • Type of Exception • Time thrown, as the time from the start of the application • Throw point as a subentry The window shows events in a hierarchical tree. • Select the View→Show Packages menu item to alternatively hide or show the Java package names to shorten the lines in the display.
Table 21 GC Duration Alert Threshold by Java Version Java Version Metric Name Default Alert Threshold HP 1.4.2.02 or later 1.4.x versions GC Duration > 2000 milliseconds (2 seconds) HP 1.5.0.12 or 6.0.01 or later GC Duration (Major) > 30 seconds HP 1.5.0.12 or 6.0.01 or later GC Duration (Minor) > 2000 milliseconds Double-click on the alert to display the GC Duration visualizer.
Mouse over the location entry to see an explanatory tool tip. Figure 33 Java Collection Leak Locations Visualizer After some time, the alert icon may disappear if: • HPjmeter determines that the previously reported leak object is not a leak. • The leak-related collection object is removed by garbage collection. HPjmeter removes the alert icon when it removes all the leak nodes and no new memory leak is detected.
Figure 34 Array Leak Locations Visualizer After some time, the alert icon may disappear if: • HPjmeter determines that the previously reported leak object is not a leak. • The leak-related array is removed by garbage collection. HPjmeter removes the alert icon when it removes all the leak nodes and no new memory leak is detected. NOTE: This alert is not available when dynamically attaching to a running Java application.
2. 3. 4. 5. Once connected to the JVM, open the Alert Controller. Scroll down if needed to see the Process CPU Usage Alert. Check this box also to turn this alert on. Set the usage threshold to reflect the expected consumption and designate a duration, if desired. Click Apply to activate checking for this threshold. When the set threshold is met, an alert notice is posted on the console. Double-click on the console alert to display the Percent CPU Utilization visualizer.
HPjmeter presents CPU utilization percentages as the number of CPUs x 100 to arrive at the total possible percent of use. Therefore, maximum use of 3 CPU would be equal to or greater than 300% of 1 CPU out of a total of 4 available CPU. By applying this value (300%) for a 5–minute duration in the Alert Controller, HPjmeter notifies you each time that value is exceeded for 5 minutes during the monitoring session. A notification is generated for display on the console and recorded in the alert log.
8 Using Visualizer Functions Visualizers behave somewhat differently depending on the type of data displayed. Visualizer Behavior When Monitoring Behavior or Analyzing Data Visualizers that open as a result of selecting metrics from the Monitor menu on the main console appear as individual, standalone windows. This behavior allows you to view a number of individual metrics at the same time.
Figure 36 The Garbage Collection Data Viewer Both standalone visualizers and tabbed viewers have tool bars and other controls for manipulating the data display. See Using Visualizer Tool Bars (page 201). Locating Information About a JVM and its Environment HPjmeter collects information about a running JVM and its environment and displays it in the JVM Summary visualizer. This feature is available by default when monitoring an application or when viewing a saved file from a monitoring session.
Monitor Memory and/or Heap Activity Menu (page 140) • Heap Monitor (page 140) • Garbage Collections (page 141) • GC Duration (page 142) • Percentage of Time Spent in Garbage Collection (page 143) • Unfinalized Objects (page 144) • Allocated Object Statistics by Class (page 145) • Allocating Method Statistics (page 146) • Current Live Heap Objects (page 147) • Process Resident Memory (page 149) • Process Virtual Memory (page 150) • Data Region Resident Memory (page 150) • Data Region V
Figure 37 Monitoring Metric: Java Methods HotSpots with Confidence Interval Graphically Displayed for Each Method Guidelines • If this metric shows a large percentage of CPU time spent in just a few methods listed at the top, it indicates a performance problem or a method performance problem that might be improved. • When the top entry represents a single-digit percentage, a performance problem is unlikely unless the entry describes a method that you did not expect to see.
NOTE: The data collection for Java Method HotSpots may be significantly delayed if one or more of these conditions exist: • The application runs on a single CPU system. • The application does not consume a lot of CPU. • The application consumes CPU exclusively in non-profiled methods. Restarting the application server or node agent will not improve data collection.
The integer is a count of how many times this exception has been thrown in this location, and caught by this method. The percentage gives information on how often this exception is being thrown in relation to all the detected thrown exceptions. NOTE: Collecting the stack trace information could impair the performance of your application if the application throws a large number of exceptions during the session.
Monitor Memory and/or Heap Activity Menu Heap Monitor Displays free, committed, and used memory sizes in the heap and garbage collection events over time. The used heap space includes live objects and dead objects that have not yet been collected as garbage. Specifically, this visualizer shows the heap in use by objects in the eden space and in the old and survivor generations, but does not include the permanent generation.
• When you select a high level of detail, 1 to 20 minutes, and the heap size does not go to the local maximum before a garbage collection happens, it could indicate excessive calls to System.gc(). See Identifying Excessive Calls to System.gc() (page 51). • When you select coarse granularity, 1 to 24 hours, you may notice the overall change of behavior in heap size and garbage collection pattern. This can help with understanding the correlation between the application load and the pressure on the heap.
Guidelines • For a healthy heap, minor collections should dominate major garbage collections. If the number of minor collections is too small compared to the number of major garbage collections, the young generation of the heap may be too small. • If the heap size shown by garbage collections converges towards the heap limit, the application has run out of memory, or soon will run out. • If the old generation is too small, the application will run out of memory.
Details • For HP Java 1.5.0.12 and later or 6.0.01 or later, this visualizer distinguishes between major and minor garbage collections such as full GC and scavenge.
Details Here are two possible ways to make improvements: • Tune the heap parameters for better performance. For HP HotSpot VM, run your application with the -Xverbosegc option and view the results in HPjmeter.
Guidelines • Escalating numbers of unfinalized objects can indicate that the finalizer queue is growing, with associated objects holding increasing space in the heap. Details • Some or many of the objects in a finalizer queue may no longer be needed by the program and are waiting to be finalized and then collected. • Check to see if the finalize () method in your application is being called at appropriate times and frequency.
Details For those classes that are instrumented (visible through the JVM agent verbose flag), every object allocation in every method is instrumented to report allocations. However, sampling is used to minimize overhead, so the metric reports allocation percentages, not total allocation counts. These percentages are not absolute across the entire application, but are computed with respect to allocations in instrumented classes. Sampling minimizes overhead and focuses attention on user code.
Sampling minimizes overhead and focuses attention on user code. To discover allocation statistics about application server classes, use the include and exclude filtering flags in the JVM agent options. The reported data is cumulative over the lifetime of the session, and accuracy will improve as the session length increases.
Figure 44 Monitoring Metric: Current Live Heap Objects Table 22 Data Shown in Current Live Heap Objects Visualizer Column Heading Description Class Name of class to which object belongs % Heap Used Percent of allocated heap used Bytes Cumulative size occupied by the object (in bytes) +/- First Bytes The total change in the amount of bytes held for this class since the first snapshot was taken.
Figure 45 Copying Selected Current Live Heap Objects Data into Buffer To select all data for use in a spreadsheet, click Copy All to Buffer in the tool bar. Figure 46 Copying All Current Live Heap Objects Data into Buffer Click File →Save to capture all data as an ASCII text file that you can save onto your local machine. Related topics • Heap Monitor (page 140) Process Resident Memory This feature displays the size of the resident memory for the Java process.
Related Topics • Heap Monitor (page 140) • Process Virtual Memory (page 150) Process Virtual Memory This feature displays the size of the virtual memory allocated for the Java process. This shows the overall virtual memory usage of the process. You can know how much virtual memory is being used by your Java application over time. This data is obtained from the PROC_MEM_VIRT attribute of Glance Plus.
heap is independent of the Java heap and is used to hold data for native code. This data is obtained from the PROC_REGION_RES_DATA attribute of Glance Plus. Related Topics • Heap Monitor (page 140) • Data Region Virtual Memory (page 151) Data Region Virtual Memory This feature displays the size of the total virtual memory occupied by data regions of the Java process. This metric helps to provide a measure of the size of the C heap for the process.
Monitor Threads and/or Locks Menu Thread Histogram Displays thread states over time. Thread data arrives in time slices. For each time slice, color-coded bars represent the percentage of time the thread spent in each state. The reported states are: Waiting The thread has been suspended using Object.wait() method. Lock Contention The thread is delayed while attempting to enter a Java monitor that is already acquired by another thread. Running All remaining cases.
Related Topics • Identifying Abnormal Thread Termination (page 54) • Identifying Multiple Short-lived Threads (page 55) • Identifying Excessive Lock Contention (page 55) • Change Color Selection for Histogram Display (page 210) Lock Contention Provides lock contention statistics. The percentages for each method represent how much of the total lock contention observed occurred in that method.
Considerations When Comparing Lock Contention and Thread Histogram Metrics: Lock contention data is sampled less frequently than thread histogram data. When there is lock contention with a short lifespan, a small amount of lock contention might appear in the Thread Histogram, but not be shown in the Lock Contention percentages. This can happen when the contention occurs during a time when the Thread Histogram sample is being taken, but the lock contention sample is not.
Related Topic • Network Output Stream (page 155) Network Input Stream This feature produces a graph that displays the rate of incoming data being provided from the network to the monitored application. This visualizer shows the number of bytes being received by sockets in the java.net package, per second.
Related Topic • Outgoing Socket Connections Frequency (page 154) Monitor JVM and/or System Activity Menu Method Compilation Count Displays a list of all the methods compiled from the time the session was opened, showing the number of times a particular method was compiled. The metric window reports “No methods compiled since the session opened.” until the next method compilation occurs, and then the metric data appears. Guidelines • The normal values for this metric are single-digit numbers.
Figure 49 Monitoring Metric: Method Compilation Count Related Topics • Identifying Excessive Method Compilation (page 56) • Method Compilation Frequency (page 157) • Find a Search Pattern (page 204) Method Compilation Frequency Produces a graph that shows the compilation frequency. This is a companion to the existing Method Compilation Count. The Method Compilation Frequency metric provides a view of how much effort the JVM is spending on method compilation.
Related Topics • Identifying Excessive Method Compilation (page 56) • Method Compilation Count (page 156) Loaded Classes HPjmeter displays the number of classes loaded into memory for processing. This number usually stabilizes over time as processing progresses. Guidelines • The number of classes loaded at any one time tends to oscillate within a narrow range; typically less than 2 percent of all loaded classes will be unloaded or reloaded during application processing.
performing well. At higher consumption rates, CPU consumption can become a bottleneck to good performance. • When system CPU consumption is significantly higher than process consumption, this may indicate that “alien” or undesired processes are using CPU resources that the preferred application could be using. It may also indicate that the application is abusing the operating system kernel.
Related Topics • Thread Histogram (page 152) • Lock Contention (page 153) Viewing Profiling or GC Data in HPjmeter To view metrics for saved data: • From the main console, click File→Open to open a file browser window and locate your saved file. You can also drag and drop the file onto the console. Selecting the file automatically opens a data viewer. In addition, a data representation for your file appears in the console main pane.
On the Summary tab, as with any field whose value is too long for its column width, if the JVM Arguments value shown at the bottom of the upper panel is extensive and runs beyond its column, you can mouse over the value to display a yellow pop-up box that shows all the JVM arguments. When you mouse away from it, the yellow box disappears.
The profile displays are categorized as follows under the Metrics menu: • Profile Code and/or CPU Activity (page 163) • Profile Memory and/or Heap Activity (page 168) • Profile by Threads (page 175) (Threads/Locks submenu) • Profile by Locks (page 177) (Threads/Locks submenu) These heuristic displays appear under the Estimate menu: • Inline Candidates (page 180) • Exceptions Thrown (page 180) • Memory Leaks (page 180) See also: 162 • Profiling Applications (page 69) • Using Visualizer Too
Menu Choices The menu bar can contain one or more of these choices, depending on the data type and visualizer you are viewing: • File — Contains selections for printing, saving, and comparing data, closing the window, or opening another window that is viewing the same data file. Opening another window is useful when you want to view more than one metric at the same time.
one is located. The gray horizontal bars provide a graphical representation of the number of times a method was called. Double-click the method name to see a list of its callers with the cumulative number of calls from each caller.
Call Graph Tree with Call Count The call graph with the method call count. Related Topics • Using Call Graph Trees (page 89) • Mark an Item for Search (page 203) • Find a Search Pattern (page 204) Call Graph Tree with CPU The inclusive CPU time shown on the call graph.
Average Exclusive Method CPU Times* Exclusive CPU Method Time divided by the number of calls to the method. This is the average CPU cost for the body of the method. Related Topics • Inclusive Versus Exclusive Time (page 82) • Time Units (page 82) • Mark an Item for Search (page 203) • Find a Search Pattern (page 204) Average Exclusive Method Clock Times* Exclusive Clock Method Time divided by the number of calls to the method. This is the average time spent within the method.
These are possible reasons for reported starvation: • the real starvation caused by the operating system scheduler - this should be distributed over all methods in proportion to the exclusive CPU time consumed by each method • waiting for the class loader to load a class • waiting on an internal VM lock (mutex) • time consuming system calls • explicit or implicit Garbage Collection invocation (frequent or massive memory allocation by the method) • excessive virtual memory paging • suspending th
• Mark an Item for Search (page 203) • Find a Search Pattern (page 204) Exclusive Class Clock Times* The total of the Exclusive Method Times (Clock), summed over all methods of the class. To see the list of all contributing methods, double click on the class name. The standard methods java.lang.Thread.sleep and java.lang.Object.wait do not contribute to this metric, however, they will be shown on the list of all contributing methods for the respective classes.
Related Topics • Allocation Site Statistics (page 185) • Mark an Item for Search (page 203) • Find a Search Pattern (page 204) Live Objects (Count) The number of live objects presented by object types. Live objects are those that could be accessed through a chain of references from the set of program variables when the heap dump took place.
Displaying large reference graphs puts a lot of stress on the underlying Swing components, so please be patient when dealing with huge heap dumps. Related Topics • Mark an Item for Search (page 203) • Find a Search Pattern (page 204) • Class Loaders (page 173) Reference Sub-Trees by Size This visualizer shows the cluster of objects that are directly or indirectly referenced from the root of the tree, along with the cumulative size of all the references.
Object and Primitive Value Tables For binary heap dump files, the tables on the right side of this visualizer display object and primitive values for any selected node object in the tree: • The Attributes table shows the names and values of the non-static fields in the node object. • The Statics table shows the names and values of the static fields in the node object. Notice that the table content changes as you choose different tree nodes.
String and char arrays For string and character arrays, the Value column has been enhanced to show the string or char value, not the object id, in the table cell. In Figure 55 (page 172), for example, the Attributes table for the selected object node shows a String type with a value beginning with ‘There either is or is not,…’, which is too long to fit in the table. If a string is too long, you can mouse over it to display a tooltip pop-up box containing the entire string.
Figure 56 Character array Related Topics • Mark an Item for Search (page 203) • Find a Search Pattern (page 204) Class Loaders This visualizer displays the name and count of class loaders found operating in the application.
Figure 57 Class Loaders Visualizer You can double click one of the listed class loaders to get a pop-up window displaying the class types that have been loaded by that class loader.
Guidelines • Class types reside in the Permanent Generation space, which is usually more limited in size than other areas of memory. Continuing incorporation of additional instances of a class loader can begin to fill this area, creating an out-of-memory error in the Perm Gen region of memory.
Figure 59 Threads Histogram with Pie Chart Thread state indicator The colored areas of each line or line segment show relative proportions of time that a thread has spent in a particular state. Double-click anywhere on the line to open the pie chart and see actual percentages for the states of the selected thread. Color change In this image, the default color for starvation has been changed from white to purple using the color chooser.
Figure 60 Threads Groups Histogram Slider In this metric visualizer, the slider lets you extend the view to encompass either the thread or thread group view or a portion of both views. Related Topics • Reading Profiling Histograms (page 87) • Change Color Selection for Histogram Display (page 210) Profile by Locks The lock delay is a measure for lock contention.
This section contains some metrics derived from other metrics. The derived metrics are marked by an asterisk (*). They are usually less frequently used, as they not always are immediately useful in detecting performance bottlenecks. For each metric that presents the average values per call, only methods called at least twice are taken into account. The following metrics are available.
Lock Contention Ratio by Method* Contested Lock Claims divided by All Lock Claims for each method. This gives the average frequency with which the method encountered a taken lock. A method must have claimed a lock at least five times to be included in this metric. Related Topics • Mark an Item for Search (page 203) • Find a Search Pattern (page 204) Average Exclusive Method Lock Delay* Method Exclusive Lock Delay divided by the number of calls to the method.
many synchronized methods, the individual methods may not exhibit excessive lock contention, but the problem might get visible using this metric. Related Topics • Inclusive Versus Exclusive Time (page 82) • Mark an Item for Search (page 203) • Find a Search Pattern (page 204) Using Heuristic Metrics from the Estimate Menu HPjmeter provides three estimated measures to aid in narrowing the field of your problem-solving or performance tuning inquiry.
Using Specialized Garbage Collection Displays GC data displays are available immediately on opening a .vgc file that was generated by running your application with -Xverbosegc or -Xloggc options.
Figure 61 GC Metric: Heap Usage After GC Related Topics • Using Visualizer Tool Bars (page 201) • Changing Time Interval in GC Data Visualizers (page 206) • Basic Garbage Collection Concepts (page 103) • Current Live Heap Objects (page 147) (available in monitoring mode) • Heap Usage Alert (page 129) (available in monitoring mode) Duration (Stop the World) This visualizer shows the amount of stop-the-world time spent for each garbage collection event.
Figure 62 GC Metric: Duration Related Topics • Using Visualizer Tool Bars (page 201) • Changing Time Interval in GC Data Visualizers (page 206) • Basic Garbage Collection Concepts (page 103) • Identifying Excessive Calls to System.gc() (page 51) (available in monitoring mode) Cumulative Allocation This visualizer shows the cumulative amount of storage for new objects requested by the application over time, which can indicate the overall progress of the application.
Figure 63 GC Metric: Cumulative Allocation Related Topics • Using Visualizer Tool Bars (page 201) • Changing Time Interval in GC Data Visualizers (page 206) • Basic Garbage Collection Concepts (page 103) • Creation Rate (page 184) Creation Rate This panel shows the incremental object creation rate for the application as observed at each garbage collection event. The increment at each data point is the amount of elapsed time since the previous GC event.
Figure 64 GC Metric: Creation Rate Related Topics • Using Visualizer Tool Bars (page 201) • Changing Time Interval in GC Data Visualizers (page 206) • Basic Garbage Collection Concepts (page 103) Allocation Site Statistics This visualizer displays the sites (methods) in the JDK and user's Java code where objects have been created.
possible, the most active allocation sites (the sites most likely to cause GC performance problems) are reported. The allocation data can be displayed as either combined or separated time sets. Use the toolbar button combined (the default) and separated to toggle between them. When the button is toggled to combined, the clock button is disabled, and the background color is white.
Figure 66 GC Metric: Separated Sets of Allocation Site Statistics. The filter button opens a pop-up dialog box where you can provide various filters to narrow down the data. A Filter entry is also listed under the Edit menu. If the sets are combined, the columns and filter dialog box show as combined. With the filter dialog box still displayed, pressing the Toggle Allocation Data into Combined or Separated Sets button would then show the columns and filter dialog box as separated.
Figure 67 GC Metric: Allocation Site Statistics - Combined Sets, Filter Dialog Box 188 Using Visualizer Functions
Figure 68 GC Metric: Allocation Site Statistics - Separated Sets, Filter Dialog Box Filter Field Descriptions: • Time – Only visible for separated sets. Select the time set from the dropdown list, which contains a sorted list of all time sets for the GC file. If there is more than one time set, the value All is also listed, which if chosen will display all time sets for this GC file in the table.
How a Snapshot of Allocation Sites Statistics is Shown in GC Visualizers Figure 69 Presentation of Allocation Sites Statistics and Xverbosegc Data Collected During the Same Time Period 190 Using Visualizer Functions
Figure 70 Presentation of Allocation Sites Statistics and Zero Preparation GC Data Collection During the Same Time Period Related Topics • Collecting Allocation Site Statistics for Viewing in HPjmeter (page 98) • Collecting GC Data with Zero Preparation (page 99) • Using Visualizer Tool Bars (page 201) • Basic Garbage Collection Concepts (page 103) • Identifying the Site of Excessive Object Allocation (page 54) User-Defined X-Y Axes This visualizer is supplied to help you to experiment with your
Figure 71 User-Defined Comparisons of GC Metrics Related Topics • Using Visualizer Tool Bars (page 201) • Changing Time Interval in GC Data Visualizers (page 206) • Basic Garbage Collection Concepts (page 103) Multiple User-Defined This visualizer is supplied to help you experiment with your own combinations of data gathered by -Xverbosegc or -Xloggc, including Glance Data (page 194) that you might have collected. The Heap Usage After GC metric is shown by default.
Figure 72 Multiple User-Defined Metrics Click the toolbar button to open a metric configuration pop-up box that lets you select the metrics you want to display: Figure 73 Multiple User-Defined Metric Configuration Box Choose the Metric Selection Mode by clicking the Single or Multiple metric radio button. Then click the check boxes to select the desired metric or combination of metrics. When you check a metric, it is immediately added to the chart. Uncheck it, and it is removed from the chart.
When displaying multiple metrics, HPjmeter tries to share the default y-axis if possible. If not, it creates additional y-axes to the right of the graph, in units appropriate to the selected metrics. For example, Figure 72 (page 193) shows four metrics, but only three y-axes. The units for the Heap Usage After GC and Old After metrics are MBytes, so they share the same default MBytes y-axis to the left of the graph.
Figure 74 GC Metric: Glance Data You can display a different Glance metric or a combination of multiple metrics using the toolbar button.
Figure 75 Glance Data Metric Configuration Box Click the Single or Multiple metric radio button to choose the Metric Selection Mode. Then click the check boxes to select the desired metric or combination of metrics. When you check a metric, it is immediately added to the chart. Uncheck it, and it is removed from the chart. You can also view these metrics together with GC metrics and Glance system call metrics on the Multiple User-Defined tab.
Table 23 Metrics Shown in Glance Adviser Visualizer (continued) Glance Adviser Metric Name HP GlancePlus Dictionary Metric Name Run Queue Average GBL_RUN_QUEUE Average number of threads waiting in the run queue over the interval. Global Private Virtual Memory GBL_MEM_VIRT Total private virtual memory (in MBs unless otherwise specified) at the end of the interval. This is the sum of the virtual allocation of private data and stack regions for all processes.
Table 23 Metrics Shown in Glance Adviser Visualizer (continued) Glance Adviser Metric Name HP GlancePlus Dictionary Metric Name Data Region Virtual Memory PROC_REGION_VIRT_DATA Size (in KBs unless otherwise indicated) of the total virtual memory occupied by data regions of this process. This value is not affected by the reference count since all data regions are private.
Figure 76 GC Metric: Glance System Call Data You can display a different Glance system call metric or a combination of multiple metrics using the toolbar button.
Figure 77 Glance System Call Metric Configuration Box Click the Single or Multiple metric radio button to choose the Metric Selection Mode. Then click the check boxes to select the desired metric or combination of metrics. When you check a metric, it is immediately added to the chart. Uncheck it, and it is removed from the chart. You can also view these metrics together with GC metrics and Glance Adviser metrics on the Multiple User-Defined tab.
Related Topics • Collecting Glance Data for Viewing in HPjmeter (page 98) • Glance Data (page 194) • Using Visualizer Tool Bars (page 201) • Changing Time Interval in GC Data Visualizers (page 206) • Basic Garbage Collection Concepts (page 103) Using Visualizer Tool Bars All visualizers offer buttons and other controls that help you manipulate the information displayed. See the following sections for details.
Refresh Live Objects Collect data and immediately display live objects in the heap at that moment. Copy Selection to Buffer Copy a selected portion of the current data display into a buffer. Use standard keyboard shortcuts to paste this temporarily stored data into a spreadsheet or other similar document. Copy All to Buffer Instantly copy all of the current data display into a buffer. Use standard keyboard shortcuts to paste this temporarily stored data into a spreadsheet or other similar document.
between a display of elapsed time in seconds and clock time. Select Metrics for Display For the Multiple User-Defined, Glance Adviser, and Glance System Call tabs, opens a pop-up window that lets you configure the metrics you want to display by using the check boxes to select the desired metric or combination of metrics. Special Button or Other Gadget Functions This section gives you more detailed information about the functions of some visualizer buttons to help you make better use of these features.
Use this list to locate marked objects across sessions or data files; anywhere within the data sets that are currently viewable in the console. To use this feature, do the following: 1. Open the Marked drop-down and view objects marked thus far in HPjmeter visualizers. 2. Click the object that you want to locate. This makes the selected object in the drop-down become the current marked object. 3. 4. Now open the visualizer where you want to search for the object.
You can also add additional items to the highlighted set by holding down the SHIFT key while clicking the left mouse button. Each new item added to the selected set is also highlighted in light yellow. Figure 78 Locating an Item Editing the Finder For each search field , you can either type in a different value, or you can click Clear to eliminate information from the search.
Figure 79 Using the Pause Clock Changing Time Interval in GC Data Visualizers The GC viewer provides several ways for you to manipulate the breadth of the data in the specialized GC visualizer displays. You can • select a subset of the available data. • continue selecting subsets to zoom in to specific events in ever smaller periods of time and/or ever narrowing range of data.
To drill in further, repeat this method of defining the area of view until you reach your desired resolution. Use the slider underneath the graph to view additional ranges in the data without changing the resolution interval. The following images show a selected area on a graph and the resulting display.
Figure 82 New Resolution After Selection Apply Selected Interval Across All Metrics When you have your desired resolution in one metric visualizer, click the button to apply the selected time interval across all metrics in the GC viewer. This resolution will apply until the reset button is clicked or a new resolution is chosen.
Figure 83 Resolution After Interval Applied to All Metrics Reset Interval to Default Settings Across All Metrics Click the button to reset all metrics to their original graphic display seen on first opening the GC viewer. Changing Time Interval in Monitoring Visualizers To change the time interval currently displayed, click the drop down in the top right of a graphical monitoring visualizer, and select the time interval that you want to see.
Figure 84 Time Interval Use in a Monitoring Visualizer Change Color Selection for Histogram Display The console provides you with the ability to change colors on thread histogram displays in order to better support variations in color vision capability and to help you selectively improve contrast for items you are particularly interested in. To access the color changer on these displays, double-click the color rectangle (in the legend) that you would like to change. A color chooser will open.
Figure 85 Using the Color Changer on Histogram Displays Related Topics • Thread Histogram (page 152) • Profile by Threads (page 175) Change Color Selection for Histogram Display 211
9 Understanding How HPjmeter Works Here are details on how HPjmeter works to collect and display metrics. Performance Overhead on Running Applications HPjmeter is designed to minimize data collection overhead on deployed applications. The most significant change you will notice is slightly longer application server startup times. Application Server Startup Time HPjmeter uses load-time bytecode instrumentation to reduce overhead in deployment situations.
agent spends almost all its time blocked on socket and first-in first-out (fifo) waits. When the node agent is managing open sessions, overhead is extremely low. The physical memory footprint of the node agent is about 1 MB when idle and 1.5 MB when active. Console Overhead You should not run the console on a machine that is a managed node to minimize impact on your system. The console uses 512 MB of memory on Microsoft Windows and 1 GB of memory on UNIX systems by default.
Until the algorithm can show the data with confidence, the graphical confidence interval is displayed as a gray bar showing only relative relationship. How Memory Leak Detection Works HPjmeter constantly monitors all open sessions and analyzes the heap size after each garbage collection. It tries to detect a longtime, increasing trend in the heap sizes.
Figure 87 Early Indicator of Possible Memory Leak Using Heap Monitor Display HPjmeter uses the long term linear upward trends in the heap size after garbage collections to calculate the presence of memory leaks. Short-term increases in heap size are normal. HPjmeter measures this rate in MB/hour. HPjmeter does not report a leak in situations in which the heap size increase is only temporary, or in situations in which the slope of the fitted line is too small to raise an alarm.
HPjmeter also uses data collected from the JVM to enrich the quality and kind of information presented in HPjmeter monitoring metrics and to expand the number of alerting notifications HPjmeter is able to provide via the Alert Controller.
10 Troubleshooting Here is information to help you find answers and possible solutions to problems you might see. Documentation and Support You can find printable PDF versions of the user's guide and the release notes in the installation directory within the doc subdirectory. Updates to documentation are made available at the HP Business Support Center at http://www.hp.com/go/hpux-java-docs.
To fix this problem, you can make either of these changes: ◦ Reinstall a JRE in the expected location specified in the HPjmeter scripts. ◦ Edit the HPjmeter scripts to specify the path to an existing JRE. The scripts containing the path to the JRE are in the location defined by JM_JAVA_HOME in the bin/ directory. These are the scripts: – hpjmeter - Runs the HPjmeter GUI console. – run_simple_jvmagent - Runs a simple Java program together with the HPjmeter JVM agent.
You can set up the node agent to run as a daemon or service for automatic, continuous operation, or as a normal process when you do not need continuous operation. The setup process depends on your platform. ◦ Managing Node Agents On HP-UX (page 47) • The console graphical interface uses the HP One Voice look and feel for presentation. When you run the console remotely, performance with this look and feel can be sluggish. Use the standard Java Metal look and feel when displaying the console remotely.
JVM agent • If the console runs out of memory during an open session, the JVM agent will close its session a few minutes later. Then the JVM agent prints out this to standard error on its controlling terminal: HPjmeter JVM agent warning: Session communication channel failure. Forcing session closure. When you start a new console, you can reopen a session with the JVM agent. • You may see an OutOfMemoryError when your application is running with the JVM agent.
Node agent On HP-UX systems, the node agent may occasionally leave files in /var/opt/hpjmeter/fifos, with names ending with *fifo. These files are used for node agent communication, and you should not remove them while a node agent or JVM agent is active. You can delete these files when the node agent is not running if they begin to accumulate. • On HP-UX you may sometimes not be able to restart the node agent with the same port.
A Quick References Connecting to the HPjmeter Node Agent For information on how to connect to the node agent, see Connect to the Node Agent from the Console (page 34). Table 24 Fields Used When Creating a Connection to the HPjmeter Node Agent. Field Expected Input Purpose Host Name Fully qualified domain name Required. Designates the location of the server running the HPjmeter node agent or the location of the designated SSH host.
Table 25 HPjmeter Features Available by JVM Version (continued) JVM HPjmeter Feature Command Line Option or Action Java 1.4x Java 5.x Java 6.x Java 7.x View results of running print GC details and print heap at GC together — — X -XX:+PrintGCDetails and -XX:+PrintHeapAtGC Start and stop verbose GC data collection with zero preparation — X (5.0.14) X (6.0.02) kill -PROF pid or kill -21 pid Obtain allocation site statistics and -Xverbosegc data together — X (5.0.14) X (6.0.
Glossary active mode An HPjmeter JVM agent that is part of an open console session and is collecting data from the JVM and application and is sending it to a console. agent A module that represents HPjmeter in your environment. HPjmeter provides two types of agents, a node agent and a JVM agent. Batch mode An HPjmeter JVM agent that is installed, configured, and is collecting data, but is not connected to a console and is not part of a session.
hot spot A section of code, typically a method, with a higher-than-normal level of execution or access requiring high CPU usage. inclusive time The accumulated total time used by all invocations of the method and all methods that were called from it directly or indirectly. inlining Replacing a call to a function with an instance of the function; usually done at compile time. Java Management Extensions See JMX . JMX™ Java Management Extensions.
SSL Secure Sockets Layer stop-the-world time Duration of a full garbage collection where processing stops during the collection time. thrashing Excessive paging due to a disparity between physical memory actually available to the application and the specified maximum size for memory usage. thread spectrum When profiling, the distribution of thread lifetime over the possible thread states. time slice Time slice entries represent the life span of the current monitoring session for an application.
Index Symbols -agentlib effect on application performance, 27, 77 JVM agent options, 27 locating Java version, 27 specifying hprof agent for Java 1.5, 77 specifying JVM agent option for Java 1.
agent running, 112 Mark item for search, 201 open alert controller, 114, 202 open session preferences, 114 Print, 201 Refresh live objects, 202 Save to file, 201 Save to snapshot file, 114 session active, 113 session unavailable, 113 Time slice life span, 113 Apply selected interval to all metrics, 202 Close file or connection, 114 Display allocation site data in combined sets, 202 Display allocation site data in separated sets, 202 Filter allocation site statistics data, 202 Generate extended profiling dat
appserver_port, 27 arc call graph tree, 90 array leak location, 130 live sizes, 169 automatic find immediately, 84 B batch JVM agent options, 29, 48 saving session data, 48 BEA lock contention, 55 benchmark creating for performance testing, 71 guidelines, 71 bytecode instrumentation filtering, 28 bytes total allocated, 106 C call count, 89 methods, 163 call graph, 70, 82 Tree menu, 163 call graph tree call count, 89 clock time, 89 CPU time, 89 cumulative calls, 90 interpreting data, 90 lock delay, 89, 178
connect to node agent through firewall, 35 via SSH, 35 connection failed node agent not running, 47 console -log option, 219 bring to front, 201 command-line options, 219 filters, 46 guide, 116 launching from HP Systems Insight Manager, 110 main data pane, 111 main window, 110 memory use, 117 OutOfMemoryError, 218 performance, 218 performance overhead, 213 reports incorrect application name, 218 requirements, 24 session preferences, 117 starting, 34, 110 starting from command line on HP-UX, 110 starting on
filters, 138, 139 metrics, 138 stack traces, 139 thrown, 180 uncaught, 54 uncaught statistics, 127 excessive compilation set threshold for alert, 124 exclude=, 28 exclusive method clock time, 81 exclusive method CPU time, 81 exclusive time, 82 F features, 21 and JVM version, 222 file -Xeprof options, 73 compressed, 49 default location for allocation sites statistics data, 223 default location for Xverbosegc data, 223 directories, 25 open, 114 File menu, 115, 163 file name suffix .eprof, 78 .hpjmeter, 49 .
behavior, 134 tabs, 134 garbage first incomplete, 104 GC data -Xloggc, 100 -Xverbosegc, 94 -Xverbosegc options, 94 collecting, 94 collecting with zero preparation, 99 GC event cumulative duration, 60 duration, 100 GC events total count, 60 get started console guide, 34 monitoring application, 34 Glance data, 194 collecting, 98 using javaGlanceAdviser.
standard management (JMX), 215 JBoss excluded packages, 28 JM_JAVA_HOME installation, 26, 217 jmap tool, 79 JMX server filter list of, 64 JMX server view, 63 JMX summary view, 58 JMX viewer, 215 change MBean attributes, 64 launching, 57 see thread list, 62 set deadlock detection, 62 set memory thresholds, 61 set notifications, 63 track live threads, 61 using, 57 view runtime data, 63 jps command, 79 JVM Summary, 116, 135 version and HPjmeter features, 222 JVM agent -logging option, 220 active mode, 212 comm
method inclusive with clock time, 179 locks uncontested, 177 log alerts, 125 changing capacity (number of lines), 125 viewing alert history, 125 logging, 29, 219, 220, 221 look and feel menu, 115 M main window close, 114 console, 110 console guide, 116 console main data pane, 111 delete, 114 edit, 114 open, 114 tool bar, 114 mark and sweep GC, 95 marked object history accessing, 203 MaxPermSize HotSpot JVM, 220 MBean attributes, 65 Attributes tab in JMX viewer, 64 change attributes in JMX viewer, 64 enabli
Allocation Site Statistics, 185 analyzing profile data, 81 Creation Rate, 184 Cumulative Allocation, 183 Current Live Heap Objects, 147 derived, 163, 177 disabled, 116, 163 Duration (Stop the World), 182 enable, 116, 163 garbage collection, 51, 53, 141 garbage collection duration, 142 Glance data, 194 Glance system call data, 198 heap monitor, 53, 140 Heap Usage After GC, 181 how sorted, 81 Java Method HotSpots, 136 load data for viewing, 160 loaded classes, 57, 158 lock contention, 153 menus, 116 method co
storage of in permanent generation, 98 unfinalized, 169 object allocation by class excessive, 54 metrics, 145 object allocation by method excessive, 54 metrics, 146 object reference call graph, 89 Object Reference Graph pruning, 91 old full GC, 104 old generation, 95, 103, 105 print data, 94 spaces, 98 old too full GC, 104 operating system version locating, 58 optimization method compilation, 56 options alert e-mail notification, 126 JVM agents, 27 node agents, 47 options used locating, 63 Oracle9i excluded
scope, 83 tabs, 134 program termination, 57 Prune, 91 public, 31 R recursive calls indicator, 91 recursive methods and time, 82 time, 91 reference graph call graph tree, 89 sub-trees by size, 170 tree of live objects, 169 Remote operation window configuration, 75 requirements agent, 24 console, 24 patch, 24, 25 system, 24 restrictions access, 48 run_simple_jvmagent script, 42, 43, 45, 218 runtime JMX tab, 63 statistics, 63 S sample application memory leak, 43 running, 42 thread deadlock, 44 sampling consi
sub-space in new generation, 98 survivor space, 103, 105 swap space locating statistics, 63 system CPU usage (%), 132 system call data, 198 System.
U U.S.
