§1. Introduction to JDWP

The Java Debug Wire Protocol (JDWP) enables remote debugging of Java applications by allowing debuggers to connect to a running JVM and inspect its state. However, JDWP brings subtle implications for memory management, particularly around class lifecycle events like ClassPrepare, which can have unexpected consequences for garbage collection and class unloading.

To understand how this works, it’s helpful to know the key components:

  • JDWP Agent: A native library running inside the debugged JVM that communicates with the debugger over a socket. It hooks into JVMTI (Java Virtual Machine Tool Interface) to receive VM events.
  • JVMTI (Java Virtual Machine Tool Interface): The low-level interface that allows agents to monitor and interact with various JVM events, like class loading and method entry.
  • JDI (Java Debug Interface): The client-side Java API that debuggers use to communicate with the JDWP agent and interact with the debugged VM.

The architecture is thus: JVM → JVMTI → JDWP Agent (native) ↔ JDWP Protocol ↔ JDI Client (debugger). When events occur in the JVM, they flow from JVMTI through the agent to the debugger client.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
┌─────────────────────────────────────────────────────────────────┐
│                         Debugged JVM                            │
│  ┌────────────────────────┐      ┌───────────────────────────┐  │
│  │    Java Application    │      │    JDWP Agent (Native)    │  │
│  │                        │      │                           │  │
│  │ [Class Loaded] ───────►│─────►│ 1. JVMTI ClassPrepare     │  │
│  └────────────────────────┘      │    (Synchronous)          │  │
│              ▲                   │    - Creates GlobalRef    │  │
│              │                   │    - Enqueues Command     │  │
│              │                   └─────────────┬─────────────┘  │
│              │                                 │ (Queue)        │
│              │                   ┌─────────────▼─────────────┐  │
│              └───────────────────│ 2. Agent Thread           │  │
│         (GC Prevented)           │    (Asynchronous)         │  │
│                                  │    - Dequeues Command     │  │
│                                  │    - Sends Packet         │  │
│                                  │    - Deletes GlobalRef    │  │
│                                  └─────────────┬─────────────┘  │
└────────────────────────────────────────────────┼────────────────┘
                                                 │
                                           JDWP Protocol
                                           (Socket I/O)
                                                 │
┌────────────────────────────────────────────────▼────────────────┐
│                    Debugger Process                             │
│                                                                 │
│   [ JDI Client ] <──────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘

§2. The Class Preparing Event

When a class reaches the prepared phase of initialization, JVMTI fires a ClassPrepare event. The JDWP agent listens for this event so it can notify the attached debugger.

§2.1 Callback Registration

The JDWP agent registers its ClassPrepare callback during initialization, in classTrack_initialize (src/jdk.jdwp.agent/share/native/libjdwp/classTrack.c):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
/*
 * Called once to initialize class-tracking.
 */
void
classTrack_initialize(JNIEnv *env)
{
    ...
    if (!setupEvents()) {
      ...
    }
}

static jboolean
setupEvents()
{
    ...
    // Enable CLASS_PREPARE events
    error = JVMTI_FUNC_PTR(trackingEnv, SetEventNotificationMode)(trackingEnv, JVMTI_ENABLE, JVMTI_EVENT_CLASS_PREPARE, NULL);

}

This tells JVMTI to invoke the registered callback whenever a class is prepared.

§2.2 Callback Firing

On the HotSpot side, JvmtiExport::post_class_prepare (src/hotspot/share/prims/jvmtiExport.cpp) iterates over active JVMTI environments and invokes their registered ClassPrepare callbacks:

1
2
3
4
5
6
7
8
void JvmtiExport::post_class_prepare(JavaThread *thread, Klass* klass) {
  ...
    jvmtiEventClassPrepare callback = env->callbacks()->ClassPrepare;
    if (callback != nullptr) {
      (*callback)(env->jvmti_external(), jem.jni_env(), jem.jni_thread(), jem.jni_class());
    }
  ...
}

§2.3 Callback Implementation

The JDWP agent’s callback is cbClassPrepare in src/jdk.jdwp.agent/share/native/libjdwp/eventHandler.c:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
/* Event callback for JVMTI_EVENT_CLASS_PREPARE */
static void JNICALL
cbClassPrepare(jvmtiEnv *jvmti_env, JNIEnv *env,
                        jthread thread, jclass klass)
{
    EventInfo info;

    LOG_CB(("cbClassPrepare: thread=%p", thread));

    BEGIN_CALLBACK() {
        (void)memset(&info,0,sizeof(info));
        info.ei         = EI_CLASS_PREPARE;
        info.thread     = thread;
        info.clazz      = klass;
        event_callback(env, &info);
    } END_CALLBACK();

    LOG_MISC(("END cbClassPrepare"));
}

event_callback() calls into eventHelper_recordEvent, which converts the local jclass reference into a JNI global reference via saveGlobalRef (NewGlobalRef) so the class remains reachable beyond this callback. The event command is then enqueued for asynchronous delivery.

The agent thread runs commandLoop (src/jdk.jdwp.agent/share/native/libjdwp/eventHelper.c), which dequeues and processes these commands:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
/*
 * The event helper thread. Dequeues commands and processes them.
 */
static void JNICALL
commandLoop(jvmtiEnv* jvmti_env, JNIEnv* jni_env, void* arg)
{
    LOG_MISC(("Begin command loop thread"));

    while (JNI_TRUE) {
        HelperCommand *command = dequeueCommand();
        if (command != NULL) {
            ...
            if (!gdata->vmDead) {
                log_debugee_location("commandLoop(): command being handled", NULL, NULL, 0);
                handleCommand(jni_env, command);
            }
        }
    }
}

When processing a command, handleCommand calls handleEventCommandSingle, which writes the event to a JDWP packet and then calls tossGlobalRef (DeleteGlobalRef) to release the reference.

§3. The Problem: Global Reference Management

As described above, processing a ClassPrepare event involves two steps on two different threads: the application thread creates a JNI global reference to the class (so it remains live across the asynchronous handoff), and the agent thread deletes that reference after sending the event to the debugger.

The cleanup—tossGlobalRef (DeleteGlobalRef)—only happens when the agent thread dequeues and processes the command. Until then, the global reference keeps the class alive.

§3.1 The Race Condition

This creates a window between global ref creation and deletion:

  1. Application thread calls saveEventInfoRefs → saveGlobalRef → NewGlobalRef, and enqueues the event command.
  2. Agent thread dequeues the command, sends the JDWP packet, and calls tossEventInfoRefs → tossGlobalRef → DeleteGlobalRef.

If the application drops all references to a class and triggers System.gc() while the event command is still in the queue, the global reference acts as a GC root, preventing the class (and its ClassLoader) from being collected.

On a loaded system, or if the agent thread is slow to drain the queue, this window can be significant, causing class unloading failure.

§4. The Issue in Action

Consider a scenario where you dynamically load a class via a custom URLClassLoader, use it, and then attempt to unload it by closing the loader and triggering garbage collection. You might expect the class to be collected when there are no more references to it. However, due to the presence of the JDWP agent, the JNI global reference created while emitting the ClassPrepare event may still be live, preventing the class from being unloaded.

Here’s a concrete example that demonstrates this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
import java.io.*;
import java.lang.ref.PhantomReference;
import java.net.*;
import java.util.*;

import com.sun.jdi.*;
import com.sun.jdi.connect.*;

class G {}

class Debuggee {
    static final String className = "G";

    static URLClassLoader loader;
    static PhantomReference<Object> ref;

    public static void main(String[] args) throws Exception {
        try (ServerSocket ss = new ServerSocket(9000); Socket s = ss.accept();
             PrintWriter out = new PrintWriter(s.getOutputStream(), true);
             BufferedReader in = new BufferedReader(new InputStreamReader(s.getInputStream()))) {

            String cmd;
            while ((cmd = in.readLine()) != null) {
                if (cmd.equals("LOAD")) {
                    loader = new URLClassLoader(new URL[]{new File("./").toURI().toURL()}, null);
                    loader.loadClass(className);
                    Class<?> cls = Class.forName(className, true /* initialize */, loader);
                    ref = new PhantomReference<>(loader, null);
                } else if (cmd.equals("UNLOAD")) {
                    loader.close();
                    loader = null;
                    System.gc();
                    boolean isUnloaded = ref.refersTo(null);
                    if (!isUnloaded) {
                      System.out.println("[Debuggee] unloading failed.");
                    }
                } else if (cmd.equals("EXIT")) break;
                out.println("OK"); // Unblock the debugger
            }
        }
    }
}

class Debugger {
    public static void main(String[] args) throws Exception {
      // 1. Find the specific SocketAttach connector
        VirtualMachineManager vmm = Bootstrap.virtualMachineManager();
        AttachingConnector ac = vmm.attachingConnectors().stream()
                .filter(c -> c.name().equals("com.sun.jdi.SocketAttach"))
                .findFirst()
                .orElseThrow(() -> new RuntimeException("SocketAttach connector not found"));

        // 2. Set the port
        Map<String, Connector.Argument> argMap = ac.defaultArguments();
        argMap.get("port").setValue("8000");
        VirtualMachine vm = ac.attach(argMap);
        System.out.println("[Debugger] JDI Attached.");

        // Socket Control
        try (Socket s = new Socket("localhost", 9000);
             PrintWriter out = new PrintWriter(s.getOutputStream(), true);
             BufferedReader in = new BufferedReader(new InputStreamReader(s.getInputStream()))) {

          for (int i = 0; i < 100; ++i) {
            for (String cmd : new String[]{"LOAD", "UNLOAD"}) {
                out.println(cmd);
                in.readLine(); // Wait for "OK"
            }
          }
          out.println("EXIT");
        }
        System.out.println("[Debugger] Disposed.");
    }
}

§4.1 How the Example Works

The code example consists of two parts: a Debuggee (the application being debugged) and a Debugger (the debugger client).

The Debuggee:

  • Listens for commands on a socket
  • When it receives a LOAD command, it loads class G into a custom URLClassLoader
  • It stores a PhantomReference to the loader (useful for detecting when the loader has been garbage collected)
  • When it receives an UNLOAD command, it closes the loader, nullifies the loader variable, and calls System.gc()
  • It then checks if the loader was garbage collected using the phantom reference

The Debugger:

  • Attaches to the debuggee using the JDWP Socket Attach connector
  • This causes JDWP events to be monitored, triggering ClassPrepare events whenever new classes are loaded
  • It then sends 100 load/unload cycles to stress-test the class unloading process

§4.2 Running the Example

Running it with:

1
2
3
4
5
javac test_jdwp.java;
java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000 Debuggee

# in another shell
java Debugger

From time to time, the unloading will fail, especially when there are heavy background tasks on the same system. (Alternatively, one can place an artificial sleep(1) inside commandLoop to induce delays in the agent thread. That way, this unloading failure becomes almost always reproducible.)

§5. References