In addition to making it easy to monitor local Java applications, JVisualVM also supports Remote monitoring of Java processes through Remote nodes. This allows you to use a visual interface to more efficiently monitor your production environment and locate CPU or memory problems.

VisualVM connects to remote nodes in two ways: JSTATD and JMX. JSTATD only supports Monitor, while JMX supports Threads and Sampler. They are explained in detail below.

A, JSTATD connection mode

To start JSTATD on a remote host, you need to configure permissions first, otherwise you will report the following security error:

Could not create remote object access denied ("java.util.PropertyPermission" "java.rmi.server.ignoreSubClasses" "write")  java.security.AccessControlException: access denied ("java.util.PropertyPermission" "java.rmi.server.ignoreSubClasses" "write") at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) at java.security.AccessController.checkPermission(AccessController.java:884) at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) at java.lang.System.setProperty(System.java:792) at sun.tools.jstatd.Jstatd.main(Jstatd.java:139)

So first create a jstatd.all.policy file under $JAVA_HOME/bin. The contents are as follows:

grant codebase "file:${java.home}/.. /lib/tools.jar" { permission java.security.AllPermission; };

Note: The file tools.jar does not exist in versions after Java9. It is said to work as follows:

grant codebase "jrt:/jdk.jstatd" {
   permission java.security.AllPermission;
};
grant codebase "jrt:/jdk.internal.jvmstat" {    
   permission java.security.AllPermission;    
};

Then start JSTATD under $JAVA_HOME/bin:

jstatd -J-Djava.security.policy=jstatd.all.policy

On the local jVisualVM clientRemoteAdd node below:

Then select the corresponding Java process to monitor. However, in JSTATD mode, you can only use the simple Monitor, not the Sampler functionality. If Sampler is required, connect using JMX mode.

Here is the Monitor screenshot:

II. JMX connection mode

JMX is Java Virtual Machine, Java comes with out-of-the-box management tools and interfaces.

The Java virtual machine (Java VM) has built-in instrumentation that enables you to monitor and manage it using the Java Management Extensions (JMX) technology. These built-in management utilities are often referred to as
out-of-the-box management tools for the Java VM. You can also monitor any appropriately instrumented applications using the JMX API.

Add the JMX configuration when the remote host starts the application, as follows:

java -Djava.rmi.server.hostname=ttg12 -Dcom.sun.management.jmxremote.port=9999 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false HelloWorld

Note: This approach that does not require validation is not secure! Only used for testing, can not be used in the external network accessible production environment! Please refer to the “Using Password Authentication” section of the official documentation for how to set up a secure connection.

If the Java application is already started and JMX is not started in the startup command, it can be proxies with the diagnostic command ManagementAgger.start. Use JCMD help managementagent.start to see the flags and how to use them. See the official documentation for more details on JCMD. Startup example: 

faceless@ttg12:~/tmp$ jcmd 26548 ManagementAgent.start jmxremote.host=ttg12 jmxremote.port=9999 jmxremote.rmi.port=9999 jmxremote.ssl=false jmxremote.authenticate=false
26548:
Command executed successfully

Further, the official recommendation is to enable JMX as soon as possible when you launch your application, because JMX does not impose any burden on your application, but can be a great convenience for later monitoring and problem solving. The well-known RocketMQ message queue provides a monitoring and management interface through JMX.

Set up JMC JMX for remote monitoring: JMX can be used to connect to a Java application remotely using tools such as Mission Control or Visual VM. Unless you can run these tools on the same machine that is running your application, setting this up can be helpful later on to monitor the application, send diagnostic commands, manage flight recordings and so on.
There is no performance overhead in enabling JMX.

Source: Enable Options/Flags for JVM Troubleshooting Point 6.

Add a JMX connection to a remote host in the VisualVM client:

Enter the host and port number, and check “No SSL” :

OK, now you can Monitor threads and use Sampler in addition to Monitor.

Threads interface is shown below:

Sampler interface as shown below:

Java: Oracle JDK 8 Server OS: Ubuntu 18.04 Client OS: MacOS 10.15.5