JDK version:

java version "1.8.0_291"
Java(TM) SE Runtime Environment (build 1.8.0_291-b10)
Java HotSpot(TM) 64-Bit Server VM (build 25.291-b10, mixed mode)

If we write the following code:

import java.util.concurrent.ThreadLocalRandom;

public class Test {

    public static void main(String[] args) {
        int exceptionCount = 0;
        while (true) {
            try {
                Object object = null;
                if (ThreadLocalRandom.current().nextInt(2) == 0) {
                    object = new Object();
                }
                object.hashCode();
            } catch (Exception e) {
                System.err.println(  exceptionCount);
                e.printStackTrace();
            }
        }
    }

}

Part of the output is:

21123
java.lang.NullPointerException
    at Test.main(Test.java:13)
21124
java.lang.NullPointerException
    at Test.main(Test.java:13)
21125
java.lang.NullPointerException
21126
java.lang.NullPointerException

That is, the stack frame is no longer printed after the exception occurs 21124 times.

Let's make a small change to the above code, pay attention to the parameters of the nextInt method:

import java.util.concurrent.ThreadLocalRandom;

public class Test {

    public static void main(String[] args) {
        int exceptionCount = 0;
        while (true) {
            try {
                Object object = null;
                if (ThreadLocalRandom.current().nextInt() == 0) {
                    object = new Object();
                }
                object.hashCode();
            } catch (Exception e) {
                System.err.println(  exceptionCount);
                e.printStackTrace();
            }
        }
    }

}

Part of the output is:

115711
java.lang.NullPointerException
    at Test.main(Test.java:13)
115712
java.lang.NullPointerException
    at Test.main(Test.java:13)
115713
java.lang.NullPointerException
115714
java.lang.NullPointerException

That is, the stack frame is no longer printed after the exception occurs 115712 times.

Now I want to know how the count of executions of this method that triggers the omitted exception stack frame is calculated?

Reference:

But in the case of a resolved Methodref, JVM throws NullPointerException from a different place in the code. This another path records the fact of a thrown exception in the MethodData structure:

MethodData is a JVM structure that holds method's runtime profile to be used for JIT compilation. The presense of the profile makes JVM compile the method directly at tier 4 (because there is no need to collect profile at tier 3, if it was already collected in the interpreter).

In this case, when a number of loop iterations reaches 40000 (-XX:Tier4BackEdgeThreshold), the method is scheduled for compilation at tier 4. But since the feedback step is 1024 (2^{Tier0BackedgeNotifyFreqLog}), the actual number of iterations before JIT compilation is 40*1024 = 40960.

Summary

Getting back to your original example. When ThreadLocalRandom.current().nextInt(2) == 0 is true, hashCode() is successfully resolved and executed. Subsequent NPEs are thrown from invokevirtual implementation, and as a side effect, JVM creates MethodData for the main method. When the number of loop iterations exceeds 40000 (half of them resulting in a normal call, and half in a thrown exception), the method gets compiled by C2 with OmitStackTraceInFastThrow optimization. Hence a little more than 20000 exceptions before the optimization.

On the contrary, ThreadLocalRandom.current().nextInt() == 0 is almost never true. Therefore hashCode() call never succeeds: NPE is thrown while resolving the constant pool entry, and the entry remains unresolved. That's why the method is first compiled by C1 (after 60K iterations), then recompiled by C2 (after 40K more iterations). So the method throws more than 100K exceptions in total before OmitStackTraceInFastThrow optimization takes place.