Inconsistent randomisation behaviour

Hi all,

I’m seeing some strange behaviour trying to reproduce a test failure in IntelliJ, compared to what happens when I run the test on the command line via gradle.

The following failed in the elasticsearch CI this morning, and reproduces when I run it on my laptop:

./gradlew :lucene:test-framework:test --tests "org.apache.lucene.tests.store.TestMockDirectoryWrapper.testIsLoadedOnSlice" -Ptests.jvms=12 -Ptests.jvmargs= -Ptests.seed=7AA64BB2BA5EE192 -Ptests.useSecurityManager=true -Ptests.nightly=true -Ptests.gui=true -Ptests.file.encoding=US-ASCII -Ptests.vectorsize=128 -Ptests.forceintegervectors=true

However, if I take that seed and fix it on TestMockDirectoryWrapper, the same test passes when run inside IntelliJ.  Adding some debug println statements indicates that the seed is indeed the same, but the random code paths taken seem to be different.  I made the following changes to the code:

---

$ git diff
diff --git a/lucene/test-framework/src/java/org/apache/lucene/tests/store/BaseDirectoryTestCase.java b/lucene/test-framework/src/java/org/apache/lucene/tests/store/BaseDirectoryTestCase.java
index 27bfa3c72f1..e522168b43c 100644
--- a/lucene/test-framework/src/java/org/apache/lucene/tests/store/BaseDirectoryTestCase.java
+++ b/lucene/test-framework/src/java/org/apache/lucene/tests/store/BaseDirectoryTestCase.java
@@ -22,6 +22,7 @@ import static org.hamcrest.Matchers.containsString;
 import static org.hamcrest.Matchers.hasItemInArray;
 import static org.hamcrest.Matchers.not;

+import com.carrotsearch.randomizedtesting.RandomizedContext;
 import com.carrotsearch.randomizedtesting.RandomizedTest;
 import com.carrotsearch.randomizedtesting.generators.RandomBytes;
 import com.carrotsearch.randomizedtesting.generators.RandomNumbers;
@@ -1647,6 +1648,10 @@ public abstract class BaseDirectoryTestCase extends LuceneTestCase {
         }
         var loaded = in.isLoaded();

+        System.out.println(RandomizedContext.current().getRunnerSeedAsString());
+        System.out.println(dir.toString());
+        System.out.println(dir.getClass().getName());
+
         if (Constants.WINDOWS) {
           // On Windows, we temporarily don't care until this is fixed: #14050
         } else if (FilterDirectory.unwrap(dir) instanceof MMapDirectory
diff --git a/lucene/test-framework/src/test/org/apache/lucene/tests/store/TestMockDirectoryWrapper.java b/lucene/test-framework/src/test/org/apache/lucene/tests/store/TestMockDirectoryWrapper.java
index b5ca29dcbb0..d06e32b4ba1 100644
--- a/lucene/test-framework/src/test/org/apache/lucene/tests/store/TestMockDirectoryWrapper.java
+++ b/lucene/test-framework/src/test/org/apache/lucene/tests/store/TestMockDirectoryWrapper.java
@@ -21,6 +21,7 @@ import java.io.FileNotFoundException;
 import java.io.IOException;
 import java.nio.file.NoSuchFileException;
 import java.nio.file.Path;
+import com.carrotsearch.randomizedtesting.annotations.Seed;
 import org.apache.lucene.document.Document;
 import org.apache.lucene.store.ByteArrayDataInput;
 import org.apache.lucene.store.ByteBuffersDirectory;
@@ -33,6 +34,7 @@ import org.apache.lucene.tests.index.RandomIndexWriter;

 // See: https://issues.apache.org/jira/browse/SOLR-12028 Tests cannot remove files on Windows
 // machines occasionally
+@Seed("7AA64BB2BA5EE192")
 public class TestMockDirectoryWrapper extends BaseDirectoryTestCase {

   @Override

—

Running from gradle I get the following output:

1> 7AA64BB2BA5EE192
1> MockDirectoryWrapper(NRTCachingDirectory(MMapDirectory@/Users/romseygeek/projects/lucene/lucene/test-framework/build/tests-tmp/lucene.tests.store.TestMockDirectoryWrapper_7AA64BB2BA5EE192-008/index-MMapDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2bb5ff68; maxCacheMB=0.9546899795532227 maxMergeSizeMB=0.9689407348632812))
1> org.apache.lucene.tests.store.MockDirectoryWrapper

And running via IntelliJ:

7AA64BB2BA5EE192
MockDirectoryWrapper(ByteBuffersDirectory@7e97aee2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@dfeade5)
org.apache.lucene.tests.store.MockDirectoryWrapper

Same seed, different Directory implementations.  This happens with both the IntelliJ and gradle test runners, and I’ve checked that both situations use the same JDK.  I am pretty stumped at the moment.  Any ideas, anyone?

- Alan