Error Doc id doesn't match the query in vector searches

Hi,

we are currently testing Solr 9.7 and experiencing an error we have not seen before with SolR 9.6.1 and we think the problem might occur in the underlying lucene code basis:

ERROR o.a.s.h.RequestHandlerBase Server exception =>
at org.apache.lucene.search.TopFieldCollector.populateScores(TopFieldCollector.java:478)
java.lang.IllegalArgumentException: Doc id 48567944 doesn't match the query
at org.apache.lucene.search.TopFieldCollector.populateScores(TopFieldCollector.java:478) ~[?:?]
at org.apache.solr.search.SolrIndexSearcher.populateScoresIfNeeded(SolrIndexSearcher.java:1766) ~[?:?]
at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1955) ~[?:?]
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1729) ~[?:?]
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:726) ~[?:?]
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:721) ~[?:?]
at org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1690) ~[?:?]
at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:432) ~[?:?]
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:456) ~[?:?]
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:226) ~[?:?]

We index the embeddings as nested fields and can reproduce the error with the following code:

We are able to reproduce the error using the same query. It seems to occur in approximately 5% of all vector queries.
We have one server running Solr 9.7 and three servers running Solr 9.6.1, all working on the same frozen index.
Only the Solr 9.7 server encounters the issue. We can rule out Java 21 and the corresponding optimizations or the new multithreading parameter as the root cause of the problem.
The index contains the document referenced in the error message.


String q = "{!knn f=vector topK=20}[0.031046804...";
SolrQuery sq = new SolrQuery("{!cache=false}" + q);
sq.addField("score"); // no error without the score field
sq.setRows(14); // Defect document must be included in result
sq.setSort("ID", ORDER.asc); // Order is not important, but ID is. No error e.g. with score
final QueryRequest r = new QueryRequest(sq, METHOD.POST);
SolrClient solrClient = SolRConnector.createServer(server);
QueryResponse response = r.process(solrClient);

Is there any additional information we can provide to help resolve this error?

Best regards

Andreas Moll