solrconfig.xml provided in solr/conf compatibility with Solr 9

[sgurnick]

I provisioned a fresh Solr 9.7 server in conjunction with an upgrade of Avalon from 7.6 to 7.7.2. It is my understanding that Avalon 7.7.2 does support the use of Solr 9.

After I confirmed my standalone Solr 9 instance was up and running, I created a new core and used the files provided in solr/conf of the Avalon repo for the 7.7.2 tag.

After restarting Solr, the core could not load. These warnings and errors appeared in the solr.log file:

2024-10-30 17:09:32.929 WARN  (coreLoadExecutor-13-thread-1) [c: s: r: x:avalon t:] o.a.s.c.SolrConfig Couldn't add files from /opt/solr-9.7.0/contrib/analysis-extras/lib filtered by null to classpath: java.nio.file.NoSuchFileException: /opt/solr-9.7.0/contrib/analysis-extras/lib
2024-10-30 17:09:32.929 WARN  (coreLoadExecutor-13-thread-1) [c: s: r: x:avalon t:] o.a.s.c.SolrConfig Couldn't add files from /opt/solr-9.7.0/contrib/analysis-extras/lucene-libs filtered by null to classpath: java.nio.file.NoSuchFileException: /opt/solr-9.7.0/contrib/analysis-extras/lucene-libs
2024-10-30 17:09:32.931 WARN  (coreLoadExecutor-13-thread-1) [c: s: r: x:avalon t:] o.a.s.c.SolrConfig Couldn't add files from /opt/solr-9.7.0/contrib/extraction/lib filtered by .*\.jar to classpath: java.nio.file.NoSuchFileException: /opt/solr-9.7.0/contrib/extraction/lib
2024-10-30 17:09:32.931 WARN  (coreLoadExecutor-13-thread-1) [c: s: r: x:avalon t:] o.a.s.c.SolrConfig Couldn't add files from /opt/solr-9.7.0/dist filtered by solr-cell-\d.*\.jar to classpath: java.nio.file.NoSuchFileException: /opt/solr-9.7.0/dist

...

2024-10-30 20:04:20.752 WARN  (coreLoadExecutor-13-thread-1) [c: s: r: x:avalon t:] o.a.s.s.FieldTypePluginLoader TokenizerFactory is using deprecated 5.0.0 emulation. You should at some point declare and reindex to at least 8.0, because 7.x emulation is deprecated and will be removed in 9.0
...

2024-10-30 17:09:34.538 ERROR (coreLoadExecutor-13-thread-1) [c: s: r: x: t:] o.a.s.c.CoreContainer SolrCore failed to load on startup => org.apache.solr.common.SolrException: Unable to create core [avalon]
        at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1787)
org.apache.solr.common.SolrException: Unable to create core [avalon]
        at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1787) ~[?:?]
        at org.apache.solr.core.CoreContainer.lambda$loadInternal$13(CoreContainer.java:1083) ~[?:?]
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:212) ~[metrics-core-4.2.26.jar:4.2.26]
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$1(ExecutorUtil.java:449) ~[?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
        at java.base/java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: org.apache.solr.common.SolrException:  Error loading class 'org.apache.solr.handler.extraction.ExtractingRequestHandler'

It appears the problem is caused by lines 31-37 in the solrconfig.xml file:

avalon/solr/conf/solrconfig.xml

Lines 31 to 37 in c1a19e0

	<luceneMatchVersion>5.0.0</luceneMatchVersion>
	<lib dir="${solr.install.dir:../../../..}/modules/analysis-extras/lib" />
	<lib dir="${solr.install.dir:../../../..}/contrib/analysis-extras/lib" />
	<lib dir="${solr.install.dir:../../../..}/contrib/analysis-extras/lucene-libs" />
	<lib dir="${solr.install.dir:../../../..}/contrib/extraction/lib" regex=".*\.jar" />
	<lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-cell-\d.*\.jar" />

In a fresh install of Solr 9:

• contrib and dist sub-directories within the solr install directory are no longer present
• solr-cell-\d.*\.jar file no longer exists

These are reflected in the Other section of the Major Changes in Solr 9 page:

• Contrib modules are now just "modules".
• The "dist" folder in the release has been removed.
• The solr-extraction module has been cleaned up to produce solr-extraction- jar instead of solr-cell- jars.

To get the avalon core loaded in my Solr 9 instance, I updated the lines 31-37 to just the following lines:

<luceneMatchVersion>9.11</luceneMatchVersion>

<lib dir="${solr.install.dir:../../../..}/modules/analysis-extras/lib" />
<lib dir="${solr.install.dir:../../../..}/modules/extraction/lib" regex=".*\.jar" />

After restarting Solr, the core loaded successfully without the above mentioned warnings/errors.

Questions:

• are the files in solr/conf in the avalon repo what I should be using when creating a new Solr 9 core for use with Avalon 7.7 or higher?
  • if yes: are these files meant to simply be dropped into the solr 9 avalon core as-is without changes?
    • if yes: I'm not sure what I might have missed because I couldn't get the avalon core to load without making changes mentioned above
    • if no: is there is documentation somewhere indicating what avalon-specific changes to the configs are necessary to support Solr 9?
  • if no: what is the location of the appropriate Solr 9 conf files that should be used?

Thank you for your help and guidance.

[cjcolvar]

@sgurnick I thought the config in solr/conf worked out of the box with the version of solr 9 (I think 9.6 locally) that we have. Your changes seem to make sense so maybe they're safe to make no matter the solr 9 version. I'll take a look at this more when I'm back in the office on Monday and get back to you.

[cjcolvar]

I started a fresh solr 9.7 locally and it loaded the the solr core for me with the config in solr/conf. FWIW it looks like the only change between Avalon 7.7.2 and 7.8 is commenting out a couple copyField configs (v7.7.2...main#diff-76699b2586a37ec4cfcb45a02364fc31ea75c15298528af372eb6e6c4fe82ed1)
Also the way we're setting up the core is running solr-precreate avalon /path/to/avalon/config in a solr docker container.

[cjcolvar]

@sgurnick Are you still having problems with this?

[sgurnick]

@cjcolvar No changes in behavior since I posted this issue. My workaround is what I stated in this issue's description, where I modify lines 31-37 in solrconfig.xml to remove references to directories/files that either no longer exist or have been moved in Solr 9. It is curious how in your deployments of Solr 9 you aren't getting these errors when using this solrconfig.xml, since these are listed as breaking changes in the release notes. Perhaps the Docker image of Solr 9 is different than a standard install? I doubt it, but there seems to be some difference of what is laid out on the filesystem between a standard server install for Solr 9 versus the container image.

I am running the Avalon stack on separate virtual machines (we're not using Docker for our deployment). Each of the following are running on standalone virtual machines:

• Avalon Web Node
• Avalon Worker Node
• Fedora
• Solr
• MySQL

When I install Solr 9, I'm using their built-in provisioning scripts. And when I set-up the Avalon core, I'm using their built-in solr-create script, which is the same as what you do, just it's meant to be run after Solr is running.

So to answer your question, yes, this issue persists for me. My workaround seems to get me going, but does require manual intervention and tracking of a solrconfig.xml that I imagine over time will diverge from what is a part of the Avalon repo.

[cjcolvar]

I hadn't considered that the official docker image would be that different from the non-docker install. I'll take another look. Thanks!