Clean install of hydra-jetty results in broken solr deployment

[kerchner]

@StudioZut encountered this problem and I was able to reproduce it in a brand new instance. I installed just sufia (no DB), java 8, and ran the hydra-jetty install steps (rake sufia:jetty:config). Jetty runs, with fedora, but solr doesn't run correctly. Error 500 with stack trace:

Problem accessing /solr/. Reason:

    {msg=SolrCore 'development' is not available due to init failure: Could not load conf for core development: Plugin init failure for [schema.xml] fieldType "text": Plugin init failure for [schema.xml] analyzer/tokenizer: Error loading class 'solr.ICUTokenizerFactory'. Schema file is /home/ubuntu/scholarspace/jetty/solr/development-core/conf/schema.xml,trace=org.apache.solr.common.SolrException: SolrCore 'development' is not available due to init failure: Could not load conf for core development: Plugin init failure for [schema.xml] fieldType "text": Plugin init failure for [schema.xml] analyzer/tokenizer: Error loading class 'solr.ICUTokenizerFactory'. Schema file is /home/ubuntu/scholarspace/jetty/solr/development-core/conf/schema.xml
    at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:745)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:307)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1476)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:370)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
    at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.common.SolrException: Could not load conf for core development: Plugin init failure for [schema.xml] fieldType "text": Plugin init failure for [schema.xml] analyzer/tokenizer: Error loading class 'solr.ICUTokenizerFactory'. Schema file is /home/ubuntu/scholarspace/jetty/solr/development-core/conf/schema.xml
    at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:66)
    at org.apache.solr.core.CoreContainer.create(CoreContainer.java:489)
    at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255)
    at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    ... 1 more
Caused by: org.apache.solr.common.SolrException: Plugin init failure for [schema.xml] fieldType "text": Plugin init failure for [schema.xml] analyzer/tokenizer: Error loading class 'solr.ICUTokenizerFactory'. Schema file is /home/ubuntu/scholarspace/jetty/solr/development-core/conf/schema.xml
    at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:595)
    at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:166)
    at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:55)
    at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)
    at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:90)
    at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:62)
    ... 7 more
Caused by: org.apache.solr.common.SolrException: Plugin init failure for [schema.xml] fieldType "text": Plugin init failure for [schema.xml] analyzer/tokenizer: Error loading class 'solr.ICUTokenizerFactory'
    at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:177)
    at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:486)
    ... 12 more
Caused by: org.apache.solr.common.SolrException: Plugin init failure for [schema.xml] analyzer/tokenizer: Error loading class 'solr.ICUTokenizerFactory'
    at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:177)
    at org.apache.solr.schema.FieldTypePluginLoader.readAnalyzer(FieldTypePluginLoader.java:362)
    at org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:95)
    at org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:43)
    at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:151)
    ... 13 more
Caused by: org.apache.solr.common.SolrException: Error loading class 'solr.ICUTokenizerFactory'
    at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:490)
    at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:593)
    at org.apache.solr.schema.FieldTypePluginLoader$2.create(FieldTypePluginLoader.java:342)
    at org.apache.solr.schema.FieldTypePluginLoader$2.create(FieldTypePluginLoader.java:335)
    at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:151)
    ... 17 more
Caused by: java.lang.ClassNotFoundException: solr.ICUTokenizerFactory
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:814)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:474)
    ... 21 more
,code=500}

I am fairly confident that this has something to do with it. https://github.com/projecthydra/sufia/pull/1300/files

I tried overriding sufia-dev.rake with the version there that adds pinning the version of hydra-jetty but haven't gotten this to work yet (this part may be worth another try).

@kilahimm sending this to you as I'm hoping you might have some bandwidth at the moment. I am severely crunched for time at the moment, and would like to get @StudioZut up and running. An alternative would be to have him set up the separate deployment of solr and fedora, but on the same server, as per the production install section of the readme.

[kilahimm]

@StudioZut did you the following again:

  % rake jetty:clean
  % rake sufia:jetty:config
  % rake jetty:start

[kerchner]

@kilahimm I know I did those on my clean instance, and tried a few times with @StudioZut on his instance. I believe there is a problem with the version of hydra-jetty; I would recommend trying to see if you can get it to pin the version back (the solution in samvera-deprecated/sufia#1300 might be worth another try )

[kerchner]

This may have been resolved in Sufia 6.4.0 by samvera-deprecated/sufia@cf4de2a / samvera-deprecated/sufia#1330 - test and close if this works now.