Occasional solr deployment failure

[dlpierce]

Every so often the solr service will fail, presumably after a automated deployment. Forcing a new deployment via the ECS dashboard will resolve the issue until the next time.

Nurax will show an error like:

RSolr::Error::Http - 500 Internal Server Error
Error: SolrCore 'nurax-pg' is not available due to init failure: Index dir '/var/solr/data/nurax-pg/data/index/' of core 'nurax-pg' is already locked. The most likely cause is another Solr server (or another solr core in this server) also configured to use this directory; other possible causes may be specific to lockType: native

URI: http://samvera-stack.svc.nurax.samvera.org:8983/solr/nurax-pg/select?wt=json
Request Headers: {"Content-Type"=>"application/x-www-form-urlencoded; charset=UTF-8"}
Request Data: "rows=1000&q=%7B%21terms+f%3Did%7D6f38d203-3350-40ee-a124-3c3b4c9a6add"

Backtrace: /usr/local/bundle/gems/rsolr-2.5.0/lib/rsolr/client.rb:220:in `rescue in execute'
/usr/local/bundle/gems/rsolr-2.5.0/lib/rsolr/client.rb:208:in `execute'
/usr/local/bundle/gems/rsolr-2.5.0/lib/rsolr/client.rb:203:in `send_and_receive'
(eval):2:in `post'
/usr/local/bundle/bundler/gems/hyrax-e28d1ca59620/app/services/hyrax/solr_service.rb:72:in `post'
/usr/local/bundle/bundler/gems/hyrax-e28d1ca59620/app/services/hyrax/solr_service.rb:40:in `post'
/usr/local/bundle/bundler/gems/hyrax-e28d1ca59620/app/presenters/hyrax/presenter_factory.rb:48:in `query'
/usr/local/bundle/bundler/gems/hyrax-e28d1ca59620/app/presenters/hyrax/presenter_factory.rb:42:in `load_docs'
/usr/local/bundle/bundler/gems/hyrax-e28d1ca59620/app/presenters/hyrax/presenter_factory.rb:31:in `build'
/usr/local/bundle/bundler/gems/hyrax-e28d1ca59620/app/presenters/hyrax/presenter_factory.rb:16:in `build_for'
/usr/local/bundle/bundler/gems/hyrax-e28d1ca59620/app/models/featured_work_list.rb:41:in `work_presenters'