FAQ Common support tasks

Table of Contents FAQ / Common support tasks Searching is not working - Restarting Solr Generate Sitemaps My data isn't right Repairing indexes Rebuilding the index completely Regenerating images Administration tasks Scheduled harvesting

FAQ / Common support tasks

Searching is not working - Restarting Solr

As all of the searches are directed to the Solr instance, this needs to be functioning correctly. Encountered problems include: * running out of memory

• fragmented index causing too many open file handles

If the web interface is available, it should be possible to check what the error is on the 'Logging' screen.

OutOfMemory errors are very infrequent. Restarting the Solr service is the quickest way of resolving this issue.

service Solr restart

This problem has been observed more frequently when the memory available to Solr (displayed on the dashboard/homepage of each instance) approaches the size of the index on disk or the number of segments exceeds 25.

A number of File exceptions are caused by the number of files open (Too many open files). This can be avoided by optimising the index and setting the file handle limits available to root (or whatever user Solr runs as) to a large number e.g.

root - nofiles 1000000

Generate Sitemaps

A periodic task to generate a sitemaps for the eMonocot portal based on the sitemaps standard for submission to search engines. via. Spring Batch: * Go to the 'Sitemap Generation' job in the eMonocot Harvester

• Manually add the parameters (copy a previous execution changing the 'attempt' parameter), e.g.

authority.name=e-monocot.org authority.last.harvested=1348142400004 authority.uri=not required attempt=12* The generated sitemaps are copied over to the sitemaps folder on the web server daily

Sitemaps have been registered in major search engines such as google using the google webmaster tools account. Sitemaps should be reharvested periodically, but can be manually resubmitted if neccessary.

My data isn't right

A catch-all for data that has been imported or search results that shouldn't appear; * Running the re-index job for the appropriate data type (see repairing indexes section) has solved a number of problems caused by inconsistent/'bad' archives and other changes

• Running the image processing job (see reprocess images section) will update image files and thumbnails that haven't been downloaded or are out of date

Repairing indexes

via. Spring Batch: * Go to the 'Re-Index' job in the eMonocot Harvester

• Manually add the parameters (copy a previous execution changing the 'attempt' parameter), e.g.

query.string=select t.id from Taxon t solr.selectedFacets=base.class_s=org.emonocot.model.Taxon query.type=org.emonocot.model.Taxon attempt=2015.04.14

The highlighted classes should be replaced appropriately * Click launch and check back periodically while the index is updated

Rebuilding the index completely

If it's necessary to rebuild all the indices (e.g. because the Solr server was rebuilt), these are all the entity types:

Entity	Base class	Query type
Annotation	org.emonocot.model.Annotation	org.emonocot.model.Annotation
Comment	org.emonocot.model.Comment	org.emonocot.model.Comment
IdentificationKey	org.emonocot.model.IdentificationKey	org.emonocot.model.IdentificationKey
Image	org.emonocot.model.Image	org.emonocot.model.Image
PhylogeneticTree	org.emonocot.model.PhylogeneticTree	org.emonocot.model.PhylogeneticTree
Place	org.emonocot.model.Place	org.emonocot.model.Place
Taxon	org.emonocot.model.Taxon	org.emonocot.model.Taxon
TypeAndSpecimen	org.emonocot.model.TypeAndSpecimen	org.emonocot.model.TypeAndSpecimen
Group	org.emonocot.model.auth.Group	org.emonocot.model.auth.Group
User	org.emonocot.model.auth.User	org.emonocot.model.auth.User
Organisation	org.emonocot.model.registry.Organisation	org.emonocot.model.registry.Organisation
Resource	org.emonocot.model.registry.Resource	org.emonocot.model.registry.Resource

Regenerating images

via. Spring Batch: * Go to the 'Image Processing' job in the eMonocot Harvester

• Manually add the parameters (copy a previous execution changing the 'attempt' parameter), e.g.

query.string=select i from Image i skip.unmodified=false attempt=2013.07.30-b* Click launch and check back periodically while the image files are re-processed

Administration tasks

A number of tasks can be achieved by any member of the portal 'Administrators' group: * Adding users to groups

• Adding/Updating/Deleting Users, Organisations (Sources) and their Resources

This is the preferred method because of the integration with Spring Security

N.B.Full administrator documentation is available on the Families and Orders scratchpad.

Scheduled harvesting

If the scheduled harvesting is not running, check whether there are any Jobs that appear to be running:

mysql> SELECT * FROM resource WHERE status NOT IN('COMPLETED', 'FAILED', 'ABANDONED', 'STOPPED') AND status IS NOT NULL

If there are results # Check Spring Batch Admin (e.g. UAT) to verify that there are no running jobs

1. Stop and abandon (buttons in prominent view) any jobs that shouldn't be running, e.g. They started days ago, they didn't fail cleanly
2. Update the database for jobs that aren't actually running

mysql> UPDATE resource SET status = 'STOPPED' WHERE id IN({the list of jobs that aren't actually running})

or if you are resonably certain that no-one will be activating a job; change the WHERE clause to update all problem jobs.