Installation instructions for Kitodo.Production 3.4

1. Preliminary Steps

1a) Check host configuration

Check before installing with the command

sudo hostname -f

whether the fully qualified domain name (FQDN) corresponds to the planned server name. This is used during the LDAP installation to form the Root Distinguished Name (RDN) and the basic configuration of the LDAP, which can only be corrected later with great effort. Here the FQDN, eg servername.rz.example.com, should be displayed, not just servername. If not, configure /etc/hostsproperly first.

1b) Setup the data partition

Digitisation generates a lot of data. Typically, this data is stored on its own partition, which is likely a network device (eg NAS). With the following command we create a mount point for the data partition as /mnt/data:

sudo mkdir /mnt/data

The set up of data storage is is highly dependent on the hardware and networks used. If you want to use a data partition, ensure that the data partition is mounted on the specified mount point. Now is a good time to consider whether you would like to set up LVM.

2. Install software packages

Install Tomcat 9, MySQL, ImageMagick, LDAP incl. Utils, Samba, the Apache web server and the Jakarta Connector with package manager. For the installation you will also need HTTPS support for apt, unzip and the dos2unix tool. During the installation you will be asked for the password for the LDAP server. Production stores the password unencrypted in the database. Don't choose a private/valuable password.

sudo apt-get update
sudo apt-get install tomcat9 mysql-server imagemagick slapd ldap-utils samba apt-transport-https unzip dos2unix

3. Determing Root Distinguished Name

Using the command

sudo slapcat

you can view the contents of the LDAP server. The entries are separated by blank lines. Here you will find the generated Root Distinguished Name (RDN) in the entry with the property objectClass:top under the key dn.

If you are setting up a test system that doesn't need a DNS name, use dc=nodomain as the RDN. The LDAP admin user has the Common Name (CN) cn=admin,dc=nodomain. This information is also used in these installation instructions. If you are installing the server on your network, it will say something like dc=rz,dc=example,dc=de. Then use this string instead of dc=nodomain later in the installation instructions.

4. Install name service switch

Now install the Name Service Switch LDAP library:

sudo apt-get install libnss-ldap

During the installation you will be asked for some settings:


LDAP server Uniform Resource Identifier:	ldapi:///
LDAP search base:	ou=users,dc=nodomain
LDAP version:	3
Make local root database admin:	No
Does the LDAP server require login?	No

If you want or have to repeat this step, enter the following two commands one after the other:

sudo apt-get purge libnss-ldap
sudo apt-get install libnss-ldap

5. Install ElasticSearch

Note that you must install ElasticSearch version 7.

export JAVA_HOME=/usr/lib/jvm/default-java
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo add-apt-repository "deb https://artifacts.elastic.co/packages/7.x/apt stable main"
sudo apt-get update
sudo apt-get install elasticsearch
sudo systemctl enable elasticsearch
sudo systemctl start elasticsearch

6. Prepare database

While the digitisation data is stored in files on the data staging point, the data for system processes (projects, employees, progress of objects in the workflow, etc.) is stored in a database. You need to create a database on the MySQL server and configure access for Production.

sudo mysql
create database kitodo;
create user 'kitodo'@'localhost' identified by 'kitodo';
grant all on kitodo.* to 'kitodo'@'localhost';
exit;

Then download the database content template and load it into the database:

wget https://github.com/kitodo/kitodo-production/releases/download/kitodo-production-3.4.0/kitodo_3-4-0.sql
mysql -u kitodo -pkitodo -D kitodo < kitodo_3-4-0.sql

The following errors currently have to be corrected manually during an installation:

Schema-validation: missing table [client_x_listColumn] https://github.com/kitodo/kitodo-production/issues/3998

Flyway migration is failing https://github.com/kitodo/kitodo-production/issues/3810

7. Prepare LDAP for Samba

Load the Samba-Schema into LDAP:

wget https://raw.githubusercontent.com/zentyal/samba/master/examples/LDAP/samba.ldif
cat samba.ldif | sudo ldapadd -Q -Y EXTERNAL -H  ldapi:///

Create an entry in the LDAP for the group of production users. To do this, create a file users.ldifwith the following content:

dn: ou=users,dc=nodomain
objectClass: organizationalUnit
ou: users

Load the file into LDAP with:

ldapadd -x -D cn=admin,dc=nodomain -w 'LDAP-Admin-Passwort' < users.ldif

Determine the group number for all users:

grep users /etc/group

It is the number between the second and third colon. (Most of the time this is 100.)

Determine the highest assigned user number. For this, look at the file /etc/passwd.

less /etc/passwd

Each row corresponds to a user account. The user number is between the second and third colon (the first of the two numbers). All users who are not system accounts have a user number from 1000 upwards. Find the first number greater than 1000 that no longer occurs. This is the next free user number. End the display by pressing the button Q.

Create a file nextfreeunixid.ldif with the following content. Enter the information you found:

dn: cn=NextFreeUnixId,dc=nodomain
objectClass: inetOrgPerson
sn: NextFreeUnixId
objectClass: posixAccount
cn: NextFreeUnixId
uid: nextfreeunixid
uidNumber: #die nächste freie Benutzendennummer#
gidNumber: #die ermittelte Gruppennummer#
homeDirectory: /dev/null

Load the file into LDAP with:

ldapadd -x -D cn=admin,dc=nodomain -w 'LDAP-Admin-Passwort' < nextfreeunixid.ldif

8. Samba configuration

Edit the file /etc/samba/smb.conf so that it contains the following:

[global]
workgroup = KITODO
unix extensions = no
wide links = yes
load printers = no
security = user
invalid users = root
encrypt passwords = yes

ntlm auth = yes
lanman auth = no
client ntlmv2 auth = yes

passdb backend = ldapsam:ldap://127.0.0.1
ldap suffix = dc=nodomain
ldap user suffix = ou=users
ldap group suffix = ou=groups
ldap admin dn = cn=admin,dc=nodomain
ldap ssl = no
ldap passwd sync = yes
ldap delete dn = no

[homes]
comment = Home Directories
path = /usr/local/kitodo/users/%U
read only = no
browseable = no
valid users = %S
guest ok = no
inherit permissions = yes

Edit the Name Service Switch config file /etc/nsswitch.conf: Add the argument ldap: to the passwd: line.

passwd:         compat systemd ldap

In Samba, set the LDAP-Password:

sudo smbpasswd -w 'LDAP-Admin-Password'

Restart the Samba services:

sudo systemctl restart smbd nmbd

9. Create program folders and files

Download the software, modules and configuration files, create a program folder, unzip the modules and configuration files. Place the metadata folder on the data partition. Assign the necessary permissions:

wget https://github.com/kitodo/kitodo-production/releases/download/kitodo-production-3.4.0/kitodo-3.4.0.war
wget https://github.com/kitodo/kitodo-production/releases/download/kitodo-production-3.4.0/kitodo_3-4-0_config_modules.zip

sudo unzip kitodo_3-4-0_config_modules.zip -x kitodo_3-4-0/scripts/*.bat -d /usr/local
sudo mv /usr/local/kitodo_3-4-0 /usr/local/kitodo
sudo mkdir /usr/local/kitodo/messages
sudo unzip -j kitodo-3.4-0.war 'WEB-INF/classes/messages/*' -d /usr/local/kitodo/messages
cd /usr/local/kitodo
sudo mv metadata /mnt/data
sudo ln -s /mnt/data/metadata
sudo mkdir logs temp
# sudo chown -R ‹Admin-User›:users config messages rulesets xslt #optional
sudo chown -R tomcat9:tomcat9 diagrams logs metadata temp users

10. Configure scripts

sudo dos2unix scripts/*.sh
sudo chown tomcat9 scripts/*.sh
sudo chmod 544 scripts/*.sh
cd

Edit the file /usr/local/kitodo/scripts/script_createDirUserHome.sh as follows:

#!/bin/sh
User="$1"
Home="$2"

/bin/mkdir "$Home"
/bin/chmod g+w "$Home"
sudo /bin/chown $User "$Home"
sudo /bin/chgrp tomcat9 "$Home"

Edit the file sudo nano /usr/local/kitodo/scripts/script_createSymLink.sh as follows:

#!/bin/sh
Source="$1"
Target="$2"
User="$3"

/bin/ln -s "$Source" "$Target"
sudo /bin/chown -R $User "$Source"

Create a file with the command sudo visudo -f /etc/sudoers.d/kitodo and the following content:

User_Alias TOMCAT_USER = tomcat9

Cmnd_Alias CHOWN_METADATA = /bin/chown -R * /usr/local/kitodo/metadata/*
Cmnd_Alias CHOWN_USERS = /bin/chown * /usr/local/kitodo/users/*
Cmnd_Alias CHGRP_USERS = /bin/chgrp tomcat9 /usr/local/kitodo/users/*

TOMCAT_USER ALL=NOPASSWD: CHOWN_METADATA, CHOWN_USERS, CHGRP_USERS

11. Configure Tomcat

Adjust the default Tomcat settings, /etc/default/tomcat9: Add the following JAVA_OPTS line in order to give Tomcat more RAM (4G, of which 256m for the program code):

JAVA_OPTS="-Djava.awt.headless=true -XX:+UseConcMarkSweepGC -Xmx4G -XX:MaxPermSize=256m"

When running systemd in full or strict ProtectSystem mode (newer Linux distributions), write access to the directory hierarchy must be enabled under /usr/local/kitodo/ by adding the line ReadWritePaths=/usr/local/kitodo/ReadWritePaths= to the file /lib/systemd/system/tomcat9.service in the [Service] section. (Repeat the line for each shared path.)

After that, restart Tomcat:

sudo systemctl daemon-reload
sudo service tomcat9 restart

Install the web application:

sudo cp kitodo-3.4.0.war /var/lib/tomcat9/webapps/kitodo##3.4.0.war

12. Configure log rotation

Create the file /etc/logrotate.d/kitodo eg with the following contents:

/usr/local/kitodo/logs/kitodo.log {
    copytruncate
    daily
    rotate 7
    ifempty
    compress
    compresscmd /usr/bin/gzip
    uncompresscmd /usr/bin/gunzip
    missingok
    nomail
 }

13. Configuration in the front end

You can reach the web application in a browser at the address: http://host:8080/kitodo. Sign in with the username testadmin and password test.

13a) Build the search index

The System-Indexing page will open. Generate the ElasticSearch Mapping and fully index the whole index.

13b) Configure LDAP

Then switch to Users via the main menu which you'll find by clicking the Rubik's cube icon, top right. Create a new LDAP server with the following settings using the “New” button:


Manager login	`cn=admin,dc=nodomain`
nextFreeUnixIdPattern	`cn=NextFreeUnixId,dc=nodomain`
URL	`ldap://localhost:389/`
Manager password	LDAP-Admin-Passwort

Switch to the LDAP Groups tab. Find out the Secure Identifier (SID) with the command:

sudo net getlocalsid

Adjust the LDAP group settings as follows:

		Example
User DN	`cn={login},ou=users,dc=nodomain`	`cn={login},ou=users,dc=nodomain`
gidNumber	Group number	`100`
sambaSID	SID`-{uidnumber*2+1001}`	`S-1-5-21-123456789-1234567890-1234567890-{uidnumber*2+1001}`
LDAP-Server	select the server you just created	Local LDAP Server
loginShell	`/bin/false`	`/bin/false`
sambaPrimaryGruopSID	SID`-1000`	`S-1-5-21-123456789-1234567890-1234567890-1000`

Further down on the LDAP Groups tab, enable LDAP authentication. Then go to the Users page – Users and edit the users one by one. Write the LDAP configuration for each user.

Adjust the configuration of the web application under /var/lib/tomcat9/webapps/kitodo##3.2/WEB-INF/classes/kitodo_config.properties: Change the line ldap_use to the value true.