Skip to content

Commit c781275

Browse files
YetAnotherGerritmgrunerMrGeneration
authoredNov 21, 2022
Introduce mysql/mariadb to pgsql-migration (#253)
* Update deprecated 'Cache.clear' calls to 'Rails.cache.clear'. * WIP: Introduce conversion to pgsql * Update .gitignore * Install of PSQL itself missing There needs to be more clarification, that PSQL itself needs to be installed still. * Typo * Update .gitignore * Moved some content to includes * Update migrate-to-postgresql.rst * Remove/Update existing documentation * Update migrate-to-postgresql.rst Testet CentOS7+8 * Update migrate-to-postgresql.rst Removed WiP-notification. * No reindex needed * Remove duplicates, adjust wording - Remove duplicate mariadb / mysql utf8 info - change order: psql > mysql/mariadb - consolidate mariadb and mysql into one line - call the link to the migration guide "proof of concept" to reduce duplicate wording * Fix typo, make this a clear PoC * Solve pgsql installation inconsistencies; Add pgsql installation include We got it any way now so why not use it consistent ... * Remove long partly duplicate intro in upcoming action and convert them to tabs * Correct SUSE wording * Remove unneeded mention of database server type There's nothing to choose from at that point. * Stop zammad with a code block, looks better * Reduce space losses - more tabs - remove migration from X to Y part as we know why we're here (hopefully) * Remove todo * Reduction of duplicates Co-authored-by: Martin Gruner <mg@zammad.com> Co-authored-by: Marcel Herrguth <github@thehomeofanime.de>
1 parent ec2714d commit c781275

15 files changed

+297
-152
lines changed
 

‎.gitignore

+9-1
Original file line numberDiff line numberDiff line change
@@ -1 +1,9 @@
1-
_build/*
1+
*.swp
2+
*~
3+
/_build
4+
/.vscode
5+
.transifexrc
6+
.tx
7+
.DS_Store
8+
locale/.doctrees
9+

‎appendix/backup-and-restore/index.rst

-8
Original file line numberDiff line numberDiff line change
@@ -63,14 +63,6 @@ or availability.
6363
not work and is out of scope of this documentation as of now
6464
* Backup & Restore is only available for PostgreSQL and MySQL / MariaDB like
6565
installations
66-
* Switching / Converting database installations *is not* possible
67-
68-
.. note::
69-
70-
If you require support with migrating your MySQL / MariaDB installation
71-
into a PostgreSQL installation, you can contact `Zammads sales team`_
72-
for commercial support.
73-
7466
* Starting with Zammad 5.0 the scripts *require* user & password
7567
authentication. This is supported by most of our installation types
7668
* Backup & Restore is always a full dump of everything (no incrementals)

‎appendix/backup-and-restore/little-helpers.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ Database Helper: (re)set password
2525
Scopes
2626
Mostly the following installation types will be affected / relevant:
2727

28-
* package installations (especially CentOS & SuSe)
28+
* package installations (especially CentOS & SUSE)
2929
* possibly source code installations
3030

3131
Functionalities

‎appendix/migrate-to-postgresql.rst

+172
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
Migrate to PostgreSQL server
2+
****************************
3+
4+
Support for MySQL/MariaDB will be dropped in Zammad 7.0 upwards. Make sure to
5+
migrate your existing instance of Zammad to PostgreSQL before that update.
6+
7+
The following guide will provide you with a rough direction through that
8+
migration process.
9+
10+
.. warning:: **Proof of concept ahead**
11+
12+
As the technical details may differ from system to system, this guide
13+
comes without any warranty. Please proceed at your own risk. In doubt
14+
please refer to the documentation of the tools used.
15+
16+
17+
Preparation
18+
===========
19+
20+
#. Stop Zammad:
21+
22+
.. code-block:: sh
23+
24+
$ systemctl stop zammad
25+
26+
#. Create a backup of your instance.
27+
28+
29+
Install PostgreSQL
30+
------------------
31+
32+
.. include:: /install/includes/postgres-installation.rst
33+
34+
Please also have a look at :doc:`/appendix/configure-database-server`.
35+
36+
.. tabs::
37+
38+
.. tab:: Package installation
39+
40+
Nothing to do, continue with the next step. 🎉
41+
42+
.. tab:: Source code installations
43+
44+
.. include:: /install/includes/postgres-dependencies.rst
45+
46+
Database Credentials
47+
--------------------
48+
49+
Get the credentials Zammad is currently using to access your MySQL/MariaDB
50+
database from ``config/database.yml``. You will need them later.
51+
52+
Now adjust the configuration file to fill in the credentials for your new
53+
PostgreSQL server. Use ``postgresql`` as ``adapter``.
54+
55+
56+
.. include:: /install/includes/postgres-permissions.rst
57+
58+
59+
Create empty database
60+
---------------------
61+
62+
Now you need to create an empty database in PostgreSQL.
63+
64+
.. tabs::
65+
66+
.. tab:: Source installation
67+
68+
.. code-block:: sh
69+
70+
$ su - zammad
71+
$ rake db:create
72+
73+
.. tab:: Package installation
74+
75+
.. code-block:: sh
76+
77+
$ zammad run rake db:create
78+
79+
80+
Install pgloader and migrate
81+
============================
82+
83+
.. tabs::
84+
85+
.. tab:: Ubuntu / Debian
86+
87+
.. code-block:: sh
88+
89+
$ apt update
90+
$ apt install pgloader
91+
92+
.. tab:: CentOS
93+
94+
.. code-block:: sh
95+
96+
$ yum install -y https://download.postgresql.org/pub/repos/yum/common/redhat/rhel-7-x86_64/pgloader-3.6.2-1.rhel7.x86_64.rpm
97+
98+
.. tab:: OpenSUSE / SLES
99+
100+
.. code-block:: sh
101+
102+
$ zypper refresh
103+
$ zypper install pgloader
104+
105+
106+
107+
Create command file
108+
-------------------
109+
110+
Create a command file for pgloader for example in ``/tmp/pgloader-command``:
111+
112+
.. code-block:: cfg
113+
114+
115+
LOAD DATABASE
116+
FROM mysql://zammad:mysql_password@localhost/zammad
117+
INTO pgsql://zammad:pgsql_password@localhost/zammad
118+
119+
ALTER SCHEMA 'zammad' RENAME TO 'public'
120+
121+
WITH BATCH CONCURRENCY = 1;
122+
123+
If your database names and/or database usernames are different from ``zammad``
124+
adjust accordingly. And don't forget to replace ``mysql_password`` and
125+
``psql_password``.
126+
127+
128+
Migrate
129+
-------
130+
131+
.. tabs::
132+
133+
.. tab:: Dry run
134+
135+
You can check your configuration by running pgloader in a dry run first:
136+
137+
.. code-block:: sh
138+
139+
$ pgloader --dry-run /tmp/pgloader-command
140+
141+
.. tab:: Actual run
142+
143+
Once you are ready and setup you can start the actual migration:
144+
145+
.. code-block:: sh
146+
147+
$ pgloader --verbose /tmp/pgloader-command
148+
149+
150+
Finishing
151+
=========
152+
153+
After the migration has completed, you'll better clear some cache files:
154+
155+
.. tabs::
156+
157+
.. tab:: Source installation
158+
159+
.. code-block:: sh
160+
161+
$ su - zammad
162+
$ rails r 'Rails.cache.clear'
163+
164+
# Run as root
165+
$ systemctl start zammad
166+
167+
.. tab:: Package installation
168+
169+
.. code-block:: sh
170+
171+
$ zammad run rails r 'Rails.cache.clear'
172+
$ systemctl start zammad

‎index.rst

+1
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,7 @@ The Zammad documentation consists of three parts:
9696
/appendix/backup-and-restore/index
9797
/appendix/configure-env-vars
9898
/appendix/configure-database-server
99+
/appendix/migrate-to-postgresql
99100
/appendix/privacy
100101
/appendix/troubleshooting/index
101102
/appendix/single-sign-on

‎install/includes/postgres-dependencies.rst

+36
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
Install PostgreSQL Dependencies
2+
.. tabs::
3+
4+
.. tab:: Ubuntu / Debian
5+
6+
.. code-block:: sh
7+
8+
$ apt install libpq-dev
9+
10+
.. tab:: CentOS
11+
12+
.. code-block:: sh
13+
14+
# CentOS 7
15+
$ yum install postgresql14-libs postgresql14-devel
16+
17+
# CentOS 8
18+
$ yum install postgresql-libs postgresql-devel
19+
20+
.. tab:: OpenSuSE
21+
22+
.. code-block:: sh
23+
24+
$ zypper install postgresql-devel
25+
26+
Install Gems for Zammad
27+
.. code-block:: sh
28+
29+
$ su - zammad
30+
$ bundle config set without "test development mysql"
31+
$ bundle install
32+
33+
# CentOS 7 users - above command might fail, run the following
34+
# command and repeat above bundle install.
35+
# Adjust pg_config path according to your environment
36+
$ gem install pg -v '0.21.0' -- --with-pg-config=/usr/pgsql-14/bin/pg_config

‎install/includes/postgres-installation.rst

+36
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
.. tabs::
2+
3+
.. tab:: Ubuntu / Debian
4+
5+
.. code-block:: sh
6+
7+
$ apt update
8+
$ apt install postgresql postgresql-contrib
9+
$ systemctl start postgresql
10+
$ systemctl enable postgresql
11+
12+
.. tab:: CentOS
13+
14+
.. code-block:: sh
15+
16+
# CentOS 7
17+
$ yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
18+
$ yum install postgresql14-server postgresql14-contrib
19+
$ postgresql13-setup initdb
20+
$ systemctl start postgresql14
21+
$ systemctl enable postgresql14
22+
23+
# CentOS 8
24+
$ yum install postgresql-server postgresql-contrib
25+
$ postgresql-setup initdb
26+
$ systemctl start postgresql
27+
$ systemctl enable postgresql
28+
29+
.. tab:: OpenSUSE / SLES
30+
31+
.. code-block:: sh
32+
33+
$ zypper refresh
34+
$ zypper install postgresql postgresql-server postgresql-contrib
35+
$ systemctl start postgresql
36+
$ systemctl enable postgresql

‎install/includes/postgres-permissions.rst

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
.. tip::
2+
3+
**🤓 For easiest usage ...**
4+
5+
If you provide your Zammad user with database creation permission, you can
6+
run ``db:create`` in the following section. If you don't want that, you'll
7+
have to create the database manually.

‎install/package.rst

-7
Original file line numberDiff line numberDiff line change
@@ -39,13 +39,6 @@ some operating systems may require additional packages if not already installed.
3939
Add Repository and install Zammad
4040
=================================
4141

42-
.. hint::
43-
44-
If you want to use MySQL instead of PostgreSQL, it's usually enough to have
45-
the MySQL server installed on your system already. Some installation
46-
managers can't differentiate and still install Zammad with PostgreSQL. In
47-
that case, you'll have to adapt manually (out of scope of this documentation).
48-
4942
Add Repository
5043
.. tabs::
5144

‎install/source.rst

+21-128
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,8 @@ Zammad requires specific ruby versions. Adapt the commands below if you install
7777
older versions. A list of required versions can be found on the
7878
:doc:`Software requirements </prerequisites/software>` page.
7979

80+
.. include:: /install/includes/postgres-installation.rst
81+
8082
.. tabs::
8183

8284
.. tab:: Ubuntu
@@ -145,7 +147,7 @@ older versions. A list of required versions can be found on the
145147
Install Ruby Environment
146148
.. include:: source/include-rvm-install-ruby.rst
147149

148-
.. tab:: OpenSuSE
150+
.. tab:: OpenSUSE
149151

150152
Install Node.js
151153
.. include:: /install/includes/nodejs/suse.rst
@@ -179,104 +181,12 @@ older versions. A list of required versions can be found on the
179181
| After installing bundler, rake and rails we'll need to install all required gems.
180182
| The command depends on the database server you are using.
181183
182-
.. tabs::
183-
184-
.. tab:: PostgreSQL (recommended)
185-
186-
Install PostgreSQL Dependencies
187-
.. tabs::
188-
189-
.. tab:: Ubuntu / Debian
190-
191-
.. code-block:: sh
192-
193-
$ apt install libpq-dev
194-
195-
.. tab:: CentOS
196-
197-
.. code-block:: sh
198-
199-
# CentOS 7
200-
$ yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
201-
$ yum install postgresql13-libs postgresql13-devel
202-
203-
# CentOS 8
204-
$ yum install postgresql-libs postgresql-devel
205-
206-
.. tab:: OpenSuSE
207-
208-
.. code-block:: sh
209-
210-
$ zypper install postgresql-devel
211-
212-
Install Gems for Zammad
213-
.. code-block:: sh
214-
215-
$ su - zammad
216-
$ bundle config set without "test development mysql"
217-
$ bundle install
218-
219-
# CentOS 7 users - above command might fail, run the following
220-
# command and repeat above bundle install.
221-
# Adjust pg_config path according to your environment
222-
$ gem install pg -v '0.21.0' -- --with-pg-config=/usr/pgsql-13/bin/pg_config
223-
224-
.. tab:: MySQL / MariaDB
225-
226-
Install MySQL/MariaDB Dependencies
227-
.. tabs::
228-
229-
.. tab:: Ubuntu / Debian
230-
231-
.. code-block:: sh
232-
233-
$ apt install libmariadb-dev
234-
235-
.. tab:: CentOS
236-
237-
.. code-block:: sh
238-
239-
$ yum install mariadb-devel
240-
241-
.. tab:: OpenSuSE
242-
243-
.. code-block:: sh
244-
245-
$ zypper install libmariadb-devel
246-
247-
Create database
248-
While that's basically an easy step, you may want to create your
249-
database as follows to minimize potential issues.
250-
251-
Below commands need adjustments to fit your environment.
252-
Choose a safe password.
253-
254-
.. code-block:: sh
255-
256-
$ mysql
257-
> create user 'zammad'@'localhost' identified by 'changeme';
258-
> CREATE DATABASE zammad DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
259-
> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, INDEX, DROP, ALTER, CREATE TEMPORARY TABLES,\
260-
LOCK TABLES ON zammad.* TO 'zammad'@'localhost';
261-
262-
263-
Install Gems for Zammad
264-
.. code-block:: sh
265-
266-
$ su - zammad
267-
$ bundle set config without "test development postgres"
268-
$ bundle install
184+
.. include:: /install/includes/postgres-dependencies.rst
269185

270186
Step 3: Configure database settings
271187
-----------------------------------
272188

273-
.. tip::
274-
275-
**🤓 For easiest usage ...**
276-
277-
If you provide your Zammad user with database creation permission, you can
278-
run the step 4 without adjustment. If you don't want that, you'll have to
279-
create the database manually.
189+
.. include:: /install/includes/postgres-permissions.rst
280190

281191
.. code-block:: sh
282192
@@ -287,45 +197,28 @@ Here's a sample configuration to give you an idea on how your configuration
287197
file could look like. Please also have a look at
288198
:doc:`/appendix/configure-database-server` for deeper details.
289199

290-
.. tabs::
291-
292-
.. tab:: PostgreSQL
293-
294-
.. code-block::
295-
296-
production:
297-
adapter: postgresql
298-
database: zammad
299-
pool: 50
300-
timeout: 5000
301-
encoding: utf8
302-
username: zammad
303-
password: changeme
304-
305-
.. hint::
306-
307-
You can remove the ``password`` line if you enable socket based
308-
authentication!
200+
.. code-block:: yaml
309201
310-
.. tab:: MySQL / MariaDB
202+
production:
203+
adapter: postgresql
204+
database: zammad
205+
pool: 50
206+
timeout: 5000
207+
encoding: utf8
208+
username: zammad
209+
password: changeme
311210
312-
.. code-block::
211+
.. hint::
313212

314-
production:
315-
adapter: mysql2
316-
database: zammad
317-
pool: 50
318-
timeout: 5000
319-
encoding: utf8
320-
username: zammad
321-
password: changeme
213+
You can remove the ``password`` line if you enable socket based
214+
authentication!
322215

323-
.. hint::
216+
.. hint::
324217

325-
If you want to use an existing database server that's not on the same
326-
machine, you can also use ``host`` and ``port`` to set that up.
218+
If you want to use an existing database server that's not on the same
219+
machine, you can also use ``host`` and ``port`` to set that up.
327220

328-
.. include:: /install/source/include-chmod-database-yml.rst
221+
.. include:: /install/source/include-chmod-database-yml.rst
329222

330223
Step 4: Initialize your database
331224
--------------------------------

‎install/source/macos.rst

-1
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,6 @@ Install Zammad
3535
3636
$ cd zammad-latest
3737
$ bundle install
38-
$ sudo ln -s /usr/local/mysql/lib/libmysqlclient.18.dylib /usr/lib/libmysqlclient.18.dylib # if needed!
3938
$ rake db:create
4039
$ rake db:migrate
4140
$ rake db:seed

‎install/update.rst

+6
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,12 @@ Updating Zammad
141141
142142
$ bundle install --without test development postgres
143143
144+
.. danger::
145+
146+
.. include:: /tmp/mysql-deprication-note.rst
147+
148+
.. include:: /tmp/mysql-deprication-link.rst
149+
144150
Step 5: Stop Zammad services
145151
Stop the application server, websocket server and scheduler.
146152

‎prerequisites/software.rst

+4-6
Original file line numberDiff line numberDiff line change
@@ -165,16 +165,14 @@ installed with the Zammad-Package.
165165
Zammad will store all content in a Database.
166166
You can choose between the following database servers:
167167

168-
* MySQL 5.7+
169-
* MariaDB 10.3+
170168
* PostgreSQL 9.3+
169+
* MySQL 5.7+ / MariaDB 10.3+ (⚠️ deprecated with Zammad 7.0+)
171170

172-
.. note::
171+
.. danger::
173172

174-
We tend to recommend PostgreSQL. For the last 10 years we had the best
175-
experience with it.
173+
.. include:: /tmp/mysql-deprication-note.rst
176174

177-
**Zammad requires UTF-8 for its database.**
175+
.. include:: /tmp/mysql-deprication-link.rst
178176

179177
.. warning:: **Required configuration for MySQL/MariaDB:**
180178

‎tmp/mysql-deprication-link.rst

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Please have a look at our guide on how to migrate an existing MySQL/MariaDB
2+
instance to PostgreSQL: :doc:`/appendix/migrate-to-postgresql`.

‎tmp/mysql-deprication-note.rst

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Support for MySQL/MariaDB will be dopped in Zammad 7.0 upwards. Make sure to
2+
migrate your existing instance of Zammad to PostgreSQL before that update.

0 commit comments

Comments
 (0)
Please sign in to comment.