Timescaledb restarts multiple time just after startup of the docker

[daniel1302]

system info

Linux Ubuntu 22.04
AMD64

docker version
Client: Docker Engine - Community
 Cloud integration: v1.0.35-desktop+001
 Version:           24.0.5
 API version:       1.43
 Go version:        go1.20.6
 Git commit:        ced0996
 Built:             Fri Jul 21 20:35:45 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Desktop
 Engine:
  Version:          24.0.5
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.6
  Git commit:       a61e2b4
  Built:            Fri Jul 21 20:35:45 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

I have the following Dockerfile:

FROM timescale/timescaledb:2.8.1-pg14

ENV TIMESCALEDB_TELEMETRY="off"
ENV NO_TS_TUNE="true"


RUN set -euo; \
    sed -i 's/^restart_after_crash.*//' /usr/local/share/postgresql/postgresql.conf.sample\
        && echo "restart_after_crash=no" >> /usr/local/share/postgresql/postgresql.conf.sample;

COPY 020_add_multiple_databases.sh /docker-entrypoint-initdb.d/

My 020_add_multiple_databases.sh script looks like following:

!/bin/sh

IFS=","
for v in $POSTGRES_DBS
do
   PGPASSWORD=${POSTGRES_PASSWORD} psql -U ${POSTGRES_USER} -c "CREATE DATABASE $v" -c "GRANT ALL PRIVILEGES ON DATABASE $v TO ${POSTGRES_USER}"
done

Then When I call the following:

docker build -t timescaledb:2.8.0-pg14-patch2 . && docker run -e POSTGRES_USER=postgres -e POSTGRES_DBS=db1,db2,db3,db4 -e POSTGRES_PASSWORD=somepassword timescaledb:2.8.0-pg14-patch2

I am getting the following output:

[+] Building 0.6s (8/8) FINISHED                                                                                           docker:default
 => [internal] load .dockerignore                                                                                                    0.0s
 => => transferring context: 2B                                                                                                      0.0s
 => [internal] load build definition from Dockerfile                                                                                 0.0s
 => => transferring dockerfile: 464B                                                                                                 0.0s
 => [internal] load metadata for docker.io/timescale/timescaledb:2.8.1-pg14                                                          0.5s
 => [1/3] FROM docker.io/timescale/timescaledb:2.8.1-pg14@sha256:96c3a5587a24bb336437403b394f5927f639322cf3fcc0eff4ee0294da540dd7    0.0s
 => [internal] load build context                                                                                                    0.0s
 => => transferring context: 51B                                                                                                     0.0s
 => CACHED [2/3] RUN set -euo;     sed -i 's/^restart_after_crash.*//' /usr/local/share/postgresql/postgresql.conf.sample        &&  0.0s
 => CACHED [3/3] COPY 020_add_multiple_databases.sh /docker-entrypoint-initdb.d/                                                     0.0s
 => exporting to image                                                                                                               0.0s
 => => exporting layers                                                                                                              0.0s
 => => writing image sha256:9b10c3bf32f7594528cc1f9d5b8048dac426fbb1dd5f50b12eecf75254b249c2                                         0.0s
 => => naming to docker.io/library/timescaledb:2.8.0-pg14-patch2                                                                     0.0s

What's Next?
  View summary of image vulnerabilities and recommendations → docker scout quickview
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... sh: locale: not found
2023-08-23 16:54:05.013 UTC [30] WARNING:  no usable system locales were found
ok
syncing data to disk ... ok


Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data -l logfile start

initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
waiting for server to start....2023-08-23 16:54:06.026 UTC [36] LOG:  starting PostgreSQL 14.5 on x86_64-pc-linux-musl, compiled by gcc (Alpine 11.2.1_git20220219) 11.2.1 20220219, 64-bit
2023-08-23 16:54:06.031 UTC [36] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-08-23 16:54:06.044 UTC [37] LOG:  database system was shut down at 2023-08-23 16:54:05 UTC
2023-08-23 16:54:06.050 UTC [36] LOG:  database system is ready to accept connections
2023-08-23 16:54:06.051 UTC [43] LOG:  TimescaleDB background worker launcher connected to shared catalogs
 done
server started

/usr/local/bin/docker-entrypoint.sh: sourcing /docker-entrypoint-initdb.d/000_install_timescaledb.sh
psql:/tmp/tmp.fPGKkL:1: WARNING:  
WELCOME TO
 _____ _                               _     ____________  
|_   _(_)                             | |    |  _  \ ___ \ 
  | |  _ _ __ ___   ___  ___  ___ __ _| | ___| | | | |_/ / 
  | | | |  _ ` _ \ / _ \/ __|/ __/ _` | |/ _ \ | | | ___ \ 
  | | | | | | | | |  __/\__ \ (_| (_| | |  __/ |/ /| |_/ /
  |_| |_|_| |_| |_|\___||___/\___\__,_|_|\___|___/ \____/
               Running version 2.8.1
For more information on TimescaleDB, please visit the following links:

 1. Getting started: https://docs.timescale.com/timescaledb/latest/getting-started
 2. API reference documentation: https://docs.timescale.com/api/latest
 3. How TimescaleDB is designed: https://docs.timescale.com/timescaledb/latest/overview/core-concepts

Note: TimescaleDB collects anonymous reports to better understand and assist our users.
For more information and how to disable, please see our docs https://docs.timescale.com/timescaledb/latest/how-to-guides/configuration/telemetry.

2023-08-23 16:54:06.472 UTC [53] WARNING:  
        WELCOME TO
         _____ _                               _     ____________  
        |_   _(_)                             | |    |  _  \ ___ \ 
          | |  _ _ __ ___   ___  ___  ___ __ _| | ___| | | | |_/ / 
          | | | |  _ ` _ \ / _ \/ __|/ __/ _` | |/ _ \ | | | ___ \ 
          | | | | | | | | |  __/\__ \ (_| (_| | |  __/ |/ /| |_/ /
          |_| |_|_| |_| |_|\___||___/\___\__,_|_|\___|___/ \____/
                       Running version 2.8.1
        For more information on TimescaleDB, please visit the following links:

         1. Getting started: https://docs.timescale.com/timescaledb/latest/getting-started
         2. API reference documentation: https://docs.timescale.com/api/latest
         3. How TimescaleDB is designed: https://docs.timescale.com/timescaledb/latest/overview/core-concepts

        Note: TimescaleDB collects anonymous reports to better understand and assist our users.
        For more information and how to disable, please see our docs https://docs.timescale.com/timescaledb/latest/how-to-guides/configuration/telemetry.

2023-08-23 16:54:06.472 UTC [53] CONTEXT:  PL/pgSQL function inline_code_block line 16 at RAISE
CREATE EXTENSION
                    alter_job                    
-------------------------------------------------
 (1,24:00:00,00:01:40,-1,01:00:00,f,,-infinity,)
(1 row)

2023-08-23 16:54:06.837 UTC [56] WARNING:  
        WELCOME TO
         _____ _                               _     ____________  
        |_   _(_)                             | |    |  _  \ ___ \ 
          | |  _ _ __ ___   ___  ___  ___ __ _| | ___| | | | |_/ / 
          | | | |  _ ` _ \ / _ \/ __|/ __/ _` | |/ _ \ | | | ___ \ 
          | | | | | | | | |  __/\__ \ (_| (_| | |  __/ |/ /| |_/ /
          |_| |_|_| |_| |_|\___||___/\___\__,_|_|\___|___/ \____/
                       Running version 2.8.1
        For more information on TimescaleDB, please visit the following links:

         1. Getting started: https://docs.timescale.com/timescaledb/latest/getting-started
         2. API reference documentation: https://docs.timescale.com/api/latest
         3. How TimescaleDB is designed: https://docs.timescale.com/timescaledb/latest/overview/core-concepts

        Note: TimescaleDB collects anonymous reports to better understand and assist our users.
        For more information and how to disable, please see our docs https://docs.timescale.com/timescaledb/latest/how-to-guides/configuration/telemetry.

2023-08-23 16:54:06.837 UTC [56] CONTEXT:  PL/pgSQL function inline_code_block line 16 at RAISE
psql:/tmp/tmp.fPGKkL:1: WARNING:  
WELCOME TO
 _____ _                               _     ____________  
|_   _(_)                             | |    |  _  \ ___ \ 
  | |  _ _ __ ___   ___  ___  ___ __ _| | ___| | | | |_/ / 
  | | | |  _ ` _ \ / _ \/ __|/ __/ _` | |/ _ \ | | | ___ \ 
  | | | | | | | | |  __/\__ \ (_| (_| | |  __/ |/ /| |_/ /
  |_| |_|_| |_| |_|\___||___/\___\__,_|_|\___|___/ \____/
               Running version 2.8.1
For more information on TimescaleDB, please visit the following links:

 1. Getting started: https://docs.timescale.com/timescaledb/latest/getting-started
 2. API reference documentation: https://docs.timescale.com/api/latest
 3. How TimescaleDB is designed: https://docs.timescale.com/timescaledb/latest/overview/core-concepts

Note: TimescaleDB collects anonymous reports to better understand and assist our users.
For more information and how to disable, please see our docs https://docs.timescale.com/timescaledb/latest/how-to-guides/configuration/telemetry.

2023-08-23 16:54:06.842 UTC [57] ERROR:  TimescaleDB background worker connected to template database, exiting
CREATE EXTENSION
2023-08-23 16:54:06.842 UTC [36] LOG:  background worker "TimescaleDB Background Worker Scheduler" (PID 57) exited with exit code 1
                    alter_job                    
-------------------------------------------------
 (1,24:00:00,00:01:40,-1,01:00:00,f,,-infinity,)
(1 row)


/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/001_timescaledb_tune.sh

/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/020_add_multiple_databases.sh
CREATE DATABASE
GRANT
CREATE DATABASE
GRANT
CREATE DATABASE
GRANT
CREATE DATABASE
GRANT

waiting for server to shut down...2023-08-23 16:54:08.445 UTC [36] LOG:  received fast shutdown request
.2023-08-23 16:54:08.450 UTC [36] LOG:  aborting any active transactions
2023-08-23 16:54:08.450 UTC [54] FATAL:  terminating background worker "TimescaleDB Background Worker Scheduler" due to administrator command
2023-08-23 16:54:08.450 UTC [43] FATAL:  terminating background worker "TimescaleDB Background Worker Launcher" due to administrator command
2023-08-23 16:54:08.451 UTC [36] LOG:  background worker "logical replication launcher" (PID 44) exited with exit code 1
2023-08-23 16:54:08.451 UTC [36] LOG:  background worker "TimescaleDB Background Worker Launcher" (PID 43) exited with exit code 1
2023-08-23 16:54:08.451 UTC [36] LOG:  background worker "TimescaleDB Background Worker Scheduler" (PID 54) exited with exit code 1
2023-08-23 16:54:08.451 UTC [38] LOG:  shutting down
2023-08-23 16:54:08.500 UTC [36] LOG:  database system is shut down
 done
server stopped

PostgreSQL init process complete; ready for start up.

2023-08-23 16:54:08.567 UTC [1] LOG:  starting PostgreSQL 14.5 on x86_64-pc-linux-musl, compiled by gcc (Alpine 11.2.1_git20220219) 11.2.1 20220219, 64-bit
2023-08-23 16:54:08.567 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2023-08-23 16:54:08.567 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2023-08-23 16:54:08.580 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-08-23 16:54:08.590 UTC [70] LOG:  database system was shut down at 2023-08-23 16:54:08 UTC
2023-08-23 16:54:08.597 UTC [1] LOG:  database system is ready to accept connections
2023-08-23 16:54:08.598 UTC [76] LOG:  TimescaleDB background worker launcher connected to shared catalogs

My question is why TimescaleDB restarts so many times, and how I could debug it?
Some times it takes seconds and sometimes about 1 minute
It's critical part because I am starting database during the tests, so i want this step to be faster.

[erimatnor]

@daniel1302 Does this problem occur with the base image as well? Unfortunately, we cannot help with custom Dockerfiles and startup scripts. I suggest you try removing your changes one-by-one until it works. That's one way to identify the issue.

The issue will be closed. Feel free to reopen if you find a specific issue/bug in our Docker image.