CouchDb issue starting Budibase with docker-compose #4269

cmthomps · 2022-01-31T13:57:15Z

cmthomps
Jan 31, 2022

Trying to follow the getting started instructions for self hosting with docker-compose. When I try to run:

docker-compose --env-file hosting.properties up

I get an error in the log:

proxy-service_1       | [2022-01-31 13:53:53.494][8][info][main] [source/server/server.cc:328]   response trailer map: 152 bytes: grpc-message,grpc-status
proxy-service_1       | [2022-01-31 13:53:53.496][8][warning][main] [source/server/server.cc:454] No admin address given, so no admin HTTP server started.
proxy-service_1       | [2022-01-31 13:53:53.496][8][info][main] [source/server/server.cc:583] runtime: layers:
proxy-service_1       |   - name: base
proxy-service_1       |     static_layer:
proxy-service_1       |       {}
proxy-service_1       |   - name: admin
proxy-service_1       |     admin_layer:
proxy-service_1       |       {}
proxy-service_1       | [2022-01-31 13:53:53.496][8][info][config] [source/server/configuration_impl.cc:95] loading tracing configuration
proxy-service_1       | [2022-01-31 13:53:53.496][8][info][config] [source/server/configuration_impl.cc:70] loading 0 static secret(s)
proxy-service_1       | [2022-01-31 13:53:53.496][8][info][config] [source/server/configuration_impl.cc:76] loading 5 cluster(s)
proxy-service_1       | [2022-01-31 13:53:53.500][8][info][config] [source/server/configuration_impl.cc:80] loading 1 listener(s)
proxy-service_1       | [2022-01-31 13:53:53.504][8][info][config] [source/server/configuration_impl.cc:121] loading stats sink configuration
proxy-service_1       | [2022-01-31 13:53:53.505][8][info][main] [source/server/server.cc:679] starting main dispatch loop
proxy-service_1       | [2022-01-31 13:53:53.506][8][info][runtime] [source/common/runtime/runtime_impl.cc:421] RTDS has finished initialization
proxy-service_1       | [2022-01-31 13:53:53.506][8][info][upstream] [source/common/upstream/cluster_manager_impl.cc:178] cm init: all clusters initialized
proxy-service_1       | [2022-01-31 13:53:53.506][8][info][main] [source/server/server.cc:660] all clusters initialized. initializing init manager
proxy-service_1       | [2022-01-31 13:53:53.506][8][info][config] [source/server/listener_manager_impl.cc:888] all dependencies initialized. starting workers
proxy-service_1       | [2022-01-31 13:53:53.507][8][warning][main] [source/server/server.cc:565] there is no configured limit to the number of allowed active connections. Set a limit via the runtime key overload.global_downstream_max_connections
bbworker              | Worker running on {"address":"::","family":"IPv6","port":4003}
bbapps                | Budibase running on {"address":"::","family":"IPv6","port":4002}
couch-init_1          |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
couch-init_1          |                                  Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
couch-init_1          | curl: (7) Failed to connect to couchdb-service port 5984 after 1 ms: Connection refused
couch-init_1          | sh: fg: line 1: No current job
hello_couch-init_1 exited with code 2

Answered by pandel

Oct 4, 2022

Hi @ALL!

Today, after upgrading Budibase from 2.0.6 to 2.0.12 I had the same problem. I pulled the proxy/app/worker images and after that the CouchDB container refused to start.

I did the following to recover:

docker-compose down
docker rmi ibmcom/couchdb3:latest
docker system prune
docker pull ibmcom/couchdb3:latest
docker-compose up -d

I retried the whole procedure a few times from a clean VM snapshot always with the same result and it seemed that pulling the new BB images somehow had a negative impact on the Couchdb image, which was already there. I clearly don't understand why, because Docker should have handled the images strictly separate, but who knows... The above mentioned step…

View full answer

pavleks · 2022-01-31T14:12:20Z

pavleks
Jan 31, 2022

@cmthomps I believe the issue Failed to connect to couchdb-service port 5984 after 1 ms: Connection refused
Perhaps you have some firewall rule on a host which somehow denies the traffic to port 5984?

Try iptables -I dockeruser -p tcp -m tcp --dport 5984 -j ACCEPT before docker-compose

0 replies

cmthomps · 2022-01-31T14:57:06Z

cmthomps
Jan 31, 2022
Author

Thanks - I think it's using the networking setup by docker compose, so I shouldn't need to open ports on my physical machine. Right?

1 reply

pavleks Jan 31, 2022

Ports mentioned in docker-compose.yml will be shared among services started by the docker-compos, but I have no idea what else could refuse connection to the port, so maybe something related to the host machine.

omichkun · 2022-02-01T14:43:00Z

omichkun
Feb 1, 2022

I have the same issue.
I've looked into container couchdb-service and seen there these lines:

budibase-couchdb-service-1     | No name found in ERL_FLAGS  -setcookie monster
budibase-couchdb-service-1     | No -name found in vm.args. Using [email protected]

I suppose (but i could be wrong) couchdb accepts connection only from 127.0.0.1 (in other words – from itself only) and rejects any connections from outside. But i don't know how to configure it correctly.

0 replies

shogunpurple · 2022-02-01T14:45:06Z

shogunpurple
Feb 1, 2022
Maintainer

@omichkun and @cmthomps -

Which operating system are you running on? Are you using OSX with a mac and M1 chip, by any chance?

1 reply

omichkun Feb 1, 2022

I am using MacOS with intel chip.

cmthomps · 2022-02-01T14:46:40Z

cmthomps
Feb 1, 2022
Author

I am using macos, but intel chip (2.3 GHz 8-Core Intel Core i9)

0 replies

shogunpurple · 2022-02-01T14:50:22Z

shogunpurple
Feb 1, 2022
Maintainer

Ok no problem - would you be able to post the result of a docker ps? If couch is actually up, can you access the platform on localhost:10000?

5 replies

omichkun Feb 1, 2022

➜  budibase docker-compose --env-file hosting.properties ps
NAME                            COMMAND                  SERVICE              STATUS               PORTS
bbapps                          "docker-entrypoint.s…"   app-service          running              0.0.0.0:4002->4002/tcp
bbworker                        "docker-entrypoint.s…"   worker-service       running              0.0.0.0:4003->4003/tcp
budibase-couch-init-1           "/entrypoint.sh sh -…"   couch-init           exited (2)
budibase-couchdb-service-1      "/docker-entrypoint.…"   couchdb-service      running              0.0.0.0:4005->5984/tcp
budibase-minio-service-1        "/usr/bin/docker-ent…"   minio-service        running (starting)   0.0.0.0:4004->9000/tcp
budibase-proxy-service-1        "/docker-entrypoint.…"   proxy-service        running              0.0.0.0:10000->10000/tcp
budibase-redis-service-1        "docker-entrypoint.s…"   redis-service        running              0.0.0.0:6379->6379/tcp
budibase-watchtower-service-1   "/watchtower --debug…"   watchtower-service   running              0.0.0.0:6161->8080/tcp

I can access localhost:10000 it redirects to http://localhost:10000/builder/admin and shows empty black page

cmthomps Feb 1, 2022
Author

I can hit localhost:10000 but get a black screen and a javascript error in the console. docker ps results below.

Uncaught (in promise) TypeError: c.adminUser is undefined
    qq http://localhost:10000/builder/assets/index.937b2767.js:445
    Qd http://localhost:10000/builder/assets/vendor.f9faf6ab.js:1
    I8 http://localhost:10000/builder/assets/vendor.f9faf6ab.js:4
    Ku http://localhost:10000/builder/assets/vendor.f9faf6ab.js:4
index.937b2767.js:445:8062

CONTAINER ID   IMAGE                                      COMMAND                  CREATED              STATUS                        PORTS                                        NAMES
0f6a9889b9b1   envoyproxy/envoy:v1.16-latest              "/docker-entrypoint.…"   About a minute ago   Up About a minute             0.0.0.0:10000->10000/tcp                     hello_proxy-service_1
57b80fdea88a   budibase.docker.scarf.sh/budibase/apps     "docker-entrypoint.s…"   About a minute ago   Up About a minute             4001/tcp, 0.0.0.0:4002->4002/tcp             bbapps
8bdd5df5db78   budibase.docker.scarf.sh/budibase/worker   "docker-entrypoint.s…"   About a minute ago   Up About a minute             4001/tcp, 0.0.0.0:4003->4003/tcp             bbworker
dd27e885250f   ibmcom/couchdb3                            "/docker-entrypoint.…"   About a minute ago   Up About a minute             4369/tcp, 9100/tcp, 0.0.0.0:4005->5984/tcp   hello_couchdb-service_1
7b7a8ba9fe71   redis                                      "docker-entrypoint.s…"   About a minute ago   Up About a minute             0.0.0.0:6379->6379/tcp                       hello_redis-service_1
5e1b74fe99d5   containrrr/watchtower                      "/watchtower --debug…"   About a minute ago   Up About a minute             0.0.0.0:6161->8080/tcp                       hello_watchtower-service_1
580def82087f   minio/minio                                "/usr/bin/docker-ent…"   About a minute ago   Up About a minute (healthy)   0.0.0.0:4004->9000/tcp                       hello_minio-service_1

shogunpurple Feb 1, 2022
Maintainer

Ok - thanks for the information. It seems like the couch-init is not waiting for couchdb to come up, hence the issue. Could you possibly try running a docker-compose --env-file hosting.properties restart couch-init and see if it runs successfully?

cmthomps Feb 1, 2022
Author

This is the result I get. Seems like something is blocking the internal docker networking to CouchDb (or isn't actually started).

couch-init_1          |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
couch-init_1          |                                  Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
couch-init_1          | curl: (7) Failed to connect to couchdb-service port 5984 after 2 ms: Connection refused
couch-init_1          | sh: fg: line 1: No current job
hello_couch-init_1 exited with code 2

mrb101 Feb 2, 2022

I've just had the same error but its actually working. by going to localhost:10000 .

cmthomps · 2022-02-01T19:13:12Z

cmthomps
Feb 1, 2022
Author

On additional piece of info. I got a terminal inside the couchdb container and tried to run "curl http://127.0.0.1:5984/". I receive "curl: (7) Failed to connect to 127.0.0.1 port 5984: Connection refused". Almost like couchdb isn't running?

0 replies

cmthomps · 2022-02-07T15:16:58Z

cmthomps
Feb 7, 2022
Author

Curious if anyone has had any luck with this?

4 replies

shogunpurple Feb 7, 2022
Maintainer

Sorry @cmthomps - missed your reply. I cannot seem to recreate this at all. I'm starting to wonder is it a problem with docker/docker-compose versions. What version of each are you on?

cmthomps Feb 7, 2022
Author

Thanks - So bizarre. I wonder if it's Mac thing since @omichkun was also on a mac.
docker: Docker version 20.10.10, build b485636
docker compose: Docker Compose version v2.1.1

omichkun Feb 8, 2022

Yeah, i have similar
➜ docker -v
Docker version 20.10.12, build e91ed57
➜ docker compose version
Docker Compose version v2.2.3

WuMaKu84 Feb 11, 2022

Exactly the same here, using Ubuntu 20.04.3:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal
$ curl http://127.0.0.1:5984/
curl: (7) Failed to connect to 127.0.0.1 port 5984: Connection refused

But everything seems to be started just fine:
$ sudo ./budi hosting --start
Starting services, this may take a moment - first time this may take a few minutes to download images.
Starting username_couchdb-service_1 ...
Starting username_redis-service_1 ...
Starting username_minio-service_1 ...
Starting username_watchtower-service_1 ...
Starting username_couchdb-service_1 ... done
Starting username_couch-init_1 ...
Starting username_minio-service_1 ... done
Starting username_redis-service_1 ... done
Starting username_watchtower-service_1 ... done
Starting username_couch-init_1 ... done
Starting bbworker ...
Starting bbworker ... done
Starting bbapps ...
Starting bbapps ... done
Starting bbproxy ...
Starting bbproxy ... done
Services started, please go to http://localhost:10000 for next steps.

Also tried both ways, using Docker only and CLI Installation...
Went ahead and even tried the installation on my Windows Computer using Docker (with disabled Firewall) -> also the same, no response on Port 5984

In the logs, no database file is found and therefore the database "users" won't get created:
temp-couchdb-service-1 | [info] 2022-02-11T21:13:41.885840Z [email protected] <0.255.0> -------- Apache CouchDB has started. Time to relax.
temp-couchdb-service-1 |
temp-couchdb-service-1 | [notice] 2022-02-11T21:13:41.897720Z [email protected] <0.310.0> -------- rexi_server : started servers
temp-couchdb-service-1 | [notice] 2022-02-11T21:13:41.900058Z [email protected] <0.314.0> -------- rexi_buffer : started servers
temp-couchdb-service-1 | [warning] 2022-02-11T21:13:41.932969Z [email protected] <0.322.0> -------- creating missing database: _nodes
temp-couchdb-service-1 | [info] 2022-02-11T21:13:41.933138Z [email protected] <0.323.0> -------- open_result error {not_found,no_db_file} for _nodes
temp-couchdb-service-1 | [warning] 2022-02-11T21:13:42.017601Z [email protected] <0.338.0> -------- creating missing database: _dbs
temp-couchdb-service-1 | [warning] 2022-02-11T21:13:42.017604Z [email protected] <0.337.0> -------- creating missing database: _dbs
temp-couchdb-service-1 | [info] 2022-02-11T21:13:42.017672Z [email protected] <0.340.0> -------- open_result error {not_found,no_db_file} for _dbs
temp-couchdb-service-1 | [notice] 2022-02-11T21:13:42.048839Z [email protected] <0.352.0> -------- mem3_reshard_dbdoc start init()
temp-couchdb-service-1 | [notice] 2022-02-11T21:13:42.056299Z [email protected] <0.354.0> -------- mem3_reshard start init()
temp-couchdb-service-1 | [notice] 2022-02-11T21:13:42.056449Z [email protected] <0.355.0> -------- mem3_reshard db monitor <0.355.0> starting
temp-couchdb-service-1 | [notice] 2022-02-11T21:13:42.059303Z [email protected] <0.354.0> -------- mem3_reshard starting reloading jobs
temp-couchdb-service-1 | [notice] 2022-02-11T21:13:42.059550Z [email protected] <0.354.0> -------- mem3_reshard finished reloading jobs
temp-couchdb-service-1 | [info] 2022-02-11T21:13:42.083029Z [email protected] <0.361.0> -------- Apache CouchDB has started. Time to relax.
temp-couchdb-service-1 |
temp-couchdb-service-1 | [info] 2022-02-11T21:13:42.083123Z [email protected] <0.361.0> -------- Apache CouchDB has started on http://any:5984/
temp-couchdb-service-1 | [notice] 2022-02-11T21:13:42.090936Z [email protected] <0.384.0> -------- chttpd_auth_cache changes listener died because the _users database does not exist. Create the database to silence this notice.
temp-couchdb-service-1 | [error] 2022-02-11T21:13:42.091104Z [email protected] emulator -------- Error in process <0.385.0> on node '[email protected]' with exit value:
temp-couchdb-service-1 | {database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,400}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,375}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,404}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,97}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,39}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,198}]},{chttpd_auth_cache,listen_for_changes,1,[{file,"src/chttpd_auth_cache.erl"},{line,145}]}]}

What went wrong? (just used everything default as for testing budibase)

cmthomps · 2022-02-17T20:18:33Z

cmthomps
Feb 17, 2022
Author

I was able get budibase working by removing all associated docker volumes, images, containers, etc.... Not sure if they released a new version of couchdb that fixed the issue.

0 replies

WuMaKu84 · 2022-02-18T15:11:59Z

WuMaKu84
Feb 18, 2022

Hi there,
Still no luck on my side... the init_database container is meant to close after DB initialization, correct?

Still looks (more or less) the same: problem loading page

Creating username_minio-service_1      ... done
Creating username_couchdb-service_1    ... done
Creating username_redis-service_1      ... done
Creating username_watchtower-service_1 ... done
Creating username_couch-init_1         ... done
Creating bbworker                       ... done
Creating bbapps                         ... done
Creating bbproxy                        ... done
Attaching to username_minio-service_1, username_redis-service_1, username_couchdb-service_1, username_watchtower-service_1, username_couch-init_1, bbworker, bbapps, bbproxy
bbapps                | yarn run v1.22.17
bbproxy               | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
bbproxy               | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
bbapps                | $ node dist/index.js
bbproxy               | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
bbworker              | yarn run v1.22.17
bbworker              | $ node dist/index.js
couchdb-service_1     | No name found in ERL_FLAGS  -setcookie monster
redis-service_1       | 1:C 18 Feb 2022 14:45:44.096 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis-service_1       | 1:C 18 Feb 2022 14:45:44.096 # Redis version=6.2.6, bits=64, commit=00000000, modified=0, pid=1, just started
redis-service_1       | 1:C 18 Feb 2022 14:45:44.096 # Configuration loaded
couchdb-service_1     | No -name found in vm.args. Using [email protected]
redis-service_1       | 1:M 18 Feb 2022 14:45:44.098 * monotonic clock: POSIX clock_gettime
watchtower-service_1  | time="2022-02-18T14:45:44Z" level=debug
watchtower-service_1  | time="2022-02-18T14:45:44Z" level=debug msg="Sleeping for a second to ensure the docker api client has been properly initialized."
redis-service_1       | 1:M 18 Feb 2022 14:45:44.099 * Running mode=standalone, port=6379.
redis-service_1       | 1:M 18 Feb 2022 14:45:44.099 # Server initialized
redis-service_1       | 1:M 18 Feb 2022 14:45:44.099 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
bbproxy               | 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
couchdb-service_1     | [info] 2022-02-18T14:45:46.241118Z [email protected] <0.255.0> -------- Preflight check: Asserting Admin Account
couchdb-service_1     |
redis-service_1       | 1:M 18 Feb 2022 14:45:44.105 * Loading RDB produced by version 6.2.6
redis-service_1       | 1:M 18 Feb 2022 14:45:44.105 * RDB age 323 seconds
redis-service_1       | 1:M 18 Feb 2022 14:45:44.105 * RDB memory usage when created 0.78 Mb
redis-service_1       | 1:M 18 Feb 2022 14:45:44.105 # Done loading RDB, keys loaded: 0, keys expired: 1.
redis-service_1       | 1:M 18 Feb 2022 14:45:44.105 * DB loaded from disk: 0.005 seconds
redis-service_1       | 1:M 18 Feb 2022 14:45:44.105 * Ready to accept connections
couchdb-service_1     | [info] 2022-02-18T14:45:46.247977Z [email protected] <0.255.0> -------- Apache CouchDB 3.2.1 is starting.
couchdb-service_1     |
couchdb-service_1     | [info] 2022-02-18T14:45:46.248138Z [email protected] <0.256.0> -------- Starting couch_sup
watchtower-service_1  | time="2022-02-18T14:45:45Z" level=debug msg="Making sure everything is sane before starting"
watchtower-service_1  | time="2022-02-18T14:45:45Z" level=debug msg="Retrieving running containers"
couchdb-service_1     | [notice] 2022-02-18T14:45:46.679804Z [email protected] <0.110.0> -------- config: [admins] BgdITnfW40W6WgK4ZWXJQo2mNTfbhJnq set to '****' for reason nil
couchdb-service_1     | [info] 2022-02-18T14:45:46.781650Z [email protected] <0.255.0> -------- Apache CouchDB has started. Time to relax.
couchdb-service_1     |
watchtower-service_1  | time="2022-02-18T14:45:45Z" level=debug msg="There are no additional watchtower containers"
watchtower-service_1  | time="2022-02-18T14:45:45Z" level=info msg="Watchtower 1.4.0"
watchtower-service_1  | time="2022-02-18T14:45:45Z" level=info msg="Using no notifications"
watchtower-service_1  | time="2022-02-18T14:45:45Z" level=info msg="Only checking containers with name \"bbapps\" or \"bbworker\" or \"bbproxy\""
watchtower-service_1  | time="2022-02-18T14:45:45Z" level=info msg="Periodic runs are not enabled."
watchtower-service_1  | time="2022-02-18T14:45:45Z" level=info msg="The HTTP API is enabled at :8080."
couchdb-service_1     | [notice] 2022-02-18T14:45:46.799483Z [email protected] <0.310.0> -------- rexi_server : started servers
couchdb-service_1     | [notice] 2022-02-18T14:45:46.803606Z [email protected] <0.314.0> -------- rexi_buffer : started servers
couchdb-service_1     | [notice] 2022-02-18T14:45:47.104757Z [email protected] <0.344.0> -------- mem3_reshard_dbdoc start init()
couchdb-service_1     | [notice] 2022-02-18T14:45:47.116937Z [email protected] <0.346.0> -------- mem3_reshard start init()
couchdb-service_1     | [notice] 2022-02-18T14:45:47.117281Z [email protected] <0.347.0> -------- mem3_reshard db monitor <0.347.0> starting
couchdb-service_1     | [notice] 2022-02-18T14:45:47.124652Z [email protected] <0.346.0> -------- mem3_reshard starting reloading jobs
couchdb-service_1     | [notice] 2022-02-18T14:45:47.124988Z [email protected] <0.346.0> -------- mem3_reshard finished reloading jobs
couchdb-service_1     | [info] 2022-02-18T14:45:47.181555Z [email protected] <0.353.0> -------- Apache CouchDB has started. Time to relax.
couchdb-service_1     |
couchdb-service_1     | [info] 2022-02-18T14:45:47.181711Z [email protected] <0.353.0> -------- Apache CouchDB has started on http://any:5984/
couchdb-service_1     | [main] INFO clouseau.main - Clouseau running as [email protected]
couchdb-service_1     | [notice] 2022-02-18T14:45:47.398794Z [email protected] <0.440.0> -------- All system databases exist.
bbproxy               | 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
minio-service_1       | WARNING: MINIO_ACCESS_KEY and MINIO_SECRET_KEY are deprecated.
minio-service_1       |          Please use MINIO_ROOT_USER and MINIO_ROOT_PASSWORD
bbproxy               | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
bbproxy               | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
bbproxy               | /docker-entrypoint.sh: Configuration complete; ready for start up
bbproxy               | 2022/02/18 14:45:47 [notice] 1#1: using the "epoll" event method
bbproxy               | 2022/02/18 14:45:47 [notice] 1#1: nginx/1.21.6
bbproxy               | 2022/02/18 14:45:47 [notice] 1#1: built by gcc 10.2.1 20210110 (Debian 10.2.1-6)
bbproxy               | 2022/02/18 14:45:47 [notice] 1#1: OS: Linux 5.4.0-99-generic
bbproxy               | 2022/02/18 14:45:47 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
bbproxy               | 2022/02/18 14:45:47 [notice] 1#1: start worker processes
bbproxy               | 2022/02/18 14:45:47 [notice] 1#1: start worker process 36
bbproxy               | 2022/02/18 14:45:47 [notice] 1#1: start worker process 37
bbproxy               | 2022/02/18 14:45:47 [notice] 1#1: start worker process 38
bbproxy               | 2022/02/18 14:45:47 [notice] 1#1: start worker process 39
bbproxy               | 2022/02/18 14:45:47 [notice] 1#1: start worker process 40
bbproxy               | 2022/02/18 14:45:47 [notice] 1#1: start worker process 41
bbproxy               | 2022/02/18 14:45:47 [notice] 1#1: start worker process 42
bbproxy               | 2022/02/18 14:45:47 [notice] 1#1: start worker process 43
minio-service_1       | API: http://172.18.0.2:9000  http://127.0.0.1:9000
minio-service_1       |
minio-service_1       | Documentation: https://docs.min.io
bbworker              | Worker running on {"address":"::","family":"IPv6","port":4003}
couchdb-service_1     | [notice] 2022-02-18T14:45:52.235598Z [email protected] <0.400.0> -------- couch_replicator_clustering : cluster stable
couchdb-service_1     | [notice] 2022-02-18T14:45:52.241986Z [email protected] <0.415.0> -------- Started replicator db changes listener <0.545.0>
couchdb-service_1     | [info] 2022-02-18T14:45:52.242801Z [email protected] <0.547.0> -------- open_result error {not_found,no_db_file} for _replicator
bbapps                | Running migrations
bbapps                | Budibase running on {"address":"::","family":"IPv6","port":4002}
couchdb-service_1     | [notice] 2022-02-18T14:45:53.164963Z [email protected] <0.529.0> 194b59d59f couchdb-service:5984 172.18.0.8 BgdITnfW40W6WgK4ZWXJQo2mNTfbhJnq GET /global-db/ 200 ok 205
couchdb-service_1     | [notice] 2022-02-18T14:45:53.164958Z [email protected] <0.530.0> 058f8eaa6a couchdb-service:5984 172.18.0.8 BgdITnfW40W6WgK4ZWXJQo2mNTfbhJnq GET /pouch__all_dbs__/ 200 ok 189
couchdb-service_1     | [notice] 2022-02-18T14:45:53.202316Z [email protected] <0.563.0> dff20faa3f couchdb-service:5984 172.18.0.8 BgdITnfW40W6WgK4ZWXJQo2mNTfbhJnq GET /pouch__all_dbs__/db_global-db? 200 ok 3
couchdb-service_1     | [notice] 2022-02-18T14:45:53.202328Z [email protected] <0.562.0> 1d4e402948 couchdb-service:5984 172.18.0.8 BgdITnfW40W6WgK4ZWXJQo2mNTfbhJnq GET /global-db/migrations? 200 ok 4
couchdb-service_1     | [notice] 2022-02-18T14:45:53.227120Z [email protected] <0.594.0> 5ccdefcb6c couchdb-service:5984 172.18.0.8 BgdITnfW40W6WgK4ZWXJQo2mNTfbhJnq GET /pouch__all_dbs__/db_global-db? 200 ok 2
couchdb-service_1     | [notice] 2022-02-18T14:45:53.227150Z [email protected] <0.593.0> d1b552109d couchdb-service:5984 172.18.0.8 BgdITnfW40W6WgK4ZWXJQo2mNTfbhJnq GET /global-db/ 200 ok 2
couchdb-service_1     | [notice] 2022-02-18T14:45:53.245477Z [email protected] <0.595.0> f9b26a15c4 couchdb-service:5984 172.18.0.8 BgdITnfW40W6WgK4ZWXJQo2mNTfbhJnq GET /global-db/migrations? 200 ok 3
couchdb-service_1     | [notice] 2022-02-18T14:45:53.320054Z [email protected] <0.596.0> 00e19f1939 couchdb-service:5984 172.18.0.8 BgdITnfW40W6WgK4ZWXJQo2mNTfbhJnq GET /_all_dbs 200 ok 65
bbapps                | Migrations complete
couch-init_1          |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
couch-init_1          |                                  Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0{"error":"file_exists","reason":"The database could not be created, the file already exists."}
couchdb-service_1     | [notice] 2022-02-18T14:45:55.454260Z [email protected] <0.651.0> 1308831880 couchdb-service:5984 172.18.0.6 BgdITnfW40W6WgK4ZWXJQo2mNTfbhJnq PUT /_users 412 ok 23
100    95  100    95    0     0   3507      0 --:--:-- --:--:-- --:--:--  3653
couch-init_1          |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
couch-init_1          |                                  Dload  Upload   Total   Spent    Left  Speed
couchdb-service_1     | [notice] 2022-02-18T14:45:55.469342Z [email protected] <0.652.0> 497d858d5b couchdb-service:5984 172.18.0.6 BgdITnfW40W6WgK4ZWXJQo2mNTfbhJnq PUT /_replicator 412 ok 2
100    95  100    95    0     0  13062      0 --:--:-- --:--:-- --:--:-- 15833
couch-init_1          | {"error":"file_exists","reason":"The database could not be created, the file already exists."}
couch-init_1          | sh: fg: line 1: No current job
username_couch-init_1 exited with code 2
couchdb-service_1     | [notice] 2022-02-18T14:46:01.800484Z [email protected] <0.310.0> -------- rexi_server : cluster stable
couchdb-service_1     | [notice] 2022-02-18T14:46:01.800603Z [email protected] <0.314.0> -------- rexi_buffer : cluster stable
couchdb-service_1     | [notice] 2022-02-18T15:04:54.158682Z [email protected] <0.9917.0> 30799c2112 couchdb-service:5984 172.18.0.3 BgdITnfW40W6WgK4ZWXJQo2mNTfbhJnq DELETE /_users 200 ok 136
couchdb-service_1     | [actor:8] INFO clouseau.cleanup - Removing shards/80000000-ffffffff/_users.1645191437
couchdb-service_1     | [actor:8] INFO clouseau.cleanup - Removing shards/00000000-7fffffff/_users.1645191437
couchdb-service_1     | [notice] 2022-02-18T15:04:59.175007Z [email protected] <0.376.0> -------- chttpd_auth_cache changes listener died because the _users database does not exist. Create the database to silence this notice.

I even deleted the databases and re-initialized (by running the container again) the database with no luck:

couchdb-service_1     | [error] 2022-02-18T15:05:34.194553Z [email protected] emulator -------- Error in process <0.10294.0> on node '[email protected]' with exit value:
couchdb-service_1     | {database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,400}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,375}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,404}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,97}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,39}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,198}]},{chttpd_auth_cache,listen_for_changes,1,[{file,"src/chttpd_auth_cache.erl"},{line,145}]}]}
couchdb-service_1     |
couch-init_1          |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
couch-init_1          |                                  Dload  Upload   Total   Spent    Left  Speed
couchdb-service_1     | [error] 2022-02-18T15:05:37.159477Z [email protected] <0.10295.0> 9c0609ffc0 Request to create N=3 DB but only 1 node(s)
couchdb-service_1     | [actor:10] INFO clouseau.cleanup - Removing shards/00000000-7fffffff/_users.1645196737
couchdb-service_1     | [actor:10] INFO clouseau.cleanup - Removing shards/80000000-ffffffff/_users.1645196737
couchdb-service_1     | [notice] 2022-02-18T15:05:37.174549Z [email protected] <0.10295.0> 9c0609ffc0 couchdb-service:5984 172.18.0.6 BgdITnfW40W6WgK4ZWXJQo2mNTfbhJnq PUT /_users 201 ok 19
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0{"ok":true}
100    12  100    12    0     0    529      0 --:--:-- --:--:-- --:--:--   545
couch-init_1          |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
couch-init_1          |                                  Dload  Upload   Total   Spent    Left  Speed
couchdb-service_1     | [error] 2022-02-18T15:05:37.191392Z [email protected] <0.10296.0> 759e01ceeb Request to create N=3 DB but only 1 node(s)
couchdb-service_1     | [actor:11] INFO clouseau.cleanup - Removing shards/00000000-7fffffff/_replicator.1645196737
couchdb-service_1     | [actor:11] INFO clouseau.cleanup - Removing shards/80000000-ffffffff/_replicator.1645196737
couchdb-service_1     | [notice] 2022-02-18T15:05:37.201789Z [email protected] <0.545.0> -------- creating replicator ddoc <<"shards/80000000-ffffffff/_replicator.1645196737">>
100    12  100    12    0     0    635      0 --:--:-- --:--:-- --:--:--   666
couch-init_1          | {"ok":true}
couch-init_1          | sh: fg: line 1: No current job
couchdb-service_1     | [notice] 2022-02-18T15:05:37.204972Z [email protected] <0.10296.0> 759e01ceeb couchdb-service:5984 172.18.0.6 BgdITnfW40W6WgK4ZWXJQo2mNTfbhJnq PUT /_replicator 201 ok 16
netzadmin_couch-init_1 exited with code 2

Should I be able to connect to the couchdb remotely? because I've still no luck with that!

2 replies

johnnycheetah Apr 20, 2022

did you ever get a fix for this? I'm having the same issue now

WuMaKu84 Apr 20, 2022

Hi Johnny,

I'm sorry, but no... I'm currently working through the workaround described below...

WuMaKu84 · 2022-02-19T19:37:27Z

WuMaKu84
Feb 19, 2022

Update:
Funny, if I run the docker compose as a stack in portainer -> everything works without any problem!! So what ever the problem is, running as a stack with portainer fixes it!

0 replies

nemu-nemu-ojisan · 2022-09-29T09:23:52Z

nemu-nemu-ojisan
Sep 29, 2022

I saw the same error. But after few settings, my self hosts environment is work properly.
If your host is located behind company's proxy, you may have set environment variable of GLOBAL_AGENT_HTTP(S)_PROXY, and same time you have to set GLOBAL_AGENT_NO_PROXY.
My docker-compose.yml is below

version: "3"

# optional ports are specified throughout for more advanced use cases.

services:
  app-service:
    restart: unless-stopped
    image: budibase.docker.scarf.sh/budibase/apps
    container_name: bbapps
    environment:
      SELF_HOSTED: 1
      COUCH_DB_URL: http://${COUCH_DB_USER}:${COUCH_DB_PASSWORD}@couchdb-service:5984
      WORKER_URL: http://worker-service:4003
      MINIO_URL: http://minio-service:9000
      MINIO_ACCESS_KEY: ${MINIO_ACCESS_KEY}
      MINIO_SECRET_KEY: ${MINIO_SECRET_KEY}
      INTERNAL_API_KEY: ${INTERNAL_API_KEY}
      BUDIBASE_ENVIRONMENT: ${BUDIBASE_ENVIRONMENT}
      PORT: 4002
      JWT_SECRET: ${JWT_SECRET}
      LOG_LEVEL: info
      SENTRY_DSN: https://[email protected]/5338131
      ENABLE_ANALYTICS: "true"
      REDIS_URL: redis-service:6379
      REDIS_PASSWORD: ${REDIS_PASSWORD}
      BB_ADMIN_USER_EMAIL: ${BB_ADMIN_USER_EMAIL}
      BB_ADMIN_USER_PASSWORD: ${BB_ADMIN_USER_PASSWORD}
      PLUGINS_DIR: ${PLUGINS_DIR}
      GLOBAL_AGENT_HTTP_PROXY: http://${PROXY_HOST}:${PROXY_PORT}
      GLOBAL_AGENT_HTTPS_PROXY: http://${PROXY_HOST}:${PROXY_PORT}
      GLOBAL_AGENT_NO_PROXY: couchdb-service,app-service,worker-service,minio-service,redis-service,localhost,127.0.0.1
    depends_on:
      - worker-service
      - redis-service
    volumes:
      - plugins_data:/plugins

  worker-service:
    restart: unless-stopped
    image: budibase.docker.scarf.sh/budibase/worker
    container_name: bbworker
    environment:
      SELF_HOSTED: 1
      PORT: 4003
      CLUSTER_PORT: ${MAIN_PORT}
      JWT_SECRET: ${JWT_SECRET}
      MINIO_ACCESS_KEY: ${MINIO_ACCESS_KEY}
      MINIO_SECRET_KEY: ${MINIO_SECRET_KEY}
      MINIO_URL: http://minio-service:9000
      APPS_URL: http://app-service:4002
      COUCH_DB_USERNAME: ${COUCH_DB_USER}
      COUCH_DB_PASSWORD: ${COUCH_DB_PASSWORD}
      COUCH_DB_URL: http://${COUCH_DB_USER}:${COUCH_DB_PASSWORD}@couchdb-service:5984
      SENTRY_DSN: https://[email protected]/5338131
      INTERNAL_API_KEY: ${INTERNAL_API_KEY}
      REDIS_URL: redis-service:6379
      REDIS_PASSWORD: ${REDIS_PASSWORD}
      GLOBAL_AGENT_HTTP_PROXY: http://${PROXY_HOST}:${PROXY_PORT}
      GLOBAL_AGENT_HTTPS_PROXY: http://${PROXY_HOST}:${PROXY_PORT}
      GLOBAL_AGENT_NO_PROXY: couchdb-service,app-service,worker-service,minio-service,redis-service,localhost,127.0.0.1
    depends_on:
      - redis-service
      - minio-service
      - couch-init

  minio-service:
    restart: unless-stopped
    image: minio/minio
    volumes:
      - minio_data:/data
    environment:
      MINIO_ACCESS_KEY: ${MINIO_ACCESS_KEY}
      MINIO_SECRET_KEY: ${MINIO_SECRET_KEY}
      MINIO_BROWSER: "off"
    command: server /data --console-address ":9001"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  proxy-service:
    restart: unless-stopped
    ports:
      - "${MAIN_PORT}:10000"
    container_name: bbproxy
    image: budibase/proxy
    environment:
      - PROXY_RATE_LIMIT_WEBHOOKS_PER_SECOND=10
      - PROXY_RATE_LIMIT_API_PER_SECOND=20
    depends_on:
      - minio-service
      - worker-service
      - app-service
      - couchdb-service

  couchdb-service:
    restart: unless-stopped
    image: ibmcom/couchdb3
    environment:
      - COUCHDB_PASSWORD=${COUCH_DB_PASSWORD}
      - COUCHDB_USER=${COUCH_DB_USER}
    volumes:
      - couchdb3_data:/opt/couchdb/data

  couch-init:
    image: curlimages/curl
    environment:
      PUT_CALL: "curl -u ${COUCH_DB_USER}:${COUCH_DB_PASSWORD} -X PUT couchdb-service:5984"
    depends_on:
      - couchdb-service
    command: ["sh","-c","sleep 10 && $${PUT_CALL}/_users && $${PUT_CALL}/_replicator; fg;"]

  redis-service:
    restart: unless-stopped
    image: redis
    command: redis-server --requirepass ${REDIS_PASSWORD}
    volumes:
      - redis_data:/data

  watchtower-service:
    restart: always
    image: containrrr/watchtower
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    command: --debug --http-api-update bbapps bbworker bbproxy
    environment:
      - WATCHTOWER_HTTP_API=true
      - WATCHTOWER_HTTP_API_TOKEN=budibase
      - WATCHTOWER_CLEANUP=true
    labels:
      - "com.centurylinklabs.watchtower.enable=false"

volumes:
  couchdb3_data:
    driver: local
  minio_data:
    driver: local
  redis_data:
    driver: local
  plugins_data:
    driver: local

I am glad, if this post become your help.

0 replies

pandel · 2022-10-04T10:19:40Z

pandel
Oct 4, 2022

Hi @ALL!

Today, after upgrading Budibase from 2.0.6 to 2.0.12 I had the same problem. I pulled the proxy/app/worker images and after that the CouchDB container refused to start.

I did the following to recover:

docker-compose down
docker rmi ibmcom/couchdb3:latest
docker system prune
docker pull ibmcom/couchdb3:latest
docker-compose up -d

I retried the whole procedure a few times from a clean VM snapshot always with the same result and it seemed that pulling the new BB images somehow had a negative impact on the Couchdb image, which was already there. I clearly don't understand why, because Docker should have handled the images strictly separate, but who knows... The above mentioned steps solved it at least.

Perhaps it helps.

Regards
Holger

2 replies

melohagan Oct 17, 2022
Maintainer

This worked for me! Legend!

melohagan Nov 4, 2022
Maintainer

Also make sure you install the latest version of the Budibase CLI if that's what you're using: https://docs.budibase.com/docs/budibase-cli-setup

Advaera · 2023-03-31T15:35:19Z

Advaera
Mar 31, 2023

I found a very similar issue to this is caused in CouchDB by creating passwords that are too complex in the .env file. Using a key generator that creates keys using special characters such as "$" and "&" seems to be the problem folks on my end were running into. We searched for the error and came to this page so if anyone else has the problem and the above solution doesn't work, either put the password in quotes, or create long passwords that don't use those characters then re-create the containers.

2 replies

melohagan Apr 3, 2023
Maintainer

Hey @Advaera I had raised a similar bug a while back: #5962

Maybe there's a few symbols that needed added into the regex.

Lord0fBytes Aug 21, 2024

Had this same issue. After removing the symbols it worked.

madukan · 2024-08-24T08:41:24Z

madukan
Aug 24, 2024

I get a Segmentation fault on Mac M1, even though I use platform: linux/amd64 for the couchdb-service.

buildbase couchdb Segmentation fault /docker-entrypoint.sh /opt/couchdb/bin/couchdb > /dev/stdout

Full couchdb-service log:

2024-08-24 18:27:38 Waiting for CouchDB to start...
2024-08-24 18:27:39 SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
2024-08-24 18:27:39 SLF4J: Defaulting to no-operation (NOP) logger implementation
2024-08-24 18:27:39 SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
2024-08-24 18:27:40 {"level":40,"time":1724488060181,"pid":18,"hostname":"fab374ba3475","msg":"the requested number of threads is larger than the number of available CPU cores (10) and will be ignored"}
2024-08-24 18:27:41 {"level":30,"time":1724488061255,"pid":18,"hostname":"fab374ba3475","module":"server","created":1724488060461,"started":1724488061152,"host":"0.0.0.0","port":4984,"protocol":"http","id":"fab374ba3475:18:m07vnzfx","uri":"http://0.0.0.0:4984","address":"0.0.0.0","msg":"server started"}
2024-08-24 18:27:43 Waiting for CouchDB to start...
2024-08-24 18:27:48 Waiting for CouchDB to start...
2024-08-24 18:27:43 /bbcouch-runner.sh: line 82:    17 Segmentation fault      /docker-entrypoint.sh /opt/couchdb/bin/couchdb > /dev/stdout 2>&1
2024-08-24 18:27:54 Waiting for CouchDB to start...
2024-08-24 18:27:59 Waiting for CouchDB to start...
2024-08-24 18:28:04 Waiting for CouchDB to start...
2024-08-24 18:28:09 Waiting for CouchDB to start...
2024-08-24 18:28:14 Waiting for CouchDB to start...
2024-08-24 18:28:19 Waiting for CouchDB to start...

Any help much appreciated. I'm super new to Buildbase.

This issue resonate with this (fixed) bug:
https://www.couchbase.com/forums/t/createbucket-causes-errors-and-or-segmentation-fault/15156/4
https://forum.communityhealthtoolkit.org/t/launching-cht-using-docker-helper-with-the-latest-version-4-5-1-results-in-a-502-bad-gateway-error/3308/4

Got it fixed!
Her's the solution: Use the couchdb:v3.2.1 image in couchdb-service:

ie: (I'm going to leave the commented lines here for reference, but you may delete them indeed to remove clutter in your docker-compose.yaml)

    # image: budibase/couchdb
    # image: budibase/couchdb:v3.3.3
    image: budibase/couchdb:v3.2.1

0 replies

CouchDb issue starting Budibase with docker-compose #4269

Replies: 15 comments · 17 replies

cmthomps Jan 31, 2022 Author

shogunpurple Feb 1, 2022 Maintainer

cmthomps Feb 1, 2022 Author

shogunpurple Feb 1, 2022 Maintainer

cmthomps Feb 1, 2022 Author

shogunpurple Feb 1, 2022 Maintainer

cmthomps Feb 1, 2022 Author

cmthomps Feb 1, 2022 Author

cmthomps Feb 7, 2022 Author

shogunpurple Feb 7, 2022 Maintainer

cmthomps Feb 7, 2022 Author

cmthomps Feb 17, 2022 Author

melohagan Oct 17, 2022 Maintainer

melohagan Nov 4, 2022 Maintainer

melohagan Apr 3, 2023 Maintainer

Replies: 15 comments 17 replies

cmthomps
Jan 31, 2022
Author

shogunpurple
Feb 1, 2022
Maintainer

cmthomps
Feb 1, 2022
Author

shogunpurple
Feb 1, 2022
Maintainer

cmthomps Feb 1, 2022
Author

shogunpurple Feb 1, 2022
Maintainer

cmthomps Feb 1, 2022
Author

cmthomps
Feb 1, 2022
Author

cmthomps
Feb 7, 2022
Author

shogunpurple Feb 7, 2022
Maintainer

cmthomps Feb 7, 2022
Author

cmthomps
Feb 17, 2022
Author

melohagan Oct 17, 2022
Maintainer

melohagan Nov 4, 2022
Maintainer

melohagan Apr 3, 2023
Maintainer