Google Cloud Run: Uncaught signal: 2

[singulart]

Environment

Google Cloud, PostreSQL 14 runs in Cloud SQL

• PostgreSQL version: 14
• PostgREST version: Docker, latest
• Operating system:

Description of issue

I am running the PostgREST on Google Cloud Run. The database is also managed by Google (Cloud SQL.) PostgREST seems to connect to it fine, but couple of minutes later crashes with the obscure message:

2022-10-10 19:41:32.679 EEST10/Oct/2022:16:41:32 +0000: Attempting to connect to the database...
2022-10-10 19:41:32.977 EEST10/Oct/2022:16:41:32 +0000: Connection successful
2022-10-10 19:41:32.978 EEST10/Oct/2022:16:41:32 +0000: Listening on port 3000
2022-10-10 19:41:33.002 EEST10/Oct/2022:16:41:32 +0000: Config reloaded
2022-10-10 19:41:33.005 EEST10/Oct/2022:16:41:32 +0000: Listening for notifications on the pgrst channel
2022-10-10 19:41:33.070 EEST10/Oct/2022:16:41:32 +0000: Schema cache loaded
2022-10-10 19:45:32.280 EESTUncaught signal: 2, pid=1, tid=1, fault_addr=0.
2022-10-10 19:45:32.362 EESTContainer terminated on signal 2.

Any help or advise on this will be greatly appreciated!

[steve-chavez]

2022-10-10 19:45:32.280 EESTUncaught signal: 2, pid=1, tid=1, fault_addr=0.
2022-10-10 19:45:32.362 EESTContainer terminated on signal 2.

Those errors come from Cloud SQL. Related: https://stackoverflow.com/questions/70272414/lots-of-uncaught-signal-6-errors-in-cloud-run

Perhaps it's lack of resources on your instance?

[singulart]

Thanks for your suggestion @steve-chavez. I tried to increase resources on both Cloud SQL side and on Cloud Run side, to no visible result. The Cloud Run still is terminated with signal 2

[steve-chavez]

Hm, since the error is so obscure I don't know what could be wrong.

How about doing db-channel-enabled=false and db-config=false. Also, try doing some requests before it dies off.

[singulart]

No luck with these settings, too. And doing a request before the Cloud Run instance finishes initialization doesn't seem possible (no assigned URL is available..)

Out of curiosity, where can I find the sequence diagram of postgrest bootstrap? What happens exactly when the app launches, that is. As a wild guess, it may happen that Google kills the container after some time as it believes the app isn't healthy, because it fails to react on some kind of readiness probes.

[singulart]

Another observation: crashes always happen 4 minutes after the DB connection is established. Is there any timeout setting within postgrest?

[wolfgangwalther]

As a wild guess, it may happen that Google kills the container after some time as it believes the app isn't healthy, because it fails to react on some kind of readiness probes.

It's very likely that you're on the right track here. Have a look at the docs for PostgREST, we have some health endpoints that you can use:

https://postgrest.org/en/latest/configuration.html#admin-server-port

https://postgrest.org/en/latest/admin.html#health-check

[singulart]

Problem solved! The default port for Cloud Run container is 8080. When changed to 3000, the app starts fine. Thanks a mil :)