feat(#9706): use Proxy Auth for communicating with Couch #9740

jkuester · 2025-01-10T23:51:39Z

Description

This is my prototype attempt at "simplifying" communication between api/sentinel and CouchDB. As noted in the ticket, there are currently several different user modes used when api/sentinel connect to Couch. Significantly, there is currently not really any way for api/sentinel to connect to Couch with just "member" access. Currently all access (not using creds provided by a user) is full "admin". Most of the time, admin access is unnecessary.

This PR tries to address both challenges by introducing Couch Proxy Auth as the sole method used to authenticate Couch traffic from api/sentinel (except for the actual user auth checks to authenticate users). The username cht-system gets added as a "member" user to the various Couch dbs that api/sentinel need to access. Then, all traffic from api/sentinel includes the proxy auth headers needed to authenticate this cht-system user (including a token calculated via the COUCHDB_SECRET). This means that by default all traffic has "member" access. For requests that need admin access (or no access, e.g. for the auth requests) this must be explicitly configured in the request. (A request for the cht-system user is made "admin" simply by specifying _admin in the role header.) IMHO this gives a nice balance of security and convenience.

Including a COUCHDB_SECRET envar when deploying via Docker Compose is part of our prod hosting instructions (not positive about the k8s setup). However, it does not seem to actually be required to run the compose config unless you are doing a multi-node Couch cluster. Otherwise, if no secret is configured, then the Couch container will generate one when it runs. This means that even if we update the compose config to inject the COUCHDB_SECRET envar, there is not a 100% guarantee it will be there. So, I added some fallback code to ping CouchDB and lookup the secret. If the COUCHDB_SECRET envar is set, then the COUCH_URL envar for api/sentinel does not actually need to include the user/pass.

The overall goal of simplifying Couch communication is to hopefully make the implementation of SSO Auth in #9735 straightforward so that SSO users can be authenticated against the 3rd party identity provider and then the api server does all the talking to Couch with no need of an actual Couch user session.

TODO

Update couch-request functionality to not use _admin role by default on the non-pouch requests. We should be able to make "member" level requests by default. The _admin requests should be made by explicitly setting that header.
Fix the server-checks code to kill the server if the Couch secret cannot be found.
Update bundled docker compose files to pass COUCHDB_SECRET in to the api and sentinel containers
Update api routing to intercept online user Couch traffic and update it to use proxy auth headers

Code review checklist

UI/UX backwards compatible: Test it works for the new design (enabled by default). And test it works in the old design, enable can_view_old_navigation permission to see the old design.
Readable: Concise, well named, follows the style guide, documented if necessary.
Documented: Configuration and user documentation on cht-docs
Tested: Unit and/or e2e where appropriate
Internationalised: All user facing text
Backwards compatible: Works with existing data and configuration or includes a migration. Any breaking changes documented in the release notes.

License

The software is provided under AGPL-3.0. Contributions to this project are accepted under the same license.

Signed-off-by: Robert <[email protected]>

…s do not fail.

jkuester · 2025-01-11T00:00:22Z

api/src/db.js

    await request.put({ url: securityUrl.toString(), json: true, body: securityObject });
  };

-  module.exports.addRoleAsAdmin = (dbname, role) => addRoleToSecurity(dbname, role, true);


This function was not being used anywhere.

jkuester · 2025-01-11T00:05:28Z

api/src/services/config-watcher.js

@@ -143,14 +144,19 @@ const updateServiceWorker = () => {
  });
 };

-const load = () => {
+const addSystemUserToDbs = () => Promise.all([...DATABASES, { name: `${environment.db}-purged-cache` }]


Technically this is not currently "config related", so it probably does not belong in here. I just stuffed it in here for now since it was the right place in the startup and nowhere else seemed better.

Also probably need a more centralized approach to listing all the DBs that the system user should have member access too....

jkuester · 2025-01-11T00:06:31Z

couchdb/10-docker-default.ini

@@ -19,6 +19,10 @@ bind_address = 0.0.0.0
 server_options = [{recbuf, 262144}]
 socket_options = [{sndbuf, 262144}, {nodelay, true}]
 require_valid_user = true
+authentication_handlers = {chttpd_auth, cookie_authentication_handler}, {chttpd_auth, proxy_authentication_handler}, {chttpd_auth, default_authentication_handler}


The default value is just:

authentication_handlers = {chttpd_auth, cookie_authentication_handler}, {chttpd_auth, default_authentication_handler}

jkuester · 2025-01-11T00:10:22Z

shared-libs/couch-request/src/couch-request.js

@@ -18,18 +20,19 @@ const methods = {
  HEAD: 'HEAD'
 };

+const promisedAdminAuthHeaders = getAuthHeaders(environment.username, '_admin');


This needs to be changed to not set the _admin role by default. I have it like this so everything runs, but the intended approach is to just use the default username with its "member" access. Then, for requests that need admin access, they will have to explicitly set the _admin role header.

jkuester · 2025-01-11T00:11:15Z

shared-libs/couch-request/src/proxy-auth.js

I put the code for dealing with the COUCHDB_SECRET in here. It could probably be its own shared-lib if we wanted but it seemed at least adjacent to the couch-request functionality so I started with it here.

Actually, after more digging, it seems like shared-libs/settings might be a the most logical place to put this. However seems like it could cause a circular dependency since that lib already depends on couch-request. I need to look more into this.

jkuester · 2025-01-11T03:12:45Z

@dianabarsan Could you please let me know what you think about the approach in this prototype? 🙏 The code is not done yet, but I think it proves enough of the functionality for us to be able to make a decision on if we want to pursue this approach.

The api/sentinel dev instances do run locally for me and I have done some smoke testing, but there are definitely still broken edge cases and incomplete flows....

dianabarsan · 2025-01-13T20:28:50Z

Hi @jkuester

I requested a review, so I get reminded about this. I had read the email over the weekend and it skipped my memory today. Apologies for the delay.

dianabarsan

I left a few questions inline.
Nice work on this. Overall I think this could be useful to overcome the difficulties in handling sso, however I'm still not convinced about how secure this will be, because it seems that right now everyone gets a free pass.

dianabarsan · 2025-01-14T15:13:01Z

shared-libs/couch-request/src/proxy-auth.js

+    return envarSecret;
+  }
+
+  // Need to look up the secret (assuming the COUCH_URL contains user/pass)


I think that, in some other file here you had edited the export COUCH_URL documentation to not include basic auth.
So, we would still have a COUCH_URL with basic auth?

In order to make things passive, we need to support the case where no COUCHDB_SECRET is set, but the COUCH_URL contains the username/password (this is likely the case for many existing single-node servers). With these changes the "recommended" case would be where COUCHDB_SECRET is set and COUCH_URL does not contain username/password.

The case we do not support is where COUCHDB_SECRET is NOT set and COUCH_URL does not contain username/password. (Since in this case we have no way of actually connecting to Couch...)

dianabarsan · 2025-01-14T15:13:52Z

shared-libs/couch-request/src/proxy-auth.js

+  };
+};
+
+const getAuthHeaders = async (username, roles) => {


This basically means that everyone has a free pass? Since we are getting the token ourselves? How are we authenticating people?

"everyone" here would be "all the calls coming from api/sentinel" and "free pass" would be "member-level access to Couch dbs (upgradeable to admin by changing the roles header value)".

The goal is to use proxy auth for all Couch traffic after the user making the request is authenticated. (More on this in my response to your next comment.)

dianabarsan · 2025-01-14T21:24:21Z

shared-libs/couch-request/src/proxy-auth.js

+
+const getAuthHeaders = async (username, roles) => {
+  if (!username) {
+    return createHeaders(username, roles);


So, if the username is null we'll create headers with the username null? I don't follow.

Yeah, so this really needs its own code path instead of the confusing overloading I am doing here. But practically, yes. When you pass in null, you get null values for the headers which means that the proxy auth headers are not actually set on the request.

So, when doing the _session checks to authenticate the user making the api REST call, we pass null in here to make sure the _session call is done without any proxy auth headers (and is just authenticated via the users username/password or their session cookie).

For all the normal traffic between api/sentinal and Couch we want to use Proxy Auth. But, for the specific user authorization checks (made against _session) we explicitly disable the proxy auth (by passing null), so we can actually check the user. For SSO flows, instead of calling _session to validate the user, we would call the 3rd party identity provider. Either way, once the user is authenticated, the subsequent communication between api and couch should be the same and should happen via proxy auth.

rmayore and others added 12 commits January 9, 2025 12:56

feat: Added couch configs to enable proxy authentication

7c5b4d0

Signed-off-by: Robert <[email protected]>

Remove unnecessary config from Couch

6dc4e7b

Update API Pouch fetch to use Proxy Auth

eba99b3

Add proxyAuth to couch-request

3b2adae

Setup couch-request at beginning of server start so that server check…

b93d2fc

…s do not fail.

Remove now unused username and password from environment

b06dc15

Remove now unused username and password from environment

1db7bda

Do not pass proxy auth headers for login/auth requests

8cf3c21

Move proxy auth header config to environment lib

4cd3784

Load secret dynamically

3c35183

Remove unnecessary null roles

9b3f65e

Add system username to dbs as member on startup.

cf4a07f

jkuester changed the title ~~feat: use Proxy Auth for communicating with Couch~~ feat(#9706): use Proxy Auth for communicating with Couch Jan 10, 2025

jkuester commented Jan 11, 2025

View reviewed changes

Cleanup

1dfdee5

jkuester commented Jan 11, 2025

View reviewed changes

dianabarsan self-requested a review January 13, 2025 20:27

dianabarsan reviewed Jan 14, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(#9706): use Proxy Auth for communicating with Couch #9740

feat(#9706): use Proxy Auth for communicating with Couch #9740

jkuester commented Jan 10, 2025 •

edited

Loading

jkuester Jan 11, 2025

jkuester Jan 11, 2025

jkuester Jan 11, 2025

jkuester Jan 11, 2025

jkuester Jan 11, 2025

jkuester Jan 13, 2025

jkuester commented Jan 11, 2025

dianabarsan commented Jan 13, 2025

dianabarsan left a comment

dianabarsan Jan 14, 2025

jkuester Jan 14, 2025

dianabarsan Jan 14, 2025

jkuester Jan 14, 2025

dianabarsan Jan 14, 2025

jkuester Jan 14, 2025

feat(#9706): use Proxy Auth for communicating with Couch #9740

Are you sure you want to change the base?

feat(#9706): use Proxy Auth for communicating with Couch #9740

Conversation

jkuester commented Jan 10, 2025 • edited Loading

Description

TODO

Code review checklist

License

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkuester commented Jan 11, 2025

dianabarsan commented Jan 13, 2025

dianabarsan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkuester commented Jan 10, 2025 •

edited

Loading