Skip to content

Commit 7d93d69

Browse files
renecannaorenecannao
committed
test(passthrough-auth): TAP test for backend-rejection cache invalidation
P2-D from the round-3 review. End-to-end test for spec §8.4's "backend rotated the password -> 1045 -> cache evicts and the next connect re-probes" flow. The eviction hook lives in handler_again___status_CONNECTING_SERVER and is gated by commit N4's session-level @c passthrough_credential flag. Test sequence: Setup - backend user with caching_sha2 + OLD_PW - mysql_users empty-pw row - passthrough_auth_enabled / require_tls=false / default_auth= caching_sha2 / FLUSH cache [1] First connect with OLD_PW -- real probe runs, cache populated. [2] Backend: ALTER USER ... IDENTIFIED BY NEW_PW The cached cleartext is now STALE. [3] New client connection through ProxySQL with OLD_PW: [3a] Frontend handshake SUCCEEDS via cache hit (PPHR_6auth2 verifies the scramble against the cached cleartext -- no backend involvement at this stage). Session is marked passthrough_credential = true. [3b] SELECT 1 issued. Backend pool has no entry for this user (the step-1 probe was a one-shot mysql_real_connect, closed and never pooled), so ProxySQL opens a fresh backend connection with the stale cached cleartext. Backend rejects with 1045. [3c] The 1045 hook in CONNECTING_SERVER: - sees sess->passthrough_credential == true, - evicts the cache entry (returns true since present), - bumps cache_invalidations. We then assert: - cache_invalidations delta >= 1 - stats_mysql_passthrough_auth_cache row for the user is gone A small retry budget (1s, 20 * 50ms) absorbs any cross-thread visibility lag between the worker emitting the eviction and the admin SELECT reading state. A subtle false-negative to be aware of (documented inline): if the proxy happens to have a usable pool connection for this user when SELECT 1 runs, the query will use it and never trigger the 1045 hook. The assertion catches that explicitly (counter unchanged -> test fail). The fixture step 1's one-shot probe doesn't pool, so the pool starts empty -- pre-condition for the test to be deterministic. Cleanup restores defaults. Registered in mysql84-g4 / mysql90-g4 / mysql95-g4 alongside the other end-to-end passthrough tests.
1 parent 762479c commit 7d93d69

2 files changed

Lines changed: 282 additions & 0 deletions

File tree

‎test/tap/groups/groups.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -392,6 +392,7 @@
392392
"test_noise_injection-t" : [ "legacy-g8","mysql84-g8","mysql90-g3","mysql95-g3" ],
393393
"test_passthrough_auth_admin-t" : [ "no-infra-g1" ],
394394
"test_passthrough_auth_e2e-t" : [ "mysql84-g4","mysql90-g4","mysql95-g4" ],
395+
"test_passthrough_auth_invalidation-t" : [ "mysql84-g4","mysql90-g4","mysql95-g4" ],
395396
"test_passthrough_auth_metrics-t" : [ "mysql84-g4","mysql90-g4","mysql95-g4" ],
396397
"test_passthrough_auth_misconfig_warning-t" : [ "no-infra-g1" ],
397398
"test_passthrough_auth_ratelimit-t" : [ "mysql84-g4","mysql90-g4","mysql95-g4" ],

‎test/tap/tests/test_passthrough_auth_invalidation-t.cpp‎

Lines changed: 281 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,281 @@
1+
/**
2+
* @file test_passthrough_auth_invalidation-t.cpp
3+
* @brief End-to-end test for backend-rejection cache invalidation
4+
* (spec §8.4) -- P2-D from the round-3 review.
5+
*
6+
* The pass-through cache holds the cleartext password ProxySQL learned
7+
* from a successful backend probe. When the backend's password later
8+
* rotates (admin runs ALTER USER on the upstream server), the cached
9+
* cleartext is stale. The next query that needs a fresh backend
10+
* connection will use the stale cleartext, the backend will reject
11+
* with @c ER_ACCESS_DENIED_ERROR (1045), and the hook in
12+
* @c handler_again___status_CONNECTING_SERVER must:
13+
*
14+
* - evict the stale cache entry so subsequent connects re-probe
15+
* against the new password, AND
16+
* - bump the @c cache_invalidations counter (gated on the session's
17+
* @c passthrough_credential flag, per commit N4).
18+
*
19+
* Sequence the test follows:
20+
*
21+
* 1. Provision backend user with @c old_password.
22+
* 2. Configure passthrough; first connect drives a real probe and
23+
* caches @c old_password.
24+
* 3. Read @c cache_invalidations and the cache row -- baseline.
25+
* 4. ALTER USER on the backend to @c new_password.
26+
* 5. Open a fresh client connection through ProxySQL using the same
27+
* @c old_password. The pass-through cache HITS (frontend
28+
* verification succeeds against the cached cleartext) and the
29+
* session is marked @c passthrough_credential.
30+
* 6. Issue SELECT 1 over that session. ProxySQL needs a backend
31+
* connection; the probe connection from step 2 was closed (one-
32+
* shot, not pooled), so the pool is empty and a fresh backend
33+
* conn is required. The connection attempt uses @c old_password
34+
* and the backend now rejects with 1045.
35+
* 7. Re-read @c cache_invalidations and the cache row:
36+
* cache_invalidations must have incremented AND the cache entry
37+
* for the user must be gone.
38+
*
39+
* Step 6 is the load-bearing path. If the pool happens to have a
40+
* usable connection (e.g. parallel test load), the query won't trigger
41+
* the eviction hook and the test would yield a false negative -- the
42+
* counter delta assertion catches that case explicitly (delta == 0
43+
* → fail).
44+
*/
45+
#include <cstring>
46+
#include <string>
47+
48+
#include "mysql.h"
49+
#include "mysqld_error.h"
50+
51+
#include "tap.h"
52+
#include "command_line.h"
53+
#include "utils.h"
54+
55+
using std::string;
56+
57+
static constexpr const char* TEST_USER = "tap_pt_invalidation_user";
58+
static constexpr const char* OLD_PW = "0ld-passw0rd-rot4t10n!";
59+
static constexpr const char* NEW_PW = "n3w-passw0rd-rot4t10n!";
60+
61+
static uint32_t MYSQL8_HG = get_env_int("TAP_MYSQL8_BACKEND_HG", 30);
62+
63+
static int do_query(MYSQL* m, const string& q) {
64+
if (mysql_query(m, q.c_str())) {
65+
diag("Query failed: %s -- %s", q.c_str(), mysql_error(m));
66+
return EXIT_FAILURE;
67+
}
68+
return EXIT_SUCCESS;
69+
}
70+
71+
static int64_t read_metric(MYSQL* admin, const string& name) {
72+
const string q =
73+
"SELECT metric_value FROM stats_mysql_passthrough_auth_metrics "
74+
"WHERE metric_name='" + name + "'";
75+
if (mysql_query(admin, q.c_str())) return -1;
76+
MYSQL_RES* res = mysql_store_result(admin);
77+
if (!res) return -1;
78+
MYSQL_ROW row = mysql_fetch_row(res);
79+
int64_t v = (row && row[0]) ? atoll(row[0]) : -1;
80+
mysql_free_result(res);
81+
return v;
82+
}
83+
84+
static int cache_entry_count_for(MYSQL* admin, const string& user) {
85+
const string q =
86+
"SELECT COUNT(*) FROM stats_mysql_passthrough_auth_cache WHERE username='" + user + "'";
87+
if (mysql_query(admin, q.c_str())) return -1;
88+
MYSQL_RES* res = mysql_store_result(admin);
89+
if (!res) return -1;
90+
MYSQL_ROW row = mysql_fetch_row(res);
91+
int n = (row && row[0]) ? atoi(row[0]) : -1;
92+
mysql_free_result(res);
93+
return n;
94+
}
95+
96+
/**
97+
* @brief Open a frontend session, run a query, and report both the
98+
* connect errno and the query errno.
99+
*
100+
* The query is what triggers the backend-pool acquisition, so a
101+
* frontend connect that succeeds via cache hit AND a backend 1045
102+
* during query execution will produce @c { 0, 1045 }.
103+
*/
104+
static std::pair<unsigned int, unsigned int> connect_and_query(
105+
const CommandLine& cl, const char* user, const char* pw
106+
) {
107+
MYSQL* m = mysql_init(NULL);
108+
mysql_options(m, MYSQL_DEFAULT_AUTH, "caching_sha2_password");
109+
const MYSQL* res = mysql_real_connect(m, cl.host, user, pw, NULL, cl.port, NULL, 0);
110+
if (!res) {
111+
const unsigned int err = mysql_errno(m);
112+
mysql_close(m);
113+
return { err, 0 };
114+
}
115+
const int qrc = mysql_query(m, "SELECT 1");
116+
const unsigned int qerr = qrc ? mysql_errno(m) : 0;
117+
if (!qrc) {
118+
MYSQL_RES* r = mysql_store_result(m);
119+
if (r) mysql_free_result(r);
120+
}
121+
mysql_close(m);
122+
return { 0, qerr };
123+
}
124+
125+
int main() {
126+
CommandLine cl;
127+
128+
/*
129+
* Plan: 4 setup + 1 probe-success + 1 ALTER + 1 baseline read +
130+
* 1 invalidation observation + 2 cleanup = 10.
131+
*/
132+
plan(10);
133+
134+
if (cl.getEnv()) {
135+
return exit_status();
136+
}
137+
138+
MYSQL* backend = mysql_init(NULL);
139+
ok(mysql_real_connect(backend, cl.mysql_host, cl.mysql_username, cl.mysql_password,
140+
NULL, cl.mysql_port, NULL, 0) != NULL,
141+
"Connected to backend MySQL");
142+
143+
MYSQL* admin = mysql_init(NULL);
144+
ok(mysql_real_connect(admin, cl.admin_host, cl.admin_username, cl.admin_password,
145+
NULL, cl.admin_port, NULL, 0) != NULL,
146+
"Connected to ProxySQL admin");
147+
148+
/* -------- backend user with old password -------- */
149+
do_query(backend, string("DROP USER IF EXISTS '") + TEST_USER + "'@'%'");
150+
const bool user_ok =
151+
(do_query(backend,
152+
string("CREATE USER '") + TEST_USER + "'@'%' "
153+
"IDENTIFIED WITH 'caching_sha2_password' BY '" + OLD_PW + "'") == EXIT_SUCCESS) &&
154+
(do_query(backend,
155+
string("GRANT SELECT ON *.* TO '") + TEST_USER + "'@'%'") == EXIT_SUCCESS);
156+
ok(user_ok, "Backend user provisioned with OLD password");
157+
158+
/* -------- empty-pw row + passthrough config -------- */
159+
do_query(admin, string("DELETE FROM mysql_users WHERE username='") + TEST_USER + "'");
160+
do_query(admin,
161+
string("INSERT INTO mysql_users (username, password, default_hostgroup, active) VALUES ('")
162+
+ TEST_USER + "', '', " + std::to_string(MYSQL8_HG) + ", 1)");
163+
do_query(admin, "LOAD MYSQL USERS TO RUNTIME");
164+
165+
bool cfg_ok = true;
166+
cfg_ok &= (do_query(admin, "SET mysql-passthrough_auth_enabled='true'") == EXIT_SUCCESS);
167+
cfg_ok &= (do_query(admin, "SET mysql-passthrough_auth_require_tls='false'") == EXIT_SUCCESS);
168+
cfg_ok &= (do_query(admin, "SET mysql-passthrough_auth_empty_password='true'") == EXIT_SUCCESS);
169+
cfg_ok &= (do_query(admin, "SET mysql-default_authentication_plugin='caching_sha2_password'") == EXIT_SUCCESS);
170+
cfg_ok &= (do_query(admin, "PROXYSQL FLUSH PASSTHROUGH_AUTH_CACHE") == EXIT_SUCCESS);
171+
cfg_ok &= (do_query(admin, "LOAD MYSQL VARIABLES TO RUNTIME") == EXIT_SUCCESS);
172+
ok(cfg_ok, "Pass-through configured");
173+
174+
/* ============================================================
175+
* Step 1 -- first connect probes & caches OLD password.
176+
* ============================================================ */
177+
{
178+
const auto [cerr, qerr] = connect_and_query(cl, TEST_USER, OLD_PW);
179+
ok(cerr == 0 && qerr == 0,
180+
"[1] First connect with OLD password succeeds "
181+
"(connect_errno=%u, query_errno=%u)", cerr, qerr);
182+
}
183+
184+
/* ============================================================
185+
* Step 2 -- ALTER USER on backend rotates to NEW password.
186+
* ============================================================ */
187+
{
188+
const string alter =
189+
string("ALTER USER '") + TEST_USER + "'@'%' "
190+
"IDENTIFIED WITH 'caching_sha2_password' BY '" + NEW_PW + "'";
191+
ok(do_query(backend, alter) == EXIT_SUCCESS,
192+
"[2] Backend ALTER USER to NEW password");
193+
}
194+
195+
/*
196+
* Capture baseline counter + cache state BEFORE the
197+
* invalidation-triggering connect.
198+
*/
199+
const int64_t inv_before = read_metric(admin, "cache_invalidations");
200+
const int cache_before = cache_entry_count_for(admin, TEST_USER);
201+
202+
/* ============================================================
203+
* Step 3 -- new client connects with OLD password.
204+
*
205+
* Frontend: cache HIT (cleartext is OLD_PW which is what the
206+
* client also sent; PPHR_6auth2 verifies the scramble against the
207+
* cached cleartext and accepts). Session is marked
208+
* @c passthrough_credential = true.
209+
*
210+
* Backend: connection pool has no entries for this user (the
211+
* probe in step 1 was a one-shot mysql_real_connect, closed
212+
* immediately, never pooled). So the SELECT 1 triggers a fresh
213+
* backend connection attempt using @c userinfo->password (OLD_PW
214+
* from the cache). The backend rejects with 1045 because it now
215+
* expects NEW_PW.
216+
*
217+
* The 1045 hook in handler_again___status_CONNECTING_SERVER:
218+
* - sees @c sess->passthrough_credential == true,
219+
* - calls @c GloMyPTAuthCache->evict() (returns true since the
220+
* entry exists),
221+
* - bumps @c cache_invalidations.
222+
*
223+
* The client gets a 1045 errno on SELECT 1. That's expected.
224+
* ============================================================ */
225+
{
226+
const auto [cerr, qerr] = connect_and_query(cl, TEST_USER, OLD_PW);
227+
/* cerr should be 0: frontend handshake succeeds because the
228+
* cache hit lets PPHR_6auth2 verify the scramble against the
229+
* cached cleartext (no backend involvement at handshake).
230+
* qerr should be 1045: SELECT 1 forces a fresh backend conn
231+
* with the stale cleartext; backend rejects. */
232+
ok(cerr == 0,
233+
"[3a] Frontend handshake succeeds via cache hit "
234+
"(connect_errno=%u)", cerr);
235+
ok(qerr == ER_ACCESS_DENIED_ERROR,
236+
"[3b] SELECT 1 fails with 1045 (stale cached cleartext) "
237+
"(query_errno=%u, expected %u)",
238+
qerr, (unsigned)ER_ACCESS_DENIED_ERROR);
239+
}
240+
241+
/*
242+
* Allow a few microseconds for the eviction hook to run on the
243+
* worker thread before reading state. In practice the eviction
244+
* is synchronous with the 1045 handling, so the next admin query
245+
* already sees the result. Add a small retry budget for
246+
* robustness on busy CI.
247+
*/
248+
int64_t inv_after = read_metric(admin, "cache_invalidations");
249+
int cache_after = cache_entry_count_for(admin, TEST_USER);
250+
for (int i = 0; i < 20 && (inv_after <= inv_before || cache_after != 0); ++i) {
251+
usleep(50000);
252+
inv_after = read_metric(admin, "cache_invalidations");
253+
cache_after = cache_entry_count_for(admin, TEST_USER);
254+
}
255+
ok(inv_after >= inv_before + 1 && cache_after == 0,
256+
"[3c] cache_invalidations++ AND cache entry evicted "
257+
"(invalidations: %lld -> %lld; cache rows for user: %d -> %d)",
258+
(long long)inv_before, (long long)inv_after,
259+
cache_before, cache_after);
260+
261+
/* -------- cleanup -------- */
262+
{
263+
const int rc = do_query(backend, string("DROP USER IF EXISTS '") + TEST_USER + "'@'%'");
264+
ok(rc == EXIT_SUCCESS, "Cleanup: DROP USER on backend");
265+
}
266+
{
267+
int rc = EXIT_SUCCESS;
268+
rc |= do_query(admin, string("DELETE FROM mysql_users WHERE username='") + TEST_USER + "'");
269+
rc |= do_query(admin, "PROXYSQL FLUSH PASSTHROUGH_AUTH_CACHE");
270+
rc |= do_query(admin, "LOAD MYSQL USERS TO RUNTIME");
271+
rc |= do_query(admin, "SET mysql-passthrough_auth_enabled='false'");
272+
rc |= do_query(admin, "SET mysql-passthrough_auth_require_tls='true'");
273+
rc |= do_query(admin, "SET mysql-default_authentication_plugin='mysql_native_password'");
274+
rc |= do_query(admin, "LOAD MYSQL VARIABLES TO RUNTIME");
275+
ok(rc == EXIT_SUCCESS, "Cleanup: globals restored");
276+
}
277+
278+
mysql_close(backend);
279+
mysql_close(admin);
280+
return exit_status();
281+
}

0 commit comments

Comments
 (0)