Skip to content

HTTP/2 connections leak when reused_http_session() races with release_http_session() #1024

Open

Description

@sebc119

Describe the bug

reused_http_session() can put an h2 connection into InUsePool twice. When the connection goes idle, only one entry is removed. The other stays in InUsePool, which has no idle timer, so Pingora never closes the connection.

How it happens:

  1. Task A calls reused_http_session(), which takes connection C out of InUsePool.
  2. Stream B on C ends. release_http_session() can't find C, and C still has other streams, so it puts C back.
  3. Task A puts C back too, without release_lock or a duplicate check. Now there are two entries.
  4. When C goes idle, one entry moves to the idle pool and times out. The other keeps C open.

Relevant code:

Pingora info

Pingora version: 0.9.0, still present on main
Rust version: 1.89
Operating system version: Debian 12

Steps to reproduce

Run test_reused_http_session_no_duplicate_in_use_entry from #1025 without the fix. It forces the race on every run.

I also reproduced it under load: 256 concurrent tasks in bursts against a local h2c upstream that never closes idle connections, with a 3 s idle_timeout.

Expected results

No connections left open after idle_timeout.

Observed results

The test finds one leftover entry in InUsePool. The load test left 4 to 10 sockets open in every run.

Additional context

#1025 fixes it by taking release_lock and removing the old entry before the insert.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions