|
| 1 | +## Context |
| 2 | + |
| 3 | +The application already uses the AWS SDK against an S3-compatible endpoint, but the implementation is coupled to Tencent COS: |
| 4 | + |
| 5 | +- `packages/config/src/index.ts` hard-codes `hackweek-public-1303088253` and the COS public host. |
| 6 | +- `packages/lib-server/src/storage/index.ts` injects Tencent's `Appid` header. |
| 7 | +- Production data stores absolute COS URLs in ordinary text, rich text, JSON, and array columns. Absolute URLs are returned unchanged, so changing only environment variables would leave historical traffic on Tencent. |
| 8 | +- The browser limits attachments to 200 MB, but the direct-upload fallback can buffer the full payload in application memory. |
| 9 | + |
| 10 | +### Live inventory (2026-07-16) |
| 11 | + |
| 12 | +| Surface | Evidence | |
| 13 | +|---|---:| |
| 14 | +| Tencent COS bucket | 1,789 objects / 16.818 GiB | |
| 15 | +| MP4 + MOV | 228 objects / 15.10 GiB / 89.8% of bytes | |
| 16 | +| Distinct keys referenced by the database or committed templates | 522 objects / 4.807 GiB | |
| 17 | +| Unreferenced submission objects approved for removal | 297 objects / 11.431 GiB | |
| 18 | +| Unreferenced submission videos in that removal set | 150 objects / 10.798 GiB | |
| 19 | +| Conservative migration set | 1,492 objects / 5.387 GiB | |
| 20 | +| Referenced keys missing from COS | 0 | |
| 21 | + |
| 22 | +The migration set excludes only objects under `events/<eventId>/submissions/` that are not referenced by the current production database. Every other object is copied, including small unreferenced templates, event content, QR codes, and historical public assets, to bias toward site stability. |
| 23 | + |
| 24 | +Cloudflare account checks confirmed that `hackathonweekly.com` is an active zone in the same account, `assets.hackathonweekly.com` is unused, and R2 access is available. An old `hackweek` R2 bucket contains one small object and is attached to the unrelated, currently broken `r2.hackweek.cn` custom domain; it will not be reused. |
| 25 | + |
| 26 | +## Goals / Non-Goals |
| 27 | + |
| 28 | +- Goals: |
| 29 | + - Stop new writes and public reads through Tencent COS. |
| 30 | + - Preserve every referenced object and conservatively retain non-submission assets. |
| 31 | + - Make `assets.hackathonweekly.com` the stable, provider-independent public asset origin. |
| 32 | + - Keep the retained R2 set within the Standard free tier where possible. |
| 33 | + - Preserve current video playback behavior and prioritize event-day stability. |
| 34 | + - Produce a reversible and auditable cutover. |
| 35 | +- Non-Goals: |
| 36 | + - Removing Tencent Cloud SMS or content-moderation integrations; this change retires object storage only. |
| 37 | + - Migrating abandoned submission files that have no production database reference. |
| 38 | + - Adding Cloudflare Stream or an FFmpeg transcoding service without evidence that its recurring/storage/compute cost is justified. |
| 39 | + - Changing the attachment data model or playback UI. |
| 40 | + |
| 41 | +## Decisions |
| 42 | + |
| 43 | +### Decision: Create a dedicated R2 Standard bucket |
| 44 | + |
| 45 | +Create `hackathonweekly-assets-prod` with an APAC location hint and Standard storage class. |
| 46 | + |
| 47 | +- The 5.387 GiB conservative migration set is below the current 10 GB Standard free tier. |
| 48 | +- R2 direct egress is free, so rare event-day playback does not create bandwidth charges. |
| 49 | +- Infrequent Access is rejected because the free tier does not apply and reads add retrieval charges. |
| 50 | +- The legacy `hackweek` bucket remains untouched. |
| 51 | + |
| 52 | +### Decision: Store video directly in R2 without transcoding |
| 53 | + |
| 54 | +Video traffic is concentrated around competition day and is otherwise close to zero. At the measured retained footprint: |
| 55 | + |
| 56 | +- R2 storage remains free under the current Standard allowance. |
| 57 | +- Cloudflare Stream adds a minimum storage purchase based on video minutes even when videos are not watched. |
| 58 | +- Browser-side FFmpeg is unreliable for 100-200 MB mobile uploads, and server-side FFmpeg adds queues, CPU pressure, temporary storage, and another failure path on the busiest day. |
| 59 | + |
| 60 | +Therefore new and historical active videos remain ordinary R2 objects with the current 200 MB upload limit. The custom domain cache and byte-range behavior are verified against representative MP4/MOV files. Before a competition, an operations command may warm the current event's referenced videos. |
| 61 | + |
| 62 | +Reconsider asynchronous transcoding only when retained R2 usage reaches 8 GB, individual videos regularly exceed 200 MB, or playback metrics show an event-day experience problem. |
| 63 | + |
| 64 | +### Decision: Separate upload and read origins |
| 65 | + |
| 66 | +- S3 API endpoint for presigned uploads: `https://<ACCOUNT_ID>.r2.cloudflarestorage.com`. |
| 67 | +- Public read origin: `https://assets.hackathonweekly.com`. |
| 68 | +- Presigned URLs cannot use the custom domain, so public URLs must never be derived from the signed URL origin. |
| 69 | +- Bucket name, API endpoint, public endpoint, and region are deployment configuration. No COS host or bucket remains as a runtime fallback. |
| 70 | + |
| 71 | +### Decision: Restrictive CORS and cache policy |
| 72 | + |
| 73 | +- Allow browser upload/read methods only from `https://hackathonweekly.com` and configured preview/local development origins. |
| 74 | +- Do not enable the public `r2.dev` hostname. |
| 75 | +- Bind R2 directly to `assets.hackathonweekly.com`; do not create a CNAME to `r2.dev`. |
| 76 | +- Uploads use the signed S3 API hostname; the custom domain serves `GET`/`HEAD` only. |
| 77 | +- Enable caching for immutable keys and verify `Range`, `Content-Type`, `ETag`, and CORS behavior for images, documents, audio, MP4, and MOV. |
| 78 | + |
| 79 | +### Decision: Selective migration with an explicit exclusion manifest |
| 80 | + |
| 81 | +1. Export a full COS manifest containing key, size, last-modified timestamp, content type, and ETag. |
| 82 | +2. Export every current database/repository-referenced COS key. |
| 83 | +3. Define the exclusion set as unreferenced keys matching `events/<eventId>/submissions/`; the audited set contains 297 objects and 11.431 GiB. |
| 84 | +4. Copy the remaining 1,492 objects and about 5.387 GiB to R2 with identical keys and metadata. |
| 85 | +5. Verify destination count/bytes, `HEAD` all 522 known referenced objects, and run full/range reads across representative file types and sizes. |
| 86 | +6. Freeze uploads briefly, regenerate the manifests, copy the final delta, and block cutover if any new key is unclassified. |
| 87 | +7. Switch writes/public URLs to R2 and backfill the 15 database columns containing the old COS hostname. |
| 88 | + |
| 89 | +The exclusion manifest remains in the migration evidence. Excluded objects stay in COS only during the rollback window; when the old bucket is deleted, those abandoned files are deleted too. |
| 90 | + |
| 91 | +### Decision: Observable rollback before Tencent retirement |
| 92 | + |
| 93 | +- Keep COS readable but stop application writes for seven days after cutover. |
| 94 | +- Alert on application responses, database values, or browser requests containing the old COS hostname. |
| 95 | +- Compare asset 404s, upload failures, and video playback errors with the pre-cutover baseline. |
| 96 | +- After seven clean days, revoke the COS storage credentials, disable public access, export a final manifest, and delete the old bucket including the exclusion set. |
| 97 | + |
| 98 | +## Rollout and Rollback |
| 99 | + |
| 100 | +### Rollout gates |
| 101 | + |
| 102 | +1. R2 bucket, custom domain, TLS, CORS, cache rules, and scoped credentials are active. |
| 103 | +2. Source and destination manifests match the 1,492-object inclusion set by key and size. |
| 104 | +3. All 522 known referenced keys return successfully through `assets.hackathonweekly.com`. |
| 105 | +4. MP4/MOV byte-range playback succeeds through the custom domain. |
| 106 | +5. Production and Mainland China probes meet the agreed success-rate threshold. |
| 107 | +6. Database backfill leaves zero old-host references. |
| 108 | +7. Authenticated upload smoke tests pass for image, document, audio, MP4, and MOV. |
| 109 | + |
| 110 | +### Rollback |
| 111 | + |
| 112 | +- Restore Tencent endpoint credentials and the previous public asset origin. |
| 113 | +- Reverse the hostname-only database replacement from the retained pre-change report. |
| 114 | +- Leave R2 data intact for diagnosis. |
| 115 | +- Rollback remains available until COS credentials and public access are revoked after the seven-day observation period. |
| 116 | + |
| 117 | +## Risks / Trade-offs |
| 118 | + |
| 119 | +- Ordinary R2 buckets are not located in Mainland China, and R2 custom domains are not supported inside Cloudflare China Network. APAC placement is only a best-effort hint, so domestic probes are a hard cutover gate. |
| 120 | +- Direct MP4/MOV playback transfers more bytes than adaptive streaming, but current playback frequency is too low for that to create R2 egress cost. The simpler path is more stable. |
| 121 | +- Database reachability cannot prove that an abandoned submission URL was never shared externally. The user has explicitly accepted deletion of abandoned submissions; the manifest preserves an audit trail until COS retirement. |
| 122 | +- Super Slurper supports generic S3-compatible sources, but Tencent COS is not listed among Cloudflare's explicitly tested providers. A test prefix must pass before bulk transfer; a local streaming copier is the fallback. |
| 123 | + |
| 124 | +## References |
| 125 | + |
| 126 | +- [Cloudflare R2 pricing](https://developers.cloudflare.com/r2/pricing/) |
| 127 | +- [R2 public buckets and custom domains](https://developers.cloudflare.com/r2/buckets/public-buckets/) |
| 128 | +- [R2 presigned URLs](https://developers.cloudflare.com/r2/api/s3/presigned-urls/) |
| 129 | +- [R2 Super Slurper](https://developers.cloudflare.com/r2/data-migration/super-slurper/) |
| 130 | +- [Cloudflare China Network product availability](https://developers.cloudflare.com/china-network/reference/available-products/) |
0 commit comments