-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: add some pages and support mermaid
- Loading branch information
Showing
12 changed files
with
1,192 additions
and
148 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
49 changes: 49 additions & 0 deletions
49
docs/dev-guide/backend/Process-of-Submission-&-SelfTest.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
# 处理提交和自测 | ||
|
||
提交和自测是 SASTOJ 中最重要的部分之一。 | ||
|
||
## 提交流程 | ||
|
||
我们的提交和自测的流程是一个很有意思的设计,涉及消息队列、负载均衡、API 回调、缓存与持久化等等。 | ||
|
||
```mermaid | ||
sequenceDiagram | ||
title: Process of Submission & SelfTest | ||
box Exam | ||
participant User | ||
participant Gateway | ||
end | ||
box SAST | ||
participant Judge-middleware | ||
participant Judge | ||
end | ||
activate User | ||
User ->>+ Gateway: submission | ||
Gateway ->>+ Judge-middleware: submission & callback token by rabbitMQ | ||
Gateway ->>- User: submission uuid | ||
User --)+ Gateway: submission uuid (polling) | ||
alt is gojudge | ||
Judge-middleware ->>+ Judge: code & cases | ||
else is rsjudge | ||
Judge-middleware ->>+ Judge: code & case version | ||
opt | ||
Judge ->> Judge-middleware: case version | ||
Judge-middleware ->> Judge: cases | ||
end | ||
end | ||
Judge--) Judge-middleware: status | ||
Judge-middleware ->> Gateway: submission status | ||
Gateway --) User: submission details | ||
Judge --)- Judge-middleware: case results | ||
Judge-middleware ->>- Gateway: submission result | ||
Gateway --)- User: submission details | ||
deactivate User | ||
``` | ||
|
||
当网关或者 `user/contest` 接收到提交请求后,会生成一个临时的 `UUID`,作为这个提交的标识符。这个 `UUID` 会进入缓存,但是不会存入数据库。当评测中间价从**消息队列**中获得到提交后,会将其存入数据库,并在 Redis 中存入 `UUID -> submission`。注意,这个时候 `submission` 已经获取到了 `submission_id`。此外,`callback` 函数中,也需要用到 UUID 作为标识符。 | ||
|
||
## Redis 存储 | ||
|
||
Key: `submission:{userID}:{UUID}` | ||
|
||
Value: `{submission}` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
# 网关开发 | ||
|
||
网关是 SASTOJ 应对学校网络问题最主要的手段,它通过横向扩容减小主服务器的网络压力,同时可以优化考场考生的体验。为了方便在考场机房(Windows 7)部署,我们需要尽可能减小网关中的服务数量,同时降低对平台的要求。目前 Gateway 包括 Gateway Service, Frontend, Static Resource, Web Server 四个部分。 | ||
|
||
```mermaid | ||
flowchart LR | ||
A[Web Server] --> Frontend | ||
A --> B[Static Resource] | ||
A --> C[Gateway Service] | ||
``` | ||
|
||
## 网关服务 | ||
|
||
网关服务是内部是 `user/gateway` 服务,它是 `user/contest` 的同 API 实现,这么实现可以让前端在开发的时候可以通过同一的 API 请求到一样的资源,无论是否有部署 Gateway 服务。同时也让我们的资源有了内部纵向分发的能力。 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
# 同步缓存 | ||
|
||
在比赛的过程中,我们所有的请求都是打到网关服务器上面,且所有 Get 请求都应该命中它的 Cache。为了达成这一目的,我们需要进行缓存的同步。这是一个简化的 “边缘存储” 的结构,Gateway 除了需要同步比赛信息,还需要主动或者被动地获取特定提交的回传。就目前而言,我们不打算让网关之间直接相互通讯。同时,由于 Gateway 部署在 Windows PC 上面,所以我们无法使用例如 `Redis` 一类的内存数据库,而只能使用 `localcache`。 | ||
|
||
## 启动服务前 | ||
|
||
这需要我们在启动服务之前,我们需要在 `user/contest` 服务中注册并从后端服务器同步当天的所有 `contest` 数据,以及它们所有的 `problem` 数据。 | ||
|
||
## 比赛开始后 | ||
|
||
在比赛开始后,我们会涉及到 `submission`, `submissionCase`, `rank` 等数据的变更 | ||
|
||
## 提交 & 测试点 | ||
|
||
这是在用户发起提交请求之后得到的数据,由于评测需要时间,这个部分去程使用 mq、回程使用grpc,以确保不会发生数据丢失,也保证了用户使用体验。 | ||
|
||
## 排名 | ||
|
||
Rank 信息是在后端完成计算,由 Gateway 拉取信息。 | ||
|
||
## 比赛 / 题目的信息更新(future) | ||
|
||
如果后端对部分信息进行了更新,我们也需要同步这些数据,以保证缓存的一致性。 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
# 简介 | ||
|
||
这个项目最大的意义在于“不触发学校网关警报的前提下,完成程序设计竞赛的任务”,从某种意义上说,是已经凉了大半截的 NOJ 的替代品。推测可能是流量过大触发了学校网关警报,目前的解决方案是减少发包次数和包的体积,从结果角度出发,与抗DDOS的解决方案有异曲同工之处。 | ||
|
||
## 基础需求 | ||
|
||
- 高并发下不被学校网关识别为DDOS攻击 (可能) 导致比赛中断 | ||
- 保证比赛过程中服务的稳定性,不能出现考试过程中服务重启的情况 | ||
- 满足IOI赛制 | ||
- 与 `go-judge` 通讯完成判题 | ||
- 能够查看比赛排行 | ||
|
||
## 基础功能 (Basic OJ) | ||
|
||
- [x] 用户管理和题库管理 | ||
- [x] 查看题面和比赛 | ||
- [x] 在线运行代码和提交代码(使用http与sandbox通讯) | ||
- [x] 查看分数和运行结果 | ||
|
||
## 高级功能 (SASTOJ) | ||
|
||
- [x] 使用grpc与sandbox通讯 | ||
- [ ] 对接口限流 | ||
- [x] 使用消息队列优化判题请求 | ||
|
||
```mermaid | ||
flowchart LR | ||
subgraph Gateway | ||
end | ||
subgraph MQ | ||
end | ||
subgraph Backend | ||
direction TB | ||
Consumer | ||
subgraph Consumer | ||
gojudge-mid | ||
rsjudge-mid | ||
end | ||
end | ||
subgraph gojudge | ||
end | ||
subgraph rsjudge | ||
end | ||
Gateway -->|Submission/ Self-Test| MQ | ||
MQ -->|When a judge-mid is free| Consumer | ||
Gateway <-->|Contests / Rank / Problems / Submissions ...| Backend | ||
gojudge-mid --> gojudge | ||
rsjudge-mid --> rsjudge | ||
Consumer --> gojudge-mid | ||
Consumer --> rsjudge-mid | ||
``` | ||
|
||
- [x] 适配IOI赛制 | ||
- [ ] 实时排行榜 | ||
- [ ] 系统监控 | ||
- [ ] 交互/special judge | ||
|
||
## 架构设计 | ||
|
||
### 独立 CDN | ||
|
||
使用CDN将题面和静态文件从大活服务器分离,减少从大活离开的、重复的大包 | ||
|
||
### 分布式网关 | ||
|
||
将大量访问的GET请求从原服务抽离(如:Rank, Contest),减少大活的吞吐 | ||
|
||
## 参考资料 | ||
|
||
**NOJ (PHP)** | ||
|
||
- <https://noj.njupt.edu.cn/> | ||
- <https://github.com/ZsgsDesign/NOJ> | ||
|
||
**Hydro (NodeJS)** | ||
|
||
- <https://github.com/hydro-dev/Hydro> | ||
|
||
**QDUOJ (Python)** | ||
|
||
- <https://github.com/QingdaoU/OnlineJudge> | ||
|
||
**UOJ (PHP+CPP)** | ||
|
||
- <https://github.com/UniversalOJ/UOJ-System> | ||
|
||
**CSGOJ** | ||
|
||
- <https://github.com/CSGrandeur/CSGOJ> |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.