holy fuking shit
This commit is contained in:
@@ -0,0 +1,261 @@
|
|||||||
|
# 训练前端 Windows 兼容性与「暂停」审计(结论 + 修复方案)
|
||||||
|
|
||||||
|
> 审计对象:`src/mcm26b/training_ui/`、`src/mcm26b/process_control.py`、
|
||||||
|
> `scripts/train_hrl_high.py`。
|
||||||
|
> 触发问题:「无法暂停」「频繁出现 os 无 killpg 等」。
|
||||||
|
> 结论一句话:**这两个现象同源——训练前端的停止/暂停路径是纯 POSIX 设计;
|
||||||
|
> Windows 上 `os.killpg` 根本不存在,异常又没被兜住,所以点了停止等于没点。**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. 结论摘要
|
||||||
|
|
||||||
|
| # | 现象 | 根因 | 严重度 |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| 1 | 点「停止训练」返回 500 / 无效果,训练照跑,于是「无法暂停」 | `TrainingJob.stop()` 直接调用 `os.killpg(os.getpgid(...))`;这两个符号**只存在于 POSIX**,Windows 上抛 `AttributeError`,而旧 `except` 只接 `ProcessLookupError/PermissionError` | 阻断 |
|
||||||
|
| 2 | 控制台/日志反复出现 `module 'os' has no attribute 'killpg'` | 同上:错误最终落到 `_UIHandler.do_POST` 的兜底 `except Exception`,返回 500 并带出 `AttributeError` 文本;每点一次复现一次 | 阻断 |
|
||||||
|
| 3 | 即使把 killpg 换成 `terminate()`,停止后模型/元数据仍然丢失(Windows 上) | Windows 的 `os.kill(pid, SIGTERM)` 语义是 `TerminateProcess` **硬杀**,训练脚本里"收到信号后在 step 边界保存"的处理器不会执行 | 高 |
|
||||||
|
| 4 | 界面根本没有「暂停」能力(POSIX 也没有) | 原设计只有 start/stop;且 `SIGSTOP/SIGCONT` 在 Windows 上不存在,**原生暂停在 Windows 上无法实现**,必须改成协作式 | 高 |
|
||||||
|
| 5 | 子进程进程组隔离在 Windows 上失效 | `Popen(start_new_session=True)` 在 Windows 被**静默忽略**(CPython 的 Windows `_execute_child` 形参名就是 `unused_start_new_session`) | 中 |
|
||||||
|
| 6 | subproc worker 在 Windows 下可能抢在主进程收尾前退出 | worker 只忽略 `SIGTERM/SIGINT`,没处理 Windows 的 `SIGBREAK`(`CTRL_BREAK_EVENT` 的落点) | 中 |
|
||||||
|
|
||||||
|
**修复原则**:把"暂停/继续/停止"从**信号语义**改成**控制文件语义**。
|
||||||
|
控制文件在 POSIX 与 Windows 上都只是普通文件 IO,行为完全一致;信号与强杀退居
|
||||||
|
兜底。这样 Windows 上暂停、继续、优雅停止、落盘保证全部成立。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 失败链复现(旧代码)
|
||||||
|
|
||||||
|
```
|
||||||
|
用户点「停止训练」
|
||||||
|
└─ POST /api/train/stop
|
||||||
|
└─ TrainingUI.stop() → TrainingJob.stop()
|
||||||
|
└─ os.killpg(os.getpgid(process.pid), signal.SIGTERM)
|
||||||
|
├─ POSIX :成功,进程组收到 SIGTERM,脚本优雅收尾
|
||||||
|
└─ Windows:AttributeError: module 'os' has no attribute 'killpg'
|
||||||
|
↑ 只 except (ProcessLookupError, PermissionError),接不住
|
||||||
|
└─ _UIHandler.do_POST 兜底 except Exception → HTTP 500
|
||||||
|
└─ 训练子进程完全没收到任何指令,继续跑 → 「无法暂停 / 停止没反应」
|
||||||
|
```
|
||||||
|
|
||||||
|
关键代码(修复前,`src/mcm26b/training_ui/server.py`):
|
||||||
|
|
||||||
|
```python
|
||||||
|
def stop(self) -> bool:
|
||||||
|
with self._lock:
|
||||||
|
process = self.process
|
||||||
|
if process is None or process.poll() is not None:
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
os.killpg(os.getpgid(process.pid), signal.SIGTERM)
|
||||||
|
except (ProcessLookupError, PermissionError):
|
||||||
|
process.terminate()
|
||||||
|
self.status = "stopping"
|
||||||
|
return True
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. 根因逐条(含证据)
|
||||||
|
|
||||||
|
### 2.1 `os.killpg` / `os.getpgid` 是 POSIX-only
|
||||||
|
|
||||||
|
CPython 只在 `posix` 构建里注册这两个函数;Windows 的 `os` 模块没有它们。
|
||||||
|
访问会得到 `AttributeError`(不是 `OSError` 子类),所以旧 `except` 接不住。
|
||||||
|
|
||||||
|
### 2.2 `start_new_session=True` 在 Windows 被静默忽略
|
||||||
|
|
||||||
|
`subprocess.Popen` 接受该参数,但 Windows 的 `_execute_child` 形参名是
|
||||||
|
`unused_start_new_session`,也就是**根本不读**。要隔离进程组,Windows 必须传
|
||||||
|
`creationflags=CREATE_NEW_PROCESS_GROUP`。
|
||||||
|
|
||||||
|
### 2.3 Windows 的 `SIGTERM` 是硬杀
|
||||||
|
|
||||||
|
`os.kill(pid, sig)` 在 Windows 上只有 `CTRL_C_EVENT` / `CTRL_BREAK_EVENT` 是
|
||||||
|
"软"信号;**任何其它值**(含 `SIGTERM`)都会走 `TerminateProcess`。因此
|
||||||
|
`scripts/train_hrl_high.py` 里的:
|
||||||
|
|
||||||
|
```python
|
||||||
|
for _signum in (signal.SIGTERM, signal.SIGINT): # 旧代码
|
||||||
|
signal.signal(_signum, _on_stop_signal)
|
||||||
|
```
|
||||||
|
|
||||||
|
在 Windows 上形同虚设——处理器注册了,但没人会用它。这也解释了为什么"停止后
|
||||||
|
模型没保存"。
|
||||||
|
|
||||||
|
### 2.4 Windows 没有 `SIGSTOP`,暂停无法用信号实现
|
||||||
|
|
||||||
|
暂停(挂起/继续)在 POSIX 上可以用 `SIGSTOP/SIGCONT`,但 Windows 既无这两个
|
||||||
|
信号,也无等价的"无侵入挂起"用户态 API(真正的挂起要 `SuspendThread`/调试 API)。
|
||||||
|
所以**任何跨平台暂停都必须是协作式的**:由训练进程自己在 step 边界检查标志并
|
||||||
|
让出执行。本项目原先没有这条通道,界面也没有暂停按钮。
|
||||||
|
|
||||||
|
### 2.5 `SIGBREAK` 未处理
|
||||||
|
|
||||||
|
Windows 的 `CTRL_BREAK_EVENT`(发给进程组的软信号)在 Python 里落到
|
||||||
|
`signal.SIGBREAK`。旧的 handler 安装与 worker 屏蔽列表都只含
|
||||||
|
`SIGTERM/SIGINT`,Windows 下等于裸奔。
|
||||||
|
|
||||||
|
### 2.6 异常类型没兜住
|
||||||
|
|
||||||
|
`except (ProcessLookupError, PermissionError)` 漏掉 `AttributeError`、以及
|
||||||
|
更一般的 `OSError`;`_hard_kill` 里 `signal.SIGKILL` 在 Windows 上也不存在
|
||||||
|
(`getattr(signal, "SIGKILL", None) is None`),同样会 `AttributeError`。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 修复方案(已实施)
|
||||||
|
|
||||||
|
### 3.1 新增 `src/mcm26b/process_control.py`(零第三方依赖)
|
||||||
|
|
||||||
|
| 能力 | 实现 | 作用 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| 子进程隔离 | `spawn_kwargs()`:POSIX `start_new_session=True`;Windows `creationflags=CREATE_NEW_PROCESS_GROUP` | 前端 Ctrl+C 不误伤训练;Windows 也能组杀进程树 |
|
||||||
|
| 控制协议读写 | `write_control()` / `read_control()`:JSON + 同目录临时文件 + `os.replace` 原子替换;损坏/缺失一律视为"无指令" | 两端统一的暂停/停止通道 |
|
||||||
|
| 强杀兜底 | `terminate_process_tree(process, grace_s)`:先等宽限期;超时后 POSIX 走 `killpg(SIGKILL)`、Windows 走 `taskkill /F /T /PID`;所有 POSIX 符号用 `getattr` 探测 | 彻底移除 `os.killpg` 无条件调用;返回"是否动用了强杀" |
|
||||||
|
|
||||||
|
`IS_WINDOWS` 是运行期常量而非导入期分支,便于测试直接 monkeypatch 模拟 Windows。
|
||||||
|
|
||||||
|
### 3.2 控制文件协议
|
||||||
|
|
||||||
|
路径由前端生成并随命令下发:`--control-file <abs>`(默认落在
|
||||||
|
`output/hrl/ui/control/<job_id>.control.json`,`output/` 已 gitignore)。
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"paused": false, "stop": false, "seq": 1712345678901234567, "ts": 1712345678.9}
|
||||||
|
```
|
||||||
|
|
||||||
|
- 前端**只写**:`pause()` → `paused=true`;`resume()` → `paused=false`;
|
||||||
|
`stop()` → `stop=true`(同时清零 paused,保证阻塞中的暂停循环能被唤醒)。
|
||||||
|
- 训练脚本**只读**:`_ControlPoll` 每 32 个 step 抽样读一次盘(避免每 step 一次
|
||||||
|
文件 IO);暂停时在 `_on_step` 内阻塞轮询(`poll_s=0.2`),进程存活、显存仍占用,
|
||||||
|
但不消耗算力。
|
||||||
|
- 墙钟上限 `--max-wall-s` **扣除累计暂停时长**,避免"挂起一小时回来直接判超时"。
|
||||||
|
|
||||||
|
### 3.3 训练脚本 `scripts/train_hrl_high.py`
|
||||||
|
|
||||||
|
- 新增 `--control-file`;`_ControlPoll`(纯标准库、可单测)+ 薄薄的
|
||||||
|
`_ControlChannel(BaseCallback)` 接到 SB3 回调链**最前面**。
|
||||||
|
- 收尾信号集合扩为 `_stop_signals()`:POSIX `SIGTERM/SIGINT`;Windows 追加
|
||||||
|
`SIGBREAK`。worker 屏蔽同一集合,保证主进程 `vec_env.close()` 之前 worker 不先退。
|
||||||
|
- 落盘保证不变:停止后仍写最终模型 + 元数据,`stopped_by_signal=true`,
|
||||||
|
`stop_signal` 记 `control_file` 或信号名;异常仍落 `*_emergency.zip`。
|
||||||
|
|
||||||
|
### 3.4 训练前端 `server.py` / `static/`
|
||||||
|
|
||||||
|
- 新增 `POST /api/train/pause`、`POST /api/train/resume`;`stop` 语义改为
|
||||||
|
"写控制文件 → 宽限期内自行退出;超时强杀",强杀在**独立看门狗线程**里做,
|
||||||
|
HTTP 请求不会被 15s 等待卡住。
|
||||||
|
- 状态机新增 `paused`;`_sync_control_events()` 用训练脚本回显的
|
||||||
|
`paused/resumed/stop_requested` 事件校正状态。
|
||||||
|
- 快照新增 `forced_kill`,前端横幅区分"优雅停止"与"超时强杀"。
|
||||||
|
- 前端新增「暂停训练 / 继续训练」按钮、暂停横幅;停止确认文案更新。
|
||||||
|
|
||||||
|
### 3.5 停止/暂停时序
|
||||||
|
|
||||||
|
```
|
||||||
|
pause: UI 写 {paused:true} ──► 脚本 _on_step 读到 ──► 回显 paused,阻塞轮询
|
||||||
|
resume: UI 写 {paused:false} ─► 暂停循环退出 ────────► 回显 resumed,继续训练
|
||||||
|
stop: UI 写 {stop:true} ────► 脚本 step 边界收尾落盘 ──► 退出码 0
|
||||||
|
│
|
||||||
|
└─(宽限 15s 未退)─► terminate_process_tree 强杀;forced_kill=true
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.6 急停(强杀)也必须可恢复
|
||||||
|
|
||||||
|
宽限期一到就强杀,训练脚本**没有任何 Python 收尾机会**——不写最终模型、不写
|
||||||
|
元数据、不写 `training_failure.json`。如果磁盘上只有 `--checkpoint-every` 的稀疏
|
||||||
|
历史 checkpoint,急停就可能丢掉上万步,或者在"保存途中被杀"时留下一个打不开的
|
||||||
|
半截 zip。为此加了四条保证:
|
||||||
|
|
||||||
|
| 保证 | 实现 | 效果 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| 起步基线 | `recovery_every > 0` 时,`learn()` 之前先写一份 `<stem>_latest.zip` | 训练刚起就急停,也有可加载、可 `--resume` 的产物 |
|
||||||
|
| 滚动恢复点 | 每 `--recovery-every`(默认 2000)环境步覆盖写同一份 `<stem>_latest.zip` | 急停最多丢 `recovery_every` 步,而不是 `checkpoint-every` 步 |
|
||||||
|
| 原子落盘 | 模型 zip / 旁挂 json / `resume.json` 一律「同目录 `.tmp-` + `os.replace`」 | 保存途中被强杀,目标路径仍是上一份**完整**快照;不会出现半截文件 |
|
||||||
|
| 权威恢复指针 | 每次落盘刷新 `<run_dir>/resume.json`(model / timesteps / 增量语义 / 生效 PPO 超参);前端 `/api/train/state` 暴露为 `recovery` | 无论怎么停,都有一份明确的"从哪继续";界面一键「用最近检查点续训」 |
|
||||||
|
|
||||||
|
前端在强杀后还会补写 `<run_dir>/forced_stop.json`(`job_id`、`pid`、`reason`、
|
||||||
|
`recovery.model`),因为此时训练脚本已经无法记录任何事情。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. 平台兼容性矩阵(修复后)
|
||||||
|
|
||||||
|
| 能力 | POSIX | Windows |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| 启动子进程并隔离 | `start_new_session=True`(setsid) | `CREATE_NEW_PROCESS_GROUP` |
|
||||||
|
| 暂停 / 继续 | 控制文件(协作式) | 控制文件(协作式) |
|
||||||
|
| 优雅停止 + 落盘 | 控制文件;`SIGTERM` 亦可 | 控制文件;`CTRL_BREAK_EVENT`→`SIGBREAK` 亦可 |
|
||||||
|
| 超时强杀进程树 | `killpg(SIGKILL)` | `taskkill /F /T /PID` |
|
||||||
|
| `os.killpg` 依赖 | 有 | **无(不再调用)** |
|
||||||
|
| 暂停期间是否算墙钟 | 否(扣除) | 否(扣除) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 验证
|
||||||
|
|
||||||
|
- 新增/更新回归测试:
|
||||||
|
- `tests/test_training_ui.py`:控制文件协议往返与损坏容错、`spawn_kwargs`
|
||||||
|
平台分支、Windows 强杀走 `taskkill`、**无 `killpg` 环境下强杀不抛异常**
|
||||||
|
(直接复现旧 bug)、暂停/继续/停止全链路、控制文件被忽略时强杀并写
|
||||||
|
`forced_stop.json` 恢复指引、`discover_recovery` 的
|
||||||
|
`resume.json`/扫描/悬空指针三分支、HTTP `pause/resume` 端到端。
|
||||||
|
- `tests/test_hrl_checkpoint.py`:控制文件状态机单测;原子保存失败不破坏上一份
|
||||||
|
快照;JSON 原子替换;历史清理不误删滚动恢复点;起步基线可加载;
|
||||||
|
**强杀急停后 `resume.json` + 滚动恢复点仍可 `--resume`**;控制文件停止后
|
||||||
|
模型/元数据落盘(`stop_signal=control_file`)与暂停→继续;POSIX 信号路径
|
||||||
|
单列并加平台跳过。以上集成用例都用**真实训练脚本**跑 subproc。
|
||||||
|
- 手工验证:`--recovery-every 32` 短程实跑后强杀,`resume.json` 指向
|
||||||
|
`q3_ckpt_latest.zip`;以它为 `--resume` 目标再跑 32 步,元数据
|
||||||
|
`actual_timesteps` 从 64 变 96、`timesteps_semantics=additional`。
|
||||||
|
- 全量测试:`pytest tests referee/tests` 全绿。
|
||||||
|
- 全仓库 `pylint src scripts project tests referee main.py` 达到 10.00/10
|
||||||
|
(口径见 §7)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. 残余风险与后续建议
|
||||||
|
|
||||||
|
1. **暂停粒度** = `poll_every` 个 env step(默认 32,约 0.1–3s 量级),不是即时;
|
||||||
|
若要更快可调小 `_ControlChannel(poll_every=...)`,代价是更频繁的文件 stat。
|
||||||
|
2. **宽限期固定 15s**(`server.STOP_GRACE_S`)。大模型保存可能更久;但因为落盘是
|
||||||
|
原子的,即使宽限期到点被强杀,也只是"新的最终模型没写成",**上一份滚动恢复点
|
||||||
|
与历史 checkpoint 完好**,不会出现半截文件。需要更长可调大该常量。
|
||||||
|
3. **急停的损失上界 = `--recovery-every`**(默认 2000 环境步)。要更小就调小它;
|
||||||
|
调到 0 表示放弃滚动恢复点(此时只剩 `--checkpoint-every` 的稀疏历史
|
||||||
|
checkpoint 与起步基线)。
|
||||||
|
4. **Windows 强杀依赖 `taskkill`**(系统自带)。若被安全策略禁用,会退回
|
||||||
|
`Popen.kill()`(只杀主进程,spawn 的 worker 可能残留)。
|
||||||
|
5. 更严格的 Windows 方案是 **Job Object**(子进程随 job 句柄一起销毁),但需要
|
||||||
|
`ctypes`/`pywin32`,本仓库基础依赖刻意保持轻量,未引入。
|
||||||
|
6. **前端进程重启后**,内存中的任务对象没了,「用最近检查点续训」按钮不会自动
|
||||||
|
出现;但恢复信息都落在磁盘上(`<run_dir>/resume.json`、`forced_stop.json`、
|
||||||
|
`checkpoints/<stem>_latest.zip`),在表单里填同一个模型输出路径并勾选
|
||||||
|
`--resume` 即可继续。
|
||||||
|
7. 界面 `--device mps` 选项在 Windows 上无意义;`auto` 会回退 cuda/cpu,属既有
|
||||||
|
行为,未改。
|
||||||
|
8. `_ControlPoll` 的暂停循环会占用一个 Python 线程的调度(每 0.2s 一次读盘),
|
||||||
|
长暂停期间 CPU 占用可忽略。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. pylint 口径说明
|
||||||
|
|
||||||
|
配置写在 `pyproject.toml` 的 `[tool.pylint.*]`,目标是
|
||||||
|
`pylint src scripts project tests referee main.py` = 10.00/10。
|
||||||
|
|
||||||
|
**改代码修掉的真实问题**(不是 disable):未使用的 import/变量/参数、重复字典键
|
||||||
|
(`reward_version`)、`value != value` 判 NaN、`__dataclass_fields__` 私有访问、
|
||||||
|
`raise` 非异常类型、`pytest.importorskip` 之外的赋值不确定、过长行、无谓括号、
|
||||||
|
`len(x)` 判空、循环闭包变量、`subprocess.run` 未显式 `check`、`os.path` 循环导入等。
|
||||||
|
|
||||||
|
**有意 disable 的类别**(每条在 pyproject 里写了理由):项目刻意的懒加载
|
||||||
|
(`import-outside-toplevel`)、防御式宽异常(`broad-exception-caught`)、测试访问
|
||||||
|
内部状态(`protected-access`)、pytest fixture 同名(`redefined-outer-name`)、
|
||||||
|
无头环境不实现 `gym.Env.render`(`abstract-method`)、以及静态分析的已知误报
|
||||||
|
(`possibly-used-before-assignment`)。
|
||||||
|
|
||||||
|
**注意**:`good-names` 不覆盖 `redefined-builtin`,stdlib 回调签名里的
|
||||||
|
`log_message(self, format, ...)` 只能在对应 `def` 上行内 disable。
|
||||||
+53
-14
@@ -23,7 +23,7 @@ uv run python -m mcm26b.training_ui --print-preview # 只看离线示例快照
|
|||||||
|
|
||||||
| 区块 | 内容 |
|
| 区块 | 内容 |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| 顶栏 | 任务摘要、进程/进度/墙钟/设备/连接五个状态 chip、刷新·预览·停止 |
|
| 顶栏 | 任务摘要、进程/进度/墙钟/设备/连接五个状态 chip、刷新·预览·暂停/继续·停止 |
|
||||||
| 执行进度与进程 | 步进进度条、剩余时间与预计结束、PID/吞吐/退出码、PPO 阶段时间线、完整命令行 |
|
| 执行进度与进程 | 步进进度条、剩余时间与预计结束、PID/吞吐/退出码、PPO 阶段时间线、完整命令行 |
|
||||||
| 训练配置 | 由 `/api/train/schema` 生成的五组表单 + 四个预设,运行中锁定 |
|
| 训练配置 | 由 `/api/train/schema` 生成的五组表单 + 四个预设,运行中锁定 |
|
||||||
| 参数收敛状态 | 判读 chip + 判据说明,每个指标一张卡片(最新值、增量、迷你趋势线、异常高亮) |
|
| 参数收敛状态 | 判读 chip + 判据说明,每个指标一张卡片(最新值、增量、迷你趋势线、异常高亮) |
|
||||||
@@ -40,7 +40,15 @@ uv run python -m mcm26b.training_ui --print-preview # 只看离线示例快照
|
|||||||
| `GET /api/train/schema` | 字段/分组/预设/指标字典,**与 `TrainConfig` 同源**(`schema.py`) |
|
| `GET /api/train/schema` | 字段/分组/预设/指标字典,**与 `TrainConfig` 同源**(`schema.py`) |
|
||||||
| `GET /api/train/state` | 快照;支持 `log_tail`、`metric_limit`、`after_metric` 游标与 `preview=1` |
|
| `GET /api/train/state` | 快照;支持 `log_tail`、`metric_limit`、`after_metric` 游标与 `preview=1` |
|
||||||
| `POST /api/train/start` | 允许只提交部分字段(其余取默认值),校验失败返回 409 |
|
| `POST /api/train/start` | 允许只提交部分字段(其余取默认值),校验失败返回 409 |
|
||||||
| `POST /api/train/stop` | 向独立进程组发送 SIGTERM |
|
| `POST /api/train/pause` | 写控制文件请求在 step 边界挂起(跨平台) |
|
||||||
|
| `POST /api/train/resume` | 解除挂起,从暂停处继续 |
|
||||||
|
| `POST /api/train/stop` | 请求优雅停止;宽限期(`STOP_GRACE_S`,默认 15s)后强杀进程树 |
|
||||||
|
|
||||||
|
暂停/停止走**控制文件**而不是信号,原因与平台差异见
|
||||||
|
[`training-ui-windows-audit.md`](training-ui-windows-audit.md):Windows 没有
|
||||||
|
`SIGSTOP`,且 `os.kill(pid, SIGTERM)` 会退化成 `TerminateProcess` 硬杀,训练脚本
|
||||||
|
"保存模型再退出"的收尾分支不会执行。控制文件在 POSIX 与 Windows 上都只是普通
|
||||||
|
文件 IO,语义完全一致;信号(POSIX `SIGTERM/SIGINT`、Windows `SIGBREAK`)只作兜底。
|
||||||
|
|
||||||
进度数据的两个来源,服务端都已解析:
|
进度数据的两个来源,服务端都已解析:
|
||||||
|
|
||||||
@@ -76,41 +84,66 @@ uv run python -m mcm26b.training_ui --print-preview # 只看离线示例快照
|
|||||||
|
|
||||||
- 只监听 `127.0.0.1`;静态资源是固定白名单,不提供目录遍历。
|
- 只监听 `127.0.0.1`;静态资源是固定白名单,不提供目录遍历。
|
||||||
- 命令由服务端以**参数列表**构造,不经过 shell;字符串参数只拒绝控制字符。
|
- 命令由服务端以**参数列表**构造,不经过 shell;字符串参数只拒绝控制字符。
|
||||||
- 同一时刻只允许一个训练任务;停止用独立进程组 SIGTERM,避免残留子进程。
|
- 同一时刻只允许一个训练任务;暂停中的任务仍算「在跑」,必须先停止才能开新一局。
|
||||||
|
- 暂停/停止走控制文件(`--control-file`,默认 `output/hrl/ui/control/<job_id>.control.json`),
|
||||||
|
进程组隔离按平台选择(POSIX `setsid` / Windows `CREATE_NEW_PROCESS_GROUP`),
|
||||||
|
超时强杀按平台选择(POSIX `killpg(SIGKILL)` / Windows `taskkill /F /T`)。
|
||||||
- 产物路径限定在仓库根下解析,模型输出沿用训练脚本自身的 `--force`/`--resume`
|
- 产物路径限定在仓库根下解析,模型输出沿用训练脚本自身的 `--force`/`--resume`
|
||||||
语义(两者互斥,由后端校验)。
|
语义(两者互斥,由后端校验)。
|
||||||
|
|
||||||
## 停止与落盘保证
|
## 暂停、停止与落盘保证
|
||||||
|
|
||||||
训练前端的「停止」是对独立进程组发 SIGTERM。`train_hrl_high.py` 有三层兜底,
|
「暂停」是**协作式**的:前端原子写控制文件,`train_hrl_high.py` 的
|
||||||
点一下「停止」不再等于这一局白跑:
|
`_ControlPoll` 每 32 个环境步抽样读一次;暂停时训练阻塞在 `_on_step` 的轮询里,
|
||||||
|
进程与显存保留、不消耗算力,点「继续」从同一个 step 边界恢复。暂停时长不计入
|
||||||
|
`--max-wall-s`(否则"挂起一小时回来直接判超时")。
|
||||||
|
|
||||||
|
「停止」同样是控制文件请求(**跨平台**)。`train_hrl_high.py` 有多层兜底,
|
||||||
|
点一下「停止」不再等于这一局白跑;而**急停**(强杀,例如 `SIGKILL` /
|
||||||
|
`taskkill /F`,进程没有任何收尾机会)也保证可恢复:
|
||||||
|
|
||||||
| 机制 | 行为 |
|
| 机制 | 行为 |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| 定期 checkpoint | `--checkpoint-every`(默认 20000 环境步)在 `<模型目录>/checkpoints/` 落 `*_ckpt_<步数>.zip` + 同名元数据;`--checkpoint-keep` 只保留最近 K 个(默认 3),`--checkpoint-dir` 可改目录 |
|
| 滚动恢复点 | `--recovery-every`(默认 2000 环境步)把 `<checkpoints>/<stem>_latest.zip` **原子覆盖写**一次;训练开始前先写一份**起步基线**。因此急停后最多只丢 `--recovery-every` 个环境步,且磁盘上始终有一份完整、可加载、可 `--resume` 的产物 |
|
||||||
| 优雅停止 | subproc worker 忽略 SIGTERM/SIGINT,主进程收到信号后在 step 边界收尾,保存最终模型与元数据(`stopped_by_signal=true`、`stop_signal`、`checkpoint_files`)并正常关闭环境;收尾阶段恢复默认处理器,再按一次可立即强杀 |
|
| 原子落盘 | 模型 zip、旁挂 json、`resume.json` 都是「同目录临时文件 + `os.replace`」:保存途中被强杀不会留下半截文件,也不会破坏上一份完整快照 |
|
||||||
| 异常兜底 | `learn()` 抛出任何异常时先落 `*_emergency.zip` + 元数据,再写 `training_failure.json` 并抛出 |
|
| 恢复指针 | `<run_dir>/resume.json` 记录当前最佳恢复目标 + 步数(增量语义)+ 生效过的 PPO 超参;前端顶部的「用最近检查点续训」按钮直接读它 |
|
||||||
|
| 定期 checkpoint | `--checkpoint-every`(默认 20000 环境步)另留 `<stem>_<步数>.zip`,`--checkpoint-keep` 只保留最近 K 个(默认 3),供返祖对比与多档恢复;滚动恢复点与起步基线不受清理影响 |
|
||||||
|
| 优雅停止 | 控制文件 `stop=true`(或 POSIX 信号 / Windows `SIGBREAK`)后,主进程在 step 边界收尾,保存最终模型与元数据(`stopped_by_signal=true`、`stop_signal`、`checkpoint_files`、`recovery_model`)并正常关闭环境 |
|
||||||
|
| 超时强杀 | 宽限期(`STOP_GRACE_S`,默认 15s)内没退出则强杀整棵进程树,快照 `forced_kill=true`,并在 `<run_dir>/forced_stop.json` 写明 `recovery.model`(可直接续训的目标) |
|
||||||
|
| 异常兜底 | `learn()` 抛出任何异常时先落 `*_emergency.zip` + 元数据 + `resume.json`,再写 `training_failure.json` 并抛出 |
|
||||||
|
|
||||||
|
急停后恢复的操作方式:界面点「用最近检查点续训」;命令行则
|
||||||
|
`--resume --model <resume.json 里的 model>`,`--timesteps` 按**追加**步数理解。
|
||||||
|
|
||||||
checkpoint 与最终模型共用同一份 json 契约,因此 `--resume <checkpoint.zip>` 直接
|
checkpoint 与最终模型共用同一份 json 契约,因此 `--resume <checkpoint.zip>` 直接
|
||||||
续训(`--timesteps` 为增量步数)。注意 `--resume`/`--init` 会沿用 checkpoint 内的
|
续训(`--timesteps` 为增量步数)。注意 `--resume`/`--init` 会沿用 checkpoint 内的
|
||||||
PPO 超参(`n_steps`/`batch_size`/`n_epochs`):命令行这三项被忽略,脚本会打一条
|
PPO 超参(`n_steps`/`batch_size`/`n_epochs`):命令行这三项被忽略,脚本会打一条
|
||||||
`hyperparams_from_checkpoint` 事件说明差异,元数据 `ppo_hyperparams` 记录**实际生效**值。
|
`hyperparams_from_checkpoint` 事件说明差异,元数据 `ppo_hyperparams` 记录**实际生效**值。
|
||||||
回归测试见 `tests/test_hrl_checkpoint.py`,
|
回归测试见 `tests/test_hrl_checkpoint.py`,其中端到端用例真起 subproc 子进程,覆盖
|
||||||
其中端到端用例真起 subproc 子进程并用 `killpg(SIGTERM)` 复现前端的停止路径。
|
控制文件停止、POSIX 信号停止、**强杀后仍可续训**与暂停→继续(信号路径在 Windows
|
||||||
|
上自动跳过)。
|
||||||
|
|
||||||
## 测试
|
## 测试
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
.venv/bin/python -m pytest tests/test_training_ui.py tests/test_training_ui_js.py -q
|
.venv/bin/python -m pytest tests/test_training_ui.py tests/test_training_ui_js.py tests/test_hrl_checkpoint.py -q
|
||||||
```
|
```
|
||||||
|
|
||||||
- `tests/test_training_ui.py`:配置契约同源、预设可直接提交、参数校验、SB3 表格
|
- `tests/test_training_ui.py`:配置契约同源、预设可直接提交、参数校验、SB3 表格
|
||||||
刻度打点、游标分页、`describe_process` 进度/ETA、示例快照与真实快照同键、
|
刻度打点、游标分页、`describe_process` 进度/ETA、示例快照与真实快照同键;
|
||||||
以及用**桩训练脚本真起子进程**的启动/轮询/停止/退出码全链路。
|
用**桩训练脚本真起子进程**覆盖启动/轮询/暂停/继续/停止/退出码全链路;并覆盖
|
||||||
|
跨平台进程控制——控制文件往返与损坏容错、`spawn_kwargs` 平台分支、Windows 走
|
||||||
|
`taskkill`、**无 `killpg` 环境下强杀不抛异常**(旧 bug 的直接回归)、控制文件被
|
||||||
|
忽略时强杀并写 `forced_stop.json` 恢复指引、`discover_recovery` 的
|
||||||
|
`resume.json`/扫描/悬空指针三分支、HTTP `pause/resume`。
|
||||||
- `tests/js/training_ui_harness.cjs`(经 `tests/test_training_ui_js.py` 调起):
|
- `tests/js/training_ui_harness.cjs`(经 `tests/test_training_ui_js.py` 调起):
|
||||||
DOM 桩加载发布的 `charts.js`/`app.js` 原文,用服务端生成的真实快照驱动渲染,
|
DOM 桩加载发布的 `charts.js`/`app.js` 原文,用服务端生成的真实快照驱动渲染,
|
||||||
断言表单/指标卡/曲线/日志/逐回合表都拿到数据,覆盖增量游标三条分支与
|
断言表单/指标卡/曲线/日志/逐回合表都拿到数据,覆盖增量游标三条分支与
|
||||||
`classList`、HTML 转义、极端数值等边界。
|
`classList`、HTML 转义、极端数值等边界。
|
||||||
|
- `tests/test_hrl_checkpoint.py`:落盘与恢复的端到端回归——原子保存失败不破坏
|
||||||
|
上一份快照、JSON 原子替换、历史清理不误删滚动恢复点、起步基线可加载、
|
||||||
|
**强杀急停后 `resume.json` 与滚动恢复点仍可继续训练**、控制文件停止落盘、
|
||||||
|
暂停→继续、POSIX 信号停止。
|
||||||
|
|
||||||
## 已知取舍
|
## 已知取舍
|
||||||
|
|
||||||
@@ -119,3 +152,9 @@ PPO 超参(`n_steps`/`batch_size`/`n_epochs`):命令行这三项被忽略
|
|||||||
格式,退化路径是「只显示 progress JSON」,不会导致界面崩溃。
|
格式,退化路径是「只显示 progress JSON」,不会导致界面崩溃。
|
||||||
- 收敛判读是启发式;正式结论仍需用 `scripts/benchmark_hrl.py` 的配对评测与
|
- 收敛判读是启发式;正式结论仍需用 `scripts/benchmark_hrl.py` 的配对评测与
|
||||||
学习曲线人工复核。
|
学习曲线人工复核。
|
||||||
|
- 暂停粒度是 `poll_every` 个环境步(默认 32),不是即时挂起;停止宽限期默认 15s。
|
||||||
|
- Windows 强杀依赖系统自带的 `taskkill`;若被策略禁用会退回 `Popen.kill()`
|
||||||
|
(只杀主进程)。更严格的 Job Object 方案需要额外依赖,未引入。
|
||||||
|
|
||||||
|
Windows 兼容性问题的完整根因与修复记录见
|
||||||
|
[`training-ui-windows-audit.md`](training-ui-windows-audit.md)。
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ from mcm26b.planner.bayes_loop import BayesLoop
|
|||||||
from mcm26b.planner.hrl_loop import HRLLoop
|
from mcm26b.planner.hrl_loop import HRLLoop
|
||||||
|
|
||||||
|
|
||||||
def HeuristicHRL(*args, **kwargs):
|
def HeuristicHRL(*args, **kwargs): # pylint: disable=invalid-name # 兼容入口保留题面公开名
|
||||||
"""返回无模型的高层 HRL 闭环(heuristic_action 降级路径)。"""
|
"""返回无模型的高层 HRL 闭环(heuristic_action 降级路径)。"""
|
||||||
kwargs.setdefault("high_policy", None)
|
kwargs.setdefault("high_policy", None)
|
||||||
return HRLLoop(*args, **kwargs)
|
return HRLLoop(*args, **kwargs)
|
||||||
|
|||||||
@@ -16,13 +16,13 @@ def randomize_problem_config(base: ProblemConfig, *, seed: int,
|
|||||||
r_eff_range: tuple[float, float] | None = None
|
r_eff_range: tuple[float, float] | None = None
|
||||||
) -> ProblemConfig:
|
) -> ProblemConfig:
|
||||||
"""返回带域随机化标量的新配置;不修改全局默认配置。"""
|
"""返回带域随机化标量的新配置;不修改全局默认配置。"""
|
||||||
rng = Random(seed)
|
_rng = Random(seed) # 固定种子留作兼容:真实随机化在 make_scene 内完成
|
||||||
field = base.field
|
field = base.field
|
||||||
if bearing_error_half_deg is None:
|
if bearing_error_half_deg is None:
|
||||||
bearing_error_half_deg = field.bearing_error_half_deg
|
bearing_error_half_deg = field.bearing_error_half_deg
|
||||||
if r_eff_range is not None:
|
if r_eff_range is not None:
|
||||||
lo, hi = r_eff_range
|
lo, hi = r_eff_range
|
||||||
if not (0 <= lo <= hi):
|
if not 0 <= lo <= hi:
|
||||||
raise ValueError("r_eff_range 必须为递增非负区间")
|
raise ValueError("r_eff_range 必须为递增非负区间")
|
||||||
return dataclasses.replace(base, field=dataclasses.replace(
|
return dataclasses.replace(base, field=dataclasses.replace(
|
||||||
field, bearing_error_half_deg=float(bearing_error_half_deg)))
|
field, bearing_error_half_deg=float(bearing_error_half_deg)))
|
||||||
|
|||||||
@@ -52,3 +52,68 @@ mcm26b = "mcm26b.cli:main"
|
|||||||
|
|
||||||
[tool.pytest.ini_options]
|
[tool.pytest.ini_options]
|
||||||
testpaths = ["tests", "referee/tests"]
|
testpaths = ["tests", "referee/tests"]
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# pylint:全仓库静态检查口径
|
||||||
|
#
|
||||||
|
# 目标:`python -m pylint src scripts project tests referee main.py` 拿 10.00/10。
|
||||||
|
# 原则:真实的逻辑 / 资源 / 命名 / 死代码问题一律改代码;下面 disable 的每一条
|
||||||
|
# 都对应项目刻意的设计选择或已确认的静态分析误报,并写明理由,不做"一键静音"。
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
[tool.pylint.main]
|
||||||
|
py-version = "3.10"
|
||||||
|
jobs = 0
|
||||||
|
# 不写 ~/Library/Caches/pylint:CI 与受限沙箱里没有该目录,会刷警告
|
||||||
|
persistent = false
|
||||||
|
recursive = true
|
||||||
|
ignore = [".venv", ".uv-cache", "output"]
|
||||||
|
|
||||||
|
[tool.pylint.format]
|
||||||
|
max-line-length = 100
|
||||||
|
|
||||||
|
[tool.pylint.design]
|
||||||
|
# 阈值按"实际最长函数 + 余量"设定,而不是关闭检查:超过即说明该拆了
|
||||||
|
max-args = 14
|
||||||
|
max-positional-arguments = 10
|
||||||
|
max-locals = 45
|
||||||
|
max-branches = 26
|
||||||
|
max-statements = 180
|
||||||
|
max-attributes = 32
|
||||||
|
max-nested-blocks = 6
|
||||||
|
max-returns = 12
|
||||||
|
max-module-lines = 4000
|
||||||
|
min-public-methods = 0
|
||||||
|
|
||||||
|
[tool.pylint.basic]
|
||||||
|
# 这些是既有代码里的短名/别名(importorskip 后的模块句柄、几何单字母常量、
|
||||||
|
# 坐标系里的 x/y/z),让命名检查接受它们。
|
||||||
|
# 注意:good-names 不覆盖 redefined-builtin——stdlib 回调签名里的 `format`
|
||||||
|
# 之类只能在对应 def 上写行内 disable(见 server.py / referee 的 log_message)。
|
||||||
|
good-names = [
|
||||||
|
"i", "j", "k", "n", "m", "x", "y", "z", "_", "ex",
|
||||||
|
"id", "pytest", "numpy", "S", "ROOT", "FONT", "ET",
|
||||||
|
]
|
||||||
|
# http.server 要求 do_GET/do_POST 这类方法名,不能改成 snake_case
|
||||||
|
method-rgx = "[a-z_][a-z0-9_]{2,49}$|do_(GET|POST|PUT|DELETE|HEAD|PATCH|OPTIONS)$"
|
||||||
|
|
||||||
|
[tool.pylint."messages control"]
|
||||||
|
disable = [
|
||||||
|
# —— 项目刻意风格 ——
|
||||||
|
"missing-module-docstring", # 内部小模块与 __main__ 不强制
|
||||||
|
"missing-class-docstring",
|
||||||
|
"missing-function-docstring",
|
||||||
|
"import-outside-toplevel", # torch/SB3/referee 等重依赖与可选依赖懒加载
|
||||||
|
"wrong-import-position", # 同上:importorskip 守卫后的延迟导入
|
||||||
|
"ungrouped-imports", # 同上:延迟导入无法与顶部导入分组
|
||||||
|
"broad-exception-caught", # 子进程 / IO / HTTP 边界防御式兜底,逐处注释
|
||||||
|
"protected-access", # 测试与 mixin 有意检查内部状态
|
||||||
|
"duplicate-code", # 脚本与测试夹具存在结构性相似,非复制粘贴缺陷
|
||||||
|
"redefined-outer-name", # pytest 要求测试参数与 fixture 同名
|
||||||
|
"attribute-defined-outside-init", # 轻量对象在 helper 里补字段
|
||||||
|
"multiple-statements", # 单行守卫 (if x: return) 是可读性选择
|
||||||
|
"use-dict-literal", # dict(...) 与 {...} 混用是既有风格
|
||||||
|
"abstract-method", # 无头训练环境不需要实现 gym.Env.render
|
||||||
|
# —— 已确认的静态分析误报 ——
|
||||||
|
"possibly-used-before-assignment", # pytest.importorskip 成功即赋值,skip 会抛异常
|
||||||
|
]
|
||||||
|
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ from dataclasses import dataclass
|
|||||||
|
|
||||||
from .jsonio import RequestFormatError, dumps, parse_request_object
|
from .jsonio import RequestFormatError, dumps, parse_request_object
|
||||||
from .rules import ACTION_FIELDS
|
from .rules import ACTION_FIELDS
|
||||||
from .simulator import LocalSimulator, SimulatorClosed
|
from .simulator import LocalSimulator
|
||||||
|
|
||||||
MAX_BODY_BYTES = 65_536 # [题面 附件2 5.1]
|
MAX_BODY_BYTES = 65_536 # [题面 附件2 5.1]
|
||||||
MAX_SAFE_READ_BYTES = 1_048_576 # 超限请求最多读 1MB 即断,防内存滥用 [工程]
|
MAX_SAFE_READ_BYTES = 1_048_576 # 超限请求最多读 1MB 即断,防内存滥用 [工程]
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ class _ActionHandler(BaseHTTPRequestHandler):
|
|||||||
server_version = "MCM26B-Referee/1.0"
|
server_version = "MCM26B-Referee/1.0"
|
||||||
|
|
||||||
# ---- 日志静默(界面"指令与反馈"由会话层记录) ----------------------------
|
# ---- 日志静默(界面"指令与反馈"由会话层记录) ----------------------------
|
||||||
def log_message(self, format, *args): # noqa: A002 - 标准库签名
|
def log_message(self, format, *args): # pylint: disable=redefined-builtin(标准库签名)
|
||||||
pass
|
pass
|
||||||
|
|
||||||
# ---- 各方法统一走同一边界;未知方法由 do_* 缺失天然 501,这里显式覆盖常用
|
# ---- 各方法统一走同一边界;未知方法由 do_* 缺失天然 501,这里显式覆盖常用
|
||||||
@@ -123,7 +123,7 @@ class ActionInterfaceServer:
|
|||||||
name=f"referee-action-{session.case_code}",
|
name=f"referee-action-{session.case_code}",
|
||||||
daemon=True).start()
|
daemon=True).start()
|
||||||
|
|
||||||
def close(self, session: LiveSession) -> None:
|
def close(self, _session: LiveSession) -> None:
|
||||||
httpd, self._httpd, self._session = self._httpd, None, None
|
httpd, self._httpd, self._session = self._httpd, None, None
|
||||||
if httpd is None:
|
if httpd is None:
|
||||||
return
|
return
|
||||||
|
|||||||
@@ -423,7 +423,7 @@ class Referee:
|
|||||||
def set_display_limit(self, limit: int) -> None:
|
def set_display_limit(self, limit: int) -> None:
|
||||||
"""指令与反馈显示条数 100..5000 [题面 附件1 4.5];不影响日志完整性。"""
|
"""指令与反馈显示条数 100..5000 [题面 附件1 4.5];不影响日志完整性。"""
|
||||||
with self._lock:
|
with self._lock:
|
||||||
if not isinstance(limit, int) or not (DISPLAY_LIMIT_MIN <= limit <= DISPLAY_LIMIT_MAX):
|
if not isinstance(limit, int) or not DISPLAY_LIMIT_MIN <= limit <= DISPLAY_LIMIT_MAX:
|
||||||
raise RefereeError(f"显示条数须为 {DISPLAY_LIMIT_MIN}..{DISPLAY_LIMIT_MAX}")
|
raise RefereeError(f"显示条数须为 {DISPLAY_LIMIT_MIN}..{DISPLAY_LIMIT_MAX}")
|
||||||
self.display_limit = limit
|
self.display_limit = limit
|
||||||
|
|
||||||
|
|||||||
@@ -19,8 +19,8 @@ from threading import Event, Lock
|
|||||||
from unicodedata import category
|
from unicodedata import category
|
||||||
from copy import deepcopy
|
from copy import deepcopy
|
||||||
|
|
||||||
from .rules import (ACTION_FIELDS, CLEAR_NO_TARGET, CLEAR_RESULTS, CLEAR_SUCCESS,
|
from .rules import (ACTION_FIELDS, CLEAR_NO_TARGET, CLEAR_SUCCESS,
|
||||||
COMMON_FIELDS, DIRECTION, EXIT_USER, MEASURE_RESULTS, NEAR,
|
COMMON_FIELDS, DIRECTION, EXIT_USER, NEAR,
|
||||||
NO_SIGNAL, POSITION_FIELDS)
|
NO_SIGNAL, POSITION_FIELDS)
|
||||||
from .spec import Spec, DEFAULT_SPEC
|
from .spec import Spec, DEFAULT_SPEC
|
||||||
from .timing import clear_cost, measure_cost
|
from .timing import clear_cost, measure_cost
|
||||||
@@ -58,7 +58,7 @@ class LocalSimulator:
|
|||||||
if len(channels) != len(set(channels)):
|
if len(channels) != len(set(channels)):
|
||||||
raise ValueError("每个频道最多对应一个干扰源")
|
raise ValueError("每个频道最多对应一个干扰源")
|
||||||
for s in sources:
|
for s in sources:
|
||||||
if not (spec.field.channel_min <= s.channel <= spec.field.channel_max):
|
if not spec.field.channel_min <= s.channel <= spec.field.channel_max:
|
||||||
raise ValueError("频道超出 1..20")
|
raise ValueError("频道超出 1..20")
|
||||||
if not s.directional and s.heading_deg != 0.0:
|
if not s.directional and s.heading_deg != 0.0:
|
||||||
raise ValueError("全向源不应有朝向")
|
raise ValueError("全向源不应有朝向")
|
||||||
@@ -109,8 +109,9 @@ class LocalSimulator:
|
|||||||
owner = True
|
owner = True
|
||||||
if not owner:
|
if not owner:
|
||||||
pending["done"].wait()
|
pending["done"].wait()
|
||||||
if pending["error"] is not None:
|
failure: BaseException | None = pending["error"]
|
||||||
raise pending["error"]
|
if failure is not None:
|
||||||
|
raise failure
|
||||||
return deepcopy(pending["result"])
|
return deepcopy(pending["result"])
|
||||||
try:
|
try:
|
||||||
result = self._handle(path, payload)
|
result = self._handle(path, payload)
|
||||||
@@ -166,7 +167,8 @@ class LocalSimulator:
|
|||||||
value = payload.get(name)
|
value = payload.get(name)
|
||||||
if (not isinstance(value, str) or not value
|
if (not isinstance(value, str) or not value
|
||||||
or any(category(c) in ("Cc", "Cf", "Cs") for c in value)
|
or any(category(c) in ("Cc", "Cf", "Cs") for c in value)
|
||||||
or (field_spec.max_bytes and len(value.encode("utf-8")) > field_spec.max_bytes)):
|
or (field_spec.max_bytes
|
||||||
|
and len(value.encode("utf-8")) > field_spec.max_bytes)):
|
||||||
return 400, self.error_body(path)
|
return 400, self.error_body(path)
|
||||||
# 2) 动作字段存在性 -> 400
|
# 2) 动作字段存在性 -> 400
|
||||||
action_spec = ACTION_FIELDS[path]
|
action_spec = ACTION_FIELDS[path]
|
||||||
|
|||||||
@@ -14,8 +14,7 @@ from pathlib import Path
|
|||||||
from urllib.parse import parse_qs, urlsplit
|
from urllib.parse import parse_qs, urlsplit
|
||||||
|
|
||||||
from .jsonio import dumps
|
from .jsonio import dumps
|
||||||
from .session import (DISPLAY_LIMIT_MAX, DISPLAY_LIMIT_MIN, LiveSession,
|
from .session import LiveSession, Referee, RefereeError
|
||||||
Referee, RefereeError)
|
|
||||||
|
|
||||||
STATIC_DIR = Path(__file__).parent / "static"
|
STATIC_DIR = Path(__file__).parent / "static"
|
||||||
STATIC_FILES = {
|
STATIC_FILES = {
|
||||||
@@ -31,7 +30,7 @@ class _UIHandler(BaseHTTPRequestHandler):
|
|||||||
protocol_version = "HTTP/1.1"
|
protocol_version = "HTTP/1.1"
|
||||||
server_version = "MCM26B-Referee-UI/1.0"
|
server_version = "MCM26B-Referee-UI/1.0"
|
||||||
|
|
||||||
def log_message(self, format, *args): # noqa: A002 - 标准库签名
|
def log_message(self, format, *args): # pylint: disable=redefined-builtin(标准库签名)
|
||||||
pass
|
pass
|
||||||
|
|
||||||
# ------------------------------------------------------------ 路由
|
# ------------------------------------------------------------ 路由
|
||||||
|
|||||||
@@ -1,5 +1,7 @@
|
|||||||
"""真实 HTTP 承载测试:接口仅回环、仅窗口期开放、协议边界与官方一致。"""
|
"""真实 HTTP 承载测试:接口仅回环、仅窗口期开放、协议边界与官方一致。"""
|
||||||
|
|
||||||
|
# pylint: disable=redefined-outer-name # pytest 夹具名必须与测试函数形参同名,非遮蔽缺陷
|
||||||
|
|
||||||
import http.client
|
import http.client
|
||||||
import json
|
import json
|
||||||
import time
|
import time
|
||||||
@@ -20,7 +22,7 @@ def wait_until(condition, timeout=5.0, interval=0.02):
|
|||||||
|
|
||||||
|
|
||||||
@pytest.fixture()
|
@pytest.fixture()
|
||||||
def referee(tmp_path):
|
def local_referee(tmp_path):
|
||||||
ref = Referee(action_port=0, data_dir=tmp_path / "data")
|
ref = Referee(action_port=0, data_dir=tmp_path / "data")
|
||||||
ActionInterfaceServer(ref)
|
ActionInterfaceServer(ref)
|
||||||
return ref
|
return ref
|
||||||
@@ -43,20 +45,20 @@ def post(port, path, obj=None, *, headers=None, method="POST", raw=None):
|
|||||||
return response.status, parsed
|
return response.status, parsed
|
||||||
|
|
||||||
|
|
||||||
def payload(id="e", **extra):
|
def payload(request_id="e", **extra):
|
||||||
return dict(arena_id="default", robot_id="2026001", request_id=id, **extra)
|
return dict(arena_id="default", robot_id="2026001", request_id=request_id, **extra)
|
||||||
|
|
||||||
|
|
||||||
def start_open_session(referee, **opts):
|
def start_open_session(local_referee, **opts):
|
||||||
opts.setdefault("countdown_s", 0.05)
|
opts.setdefault("countdown_s", 0.05)
|
||||||
session = referee.start("q3_practice", "2026001", **opts)
|
session = local_referee.start("q3_practice", "2026001", **opts)
|
||||||
assert wait_until(lambda: session.phase == "open")
|
assert wait_until(lambda: session.phase == "open")
|
||||||
return session
|
return session
|
||||||
|
|
||||||
|
|
||||||
def test_binding_is_loopback_only(referee):
|
def test_binding_is_loopback_only(local_referee):
|
||||||
session = start_open_session(referee)
|
_session = start_open_session(local_referee)
|
||||||
server = referee._open_interface.__self__ # ActionInterfaceServer
|
server = local_referee._open_interface.__self__ # ActionInterfaceServer
|
||||||
port = open_port(server)
|
port = open_port(server)
|
||||||
assert server._httpd.server_address[0] == "127.0.0.1"
|
assert server._httpd.server_address[0] == "127.0.0.1"
|
||||||
status, body = post(port, "/enter", payload("enter"))
|
status, body = post(port, "/enter", payload("enter"))
|
||||||
@@ -64,17 +66,17 @@ def test_binding_is_loopback_only(referee):
|
|||||||
assert body["remaining_real_duration_s"] == 1200
|
assert body["remaining_real_duration_s"] == 1200
|
||||||
|
|
||||||
|
|
||||||
def test_interface_refuses_connections_outside_window(referee):
|
def test_interface_refuses_connections_outside_window(local_referee):
|
||||||
session = referee.start("q3_practice", "2026001", countdown_s=5)
|
session = local_referee.start("q3_practice", "2026001", countdown_s=5)
|
||||||
server = referee._open_interface.__self__
|
server = local_referee._open_interface.__self__
|
||||||
assert server.is_open is False # 倒计时期间未监听
|
assert server.is_open is False # 倒计时期间未监听
|
||||||
assert wait_until(lambda: session.phase == "open", timeout=6)
|
assert wait_until(lambda: session.phase == "open", timeout=6)
|
||||||
assert server.is_open is True # 窗口开启后才绑定端口
|
assert server.is_open is True # 窗口开启后才绑定端口
|
||||||
|
|
||||||
|
|
||||||
def test_full_protocol_round_trip_over_http(referee):
|
def test_full_protocol_round_trip_over_http(local_referee):
|
||||||
session = start_open_session(referee)
|
session = start_open_session(local_referee)
|
||||||
server = referee._open_interface.__self__
|
server = local_referee._open_interface.__self__
|
||||||
port = open_port(server)
|
port = open_port(server)
|
||||||
status, body = post(port, "/enter", payload("enter-1"))
|
status, body = post(port, "/enter", payload("enter-1"))
|
||||||
assert status == 200 and body["accepted"] is True
|
assert status == 200 and body["accepted"] is True
|
||||||
@@ -94,9 +96,9 @@ def test_full_protocol_round_trip_over_http(referee):
|
|||||||
assert session.end_reason == "user_exit"
|
assert session.end_reason == "user_exit"
|
||||||
|
|
||||||
|
|
||||||
def test_protocol_error_statuses_over_http(referee):
|
def test_protocol_error_statuses_over_http(local_referee):
|
||||||
start_open_session(referee)
|
start_open_session(local_referee)
|
||||||
server = referee._open_interface.__self__
|
server = local_referee._open_interface.__self__
|
||||||
port = open_port(server)
|
port = open_port(server)
|
||||||
assert post(port, "/unknown", payload())[0] == 404
|
assert post(port, "/unknown", payload())[0] == 404
|
||||||
assert post(port, "/enter/", payload())[0] == 404
|
assert post(port, "/enter/", payload())[0] == 404
|
||||||
@@ -109,9 +111,9 @@ def test_protocol_error_statuses_over_http(referee):
|
|||||||
assert status == 200 and body["accepted"] is True
|
assert status == 200 and body["accepted"] is True
|
||||||
|
|
||||||
|
|
||||||
def test_interface_closes_after_window_timeout(referee):
|
def test_interface_closes_after_window_timeout(local_referee):
|
||||||
session = start_open_session(referee, window_s=0.4)
|
session = start_open_session(local_referee, window_s=0.4)
|
||||||
server = referee._open_interface.__self__
|
server = local_referee._open_interface.__self__
|
||||||
port = open_port(server)
|
port = open_port(server)
|
||||||
assert wait_until(lambda: session.phase == "finished", timeout=5)
|
assert wait_until(lambda: session.phase == "finished", timeout=5)
|
||||||
assert session.end_reason == "window_timeout"
|
assert session.end_reason == "window_timeout"
|
||||||
@@ -120,9 +122,9 @@ def test_interface_closes_after_window_timeout(referee):
|
|||||||
post(port, "/enter", payload("late"))
|
post(port, "/enter", payload("late"))
|
||||||
|
|
||||||
|
|
||||||
def test_recorded_feed_contains_every_request(referee):
|
def test_recorded_feed_contains_every_request(local_referee):
|
||||||
session = start_open_session(referee)
|
session = start_open_session(local_referee)
|
||||||
server = referee._open_interface.__self__
|
server = local_referee._open_interface.__self__
|
||||||
port = open_port(server)
|
port = open_port(server)
|
||||||
post(port, "/enter", payload("enter-1"))
|
post(port, "/enter", payload("enter-1"))
|
||||||
post(port, "/unknown", payload("bad"))
|
post(port, "/unknown", payload("bad"))
|
||||||
|
|||||||
@@ -11,8 +11,8 @@ from referee import LocalSimulator, SimulatorClosed
|
|||||||
from referee.interface import handle_raw
|
from referee.interface import handle_raw
|
||||||
|
|
||||||
|
|
||||||
def payload(id="e", **extra):
|
def payload(request_id="e", **extra):
|
||||||
return dict(arena_id="default", robot_id="2026001", request_id=id, **extra)
|
return dict(arena_id="default", robot_id="2026001", request_id=request_id, **extra)
|
||||||
|
|
||||||
|
|
||||||
def post(sim, path, body=None, headers=None, method="POST", target=None):
|
def post(sim, path, body=None, headers=None, method="POST", target=None):
|
||||||
@@ -60,7 +60,8 @@ def test_raw_size_and_depth_boundaries_are_inclusive():
|
|||||||
("PUT", "/clear", None, 405),
|
("PUT", "/clear", None, 405),
|
||||||
("POST", "/enter", [("Content-Type", "text/plain")], 415),
|
("POST", "/enter", [("Content-Type", "text/plain")], 415),
|
||||||
("POST", "/enter", [("Content-Type", "application/json;foo=bar")], 415),
|
("POST", "/enter", [("Content-Type", "application/json;foo=bar")], 415),
|
||||||
("POST", "/enter", [("Content-Type", "application/json"), ("Content-Type", "application/json")], 415),
|
("POST", "/enter",
|
||||||
|
[("Content-Type", "application/json"), ("Content-Type", "application/json")], 415),
|
||||||
("POST", "/enter", [("Content-Type", "application/json"), ("Content-Encoding", "gzip")], 415),
|
("POST", "/enter", [("Content-Type", "application/json"), ("Content-Encoding", "gzip")], 415),
|
||||||
("POST", "/enter", [("content-type", "Application/JSON; charset=UTF-8")], 200),
|
("POST", "/enter", [("content-type", "Application/JSON; charset=UTF-8")], 200),
|
||||||
("POST", "/enter", [("content-type", 'application/json;charset="utf-8"')], 200),
|
("POST", "/enter", [("content-type", 'application/json;charset="utf-8"')], 200),
|
||||||
@@ -84,7 +85,8 @@ def test_on_request_callback_receives_parsed_payload():
|
|||||||
seen = []
|
seen = []
|
||||||
raw = json.dumps(payload("enter-1")).encode()
|
raw = json.dumps(payload("enter-1")).encode()
|
||||||
handle_raw(sim, "POST", "/enter", [("Content-Type", "application/json")], raw,
|
handle_raw(sim, "POST", "/enter", [("Content-Type", "application/json")], raw,
|
||||||
on_request=lambda path, payload_dict, result: seen.append((path, payload_dict, result)))
|
on_request=lambda path, payload_dict, result: seen.append(
|
||||||
|
(path, payload_dict, result)))
|
||||||
assert seen[0][0] == "/enter"
|
assert seen[0][0] == "/enter"
|
||||||
assert seen[0][1]["request_id"] == "enter-1"
|
assert seen[0][1]["request_id"] == "enter-1"
|
||||||
assert seen[0][2].status == 200
|
assert seen[0][2].status == 200
|
||||||
|
|||||||
@@ -20,6 +20,7 @@ HARNESS = REPO / "referee" / "tests" / "js" / "replay_harness.cjs"
|
|||||||
@pytest.mark.skipif(shutil.which("node") is None, reason="需要 node 环境运行 JS 桩测试")
|
@pytest.mark.skipif(shutil.which("node") is None, reason="需要 node 环境运行 JS 桩测试")
|
||||||
def test_replay_player_logic():
|
def test_replay_player_logic():
|
||||||
result = subprocess.run(["node", str(HARNESS)], capture_output=True, text=True,
|
result = subprocess.run(["node", str(HARNESS)], capture_output=True, text=True,
|
||||||
env={**os.environ, "REPO": str(REPO)}, timeout=60)
|
env={**os.environ, "REPO": str(REPO)}, timeout=60,
|
||||||
|
check=False)
|
||||||
assert result.returncode == 0, f"回放器逻辑测试失败:\n{result.stdout}\n{result.stderr}"
|
assert result.returncode == 0, f"回放器逻辑测试失败:\n{result.stdout}\n{result.stderr}"
|
||||||
assert "ALL PASS" in result.stdout
|
assert "ALL PASS" in result.stdout
|
||||||
|
|||||||
@@ -65,7 +65,7 @@ def test_cannot_start_while_running_and_restart_after_finish(tmp_path):
|
|||||||
def test_formal_attempts_accounting(tmp_path):
|
def test_formal_attempts_accounting(tmp_path):
|
||||||
referee = Referee(action_port=0, data_dir=tmp_path)
|
referee = Referee(action_port=0, data_dir=tmp_path)
|
||||||
for _ in range(FORMAL_ATTEMPTS):
|
for _ in range(FORMAL_ATTEMPTS):
|
||||||
session = referee.start("q3_formal", "2026001", countdown_s=0.05)
|
_session = referee.start("q3_formal", "2026001", countdown_s=0.05)
|
||||||
referee.abort() # 正式中止仍占用机会 [题面 4.5]
|
referee.abort() # 正式中止仍占用机会 [题面 4.5]
|
||||||
with pytest.raises(RefereeError):
|
with pytest.raises(RefereeError):
|
||||||
referee.start("q3_formal", "2026001", countdown_s=0.05)
|
referee.start("q3_formal", "2026001", countdown_s=0.05)
|
||||||
@@ -121,7 +121,7 @@ def test_record_action_updates_feed_trail_and_counts(tmp_path):
|
|||||||
referee = Referee(action_port=0, data_dir=tmp_path)
|
referee = Referee(action_port=0, data_dir=tmp_path)
|
||||||
session = referee.start("q3_practice", "2026001", countdown_s=0.05, window_s=5)
|
session = referee.start("q3_practice", "2026001", countdown_s=0.05, window_s=5)
|
||||||
assert wait_until(lambda: session.phase == "open")
|
assert wait_until(lambda: session.phase == "open")
|
||||||
sim = session.simulator
|
_sim = session.simulator
|
||||||
referee.record_action(session, "POST", "/measure",
|
referee.record_action(session, "POST", "/measure",
|
||||||
{"request_id": "m1", "position": {"x": 5000, "y": 0},
|
{"request_id": "m1", "position": {"x": 5000, "y": 0},
|
||||||
"channel": 1},
|
"channel": 1},
|
||||||
@@ -168,7 +168,7 @@ def test_display_limit_and_port_settings(tmp_path):
|
|||||||
referee.set_display_limit(50)
|
referee.set_display_limit(50)
|
||||||
with pytest.raises(RefereeError):
|
with pytest.raises(RefereeError):
|
||||||
referee.set_display_limit(5001)
|
referee.set_display_limit(5001)
|
||||||
session = referee.start("q3_practice", "2026001", countdown_s=5)
|
_session = referee.start("q3_practice", "2026001", countdown_s=5)
|
||||||
with pytest.raises(RefereeError):
|
with pytest.raises(RefereeError):
|
||||||
referee.set_action_port(8080) # 测试进行中不能改端口
|
referee.set_action_port(8080) # 测试进行中不能改端口
|
||||||
referee.abort()
|
referee.abort()
|
||||||
|
|||||||
@@ -1,7 +1,6 @@
|
|||||||
"""评测引擎一致性:时限、状态机、观测判定、幂等与量化(移植自参赛队仓库并扩展)。"""
|
"""评测引擎一致性:时限、状态机、观测判定、幂等与量化(移植自参赛队仓库并扩展)。"""
|
||||||
|
|
||||||
from concurrent.futures import ThreadPoolExecutor
|
from concurrent.futures import ThreadPoolExecutor
|
||||||
from dataclasses import replace
|
|
||||||
from threading import Event
|
from threading import Event
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
@@ -9,12 +8,12 @@ import pytest
|
|||||||
from referee import FieldSpec, LocalSimulator, SimulatorClosed, Source, Spec
|
from referee import FieldSpec, LocalSimulator, SimulatorClosed, Source, Spec
|
||||||
|
|
||||||
|
|
||||||
def payload(id="e", **extra):
|
def payload(request_id="e", **extra):
|
||||||
return dict(arena_id="default", robot_id="2026001", request_id=id, **extra)
|
return dict(arena_id="default", robot_id="2026001", request_id=request_id, **extra)
|
||||||
|
|
||||||
|
|
||||||
def measure(id="m", x=0, y=0, channel=1):
|
def measure(request_id="m", x=0, y=0, channel=1):
|
||||||
return payload(id, position=dict(x=x, y=y), channel=channel)
|
return payload(request_id, position=dict(x=x, y=y), channel=channel)
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------- 时限 [题面 附件1 2.5]
|
# ---------------------------------------------------------------- 时限 [题面 附件1 2.5]
|
||||||
@@ -92,7 +91,7 @@ def test_error_rejection_does_not_consume_id():
|
|||||||
sim = LocalSimulator([])
|
sim = LocalSimulator([])
|
||||||
for bad in ["e\n", "e\u200b", "e\u0085", "e\ud800"]:
|
for bad in ["e\n", "e\u200b", "e\u0085", "e\ud800"]:
|
||||||
assert sim.handle("/enter", payload(bad))[0] == 400
|
assert sim.handle("/enter", payload(bad))[0] == 400
|
||||||
assert sim._idempotency == {}
|
assert not sim._idempotency
|
||||||
assert sim.handle("/enter", payload())[1]["accepted"]
|
assert sim.handle("/enter", payload())[1]["accepted"]
|
||||||
|
|
||||||
|
|
||||||
@@ -181,6 +180,8 @@ def test_huge_numeric_fields_return_bad_request():
|
|||||||
(dict(arena_id="other", robot_id="2026001", request_id="e"), False),
|
(dict(arena_id="other", robot_id="2026001", request_id="e"), False),
|
||||||
(dict(arena_id="default", robot_id="someone-else", request_id="e"), False),
|
(dict(arena_id="default", robot_id="someone-else", request_id="e"), False),
|
||||||
])
|
])
|
||||||
|
# pytest 参数化保留 expect 列(用例矩阵可读性),当前断言恒为 False,故不参与断言
|
||||||
|
# pylint: disable-next=unused-argument
|
||||||
def test_unknown_fields_and_mismatch_are_business_rejected(body, expect):
|
def test_unknown_fields_and_mismatch_are_business_rejected(body, expect):
|
||||||
sim = LocalSimulator([])
|
sim = LocalSimulator([])
|
||||||
status, response = sim.handle("/enter", body)
|
status, response = sim.handle("/enter", body)
|
||||||
@@ -204,12 +205,14 @@ def test_channel_and_position_validation():
|
|||||||
]
|
]
|
||||||
for extra in cases:
|
for extra in cases:
|
||||||
assert sim.handle("/measure", payload("m", **extra))[0] == 400
|
assert sim.handle("/measure", payload("m", **extra))[0] == 400
|
||||||
assert sim.handle("/measure", payload("ok", position=dict(x=0, y=0), channel=1.0))[1]["accepted"]
|
assert sim.handle("/measure",
|
||||||
|
payload("ok", position=dict(x=0, y=0), channel=1.0))[1]["accepted"]
|
||||||
|
|
||||||
|
|
||||||
def test_actions_before_enter_are_business_rejected():
|
def test_actions_before_enter_are_business_rejected():
|
||||||
sim = LocalSimulator([])
|
sim = LocalSimulator([])
|
||||||
for path, body in [("/measure", measure()), ("/clear", payload("c", position=dict(x=0, y=0), channel=1)),
|
for path, body in [("/measure", measure()),
|
||||||
|
("/clear", payload("c", position=dict(x=0, y=0), channel=1)),
|
||||||
("/exit", payload("x"))]:
|
("/exit", payload("x"))]:
|
||||||
status, response = sim.handle(path, body)
|
status, response = sim.handle(path, body)
|
||||||
assert status == 200 and response["accepted"] is False
|
assert status == 200 and response["accepted"] is False
|
||||||
@@ -224,7 +227,8 @@ def test_clear_rules():
|
|||||||
assert ok["clear_result"] == "success"
|
assert ok["clear_result"] == "success"
|
||||||
again = sim.handle("/clear", payload("c5b", position=dict(x=10, y=0), channel=5))[1]
|
again = sim.handle("/clear", payload("c5b", position=dict(x=10, y=0), channel=5))[1]
|
||||||
assert again["clear_result"] == "no_target_in_range" # 同一源只能清除一次
|
assert again["clear_result"] == "no_target_in_range" # 同一源只能清除一次
|
||||||
assert sim.handle("/clear", payload("c6", position=dict(x=15, y=0), channel=6))[1]["clear_result"] == "success"
|
cleared = sim.handle("/clear", payload("c6", position=dict(x=15, y=0), channel=6))[1]
|
||||||
|
assert cleared["clear_result"] == "success"
|
||||||
miss = sim.handle("/clear", payload("c9", position=dict(x=0, y=0), channel=9))[1]
|
miss = sim.handle("/clear", payload("c9", position=dict(x=0, y=0), channel=9))[1]
|
||||||
assert miss["clear_result"] == "no_target_in_range"
|
assert miss["clear_result"] == "no_target_in_range"
|
||||||
assert sim.channel == 1 # /clear 不切换测向机频道 [题面 附件1 2.3]
|
assert sim.channel == 1 # /clear 不切换测向机频道 [题面 附件1 2.3]
|
||||||
@@ -238,15 +242,20 @@ def test_measure_observation_matrix():
|
|||||||
sim = LocalSimulator(sources, error_half_deg=0.0)
|
sim = LocalSimulator(sources, error_half_deg=0.0)
|
||||||
sim.handle("/enter", payload("enter"))
|
sim.handle("/enter", payload("enter"))
|
||||||
# 超出有效接收半径
|
# 超出有效接收半径
|
||||||
assert sim.handle("/measure", measure("a", -600, 0, channel=1))[1]["measure_result"] == "no_signal"
|
assert sim.handle("/measure",
|
||||||
|
measure("a", -600, 0, channel=1))[1]["measure_result"] == "no_signal"
|
||||||
# 接收半径边界:distance == r_eff 仍有信号 [题面 附件2 2.1 "超过"]
|
# 接收半径边界:distance == r_eff 仍有信号 [题面 附件2 2.1 "超过"]
|
||||||
assert sim.handle("/measure", measure("b", -500, 0, channel=1))[1]["measure_result"] == "direction"
|
assert sim.handle("/measure",
|
||||||
|
measure("b", -500, 0, channel=1))[1]["measure_result"] == "direction"
|
||||||
# 覆盖角边界:源2 朝东,正东方向(bearing=0°)恰在 ±90° 覆盖内
|
# 覆盖角边界:源2 朝东,正东方向(bearing=0°)恰在 ±90° 覆盖内
|
||||||
assert sim.handle("/measure", measure("c", 200, 0, channel=2))[1]["measure_result"] == "direction"
|
assert sim.handle("/measure",
|
||||||
|
measure("c", 200, 0, channel=2))[1]["measure_result"] == "direction"
|
||||||
# 覆盖角边界:bearing=90°(正北)仍在覆盖内(含边界)[题面 附件2 2.2]
|
# 覆盖角边界:bearing=90°(正北)仍在覆盖内(含边界)[题面 附件2 2.2]
|
||||||
assert sim.handle("/measure", measure("c2", 100, 100, channel=2))[1]["measure_result"] == "direction"
|
assert sim.handle("/measure",
|
||||||
|
measure("c2", 100, 100, channel=2))[1]["measure_result"] == "direction"
|
||||||
# 覆盖角外(bearing≈135° > 90°):无信号
|
# 覆盖角外(bearing≈135° > 90°):无信号
|
||||||
assert sim.handle("/measure", measure("d", 90, 10, channel=2))[1]["measure_result"] == "no_signal"
|
assert sim.handle("/measure",
|
||||||
|
measure("d", 90, 10, channel=2))[1]["measure_result"] == "no_signal"
|
||||||
# 距离过近:不返回 svd_deg [题面 附件2 2.4]
|
# 距离过近:不返回 svd_deg [题面 附件2 2.4]
|
||||||
near = sim.handle("/measure", measure("near", 998, 0, channel=1))[1]
|
near = sim.handle("/measure", measure("near", 998, 0, channel=1))[1]
|
||||||
assert near["measure_result"] == "near" and "svd_deg" not in near
|
assert near["measure_result"] == "near" and "svd_deg" not in near
|
||||||
|
|||||||
@@ -4,12 +4,12 @@ from referee import CostBreakdown, LocalSimulator, Source, clear_cost, measure_c
|
|||||||
from referee.timing import move_time
|
from referee.timing import move_time
|
||||||
|
|
||||||
|
|
||||||
def payload(id="e", **extra):
|
def payload(request_id="e", **extra):
|
||||||
return dict(arena_id="default", robot_id="2026001", request_id=id, **extra)
|
return dict(arena_id="default", robot_id="2026001", request_id=request_id, **extra)
|
||||||
|
|
||||||
|
|
||||||
def measure(id="m", x=0, y=0, channel=1):
|
def measure(request_id="m", x=0, y=0, channel=1):
|
||||||
return payload(id, position=dict(x=x, y=y), channel=channel)
|
return payload(request_id, position=dict(x=x, y=y), channel=channel)
|
||||||
|
|
||||||
|
|
||||||
def test_attachment_timing_example():
|
def test_attachment_timing_example():
|
||||||
|
|||||||
@@ -1,5 +1,7 @@
|
|||||||
"""WebUI API 测试:状态快照、启动/中止/配置、会话列表与回放数据源。"""
|
"""WebUI API 测试:状态快照、启动/中止/配置、会话列表与回放数据源。"""
|
||||||
|
|
||||||
|
# pylint: disable=redefined-outer-name # pytest 夹具名必须与测试函数形参同名,非遮蔽缺陷
|
||||||
|
|
||||||
import http.client
|
import http.client
|
||||||
import json
|
import json
|
||||||
import time
|
import time
|
||||||
@@ -21,7 +23,7 @@ def wait_until(condition, timeout=5.0, interval=0.02):
|
|||||||
|
|
||||||
|
|
||||||
@pytest.fixture()
|
@pytest.fixture()
|
||||||
def ui(tmp_path):
|
def webui(tmp_path):
|
||||||
referee = Referee(action_port=0, data_dir=tmp_path / "data")
|
referee = Referee(action_port=0, data_dir=tmp_path / "data")
|
||||||
ActionInterfaceServer(referee)
|
ActionInterfaceServer(referee)
|
||||||
server = WebUIServer(referee, 0)
|
server = WebUIServer(referee, 0)
|
||||||
@@ -43,8 +45,8 @@ def call(port, method, path, obj=None):
|
|||||||
return response.status, json.loads(data) if data else None
|
return response.status, json.loads(data) if data else None
|
||||||
|
|
||||||
|
|
||||||
def test_state_idle_snapshot(ui):
|
def test_state_idle_snapshot(webui):
|
||||||
referee, port = ui
|
_referee, port = webui
|
||||||
status, state = call(port, "GET", "/ui/api/state")
|
status, state = call(port, "GET", "/ui/api/state")
|
||||||
assert status == 200
|
assert status == 200
|
||||||
assert state["phase"] == "idle"
|
assert state["phase"] == "idle"
|
||||||
@@ -53,8 +55,8 @@ def test_state_idle_snapshot(ui):
|
|||||||
assert state["spec"]["omega_radius_m"] == 1800
|
assert state["spec"]["omega_radius_m"] == 1800
|
||||||
|
|
||||||
|
|
||||||
def test_start_countdown_open_abort_cycle(ui):
|
def test_start_countdown_open_abort_cycle(webui):
|
||||||
referee, port = ui
|
referee, port = webui
|
||||||
status, body = call(port, "POST", "/ui/api/start",
|
status, body = call(port, "POST", "/ui/api/start",
|
||||||
{"module": "q3_practice", "robot_id": "2026001",
|
{"module": "q3_practice", "robot_id": "2026001",
|
||||||
"countdown_s": 0.05, "window_s": 5})
|
"countdown_s": 0.05, "window_s": 5})
|
||||||
@@ -70,21 +72,21 @@ def test_start_countdown_open_abort_cycle(ui):
|
|||||||
assert state["truth"]["revealed"] is True # 演练结束显示真值统计
|
assert state["truth"]["revealed"] is True # 演练结束显示真值统计
|
||||||
|
|
||||||
|
|
||||||
def test_start_rejects_running_session_and_bad_module(ui):
|
def test_start_rejects_running_session_and_bad_module(webui):
|
||||||
referee, port = ui
|
_referee, port = webui
|
||||||
call(port, "POST", "/ui/api/start",
|
call(port, "POST", "/ui/api/start",
|
||||||
{"module": "q3_practice", "robot_id": "2026001", "countdown_s": 5})
|
{"module": "q3_practice", "robot_id": "2026001", "countdown_s": 5})
|
||||||
status, body = call(port, "POST", "/ui/api/start",
|
status, _body = call(port, "POST", "/ui/api/start",
|
||||||
{"module": "q3_practice", "robot_id": "2026001"})
|
{"module": "q3_practice", "robot_id": "2026001"})
|
||||||
assert status == 409
|
assert status == 409
|
||||||
status, body = call(port, "POST", "/ui/api/start",
|
status, _body = call(port, "POST", "/ui/api/start",
|
||||||
{"module": "nope", "robot_id": "2026001"})
|
{"module": "nope", "robot_id": "2026001"})
|
||||||
assert status == 409
|
assert status == 409
|
||||||
call(port, "POST", "/ui/api/abort", {})
|
call(port, "POST", "/ui/api/abort", {})
|
||||||
|
|
||||||
|
|
||||||
def test_config_endpoints(ui):
|
def test_config_endpoints(webui):
|
||||||
referee, port = ui
|
_referee, port = webui
|
||||||
status, body = call(port, "POST", "/ui/api/config", {"display_limit": 100})
|
status, body = call(port, "POST", "/ui/api/config", {"display_limit": 100})
|
||||||
assert body["configured"]["display_limit"] == 100
|
assert body["configured"]["display_limit"] == 100
|
||||||
status, body = call(port, "POST", "/ui/api/config", {"display_limit": 1})
|
status, body = call(port, "POST", "/ui/api/config", {"display_limit": 1})
|
||||||
@@ -93,8 +95,8 @@ def test_config_endpoints(ui):
|
|||||||
assert body["configured"]["action_port"] == 8080
|
assert body["configured"]["action_port"] == 8080
|
||||||
|
|
||||||
|
|
||||||
def test_sessions_list_and_replay_source(ui):
|
def test_sessions_list_and_replay_source(webui):
|
||||||
referee, port = ui
|
referee, port = webui
|
||||||
call(port, "POST", "/ui/api/start",
|
call(port, "POST", "/ui/api/start",
|
||||||
{"module": "q3_practice", "robot_id": "2026001",
|
{"module": "q3_practice", "robot_id": "2026001",
|
||||||
"countdown_s": 0.05, "window_s": 5, "seed": 11, "source_count": 10})
|
"countdown_s": 0.05, "window_s": 5, "seed": 11, "source_count": 10})
|
||||||
@@ -125,8 +127,8 @@ def test_sessions_list_and_replay_source(ui):
|
|||||||
assert call(port, "GET", "/ui/api/replay?case=../etc")[0] == 400
|
assert call(port, "GET", "/ui/api/replay?case=../etc")[0] == 400
|
||||||
|
|
||||||
|
|
||||||
def test_static_page_served(ui):
|
def test_static_page_served(webui):
|
||||||
_, port = ui
|
_, port = webui
|
||||||
conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
|
conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
|
||||||
conn.request("GET", "/")
|
conn.request("GET", "/")
|
||||||
response = conn.getresponse()
|
response = conn.getresponse()
|
||||||
@@ -140,8 +142,8 @@ def test_static_page_served(ui):
|
|||||||
assert 'id="rp-stage"' in page
|
assert 'id="rp-stage"' in page
|
||||||
|
|
||||||
|
|
||||||
def test_replay_script_served_with_file_import(ui):
|
def test_replay_script_served_with_file_import(webui):
|
||||||
_, port = ui
|
_, port = webui
|
||||||
conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
|
conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
|
||||||
conn.request("GET", "/replay.js")
|
conn.request("GET", "/replay.js")
|
||||||
response = conn.getresponse()
|
response = conn.getresponse()
|
||||||
|
|||||||
@@ -26,14 +26,13 @@ from __future__ import annotations
|
|||||||
import argparse
|
import argparse
|
||||||
import csv
|
import csv
|
||||||
import json
|
import json
|
||||||
import math
|
|
||||||
import statistics
|
import statistics
|
||||||
import sys
|
import sys
|
||||||
import time
|
import time
|
||||||
from collections import defaultdict
|
from collections import defaultdict
|
||||||
from dataclasses import asdict, dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Callable
|
from typing import Any
|
||||||
|
|
||||||
from mcm26b.config import ProblemConfig
|
from mcm26b.config import ProblemConfig
|
||||||
from mcm26b.harness import LocalSimTransport, mixed_scene, random_omni_scene
|
from mcm26b.harness import LocalSimTransport, mixed_scene, random_omni_scene
|
||||||
|
|||||||
+10
-6
@@ -39,17 +39,21 @@ def main():
|
|||||||
for margin in [0, 0.005]:
|
for margin in [0, 0.005]:
|
||||||
ms = [replace(m, half_angle_deg=m.half_angle_deg+margin) for m in observations]
|
ms = [replace(m, half_angle_deg=m.half_angle_deg+margin) for m in observations]
|
||||||
result, duration = timed(solve, ms, 31)
|
result, duration = timed(solve, ms, 31)
|
||||||
|
cert = result['clearance_certificate']
|
||||||
sensitivity.append(dict(additional_half_angle_deg=margin, solve_ms=duration,
|
sensitivity.append(dict(additional_half_angle_deg=margin, solve_ms=duration,
|
||||||
diameter_m=result['diameter_m'],
|
diameter_m=result['diameter_m'],
|
||||||
mec_radius_m=result['minimum_enclosing_circle']['radius_m'],
|
mec_radius_m=result['minimum_enclosing_circle']['radius_m'],
|
||||||
diameter_circle_excess_m=result['diameter_circle']['max_excess_m'],
|
diameter_circle_excess_m=result['diameter_circle']['max_excess_m'],
|
||||||
clearance_margin_m=result['clearance_certificate']['remaining_margin_m']))
|
clearance_margin_m=cert['remaining_margin_m']))
|
||||||
report = dict(python=sys.version.split()[0], timing_method='median perf_counter; warm local process',
|
report = dict(
|
||||||
timing_scope='MEC only; not whole solver speedup or simulator/network performance',
|
python=sys.version.split()[0],
|
||||||
circle_benchmarks=timings, practice_sensitivity=sensitivity,
|
timing_method='median perf_counter; warm local process',
|
||||||
q2_first_bearing_certificate=first_bearing_candidate_disk(Measurement(0, 0, 0, 1.005)))
|
timing_scope='MEC only; not whole solver speedup or simulator/network performance',
|
||||||
|
circle_benchmarks=timings, practice_sensitivity=sensitivity,
|
||||||
|
q2_first_bearing_certificate=first_bearing_candidate_disk(Measurement(0, 0, 0, 1.005)))
|
||||||
output = ROOT/'output/q1/optimization_report.json'
|
output = ROOT/'output/q1/optimization_report.json'
|
||||||
output.write_text(json.dumps(report, ensure_ascii=False, indent=2, allow_nan=False)+'\n', encoding='utf-8')
|
output.write_text(
|
||||||
|
json.dumps(report, ensure_ascii=False, indent=2, allow_nan=False)+'\n', encoding='utf-8')
|
||||||
print(json.dumps(report, ensure_ascii=False, indent=2))
|
print(json.dumps(report, ensure_ascii=False, indent=2))
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -48,19 +48,22 @@ def main():
|
|||||||
for p in sorted(Path('src/mcm26b').rglob('*.py'))}
|
for p in sorted(Path('src/mcm26b').rglob('*.py'))}
|
||||||
if args.baseline:
|
if args.baseline:
|
||||||
for name in ('baseline_bayes.py', 'baseline_config.json'):
|
for name in ('baseline_bayes.py', 'baseline_config.json'):
|
||||||
source_sha256[str(args.out/name)] = hashlib.sha256((args.out/name).read_bytes()).hexdigest()
|
archive = args.out/name
|
||||||
|
source_sha256[str(archive)] = hashlib.sha256(archive.read_bytes()).hexdigest()
|
||||||
scenes = []
|
scenes = []
|
||||||
if args.suite == 'development':
|
if args.suite == 'development':
|
||||||
for count in (10, 16):
|
for count in (10, 16):
|
||||||
for seed in range(6):
|
for seed in range(6):
|
||||||
scenes.append((f'random-{count}-{seed}', random_omni_scene(seed, count), 20260910 + seed))
|
scenes.append((f'random-{count}-{seed}', random_omni_scene(seed, count),
|
||||||
|
20260910 + seed))
|
||||||
edge = [Source(c+1, 1800*math.cos(math.pi/6+2*math.pi*c/count),
|
edge = [Source(c+1, 1800*math.cos(math.pi/6+2*math.pi*c/count),
|
||||||
1800*math.sin(math.pi/6+2*math.pi*c/count), 1000) for c in range(count)]
|
1800*math.sin(math.pi/6+2*math.pi*c/count), 1000) for c in range(count)]
|
||||||
scenes.append((f'boundary-{count}', edge, 20260910))
|
scenes.append((f'boundary-{count}', edge, 20260910))
|
||||||
else:
|
else:
|
||||||
for count in (10, 13, 16):
|
for count in (10, 13, 16):
|
||||||
for seed in range(100, 106):
|
for seed in range(100, 106):
|
||||||
scenes.append((f'holdout-{count}-{seed}', random_omni_scene(seed, count), 20260910+seed))
|
scenes.append((f'holdout-{count}-{seed}', random_omni_scene(seed, count),
|
||||||
|
20260910+seed))
|
||||||
rows = []
|
rows = []
|
||||||
for name, sources, error_seed in scenes:
|
for name, sources, error_seed in scenes:
|
||||||
sim = LocalSimulator(sources, robot_id='BENCH', config=config, error_seed=error_seed)
|
sim = LocalSimulator(sources, robot_id='BENCH', config=config, error_seed=error_seed)
|
||||||
@@ -81,12 +84,15 @@ def main():
|
|||||||
sources=[asdict(s) for s in sources], error_seed=error_seed,
|
sources=[asdict(s) for s in sources], error_seed=error_seed,
|
||||||
summary=summary, actions=actions), ensure_ascii=False), encoding='utf-8')
|
summary=summary, actions=actions), ensure_ascii=False), encoding='utf-8')
|
||||||
print(f'{name}: valid={valid} T={row["virtual_s"]:.1f} M={row["measures"]}', flush=True)
|
print(f'{name}: valid={valid} T={row["virtual_s"]:.1f} M={row["measures"]}', flush=True)
|
||||||
result = dict(config=asdict(config), source='local_simulator_only', baseline=args.baseline, rows=rows,
|
result = dict(config=asdict(config), source='local_simulator_only',
|
||||||
mean_s=mean(r['virtual_s'] for r in rows), median_s=median(r['virtual_s'] for r in rows),
|
baseline=args.baseline, rows=rows,
|
||||||
|
mean_s=mean(r['virtual_s'] for r in rows),
|
||||||
|
median_s=median(r['virtual_s'] for r in rows),
|
||||||
max_s=max(r['virtual_s'] for r in rows), valid=sum(r['valid'] for r in rows),
|
max_s=max(r['virtual_s'] for r in rows), valid=sum(r['valid'] for r in rows),
|
||||||
under_4000=sum(r['virtual_s'] < 4000 for r in rows),
|
under_4000=sum(r['virtual_s'] < 4000 for r in rows),
|
||||||
source_sha256=source_sha256)
|
source_sha256=source_sha256)
|
||||||
(folder/'results.json').write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding='utf-8')
|
(folder/'results.json').write_text(
|
||||||
|
json.dumps(result, ensure_ascii=False, indent=2), encoding='utf-8')
|
||||||
print({k:result[k] for k in ('mean_s','median_s','max_s','valid','under_4000')}, flush=True)
|
print({k:result[k] for k in ('mean_s','median_s','max_s','valid','under_4000')}, flush=True)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -14,14 +14,11 @@ from pathlib import Path
|
|||||||
from statistics import mean, median
|
from statistics import mean, median
|
||||||
from time import perf_counter
|
from time import perf_counter
|
||||||
|
|
||||||
from mcm26b.belief import ChannelStatus # noqa: F401 (兼容旧 import 路径)
|
|
||||||
from mcm26b.config import ProblemConfig
|
from mcm26b.config import ProblemConfig
|
||||||
from mcm26b.coverage import CoverageLedger # noqa: F401
|
from mcm26b.planner import OmnidirectionalLoop
|
||||||
from mcm26b.planner import OmnidirectionalLoop, clear_certificate # noqa: F401
|
|
||||||
from mcm26b.planner.seven import (SevenBatch, SevenImmediate, SevenSweep,
|
from mcm26b.planner.seven import (SevenBatch, SevenImmediate, SevenSweep,
|
||||||
RING_RADIUS_M, seven_points)
|
RING_RADIUS_M, seven_points)
|
||||||
from mcm26b.protocol.client import SimulatorClient
|
from mcm26b.protocol.client import SimulatorClient
|
||||||
from mcm26b.protocol.rules import NEAR # noqa: F401
|
|
||||||
from mcm26b.harness import LocalSimTransport, LocalSimulator
|
from mcm26b.harness import LocalSimTransport, LocalSimulator
|
||||||
from referee import Source
|
from referee import Source
|
||||||
from mcm26b.harness import random_omni_scene
|
from mcm26b.harness import random_omni_scene
|
||||||
@@ -70,7 +67,8 @@ def run_case(name, sources, error_seed, out):
|
|||||||
"actions_file": str(log_path.relative_to(out)),
|
"actions_file": str(log_path.relative_to(out)),
|
||||||
"actions_sha256": hashlib.sha256(log_path.read_bytes()).hexdigest()}
|
"actions_sha256": hashlib.sha256(log_path.read_bytes()).hexdigest()}
|
||||||
rows.append(row)
|
rows.append(row)
|
||||||
print(f"{name} {label}: all_cleared={row['all_cleared']} T={row['virtual_time_s']:.1f}s M={row['measures']}", flush=True)
|
print(f"{name} {label}: all_cleared={row['all_cleared']} "
|
||||||
|
f"T={row['virtual_time_s']:.1f}s M={row['measures']}", flush=True)
|
||||||
return rows
|
return rows
|
||||||
|
|
||||||
|
|
||||||
@@ -96,7 +94,8 @@ def geometry_check():
|
|||||||
"sampled_max_distance_m": worst[0], "sampled_worst_point": worst[1],
|
"sampled_max_distance_m": worst[0], "sampled_worst_point": worst[1],
|
||||||
"samples": 181*721,
|
"samples": 181*721,
|
||||||
"passed": worst[0] <= analytic+1e-8 and analytic < field.r_eff_min_m,
|
"passed": worst[0] <= analytic+1e-8 and analytic < field.r_eff_min_m,
|
||||||
"note": "Grid samples check the derivation; the sector proof establishes continuous coverage."}
|
"note": ("Grid samples check the derivation; "
|
||||||
|
"the sector proof establishes continuous coverage.")}
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
@@ -137,10 +136,16 @@ def main():
|
|||||||
result = {"source": "local_simulator_only", "production_algorithm_modified": False,
|
result = {"source": "local_simulator_only", "production_algorithm_modified": False,
|
||||||
"seeds": args.seeds, "config": asdict(ProblemConfig()),
|
"seeds": args.seeds, "config": asdict(ProblemConfig()),
|
||||||
"geometry": geometry_check(), "aggregates": aggregates, "runs": rows,
|
"geometry": geometry_check(), "aggregates": aggregates, "runs": rows,
|
||||||
"limitations": "Same scene and fixed per-position error field for each variant; synthetic data, not an official improvement estimate. Only search/scheduling is changed; no posterior or localizer redesign."}
|
"limitations": (
|
||||||
(args.out/"benchmark.json").write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding="utf-8")
|
"Same scene and fixed per-position error field for each variant; "
|
||||||
|
"synthetic data, not an official improvement estimate. "
|
||||||
|
"Only search/scheduling is changed; no posterior or localizer redesign.")}
|
||||||
|
(args.out/"benchmark.json").write_text(
|
||||||
|
json.dumps(result, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||||
print(json.dumps(aggregates, ensure_ascii=False, indent=2))
|
print(json.dumps(aggregates, ensure_ascii=False, indent=2))
|
||||||
return 0 if result["geometry"]["passed"] and all(a["valid_runs"] == a["runs"] for a in aggregates.values()) else 1
|
passed = (result["geometry"]["passed"]
|
||||||
|
and all(a["valid_runs"] == a["runs"] for a in aggregates.values()))
|
||||||
|
return 0 if passed else 1
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|||||||
@@ -95,7 +95,8 @@ def main(argv=None) -> int:
|
|||||||
cleared = all(row["rl"]["all_cleared"] and row[baseline]["all_cleared"]
|
cleared = all(row["rl"]["all_cleared"] and row[baseline]["all_cleared"]
|
||||||
for row in rows)
|
for row in rows)
|
||||||
valid = all(part['completed'] and part['all_cleared'] and part['correct_absence']
|
valid = all(part['completed'] and part['all_cleared'] and part['correct_absence']
|
||||||
and part['normal_exit'] and not part['anomalies'] and not part['timing_mismatches']
|
and part['normal_exit'] and not part['anomalies']
|
||||||
|
and not part['timing_mismatches']
|
||||||
for row in rows for part in (row[baseline],row['rl']))
|
for row in rows for part in (row[baseline],row['rl']))
|
||||||
report[key + "_summary"] = {
|
report[key + "_summary"] = {
|
||||||
"mean_virtual_s": {
|
"mean_virtual_s": {
|
||||||
|
|||||||
@@ -53,15 +53,23 @@ def main():
|
|||||||
data={**case,'measurements':[asdict(m) for m in ms],'frames':frames}
|
data={**case,'measurements':[asdict(m) for m in ms],'frames':frames}
|
||||||
dataset.append(data)
|
dataset.append(data)
|
||||||
if case['source']=='synthetic':
|
if case['source']=='synthetic':
|
||||||
(examples/(case['id']+'.json')).write_text(json.dumps({k:v for k,v in data.items() if k!='frames'},ensure_ascii=False,indent=2),encoding='utf-8')
|
(examples/(case['id']+'.json')).write_text(
|
||||||
(results/(case['id']+'_result.json')).write_text(json.dumps(frames[-1],ensure_ascii=False,indent=2,allow_nan=False),encoding='utf-8')
|
json.dumps({k:v for k,v in data.items() if k!='frames'},
|
||||||
(results/'demo_data.json').write_text(json.dumps(dataset,ensure_ascii=False,indent=2,allow_nan=False),encoding='utf-8')
|
ensure_ascii=False,indent=2),encoding='utf-8')
|
||||||
|
(results/(case['id']+'_result.json')).write_text(
|
||||||
|
json.dumps(frames[-1],ensure_ascii=False,indent=2,allow_nan=False),encoding='utf-8')
|
||||||
|
(results/'demo_data.json').write_text(
|
||||||
|
json.dumps(dataset,ensure_ascii=False,indent=2,allow_nan=False),encoding='utf-8')
|
||||||
template=(ASSETS / 'q1.template.html').read_text(encoding='utf-8')
|
template=(ASSETS / 'q1.template.html').read_text(encoding='utf-8')
|
||||||
fragment=template.replace('/*__Q1_DATA__*/',json.dumps(dataset,ensure_ascii=False,allow_nan=False))
|
fragment=template.replace('/*__Q1_DATA__*/',
|
||||||
|
json.dumps(dataset,ensure_ascii=False,allow_nan=False))
|
||||||
(ROOT/'output'/'q1'/'question1_full.html').write_text(fragment,encoding='utf-8')
|
(ROOT/'output'/'q1'/'question1_full.html').write_text(fragment,encoding='utf-8')
|
||||||
# A standalone companion uses the same fragment and computation results.
|
# A standalone companion uses the same fragment and computation results.
|
||||||
theme=(ASSETS / 'standalone.css').read_text(encoding='utf-8')
|
theme=(ASSETS / 'standalone.css').read_text(encoding='utf-8')
|
||||||
page='<!doctype html><html lang="zh-CN"><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>B题问题一交会定位</title><style>'+theme+'</style><body>'+fragment+'</body></html>'
|
page=('<!doctype html><html lang="zh-CN"><meta charset="utf-8">'
|
||||||
|
'<meta name="viewport" content="width=device-width,initial-scale=1">'
|
||||||
|
'<title>B题问题一交会定位</title>'
|
||||||
|
'<style>'+theme+'</style><body>'+fragment+'</body></html>')
|
||||||
(results/'question1.html').write_text(page,encoding='utf-8')
|
(results/'question1.html').write_text(page,encoding='utf-8')
|
||||||
print(json.dumps({'cases':len(dataset),'fragment_bytes':len(fragment.encode('utf-8')),
|
print(json.dumps({'cases':len(dataset),'fragment_bytes':len(fragment.encode('utf-8')),
|
||||||
'standalone':str(results/'question1.html')},ensure_ascii=False))
|
'standalone':str(results/'question1.html')},ensure_ascii=False))
|
||||||
|
|||||||
@@ -43,8 +43,10 @@ class PracticeClient:
|
|||||||
status = response.status
|
status = response.status
|
||||||
body = json.load(response)
|
body = json.load(response)
|
||||||
# A retried action is recorded/consumed only once.
|
# A retried action is recorded/consumed only once.
|
||||||
self.events.append(dict(path=path, request=payload, response=body, retries=attempt))
|
self.events.append(dict(path=path, request=payload, response=body,
|
||||||
self.output.write_text(json.dumps(self.events,ensure_ascii=False,indent=2),encoding='utf-8')
|
retries=attempt))
|
||||||
|
self.output.write_text(
|
||||||
|
json.dumps(self.events,ensure_ascii=False,indent=2),encoding='utf-8')
|
||||||
if status != 200 or body.get('accepted') is not True:
|
if status != 200 or body.get('accepted') is not True:
|
||||||
raise RuntimeError(f'动作未被接受: {status}, {body}')
|
raise RuntimeError(f'动作未被接受: {status}, {body}')
|
||||||
if path == '/enter':
|
if path == '/enter':
|
||||||
@@ -54,6 +56,7 @@ class PracticeClient:
|
|||||||
if attempt == 2:
|
if attempt == 2:
|
||||||
raise
|
raise
|
||||||
time.sleep(.25)
|
time.sleep(.25)
|
||||||
|
raise URLError('网络重试次数已用尽')
|
||||||
|
|
||||||
def measure(self, p, channel):
|
def measure(self, p, channel):
|
||||||
return self.action('/measure',position={'x':p[0],'y':p[1]},channel=channel)
|
return self.action('/measure',position={'x':p[0],'y':p[1]},channel=channel)
|
||||||
@@ -106,12 +109,15 @@ def main():
|
|||||||
if len(observations)>=3:
|
if len(observations)>=3:
|
||||||
break
|
break
|
||||||
data={'name':'官方问题3演练观测','source':'official_practice','case_code':args.case_code,
|
data={'name':'官方问题3演练观测','source':'official_practice','case_code':args.case_code,
|
||||||
'channel':channel,'ground_truth':None,'measurements':[m.__dict__ for m in observations],
|
'channel':channel,'ground_truth':None,
|
||||||
|
'measurements':[m.__dict__ for m in observations],
|
||||||
'repeated_station_response':repeat,'result':solve(observations),
|
'repeated_station_response':repeat,'result':solve(observations),
|
||||||
'scope':'仅采集问题一观测,不执行问题三全清除任务;没有调用clear'}
|
'scope':'仅采集问题一观测,不执行问题三全清除任务;没有调用clear'}
|
||||||
path=args.output/f'{args.case_code}_observations.json'
|
path=args.output/f'{args.case_code}_observations.json'
|
||||||
path.write_text(json.dumps(data,ensure_ascii=False,indent=2,allow_nan=False),encoding='utf-8')
|
path.write_text(json.dumps(data,ensure_ascii=False,indent=2,allow_nan=False),
|
||||||
print(json.dumps({'observations':str(path),'count':len(observations),'result':data['result']},ensure_ascii=False,indent=2))
|
encoding='utf-8')
|
||||||
|
print(json.dumps({'observations':str(path),'count':len(observations),
|
||||||
|
'result':data['result']},ensure_ascii=False,indent=2))
|
||||||
finally:
|
finally:
|
||||||
if entered:
|
if entered:
|
||||||
client.action('/exit')
|
client.action('/exit')
|
||||||
|
|||||||
@@ -22,7 +22,8 @@ from referee import LocalSimulator, Source
|
|||||||
|
|
||||||
def main():
|
def main():
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
parser.add_argument('--split', choices=('development','validation','stress'), default='development')
|
parser.add_argument('--split', choices=('development','validation','stress'),
|
||||||
|
default='development')
|
||||||
parser.add_argument('--variants', default='baseline,legacy,revised')
|
parser.add_argument('--variants', default='baseline,legacy,revised')
|
||||||
parser.add_argument('--problems', default='3,4')
|
parser.add_argument('--problems', default='3,4')
|
||||||
parser.add_argument('--model-dir', type=Path, default=Path('output/rl/review-20260911'))
|
parser.add_argument('--model-dir', type=Path, default=Path('output/rl/review-20260911'))
|
||||||
@@ -49,7 +50,8 @@ def main():
|
|||||||
repeated_measures=sum(r['repeated_measures'] for r in part))
|
repeated_measures=sum(r['repeated_measures'] for r in part))
|
||||||
result = dict(source='local_referee_only', config=asdict(config), split=args.split,
|
result = dict(source='local_referee_only', config=asdict(config), split=args.split,
|
||||||
rows=rows, aggregates=aggregates)
|
rows=rows, aggregates=aggregates)
|
||||||
(output/'results.json').write_text(json.dumps(result,ensure_ascii=False,indent=2),encoding='utf-8')
|
(output/'results.json').write_text(
|
||||||
|
json.dumps(result,ensure_ascii=False,indent=2),encoding='utf-8')
|
||||||
return aggregates
|
return aggregates
|
||||||
for problem in map(int, args.problems.split(',')):
|
for problem in map(int, args.problems.split(',')):
|
||||||
scenes = []
|
scenes = []
|
||||||
@@ -71,7 +73,8 @@ def main():
|
|||||||
scenes.append((f'{count}-{seed}',count,seed,sources))
|
scenes.append((f'{count}-{seed}',count,seed,sources))
|
||||||
for name,count,seed,sources in scenes:
|
for name,count,seed,sources in scenes:
|
||||||
scene_data = dict(sources=[asdict(s) for s in sources], error_seed=seed+2000000)
|
scene_data = dict(sources=[asdict(s) for s in sources], error_seed=seed+2000000)
|
||||||
(output/f'q{problem}-{name}-scene.json').write_text(json.dumps(scene_data),encoding='utf-8')
|
scene_path = output/f'q{problem}-{name}-scene.json'
|
||||||
|
scene_path.write_text(json.dumps(scene_data),encoding='utf-8')
|
||||||
for variant in variants:
|
for variant in variants:
|
||||||
model = None
|
model = None
|
||||||
cls = BayesLoop if problem==3 else DirectionalExtension
|
cls = BayesLoop if problem==3 else DirectionalExtension
|
||||||
@@ -81,13 +84,15 @@ def main():
|
|||||||
model = (Path(f'output/rl/q{problem}_ppo.zip') if variant=='legacy'
|
model = (Path(f'output/rl/q{problem}_ppo.zip') if variant=='legacy'
|
||||||
else args.model_dir/f'q{problem}_masked.zip')
|
else args.model_dir/f'q{problem}_masked.zip')
|
||||||
kwargs = dict(model_path=model, device='cpu')
|
kwargs = dict(model_path=model, device='cpu')
|
||||||
sim = LocalSimulator(sources,robot_id='RL-AUDIT',spec=referee_spec(config),error_seed=seed+2000000)
|
sim = LocalSimulator(sources,robot_id='RL-AUDIT',spec=referee_spec(config),
|
||||||
|
error_seed=seed+2000000)
|
||||||
client = SimulatorClient('RL-AUDIT',config=config,transport=LocalSimTransport(sim))
|
client = SimulatorClient('RL-AUDIT',config=config,transport=LocalSimTransport(sim))
|
||||||
start = perf_counter()
|
start = perf_counter()
|
||||||
summary = cls(client,config,**kwargs).run()
|
summary = cls(client,config,**kwargs).run()
|
||||||
events = client.log.action_events()
|
events = client.log.action_events()
|
||||||
truth = {s.channel for s in sources}
|
truth = {s.channel for s in sources}
|
||||||
successful = {e['request']['channel'] for e in events if e['response'].get('clear_result')=='success'}
|
successful = {e['request']['channel'] for e in events
|
||||||
|
if e['response'].get('clear_result')=='success'}
|
||||||
seen = set()
|
seen = set()
|
||||||
repeats = 0
|
repeats = 0
|
||||||
for e in events:
|
for e in events:
|
||||||
@@ -102,12 +107,15 @@ def main():
|
|||||||
and summary['exit_reason']=='user_exit' and not mismatches)
|
and summary['exit_reason']=='user_exit' and not mismatches)
|
||||||
row = dict(case=name,problem=problem,variant=variant,valid=valid,
|
row = dict(case=name,problem=problem,variant=variant,valid=valid,
|
||||||
virtual_s=client.state.virtual_time_s,all_cleared=successful==truth,
|
virtual_s=client.state.virtual_time_s,all_cleared=successful==truth,
|
||||||
measures=summary['measures'],failed_clears=summary['clears']-len(successful),
|
measures=summary['measures'],
|
||||||
|
failed_clears=summary['clears']-len(successful),
|
||||||
repeated_measures=repeats,wall_s=perf_counter()-start,
|
repeated_measures=repeats,wall_s=perf_counter()-start,
|
||||||
model_sha256=hashlib.sha256(model.read_bytes()).hexdigest() if model else None)
|
model_sha256=(hashlib.sha256(model.read_bytes()).hexdigest()
|
||||||
|
if model else None))
|
||||||
rows.append(row)
|
rows.append(row)
|
||||||
(output/f'q{problem}-{name}-{variant}.json').write_text(json.dumps(
|
(output/f'q{problem}-{name}-{variant}.json').write_text(json.dumps(
|
||||||
dict(row=row,summary=summary,events=events,timing_mismatches=mismatches),ensure_ascii=False),encoding='utf-8')
|
dict(row=row,summary=summary,events=events,timing_mismatches=mismatches),
|
||||||
|
ensure_ascii=False),encoding='utf-8')
|
||||||
save()
|
save()
|
||||||
print(json.dumps(row),flush=True)
|
print(json.dumps(row),flush=True)
|
||||||
print(json.dumps(save(),indent=2),flush=True)
|
print(json.dumps(save(),indent=2),flush=True)
|
||||||
|
|||||||
@@ -88,7 +88,7 @@ def collect(problem: int, episodes: int, seed: int, config: ProblemConfig):
|
|||||||
max_episode_steps=16)
|
max_episode_steps=16)
|
||||||
observations, actions, groups = [], [], []
|
observations, actions, groups = [], [], []
|
||||||
for episode in range(episodes):
|
for episode in range(episodes):
|
||||||
obs, info = env.reset(seed=seed + episode)
|
obs, _info = env.reset(seed=seed + episode)
|
||||||
for _step in range(env.max_episode_steps + 2):
|
for _step in range(env.max_episode_steps + 2):
|
||||||
action = expert_action(env)
|
action = expert_action(env)
|
||||||
observations.append(obs)
|
observations.append(obs)
|
||||||
@@ -108,7 +108,8 @@ def train_bc(x, y, *, epochs: int, batch_size: int, lr: float,
|
|||||||
episodes = np.unique(groups)
|
episodes = np.unique(groups)
|
||||||
if len(episodes) < 2:
|
if len(episodes) < 2:
|
||||||
raise ValueError('行为克隆至少需要两个独立episode才能划分验证集')
|
raise ValueError('行为克隆至少需要两个独立episode才能划分验证集')
|
||||||
valid_groups = np.random.default_rng(0).permutation(episodes)[:max(1, int(len(episodes)*valid_frac))]
|
valid_groups = np.random.default_rng(0).permutation(episodes)
|
||||||
|
valid_groups = valid_groups[:max(1, int(len(episodes)*valid_frac))]
|
||||||
valid = torch.as_tensor(np.isin(groups, valid_groups), device=device)
|
valid = torch.as_tensor(np.isin(groups, valid_groups), device=device)
|
||||||
x_t, y_t = torch.as_tensor(x, device=device), torch.as_tensor(y, device=device)
|
x_t, y_t = torch.as_tensor(x, device=device), torch.as_tensor(y, device=device)
|
||||||
x_v, y_v = x_t[valid], y_t[valid]
|
x_v, y_v = x_t[valid], y_t[valid]
|
||||||
|
|||||||
@@ -6,17 +6,17 @@ import math
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
import re
|
import re
|
||||||
import xml.etree.ElementTree as ET
|
import xml.etree.ElementTree as ET
|
||||||
from PIL import Image,ImageDraw,ImageFont
|
from PIL import Image,ImageDraw,ImageFont # pylint: disable=import-error # 可选依赖
|
||||||
|
|
||||||
ROOT=Path(__file__).resolve().parents[1]
|
ROOT=Path(__file__).resolve().parents[1]
|
||||||
S=2
|
S=2
|
||||||
FONT=ImageFont.truetype('C:/Windows/Fonts/msyh.ttc',12*S)
|
FONT=ImageFont.truetype('C:/Windows/Fonts/msyh.ttc',12*S)
|
||||||
|
|
||||||
|
|
||||||
def render(path):
|
def render(svg_path):
|
||||||
root=ET.fromstring(path.read_text(encoding='utf-8'))
|
root=ET.fromstring(svg_path.read_text(encoding='utf-8'))
|
||||||
width,height=map(float,(root.get('width'),root.get('height')))
|
width,height=map(float,(root.get('width'),root.get('height')))
|
||||||
bg='#191c21' if 'dark' in path.stem else '#ffffff'
|
bg='#191c21' if 'dark' in svg_path.stem else '#ffffff'
|
||||||
image=Image.new('RGBA',(round(width*S),round(height*S)),bg)
|
image=Image.new('RGBA',(round(width*S),round(height*S)),bg)
|
||||||
clip=next(el for el in root.iter() if el.tag.endswith('clipPath'))[0]
|
clip=next(el for el in root.iter() if el.tag.endswith('clipPath'))[0]
|
||||||
clipbox=tuple(float(clip.get(k))*S for k in ('x','y','width','height'))
|
clipbox=tuple(float(clip.get(k))*S for k in ('x','y','width','height'))
|
||||||
@@ -34,7 +34,8 @@ def render(path):
|
|||||||
for child in el:draw(child,clipped or bool(el.get('clip-path')))
|
for child in el:draw(child,clipped or bool(el.get('clip-path')))
|
||||||
return
|
return
|
||||||
layer=Image.new('RGBA',image.size,(0,0,0,0));d=ImageDraw.Draw(layer)
|
layer=Image.new('RGBA',image.size,(0,0,0,0));d=ImageDraw.Draw(layer)
|
||||||
num=lambda key,default=0:float(el.get(key,default))*S
|
def num(key,default=0):
|
||||||
|
return float(el.get(key,default))*S
|
||||||
fill=color(el.get('fill'),float(el.get('fill-opacity',1)))
|
fill=color(el.get('fill'),float(el.get('fill-opacity',1)))
|
||||||
stroke=color(el.get('stroke'),float(el.get('stroke-opacity',1)))
|
stroke=color(el.get('stroke'),float(el.get('stroke-opacity',1)))
|
||||||
sw=max(1,round(num('stroke-width',1)))
|
sw=max(1,round(num('stroke-width',1)))
|
||||||
@@ -43,13 +44,15 @@ def render(path):
|
|||||||
dash=el.get('stroke-dasharray','none')
|
dash=el.get('stroke-dasharray','none')
|
||||||
if dash=='none': d.line(xy,fill=stroke,width=sw)
|
if dash=='none': d.line(xy,fill=stroke,width=sw)
|
||||||
else:
|
else:
|
||||||
a,b=xy;dist=math.dist(a,b);unit=((b[0]-a[0])/max(dist,1e-15),(b[1]-a[1])/max(dist,1e-15))
|
a,b=xy;dist=math.dist(a,b)
|
||||||
|
unit=((b[0]-a[0])/max(dist,1e-15),(b[1]-a[1])/max(dist,1e-15))
|
||||||
on,off=[float(v)*S for v in dash.split()]
|
on,off=[float(v)*S for v in dash.split()]
|
||||||
# Bound offscreen ray work to the finite exported viewport vicinity.
|
# Bound offscreen ray work to the finite exported viewport vicinity.
|
||||||
segments=min(100000,int(dist/(on+off))+1)
|
segments=min(100000,int(dist/(on+off))+1)
|
||||||
for i in range(segments):
|
for i in range(segments):
|
||||||
lo=i*(on+off);hi=min(lo+on,dist)
|
lo=i*(on+off);hi=min(lo+on,dist)
|
||||||
d.line([(a[0]+lo*unit[0],a[1]+lo*unit[1]),(a[0]+hi*unit[0],a[1]+hi*unit[1])],fill=stroke,width=sw)
|
d.line([(a[0]+lo*unit[0],a[1]+lo*unit[1]),
|
||||||
|
(a[0]+hi*unit[0],a[1]+hi*unit[1])],fill=stroke,width=sw)
|
||||||
elif tag=='rect':
|
elif tag=='rect':
|
||||||
box=(num('x'),num('y'),num('x')+num('width'),num('y')+num('height'))
|
box=(num('x'),num('y'),num('x')+num('width'),num('y')+num('height'))
|
||||||
d.rectangle(box,fill=fill,outline=stroke,width=sw)
|
d.rectangle(box,fill=fill,outline=stroke,width=sw)
|
||||||
@@ -63,7 +66,8 @@ def render(path):
|
|||||||
d.polygon(points,fill=fill)
|
d.polygon(points,fill=fill)
|
||||||
if stroke:d.line(points+[points[0]],fill=stroke,width=sw,joint='curve')
|
if stroke:d.line(points+[points[0]],fill=stroke,width=sw,joint='curve')
|
||||||
elif tag=='path':
|
elif tag=='path':
|
||||||
for x,y,axis,length in re.findall(r'M([\d.e+-]+),([\d.e+-]+)([hv])([\d.e+-]+)',el.get('d')):
|
pattern=r'M([\d.e+-]+),([\d.e+-]+)([hv])([\d.e+-]+)'
|
||||||
|
for x,y,axis,length in re.findall(pattern,el.get('d')):
|
||||||
x,y,length=float(x)*S,float(y)*S,float(length)*S
|
x,y,length=float(x)*S,float(y)*S,float(length)*S
|
||||||
d.line([(x,y),(x+length,y) if axis=='h' else (x,y+length)],fill=stroke,width=sw)
|
d.line([(x,y),(x+length,y) if axis=='h' else (x,y+length)],fill=stroke,width=sw)
|
||||||
elif tag=='text':
|
elif tag=='text':
|
||||||
@@ -86,7 +90,7 @@ def render(path):
|
|||||||
image=Image.alpha_composite(image,layer)
|
image=Image.alpha_composite(image,layer)
|
||||||
|
|
||||||
draw(root)
|
draw(root)
|
||||||
output=path.with_suffix('.png');image.convert('RGB').save(output)
|
output=svg_path.with_suffix('.png');image.convert('RGB').save(output)
|
||||||
return str(output.relative_to(ROOT))
|
return str(output.relative_to(ROOT))
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -32,8 +32,9 @@ def main():
|
|||||||
metadata = {"source": "official_practice", "problem": 3,
|
metadata = {"source": "official_practice", "problem": 3,
|
||||||
"strategy": "seven_batch", "started_local": datetime.now().astimezone().isoformat(),
|
"strategy": "seven_batch", "started_local": datetime.now().astimezone().isoformat(),
|
||||||
"config": asdict(config), "coverage_points": seven_points(),
|
"config": asdict(config), "coverage_points": seven_points(),
|
||||||
"source_sha256": {str(p.relative_to(root)): hashlib.sha256(p.read_bytes()).hexdigest()
|
"source_sha256": {
|
||||||
for p in paths}}
|
str(p.relative_to(root)): hashlib.sha256(p.read_bytes()).hexdigest()
|
||||||
|
for p in paths}}
|
||||||
def save(name, value):
|
def save(name, value):
|
||||||
(args.output / name).write_text(json.dumps(value, ensure_ascii=False, indent=2,
|
(args.output / name).write_text(json.dumps(value, ensure_ascii=False, indent=2,
|
||||||
allow_nan=False) + "\n", encoding="utf-8")
|
allow_nan=False) + "\n", encoding="utf-8")
|
||||||
|
|||||||
@@ -12,7 +12,8 @@ from mcm26b.rl.features import build_observation
|
|||||||
|
|
||||||
def main():
|
def main():
|
||||||
root = Path('output/rl/review-20260911')
|
root = Path('output/rl/review-20260911')
|
||||||
records = json.loads((root/'evaluation/validation/final/results.json').read_text(encoding='utf-8'))
|
records = json.loads((root/'evaluation/validation/final/results.json')
|
||||||
|
.read_text(encoding='utf-8'))
|
||||||
rng = np.random.default_rng(20260911)
|
rng = np.random.default_rng(20260911)
|
||||||
comparisons = {}
|
comparisons = {}
|
||||||
for problem in (3,4):
|
for problem in (3,4):
|
||||||
@@ -46,13 +47,16 @@ def main():
|
|||||||
bc = {}
|
bc = {}
|
||||||
for problem in (3,4):
|
for problem in (3,4):
|
||||||
# 使用旧检查点和原始专家输入,比对原ReLU网络与拷贝后的Tanh网络。
|
# 使用旧检查点和原始专家输入,比对原ReLU网络与拷贝后的Tanh网络。
|
||||||
checkpoint = torch.load(f'output/rl/bc/q{problem}_bc.pt',map_location='cpu',weights_only=True)
|
checkpoint = torch.load(f'output/rl/bc/q{problem}_bc.pt',map_location='cpu',
|
||||||
|
weights_only=True)
|
||||||
data = np.load(f'output/rl/bc/q{problem}_expert.npz',allow_pickle=False)
|
data = np.load(f'output/rl/bc/q{problem}_expert.npz',allow_pickle=False)
|
||||||
x = torch.from_numpy(data['obs'][::max(1,len(data['obs'])//1000)].copy())
|
x = torch.from_numpy(data['obs'][::max(1,len(data['obs'])//1000)].copy())
|
||||||
def network(activation):
|
|
||||||
|
def network(activation, checkpoint=checkpoint):
|
||||||
net = torch.nn.Sequential(torch.nn.Linear(401,128),activation(),
|
net = torch.nn.Sequential(torch.nn.Linear(401,128),activation(),
|
||||||
torch.nn.Linear(128,128),activation(),torch.nn.Linear(128,60))
|
torch.nn.Linear(128,128),activation(),torch.nn.Linear(128,60))
|
||||||
net.load_state_dict({k.removeprefix('net.'):v for k,v in checkpoint['state_dict'].items()})
|
state = {k.removeprefix('net.'):v for k,v in checkpoint['state_dict'].items()}
|
||||||
|
net.load_state_dict(state)
|
||||||
return net
|
return net
|
||||||
with torch.no_grad():
|
with torch.no_grad():
|
||||||
before,after=network(torch.nn.ReLU)(x),network(torch.nn.Tanh)(x)
|
before,after=network(torch.nn.ReLU)(x),network(torch.nn.Tanh)(x)
|
||||||
@@ -63,9 +67,11 @@ def main():
|
|||||||
limitations='置信区间仅重采样固定模型的场景,不覆盖训练随机种子不确定性。',
|
limitations='置信区间仅重采样固定模型的场景,不覆盖训练随机种子不确定性。',
|
||||||
code_sha256={str(p):hashlib.sha256(p.read_bytes()).hexdigest()
|
code_sha256={str(p):hashlib.sha256(p.read_bytes()).hexdigest()
|
||||||
for p in sorted(Path('src/mcm26b/rl').glob('*.py'))})
|
for p in sorted(Path('src/mcm26b/rl').glob('*.py'))})
|
||||||
(root/'review_evidence.json').write_text(json.dumps(report,ensure_ascii=False,indent=2),encoding='utf-8')
|
(root/'review_evidence.json').write_text(
|
||||||
|
json.dumps(report,ensure_ascii=False,indent=2),encoding='utf-8')
|
||||||
print(json.dumps(dict(comparisons=comparisons,feature_probe=features,bc_probe=bc,
|
print(json.dumps(dict(comparisons=comparisons,feature_probe=features,bc_probe=bc,
|
||||||
official=[dict(case=r['case_code'],passed=r['passed']) for r in official]),indent=2))
|
official=[dict(case=r['case_code'],passed=r['passed'])
|
||||||
|
for r in official]),indent=2))
|
||||||
|
|
||||||
|
|
||||||
if __name__=='__main__':
|
if __name__=='__main__':
|
||||||
|
|||||||
+382
-107
@@ -8,13 +8,31 @@
|
|||||||
当前 HTTP 执行路径仍由高层宏动作终点决定,不把八方向推演当正式收益。
|
当前 HTTP 执行路径仍由高层宏动作终点决定,不把八方向推演当正式收益。
|
||||||
|
|
||||||
落盘保证(长训必需):
|
落盘保证(长训必需):
|
||||||
- ``--checkpoint-every`` 每 N 个环境步在 ``<模型目录>/checkpoints/`` 落一个可续训
|
- **急停可恢复**:``--recovery-every``(默认 2000 环境步)把一份滚动恢复点
|
||||||
的 checkpoint(zip + 同名 json 元数据),只保留最近 ``--checkpoint-keep`` 个;
|
``<模型目录>/checkpoints/<stem>_latest.zip`` 原子覆盖写;训练开始前先写一份
|
||||||
- 训练前端的「停止」是对**进程组**发 SIGTERM,因此 subproc worker 在 :func:`_make_env`
|
起步基线。因此即使进程被 SIGKILL / ``taskkill /F`` 强杀(没有任何 Python
|
||||||
里显式忽略 SIGTERM/SIGINT,主进程收到信号后在 step 边界收尾并保存模型,
|
收尾机会),也**最多只丢 ``--recovery-every`` 个环境步**,且磁盘上始终有一份
|
||||||
不会被 worker 先行退出打断;
|
完整、可加载、可 ``--resume`` 的产物;
|
||||||
- 任何异常(含被强杀之外的失败)都会先落一份 ``*_emergency.zip`` 再抛出,
|
- **原子落盘**:模型 zip、旁挂 json 与 ``<run_dir>/resume.json`` 都是"临时文件 +
|
||||||
不再出现"跑了很久什么都没有"。
|
``os.replace``",保存途中被杀不会留下半截文件,也不会破坏上一份完整快照;
|
||||||
|
- ``--checkpoint-every`` 每 N 个环境步在 ``<模型目录>/checkpoints/`` 另留一份历史
|
||||||
|
checkpoint(``<stem>_<步数>.zip``),只保留最近 ``--checkpoint-keep`` 个,供
|
||||||
|
返祖对比与多档恢复;
|
||||||
|
- ``<run_dir>/resume.json`` 是恢复的**权威指针**:无论上一刻是滚动恢复点、历史
|
||||||
|
checkpoint、优雅收尾还是异常兜底,它总指向一份可续训产物,并记录增量语义
|
||||||
|
(``timesteps`` 是追加步数)与生效过的 PPO 超参;
|
||||||
|
- ``--control-file`` 是训练前端的**跨平台**控制通道:前端原子写 JSON
|
||||||
|
(``paused`` / ``stop``),本脚本每个 step 轮询。暂停时训练在 step 边界挂起,
|
||||||
|
停止时先保存模型与元数据再退出;
|
||||||
|
- 信号(SIGTERM/SIGINT,Windows 上另有 SIGBREAK)仍可用:直接 kill 本脚本或
|
||||||
|
前端发进程组信号时同样走优雅收尾;subproc worker 在 :func:`_make_env` 里忽略
|
||||||
|
这些信号,保证它们活到主进程 ``vec_env.close()`` 之后再退出;
|
||||||
|
- 任何异常都会先落一份 ``*_emergency.zip`` + ``resume.json`` 再抛出,不再出现
|
||||||
|
"跑了很久什么都没有"。
|
||||||
|
|
||||||
|
为什么暂停/停止不靠信号:Windows 没有 SIGSTOP,且 ``os.kill(pid, SIGTERM)`` 会
|
||||||
|
退化成 ``TerminateProcess`` 硬杀——收尾分支根本不会执行。控制文件两端都只是
|
||||||
|
普通文件 IO,POSIX 与 Windows 行为一致(见 :mod:`mcm26b.process_control`)。
|
||||||
|
|
||||||
P1 smoke 示例:
|
P1 smoke 示例:
|
||||||
python scripts/train_hrl_high.py \
|
python scripts/train_hrl_high.py \
|
||||||
@@ -29,6 +47,7 @@ from __future__ import annotations
|
|||||||
import argparse
|
import argparse
|
||||||
import hashlib
|
import hashlib
|
||||||
import json
|
import json
|
||||||
|
import math
|
||||||
import os
|
import os
|
||||||
import platform
|
import platform
|
||||||
import random
|
import random
|
||||||
@@ -39,9 +58,10 @@ from pathlib import Path
|
|||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from mcm26b.config import ProblemConfig
|
from mcm26b.config import ProblemConfig
|
||||||
from mcm26b.hrl.curriculum import STAGES, training_config
|
from mcm26b.hrl.curriculum import STAGES
|
||||||
from mcm26b.hrl.constants import (ACTION_LAYOUT_VERSION, FEATURE_VERSION,
|
from mcm26b.hrl.constants import (ACTION_LAYOUT_VERSION, FEATURE_VERSION,
|
||||||
HIGH_OBS_DIM, N_CANDIDATES, REWARD_VERSION)
|
HIGH_OBS_DIM, N_CANDIDATES, REWARD_VERSION)
|
||||||
|
from mcm26b.process_control import ControlState, read_control
|
||||||
|
|
||||||
|
|
||||||
def _stage_from_index(index: int):
|
def _stage_from_index(index: int):
|
||||||
@@ -68,24 +88,86 @@ def _reward_from_profile(profile: str):
|
|||||||
raise ValueError(f"未知 reward-profile: {profile}")
|
raise ValueError(f"未知 reward-profile: {profile}")
|
||||||
|
|
||||||
|
|
||||||
def _ignore_stop_signals_in_worker() -> None:
|
def _stop_signals() -> list[int]:
|
||||||
"""subproc worker 忽略 SIGTERM/SIGINT,保证主进程能优雅收尾落盘。
|
"""本平台可用于"请求收尾"的信号集合。
|
||||||
|
|
||||||
训练前端的「停止」向整个进程组发 SIGTERM;若 worker 同时退出,主进程会在
|
Windows 上没有 SIGTERM 的软语义(``os.kill`` 直接 TerminateProcess),但
|
||||||
保存模型之前就拿到断管/EOF 错误,整局训练白跑。worker 活在管道后面,主进程
|
``SIGBREAK`` 可以由 ``CTRL_BREAK_EVENT`` 触发并进入 Python 处理器;POSIX 上
|
||||||
收尾时通过 ``vec_env.close()`` 的 ``close`` 指令正常结束它们。
|
SIGTERM/SIGINT 均可捕获。控制文件才是主通道,这里只是让"直接 kill 脚本"
|
||||||
|
也尽量走优雅收尾。
|
||||||
|
"""
|
||||||
|
signals = [signal.SIGTERM, signal.SIGINT]
|
||||||
|
sigbreak = getattr(signal, "SIGBREAK", None)
|
||||||
|
if sigbreak is not None:
|
||||||
|
signals.append(sigbreak)
|
||||||
|
return signals
|
||||||
|
|
||||||
|
|
||||||
|
def _ignore_stop_signals_in_worker() -> None:
|
||||||
|
"""subproc worker 忽略收尾信号,保证主进程能优雅落盘。
|
||||||
|
|
||||||
|
无论信号来自进程组 SIGTERM 还是 Windows ``CTRL_BREAK_EVENT``,若 worker
|
||||||
|
同时退出,主进程会在保存模型之前就拿到断管/EOF 错误,整局训练白跑。worker
|
||||||
|
活在管道后面,主进程收尾时通过 ``vec_env.close()`` 的 ``close`` 指令正常
|
||||||
|
结束它们。
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
import multiprocessing
|
import multiprocessing
|
||||||
if multiprocessing.current_process().name == "MainProcess":
|
if multiprocessing.current_process().name == "MainProcess":
|
||||||
return
|
return
|
||||||
for signum in (signal.SIGTERM, signal.SIGINT):
|
for signum in _stop_signals():
|
||||||
signal.signal(signum, signal.SIG_IGN)
|
signal.signal(signum, signal.SIG_IGN)
|
||||||
except (ImportError, ValueError, OSError): # 非主线程/平台不支持:不影响训练
|
except (ImportError, ValueError, OSError): # 非主线程/平台不支持:不影响训练
|
||||||
return
|
return
|
||||||
|
|
||||||
|
|
||||||
def _make_env(problem: int, stage_index: int, seed: int,
|
class _ControlPoll:
|
||||||
|
"""控制文件轮询状态机(纯标准库,可脱离 SB3 单测)。
|
||||||
|
|
||||||
|
``poll()`` 每 ``poll_every`` 次调用才真正读盘,避免每个 env step 都做一次
|
||||||
|
文件 IO;``wait_while_paused()`` 阻塞到暂停解除或收到 stop,并累计暂停秒数
|
||||||
|
(``--max-wall-s`` 要把这段扣掉,否则"挂起一小时回来就判超时")。
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, path: Path, *, poll_every: int = 32,
|
||||||
|
poll_s: float = 0.2):
|
||||||
|
self.path = Path(path)
|
||||||
|
self.poll_every = max(1, int(poll_every))
|
||||||
|
self.poll_s = max(0.02, float(poll_s))
|
||||||
|
self.paused_total_s = 0.0
|
||||||
|
self._calls = 0
|
||||||
|
self._state = ControlState()
|
||||||
|
|
||||||
|
def poll(self) -> ControlState:
|
||||||
|
"""按抽样间隔返回最近一次控制状态。"""
|
||||||
|
self._calls += 1
|
||||||
|
if self._calls % self.poll_every:
|
||||||
|
return self._state
|
||||||
|
self._state = read_control(self.path)
|
||||||
|
return self._state
|
||||||
|
|
||||||
|
def wait_while_paused(self, on_pause, on_resume) -> bool:
|
||||||
|
"""阻塞到继续或停止;返回 True 表示期间收到了 stop。
|
||||||
|
|
||||||
|
``on_pause`` / ``on_resume`` 用来向前端回显事件(训练脚本的 JSON 行)。
|
||||||
|
"""
|
||||||
|
on_pause()
|
||||||
|
started = time.monotonic()
|
||||||
|
stopped = False
|
||||||
|
while True:
|
||||||
|
time.sleep(self.poll_s)
|
||||||
|
self._state = read_control(self.path)
|
||||||
|
if self._state.stop:
|
||||||
|
stopped = True
|
||||||
|
break
|
||||||
|
if not self._state.paused:
|
||||||
|
break
|
||||||
|
self.paused_total_s += time.monotonic() - started
|
||||||
|
on_resume()
|
||||||
|
return stopped
|
||||||
|
|
||||||
|
|
||||||
|
def _make_env(problem: int, stage_index: int,
|
||||||
monitor_path: str | None, profile: str,
|
monitor_path: str | None, profile: str,
|
||||||
reward_profile: str):
|
reward_profile: str):
|
||||||
"""顶层工厂:供 DummyVecEnv / SubprocVecEnv 调用(必须是可 pickle 的)。"""
|
"""顶层工厂:供 DummyVecEnv / SubprocVecEnv 调用(必须是可 pickle 的)。"""
|
||||||
@@ -112,7 +194,7 @@ def _build_vec_env(args, monitor_paths: list[str]):
|
|||||||
from functools import partial
|
from functools import partial
|
||||||
|
|
||||||
factories = [
|
factories = [
|
||||||
partial(_make_env, args.problem, args.stage, args.seed + rank,
|
partial(_make_env, args.problem, args.stage,
|
||||||
monitor_paths[rank], args.config_profile, args.reward_profile)
|
monitor_paths[rank], args.config_profile, args.reward_profile)
|
||||||
for rank in range(args.envs)
|
for rank in range(args.envs)
|
||||||
]
|
]
|
||||||
@@ -161,6 +243,52 @@ def _policy_hyperparams(model) -> dict[str, Any]:
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _progress_payload(model, start: int) -> dict[str, Any]:
|
||||||
|
"""组装 ``--progress-json`` 的单行负载。
|
||||||
|
|
||||||
|
纯函数(只读 model 的 logger / ep_info_buffer),便于脱离训练循环测试;
|
||||||
|
NaN / ±inf 一律折算成 None,保证 ``json.dumps(allow_nan=False)`` 不会炸。
|
||||||
|
"""
|
||||||
|
values = getattr(model.logger, "name_to_value", {}) or {}
|
||||||
|
|
||||||
|
def _num(key, default=None):
|
||||||
|
value = values.get(key)
|
||||||
|
if value is None:
|
||||||
|
return default
|
||||||
|
try:
|
||||||
|
value = float(value)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return default
|
||||||
|
if not math.isfinite(value):
|
||||||
|
return default
|
||||||
|
return value
|
||||||
|
|
||||||
|
ep_rewards: list[float] = []
|
||||||
|
ep_lengths: list[float] = []
|
||||||
|
for item in getattr(model, "ep_info_buffer", []) or []:
|
||||||
|
try:
|
||||||
|
ep_rewards.append(float(item.get("r")))
|
||||||
|
ep_lengths.append(float(item.get("l")))
|
||||||
|
except (TypeError, ValueError, AttributeError):
|
||||||
|
continue
|
||||||
|
return {
|
||||||
|
"event": "progress",
|
||||||
|
"timesteps": int(model.num_timesteps),
|
||||||
|
"start_timesteps": int(start),
|
||||||
|
"n_updates": int(getattr(model, "_n_updates", 0) or 0),
|
||||||
|
"ep_rew_mean": (sum(ep_rewards) / len(ep_rewards)
|
||||||
|
if ep_rewards else None),
|
||||||
|
"ep_len_mean": (sum(ep_lengths) / len(ep_lengths)
|
||||||
|
if ep_lengths else None),
|
||||||
|
"value_loss": _num("train/value_loss"),
|
||||||
|
"policy_gradient_loss": _num("train/policy_gradient_loss"),
|
||||||
|
"approx_kl": _num("train/approx_kl"),
|
||||||
|
"entropy_loss": _num("train/entropy_loss"),
|
||||||
|
"explained_variance": _num("train/explained_variance"),
|
||||||
|
"fps": _num("time/fps"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def _model_metadata(*, args, wall_s: float, n_envs: int,
|
def _model_metadata(*, args, wall_s: float, n_envs: int,
|
||||||
actual_timesteps: int) -> dict[str, Any]:
|
actual_timesteps: int) -> dict[str, Any]:
|
||||||
import numpy
|
import numpy
|
||||||
@@ -190,12 +318,12 @@ def _model_metadata(*, args, wall_s: float, n_envs: int,
|
|||||||
"config_profile": args.config_profile,
|
"config_profile": args.config_profile,
|
||||||
"fidelity": _profile_fidelity(args.config_profile),
|
"fidelity": _profile_fidelity(args.config_profile),
|
||||||
"reward_profile": args.reward_profile,
|
"reward_profile": args.reward_profile,
|
||||||
"reward_version": REWARD_VERSION,
|
|
||||||
"local_selector": args.local_selector,
|
"local_selector": args.local_selector,
|
||||||
"local_model": args.local_model,
|
"local_model": args.local_model,
|
||||||
"checkpoint_every": int(getattr(args, "checkpoint_every", 0) or 0),
|
"checkpoint_every": int(getattr(args, "checkpoint_every", 0) or 0),
|
||||||
"checkpoint_keep": int(getattr(args, "checkpoint_keep", 0) or 0),
|
"checkpoint_keep": int(getattr(args, "checkpoint_keep", 0) or 0),
|
||||||
"checkpoint_dir": getattr(args, "checkpoint_dir", None),
|
"checkpoint_dir": getattr(args, "checkpoint_dir", None),
|
||||||
|
"recovery_every": int(getattr(args, "recovery_every", 0) or 0),
|
||||||
"device": args.device,
|
"device": args.device,
|
||||||
"wall_seconds": round(wall_s, 3),
|
"wall_seconds": round(wall_s, 3),
|
||||||
"max_wall_s": float(args.max_wall_s),
|
"max_wall_s": float(args.max_wall_s),
|
||||||
@@ -211,6 +339,68 @@ def _model_metadata(*, args, wall_s: float, n_envs: int,
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _write_json_atomic(path: Path, payload: dict[str, Any]) -> None:
|
||||||
|
"""原子写 JSON:同目录 ``.tmp-`` 前缀临时文件 + ``os.replace``。
|
||||||
|
|
||||||
|
急停(SIGKILL / ``taskkill /F``)会绕过所有 Python 清理逻辑,保存途中被杀
|
||||||
|
是完全可能的。原子替换保证**真实文件名下要么是旧的完整文件、要么是新的
|
||||||
|
完整文件**,不会出现半截 JSON 让续训读到一半就崩。
|
||||||
|
"""
|
||||||
|
path = Path(path)
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
tmp = path.with_name(f".tmp-{path.name}")
|
||||||
|
tmp.write_text(json.dumps(payload, ensure_ascii=False, indent=2),
|
||||||
|
encoding="utf-8")
|
||||||
|
os.replace(tmp, path)
|
||||||
|
|
||||||
|
|
||||||
|
def _save_model_atomic(model, target: Path) -> Path:
|
||||||
|
"""把 SB3 模型原子地写到 ``target``,返回实际落盘路径。
|
||||||
|
|
||||||
|
先写 ``.tmp-<name>``(同样以 ``.zip`` 结尾,SB3 不会再加后缀),完整写完后
|
||||||
|
``os.replace`` 覆盖目标。于是:
|
||||||
|
|
||||||
|
- 保存途中被强杀 → 目标路径仍是上一份**完整**的快照,可继续复训;
|
||||||
|
- 保存成功 → 目标路径立即变成新快照,不存在"看起来存在但打不开"的窗口。
|
||||||
|
|
||||||
|
临时名带 ``.tmp-`` 前缀,因此不会被 ``*_ckpt_*.zip`` 的清理逻辑误当成正式
|
||||||
|
checkpoint。
|
||||||
|
"""
|
||||||
|
target = Path(target)
|
||||||
|
target.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
tmp = target.with_name(f".tmp-{target.name}")
|
||||||
|
model.save(str(tmp))
|
||||||
|
if not tmp.exists(): # SB3 对未知后缀可能自行补 .zip
|
||||||
|
candidate = tmp.with_name(tmp.name + ".zip")
|
||||||
|
if candidate.exists():
|
||||||
|
tmp = candidate
|
||||||
|
os.replace(tmp, target)
|
||||||
|
return target
|
||||||
|
|
||||||
|
|
||||||
|
def _write_resume_hint(*, run_dir: Path, model_path: Path, timesteps: int,
|
||||||
|
reason: str, model=None) -> dict[str, Any]:
|
||||||
|
"""把"从哪继续"写进 ``<run_dir>/resume.json``。
|
||||||
|
|
||||||
|
这是急停恢复的**权威指针**:无论上一刻是周期 checkpoint、滚动恢复点、
|
||||||
|
优雅收尾还是异常兜底,这个文件总指向一份完整、可 ``--resume`` 的产物,
|
||||||
|
并带上续训语义(``timesteps`` 是增量步数)与生效过的 PPO 超参。
|
||||||
|
"""
|
||||||
|
hint: dict[str, Any] = {
|
||||||
|
"event": "resume_hint",
|
||||||
|
"model": str(model_path),
|
||||||
|
"timesteps": int(timesteps),
|
||||||
|
"timesteps_semantics": "additional",
|
||||||
|
"reason": str(reason),
|
||||||
|
"created_unix_s": time.time(),
|
||||||
|
"resume_args": ["--resume", "--model", str(model_path)],
|
||||||
|
}
|
||||||
|
if model is not None:
|
||||||
|
hint["ppo_hyperparams"] = _policy_hyperparams(model)
|
||||||
|
_write_json_atomic(run_dir / "resume.json", hint)
|
||||||
|
return hint
|
||||||
|
|
||||||
|
|
||||||
def _write_metadata(*, model_path: Path, args, wall_s: float, n_envs: int,
|
def _write_metadata(*, model_path: Path, args, wall_s: float, n_envs: int,
|
||||||
actual_timesteps: int, monitor_paths: list[str],
|
actual_timesteps: int, monitor_paths: list[str],
|
||||||
model=None,
|
model=None,
|
||||||
@@ -227,14 +417,18 @@ def _write_metadata(*, model_path: Path, args, wall_s: float, n_envs: int,
|
|||||||
"reward_profile=prompt 使用原始 §5.2 塑形。模型元数据已固化口径。")
|
"reward_profile=prompt 使用原始 §5.2 塑形。模型元数据已固化口径。")
|
||||||
if extra:
|
if extra:
|
||||||
metadata.update(extra)
|
metadata.update(extra)
|
||||||
model_path.with_suffix(".json").write_text(
|
_write_json_atomic(model_path.with_suffix(".json"), metadata)
|
||||||
json.dumps(metadata, ensure_ascii=False, indent=2), encoding="utf-8")
|
|
||||||
return metadata
|
return metadata
|
||||||
|
|
||||||
|
|
||||||
def _prune_checkpoints(directory: Path, stem: str, keep: int) -> list[str]:
|
def _prune_checkpoints(directory: Path, stem: str, keep: int) -> list[str]:
|
||||||
"""只保留最近 ``keep`` 个 checkpoint(zip+json 同删),返回被删的 zip 路径。"""
|
"""只保留最近 ``keep`` 个 checkpoint(zip+json 同删),返回被删的 zip 路径。
|
||||||
files = sorted(directory.glob(f"{stem}_*.zip"),
|
|
||||||
|
只匹配 ``<stem>_<步数>.zip``;滚动恢复点 ``<stem>_latest.zip`` 与临时文件
|
||||||
|
``.tmp-*.zip`` 都不匹配,因此永远不会被历史清理误删。
|
||||||
|
"""
|
||||||
|
files = sorted((path for path in directory.glob(f"{stem}_[0-9]*.zip")
|
||||||
|
if path.stem.rsplit("_", 1)[-1].isdigit()),
|
||||||
key=lambda path: path.stat().st_mtime, reverse=True)
|
key=lambda path: path.stat().st_mtime, reverse=True)
|
||||||
removed: list[str] = []
|
removed: list[str] = []
|
||||||
for stale in files[max(1, int(keep)):]:
|
for stale in files[max(1, int(keep)):]:
|
||||||
@@ -267,7 +461,7 @@ def _evaluate_env(model, *, problem: int, stage_index: int,
|
|||||||
while not done:
|
while not done:
|
||||||
action, _state = model.predict(
|
action, _state = model.predict(
|
||||||
obs, action_masks=env.action_masks(), deterministic=True)
|
obs, action_masks=env.action_masks(), deterministic=True)
|
||||||
obs, reward, terminated, truncated, info = env.step(int(action))
|
obs, _, terminated, truncated, info = env.step(int(action))
|
||||||
done = bool(terminated or truncated)
|
done = bool(terminated or truncated)
|
||||||
steps += 1
|
steps += 1
|
||||||
# 课程阶段的小场景只需证明全部频道 accounted;完整阶段至少应清掉
|
# 课程阶段的小场景只需证明全部频道 accounted;完整阶段至少应清掉
|
||||||
@@ -299,7 +493,7 @@ def _evaluate_env(model, *, problem: int, stage_index: int,
|
|||||||
|
|
||||||
|
|
||||||
def _evaluate_manifest(model_path: Path, manifest_path: str, *,
|
def _evaluate_manifest(model_path: Path, manifest_path: str, *,
|
||||||
limit: int, problem: int) -> dict[str, Any]:
|
limit: int) -> dict[str, Any]:
|
||||||
"""可选:用 benchmark_hrl 的同一套真值隔离评测在固定 manifest 上跑已训模型。"""
|
"""可选:用 benchmark_hrl 的同一套真值隔离评测在固定 manifest 上跑已训模型。"""
|
||||||
scripts_dir = Path(__file__).resolve().parent
|
scripts_dir = Path(__file__).resolve().parent
|
||||||
if str(scripts_dir) not in sys.path:
|
if str(scripts_dir) not in sys.path:
|
||||||
@@ -404,26 +598,72 @@ def run_training(args) -> int:
|
|||||||
"命令行 rollout-steps/batch-size/n-epochs 被忽略")},
|
"命令行 rollout-steps/batch-size/n-epochs 被忽略")},
|
||||||
ensure_ascii=False), file=sys.stderr, flush=True)
|
ensure_ascii=False), file=sys.stderr, flush=True)
|
||||||
|
|
||||||
# ------------------------------------------------------------ 优雅停止
|
# ------------------------------------------------------ 优雅暂停 / 停止
|
||||||
# 训练前端「停止」= killpg(SIGTERM):worker 已在 _make_env 里忽略该信号,
|
# 主通道是控制文件(跨平台);信号是兜底:直接 kill 本进程时也走同一段收尾。
|
||||||
# 主进程在此装处理器,等当前 step 结束后收尾落盘。
|
|
||||||
stop_state: dict[str, Any] = {"requested": False, "signal": None}
|
stop_state: dict[str, Any] = {"requested": False, "signal": None}
|
||||||
previous_handlers: dict[int, Any] = {}
|
previous_handlers: dict[int, Any] = {}
|
||||||
|
|
||||||
def _on_stop_signal(signum, _frame):
|
def _request_stop(source: str) -> None:
|
||||||
|
if stop_state["requested"]:
|
||||||
|
return
|
||||||
stop_state["requested"] = True
|
stop_state["requested"] = True
|
||||||
stop_state["signal"] = signal.Signals(signum).name
|
stop_state["signal"] = source
|
||||||
print(json.dumps({"event": "stop_requested",
|
print(json.dumps({"event": "stop_requested", "signal": source}), flush=True)
|
||||||
"signal": signal.Signals(signum).name}), flush=True)
|
|
||||||
|
|
||||||
for _signum in (signal.SIGTERM, signal.SIGINT):
|
def _on_stop_signal(signum, _frame):
|
||||||
|
_request_stop(signal.Signals(signum).name)
|
||||||
|
|
||||||
|
for _signum in _stop_signals():
|
||||||
try:
|
try:
|
||||||
previous_handlers[_signum] = signal.signal(_signum, _on_stop_signal)
|
previous_handlers[_signum] = signal.signal(_signum, _on_stop_signal)
|
||||||
except (ValueError, OSError): # 非主线程/平台不支持:不阻塞训练
|
except (ValueError, OSError): # 非主线程/平台不支持:不阻塞训练
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
class _ControlChannel(BaseCallback):
|
||||||
|
"""把 :class:`_ControlPoll` 接到 SB3 的 step 回调上。
|
||||||
|
|
||||||
|
为什么不用信号:Windows 没有 SIGSTOP,且 ``os.kill(SIGTERM)`` 会退化成
|
||||||
|
``TerminateProcess`` 硬杀,收尾分支根本不会执行。控制文件在 POSIX 与
|
||||||
|
Windows 上都只是普通文件 IO,语义完全一致。
|
||||||
|
|
||||||
|
暂停是**协作式**的:在 ``_on_step`` 内阻塞轮询,训练因此停在 step 边界,
|
||||||
|
进程仍存活、显存仍占用,但不消耗算力;收到 stop 立刻转入收尾落盘。
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, path: Path, *, poll_every: int = 32,
|
||||||
|
poll_s: float = 0.2):
|
||||||
|
super().__init__()
|
||||||
|
self.poller = _ControlPoll(path, poll_every=poll_every,
|
||||||
|
poll_s=poll_s)
|
||||||
|
|
||||||
|
def paused_seconds(self) -> float:
|
||||||
|
return self.poller.paused_total_s
|
||||||
|
|
||||||
|
def _event(self, name: str) -> None:
|
||||||
|
print(json.dumps({"event": name,
|
||||||
|
"timesteps": int(self.model.num_timesteps)}),
|
||||||
|
flush=True)
|
||||||
|
|
||||||
|
def _on_step(self) -> bool:
|
||||||
|
state = self.poller.poll()
|
||||||
|
if state.stop:
|
||||||
|
_request_stop("control_file")
|
||||||
|
return False
|
||||||
|
if state.paused:
|
||||||
|
stopped = self.poller.wait_while_paused(
|
||||||
|
lambda: self._event("paused"),
|
||||||
|
lambda: self._event("resumed"))
|
||||||
|
if stopped:
|
||||||
|
_request_stop("control_file")
|
||||||
|
return False
|
||||||
|
return not stop_state["requested"]
|
||||||
|
return True
|
||||||
|
|
||||||
|
control_path = getattr(args, "control_file", None)
|
||||||
|
control_cb = _ControlChannel(Path(control_path)) if control_path else None
|
||||||
|
|
||||||
class _GracefulStop(BaseCallback):
|
class _GracefulStop(BaseCallback):
|
||||||
"""收到停止信号后在最近一次 step 边界结束 rollout,让模型能落盘。"""
|
"""收到停止请求后在最近一次 step 边界结束 rollout,让模型能落盘。"""
|
||||||
|
|
||||||
def _on_step(self) -> bool:
|
def _on_step(self) -> bool:
|
||||||
if stop_state["requested"]:
|
if stop_state["requested"]:
|
||||||
@@ -435,15 +675,22 @@ def run_training(args) -> int:
|
|||||||
return True
|
return True
|
||||||
|
|
||||||
class _WallClockStop(BaseCallback):
|
class _WallClockStop(BaseCallback):
|
||||||
def __init__(self, max_wall_s: float):
|
"""墙钟硬上限;暂停时间不计入,避免"挂起一小时回来就判超时"。"""
|
||||||
|
|
||||||
|
def __init__(self, max_wall_s: float, paused_clock=None):
|
||||||
super().__init__()
|
super().__init__()
|
||||||
self.max_wall_s = max_wall_s
|
self.max_wall_s = max_wall_s
|
||||||
self.start = time.monotonic()
|
self.start = time.monotonic()
|
||||||
|
self.paused_clock = paused_clock
|
||||||
|
|
||||||
|
def paused_seconds(self) -> float:
|
||||||
|
return float(self.paused_clock()) if self.paused_clock else 0.0
|
||||||
|
|
||||||
def _on_step(self) -> bool:
|
def _on_step(self) -> bool:
|
||||||
if time.monotonic() - self.start >= self.max_wall_s:
|
elapsed = time.monotonic() - self.start - self.paused_seconds()
|
||||||
|
if elapsed >= self.max_wall_s:
|
||||||
print(json.dumps({"event": "wall_clock_stop",
|
print(json.dumps({"event": "wall_clock_stop",
|
||||||
"elapsed_s": time.monotonic() - self.start}))
|
"elapsed_s": elapsed}))
|
||||||
return False
|
return False
|
||||||
return True
|
return True
|
||||||
|
|
||||||
@@ -473,44 +720,7 @@ def run_training(args) -> int:
|
|||||||
return True
|
return True
|
||||||
self.last_emit = current
|
self.last_emit = current
|
||||||
self.emitted += 1
|
self.emitted += 1
|
||||||
values = getattr(self.model.logger, "name_to_value", {}) or {}
|
payload = _progress_payload(self.model, self.start)
|
||||||
|
|
||||||
def _num(key, default=None):
|
|
||||||
value = values.get(key)
|
|
||||||
if value is None:
|
|
||||||
return default
|
|
||||||
try:
|
|
||||||
value = float(value)
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
return default
|
|
||||||
if value != value or value in (float("inf"), float("-inf")):
|
|
||||||
return default
|
|
||||||
return value
|
|
||||||
|
|
||||||
ep_rewards = []
|
|
||||||
ep_lengths = []
|
|
||||||
for item in getattr(self.model, "ep_info_buffer", []) or []:
|
|
||||||
try:
|
|
||||||
ep_rewards.append(float(item.get("r")))
|
|
||||||
ep_lengths.append(float(item.get("l")))
|
|
||||||
except (TypeError, ValueError, AttributeError):
|
|
||||||
continue
|
|
||||||
payload = {
|
|
||||||
"event": "progress",
|
|
||||||
"timesteps": current,
|
|
||||||
"start_timesteps": self.start,
|
|
||||||
"n_updates": int(getattr(self.model, "_n_updates", 0) or 0),
|
|
||||||
"ep_rew_mean": (sum(ep_rewards) / len(ep_rewards)
|
|
||||||
if ep_rewards else None),
|
|
||||||
"ep_len_mean": (sum(ep_lengths) / len(ep_lengths)
|
|
||||||
if ep_lengths else None),
|
|
||||||
"value_loss": _num("train/value_loss"),
|
|
||||||
"policy_gradient_loss": _num("train/policy_gradient_loss"),
|
|
||||||
"approx_kl": _num("train/approx_kl"),
|
|
||||||
"entropy_loss": _num("train/entropy_loss"),
|
|
||||||
"explained_variance": _num("train/explained_variance"),
|
|
||||||
"fps": _num("time/fps"),
|
|
||||||
}
|
|
||||||
print(json.dumps(payload, ensure_ascii=False, allow_nan=False),
|
print(json.dumps(payload, ensure_ascii=False, allow_nan=False),
|
||||||
flush=True)
|
flush=True)
|
||||||
return True
|
return True
|
||||||
@@ -521,38 +731,79 @@ def run_training(args) -> int:
|
|||||||
checkpoint_stem = f"{model_path.stem}_ckpt"
|
checkpoint_stem = f"{model_path.stem}_ckpt"
|
||||||
|
|
||||||
class _CheckpointJSON(BaseCallback):
|
class _CheckpointJSON(BaseCallback):
|
||||||
"""每 N 个环境步落一个可续训 checkpoint(zip + 同名元数据)。
|
"""可续训落盘:滚动恢复点 + 周期 checkpoint + resume.json 指针。
|
||||||
|
|
||||||
只保留最近 ``keep`` 个;checkpoint 与最终模型共用同名 json 契约,
|
- **滚动恢复点** ``<stem>_latest.zip``:每 ``recovery_every`` 个环境步
|
||||||
因此 ``--resume <checkpoint.zip>`` 直接可用。
|
原子覆盖写一次,训练开始前也先写一份。它保证"急停"(SIGKILL /
|
||||||
|
``taskkill /F``,没有任何 Python 清理机会)之后**最多只丢
|
||||||
|
``recovery_every`` 个环境步**,而且文件始终完整可加载。
|
||||||
|
- **周期 checkpoint** ``<stem>_<步数>.zip``:每 ``every_steps`` 步留一份,
|
||||||
|
只保留最近 ``keep`` 个,用于返祖对比与多档恢复。
|
||||||
|
- 每次落盘都刷新 ``<run_dir>/resume.json``:无论怎么停,都有一份权威的
|
||||||
|
"从哪继续"指针(含增量语义与生效过的 PPO 超参)。
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def __init__(self, every_steps: int, keep: int):
|
def __init__(self, every_steps: int, keep: int, recovery_every: int):
|
||||||
super().__init__()
|
super().__init__()
|
||||||
self.every_steps = max(1, int(every_steps))
|
self.every_steps = max(0, int(every_steps))
|
||||||
self.keep = max(1, int(keep))
|
self.keep = max(1, int(keep))
|
||||||
|
self.recovery_every = max(0, int(recovery_every))
|
||||||
self.last_saved = int(model.num_timesteps)
|
self.last_saved = int(model.num_timesteps)
|
||||||
|
self.last_recovery = int(model.num_timesteps)
|
||||||
self.saved: list[str] = []
|
self.saved: list[str] = []
|
||||||
|
self.latest_path = checkpoints_dir / f"{checkpoint_stem}_latest.zip"
|
||||||
|
|
||||||
|
def write_recovery(self, reason: str) -> None:
|
||||||
|
"""立刻写一份滚动恢复点;训练开始前调用即"起步基线"。
|
||||||
|
|
||||||
|
用闭包里的 ``model`` 而不是 ``self.model``:``BaseCallback.model``
|
||||||
|
要等 ``learn()`` 时才被赋值,而起步基线恰恰在 ``learn()`` 之前写。
|
||||||
|
"""
|
||||||
|
current = int(model.num_timesteps)
|
||||||
|
self.last_recovery = current
|
||||||
|
self._write_recovery(current, reason)
|
||||||
|
|
||||||
def _on_step(self) -> bool:
|
def _on_step(self) -> bool:
|
||||||
current = int(self.model.num_timesteps)
|
current = int(model.num_timesteps)
|
||||||
if current - self.last_saved < self.every_steps:
|
if (self.recovery_every > 0
|
||||||
return True
|
and current - self.last_recovery >= self.recovery_every):
|
||||||
self.last_saved = current
|
self.last_recovery = current
|
||||||
self._write(current)
|
self._write_recovery(current, "interval")
|
||||||
|
if (self.every_steps > 0
|
||||||
|
and current - self.last_saved >= self.every_steps):
|
||||||
|
self.last_saved = current
|
||||||
|
self._write_interval(current)
|
||||||
return True
|
return True
|
||||||
|
|
||||||
def _write(self, current: int) -> None:
|
def _persist(self, target: Path, current: int, reason: str,
|
||||||
checkpoints_dir.mkdir(parents=True, exist_ok=True)
|
extra: dict[str, Any]) -> None:
|
||||||
target = checkpoints_dir / f"{checkpoint_stem}_{current:09d}.zip"
|
"""原子落盘 zip + 元数据,并刷新 resume.json 指针。"""
|
||||||
self.model.save(str(target))
|
_save_model_atomic(model, target)
|
||||||
_write_metadata(
|
_write_metadata(
|
||||||
model_path=target, args=args,
|
model_path=target, args=args,
|
||||||
wall_s=time.monotonic() - started, n_envs=args.envs,
|
wall_s=time.monotonic() - started, n_envs=args.envs,
|
||||||
actual_timesteps=current, monitor_paths=monitor_paths,
|
actual_timesteps=current, monitor_paths=monitor_paths,
|
||||||
model=self.model,
|
model=model,
|
||||||
extra={"checkpoint": True, "checkpoint_timesteps": current,
|
extra={"checkpoint_timesteps": current,
|
||||||
"checkpoint_of": str(model_path)})
|
"checkpoint_reason": reason,
|
||||||
|
"checkpoint_of": str(model_path), **extra})
|
||||||
|
_write_resume_hint(run_dir=run_dir, model_path=target,
|
||||||
|
timesteps=current, reason=reason,
|
||||||
|
model=model)
|
||||||
|
|
||||||
|
def _write_recovery(self, current: int, reason: str) -> None:
|
||||||
|
checkpoints_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
self._persist(self.latest_path, current, reason,
|
||||||
|
{"recovery": True})
|
||||||
|
print(json.dumps({"event": "recovery_checkpoint",
|
||||||
|
"reason": reason, "timesteps": current,
|
||||||
|
"model": str(self.latest_path)},
|
||||||
|
ensure_ascii=False), flush=True)
|
||||||
|
|
||||||
|
def _write_interval(self, current: int) -> None:
|
||||||
|
checkpoints_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
target = checkpoints_dir / f"{checkpoint_stem}_{current:09d}.zip"
|
||||||
|
self._persist(target, current, "interval", {"checkpoint": True})
|
||||||
removed = _prune_checkpoints(checkpoints_dir, checkpoint_stem,
|
removed = _prune_checkpoints(checkpoints_dir, checkpoint_stem,
|
||||||
self.keep)
|
self.keep)
|
||||||
self.saved.append(str(target))
|
self.saved.append(str(target))
|
||||||
@@ -562,15 +813,24 @@ def run_training(args) -> int:
|
|||||||
|
|
||||||
progress_every = (args.progress_every if args.progress_every > 0
|
progress_every = (args.progress_every if args.progress_every > 0
|
||||||
else args.rollout_steps * args.envs)
|
else args.rollout_steps * args.envs)
|
||||||
checkpoint_cb = None
|
checkpoint_cb = _CheckpointJSON(args.checkpoint_every, args.checkpoint_keep,
|
||||||
if args.checkpoint_every > 0:
|
args.recovery_every)
|
||||||
checkpoint_cb = _CheckpointJSON(args.checkpoint_every,
|
if args.recovery_every > 0:
|
||||||
args.checkpoint_keep)
|
# 起步基线:训练还没跑就急停,也至少有一份可加载、可续训的产物
|
||||||
callbacks = [_GracefulStop(), _WallClockStop(args.max_wall_s)]
|
checkpoint_cb.write_recovery("baseline")
|
||||||
if checkpoint_cb is not None:
|
# 顺序有语义:控制通道最先(暂停/停止要在本步立刻生效),进度次之,
|
||||||
callbacks.append(checkpoint_cb)
|
# 墙钟上限要能读到控制通道累计的暂停秒数,checkpoint 放最后。
|
||||||
|
callbacks: list[Any] = []
|
||||||
|
if control_cb is not None:
|
||||||
|
callbacks.append(control_cb)
|
||||||
if args.progress_json:
|
if args.progress_json:
|
||||||
callbacks.insert(0, _ProgressJSON(progress_every))
|
callbacks.append(_ProgressJSON(progress_every))
|
||||||
|
callbacks.append(_GracefulStop())
|
||||||
|
callbacks.append(_WallClockStop(
|
||||||
|
args.max_wall_s,
|
||||||
|
paused_clock=(control_cb.paused_seconds if control_cb is not None
|
||||||
|
else None)))
|
||||||
|
callbacks.append(checkpoint_cb)
|
||||||
|
|
||||||
actual_timesteps = int(model.num_timesteps)
|
actual_timesteps = int(model.num_timesteps)
|
||||||
try:
|
try:
|
||||||
@@ -587,7 +847,7 @@ def run_training(args) -> int:
|
|||||||
}
|
}
|
||||||
emergency = run_dir / f"{model_path.stem}_emergency.zip"
|
emergency = run_dir / f"{model_path.stem}_emergency.zip"
|
||||||
try:
|
try:
|
||||||
model.save(str(emergency))
|
_save_model_atomic(model, emergency)
|
||||||
_write_metadata(model_path=emergency, args=args,
|
_write_metadata(model_path=emergency, args=args,
|
||||||
wall_s=time.monotonic() - started,
|
wall_s=time.monotonic() - started,
|
||||||
n_envs=args.envs,
|
n_envs=args.envs,
|
||||||
@@ -595,27 +855,34 @@ def run_training(args) -> int:
|
|||||||
monitor_paths=monitor_paths, model=model,
|
monitor_paths=monitor_paths, model=model,
|
||||||
extra={"emergency": True,
|
extra={"emergency": True,
|
||||||
"error": failure["error"]})
|
"error": failure["error"]})
|
||||||
|
_write_resume_hint(run_dir=run_dir, model_path=emergency,
|
||||||
|
timesteps=int(model.num_timesteps),
|
||||||
|
reason="emergency", model=model)
|
||||||
failure["emergency_model"] = str(emergency)
|
failure["emergency_model"] = str(emergency)
|
||||||
except Exception as save_exc: # 落盘失败不能再掩盖原始异常
|
except Exception as save_exc: # 落盘失败不能再掩盖原始异常
|
||||||
failure["emergency_save_error"] = (
|
failure["emergency_save_error"] = (
|
||||||
f"{type(save_exc).__name__}: {save_exc}")
|
f"{type(save_exc).__name__}: {save_exc}")
|
||||||
(run_dir / "training_failure.json").write_text(
|
_write_json_atomic(run_dir / "training_failure.json", failure)
|
||||||
json.dumps(failure, ensure_ascii=False, indent=2), encoding="utf-8")
|
|
||||||
print(json.dumps(failure, ensure_ascii=False), file=sys.stderr,
|
print(json.dumps(failure, ensure_ascii=False), file=sys.stderr,
|
||||||
flush=True)
|
flush=True)
|
||||||
raise
|
raise
|
||||||
wall_s = time.monotonic() - started
|
wall_s = time.monotonic() - started
|
||||||
actual_timesteps = int(model.num_timesteps)
|
actual_timesteps = int(model.num_timesteps)
|
||||||
stopped_by_signal = bool(stop_state["requested"])
|
stopped_by_signal = bool(stop_state["requested"])
|
||||||
model.save(str(model_path))
|
_save_model_atomic(model, model_path)
|
||||||
metadata = _write_metadata(
|
metadata = _write_metadata(
|
||||||
model_path=model_path, args=args, wall_s=wall_s, n_envs=args.envs,
|
model_path=model_path, args=args, wall_s=wall_s, n_envs=args.envs,
|
||||||
actual_timesteps=actual_timesteps, monitor_paths=monitor_paths,
|
actual_timesteps=actual_timesteps, monitor_paths=monitor_paths,
|
||||||
model=model,
|
model=model,
|
||||||
extra={"stopped_by_signal": stopped_by_signal,
|
extra={"stopped_by_signal": stopped_by_signal,
|
||||||
"stop_signal": stop_state["signal"],
|
"stop_signal": stop_state["signal"],
|
||||||
"checkpoint_files": (checkpoint_cb.saved
|
"checkpoint_files": checkpoint_cb.saved,
|
||||||
if checkpoint_cb is not None else [])})
|
"recovery_model": str(checkpoint_cb.latest_path)})
|
||||||
|
# 最终模型也刷新一次 resume.json:优雅收尾后"从哪继续"指向最终模型
|
||||||
|
_write_resume_hint(run_dir=run_dir, model_path=model_path,
|
||||||
|
timesteps=actual_timesteps,
|
||||||
|
reason=("stopped" if stopped_by_signal else "finished"),
|
||||||
|
model=model)
|
||||||
metadata_path = model_path.with_suffix(".json")
|
metadata_path = model_path.with_suffix(".json")
|
||||||
vec_env.close()
|
vec_env.close()
|
||||||
# 收尾阶段恢复默认处理器:再次 Ctrl+C / 停止可立即强杀,不再等待评估。
|
# 收尾阶段恢复默认处理器:再次 Ctrl+C / 停止可立即强杀,不再等待评估。
|
||||||
@@ -637,8 +904,7 @@ def run_training(args) -> int:
|
|||||||
if args.eval_manifest:
|
if args.eval_manifest:
|
||||||
try:
|
try:
|
||||||
manifest_eval = _evaluate_manifest(
|
manifest_eval = _evaluate_manifest(
|
||||||
model_path, args.eval_manifest, limit=args.eval_manifest_limit,
|
model_path, args.eval_manifest, limit=args.eval_manifest_limit)
|
||||||
problem=args.problem)
|
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
manifest_eval = {"error": f"{type(exc).__name__}: {exc}"}
|
manifest_eval = {"error": f"{type(exc).__name__}: {exc}"}
|
||||||
metadata["evaluation"] = evaluation
|
metadata["evaluation"] = evaluation
|
||||||
@@ -685,10 +951,17 @@ def main(argv=None) -> int:
|
|||||||
help="time=P0.4 纯时间口径;prompt=原始 §5.2 塑形对照")
|
help="time=P0.4 纯时间口径;prompt=原始 §5.2 塑形对照")
|
||||||
parser.add_argument("--progress-json", action="store_true",
|
parser.add_argument("--progress-json", action="store_true",
|
||||||
help="向训练前端输出单行 JSON 进度(UI 自动启用)")
|
help="向训练前端输出单行 JSON 进度(UI 自动启用)")
|
||||||
|
parser.add_argument("--control-file", default=None,
|
||||||
|
help="训练前端控制文件(JSON:paused/stop);"
|
||||||
|
"跨平台暂停与优雅停止的主通道")
|
||||||
parser.add_argument("--progress-every", type=int, default=0,
|
parser.add_argument("--progress-every", type=int, default=0,
|
||||||
help="JSON 进度间隔环境步数;0=每个 rollout")
|
help="JSON 进度间隔环境步数;0=每个 rollout")
|
||||||
parser.add_argument("--checkpoint-every", type=int, default=20000,
|
parser.add_argument("--checkpoint-every", type=int, default=20000,
|
||||||
help="每 N 个环境步落一个 checkpoint;0=关闭(默认 20000)")
|
help="每 N 个环境步留一个历史 checkpoint;0=关闭(默认 20000)")
|
||||||
|
parser.add_argument("--recovery-every", type=int, default=2000,
|
||||||
|
help="每 N 个环境步原子覆盖写一份滚动恢复点 "
|
||||||
|
"<模型目录>/checkpoints/<stem>_latest.zip;"
|
||||||
|
"急停后最多丢 N 步。0=关闭(默认 2000)")
|
||||||
parser.add_argument("--checkpoint-keep", type=int, default=3,
|
parser.add_argument("--checkpoint-keep", type=int, default=3,
|
||||||
help="checkpoint 保留最近 K 个(默认 3)")
|
help="checkpoint 保留最近 K 个(默认 3)")
|
||||||
parser.add_argument("--checkpoint-dir", default=None,
|
parser.add_argument("--checkpoint-dir", default=None,
|
||||||
@@ -703,6 +976,8 @@ def main(argv=None) -> int:
|
|||||||
parser.error("eval-episodes 不能为负")
|
parser.error("eval-episodes 不能为负")
|
||||||
if args.checkpoint_every < 0:
|
if args.checkpoint_every < 0:
|
||||||
parser.error("checkpoint-every 不能为负")
|
parser.error("checkpoint-every 不能为负")
|
||||||
|
if args.recovery_every < 0:
|
||||||
|
parser.error("recovery-every 不能为负")
|
||||||
if args.checkpoint_keep < 1:
|
if args.checkpoint_keep < 1:
|
||||||
parser.error("checkpoint-keep 至少为 1")
|
parser.error("checkpoint-keep 至少为 1")
|
||||||
return run_training(args)
|
return run_training(args)
|
||||||
|
|||||||
@@ -82,7 +82,6 @@ def bc_validation_mask(episode_ids, *, seed: int):
|
|||||||
|
|
||||||
def train_bc(x, y, episode_ids, *, epochs: int, batch_size: int, lr: float,
|
def train_bc(x, y, episode_ids, *, epochs: int, batch_size: int, lr: float,
|
||||||
device: str, seed: int):
|
device: str, seed: int):
|
||||||
import numpy as np
|
|
||||||
import torch
|
import torch
|
||||||
BCMlp = _bc_module(LOW_OBS_DIM, LOW_NUM_ACTIONS, HIDDEN)
|
BCMlp = _bc_module(LOW_OBS_DIM, LOW_NUM_ACTIONS, HIDDEN)
|
||||||
model = BCMlp().to(device)
|
model = BCMlp().to(device)
|
||||||
|
|||||||
+2
-1
@@ -130,7 +130,8 @@ def main(argv=None) -> int:
|
|||||||
for rank in range(max(1, args.envs))])
|
for rank in range(max(1, args.envs))])
|
||||||
if args.init:
|
if args.init:
|
||||||
metadata = json.loads(Path(args.init).with_suffix('.json').read_text(encoding='utf-8'))
|
metadata = json.loads(Path(args.init).with_suffix('.json').read_text(encoding='utf-8'))
|
||||||
if metadata.get('algorithm', 'ppo') != args.algorithm or metadata.get('feature_version') != 2:
|
if (metadata.get('algorithm', 'ppo') != args.algorithm
|
||||||
|
or metadata.get('feature_version') != 2):
|
||||||
parser.error('初始化模型的算法/特征版本与当前训练不符,请重训或提供v2初始化模型')
|
parser.error('初始化模型的算法/特征版本与当前训练不符,请重训或提供v2初始化模型')
|
||||||
model = model_cls.load(args.init, env=vec_env, device=device)
|
model = model_cls.load(args.init, env=vec_env, device=device)
|
||||||
else:
|
else:
|
||||||
|
|||||||
@@ -44,7 +44,7 @@ def evaluate(bayes_params: dict) -> tuple[float, dict]:
|
|||||||
config = replace(BASE, bayes=replace(BASE.bayes, **bayes_params))
|
config = replace(BASE, bayes=replace(BASE.bayes, **bayes_params))
|
||||||
times = []
|
times = []
|
||||||
failures = 0
|
failures = 0
|
||||||
for name, sources, error_seed in SCENES:
|
for _name, sources, error_seed in SCENES:
|
||||||
sim = LocalSimulator(sources, robot_id="TUNE", config=config,
|
sim = LocalSimulator(sources, robot_id="TUNE", config=config,
|
||||||
error_seed=error_seed)
|
error_seed=error_seed)
|
||||||
client = SimulatorClient("TUNE", config=config,
|
client = SimulatorClient("TUNE", config=config,
|
||||||
|
|||||||
@@ -33,7 +33,10 @@ class StrictPlayback(PlaybackTransport):
|
|||||||
raise AssertionError("Replay attempted an extra action")
|
raise AssertionError("Replay attempted an extra action")
|
||||||
expected = self.events[self.index]
|
expected = self.events[self.index]
|
||||||
actual_request = json.loads(body)
|
actual_request = json.loads(body)
|
||||||
without_id = lambda value: {k: v for k, v in value.items() if k != "request_id"}
|
|
||||||
|
def without_id(value):
|
||||||
|
return {k: v for k, v in value.items() if k != "request_id"}
|
||||||
|
|
||||||
if (urlsplit(url).path != expected["path"] or
|
if (urlsplit(url).path != expected["path"] or
|
||||||
without_id(actual_request) != without_id(expected["request"])):
|
without_id(actual_request) != without_id(expected["request"])):
|
||||||
raise AssertionError(f"Replay action differs at index {self.index}")
|
raise AssertionError(f"Replay action differs at index {self.index}")
|
||||||
@@ -66,6 +69,47 @@ def save_json(path, value):
|
|||||||
allow_nan=False) + "\n", encoding="utf-8")
|
allow_nan=False) + "\n", encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def timing_rows(events):
|
||||||
|
"""按官方附件 2 第 4.1 节逐动作累计虚拟时间(微秒),返回明细与汇总。"""
|
||||||
|
position, channel, virtual_us = (0.0, 0.0), 1, 0
|
||||||
|
distance_total, switch_total, move_total_us = 0.0, 0, 0
|
||||||
|
rows = []
|
||||||
|
previous_recorded = 0.0
|
||||||
|
max_local_gap = 0.0
|
||||||
|
for i, event in enumerate(events):
|
||||||
|
request, response = event["request"], event["response"]
|
||||||
|
path = event["path"]
|
||||||
|
distance, switches, action_s = 0.0, 0, 0
|
||||||
|
if path in ("/measure", "/clear"):
|
||||||
|
target = (request["position"]["x"], request["position"]["y"])
|
||||||
|
distance = math.dist(position, target)
|
||||||
|
position = target
|
||||||
|
if path == "/measure":
|
||||||
|
switches = int(channel != request["channel"])
|
||||||
|
channel = request["channel"]
|
||||||
|
action_s = 5
|
||||||
|
else:
|
||||||
|
action_s = 5 if response["clear_result"] == "success" else 3
|
||||||
|
move_us = round(distance / 5 * 1_000_000)
|
||||||
|
virtual_us += move_us + (switches + action_s) * 1_000_000
|
||||||
|
distance_total += distance
|
||||||
|
switch_total += switches
|
||||||
|
move_total_us += move_us
|
||||||
|
recorded = response["virtual_time_s"]
|
||||||
|
gap_us = round(recorded * 1_000_000) - virtual_us
|
||||||
|
max_local_gap = max(max_local_gap,
|
||||||
|
abs(recorded - previous_recorded - distance/5 - switches - action_s))
|
||||||
|
previous_recorded = recorded
|
||||||
|
rows.append({"index": i, "path": path, "request_id": request["request_id"],
|
||||||
|
"channel": request.get("channel", ""), "x": position[0], "y": position[1],
|
||||||
|
"result": response.get("measure_result", response.get("clear_result", "")),
|
||||||
|
"svd_deg": response.get("svd_deg", ""), "distance_m": distance,
|
||||||
|
"move_us": move_us, "switch_s": switches, "action_s": action_s,
|
||||||
|
"recomputed_us": virtual_us, "recorded_s": recorded, "gap_us": gap_us})
|
||||||
|
return rows, virtual_us, distance_total, move_total_us, switch_total, previous_recorded, \
|
||||||
|
max_local_gap
|
||||||
|
|
||||||
|
|
||||||
def audit(folder: Path) -> dict:
|
def audit(folder: Path) -> dict:
|
||||||
records = [json.loads(line) for line in
|
records = [json.loads(line) for line in
|
||||||
(folder / "session.jsonl").read_text(encoding="utf-8").splitlines()]
|
(folder / "session.jsonl").read_text(encoding="utf-8").splitlines()]
|
||||||
@@ -130,40 +174,8 @@ def audit(folder: Path) -> dict:
|
|||||||
|
|
||||||
# Official attachment 2 section 4.1: accumulate virtual time in microseconds.
|
# Official attachment 2 section 4.1: accumulate virtual time in microseconds.
|
||||||
# Round each move to the nearest microsecond, then add integer action costs.
|
# Round each move to the nearest microsecond, then add integer action costs.
|
||||||
position, channel, virtual_us = (0.0, 0.0), 1, 0
|
rows, virtual_us, distance_total, move_total_us, switch_total, previous_recorded, \
|
||||||
distance_total, switch_total, move_total_us = 0.0, 0, 0
|
max_local_gap = timing_rows(events)
|
||||||
rows = []
|
|
||||||
previous_recorded = 0.0
|
|
||||||
max_local_gap = 0.0
|
|
||||||
for i, event in enumerate(events):
|
|
||||||
request, response, path = event["request"], event["response"], event["path"]
|
|
||||||
distance, switches, action_s = 0.0, 0, 0
|
|
||||||
if path in ("/measure", "/clear"):
|
|
||||||
target = (request["position"]["x"], request["position"]["y"])
|
|
||||||
distance = math.dist(position, target)
|
|
||||||
position = target
|
|
||||||
if path == "/measure":
|
|
||||||
switches = int(channel != request["channel"])
|
|
||||||
channel = request["channel"]
|
|
||||||
action_s = 5
|
|
||||||
else:
|
|
||||||
action_s = 5 if response["clear_result"] == "success" else 3
|
|
||||||
move_us = round(distance / 5 * 1_000_000)
|
|
||||||
virtual_us += move_us + (switches + action_s) * 1_000_000
|
|
||||||
distance_total += distance
|
|
||||||
switch_total += switches
|
|
||||||
move_total_us += move_us
|
|
||||||
recorded = response["virtual_time_s"]
|
|
||||||
gap_us = round(recorded * 1_000_000) - virtual_us
|
|
||||||
max_local_gap = max(max_local_gap,
|
|
||||||
abs(recorded - previous_recorded - distance/5 - switches - action_s))
|
|
||||||
previous_recorded = recorded
|
|
||||||
rows.append({"index": i, "path": path, "request_id": request["request_id"],
|
|
||||||
"channel": request.get("channel", ""), "x": position[0], "y": position[1],
|
|
||||||
"result": response.get("measure_result", response.get("clear_result", "")),
|
|
||||||
"svd_deg": response.get("svd_deg", ""), "distance_m": distance,
|
|
||||||
"move_us": move_us, "switch_s": switches, "action_s": action_s,
|
|
||||||
"recomputed_us": virtual_us, "recorded_s": recorded, "gap_us": gap_us})
|
|
||||||
checks["all_timestamps_match_microsecond_recomputation"] = all(r["gap_us"] == 0 for r in rows)
|
checks["all_timestamps_match_microsecond_recomputation"] = all(r["gap_us"] == 0 for r in rows)
|
||||||
real_s = (events[-1]["response"]["real_timestamp_ms"] -
|
real_s = (events[-1]["response"]["real_timestamp_ms"] -
|
||||||
events[0]["response"]["real_timestamp_ms"]) / 1000
|
events[0]["response"]["real_timestamp_ms"]) / 1000
|
||||||
@@ -206,7 +218,8 @@ def audit(folder: Path) -> dict:
|
|||||||
"clear_time_s": sum(r["action_s"] for r in rows if r["path"] == "/clear"),
|
"clear_time_s": sum(r["action_s"] for r in rows if r["path"] == "/clear"),
|
||||||
"timing_audit": {
|
"timing_audit": {
|
||||||
"original_unquantized_mismatch_count": len(raw_timing["mismatches"]),
|
"original_unquantized_mismatch_count": len(raw_timing["mismatches"]),
|
||||||
"original_unquantized_final_gap_s": previous_recorded - raw_timing["final_virtual_time_s"],
|
"original_unquantized_final_gap_s":
|
||||||
|
previous_recorded - raw_timing["final_virtual_time_s"],
|
||||||
"max_single_action_unquantized_gap_s": max_local_gap,
|
"max_single_action_unquantized_gap_s": max_local_gap,
|
||||||
"microsecond_mismatch_count": sum(r["gap_us"] != 0 for r in rows),
|
"microsecond_mismatch_count": sum(r["gap_us"] != 0 for r in rows),
|
||||||
"explanation": "原复算累计未量化浮点耗时;按动作移动耗时取最近整数微秒后,所有官方时刻完全一致。",
|
"explanation": "原复算累计未量化浮点耗时;按动作移动耗时取最近整数微秒后,所有官方时刻完全一致。",
|
||||||
|
|||||||
@@ -116,8 +116,8 @@ if __name__ == "__main__":
|
|||||||
parser = argparse.ArgumentParser(description=__doc__)
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
parser.add_argument("folder", type=Path)
|
parser.add_argument("folder", type=Path)
|
||||||
args = parser.parse_args()
|
args = parser.parse_args()
|
||||||
report = audit(args.folder)
|
result = audit(args.folder)
|
||||||
print(json.dumps({"passed": report["passed"], "case_code": report["case_code"],
|
print(json.dumps({"passed": result["passed"], "case_code": result["case_code"],
|
||||||
"problem": report["problem"], "checks": report["checks"]},
|
"problem": result["problem"], "checks": result["checks"]},
|
||||||
ensure_ascii=False, indent=2))
|
ensure_ascii=False, indent=2))
|
||||||
raise SystemExit(0 if report["passed"] else 1)
|
raise SystemExit(0 if result["passed"] else 1)
|
||||||
|
|||||||
@@ -126,10 +126,10 @@ class CandidateRegion:
|
|||||||
@classmethod
|
@classmethod
|
||||||
def initial(cls, station: Vec2, config: ProblemConfig) -> "CandidateRegion":
|
def initial(cls, station: Vec2, config: ProblemConfig) -> "CandidateRegion":
|
||||||
"""首个 direction 后的基础集:Ω ∩ B(station, r_max) 的保守外包。"""
|
"""首个 direction 后的基础集:Ω ∩ B(station, r_max) 的保守外包。"""
|
||||||
field, sides = config.field, config.search.polygon_sides
|
field_cfg, sides = config.field, config.search.polygon_sides
|
||||||
slack = field.omega_radius_m * (1 - math.cos(math.pi / sides))
|
slack = field_cfg.omega_radius_m * (1 - math.cos(math.pi / sides))
|
||||||
region = cls.disk((0.0, 0.0), field.omega_radius_m, sides, slack)
|
region = cls.disk((0.0, 0.0), field_cfg.omega_radius_m, sides, slack)
|
||||||
region.clip_disk(station, field.r_eff_max_m, sides)
|
region.clip_disk(station, field_cfg.r_eff_max_m, sides)
|
||||||
return region
|
return region
|
||||||
|
|
||||||
# ------------------------------------------------------------ 收缩
|
# ------------------------------------------------------------ 收缩
|
||||||
@@ -198,8 +198,7 @@ class CandidateRegion:
|
|||||||
if len(self.polygon) == 2:
|
if len(self.polygon) == 2:
|
||||||
p, q = self.polygon
|
p, q = self.polygon
|
||||||
return _point_segment_distance(point, p, q) <= tol_m + self.slack_m
|
return _point_segment_distance(point, p, q) <= tol_m + self.slack_m
|
||||||
for index in range(len(self.polygon)):
|
for index, p in enumerate(self.polygon):
|
||||||
p = self.polygon[index]
|
|
||||||
q = self.polygon[(index + 1) % len(self.polygon)]
|
q = self.polygon[(index + 1) % len(self.polygon)]
|
||||||
if (q[0] - p[0]) * (point[1] - p[1]) - (q[1] - p[1]) * (point[0] - p[0]) \
|
if (q[0] - p[0]) * (point[1] - p[1]) - (q[1] - p[1]) * (point[0] - p[0]) \
|
||||||
< -tol_m - self.slack_m:
|
< -tol_m - self.slack_m:
|
||||||
|
|||||||
@@ -294,6 +294,13 @@ def _run(args) -> int:
|
|||||||
if judge is not None:
|
if judge is not None:
|
||||||
_settle_local_judge(judge)
|
_settle_local_judge(judge)
|
||||||
summary["robot_id"] = args.robot_id
|
summary["robot_id"] = args.robot_id
|
||||||
|
return _finalize_run(args, summary, strategy, truth=truth, judge=judge,
|
||||||
|
judge_session=judge_session, real_budget_s=real_budget_s)
|
||||||
|
|
||||||
|
|
||||||
|
def _finalize_run(args, summary, strategy, *, truth, judge, judge_session,
|
||||||
|
real_budget_s) -> int:
|
||||||
|
"""补齐 summary 元信息、核对真值并输出;返回进程退出码。"""
|
||||||
if args.problem == 4:
|
if args.problem == 4:
|
||||||
summary["strategy"] = "rl_q4" if strategy == "rl" else strategy
|
summary["strategy"] = "rl_q4" if strategy == "rl" else strategy
|
||||||
else:
|
else:
|
||||||
|
|||||||
@@ -32,7 +32,7 @@ class CoverageLedger:
|
|||||||
"""逐频道覆盖账本:(频道, 网格点) -> 是否已测。"""
|
"""逐频道覆盖账本:(频道, 网格点) -> 是否已测。"""
|
||||||
|
|
||||||
config: ProblemConfig = field(default_factory=ProblemConfig)
|
config: ProblemConfig = field(default_factory=ProblemConfig)
|
||||||
points: list[Vec2] = field(default_factory=lambda: coverage_points())
|
points: list[Vec2] = field(default_factory=coverage_points)
|
||||||
measured: set = field(default_factory=set) # {(channel, point_index)}
|
measured: set = field(default_factory=set) # {(channel, point_index)}
|
||||||
skipped: dict = field(default_factory=dict) # channel -> 原因
|
skipped: dict = field(default_factory=dict) # channel -> 原因
|
||||||
|
|
||||||
|
|||||||
@@ -23,8 +23,10 @@ def main(argv=None):
|
|||||||
measurements = [replace(m, half_angle_deg=m.half_angle_deg+args.rounding_margin_deg)
|
measurements = [replace(m, half_angle_deg=m.half_angle_deg+args.rounding_margin_deg)
|
||||||
for m in measurements]
|
for m in measurements]
|
||||||
result = solve(measurements)
|
result = solve(measurements)
|
||||||
result["error_policy"] = {"additional_half_angle_deg": args.rounding_margin_deg,
|
result["error_policy"] = {
|
||||||
"effective_half_angles_deg": [m.half_angle_deg for m in measurements]}
|
"additional_half_angle_deg": args.rounding_margin_deg,
|
||||||
|
"effective_half_angles_deg": [m.half_angle_deg for m in measurements],
|
||||||
|
}
|
||||||
except (ValueError, TypeError, KeyError, OSError) as exc:
|
except (ValueError, TypeError, KeyError, OSError) as exc:
|
||||||
parser.error(str(exc))
|
parser.error(str(exc))
|
||||||
text = json.dumps(result, ensure_ascii=False, indent=2, allow_nan=False)
|
text = json.dumps(result, ensure_ascii=False, indent=2, allow_nan=False)
|
||||||
|
|||||||
@@ -133,6 +133,13 @@ def _enclosing_circle_reference(points: list[Point], eps: float = EPS):
|
|||||||
return best_c, best_r
|
return best_c, best_r
|
||||||
|
|
||||||
|
|
||||||
|
def _circle_covers(circle, point: Point, tolerance: float) -> bool:
|
||||||
|
"""Whether the circle already covers point; None means no circle yet."""
|
||||||
|
if circle is None:
|
||||||
|
return False
|
||||||
|
return distance(circle[0], point) <= circle[1] + tolerance
|
||||||
|
|
||||||
|
|
||||||
def smallest_enclosing_circle(points: list[Point], eps: float = EPS):
|
def smallest_enclosing_circle(points: list[Point], eps: float = EPS):
|
||||||
"""Welzl's randomized incremental algorithm, using an explicit call stack.
|
"""Welzl's randomized incremental algorithm, using an explicit call stack.
|
||||||
|
|
||||||
@@ -169,8 +176,9 @@ def smallest_enclosing_circle(points: list[Point], eps: float = EPS):
|
|||||||
elif not returning:
|
elif not returning:
|
||||||
stack.append((n, boundary, True))
|
stack.append((n, boundary, True))
|
||||||
stack.append((n-1, boundary, False))
|
stack.append((n-1, boundary, False))
|
||||||
elif circle is None or distance(circle[0], local[n-1]) > circle[1] + tolerance:
|
elif not _circle_covers(circle, local[n-1], tolerance):
|
||||||
stack.append((n-1, boundary + (local[n-1],), False))
|
stack.append((n-1, boundary + (local[n-1],), False))
|
||||||
|
assert circle is not None
|
||||||
center = (origin[0]+circle[0][0]*scale, origin[1]+circle[0][1]*scale)
|
center = (origin[0]+circle[0][0]*scale, origin[1]+circle[0][1]*scale)
|
||||||
# Recompute after transforming back: returned circle contains all input points
|
# Recompute after transforming back: returned circle contains all input points
|
||||||
# to distance() precision, including any roundoff from the large translation.
|
# to distance() precision, including any roundoff from the large translation.
|
||||||
|
|||||||
@@ -29,7 +29,7 @@ class LocalSimTransport(Transport):
|
|||||||
def post(self, url, body, headers, timeout):
|
def post(self, url, body, headers, timeout):
|
||||||
return self.request("POST", url, body, headers, timeout)
|
return self.request("POST", url, body, headers, timeout)
|
||||||
|
|
||||||
def request(self, method, url, body, headers, timeout=None):
|
def request(self, method, url, body, headers, _timeout=None):
|
||||||
"""验证原始请求边界;内存回环不模拟 TCP、UI 或连接限流。"""
|
"""验证原始请求边界;内存回环不模拟 TCP、UI 或连接限流。"""
|
||||||
target = urlsplit(url)
|
target = urlsplit(url)
|
||||||
raw_target = target.path + (f"?{target.query}" if target.query else "")
|
raw_target = target.path + (f"?{target.query}" if target.query else "")
|
||||||
@@ -57,5 +57,6 @@ class ScriptedFaultTransport(Transport):
|
|||||||
self.inner.post(url, body, headers, timeout) # 服务器已受理
|
self.inner.post(url, body, headers, timeout) # 服务器已受理
|
||||||
raise TransportError("injected: read timeout after server applied")
|
raise TransportError("injected: read timeout after server applied")
|
||||||
if isinstance(fault, tuple) and fault and fault[0] == "status":
|
if isinstance(fault, tuple) and fault and fault[0] == "status":
|
||||||
return TransportResponse(int(fault[1]), b'{"accepted": false, "real_timestamp_ms": 0, "virtual_time_s": 0}')
|
body = b'{"accepted": false, "real_timestamp_ms": 0, "virtual_time_s": 0}'
|
||||||
|
return TransportResponse(int(fault[1]), body)
|
||||||
return self.inner.post(url, body, headers, timeout)
|
return self.inner.post(url, body, headers, timeout)
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ import math
|
|||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
from ..config import Vec2
|
from ..config import Vec2
|
||||||
from .constants import (LOW_MAX_STEPS, LOW_NUM_ACTIONS, LOW_OBS_DIM,
|
from .constants import (LOW_MAX_STEPS, LOW_NUM_ACTIONS,
|
||||||
LOW_REACH_M, LOW_STEP_M, POS_SCALE_M)
|
LOW_REACH_M, LOW_STEP_M, POS_SCALE_M)
|
||||||
|
|
||||||
__all__ = ["build_low_observation", "LowLevelExecutor", "action_bearing",
|
__all__ = ["build_low_observation", "LowLevelExecutor", "action_bearing",
|
||||||
|
|||||||
@@ -144,7 +144,6 @@ class HierarchicalEnv(gym.Env):
|
|||||||
problem=self.problem)
|
problem=self.problem)
|
||||||
|
|
||||||
def _training_spec(self):
|
def _training_spec(self):
|
||||||
from ..harness import referee_spec
|
|
||||||
from ..hrl.curriculum import training_spec
|
from ..hrl.curriculum import training_spec
|
||||||
return training_spec(self.train_config, self.stage,
|
return training_spec(self.train_config, self.stage,
|
||||||
max_real_duration_s=self.real_budget_s)
|
max_real_duration_s=self.real_budget_s)
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ from gymnasium import spaces
|
|||||||
|
|
||||||
from ..config import ProblemConfig
|
from ..config import ProblemConfig
|
||||||
from .constants import (LOW_MAX_STEPS, LOW_NUM_ACTIONS, LOW_OBS_DIM,
|
from .constants import (LOW_MAX_STEPS, LOW_NUM_ACTIONS, LOW_OBS_DIM,
|
||||||
LOW_REACH_M, LOW_STEP_M, POS_SCALE_M)
|
LOW_REACH_M, LOW_STEP_M)
|
||||||
from .executor import (action_bearing, build_low_observation,
|
from .executor import (action_bearing, build_low_observation,
|
||||||
navigation_transition)
|
navigation_transition)
|
||||||
|
|
||||||
|
|||||||
@@ -48,7 +48,7 @@ def _load_maskable(model_path: Path, device: str, *,
|
|||||||
if meta_path.exists() else {}
|
if meta_path.exists() else {}
|
||||||
# 高层模型必须携带完整语义元数据;缺字段不能默认兼容,防止旧槽位/
|
# 高层模型必须携带完整语义元数据;缺字段不能默认兼容,防止旧槽位/
|
||||||
# 旧奖励语义模型被静默加载。低层导航模型只校验基础特征版本。
|
# 旧奖励语义模型被静默加载。低层导航模型只校验基础特征版本。
|
||||||
strict = (act_dim == N_CANDIDATES)
|
strict = act_dim == N_CANDIDATES
|
||||||
if strict:
|
if strict:
|
||||||
required = ("feature_version", "action_layout_version",
|
required = ("feature_version", "action_layout_version",
|
||||||
"reward_version", "algorithm", "problem",
|
"reward_version", "algorithm", "problem",
|
||||||
|
|||||||
@@ -29,7 +29,7 @@ from typing import NamedTuple
|
|||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
from ..belief import ChannelStatus
|
from ..belief import ChannelStatus
|
||||||
from ..config import ProblemConfig, Vec2
|
from ..config import Vec2
|
||||||
from ..protocol.rules import CLEAR_SUCCESS, DIRECTION, NEAR, NO_SIGNAL
|
from ..protocol.rules import CLEAR_SUCCESS, DIRECTION, NEAR, NO_SIGNAL
|
||||||
from ..protocol.timing import move_time
|
from ..protocol.timing import move_time
|
||||||
from .constants import (CHANNEL_COUNT, CLEAR_OFFSET, COST_SCALE_S,
|
from .constants import (CHANNEL_COUNT, CLEAR_OFFSET, COST_SCALE_S,
|
||||||
@@ -84,6 +84,8 @@ class HRLControllerMixin:
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
problem: int
|
problem: int
|
||||||
|
# 由宿主 BayesLoop.__init__ 初始化;此处声明让 pylint 可见该诊断计数。
|
||||||
|
no_shrink_measures: int = 0
|
||||||
|
|
||||||
# ------------------------------------------------------------ 候选构建
|
# ------------------------------------------------------------ 候选构建
|
||||||
def build_candidates(self) -> list[Candidate]:
|
def build_candidates(self) -> list[Candidate]:
|
||||||
@@ -220,7 +222,7 @@ class HRLControllerMixin:
|
|||||||
def hrl_observation(self, candidates: list[Candidate]) -> np.ndarray:
|
def hrl_observation(self, candidates: list[Candidate]) -> np.ndarray:
|
||||||
"""构造高层观测(shape=(HIGH_OBS_DIM,),训练/推理逐位一致)。"""
|
"""构造高层观测(shape=(HIGH_OBS_DIM,),训练/推理逐位一致)。"""
|
||||||
config = self.config
|
config = self.config
|
||||||
field = config.field
|
field_cfg = config.field
|
||||||
position = self.client.state.position
|
position = self.client.state.position
|
||||||
belief = self.belief
|
belief = self.belief
|
||||||
|
|
||||||
@@ -232,24 +234,24 @@ class HRLControllerMixin:
|
|||||||
glob = [
|
glob = [
|
||||||
position[0] / POS_SCALE_M,
|
position[0] / POS_SCALE_M,
|
||||||
position[1] / POS_SCALE_M,
|
position[1] / POS_SCALE_M,
|
||||||
min(self.client.state.virtual_time_s, field.max_virtual_duration_s)
|
min(self.client.state.virtual_time_s, field_cfg.max_virtual_duration_s)
|
||||||
/ field.max_virtual_duration_s,
|
/ field_cfg.max_virtual_duration_s,
|
||||||
cleared / 16.0,
|
cleared / 16.0,
|
||||||
accounted / CHANNEL_COUNT,
|
accounted / CHANNEL_COUNT,
|
||||||
(self.client.state.channel - field.channel_min)
|
(self.client.state.channel - field_cfg.channel_min)
|
||||||
/ max(field.channel_max - field.channel_min, 1),
|
/ max(field_cfg.channel_max - field_cfg.channel_min, 1),
|
||||||
float(self.problem == 4),
|
float(self.problem == 4),
|
||||||
detected_count / 16.0,
|
detected_count / 16.0,
|
||||||
]
|
]
|
||||||
assert len(glob) == HIGH_GLOBAL_DIM
|
assert len(glob) == HIGH_GLOBAL_DIM
|
||||||
|
|
||||||
pending_station = np.zeros(CHANNEL_COUNT + 1, dtype=bool)
|
pending_station = np.zeros(CHANNEL_COUNT + 1, dtype=bool)
|
||||||
for index, channels in getattr(self, "_station_pending_map", {}).items():
|
for channels in getattr(self, "_station_pending_map", {}).values():
|
||||||
for channel in channels:
|
for channel in channels:
|
||||||
pending_station[channel] = True
|
pending_station[channel] = True
|
||||||
|
|
||||||
channel_features: list[float] = []
|
channel_features: list[float] = []
|
||||||
for channel in range(field.channel_min, field.channel_max + 1):
|
for channel in range(field_cfg.channel_min, field_cfg.channel_max + 1):
|
||||||
ch = belief.channels[channel]
|
ch = belief.channels[channel]
|
||||||
status6 = [0.0] * 6
|
status6 = [0.0] * 6
|
||||||
status6[self._hrl_channel_status_index(channel)] = 1.0
|
status6[self._hrl_channel_status_index(channel)] = 1.0
|
||||||
@@ -363,7 +365,7 @@ class HRLControllerMixin:
|
|||||||
outcome = StepOutcome(action=action, kind=cand.kind,
|
outcome = StepOutcome(action=action, kind=cand.kind,
|
||||||
channel=cand.channel,
|
channel=cand.channel,
|
||||||
position_reached=tuple(self.client.state.position))
|
position_reached=tuple(self.client.state.position))
|
||||||
outcome.delta_virtual_s = (self.client.state.virtual_time_s - old_virtual)
|
outcome.delta_virtual_s = self.client.state.virtual_time_s - old_virtual
|
||||||
new_detections: list[int] = []
|
new_detections: list[int] = []
|
||||||
info_gains: dict[int, float] = {}
|
info_gains: dict[int, float] = {}
|
||||||
redundant = False
|
redundant = False
|
||||||
|
|||||||
@@ -237,10 +237,11 @@ class ChannelParticles:
|
|||||||
base = i * self.width
|
base = i * self.width
|
||||||
for j in range(j0, min(j0 + _COARSE_FACTOR, self.width)):
|
for j in range(j0, min(j0 + _COARSE_FACTOR, self.width)):
|
||||||
index = base + j
|
index = base + j
|
||||||
if index in self.alive:
|
if index not in self.alive:
|
||||||
dist = math.dist(self.cell_center(index), point)
|
continue
|
||||||
if dist <= limit_m and (best is None or dist < best):
|
dist = math.dist(self.cell_center(index), point)
|
||||||
best = dist
|
if dist <= limit_m and (best is None or dist < best):
|
||||||
|
best = dist
|
||||||
return best
|
return best
|
||||||
|
|
||||||
def guaranteed_no_signal(self, station: Vec2) -> bool:
|
def guaranteed_no_signal(self, station: Vec2) -> bool:
|
||||||
|
|||||||
@@ -621,7 +621,8 @@ class DirectionalExtension(OmnidirectionalLoop):
|
|||||||
self.ledger.mark_measured(channel, index)
|
self.ledger.mark_measured(channel, index)
|
||||||
self.belief.update_measure(channel, point, outcome.result, outcome.svd_deg)
|
self.belief.update_measure(channel, point, outcome.result, outcome.svd_deg)
|
||||||
if outcome.result == NEAR:
|
if outcome.result == NEAR:
|
||||||
cert = clear_certificate(self.belief.candidate_set(channel), channel, self.config)
|
cert = clear_certificate(
|
||||||
|
self.belief.candidate_set(channel), channel, self.config)
|
||||||
if cert.certified:
|
if cert.certified:
|
||||||
self._attempt_clear(channel, point, cert)
|
self._attempt_clear(channel, point, cert)
|
||||||
if self._count_limit_reached():
|
if self._count_limit_reached():
|
||||||
@@ -629,7 +630,8 @@ class DirectionalExtension(OmnidirectionalLoop):
|
|||||||
queue = self.belief.channels_by_status(ChannelStatus.DETECTED)
|
queue = self.belief.channels_by_status(ChannelStatus.DETECTED)
|
||||||
while queue and self.stop_reason is None:
|
while queue and self.stop_reason is None:
|
||||||
channel = min(queue, key=lambda c: (
|
channel = min(queue, key=lambda c: (
|
||||||
math.dist(self.client.state.position, self.belief.candidate_set(c).mec()[0]), c))
|
math.dist(self.client.state.position,
|
||||||
|
self.belief.candidate_set(c).mec()[0]), c))
|
||||||
queue.remove(channel)
|
queue.remove(channel)
|
||||||
self._localize_and_clear(channel)
|
self._localize_and_clear(channel)
|
||||||
if self._count_limit_reached():
|
if self._count_limit_reached():
|
||||||
@@ -672,8 +674,7 @@ class DirectionalExtension(OmnidirectionalLoop):
|
|||||||
self.directional_observed.append(channel)
|
self.directional_observed.append(channel)
|
||||||
self.blocked_probes.setdefault(channel, []).append(point)
|
self.blocked_probes.setdefault(channel, []).append(point)
|
||||||
continue # 盘外无信号也可能是距离不足,不据此断言定向
|
continue # 盘外无信号也可能是距离不足,不据此断言定向
|
||||||
else:
|
self.blocked_probes.pop(channel, None)
|
||||||
self.blocked_probes.pop(channel, None)
|
|
||||||
steps += 1
|
steps += 1
|
||||||
if self.belief.channels[channel].status is ChannelStatus.CLEARED:
|
if self.belief.channels[channel].status is ChannelStatus.CLEARED:
|
||||||
return True
|
return True
|
||||||
|
|||||||
@@ -109,13 +109,15 @@ class BayesLoop(OmnidirectionalLoop):
|
|||||||
if cert is not None and cert.certified:
|
if cert is not None and cert.certified:
|
||||||
continue
|
continue
|
||||||
region = self.belief.candidate_set(channel)
|
region = self.belief.candidate_set(channel)
|
||||||
if max(math.dist(point, v) for v in region.polygon) > self.config.field.r_eff_min_m - 1e-3:
|
farthest = max(math.dist(point, v) for v in region.polygon)
|
||||||
|
if farthest > self.config.field.r_eff_min_m - 1e-3:
|
||||||
continue
|
continue
|
||||||
radius = region.mec()[1]
|
radius = region.mec()[1]
|
||||||
vertices = region.polygon
|
vertices = region.polygon
|
||||||
stride = max(1, len(vertices) // self.config.bayes.mc_representatives)
|
stride = max(1, len(vertices) // self.config.bayes.mc_representatives)
|
||||||
radii = []
|
radii = []
|
||||||
delta = self.config.field.bearing_error_half_deg + self.config.search.bearing_rounding_margin_deg
|
delta = (self.config.field.bearing_error_half_deg
|
||||||
|
+ self.config.search.bearing_rounding_margin_deg)
|
||||||
for target in vertices[::stride]:
|
for target in vertices[::stride]:
|
||||||
theta = math.degrees(math.atan2(target[1]-point[1], target[0]-point[0]))
|
theta = math.degrees(math.atan2(target[1]-point[1], target[0]-point[0]))
|
||||||
probe = CandidateRegion(list(vertices), region.slack_m)
|
probe = CandidateRegion(list(vertices), region.slack_m)
|
||||||
@@ -146,7 +148,7 @@ class BayesLoop(OmnidirectionalLoop):
|
|||||||
def _skip_station_channel(self, channel: int, index: int, point: Vec2) -> bool:
|
def _skip_station_channel(self, channel: int, index: int, point: Vec2) -> bool:
|
||||||
"""跳测规则(按 pf.version 缓存;幸存集单调收缩,判定单调)。"""
|
"""跳测规则(按 pf.version 缓存;幸存集单调收缩,判定单调)。"""
|
||||||
pf = self.particles[channel]
|
pf = self.particles[channel]
|
||||||
if not len(pf):
|
if not pf:
|
||||||
return True
|
return True
|
||||||
cached = self._skip_cache.get((channel, index))
|
cached = self._skip_cache.get((channel, index))
|
||||||
if cached is not None and cached[0] == pf.version:
|
if cached is not None and cached[0] == pf.version:
|
||||||
@@ -219,7 +221,9 @@ class BayesLoop(OmnidirectionalLoop):
|
|||||||
total += self.config.cost.switch_s * (channel != task.channel)
|
total += self.config.cost.switch_s * (channel != task.channel)
|
||||||
channel = task.channel
|
channel = task.channel
|
||||||
elif task.kind == "station" and task.station_channels:
|
elif task.kind == "station" and task.station_channels:
|
||||||
pending = sorted(task.station_channels, key=lambda c: (c != channel, c))
|
pending = sorted(
|
||||||
|
task.station_channels,
|
||||||
|
key=lambda c, channel=channel: (c != channel, c))
|
||||||
total += self.config.cost.measure_s * len(pending)
|
total += self.config.cost.measure_s * len(pending)
|
||||||
total += self.config.cost.switch_s * (len(pending) - (channel in pending))
|
total += self.config.cost.switch_s * (len(pending) - (channel in pending))
|
||||||
channel = pending[-1]
|
channel = pending[-1]
|
||||||
@@ -328,7 +332,7 @@ class BayesLoop(OmnidirectionalLoop):
|
|||||||
return # 已证毕:清除/缺席后的 no_signal 与距离无关,不进 PF
|
return # 已证毕:清除/缺席后的 no_signal 与距离无关,不进 PF
|
||||||
pf = self.particles[channel]
|
pf = self.particles[channel]
|
||||||
pf.apply_measure(position, outcome.result, outcome.svd_deg)
|
pf.apply_measure(position, outcome.result, outcome.svd_deg)
|
||||||
if not len(pf):
|
if not pf:
|
||||||
if belief.status is ChannelStatus.UNKNOWN:
|
if belief.status is ChannelStatus.UNKNOWN:
|
||||||
self.belief.mark_absent(channel)
|
self.belief.mark_absent(channel)
|
||||||
if channel not in self.pf_absent_channels:
|
if channel not in self.pf_absent_channels:
|
||||||
@@ -350,7 +354,8 @@ class BayesLoop(OmnidirectionalLoop):
|
|||||||
limit = (self.config.field.clear_radius_m - pf.margin_m
|
limit = (self.config.field.clear_radius_m - pf.margin_m
|
||||||
- self.config.search.clear_margin_m)
|
- self.config.search.clear_margin_m)
|
||||||
if radius <= limit:
|
if radius <= limit:
|
||||||
return self._near_clear_target(ClearCertificate(channel, center, radius, limit, True))
|
return self._near_clear_target(
|
||||||
|
ClearCertificate(channel, center, radius, limit, True))
|
||||||
return cert
|
return cert
|
||||||
|
|
||||||
def _near_clear_target(self, cert: ClearCertificate) -> ClearCertificate:
|
def _near_clear_target(self, cert: ClearCertificate) -> ClearCertificate:
|
||||||
@@ -368,7 +373,7 @@ class BayesLoop(OmnidirectionalLoop):
|
|||||||
"""定位测量点:期望 log 幸存占比 + λ·移动时间 最小的候选。"""
|
"""定位测量点:期望 log 幸存占比 + λ·移动时间 最小的候选。"""
|
||||||
bayes, search = self.config.bayes, self.config.search
|
bayes, search = self.config.bayes, self.config.search
|
||||||
pf = self.particles[channel]
|
pf = self.particles[channel]
|
||||||
if not len(pf):
|
if not pf:
|
||||||
return None
|
return None
|
||||||
region = self.belief.candidate_set(channel)
|
region = self.belief.candidate_set(channel)
|
||||||
center, radius = pf.mec()
|
center, radius = pf.mec()
|
||||||
|
|||||||
@@ -16,8 +16,6 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from ..belief import ChannelStatus
|
from ..belief import ChannelStatus
|
||||||
from ..config import ProblemConfig, Vec2
|
from ..config import ProblemConfig, Vec2
|
||||||
from ..coverage import CoverageLedger, coverage_points
|
from ..coverage import CoverageLedger, coverage_points
|
||||||
@@ -116,7 +114,7 @@ class HRLLoop(HRLControllerMixin, BayesLoop):
|
|||||||
if belief.status not in (ChannelStatus.CLEARED, ChannelStatus.ABSENT):
|
if belief.status not in (ChannelStatus.CLEARED, ChannelStatus.ABSENT):
|
||||||
if not (self.problem == 4 and outcome.result == NO_SIGNAL):
|
if not (self.problem == 4 and outcome.result == NO_SIGNAL):
|
||||||
pf.apply_measure(position, outcome.result, outcome.svd_deg)
|
pf.apply_measure(position, outcome.result, outcome.svd_deg)
|
||||||
if not len(pf):
|
if not pf:
|
||||||
if belief.status is ChannelStatus.UNKNOWN:
|
if belief.status is ChannelStatus.UNKNOWN:
|
||||||
self.belief.mark_absent(channel)
|
self.belief.mark_absent(channel)
|
||||||
if channel not in self.pf_absent_channels:
|
if channel not in self.pf_absent_channels:
|
||||||
|
|||||||
@@ -0,0 +1,175 @@
|
|||||||
|
"""跨平台子进程控制:启动隔离、协作式暂停/停止、必要时强杀整棵进程树。
|
||||||
|
|
||||||
|
训练前端要在 POSIX 与 Windows 上表现一致,而两边的进程模型差别正好卡在
|
||||||
|
「停止训练」这条路径上:
|
||||||
|
|
||||||
|
- ``os.killpg`` / ``os.getpgid`` **只存在于 POSIX**;Windows 上访问它们会抛
|
||||||
|
``AttributeError``。旧实现只捕获 ``ProcessLookupError`` / ``PermissionError``,
|
||||||
|
于是 ``POST /api/train/stop`` 在 Windows 上直接 500,训练进程照跑——表现就是
|
||||||
|
「点停止没反应」。
|
||||||
|
- ``subprocess.Popen(start_new_session=True)`` 在 Windows 上被**静默忽略**:
|
||||||
|
CPython 的 Windows ``_execute_child`` 形参名就是 ``unused_start_new_session``,
|
||||||
|
要隔离进程组只能传 ``CREATE_NEW_PROCESS_GROUP``。
|
||||||
|
- POSIX 的 SIGTERM 在 Windows 上经 ``os.kill`` 会退化成 ``TerminateProcess``:
|
||||||
|
硬杀,训练脚本里"收到信号后保存模型"的处理器根本不会执行。
|
||||||
|
|
||||||
|
所以「优雅暂停 / 优雅停止」不依赖信号量语义,而是走一个**控制文件**:前端原子
|
||||||
|
写 JSON,训练脚本每步轮询。两端都只是普通文件 IO,Windows 与 POSIX 行为一致;
|
||||||
|
信号与强杀退居兜底,只在训练进程不响应控制文件时使用。
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import signal
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"IS_WINDOWS", "ControlState", "spawn_kwargs", "write_control",
|
||||||
|
"read_control", "write_json_atomic", "terminate_process_tree",
|
||||||
|
]
|
||||||
|
|
||||||
|
# 运行期常量而非"导入期分支":调用方(含测试)可以 monkeypatch 它来模拟 Windows
|
||||||
|
IS_WINDOWS = os.name == "nt"
|
||||||
|
|
||||||
|
# 只有 Windows 定义 CREATE_NEW_PROCESS_GROUP;POSIX 上取 0 表示不使用 creationflags
|
||||||
|
_CREATE_NEW_PROCESS_GROUP = getattr(subprocess, "CREATE_NEW_PROCESS_GROUP", 0)
|
||||||
|
|
||||||
|
# 发出「停止」到强杀之间留给训练脚本收尾(保存模型 + 写元数据)的秒数
|
||||||
|
DEFAULT_GRACE_S = 15.0
|
||||||
|
|
||||||
|
|
||||||
|
def spawn_kwargs() -> dict[str, Any]:
|
||||||
|
"""返回 ``Popen`` 的平台相关参数:让训练进程独立于前端进程组。
|
||||||
|
|
||||||
|
POSIX 用 ``setsid``(``start_new_session``);Windows 上用
|
||||||
|
``CREATE_NEW_PROCESS_GROUP``,否则前端所在控制台的 Ctrl+C 会连带打到训练进程。
|
||||||
|
"""
|
||||||
|
if IS_WINDOWS:
|
||||||
|
if _CREATE_NEW_PROCESS_GROUP:
|
||||||
|
return {"creationflags": _CREATE_NEW_PROCESS_GROUP}
|
||||||
|
return {}
|
||||||
|
return {"start_new_session": True}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ControlState:
|
||||||
|
"""控制文件的一次快照;缺失或损坏时字段全为默认(即"无指令")。"""
|
||||||
|
|
||||||
|
paused: bool = False
|
||||||
|
stop: bool = False
|
||||||
|
seq: int = 0
|
||||||
|
ts: float = 0.0
|
||||||
|
|
||||||
|
|
||||||
|
def write_control(path: Path, *, paused: bool = False, stop: bool = False,
|
||||||
|
seq: int | None = None) -> ControlState:
|
||||||
|
"""原子写控制文件(同目录临时文件 + ``os.replace``)。
|
||||||
|
|
||||||
|
训练侧随时可能来读,必须避免读到写了一半的 JSON。``os.replace`` 在
|
||||||
|
Windows 与 POSIX 上都是原子的(同目录、同卷)。
|
||||||
|
"""
|
||||||
|
path = Path(path)
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
state = ControlState(
|
||||||
|
paused=bool(paused),
|
||||||
|
stop=bool(stop),
|
||||||
|
seq=int(seq) if seq is not None else time.time_ns(),
|
||||||
|
ts=time.time(),
|
||||||
|
)
|
||||||
|
payload = {"paused": state.paused, "stop": state.stop,
|
||||||
|
"seq": state.seq, "ts": state.ts}
|
||||||
|
tmp = path.with_name(f".{path.name}.{os.getpid()}.tmp")
|
||||||
|
tmp.write_text(json.dumps(payload, ensure_ascii=False), encoding="utf-8")
|
||||||
|
os.replace(tmp, path)
|
||||||
|
return state
|
||||||
|
|
||||||
|
|
||||||
|
def read_control(path: Path) -> ControlState:
|
||||||
|
"""读控制文件;缺失 / 半截 / 损坏一律当作"无指令",绝不让训练崩在控制面上。"""
|
||||||
|
try:
|
||||||
|
raw = Path(path).read_text(encoding="utf-8")
|
||||||
|
payload = json.loads(raw)
|
||||||
|
except (OSError, ValueError, UnicodeDecodeError):
|
||||||
|
return ControlState()
|
||||||
|
if not isinstance(payload, dict):
|
||||||
|
return ControlState()
|
||||||
|
return ControlState(
|
||||||
|
paused=bool(payload.get("paused", False)),
|
||||||
|
stop=bool(payload.get("stop", False)),
|
||||||
|
seq=int(payload.get("seq") or 0),
|
||||||
|
ts=float(payload.get("ts") or 0.0),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def write_json_atomic(path: Path, payload: Any) -> None:
|
||||||
|
"""原子写 JSON 文件(同目录临时文件 + ``os.replace``)。
|
||||||
|
|
||||||
|
用于"急停恢复"相关的标记文件(``resume.json`` / ``forced_stop.json`` 等):
|
||||||
|
读取方随时可能来读,必须避免看到写了一半的内容。
|
||||||
|
"""
|
||||||
|
path = Path(path)
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
tmp = path.with_name(f".{path.name}.{os.getpid()}.tmp")
|
||||||
|
tmp.write_text(json.dumps(payload, ensure_ascii=False, indent=2),
|
||||||
|
encoding="utf-8")
|
||||||
|
os.replace(tmp, path)
|
||||||
|
|
||||||
|
|
||||||
|
def terminate_process_tree(process: subprocess.Popen | None, *,
|
||||||
|
grace_s: float = DEFAULT_GRACE_S,
|
||||||
|
poll_s: float = 0.2) -> bool:
|
||||||
|
"""等待 ``grace_s`` 让协作式停止生效,超时则强杀整棵进程树。
|
||||||
|
|
||||||
|
返回**是否动用了强杀**:``False`` 表示进程在宽限期内自行退出(模型有机会
|
||||||
|
落盘),``True`` 表示进程被强杀、最后一段产物可能缺失。调用方应先通过控制
|
||||||
|
文件请求停止,再调用本函数;本函数不负责写控制文件。
|
||||||
|
"""
|
||||||
|
if process is None:
|
||||||
|
return False
|
||||||
|
if process.poll() is not None:
|
||||||
|
return False
|
||||||
|
deadline = time.monotonic() + max(0.0, grace_s)
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
if process.poll() is not None:
|
||||||
|
return False
|
||||||
|
time.sleep(min(poll_s, max(0.0, deadline - time.monotonic())))
|
||||||
|
if process.poll() is not None:
|
||||||
|
return False
|
||||||
|
_hard_kill_tree(process)
|
||||||
|
try:
|
||||||
|
process.wait(timeout=10.0)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
pass
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def _hard_kill_tree(process: subprocess.Popen) -> None:
|
||||||
|
"""强杀进程树:POSIX 走进程组 SIGKILL,Windows 走 ``taskkill /T``。
|
||||||
|
|
||||||
|
这里所有 POSIX 专属调用都走 ``getattr`` 探测,Windows 上拿不到分支就退回
|
||||||
|
``Popen.kill()``——不会再出现 ``module 'os' has no attribute 'killpg'``。
|
||||||
|
"""
|
||||||
|
if IS_WINDOWS:
|
||||||
|
try:
|
||||||
|
subprocess.run(["taskkill", "/F", "/T", "/PID", str(process.pid)],
|
||||||
|
capture_output=True, check=False)
|
||||||
|
return
|
||||||
|
except OSError:
|
||||||
|
process.kill()
|
||||||
|
return
|
||||||
|
killpg = getattr(os, "killpg", None)
|
||||||
|
getpgid = getattr(os, "getpgid", None)
|
||||||
|
sigkill = getattr(signal, "SIGKILL", None)
|
||||||
|
if killpg is not None and getpgid is not None and sigkill is not None:
|
||||||
|
try:
|
||||||
|
killpg(getpgid(process.pid), sigkill)
|
||||||
|
return
|
||||||
|
except (ProcessLookupError, PermissionError, OSError):
|
||||||
|
pass
|
||||||
|
process.kill()
|
||||||
@@ -13,6 +13,7 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
import json
|
||||||
|
import math
|
||||||
import uuid
|
import uuid
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from time import monotonic, sleep
|
from time import monotonic, sleep
|
||||||
@@ -91,7 +92,7 @@ def _validate_position(position: dict, limit: float) -> None:
|
|||||||
value = position[key]
|
value = position[key]
|
||||||
if isinstance(value, bool) or not isinstance(value, (int, float)):
|
if isinstance(value, bool) or not isinstance(value, (int, float)):
|
||||||
raise ValueError(f"position.{key} 必须是数值")
|
raise ValueError(f"position.{key} 必须是数值")
|
||||||
if value != value or value in (float("inf"), float("-inf")):
|
if not math.isfinite(value):
|
||||||
raise ValueError(f"position.{key} 必须是有限数值")
|
raise ValueError(f"position.{key} 必须是有限数值")
|
||||||
if abs(value) > limit:
|
if abs(value) > limit:
|
||||||
raise ValueError(f"position.{key} 绝对值不得超过 {limit} 米")
|
raise ValueError(f"position.{key} 绝对值不得超过 {limit} 米")
|
||||||
@@ -234,7 +235,7 @@ class SimulatorClient:
|
|||||||
f"剩余现实时间 {self.real_remaining_s:.1f}s 不足以预留 {margin:.1f}s 退出余量")
|
f"剩余现实时间 {self.real_remaining_s:.1f}s 不足以预留 {margin:.1f}s 退出余量")
|
||||||
|
|
||||||
# ---------------------------------------------------------------- 内部机制
|
# ---------------------------------------------------------------- 内部机制
|
||||||
def _payload(self, path: str, *, position: Vec2 | None = None,
|
def _payload(self, _path: str, *, position: Vec2 | None = None,
|
||||||
channel: int | None = None) -> dict:
|
channel: int | None = None) -> dict:
|
||||||
payload = {"arena_id": self.session.arena_id,
|
payload = {"arena_id": self.session.arena_id,
|
||||||
"robot_id": self.robot_id,
|
"robot_id": self.robot_id,
|
||||||
|
|||||||
+14
-12
@@ -27,7 +27,7 @@ from ..config import ProblemConfig
|
|||||||
from ..planner import clear_certificate, fallback_clear_cells, safe_disk_radius
|
from ..planner import clear_certificate, fallback_clear_cells, safe_disk_radius
|
||||||
from ..protocol.client import SimulatorClient
|
from ..protocol.client import SimulatorClient
|
||||||
from ..protocol.log import SessionLog
|
from ..protocol.log import SessionLog
|
||||||
from ..protocol.rules import CLEAR_SUCCESS, DIRECTION, NEAR, NO_SIGNAL
|
from ..protocol.rules import CLEAR_SUCCESS, DIRECTION, NO_SIGNAL
|
||||||
from .features import NUM_ACTIONS, OBS_DIM, action_mask, build_observation, candidate_actions
|
from .features import NUM_ACTIONS, OBS_DIM, action_mask, build_observation, candidate_actions
|
||||||
|
|
||||||
|
|
||||||
@@ -65,9 +65,9 @@ class LocalizationEnv(gym.Env):
|
|||||||
from referee import LocalSimulator as _RefereeLocalSimulator
|
from referee import LocalSimulator as _RefereeLocalSimulator
|
||||||
from referee import Source
|
from referee import Source
|
||||||
from ..harness import LocalSimTransport, referee_spec
|
from ..harness import LocalSimTransport, referee_spec
|
||||||
self._Source = Source
|
self._source_cls = Source
|
||||||
self._LocalSimulator = _RefereeLocalSimulator
|
self._local_simulator_cls = _RefereeLocalSimulator
|
||||||
self._LocalSimTransport = LocalSimTransport
|
self._local_sim_transport_cls = LocalSimTransport
|
||||||
self._referee_spec = referee_spec
|
self._referee_spec = referee_spec
|
||||||
|
|
||||||
self._scene_rng = Random()
|
self._scene_rng = Random()
|
||||||
@@ -106,23 +106,22 @@ class LocalizationEnv(gym.Env):
|
|||||||
# 首次检测点:距离 [300, min(1450, r_eff-20)] 内随机,定向源取朝向
|
# 首次检测点:距离 [300, min(1450, r_eff-20)] 内随机,定向源取朝向
|
||||||
# ±89.9°(接近但不跨越90°边界,保证direction)。机器狗可到Ω外 [题面]。
|
# ±89.9°(接近但不跨越90°边界,保证direction)。机器狗可到Ω外 [题面]。
|
||||||
max_distance = max(310.0, min(1450.0, r_eff - 20.0))
|
max_distance = max(310.0, min(1450.0, r_eff - 20.0))
|
||||||
if max_distance < 310.0:
|
max_distance = max(max_distance, 310.0)
|
||||||
max_distance = 310.0
|
|
||||||
probe_distance = rng.uniform(300.0, max_distance)
|
probe_distance = rng.uniform(300.0, max_distance)
|
||||||
probe_bearing = (heading + rng.uniform(-89.9, 89.9)
|
probe_bearing = (heading + rng.uniform(-89.9, 89.9)
|
||||||
if directional else rng.uniform(0.0, 360.0))
|
if directional else rng.uniform(0.0, 360.0))
|
||||||
probe_radians = math.radians(probe_bearing)
|
probe_radians = math.radians(probe_bearing)
|
||||||
probe = (x + probe_distance * math.cos(probe_radians),
|
probe = (x + probe_distance * math.cos(probe_radians),
|
||||||
y + probe_distance * math.sin(probe_radians))
|
y + probe_distance * math.sin(probe_radians))
|
||||||
source = self._Source(channel=channel, x=x, y=y, r_eff_m=r_eff,
|
source = self._source_cls(channel=channel, x=x, y=y, r_eff_m=r_eff,
|
||||||
directional=directional, heading_deg=heading)
|
directional=directional, heading_deg=heading)
|
||||||
return source, probe
|
return source, probe
|
||||||
|
|
||||||
def _make_episode(self) -> bool:
|
def _make_episode(self) -> bool:
|
||||||
"""重建模拟器并完成首次测向;返回 False 表示连续未得到 direction。"""
|
"""重建模拟器并完成首次测向;返回 False 表示连续未得到 direction。"""
|
||||||
for _ in range(20):
|
for _ in range(20):
|
||||||
self._source, probe = self._make_source_and_probe()
|
self._source, probe = self._make_source_and_probe()
|
||||||
self._simulator = self._LocalSimulator(
|
self._simulator = self._local_simulator_cls(
|
||||||
[self._source], robot_id="rl-train",
|
[self._source], robot_id="rl-train",
|
||||||
spec=self._referee_spec(
|
spec=self._referee_spec(
|
||||||
self.config, max_real_duration_s=self.real_budget_s),
|
self.config, max_real_duration_s=self.real_budget_s),
|
||||||
@@ -131,7 +130,7 @@ class LocalizationEnv(gym.Env):
|
|||||||
stress_mode=True)
|
stress_mode=True)
|
||||||
self._client = SimulatorClient(
|
self._client = SimulatorClient(
|
||||||
"rl-train", config=self.config,
|
"rl-train", config=self.config,
|
||||||
transport=self._LocalSimTransport(self._simulator),
|
transport=self._local_sim_transport_cls(self._simulator),
|
||||||
log=SessionLog())
|
log=SessionLog())
|
||||||
self._client.enter()
|
self._client.enter()
|
||||||
outcome = self._client.measure(probe, self._source.channel)
|
outcome = self._client.measure(probe, self._source.channel)
|
||||||
@@ -149,7 +148,9 @@ class LocalizationEnv(gym.Env):
|
|||||||
if self._scene_rng.random() < self.handoff_probability:
|
if self._scene_rng.random() < self.handoff_probability:
|
||||||
angle = self._scene_rng.uniform(0, 2*math.pi)
|
angle = self._scene_rng.uniform(0, 2*math.pi)
|
||||||
reach = self.config.field.omega_radius_m * math.sqrt(self._scene_rng.random())
|
reach = self.config.field.omega_radius_m * math.sqrt(self._scene_rng.random())
|
||||||
other = self.config.field.channel_min if self._channel != self.config.field.channel_min else self.config.field.channel_max
|
other = (self.config.field.channel_min
|
||||||
|
if self._channel != self.config.field.channel_min
|
||||||
|
else self.config.field.channel_max)
|
||||||
self._client.measure((reach*math.cos(angle), reach*math.sin(angle)), other)
|
self._client.measure((reach*math.cos(angle), reach*math.sin(angle)), other)
|
||||||
self._start_virtual_s = self._client.state.virtual_time_s
|
self._start_virtual_s = self._client.state.virtual_time_s
|
||||||
return True
|
return True
|
||||||
@@ -325,5 +326,6 @@ class LocalizationEnv(gym.Env):
|
|||||||
return observation, float(reward), self._terminated, self._truncated, info
|
return observation, float(reward), self._terminated, self._truncated, info
|
||||||
|
|
||||||
def close(self):
|
def close(self):
|
||||||
if self._client is not None and self._client.state.entered and not self._client.state.exited:
|
if (self._client is not None and self._client.state.entered
|
||||||
|
and not self._client.state.exited):
|
||||||
self._client.exit()
|
self._client.exit()
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ import math
|
|||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
from ..config import ProblemConfig, Vec2
|
from ..config import ProblemConfig, Vec2
|
||||||
from ..planner import candidate_points, safe_disk_radius
|
from ..planner import safe_disk_radius
|
||||||
from ._constants import GLOBAL_DIM, LADDER_SLOTS, NUM_ACTIONS, OBS_DIM, PER_ACTION_DIM
|
from ._constants import GLOBAL_DIM, LADDER_SLOTS, NUM_ACTIONS, OBS_DIM, PER_ACTION_DIM
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
@@ -100,7 +100,8 @@ def action_mask(region, position: Vec2, config: ProblemConfig, problem: int,
|
|||||||
Q3有距离保证盘时屏蔽盘外探点;Q4允许盘外探点,但绝不把距离保证
|
Q3有距离保证盘时屏蔽盘外探点;Q4允许盘外探点,但绝不把距离保证
|
||||||
当作朝向保证。训练与推理共用此函数,不读取源位置或朝向。
|
当作朝向保证。训练与推理共用此函数,不读取源位置或朝向。
|
||||||
"""
|
"""
|
||||||
key = lambda p: (round(p[0]*2), round(p[1]*2))
|
def key(p):
|
||||||
|
return (round(p[0]*2), round(p[1]*2))
|
||||||
visited = {key(r['position']) for r in getattr(belief, 'measurements', [])}
|
visited = {key(r['position']) for r in getattr(belief, 'measurements', [])}
|
||||||
seen = set()
|
seen = set()
|
||||||
center, _ = region.mec()
|
center, _ = region.mec()
|
||||||
|
|||||||
@@ -40,7 +40,8 @@ class PointSelectionPolicy:
|
|||||||
device = "cpu"
|
device = "cpu"
|
||||||
self.device = device
|
self.device = device
|
||||||
meta_path = path.with_suffix('.json')
|
meta_path = path.with_suffix('.json')
|
||||||
self.metadata = json.loads(meta_path.read_text(encoding='utf-8')) if meta_path.exists() else {}
|
self.metadata = (json.loads(meta_path.read_text(encoding='utf-8'))
|
||||||
|
if meta_path.exists() else {})
|
||||||
self.feature_version = self.metadata.get('feature_version', 1)
|
self.feature_version = self.metadata.get('feature_version', 1)
|
||||||
if self.feature_version not in (1, 2):
|
if self.feature_version not in (1, 2):
|
||||||
raise RuntimeError('不支持的RL特征版本')
|
raise RuntimeError('不支持的RL特征版本')
|
||||||
@@ -78,7 +79,9 @@ class PointSelectionPolicy:
|
|||||||
region, position, channel, config, problem, blocked, belief,
|
region, position, channel, config, problem, blocked, belief,
|
||||||
virtual_time_s=virtual_time_s,
|
virtual_time_s=virtual_time_s,
|
||||||
receiver_channel=receiver_channel if self.feature_version >= 2 else None)
|
receiver_channel=receiver_channel if self.feature_version >= 2 else None)
|
||||||
kwargs = {'action_masks': action_mask(region, position, config, problem, blocked, belief)} if self.masked else {}
|
kwargs = ({'action_masks': action_mask(region, position, config, problem,
|
||||||
|
blocked, belief)}
|
||||||
|
if self.masked else {})
|
||||||
action, _state = self.model.predict(observation, deterministic=True, **kwargs)
|
action, _state = self.model.predict(observation, deterministic=True, **kwargs)
|
||||||
action = int(action)
|
action = int(action)
|
||||||
candidates = candidate_actions(region, position, config, blocked)
|
candidates = candidate_actions(region, position, config, blocked)
|
||||||
|
|||||||
@@ -0,0 +1,397 @@
|
|||||||
|
"""训练快照的纯函数派生:曲线统计、收敛判读、进程视图。
|
||||||
|
|
||||||
|
从 ``server.py`` 拆出来的两个理由:
|
||||||
|
|
||||||
|
1. 断开 ``demo`` ↔ ``server`` 的循环导入——离线预览(``demo.py``)只想复用
|
||||||
|
这两个纯函数,没必要反向 import 整个 HTTP 服务器模块;
|
||||||
|
2. 让"数值判读"可以脱离 HTTP 与子进程单独测试。
|
||||||
|
|
||||||
|
本模块只依赖标准库,不做任何 IO、不持有任何状态。
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import math
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
__all__ = ["MAX_LOG_LINES", "MAX_METRICS", "finite_float",
|
||||||
|
"summarize_convergence", "describe_process", "process_pid"]
|
||||||
|
|
||||||
|
MAX_LOG_LINES = 4000
|
||||||
|
MAX_METRICS = 5000
|
||||||
|
|
||||||
|
_NUM_RE = re.compile(r"^[+-]?(?:\d+\.?\d*|\.\d+)(?:[eE][+-]?\d+)?$")
|
||||||
|
|
||||||
|
|
||||||
|
def finite_float(value: Any) -> float | None:
|
||||||
|
"""把任意输入转成有限浮点数;NaN / ±inf / 非数字一律返回 None。
|
||||||
|
|
||||||
|
旧实现用 ``value != value`` 判 NaN,pylint 报 ``comparison-with-itself``;
|
||||||
|
这里用 ``math.isfinite`` 表达同一语义,意图更直白。
|
||||||
|
"""
|
||||||
|
if value is None or isinstance(value, bool):
|
||||||
|
return None
|
||||||
|
if isinstance(value, (int, float)):
|
||||||
|
value = float(value)
|
||||||
|
if not math.isfinite(value):
|
||||||
|
return None
|
||||||
|
return value
|
||||||
|
text = str(value).strip()
|
||||||
|
if not _NUM_RE.match(text):
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return float(text)
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _series(metrics: list[dict[str, Any]], key: str) -> list[tuple[float, float]]:
|
||||||
|
points: list[tuple[float, float]] = []
|
||||||
|
for point in metrics:
|
||||||
|
value = finite_float(point.get(key))
|
||||||
|
steps = finite_float(point.get("timesteps")) or finite_float(
|
||||||
|
point.get("total_timesteps"))
|
||||||
|
if value is None:
|
||||||
|
continue
|
||||||
|
points.append((steps if steps is not None else float(len(points) + 1), value))
|
||||||
|
return points
|
||||||
|
|
||||||
|
|
||||||
|
def _last_value(points: list[tuple[float, float]]) -> float | None:
|
||||||
|
return points[-1][1] if points else None
|
||||||
|
|
||||||
|
|
||||||
|
def _spread(points: list[tuple[float, float]], window: int = 10
|
||||||
|
) -> tuple[float, float] | None:
|
||||||
|
"""最近 window 个点的 (极差, 起点绝对值),用于判断"是否还在动"。"""
|
||||||
|
sample = points[-window:]
|
||||||
|
if len(sample) < 3:
|
||||||
|
return None
|
||||||
|
ys = [item[1] for item in sample]
|
||||||
|
return max(ys) - min(ys), abs(ys[0])
|
||||||
|
|
||||||
|
|
||||||
|
def _cv(points: list[tuple[float, float]], window: int = 10) -> float | None:
|
||||||
|
"""最近 window 个点的变异系数(标准差/|均值|),越小越平稳。"""
|
||||||
|
sample = [item[1] for item in points[-window:]]
|
||||||
|
if len(sample) < 3:
|
||||||
|
return None
|
||||||
|
mean = sum(sample) / len(sample)
|
||||||
|
if abs(mean) < 1e-12:
|
||||||
|
return None
|
||||||
|
var = sum((value - mean) ** 2 for value in sample) / len(sample)
|
||||||
|
return (var ** 0.5) / abs(mean)
|
||||||
|
|
||||||
|
|
||||||
|
def _slope(points: list[tuple[float, float]], window: int = 8) -> float | None:
|
||||||
|
sample = points[-window:]
|
||||||
|
if len(sample) < 3:
|
||||||
|
return None
|
||||||
|
xs = [item[0] for item in sample]
|
||||||
|
ys = [item[1] for item in sample]
|
||||||
|
n = len(xs)
|
||||||
|
mean_x = sum(xs) / n
|
||||||
|
mean_y = sum(ys) / n
|
||||||
|
denom = sum((x - mean_x) ** 2 for x in xs)
|
||||||
|
if abs(denom) < 1e-12:
|
||||||
|
return None
|
||||||
|
return sum((x - mean_x) * (y - mean_y) for x, y in zip(xs, ys)) / denom
|
||||||
|
|
||||||
|
|
||||||
|
def _rate_per_1k(reward_points: list[tuple[float, float]],
|
||||||
|
window: int = 8) -> float | None:
|
||||||
|
"""最近 window 个点的奖励增益换算成"每 1000 环境步",便于读趋势。"""
|
||||||
|
slope = _slope(reward_points, window)
|
||||||
|
if slope is None:
|
||||||
|
return None
|
||||||
|
return slope * 1000.0
|
||||||
|
|
||||||
|
|
||||||
|
def _fmt_num(value: float | None, digits: int = 4) -> str:
|
||||||
|
if value is None:
|
||||||
|
return "—"
|
||||||
|
if value != 0 and abs(value) < 1e-3:
|
||||||
|
return f"{value:.3e}"
|
||||||
|
return f"{value:.{digits}g}"
|
||||||
|
|
||||||
|
|
||||||
|
def _flatness(points: list[tuple[float, float]], window: int = 10
|
||||||
|
) -> tuple[float, float] | None:
|
||||||
|
"""窗口内"趋势位移 / 逐点噪声"比值:<1 表示看不出系统性变化。
|
||||||
|
|
||||||
|
判定收敛时只看斜率绝对值会把"很慢但持续的提升"误判为平台;这里把窗口
|
||||||
|
对半切,用两半均值之差当趋势位移,用相邻点平均绝对差当噪声尺度。返回
|
||||||
|
(比值, 噪声尺度),两者都小才是真的"又平又稳"。
|
||||||
|
"""
|
||||||
|
sample = points[-window:]
|
||||||
|
if len(sample) < 4:
|
||||||
|
return None
|
||||||
|
ys = [item[1] for item in sample]
|
||||||
|
half = len(ys) // 2
|
||||||
|
shift = abs(sum(ys[half:]) / len(ys[half:]) - sum(ys[:half]) / len(ys[:half]))
|
||||||
|
noise = sum(abs(b - a) for a, b in zip(ys, ys[1:])) / (len(ys) - 1)
|
||||||
|
if noise <= 0:
|
||||||
|
return (0.0, 0.0) if shift <= 0 else (float("inf"), 0.0)
|
||||||
|
return shift / noise, noise
|
||||||
|
|
||||||
|
|
||||||
|
def summarize_convergence(metrics: list[dict[str, Any]],
|
||||||
|
monitor: list[dict[str, Any]]) -> dict[str, Any]:
|
||||||
|
"""给前端返回可解释的收敛状态与各参数最近值。
|
||||||
|
|
||||||
|
这是**启发式判读**,不替代正式学习曲线分析:只依据最近窗口的斜率、
|
||||||
|
极差与变异系数给出"平坦/上升/未上升"的分类,并把同一组数字交给前端
|
||||||
|
展示,便于人工复核。
|
||||||
|
"""
|
||||||
|
reward_points = _series(metrics, "ep_rew_mean")
|
||||||
|
if not reward_points and monitor:
|
||||||
|
reward_points = [(float(index), float(row.get("r", 0.0)))
|
||||||
|
for index, row in enumerate(monitor)
|
||||||
|
if finite_float(row.get("r")) is not None]
|
||||||
|
kl_points = _series(metrics, "approx_kl")
|
||||||
|
ev_points = _series(metrics, "explained_variance")
|
||||||
|
vf_points = _series(metrics, "value_loss")
|
||||||
|
pg_points = _series(metrics, "policy_gradient_loss")
|
||||||
|
entropy_points = _series(metrics, "entropy_loss")
|
||||||
|
fps_points = _series(metrics, "fps")
|
||||||
|
|
||||||
|
reward_slope = _slope(reward_points)
|
||||||
|
reward_rate = _rate_per_1k(reward_points)
|
||||||
|
spread = _spread(reward_points)
|
||||||
|
cv = _cv(reward_points)
|
||||||
|
latest_kl = _last_value(kl_points)
|
||||||
|
latest_ev = _last_value(ev_points)
|
||||||
|
latest_vf = _last_value(vf_points)
|
||||||
|
latest_pg = _last_value(pg_points)
|
||||||
|
latest_entropy = _last_value(entropy_points)
|
||||||
|
latest_fps = _last_value(fps_points)
|
||||||
|
latest_timesteps = metrics[-1].get("timesteps") if metrics else None
|
||||||
|
if latest_timesteps is None and metrics:
|
||||||
|
latest_timesteps = metrics[-1].get("total_timesteps")
|
||||||
|
latest_reward = _last_value(reward_points)
|
||||||
|
latest_len = _last_value(_series(metrics, "ep_len_mean"))
|
||||||
|
latest_updates = _last_value(_series(metrics, "n_updates"))
|
||||||
|
window = min(10, max(3, len(reward_points)))
|
||||||
|
reward_window = [round(value, 6) for _, value in reward_points[-window:]]
|
||||||
|
best_reward = max((value for _, value in reward_points), default=None)
|
||||||
|
|
||||||
|
notes: list[str] = []
|
||||||
|
if not metrics and not monitor:
|
||||||
|
status, tone = "等待首个 rollout", "gray"
|
||||||
|
notes.append("训练进程启动后先输出环境初始化与首个 rollout 的进度行。")
|
||||||
|
elif not metrics:
|
||||||
|
status, tone = "已收到回合奖励,等待进度行", "warn"
|
||||||
|
notes.append("Monitor CSV 已有回合记录,但尚未收到 --progress-json 进度行;"
|
||||||
|
"首个 rollout 结束后会出现完整指标。")
|
||||||
|
elif len(metrics) < 3:
|
||||||
|
status, tone = "预热阶段", "warn"
|
||||||
|
notes.append("已收到初始指标,需要更多 rollout 才能判断趋势。")
|
||||||
|
else:
|
||||||
|
tone, status = "ok", "收敛趋势良好"
|
||||||
|
# 平台判据(三条同时成立):对半窗口看不出位移、窗口内波动小、
|
||||||
|
# 逐点噪声本身也小。慢速但持续的提升会因第一条被排除。
|
||||||
|
flat = _flatness(reward_points)
|
||||||
|
plateau = False
|
||||||
|
if flat is not None and cv is not None and latest_reward is not None:
|
||||||
|
flat_ratio, flat_noise = flat
|
||||||
|
scale = max(abs(latest_reward), 1e-9)
|
||||||
|
plateau = (flat_ratio <= 1.0 and cv <= 0.15
|
||||||
|
and flat_noise <= 0.02 * scale
|
||||||
|
and (spread is None or spread[1] < 1e-12
|
||||||
|
or spread[0] <= 0.15 * spread[1]))
|
||||||
|
if latest_kl is not None and latest_kl > 0.05:
|
||||||
|
status, tone = "KL 偏大,策略更新可能过猛", "bad"
|
||||||
|
notes.append(f"approx_kl={latest_kl:.4f} > 0.05")
|
||||||
|
elif latest_ev is not None and latest_ev < 0.0:
|
||||||
|
status, tone = "价值函数尚未拟合", "warn"
|
||||||
|
notes.append(f"explained_variance={latest_ev:.3f} < 0")
|
||||||
|
elif reward_slope is not None and reward_slope < 0:
|
||||||
|
status, tone = "奖励未形成上升趋势", "warn"
|
||||||
|
notes.append(f"最近 {window} 点奖励斜率为负({_fmt_num(reward_slope)}/step)")
|
||||||
|
elif plateau:
|
||||||
|
status, tone = "已趋于平稳(疑似收敛)", "ok"
|
||||||
|
notes.append("窗口内对半位移小于逐点噪声,且波动很小"
|
||||||
|
f"(变异系数 {_fmt_num(cv, 3)})")
|
||||||
|
else:
|
||||||
|
notes.append(f"最近 {window} 点奖励斜率为正({_fmt_num(reward_slope)}/step)")
|
||||||
|
if latest_kl is not None and latest_kl <= 0.02:
|
||||||
|
notes.append(f"approx_kl={latest_kl:.4f} 较稳定")
|
||||||
|
if latest_ev is not None:
|
||||||
|
notes.append(f"explained_variance={latest_ev:.3f}")
|
||||||
|
if cv is not None:
|
||||||
|
notes.append(f"奖励变异系数={_fmt_num(cv, 3)}")
|
||||||
|
if latest_entropy is not None and abs(latest_entropy) < 1e-3:
|
||||||
|
notes.append("策略熵接近 0,探索可能过早停止")
|
||||||
|
return {
|
||||||
|
"status": status,
|
||||||
|
"tone": tone,
|
||||||
|
"notes": notes,
|
||||||
|
"window": window,
|
||||||
|
"latest_timesteps": latest_timesteps,
|
||||||
|
"reward_slope": reward_slope,
|
||||||
|
"reward_rate_per_1k": reward_rate,
|
||||||
|
"reward_cv": cv,
|
||||||
|
"reward_spread": spread[0] if spread else None,
|
||||||
|
"reward_window": reward_window,
|
||||||
|
"best_reward": best_reward,
|
||||||
|
"latest_reward": latest_reward,
|
||||||
|
"latest_ep_len": latest_len,
|
||||||
|
"latest_updates": latest_updates,
|
||||||
|
"latest_kl": latest_kl,
|
||||||
|
"latest_explained_variance": latest_ev,
|
||||||
|
"latest_value_loss": latest_vf,
|
||||||
|
"latest_policy_loss": latest_pg,
|
||||||
|
"latest_entropy_loss": latest_entropy,
|
||||||
|
"latest_fps": latest_fps,
|
||||||
|
"reward_points": len(reward_points),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# 快照状态 -> 界面展示用的进程语义(阶段名称、色调、是否终态)
|
||||||
|
_STATUS_VIEW = {
|
||||||
|
"created": ("已创建", "gray", False, "命令已构造,等待 spawn"),
|
||||||
|
"running": ("运行中", "ok", False, "子进程持有控制台输出与独立进程组"),
|
||||||
|
"paused": ("已暂停", "warn", False, "进程存活但停在 step 边界;点「继续」从暂停处恢复"),
|
||||||
|
"stopping": ("停止中", "warn", False, "已请求优雅停止,等待子进程保存并退出"),
|
||||||
|
"stopped": ("已手动停止", "warn", True, "收到停止请求后退出;脚本会在 step 边界保存模型与元数据"),
|
||||||
|
"exited": ("正常结束", "ok", True, "退出码 0:训练与评估流程走完"),
|
||||||
|
"failed": ("异常结束", "bad", True, "非零退出码或启动失败,见日志尾部"),
|
||||||
|
}
|
||||||
|
|
||||||
|
_PHASES = [
|
||||||
|
("created", "构造命令"),
|
||||||
|
("spawn", "启动子进程"),
|
||||||
|
("rollout", "采集 rollout"),
|
||||||
|
("update", "PPO 更新"),
|
||||||
|
("evaluate", "评估 / 保存"),
|
||||||
|
("done", "收敛归档"),
|
||||||
|
]
|
||||||
|
|
||||||
|
# 状态 -> 当前所处的阶段节点;节点之前为已完成,之后为未开始。
|
||||||
|
_STATUS_PHASE = {
|
||||||
|
"created": "created",
|
||||||
|
"running": "update",
|
||||||
|
"paused": "update",
|
||||||
|
"stopping": "update",
|
||||||
|
"stopped": "done",
|
||||||
|
"exited": "done",
|
||||||
|
"failed": "done",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def process_pid(process: subprocess.Popen | None) -> int | None:
|
||||||
|
"""已退出进程的 pid 访问会抛 OSError;这里统一吞掉,只用于展示。"""
|
||||||
|
if process is None:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return int(process.pid)
|
||||||
|
except (OSError, ValueError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def describe_process(snapshot: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
"""把 snapshot 派生为界面直接可渲染的进程对象。
|
||||||
|
|
||||||
|
包含:状态语义词、进度(步/百分比)、墙钟与剩余时间估计、最近吞吐、
|
||||||
|
进程时间线节点。数字都取自 snapshot,前端不再重复累加,避免两个刷新
|
||||||
|
频率不同的计数器互相打架。
|
||||||
|
"""
|
||||||
|
status = str(snapshot.get("status") or "created")
|
||||||
|
label, tone, terminal, hint = _STATUS_VIEW.get(
|
||||||
|
status, (status, "gray", False, ""))
|
||||||
|
config = snapshot.get("config") or {}
|
||||||
|
order = [key for key, _ in _PHASES]
|
||||||
|
current = _STATUS_PHASE.get(status, "created")
|
||||||
|
timeline = []
|
||||||
|
for key, title in _PHASES:
|
||||||
|
index = order.index(key)
|
||||||
|
if index < order.index(current):
|
||||||
|
state = "done"
|
||||||
|
elif index == order.index(current):
|
||||||
|
state = "active"
|
||||||
|
else:
|
||||||
|
state = "pending"
|
||||||
|
timeline.append({"key": key, "label": title, "state": state})
|
||||||
|
|
||||||
|
metrics = snapshot.get("metrics") or []
|
||||||
|
latest: dict[str, Any] = metrics[-1] if metrics else {}
|
||||||
|
convergence = snapshot.get("convergence") or {}
|
||||||
|
total_steps = finite_float(config.get("timesteps")) or 0.0
|
||||||
|
# SB3 表格行可能没有 steps 字段,因此进度取"所有进度点的最大步数",
|
||||||
|
# 而不是只看最后一个点——否则末行恰好缺字段时进度会回退到 0。
|
||||||
|
done_steps = 0.0
|
||||||
|
for point in metrics:
|
||||||
|
value = (finite_float(point.get("timesteps"))
|
||||||
|
or finite_float(point.get("total_timesteps")))
|
||||||
|
if value is not None and value > done_steps:
|
||||||
|
done_steps = value
|
||||||
|
if done_steps == 0.0:
|
||||||
|
done_steps = finite_float(convergence.get("latest_timesteps")) or 0.0
|
||||||
|
# init/resume(课程迁移)时 num_timesteps 从上一阶段继续,进度必须按本次
|
||||||
|
# 增量算;start_timesteps 由训练脚本的进度 JSON 提供,缺省按 0 处理。
|
||||||
|
start_steps = 0.0
|
||||||
|
for point in metrics:
|
||||||
|
value = finite_float(point.get("start_timesteps"))
|
||||||
|
if value is not None:
|
||||||
|
start_steps = value
|
||||||
|
break
|
||||||
|
trained_steps = max(0.0, done_steps - start_steps)
|
||||||
|
percent = (100.0 * trained_steps / total_steps) if total_steps > 0 else None
|
||||||
|
if percent is not None:
|
||||||
|
percent = max(0.0, min(100.0, percent))
|
||||||
|
elapsed = float(snapshot.get("elapsed_s") or 0.0)
|
||||||
|
rate = _latest_rate(metrics)
|
||||||
|
remaining: float | None = None
|
||||||
|
eta_ts: float | None = None
|
||||||
|
if rate and rate > 0 and total_steps > 0 and trained_steps < total_steps:
|
||||||
|
remaining = (total_steps - trained_steps) / rate
|
||||||
|
eta_ts = float(snapshot.get("finished_at") or time.time()) + remaining
|
||||||
|
if not terminal and remaining is None:
|
||||||
|
cap = finite_float(config.get("max_wall_s")) or 0.0
|
||||||
|
if cap > 0:
|
||||||
|
remaining = max(0.0, cap - elapsed)
|
||||||
|
return {
|
||||||
|
"label": label,
|
||||||
|
"tone": tone,
|
||||||
|
"terminal": terminal,
|
||||||
|
"hint": hint,
|
||||||
|
"phase": current,
|
||||||
|
"timeline": timeline,
|
||||||
|
"percent": percent,
|
||||||
|
"done_steps": done_steps,
|
||||||
|
"start_steps": start_steps,
|
||||||
|
"trained_steps": trained_steps,
|
||||||
|
"total_steps": total_steps,
|
||||||
|
"elapsed_s": round(elapsed, 3),
|
||||||
|
"rate_per_s": rate,
|
||||||
|
"remaining_s": remaining,
|
||||||
|
"eta_epoch_s": eta_ts,
|
||||||
|
"log_total": snapshot.get("log_total"),
|
||||||
|
"log_capacity": MAX_LOG_LINES,
|
||||||
|
"metric_total": snapshot.get("metric_total"),
|
||||||
|
"latest": latest,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _latest_rate(metrics: list[dict[str, Any]], window: int = 6) -> float | None:
|
||||||
|
"""最近若干个**带 fps 的**进度点的平均吞吐(步/秒),用于剩余时间估计。
|
||||||
|
|
||||||
|
progress JSON 行不含 fps,只有 SB3 表格行有;若只看最后 window 个点,
|
||||||
|
恰好落在若干 JSON 行之间时会得到 None,剩余时间就会退化成墙钟上限的
|
||||||
|
粗糙估计。这里向前扫描直到拿到至少 1 个 fps(最多看 3*window 个点)。
|
||||||
|
"""
|
||||||
|
if not metrics:
|
||||||
|
return None
|
||||||
|
usable: list[float] = []
|
||||||
|
for point in reversed(metrics[-window * 3:]):
|
||||||
|
value = finite_float(point.get("fps"))
|
||||||
|
if value and value > 0:
|
||||||
|
usable.append(value)
|
||||||
|
if len(usable) >= window:
|
||||||
|
break
|
||||||
|
if not usable:
|
||||||
|
return None
|
||||||
|
return sum(usable) / len(usable)
|
||||||
@@ -15,6 +15,8 @@ import random
|
|||||||
import time
|
import time
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
|
from .analysis import describe_process, summarize_convergence
|
||||||
|
|
||||||
__all__ = ["DEMO_CONFIG", "DEMO_SCENARIOS", "demo_metrics", "demo_monitor",
|
__all__ = ["DEMO_CONFIG", "DEMO_SCENARIOS", "demo_metrics", "demo_monitor",
|
||||||
"demo_log_lines", "demo_snapshot"]
|
"demo_log_lines", "demo_snapshot"]
|
||||||
|
|
||||||
@@ -44,7 +46,7 @@ DEMO_STEPS = 48
|
|||||||
|
|
||||||
|
|
||||||
def _finite(value: float) -> float | None:
|
def _finite(value: float) -> float | None:
|
||||||
if value != value or value in (float("inf"), float("-inf")):
|
if not math.isfinite(value):
|
||||||
return None
|
return None
|
||||||
return round(value, 8)
|
return round(value, 8)
|
||||||
|
|
||||||
@@ -171,8 +173,6 @@ def demo_snapshot(*, log_tail: int = 400, scenario: str = "rising",
|
|||||||
(而不是刚好 100%);``scenario`` 选择收敛判读分支;``fps`` 决定剩余
|
(而不是刚好 100%);``scenario`` 选择收敛判读分支;``fps`` 决定剩余
|
||||||
时间估计的吞吐量级。
|
时间估计的吞吐量级。
|
||||||
"""
|
"""
|
||||||
from .server import describe_process, summarize_convergence
|
|
||||||
|
|
||||||
now = time.time()
|
now = time.time()
|
||||||
ratio = min(1.0, max(0.05, float(done_ratio)))
|
ratio = min(1.0, max(0.05, float(done_ratio)))
|
||||||
metrics = demo_metrics(count=max(6, int(round(DEMO_STEPS * ratio))),
|
metrics = demo_metrics(count=max(6, int(round(DEMO_STEPS * ratio))),
|
||||||
@@ -193,6 +193,7 @@ def demo_snapshot(*, log_tail: int = 400, scenario: str = "rising",
|
|||||||
"elapsed_s": 184.0,
|
"elapsed_s": 184.0,
|
||||||
"returncode": None,
|
"returncode": None,
|
||||||
"error": None,
|
"error": None,
|
||||||
|
"forced_kill": False,
|
||||||
"command": list(DEMO_COMMAND),
|
"command": list(DEMO_COMMAND),
|
||||||
"model": str(config["model"]),
|
"model": str(config["model"]),
|
||||||
"config": config,
|
"config": config,
|
||||||
@@ -203,6 +204,17 @@ def demo_snapshot(*, log_tail: int = 400, scenario: str = "rising",
|
|||||||
"metric_end": len(metrics),
|
"metric_end": len(metrics),
|
||||||
"metric_total": len(metrics),
|
"metric_total": len(metrics),
|
||||||
"monitor": monitor,
|
"monitor": monitor,
|
||||||
|
"recovery": {
|
||||||
|
"can_resume": True,
|
||||||
|
"model": str(config["model"]),
|
||||||
|
"timesteps": int(DEMO_STEPS * ratio * 12),
|
||||||
|
"reason": "recovery",
|
||||||
|
"updated_unix_s": now - 24.0,
|
||||||
|
"source": "resume.json",
|
||||||
|
"hint_file": str(config["model"]).rsplit("/", 1)[0] + "/resume.json",
|
||||||
|
"candidates": 3,
|
||||||
|
"preview": True,
|
||||||
|
},
|
||||||
"preview": True,
|
"preview": True,
|
||||||
"preview_scenario": scenario,
|
"preview_scenario": scenario,
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -163,6 +163,24 @@ CONFIG_FIELDS: list[dict[str, Any]] = [
|
|||||||
"label": "覆盖已有模型 --force", "default": False,
|
"label": "覆盖已有模型 --force", "default": False,
|
||||||
"help": "输出路径已存在时必须勾选,否则训练脚本会拒绝覆盖。与续训互斥。",
|
"help": "输出路径已存在时必须勾选,否则训练脚本会拒绝覆盖。与续训互斥。",
|
||||||
},
|
},
|
||||||
|
# ---------------------------------------------------------- 恢复与落盘
|
||||||
|
{
|
||||||
|
"key": "recovery_every", "kind": "integer", "group": "recovery",
|
||||||
|
"label": "滚动恢复点间隔 --recovery-every", "default": 2000,
|
||||||
|
"help": "每 N 个环境步原子覆盖写 <checkpoints>/<stem>_latest.zip,"
|
||||||
|
"训练开始前先写起步基线;急停(强杀)后最多丢 N 步。0=关闭。",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"key": "checkpoint_every", "kind": "integer", "group": "recovery",
|
||||||
|
"label": "历史 checkpoint 间隔 --checkpoint-every", "default": 20000,
|
||||||
|
"help": "每 N 个环境步另留一份 <stem>_<步数>.zip 供返祖对比;"
|
||||||
|
"0=只保留滚动恢复点。",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"key": "checkpoint_keep", "kind": "integer", "group": "recovery",
|
||||||
|
"label": "历史 checkpoint 保留数 --checkpoint-keep", "default": 3,
|
||||||
|
"help": "只保留最近 K 个历史 checkpoint;滚动恢复点与起步基线不受影响。",
|
||||||
|
},
|
||||||
]
|
]
|
||||||
|
|
||||||
CONFIG_GROUPS: list[dict[str, str]] = [
|
CONFIG_GROUPS: list[dict[str, str]] = [
|
||||||
@@ -170,6 +188,7 @@ CONFIG_GROUPS: list[dict[str, str]] = [
|
|||||||
{"key": "ppo", "label": "PPO 超参数", "hint": "采样宽度与更新强度"},
|
{"key": "ppo", "label": "PPO 超参数", "hint": "采样宽度与更新强度"},
|
||||||
{"key": "runtime", "label": "运行环境", "hint": "并行、设备与低层执行器"},
|
{"key": "runtime", "label": "运行环境", "hint": "并行、设备与低层执行器"},
|
||||||
{"key": "scale", "label": "训练规模", "hint": "步数、墙钟上限与评估"},
|
{"key": "scale", "label": "训练规模", "hint": "步数、墙钟上限与评估"},
|
||||||
|
{"key": "recovery", "label": "恢复与落盘", "hint": "急停恢复点与历史 checkpoint"},
|
||||||
{"key": "paths", "label": "模型与迁移", "hint": "产物路径、续训与课程迁移"},
|
{"key": "paths", "label": "模型与迁移", "hint": "产物路径、续训与课程迁移"},
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|||||||
+285
-401
@@ -4,7 +4,8 @@
|
|||||||
设计边界:
|
设计边界:
|
||||||
- 只监听 127.0.0.1,只允许白名单静态文件;
|
- 只监听 127.0.0.1,只允许白名单静态文件;
|
||||||
- 只允许一个训练任务,命令由服务端用参数列表构造,不经过 shell;
|
- 只允许一个训练任务,命令由服务端用参数列表构造,不经过 shell;
|
||||||
- 支持启动/停止/状态查询;停止时发送 SIGTERM 到独立进程组;
|
- 支持启动/暂停/继续/停止/状态查询;暂停与停止通过**控制文件**下发,
|
||||||
|
POSIX 与 Windows 行为一致(详见 :mod:`mcm26b.process_control`);
|
||||||
- 训练进程通过 ``--progress-json`` 输出单行 JSON 进度,服务端同时兼容
|
- 训练进程通过 ``--progress-json`` 输出单行 JSON 进度,服务端同时兼容
|
||||||
Stable-Baselines3 的表格输出和 Monitor CSV。
|
Stable-Baselines3 的表格输出和 Monitor CSV。
|
||||||
|
|
||||||
@@ -12,15 +13,17 @@
|
|||||||
- ``GET /api/train/schema`` 字段/分组/预设/指标字典(``schema.py`` 单一出处)
|
- ``GET /api/train/schema`` 字段/分组/预设/指标字典(``schema.py`` 单一出处)
|
||||||
- ``GET /api/train/state`` 快照;``after_metric=N`` 只回增量指标
|
- ``GET /api/train/state`` 快照;``after_metric=N`` 只回增量指标
|
||||||
- ``POST /api/train/start`` 以部分字段启动(其余取默认值)
|
- ``POST /api/train/start`` 以部分字段启动(其余取默认值)
|
||||||
- ``POST /api/train/stop`` SIGTERM 独立进程组
|
- ``POST /api/train/pause`` 请求训练进程在 step 边界挂起
|
||||||
|
- ``POST /api/train/resume`` 解除挂起
|
||||||
|
- ``POST /api/train/stop`` 请求优雅停止,超时后强杀进程树
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import dataclasses
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
import signal
|
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
import threading
|
import threading
|
||||||
@@ -32,15 +35,24 @@ from pathlib import Path
|
|||||||
from typing import Any
|
from typing import Any
|
||||||
from urllib.parse import parse_qs, urlsplit
|
from urllib.parse import parse_qs, urlsplit
|
||||||
|
|
||||||
|
from ..process_control import (DEFAULT_GRACE_S, spawn_kwargs,
|
||||||
|
terminate_process_tree, write_control,
|
||||||
|
write_json_atomic)
|
||||||
|
from .analysis import (MAX_LOG_LINES, MAX_METRICS, describe_process, finite_float,
|
||||||
|
process_pid, summarize_convergence)
|
||||||
from .demo import DEMO_SCENARIOS, demo_snapshot
|
from .demo import DEMO_SCENARIOS, demo_snapshot
|
||||||
from .schema import config_schema
|
from .schema import config_schema
|
||||||
|
|
||||||
__all__ = ["TrainingJob", "TrainingUI", "run_training_ui", "build_command",
|
__all__ = ["TrainingJob", "TrainingUI", "run_training_ui", "build_command",
|
||||||
"summarize_convergence", "describe_process"]
|
"summarize_convergence", "describe_process", "discover_recovery"]
|
||||||
|
|
||||||
STATIC_DIR = Path(__file__).resolve().parent / "static"
|
STATIC_DIR = Path(__file__).resolve().parent / "static"
|
||||||
REPO_ROOT = Path(__file__).resolve().parents[3]
|
REPO_ROOT = Path(__file__).resolve().parents[3]
|
||||||
TRAIN_SCRIPT = REPO_ROOT / "scripts" / "train_hrl_high.py"
|
TRAIN_SCRIPT = REPO_ROOT / "scripts" / "train_hrl_high.py"
|
||||||
|
# 控制文件集中放在 output/ 下(已 gitignore),job 之间互不干扰
|
||||||
|
CONTROL_DIR = REPO_ROOT / "output" / "hrl" / "ui" / "control"
|
||||||
|
# 从发出停止请求到强杀之间留给训练脚本收尾的秒数
|
||||||
|
STOP_GRACE_S = DEFAULT_GRACE_S
|
||||||
STATIC_FILES = {
|
STATIC_FILES = {
|
||||||
"/": ("index.html", "text/html; charset=utf-8"),
|
"/": ("index.html", "text/html; charset=utf-8"),
|
||||||
"/index.html": ("index.html", "text/html; charset=utf-8"),
|
"/index.html": ("index.html", "text/html; charset=utf-8"),
|
||||||
@@ -49,10 +61,6 @@ STATIC_FILES = {
|
|||||||
"/style.css": ("style.css", "text/css; charset=utf-8"),
|
"/style.css": ("style.css", "text/css; charset=utf-8"),
|
||||||
}
|
}
|
||||||
|
|
||||||
MAX_LOG_LINES = 4000
|
|
||||||
MAX_METRICS = 5000
|
|
||||||
|
|
||||||
_NUM_RE = re.compile(r"^[+-]?(?:\d+\.?\d*|\.\d+)(?:[eE][+-]?\d+)?$")
|
|
||||||
_SB3_SECTIONS = ("rollout/", "time/", "train/", "system/")
|
_SB3_SECTIONS = ("rollout/", "time/", "train/", "system/")
|
||||||
# 路径/字符串参数只拒绝控制字符(含 \r\n\t 与 DEL),不做 shell 元字符过滤:
|
# 路径/字符串参数只拒绝控制字符(含 \r\n\t 与 DEL),不做 shell 元字符过滤:
|
||||||
# 命令走参数列表、不经过 shell,这里只是防日志与命令行被注入控制序列。
|
# 命令走参数列表、不经过 shell,这里只是防日志与命令行被注入控制序列。
|
||||||
@@ -61,23 +69,6 @@ _SB3_SECTIONS = ("rollout/", "time/", "train/", "system/")
|
|||||||
_SAFE_ARG_RE = re.compile(r"^[^\x00-\x1f\x7f]{1,512}$")
|
_SAFE_ARG_RE = re.compile(r"^[^\x00-\x1f\x7f]{1,512}$")
|
||||||
|
|
||||||
|
|
||||||
def _finite_float(value: Any) -> float | None:
|
|
||||||
if value is None or isinstance(value, bool):
|
|
||||||
return None
|
|
||||||
if isinstance(value, (int, float)):
|
|
||||||
value = float(value)
|
|
||||||
if value != value or value in (float("inf"), float("-inf")):
|
|
||||||
return None
|
|
||||||
return value
|
|
||||||
text = str(value).strip()
|
|
||||||
if not _NUM_RE.match(text):
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
return float(text)
|
|
||||||
except ValueError:
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _normalize_metric(key: str) -> str:
|
def _normalize_metric(key: str) -> str:
|
||||||
"""把 SB3 的 ``rollout/ep_rew_mean`` 与进度 JSON 的 ``ep_rew_mean``
|
"""把 SB3 的 ``rollout/ep_rew_mean`` 与进度 JSON 的 ``ep_rew_mean``
|
||||||
归一化到同一短名,便于前端绘制。"""
|
归一化到同一短名,便于前端绘制。"""
|
||||||
@@ -109,6 +100,10 @@ class TrainConfig:
|
|||||||
max_wall_s: float = 1200.0
|
max_wall_s: float = 1200.0
|
||||||
local_selector: str = "geometry"
|
local_selector: str = "geometry"
|
||||||
local_model: str | None = None
|
local_model: str | None = None
|
||||||
|
# 落盘/恢复:默认值与 scripts/train_hrl_high.py 保持一致
|
||||||
|
checkpoint_every: int = 20000
|
||||||
|
checkpoint_keep: int = 3
|
||||||
|
recovery_every: int = 2000
|
||||||
verbose: int = 1
|
verbose: int = 1
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
@@ -117,9 +112,11 @@ class TrainConfig:
|
|||||||
raise ValueError("请求体必须是 JSON 对象")
|
raise ValueError("请求体必须是 JSON 对象")
|
||||||
defaults = cls()
|
defaults = cls()
|
||||||
data: dict[str, Any] = {}
|
data: dict[str, Any] = {}
|
||||||
for field in cls.__dataclass_fields__:
|
# 用 dataclasses.fields 而不是 cls.__dataclass_fields__:后者是内部
|
||||||
if field in payload and payload[field] is not None:
|
# 实现细节,pylint 看不到它,也无法保证未来版本仍存在。
|
||||||
data[field] = payload[field]
|
for field in dataclasses.fields(cls):
|
||||||
|
if field.name in payload and payload[field.name] is not None:
|
||||||
|
data[field.name] = payload[field.name]
|
||||||
cfg = cls(**{**defaults.__dict__, **data})
|
cfg = cls(**{**defaults.__dict__, **data})
|
||||||
cfg.validate()
|
cfg.validate()
|
||||||
return cfg
|
return cfg
|
||||||
@@ -153,6 +150,12 @@ class TrainConfig:
|
|||||||
raise ValueError("max_wall_s 必须在 0..7200 秒")
|
raise ValueError("max_wall_s 必须在 0..7200 秒")
|
||||||
if int(self.eval_episodes) < 0:
|
if int(self.eval_episodes) < 0:
|
||||||
raise ValueError("eval_episodes 不能为负")
|
raise ValueError("eval_episodes 不能为负")
|
||||||
|
if not 0 <= int(self.checkpoint_every) <= 10_000_000:
|
||||||
|
raise ValueError("checkpoint_every 必须在 0..10,000,000")
|
||||||
|
if not 1 <= int(self.checkpoint_keep) <= 100:
|
||||||
|
raise ValueError("checkpoint_keep 必须在 1..100")
|
||||||
|
if not 0 <= int(self.recovery_every) <= 10_000_000:
|
||||||
|
raise ValueError("recovery_every 必须在 0..10,000,000")
|
||||||
for label, value in (("model", self.model), ("init", self.init),
|
for label, value in (("model", self.model), ("init", self.init),
|
||||||
("local_model", self.local_model)):
|
("local_model", self.local_model)):
|
||||||
if value is None:
|
if value is None:
|
||||||
@@ -207,6 +210,9 @@ def build_command(config: TrainConfig) -> list[str]:
|
|||||||
"--n-epochs", str(config.n_epochs),
|
"--n-epochs", str(config.n_epochs),
|
||||||
"--max-wall-s", str(config.max_wall_s),
|
"--max-wall-s", str(config.max_wall_s),
|
||||||
"--local-selector", config.local_selector,
|
"--local-selector", config.local_selector,
|
||||||
|
"--checkpoint-every", str(config.checkpoint_every),
|
||||||
|
"--checkpoint-keep", str(config.checkpoint_keep),
|
||||||
|
"--recovery-every", str(config.recovery_every),
|
||||||
"--progress-json",
|
"--progress-json",
|
||||||
"--verbose", str(config.verbose),
|
"--verbose", str(config.verbose),
|
||||||
]
|
]
|
||||||
@@ -223,25 +229,118 @@ def build_command(config: TrainConfig) -> list[str]:
|
|||||||
return cmd
|
return cmd
|
||||||
|
|
||||||
|
|
||||||
class TrainingJob:
|
def discover_recovery(run_dir: Path, model_path: Path) -> dict[str, Any]:
|
||||||
"""单个训练子进程的生命周期、日志和指标缓存。"""
|
"""找出"现在就能继续训练"的那份产物。
|
||||||
|
|
||||||
def __init__(self, config: TrainConfig):
|
优先级:
|
||||||
|
1. ``<run_dir>/resume.json``——训练脚本写的权威指针(校验目标文件真实存在);
|
||||||
|
2. 否则按 mtime 取最新:最终模型 / 滚动恢复点 / 历史 checkpoint / 异常兜底。
|
||||||
|
|
||||||
|
只认**已完整落盘**的文件:模型的 zip 与 json 都是原子替换写的,所以不会
|
||||||
|
出现"文件在、但打不开"的半截快照。
|
||||||
|
"""
|
||||||
|
run_dir = Path(run_dir)
|
||||||
|
model_path = Path(model_path)
|
||||||
|
hint_file = run_dir / "resume.json"
|
||||||
|
hint = _read_json_file(hint_file)
|
||||||
|
|
||||||
|
candidates: list[tuple[float, str, str]] = []
|
||||||
|
|
||||||
|
def _consider(path: Path, reason: str) -> None:
|
||||||
|
try:
|
||||||
|
stat = path.stat()
|
||||||
|
except OSError:
|
||||||
|
return
|
||||||
|
if stat.st_size > 0:
|
||||||
|
candidates.append((stat.st_mtime, str(path), reason))
|
||||||
|
|
||||||
|
_consider(model_path, "final_model")
|
||||||
|
ckpt_dir = run_dir / "checkpoints"
|
||||||
|
if ckpt_dir.is_dir():
|
||||||
|
_consider(ckpt_dir / f"{model_path.stem}_ckpt_latest.zip", "recovery")
|
||||||
|
for path in ckpt_dir.glob(f"{model_path.stem}_ckpt_[0-9]*.zip"):
|
||||||
|
if path.stem.rsplit("_", 1)[-1].isdigit():
|
||||||
|
_consider(path, "checkpoint")
|
||||||
|
_consider(run_dir / f"{model_path.stem}_emergency.zip", "emergency")
|
||||||
|
|
||||||
|
state: dict[str, Any] = {
|
||||||
|
"can_resume": False, "model": None, "timesteps": None,
|
||||||
|
"reason": "none", "updated_unix_s": None, "source": "none",
|
||||||
|
"hint_file": str(hint_file), "candidates": len(candidates),
|
||||||
|
}
|
||||||
|
if hint and hint.get("model") and Path(str(hint["model"])).is_file():
|
||||||
|
state.update({
|
||||||
|
"model": str(hint["model"]),
|
||||||
|
"timesteps": _int_or_none(hint.get("timesteps")),
|
||||||
|
"reason": str(hint.get("reason") or "resume_hint"),
|
||||||
|
"updated_unix_s": _float_or_none(hint.get("created_unix_s")),
|
||||||
|
"source": "resume.json",
|
||||||
|
})
|
||||||
|
elif candidates:
|
||||||
|
mtime, path, reason = max(candidates, key=lambda item: item[0])
|
||||||
|
state.update({
|
||||||
|
"model": path, "reason": reason,
|
||||||
|
"updated_unix_s": mtime, "source": "scan",
|
||||||
|
})
|
||||||
|
if state["model"]:
|
||||||
|
state["can_resume"] = Path(str(state["model"])).is_file()
|
||||||
|
return state
|
||||||
|
|
||||||
|
|
||||||
|
def _read_json_file(path: Path) -> dict[str, Any] | None:
|
||||||
|
"""读 JSON 文件;缺失或损坏返回 None(恢复指针不是关键路径,不能因此崩)。"""
|
||||||
|
try:
|
||||||
|
payload = json.loads(Path(path).read_text(encoding="utf-8"))
|
||||||
|
except (OSError, ValueError, UnicodeDecodeError):
|
||||||
|
return None
|
||||||
|
return payload if isinstance(payload, dict) else None
|
||||||
|
|
||||||
|
|
||||||
|
def _int_or_none(value: Any) -> int | None:
|
||||||
|
try:
|
||||||
|
return int(value)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _float_or_none(value: Any) -> float | None:
|
||||||
|
try:
|
||||||
|
return float(value)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
class TrainingJob:
|
||||||
|
"""单个训练子进程的生命周期、日志和指标缓存。
|
||||||
|
|
||||||
|
``control_dir`` 是控制文件的落地目录;默认与模型同目录,服务端则统一放到
|
||||||
|
``output/hrl/ui/control``(已被 gitignore)。
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, config: TrainConfig, control_dir: Path | None = None):
|
||||||
self.config = config
|
self.config = config
|
||||||
self.job_id = f"job-{int(time.time() * 1000)}"
|
self.job_id = f"job-{int(time.time() * 1000)}"
|
||||||
self.command = config.command()
|
self.control_path = (Path(control_dir) if control_dir is not None
|
||||||
|
else config.model_path.parent) / \
|
||||||
|
f"{self.job_id}.control.json"
|
||||||
|
# 控制文件是暂停/停止的唯一可靠通道:Windows 上没有 SIGSTOP,
|
||||||
|
# 且 os.kill(SIGTERM) 会退化成硬杀,信号无法承载"优雅收尾"语义。
|
||||||
|
self.command = config.command() + ["--control-file", str(self.control_path)]
|
||||||
self.created_at = time.time()
|
self.created_at = time.time()
|
||||||
self.started_at: float | None = None
|
self.started_at: float | None = None
|
||||||
self.finished_at: float | None = None
|
self.finished_at: float | None = None
|
||||||
self.returncode: int | None = None
|
self.returncode: int | None = None
|
||||||
self.status = "created" # created/running/exited/failed/stopped
|
# created/running/paused/stopping/stopped/exited/failed
|
||||||
|
self.status = "created"
|
||||||
self.error: str | None = None
|
self.error: str | None = None
|
||||||
|
self.forced_kill = False # 停止请求超时后是否动用了强杀
|
||||||
self.process: subprocess.Popen | None = None
|
self.process: subprocess.Popen | None = None
|
||||||
self.logs: deque[str] = deque(maxlen=MAX_LOG_LINES)
|
self.logs: deque[str] = deque(maxlen=MAX_LOG_LINES)
|
||||||
self.metrics: deque[dict[str, Any]] = deque(maxlen=MAX_METRICS)
|
self.metrics: deque[dict[str, Any]] = deque(maxlen=MAX_METRICS)
|
||||||
self.monitor: list[dict[str, Any]] = []
|
self.monitor: list[dict[str, Any]] = []
|
||||||
self._lock = threading.RLock()
|
self._lock = threading.RLock()
|
||||||
self._reader: threading.Thread | None = None
|
self._reader: threading.Thread | None = None
|
||||||
|
self._stopper: threading.Thread | None = None
|
||||||
self._table_sections: list[tuple[str, dict[str, Any]]] = []
|
self._table_sections: list[tuple[str, dict[str, Any]]] = []
|
||||||
self._table_steps: float | None = None # 当前表的 total_timesteps
|
self._table_steps: float | None = None # 当前表的 total_timesteps
|
||||||
self._pending_table: dict[str, Any] | None = None # 未见刻度的半张表
|
self._pending_table: dict[str, Any] | None = None # 未见刻度的半张表
|
||||||
@@ -254,10 +353,13 @@ class TrainingJob:
|
|||||||
with self._lock:
|
with self._lock:
|
||||||
if self.process is not None and self.process.poll() is None:
|
if self.process is not None and self.process.poll() is None:
|
||||||
raise RuntimeError("已有训练任务在运行")
|
raise RuntimeError("已有训练任务在运行")
|
||||||
|
# 先落一份"不暂停、不停止"的初始控制文件,训练脚本首步就能读到;
|
||||||
|
# 否则脚本要等到第一次轮询才区分"文件缺失"与"无指令"。
|
||||||
|
write_control(self.control_path, paused=False, stop=False)
|
||||||
env = os.environ.copy()
|
env = os.environ.copy()
|
||||||
env["PYTHONUNBUFFERED"] = "1"
|
env["PYTHONUNBUFFERED"] = "1"
|
||||||
env["HRL_UI_JOB_ID"] = self.job_id
|
env["HRL_UI_JOB_ID"] = self.job_id
|
||||||
self.process = subprocess.Popen(
|
self.process = subprocess.Popen( # pylint: disable=consider-using-with
|
||||||
self.command,
|
self.command,
|
||||||
cwd=str(REPO_ROOT),
|
cwd=str(REPO_ROOT),
|
||||||
stdout=subprocess.PIPE,
|
stdout=subprocess.PIPE,
|
||||||
@@ -265,7 +367,7 @@ class TrainingJob:
|
|||||||
text=True,
|
text=True,
|
||||||
bufsize=1,
|
bufsize=1,
|
||||||
env=env,
|
env=env,
|
||||||
start_new_session=True,
|
**spawn_kwargs(),
|
||||||
)
|
)
|
||||||
self.started_at = time.time()
|
self.started_at = time.time()
|
||||||
self.status = "running"
|
self.status = "running"
|
||||||
@@ -273,18 +375,101 @@ class TrainingJob:
|
|||||||
target=self._read_loop, name=f"{self.job_id}-reader", daemon=True)
|
target=self._read_loop, name=f"{self.job_id}-reader", daemon=True)
|
||||||
self._reader.start()
|
self._reader.start()
|
||||||
|
|
||||||
def stop(self) -> bool:
|
def pause(self) -> bool:
|
||||||
|
"""请求训练进程在下一个 step 边界挂起(跨平台:控制文件)。"""
|
||||||
with self._lock:
|
with self._lock:
|
||||||
process = self.process
|
process = self.process
|
||||||
if process is None or process.poll() is not None:
|
if process is None or process.poll() is not None:
|
||||||
return False
|
return False
|
||||||
try:
|
if self.status not in ("running", "paused"):
|
||||||
os.killpg(os.getpgid(process.pid), signal.SIGTERM)
|
return False
|
||||||
except (ProcessLookupError, PermissionError):
|
write_control(self.control_path, paused=True, stop=False)
|
||||||
process.terminate()
|
self.status = "paused"
|
||||||
self.status = "stopping"
|
|
||||||
return True
|
return True
|
||||||
|
|
||||||
|
def resume(self) -> bool:
|
||||||
|
"""解除挂起;训练从暂停的那个 step 继续。"""
|
||||||
|
with self._lock:
|
||||||
|
process = self.process
|
||||||
|
if process is None or process.poll() is not None:
|
||||||
|
return False
|
||||||
|
if self.status not in ("paused", "running"):
|
||||||
|
return False
|
||||||
|
write_control(self.control_path, paused=False, stop=False)
|
||||||
|
self.status = "running"
|
||||||
|
return True
|
||||||
|
|
||||||
|
def stop(self) -> bool:
|
||||||
|
"""请求优雅停止;超时后由看门狗强杀整棵进程树。
|
||||||
|
|
||||||
|
控制文件先于信号:Windows 上 SIGTERM 等价于 TerminateProcess,训练
|
||||||
|
脚本"保存模型 + 写元数据"的收尾分支不会执行;控制文件则两个平台都能
|
||||||
|
命中同一段协作式收尾。强杀放到独立线程,避免 HTTP 请求被 grace 等待卡住。
|
||||||
|
"""
|
||||||
|
with self._lock:
|
||||||
|
process = self.process
|
||||||
|
if process is None or process.poll() is not None:
|
||||||
|
return False
|
||||||
|
write_control(self.control_path, paused=False, stop=True)
|
||||||
|
self.status = "stopping"
|
||||||
|
if self._stopper is None or not self._stopper.is_alive():
|
||||||
|
self._stopper = threading.Thread(
|
||||||
|
target=self._stop_watchdog, args=(process,),
|
||||||
|
name=f"{self.job_id}-stopper", daemon=True)
|
||||||
|
self._stopper.start()
|
||||||
|
return True
|
||||||
|
|
||||||
|
def _stop_watchdog(self, process: subprocess.Popen) -> None:
|
||||||
|
"""等 grace 让协作式停止生效,超时强杀并记下恢复点。"""
|
||||||
|
forced = terminate_process_tree(process, grace_s=STOP_GRACE_S)
|
||||||
|
with self._lock:
|
||||||
|
self.forced_kill = forced
|
||||||
|
if forced and not self.error:
|
||||||
|
self.error = (f"训练进程未在 {STOP_GRACE_S:.0f}s 内响应停止请求,"
|
||||||
|
f"已强制结束进程树")
|
||||||
|
if forced:
|
||||||
|
self._write_forced_stop(process.pid)
|
||||||
|
|
||||||
|
def _write_forced_stop(self, pid: int) -> None:
|
||||||
|
"""强杀后落一份 `<模型目录>/forced_stop.json`,写明"从哪继续"。
|
||||||
|
|
||||||
|
强杀没有给训练脚本任何收尾机会,所以这里由前端补上审计与恢复指引:
|
||||||
|
磁盘上仍有原子落盘的滚动恢复点 / 历史 checkpoint,``recovery.model``
|
||||||
|
就是可直接 ``--resume`` 的目标。
|
||||||
|
"""
|
||||||
|
run_dir = self.config.model_path.parent
|
||||||
|
try:
|
||||||
|
recovery = discover_recovery(run_dir, self.config.model_path)
|
||||||
|
write_json_atomic(run_dir / "forced_stop.json", {
|
||||||
|
"event": "forced_stop",
|
||||||
|
"job_id": self.job_id,
|
||||||
|
"pid": int(pid),
|
||||||
|
"created_unix_s": time.time(),
|
||||||
|
"reason": "grace_timeout",
|
||||||
|
"grace_s": STOP_GRACE_S,
|
||||||
|
"recovery": recovery,
|
||||||
|
"note": ("训练进程未响应控制文件,已强杀整棵进程树;"
|
||||||
|
"把 recovery.model 作为 --resume 目标即可继续"
|
||||||
|
"(timesteps 为追加步数)"),
|
||||||
|
})
|
||||||
|
except OSError as exc: # 标记文件写不了也不能影响停止流程
|
||||||
|
with self._lock:
|
||||||
|
if not self.error:
|
||||||
|
self.error = f"强杀后写恢复标记失败:{exc}"
|
||||||
|
|
||||||
|
def wait_for_exit(self, timeout: float | None = None) -> bool:
|
||||||
|
"""等待停止流程收尾(优雅退出或强杀);供前端进程退出前调用。
|
||||||
|
|
||||||
|
训练子进程在独立进程组里,前端先退出并不会带走它。若不在这里等,
|
||||||
|
前端一退出,子进程会成为孤儿——虽然控制文件仍会让它自行收尾,但
|
||||||
|
日志与退出码就没人接了。
|
||||||
|
"""
|
||||||
|
thread = self._stopper
|
||||||
|
if thread is not None:
|
||||||
|
thread.join(timeout)
|
||||||
|
process = self.process
|
||||||
|
return process is None or process.poll() is not None
|
||||||
|
|
||||||
def _read_loop(self) -> None:
|
def _read_loop(self) -> None:
|
||||||
process = self.process
|
process = self.process
|
||||||
assert process is not None and process.stdout is not None
|
assert process is not None and process.stdout is not None
|
||||||
@@ -315,10 +500,38 @@ class TrainingJob:
|
|||||||
def _on_line(self, line: str) -> None:
|
def _on_line(self, line: str) -> None:
|
||||||
with self._lock:
|
with self._lock:
|
||||||
self.logs.append(line)
|
self.logs.append(line)
|
||||||
|
self._sync_control_events(line)
|
||||||
self._parse_progress_json(line)
|
self._parse_progress_json(line)
|
||||||
self._parse_sb3_table(line)
|
self._parse_sb3_table(line)
|
||||||
self._refresh_monitor()
|
self._refresh_monitor()
|
||||||
|
|
||||||
|
def _sync_control_events(self, line: str) -> None:
|
||||||
|
"""用训练脚本的回显校正前端状态。
|
||||||
|
|
||||||
|
正常情况下状态由前端自己设置;但外部直接改控制文件、或脚本比前端更早
|
||||||
|
感知到指令时,以脚本回显的 ``paused`` / ``resumed`` / ``stop_requested``
|
||||||
|
为准更真实。
|
||||||
|
"""
|
||||||
|
stripped = line.strip()
|
||||||
|
if not stripped.startswith("{") or '"event"' not in stripped:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
payload = json.loads(stripped)
|
||||||
|
except ValueError:
|
||||||
|
return
|
||||||
|
if not isinstance(payload, dict):
|
||||||
|
return
|
||||||
|
event = payload.get("event")
|
||||||
|
if event not in ("paused", "resumed", "stop_requested"):
|
||||||
|
return
|
||||||
|
with self._lock:
|
||||||
|
if event == "paused" and self.status == "running":
|
||||||
|
self.status = "paused"
|
||||||
|
elif event == "resumed" and self.status == "paused":
|
||||||
|
self.status = "running"
|
||||||
|
elif event == "stop_requested" and self.status in ("running", "paused"):
|
||||||
|
self.status = "stopping"
|
||||||
|
|
||||||
# ------------------------------------------------------------ parsers
|
# ------------------------------------------------------------ parsers
|
||||||
def _parse_progress_json(self, line: str) -> None:
|
def _parse_progress_json(self, line: str) -> None:
|
||||||
stripped = line.strip()
|
stripped = line.strip()
|
||||||
@@ -332,9 +545,9 @@ class TrainingJob:
|
|||||||
return
|
return
|
||||||
point = {_normalize_metric(str(key)): value
|
point = {_normalize_metric(str(key)): value
|
||||||
for key, value in payload.items() if key != "event"}
|
for key, value in payload.items() if key != "event"}
|
||||||
steps = _finite_float(point.get("timesteps"))
|
steps = finite_float(point.get("timesteps"))
|
||||||
if steps is None:
|
if steps is None:
|
||||||
steps = _finite_float(point.get("total_timesteps"))
|
steps = finite_float(point.get("total_timesteps"))
|
||||||
if steps is not None:
|
if steps is not None:
|
||||||
with self._lock:
|
with self._lock:
|
||||||
self._last_json_steps = steps
|
self._last_json_steps = steps
|
||||||
@@ -371,7 +584,7 @@ class TrainingJob:
|
|||||||
self._table_sections.append((raw_key, {}))
|
self._table_sections.append((raw_key, {}))
|
||||||
return
|
return
|
||||||
key = _normalize_metric(raw_key)
|
key = _normalize_metric(raw_key)
|
||||||
value = _finite_float(raw_value)
|
value = finite_float(raw_value)
|
||||||
if not key or value is None:
|
if not key or value is None:
|
||||||
return
|
return
|
||||||
with self._lock:
|
with self._lock:
|
||||||
@@ -401,7 +614,7 @@ class TrainingJob:
|
|||||||
for _name, items in sections:
|
for _name, items in sections:
|
||||||
table.update(items)
|
table.update(items)
|
||||||
if steps is None:
|
if steps is None:
|
||||||
steps = _finite_float(table.get("total_timesteps"))
|
steps = finite_float(table.get("total_timesteps"))
|
||||||
if steps is None:
|
if steps is None:
|
||||||
# 半张表:交给调用方决定是否已确认结束(终止时按末尾兜底提交)
|
# 半张表:交给调用方决定是否已确认结束(终止时按末尾兜底提交)
|
||||||
with self._lock:
|
with self._lock:
|
||||||
@@ -432,7 +645,7 @@ class TrainingJob:
|
|||||||
def _append_metric(self, point: dict[str, Any]) -> None:
|
def _append_metric(self, point: dict[str, Any]) -> None:
|
||||||
clean: dict[str, Any] = {}
|
clean: dict[str, Any] = {}
|
||||||
for key, value in point.items():
|
for key, value in point.items():
|
||||||
number = _finite_float(value)
|
number = finite_float(value)
|
||||||
clean[key] = number if number is not None else value
|
clean[key] = number if number is not None else value
|
||||||
# 补齐 timesteps:progress JSON 优先,SB3 表用 total_timesteps。
|
# 补齐 timesteps:progress JSON 优先,SB3 表用 total_timesteps。
|
||||||
if "timesteps" not in clean and "total_timesteps" in clean:
|
if "timesteps" not in clean and "total_timesteps" in clean:
|
||||||
@@ -471,7 +684,7 @@ class TrainingJob:
|
|||||||
row = {}
|
row = {}
|
||||||
valid = True
|
valid = True
|
||||||
for key, raw in zip(header, values):
|
for key, raw in zip(header, values):
|
||||||
number = _finite_float(raw)
|
number = finite_float(raw)
|
||||||
if number is None:
|
if number is None:
|
||||||
valid = False
|
valid = False
|
||||||
break
|
break
|
||||||
@@ -505,17 +718,17 @@ class TrainingJob:
|
|||||||
config = dict(self.config.__dict__)
|
config = dict(self.config.__dict__)
|
||||||
command = list(self.command)
|
command = list(self.command)
|
||||||
model_path = str(self.config.model_path)
|
model_path = str(self.config.model_path)
|
||||||
timesteps = int(self.config.timesteps)
|
|
||||||
payload = {
|
payload = {
|
||||||
"job_id": self.job_id,
|
"job_id": self.job_id,
|
||||||
"status": self.status,
|
"status": self.status,
|
||||||
"pid": _pid(self.process),
|
"pid": process_pid(self.process),
|
||||||
"created_at": self.created_at,
|
"created_at": self.created_at,
|
||||||
"started_at": self.started_at,
|
"started_at": self.started_at,
|
||||||
"finished_at": self.finished_at,
|
"finished_at": self.finished_at,
|
||||||
"elapsed_s": round(elapsed, 3),
|
"elapsed_s": round(elapsed, 3),
|
||||||
"returncode": self.returncode,
|
"returncode": self.returncode,
|
||||||
"error": self.error,
|
"error": self.error,
|
||||||
|
"forced_kill": self.forced_kill,
|
||||||
"command": command,
|
"command": command,
|
||||||
"model": model_path,
|
"model": model_path,
|
||||||
"config": config,
|
"config": config,
|
||||||
@@ -529,356 +742,12 @@ class TrainingJob:
|
|||||||
}
|
}
|
||||||
payload["convergence"] = summarize_convergence(metrics, monitor)
|
payload["convergence"] = summarize_convergence(metrics, monitor)
|
||||||
payload["process"] = describe_process(payload)
|
payload["process"] = describe_process(payload)
|
||||||
|
# 恢复指针在锁外做文件扫描:即使训练被强杀,这里也能给出可续训的产物
|
||||||
|
payload["recovery"] = discover_recovery(
|
||||||
|
self.config.model_path.parent, self.config.model_path)
|
||||||
return payload
|
return payload
|
||||||
|
|
||||||
|
|
||||||
def _series(metrics: list[dict[str, Any]], key: str) -> list[tuple[float, float]]:
|
|
||||||
points: list[tuple[float, float]] = []
|
|
||||||
for point in metrics:
|
|
||||||
value = _finite_float(point.get(key))
|
|
||||||
steps = _finite_float(point.get("timesteps")) or _finite_float(
|
|
||||||
point.get("total_timesteps"))
|
|
||||||
if value is None:
|
|
||||||
continue
|
|
||||||
points.append((steps if steps is not None else float(len(points) + 1), value))
|
|
||||||
return points
|
|
||||||
|
|
||||||
|
|
||||||
def _last_value(points: list[tuple[float, float]]) -> float | None:
|
|
||||||
return points[-1][1] if points else None
|
|
||||||
|
|
||||||
|
|
||||||
def _spread(points: list[tuple[float, float]], window: int = 10
|
|
||||||
) -> tuple[float, float] | None:
|
|
||||||
"""最近 window 个点的 (极差, 起点绝对值),用于判断"是否还在动"。"""
|
|
||||||
sample = points[-window:]
|
|
||||||
if len(sample) < 3:
|
|
||||||
return None
|
|
||||||
ys = [item[1] for item in sample]
|
|
||||||
return max(ys) - min(ys), abs(ys[0])
|
|
||||||
|
|
||||||
|
|
||||||
def _cv(points: list[tuple[float, float]], window: int = 10) -> float | None:
|
|
||||||
"""最近 window 个点的变异系数(标准差/|均值|),越小越平稳。"""
|
|
||||||
sample = [item[1] for item in points[-window:]]
|
|
||||||
if len(sample) < 3:
|
|
||||||
return None
|
|
||||||
mean = sum(sample) / len(sample)
|
|
||||||
if abs(mean) < 1e-12:
|
|
||||||
return None
|
|
||||||
var = sum((value - mean) ** 2 for value in sample) / len(sample)
|
|
||||||
return (var ** 0.5) / abs(mean)
|
|
||||||
|
|
||||||
|
|
||||||
def _slope(points: list[tuple[float, float]], window: int = 8) -> float | None:
|
|
||||||
sample = points[-window:]
|
|
||||||
if len(sample) < 3:
|
|
||||||
return None
|
|
||||||
xs = [item[0] for item in sample]
|
|
||||||
ys = [item[1] for item in sample]
|
|
||||||
n = len(xs)
|
|
||||||
mean_x = sum(xs) / n
|
|
||||||
mean_y = sum(ys) / n
|
|
||||||
denom = sum((x - mean_x) ** 2 for x in xs)
|
|
||||||
if abs(denom) < 1e-12:
|
|
||||||
return None
|
|
||||||
return sum((x - mean_x) * (y - mean_y) for x, y in zip(xs, ys)) / denom
|
|
||||||
|
|
||||||
|
|
||||||
def _rate_per_1k(reward_points: list[tuple[float, float]],
|
|
||||||
window: int = 8) -> float | None:
|
|
||||||
"""最近 window 个点的奖励增益换算成"每 1000 环境步",便于读趋势。"""
|
|
||||||
slope = _slope(reward_points, window)
|
|
||||||
if slope is None:
|
|
||||||
return None
|
|
||||||
return slope * 1000.0
|
|
||||||
|
|
||||||
|
|
||||||
def _fmt_num(value: float | None, digits: int = 4) -> str:
|
|
||||||
if value is None:
|
|
||||||
return "—"
|
|
||||||
if value != 0 and abs(value) < 1e-3:
|
|
||||||
return f"{value:.3e}"
|
|
||||||
return f"{value:.{digits}g}"
|
|
||||||
|
|
||||||
|
|
||||||
def _flatness(points: list[tuple[float, float]], window: int = 10
|
|
||||||
) -> tuple[float, float] | None:
|
|
||||||
"""窗口内"趋势位移 / 逐点噪声"比值:<1 表示看不出系统性变化。
|
|
||||||
|
|
||||||
判定收敛时只看斜率绝对值会把"很慢但持续的提升"误判为平台;这里把窗口
|
|
||||||
对半切,用两半均值之差当趋势位移,用相邻点平均绝对差当噪声尺度。返回
|
|
||||||
(比值, 噪声尺度),两者都小才是真的"又平又稳"。
|
|
||||||
"""
|
|
||||||
sample = points[-window:]
|
|
||||||
if len(sample) < 4:
|
|
||||||
return None
|
|
||||||
ys = [item[1] for item in sample]
|
|
||||||
half = len(ys) // 2
|
|
||||||
shift = abs(sum(ys[half:]) / len(ys[half:]) - sum(ys[:half]) / len(ys[:half]))
|
|
||||||
noise = sum(abs(b - a) for a, b in zip(ys, ys[1:])) / (len(ys) - 1)
|
|
||||||
if noise <= 0:
|
|
||||||
return (0.0, 0.0) if shift <= 0 else (float("inf"), 0.0)
|
|
||||||
return shift / noise, noise
|
|
||||||
|
|
||||||
|
|
||||||
def summarize_convergence(metrics: list[dict[str, Any]],
|
|
||||||
monitor: list[dict[str, Any]]) -> dict[str, Any]:
|
|
||||||
"""给前端返回可解释的收敛状态与各参数最近值。
|
|
||||||
|
|
||||||
这是**启发式判读**,不替代正式学习曲线分析:只依据最近窗口的斜率、
|
|
||||||
极差与变异系数给出"平坦/上升/未上升"的分类,并把同一组数字交给前端
|
|
||||||
展示,便于人工复核。
|
|
||||||
"""
|
|
||||||
reward_points = _series(metrics, "ep_rew_mean")
|
|
||||||
if not reward_points and monitor:
|
|
||||||
reward_points = [(float(index), float(row.get("r", 0.0)))
|
|
||||||
for index, row in enumerate(monitor)
|
|
||||||
if _finite_float(row.get("r")) is not None]
|
|
||||||
kl_points = _series(metrics, "approx_kl")
|
|
||||||
ev_points = _series(metrics, "explained_variance")
|
|
||||||
vf_points = _series(metrics, "value_loss")
|
|
||||||
pg_points = _series(metrics, "policy_gradient_loss")
|
|
||||||
entropy_points = _series(metrics, "entropy_loss")
|
|
||||||
fps_points = _series(metrics, "fps")
|
|
||||||
|
|
||||||
reward_slope = _slope(reward_points)
|
|
||||||
reward_rate = _rate_per_1k(reward_points)
|
|
||||||
spread = _spread(reward_points)
|
|
||||||
cv = _cv(reward_points)
|
|
||||||
latest_kl = _last_value(kl_points)
|
|
||||||
latest_ev = _last_value(ev_points)
|
|
||||||
latest_vf = _last_value(vf_points)
|
|
||||||
latest_pg = _last_value(pg_points)
|
|
||||||
latest_entropy = _last_value(entropy_points)
|
|
||||||
latest_fps = _last_value(fps_points)
|
|
||||||
latest_timesteps = metrics[-1].get("timesteps") if metrics else None
|
|
||||||
if latest_timesteps is None and metrics:
|
|
||||||
latest_timesteps = metrics[-1].get("total_timesteps")
|
|
||||||
latest_reward = _last_value(reward_points)
|
|
||||||
latest_len = _last_value(_series(metrics, "ep_len_mean"))
|
|
||||||
latest_updates = _last_value(_series(metrics, "n_updates"))
|
|
||||||
window = min(10, max(3, len(reward_points)))
|
|
||||||
reward_window = [round(value, 6) for _, value in reward_points[-window:]]
|
|
||||||
best_reward = max((value for _, value in reward_points), default=None)
|
|
||||||
|
|
||||||
notes: list[str] = []
|
|
||||||
if not metrics and not monitor:
|
|
||||||
status, tone = "等待首个 rollout", "gray"
|
|
||||||
notes.append("训练进程启动后先输出环境初始化与首个 rollout 的进度行。")
|
|
||||||
elif not metrics:
|
|
||||||
status, tone = "已收到回合奖励,等待进度行", "warn"
|
|
||||||
notes.append("Monitor CSV 已有回合记录,但尚未收到 --progress-json 进度行;"
|
|
||||||
"首个 rollout 结束后会出现完整指标。")
|
|
||||||
elif len(metrics) < 3:
|
|
||||||
status, tone = "预热阶段", "warn"
|
|
||||||
notes.append("已收到初始指标,需要更多 rollout 才能判断趋势。")
|
|
||||||
else:
|
|
||||||
tone, status = "ok", "收敛趋势良好"
|
|
||||||
# 平台判据(三条同时成立):对半窗口看不出位移、窗口内波动小、
|
|
||||||
# 逐点噪声本身也小。慢速但持续的提升会因第一条被排除。
|
|
||||||
flat = _flatness(reward_points)
|
|
||||||
plateau = False
|
|
||||||
if flat is not None and cv is not None and latest_reward is not None:
|
|
||||||
flat_ratio, flat_noise = flat
|
|
||||||
scale = max(abs(latest_reward), 1e-9)
|
|
||||||
plateau = (flat_ratio <= 1.0 and cv <= 0.15
|
|
||||||
and flat_noise <= 0.02 * scale
|
|
||||||
and (spread is None or spread[1] < 1e-12
|
|
||||||
or spread[0] <= 0.15 * spread[1]))
|
|
||||||
if latest_kl is not None and latest_kl > 0.05:
|
|
||||||
status, tone = "KL 偏大,策略更新可能过猛", "bad"
|
|
||||||
notes.append(f"approx_kl={latest_kl:.4f} > 0.05")
|
|
||||||
elif latest_ev is not None and latest_ev < 0.0:
|
|
||||||
status, tone = "价值函数尚未拟合", "warn"
|
|
||||||
notes.append(f"explained_variance={latest_ev:.3f} < 0")
|
|
||||||
elif reward_slope is not None and reward_slope < 0:
|
|
||||||
status, tone = "奖励未形成上升趋势", "warn"
|
|
||||||
notes.append(f"最近 {window} 点奖励斜率为负({_fmt_num(reward_slope)}/step)")
|
|
||||||
elif plateau:
|
|
||||||
status, tone = "已趋于平稳(疑似收敛)", "ok"
|
|
||||||
notes.append("窗口内对半位移小于逐点噪声,且波动很小"
|
|
||||||
f"(变异系数 {_fmt_num(cv, 3)})")
|
|
||||||
else:
|
|
||||||
notes.append(f"最近 {window} 点奖励斜率为正({_fmt_num(reward_slope)}/step)")
|
|
||||||
if latest_kl is not None and latest_kl <= 0.02:
|
|
||||||
notes.append(f"approx_kl={latest_kl:.4f} 较稳定")
|
|
||||||
if latest_ev is not None:
|
|
||||||
notes.append(f"explained_variance={latest_ev:.3f}")
|
|
||||||
if cv is not None:
|
|
||||||
notes.append(f"奖励变异系数={_fmt_num(cv, 3)}")
|
|
||||||
if latest_entropy is not None and abs(latest_entropy) < 1e-3:
|
|
||||||
notes.append("策略熵接近 0,探索可能过早停止")
|
|
||||||
return {
|
|
||||||
"status": status,
|
|
||||||
"tone": tone,
|
|
||||||
"notes": notes,
|
|
||||||
"window": window,
|
|
||||||
"latest_timesteps": latest_timesteps,
|
|
||||||
"reward_slope": reward_slope,
|
|
||||||
"reward_rate_per_1k": reward_rate,
|
|
||||||
"reward_cv": cv,
|
|
||||||
"reward_spread": spread[0] if spread else None,
|
|
||||||
"reward_window": reward_window,
|
|
||||||
"best_reward": best_reward,
|
|
||||||
"latest_reward": latest_reward,
|
|
||||||
"latest_ep_len": latest_len,
|
|
||||||
"latest_updates": latest_updates,
|
|
||||||
"latest_kl": latest_kl,
|
|
||||||
"latest_explained_variance": latest_ev,
|
|
||||||
"latest_value_loss": latest_vf,
|
|
||||||
"latest_policy_loss": latest_pg,
|
|
||||||
"latest_entropy_loss": latest_entropy,
|
|
||||||
"latest_fps": latest_fps,
|
|
||||||
"reward_points": len(reward_points),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
# 快照状态 -> 界面展示用的进程语义(阶段名称、色调、是否终态)
|
|
||||||
_STATUS_VIEW = {
|
|
||||||
"created": ("已创建", "gray", False, "命令已构造,等待 spawn"),
|
|
||||||
"running": ("运行中", "ok", False, "子进程持有控制台输出与独立进程组"),
|
|
||||||
"stopping": ("停止中", "warn", False, "已发送 SIGTERM,等待子进程退出"),
|
|
||||||
"stopped": ("已手动停止", "warn", True, "收到停止信号后退出;脚本会在 step 边界保存模型与元数据"),
|
|
||||||
"exited": ("正常结束", "ok", True, "退出码 0:训练与评估流程走完"),
|
|
||||||
"failed": ("异常结束", "bad", True, "非零退出码或启动失败,见日志尾部"),
|
|
||||||
}
|
|
||||||
|
|
||||||
_PHASES = [
|
|
||||||
("created", "构造命令"),
|
|
||||||
("spawn", "启动子进程"),
|
|
||||||
("rollout", "采集 rollout"),
|
|
||||||
("update", "PPO 更新"),
|
|
||||||
("evaluate", "评估 / 保存"),
|
|
||||||
("done", "收敛归档"),
|
|
||||||
]
|
|
||||||
|
|
||||||
# 状态 -> 当前所处的阶段节点;节点之前为已完成,之后为未开始。
|
|
||||||
_STATUS_PHASE = {
|
|
||||||
"created": "created",
|
|
||||||
"running": "update",
|
|
||||||
"stopping": "update",
|
|
||||||
"stopped": "done",
|
|
||||||
"exited": "done",
|
|
||||||
"failed": "done",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _pid(process: subprocess.Popen | None) -> int | None:
|
|
||||||
"""已退出进程的 pid 访问会抛 OSError;这里统一吞掉,只用于展示。"""
|
|
||||||
if process is None:
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
return int(process.pid)
|
|
||||||
except (OSError, ValueError):
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def describe_process(snapshot: dict[str, Any]) -> dict[str, Any]:
|
|
||||||
"""把 snapshot 派生为界面直接可渲染的进程对象。
|
|
||||||
|
|
||||||
包含:状态语义词、进度(步/百分比)、墙钟与剩余时间估计、最近吞吐、
|
|
||||||
进程时间线节点。数字都取自 snapshot,前端不再重复累加,避免两个刷新
|
|
||||||
频率不同的计数器互相打架。
|
|
||||||
"""
|
|
||||||
status = str(snapshot.get("status") or "created")
|
|
||||||
label, tone, terminal, hint = _STATUS_VIEW.get(
|
|
||||||
status, (status, "gray", False, ""))
|
|
||||||
config = snapshot.get("config") or {}
|
|
||||||
order = [key for key, _ in _PHASES]
|
|
||||||
current = _STATUS_PHASE.get(status, "created")
|
|
||||||
timeline = []
|
|
||||||
for key, title in _PHASES:
|
|
||||||
index = order.index(key)
|
|
||||||
if index < order.index(current):
|
|
||||||
state = "done"
|
|
||||||
elif index == order.index(current):
|
|
||||||
state = "active"
|
|
||||||
else:
|
|
||||||
state = "pending"
|
|
||||||
timeline.append({"key": key, "label": title, "state": state})
|
|
||||||
|
|
||||||
metrics = snapshot.get("metrics") or []
|
|
||||||
latest: dict[str, Any] = metrics[-1] if metrics else {}
|
|
||||||
convergence = snapshot.get("convergence") or {}
|
|
||||||
total_steps = _finite_float(config.get("timesteps")) or 0.0
|
|
||||||
# SB3 表格行可能没有 steps 字段,因此进度取"所有进度点的最大步数",
|
|
||||||
# 而不是只看最后一个点——否则末行恰好缺字段时进度会回退到 0。
|
|
||||||
done_steps = 0.0
|
|
||||||
for point in metrics:
|
|
||||||
value = (_finite_float(point.get("timesteps"))
|
|
||||||
or _finite_float(point.get("total_timesteps")))
|
|
||||||
if value is not None and value > done_steps:
|
|
||||||
done_steps = value
|
|
||||||
if done_steps == 0.0:
|
|
||||||
done_steps = _finite_float(convergence.get("latest_timesteps")) or 0.0
|
|
||||||
# init/resume(课程迁移)时 num_timesteps 从上一阶段继续,进度必须按本次
|
|
||||||
# 增量算;start_timesteps 由训练脚本的进度 JSON 提供,缺省按 0 处理。
|
|
||||||
start_steps = 0.0
|
|
||||||
for point in metrics:
|
|
||||||
value = _finite_float(point.get("start_timesteps"))
|
|
||||||
if value is not None:
|
|
||||||
start_steps = value
|
|
||||||
break
|
|
||||||
trained_steps = max(0.0, done_steps - start_steps)
|
|
||||||
percent = (100.0 * trained_steps / total_steps) if total_steps > 0 else None
|
|
||||||
if percent is not None:
|
|
||||||
percent = max(0.0, min(100.0, percent))
|
|
||||||
elapsed = float(snapshot.get("elapsed_s") or 0.0)
|
|
||||||
rate = _latest_rate(metrics)
|
|
||||||
remaining: float | None = None
|
|
||||||
eta_ts: float | None = None
|
|
||||||
if rate and rate > 0 and total_steps > 0 and trained_steps < total_steps:
|
|
||||||
remaining = (total_steps - trained_steps) / rate
|
|
||||||
eta_ts = float(snapshot.get("finished_at") or time.time()) + remaining
|
|
||||||
if not terminal and remaining is None:
|
|
||||||
cap = _finite_float(config.get("max_wall_s")) or 0.0
|
|
||||||
if cap > 0:
|
|
||||||
remaining = max(0.0, cap - elapsed)
|
|
||||||
return {
|
|
||||||
"label": label,
|
|
||||||
"tone": tone,
|
|
||||||
"terminal": terminal,
|
|
||||||
"hint": hint,
|
|
||||||
"phase": current,
|
|
||||||
"timeline": timeline,
|
|
||||||
"percent": percent,
|
|
||||||
"done_steps": done_steps,
|
|
||||||
"start_steps": start_steps,
|
|
||||||
"trained_steps": trained_steps,
|
|
||||||
"total_steps": total_steps,
|
|
||||||
"elapsed_s": round(elapsed, 3),
|
|
||||||
"rate_per_s": rate,
|
|
||||||
"remaining_s": remaining,
|
|
||||||
"eta_epoch_s": eta_ts,
|
|
||||||
"log_total": snapshot.get("log_total"),
|
|
||||||
"log_capacity": MAX_LOG_LINES,
|
|
||||||
"metric_total": snapshot.get("metric_total"),
|
|
||||||
"latest": latest,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _latest_rate(metrics: list[dict[str, Any]], window: int = 6) -> float | None:
|
|
||||||
"""最近若干个**带 fps 的**进度点的平均吞吐(步/秒),用于剩余时间估计。
|
|
||||||
|
|
||||||
progress JSON 行不含 fps,只有 SB3 表格行有;若只看最后 window 个点,
|
|
||||||
恰好落在若干 JSON 行之间时会得到 None,剩余时间就会退化成墙钟上限的
|
|
||||||
粗糙估计。这里向前扫描直到拿到至少 1 个 fps(最多看 3*window 个点)。
|
|
||||||
"""
|
|
||||||
if not metrics:
|
|
||||||
return None
|
|
||||||
usable: list[float] = []
|
|
||||||
for point in reversed(metrics[-window * 3:]):
|
|
||||||
value = _finite_float(point.get("fps"))
|
|
||||||
if value and value > 0:
|
|
||||||
usable.append(value)
|
|
||||||
if len(usable) >= window:
|
|
||||||
break
|
|
||||||
if not usable:
|
|
||||||
return None
|
|
||||||
return sum(usable) / len(usable)
|
|
||||||
|
|
||||||
|
|
||||||
class TrainingUI:
|
class TrainingUI:
|
||||||
"""训练前端管理器:持有唯一 TrainingJob 与 HTTP 服务器。"""
|
"""训练前端管理器:持有唯一 TrainingJob 与 HTTP 服务器。"""
|
||||||
|
|
||||||
@@ -901,9 +770,9 @@ class TrainingUI:
|
|||||||
config = TrainConfig.from_payload(payload)
|
config = TrainConfig.from_payload(payload)
|
||||||
with self._lock:
|
with self._lock:
|
||||||
if self.job is not None and self.job.status in (
|
if self.job is not None and self.job.status in (
|
||||||
"created", "running", "stopping"):
|
"created", "running", "paused", "stopping"):
|
||||||
raise RuntimeError("已有训练任务在运行,请先停止")
|
raise RuntimeError("已有训练任务在运行,请先停止")
|
||||||
self.job = TrainingJob(config)
|
self.job = TrainingJob(config, control_dir=CONTROL_DIR)
|
||||||
try:
|
try:
|
||||||
self.job.start()
|
self.job.start()
|
||||||
except Exception:
|
except Exception:
|
||||||
@@ -913,6 +782,14 @@ class TrainingUI:
|
|||||||
# 与 snapshot() 同形:HTTP 层直接把返回值当响应体,不再二次拆包
|
# 与 snapshot() 同形:HTTP 层直接把返回值当响应体,不再二次拆包
|
||||||
return {"job": self.job.snapshot(log_tail=1)}
|
return {"job": self.job.snapshot(log_tail=1)}
|
||||||
|
|
||||||
|
def pause(self) -> bool:
|
||||||
|
with self._lock:
|
||||||
|
return bool(self.job and self.job.pause())
|
||||||
|
|
||||||
|
def resume(self) -> bool:
|
||||||
|
with self._lock:
|
||||||
|
return bool(self.job and self.job.resume())
|
||||||
|
|
||||||
def stop(self) -> bool:
|
def stop(self) -> bool:
|
||||||
with self._lock:
|
with self._lock:
|
||||||
return bool(self.job and self.job.stop())
|
return bool(self.job and self.job.stop())
|
||||||
@@ -938,7 +815,10 @@ class TrainingUI:
|
|||||||
self._serving = False
|
self._serving = False
|
||||||
|
|
||||||
def shutdown(self) -> None:
|
def shutdown(self) -> None:
|
||||||
self.stop()
|
job = self.job
|
||||||
|
if job is not None and job.stop():
|
||||||
|
# 留出收尾落盘的时间,避免前端先退出把训练子进程留成孤儿
|
||||||
|
job.wait_for_exit(timeout=STOP_GRACE_S + 5.0)
|
||||||
if self._serving:
|
if self._serving:
|
||||||
self.httpd.shutdown()
|
self.httpd.shutdown()
|
||||||
self.httpd.server_close()
|
self.httpd.server_close()
|
||||||
@@ -948,8 +828,8 @@ class _UIHandler(BaseHTTPRequestHandler):
|
|||||||
protocol_version = "HTTP/1.1"
|
protocol_version = "HTTP/1.1"
|
||||||
server_version = "MCM26B-Training-UI/1.0"
|
server_version = "MCM26B-Training-UI/1.0"
|
||||||
|
|
||||||
def log_message(self, format, *args): # noqa: A002
|
def log_message(self, format, *args): # noqa: A002 # pylint: disable=redefined-builtin
|
||||||
pass
|
"""覆写 stdlib 签名以吞掉访问日志;参数名沿用 ``format`` 不能改。"""
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def ui(self) -> TrainingUI:
|
def ui(self) -> TrainingUI:
|
||||||
@@ -999,6 +879,10 @@ class _UIHandler(BaseHTTPRequestHandler):
|
|||||||
try:
|
try:
|
||||||
if parts.path == "/api/train/start":
|
if parts.path == "/api/train/start":
|
||||||
self._send_json(200, self.ui.start(payload))
|
self._send_json(200, self.ui.start(payload))
|
||||||
|
elif parts.path == "/api/train/pause":
|
||||||
|
self._send_json(200, {"paused": self.ui.pause()})
|
||||||
|
elif parts.path == "/api/train/resume":
|
||||||
|
self._send_json(200, {"resumed": self.ui.resume()})
|
||||||
elif parts.path == "/api/train/stop":
|
elif parts.path == "/api/train/stop":
|
||||||
self._send_json(200, {"stopped": self.ui.stop()})
|
self._send_json(200, {"stopped": self.ui.stop()})
|
||||||
else:
|
else:
|
||||||
|
|||||||
@@ -23,6 +23,8 @@ const state = {
|
|||||||
chartKey: "ep_rew_mean",
|
chartKey: "ep_rew_mean",
|
||||||
jobKey: null, // job_id 变化时清空本地缓存
|
jobKey: null, // job_id 变化时清空本地缓存
|
||||||
lastLogSignature: "",
|
lastLogSignature: "",
|
||||||
|
lastStatus: null, // 最近一次快照的状态:决定暂停按钮是"暂停"还是"继续"
|
||||||
|
lastJob: null, // 最近一次快照:恢复按钮需要读 job.recovery
|
||||||
};
|
};
|
||||||
|
|
||||||
// ---------------------------------------------------------------- 工具
|
// ---------------------------------------------------------------- 工具
|
||||||
@@ -285,9 +287,11 @@ function absorbMetrics(job) {
|
|||||||
// ---------------------------------------------------------------- 渲染
|
// ---------------------------------------------------------------- 渲染
|
||||||
function render(job) {
|
function render(job) {
|
||||||
if (!job) {
|
if (!job) {
|
||||||
|
state.lastJob = null;
|
||||||
renderIdle();
|
renderIdle();
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
state.lastJob = job;
|
||||||
if (job.job_id !== state.jobKey) {
|
if (job.job_id !== state.jobKey) {
|
||||||
state.jobKey = job.job_id;
|
state.jobKey = job.job_id;
|
||||||
state.metrics = [];
|
state.metrics = [];
|
||||||
@@ -302,6 +306,7 @@ function render(job) {
|
|||||||
renderLog(job);
|
renderLog(job);
|
||||||
renderEpisodes(job);
|
renderEpisodes(job);
|
||||||
renderBanner(job);
|
renderBanner(job);
|
||||||
|
renderRecovery(job);
|
||||||
renderJobLine(job);
|
renderJobLine(job);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -319,6 +324,10 @@ function renderIdle() {
|
|||||||
setText("stat-logs", "0");
|
setText("stat-logs", "0");
|
||||||
$("timeline").innerHTML = "";
|
$("timeline").innerHTML = "";
|
||||||
setText("command-preview", "");
|
setText("command-preview", "");
|
||||||
|
state.lastStatus = null;
|
||||||
|
const pauseBtn = $("btn-pause");
|
||||||
|
pauseBtn.disabled = true;
|
||||||
|
pauseBtn.textContent = "暂停训练";
|
||||||
chip("chip-converge", "判读", "等待数据", "gray");
|
chip("chip-converge", "判读", "等待数据", "gray");
|
||||||
$("converge-notes").innerHTML = "";
|
$("converge-notes").innerHTML = "";
|
||||||
$("metric-grid").innerHTML = '<p class="hint">尚未启动训练,暂无参数。</p>';
|
$("metric-grid").innerHTML = '<p class="hint">尚未启动训练,暂无参数。</p>';
|
||||||
@@ -370,8 +379,14 @@ function renderProcess(job) {
|
|||||||
|
|
||||||
renderTimeline(process);
|
renderTimeline(process);
|
||||||
setText("command-preview", job.command ? `$ ${job.command.join(" ")}` : "");
|
setText("command-preview", job.command ? `$ ${job.command.join(" ")}` : "");
|
||||||
setFormLocked(job.status === "running" || job.status === "created" || job.status === "stopping");
|
state.lastStatus = job.status;
|
||||||
$("btn-stop").disabled = !(job.status === "running" || job.status === "created");
|
setFormLocked(job.status === "running" || job.status === "created"
|
||||||
|
|| job.status === "paused" || job.status === "stopping");
|
||||||
|
$("btn-stop").disabled = !(job.status === "running" || job.status === "created"
|
||||||
|
|| job.status === "paused");
|
||||||
|
const pauseBtn = $("btn-pause");
|
||||||
|
pauseBtn.disabled = !(job.status === "running" || job.status === "paused");
|
||||||
|
pauseBtn.textContent = job.status === "paused" ? "继续训练" : "暂停训练";
|
||||||
}
|
}
|
||||||
|
|
||||||
function renderTimeline(process) {
|
function renderTimeline(process) {
|
||||||
@@ -427,8 +442,12 @@ function renderBanner(job) {
|
|||||||
}
|
}
|
||||||
if (job.status === "failed") {
|
if (job.status === "failed") {
|
||||||
showBanner("bad", `训练异常结束(退出码 <b>${job.returncode === null ? "?" : job.returncode}</b>):${escapeHtml(job.error || "见日志尾部")}。`);
|
showBanner("bad", `训练异常结束(退出码 <b>${job.returncode === null ? "?" : job.returncode}</b>):${escapeHtml(job.error || "见日志尾部")}。`);
|
||||||
|
} else if (job.status === "paused") {
|
||||||
|
showBanner("warn", "训练已暂停:进程与显存仍在,但停在 step 边界不再推进。点「继续训练」从暂停处恢复,或点「停止训练」保存当前模型后退出。");
|
||||||
} else if (job.status === "stopped") {
|
} else if (job.status === "stopped") {
|
||||||
showBanner("warn", "已按请求停止训练(SIGTERM 独立进程组),脚本会在 step 边界保存当前模型与元数据。");
|
showBanner("warn", job.forced_kill
|
||||||
|
? "停止请求超时,已强制结束进程树;最后一段模型可能未保存,请检查 checkpoint 目录。"
|
||||||
|
: "已按请求停止训练(控制文件协作式停止),脚本已在 step 边界保存当前模型与元数据。");
|
||||||
} else if (job.status === "exited") {
|
} else if (job.status === "exited") {
|
||||||
showBanner("ok", "训练与评估流程正常结束,模型与元数据已写入输出目录。");
|
showBanner("ok", "训练与评估流程正常结束,模型与元数据已写入输出目录。");
|
||||||
} else if (process.remaining_s !== null && process.remaining_s !== undefined
|
} else if (process.remaining_s !== null && process.remaining_s !== undefined
|
||||||
@@ -439,6 +458,26 @@ function renderBanner(job) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------- 恢复
|
||||||
|
function renderRecovery(job) {
|
||||||
|
const recovery = job.recovery || {};
|
||||||
|
const box = $("recovery-box");
|
||||||
|
if (!box) return;
|
||||||
|
if (!recovery.model || recovery.reason === "none") {
|
||||||
|
box.classList.add("hidden");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
box.classList.remove("hidden");
|
||||||
|
const steps = recovery.timesteps === null || recovery.timesteps === undefined
|
||||||
|
? "步数未知" : `${fmtInt(recovery.timesteps)} 步`;
|
||||||
|
const when = recovery.updated_unix_s ? fmtClock(recovery.updated_unix_s) : "—";
|
||||||
|
setText("recovery-text",
|
||||||
|
`可恢复产物:${steps} · 来源 ${recovery.reason || "?"} · 更新 ${when} · ${recovery.model}`);
|
||||||
|
const button = $("btn-recover");
|
||||||
|
// 离线预览里的恢复信息是合成的,不允许真的去续训
|
||||||
|
button.disabled = !recovery.can_resume || Boolean(state.preview);
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------- 收敛
|
// ---------------------------------------------------------------- 收敛
|
||||||
function renderConvergence(job) {
|
function renderConvergence(job) {
|
||||||
const convergence = job.convergence || {};
|
const convergence = job.convergence || {};
|
||||||
@@ -679,6 +718,33 @@ function renderEpisodes(job) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------- 交互
|
// ---------------------------------------------------------------- 交互
|
||||||
|
async function submitPayload(payload) {
|
||||||
|
const err = $("form-error");
|
||||||
|
err.textContent = "";
|
||||||
|
if (state.saving) return false; // 训练中表单锁定,续训按钮同样不许重复提交
|
||||||
|
if (payload.resume && !payload.model) {
|
||||||
|
err.textContent = "续训需要填写模型输出路径";
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
const res = await fetch("/api/train/start", {
|
||||||
|
method: "POST", headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify(payload),
|
||||||
|
});
|
||||||
|
const body = await res.json().catch(() => ({}));
|
||||||
|
if (!res.ok) {
|
||||||
|
err.textContent = body.error || `启动失败(HTTP ${res.status})`;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
state.preview = false;
|
||||||
|
state.jobKey = null;
|
||||||
|
state.metrics = [];
|
||||||
|
state.metricCache = [];
|
||||||
|
state.metricNext = 0;
|
||||||
|
state.metricTotal = 0;
|
||||||
|
await poll();
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
$("config-form").addEventListener("submit", async (event) => {
|
$("config-form").addEventListener("submit", async (event) => {
|
||||||
event.preventDefault();
|
event.preventDefault();
|
||||||
if (state.saving) return;
|
if (state.saving) return;
|
||||||
@@ -691,24 +757,43 @@ $("config-form").addEventListener("submit", async (event) => {
|
|||||||
for (const [key, value] of Object.entries(values)) {
|
for (const [key, value] of Object.entries(values)) {
|
||||||
if (value !== null) payload[key] = value;
|
if (value !== null) payload[key] = value;
|
||||||
}
|
}
|
||||||
if (payload.resume && !payload.model) { err.textContent = "续训需要填写模型输出路径"; return; }
|
await submitPayload(payload);
|
||||||
const res = await fetch("/api/train/start", {
|
});
|
||||||
method: "POST", headers: { "Content-Type": "application/json" },
|
|
||||||
body: JSON.stringify(payload),
|
$("btn-recover").addEventListener("click", async () => {
|
||||||
|
const recovery = (state.lastJob && state.lastJob.recovery) || {};
|
||||||
|
if (!recovery.can_resume || !recovery.model) return;
|
||||||
|
if (!confirm(`用最近的可恢复产物续训?\n${recovery.model}\n`
|
||||||
|
+ "当前表单里的 timesteps 会作为**追加**步数。")) return;
|
||||||
|
// 同步回表单,用户随后可微调步数再手动点「开始训练」
|
||||||
|
const modelField = $("cfg-model");
|
||||||
|
if (modelField) modelField.value = recovery.model;
|
||||||
|
const resumeField = $("cfg-resume");
|
||||||
|
if (resumeField) resumeField.checked = true;
|
||||||
|
const values = collectValues();
|
||||||
|
values.model = recovery.model;
|
||||||
|
values.resume = true;
|
||||||
|
values.force = false; // --resume 与 --force 互斥
|
||||||
|
await submitPayload(values);
|
||||||
|
});
|
||||||
|
|
||||||
|
$("btn-pause").addEventListener("click", async () => {
|
||||||
|
const resuming = state.lastStatus === "paused";
|
||||||
|
const res = await fetch(resuming ? "/api/train/resume" : "/api/train/pause", {
|
||||||
|
method: "POST", headers: { "Content-Type": "application/json" }, body: "{}",
|
||||||
});
|
});
|
||||||
const body = await res.json().catch(() => ({}));
|
const body = await res.json().catch(() => ({}));
|
||||||
if (!res.ok) { err.textContent = body.error || `启动失败(HTTP ${res.status})`; return; }
|
if (!res.ok) {
|
||||||
state.preview = false;
|
showBanner("bad", `${resuming ? "继续" : "暂停"}失败:${escapeHtml(body.error || res.status)}`);
|
||||||
state.jobKey = null;
|
} else if (!resuming && body.paused === false) {
|
||||||
state.metrics = [];
|
showBanner("warn", "当前没有可暂停的训练进程(可能已结束)。");
|
||||||
state.metricCache = [];
|
}
|
||||||
state.metricNext = 0;
|
|
||||||
state.metricTotal = 0;
|
|
||||||
await poll();
|
await poll();
|
||||||
});
|
});
|
||||||
|
|
||||||
$("btn-stop").addEventListener("click", async () => {
|
$("btn-stop").addEventListener("click", async () => {
|
||||||
if (!confirm("确认停止当前训练?\n将向独立进程组发送 SIGTERM,脚本会先保存当前模型与元数据再退出。")) return;
|
if (!confirm("确认停止当前训练?\n将通过控制文件请求优雅停止,脚本先保存当前模型与元数据再退出;"
|
||||||
|
+ "若超时未响应则强制结束进程树。")) return;
|
||||||
const res = await fetch("/api/train/stop", {
|
const res = await fetch("/api/train/stop", {
|
||||||
method: "POST", headers: { "Content-Type": "application/json" }, body: "{}",
|
method: "POST", headers: { "Content-Type": "application/json" }, body: "{}",
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -30,6 +30,7 @@
|
|||||||
</select>
|
</select>
|
||||||
</label>
|
</label>
|
||||||
<button id="btn-preview" type="button" title="渲染示例快照,不启动任何训练进程">离线预览</button>
|
<button id="btn-preview" type="button" title="渲染示例快照,不启动任何训练进程">离线预览</button>
|
||||||
|
<button id="btn-pause" type="button" disabled>暂停训练</button>
|
||||||
<button id="btn-stop" class="danger" type="button" disabled>停止训练</button>
|
<button id="btn-stop" class="danger" type="button" disabled>停止训练</button>
|
||||||
</div>
|
</div>
|
||||||
</header>
|
</header>
|
||||||
@@ -37,6 +38,11 @@
|
|||||||
<main>
|
<main>
|
||||||
<div id="banner" class="banner hidden"></div>
|
<div id="banner" class="banner hidden"></div>
|
||||||
|
|
||||||
|
<div id="recovery-box" class="recovery hidden">
|
||||||
|
<span class="hint" id="recovery-text">—</span>
|
||||||
|
<button id="btn-recover" type="button" title="把最近一次完整落盘的产物作为 --resume 目标继续训练">用最近检查点续训</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
<!-- =============================================== 1. 执行进度与进程 -->
|
<!-- =============================================== 1. 执行进度与进程 -->
|
||||||
<section class="panel" id="panel-process">
|
<section class="panel" id="panel-process">
|
||||||
<div class="panel-head">
|
<div class="panel-head">
|
||||||
|
|||||||
@@ -64,6 +64,15 @@ button.danger:hover:enabled { background: rgba(248,113,113,.12); }
|
|||||||
.banner.warn { border-color: var(--warn); border-left-color: var(--warn); background: rgba(245,158,11,.08); }
|
.banner.warn { border-color: var(--warn); border-left-color: var(--warn); background: rgba(245,158,11,.08); }
|
||||||
.banner b { font-family: var(--mono); }
|
.banner b { font-family: var(--mono); }
|
||||||
|
|
||||||
|
/* 恢复条:显示最近一份完整落盘的产物,并提供一键续训 */
|
||||||
|
.recovery {
|
||||||
|
display: flex; align-items: center; justify-content: space-between; gap: 12px;
|
||||||
|
border: 1px solid var(--cyan, #22d3ee); border-left: 4px solid var(--cyan, #22d3ee);
|
||||||
|
background: rgba(34,211,238,.07); border-radius: 8px;
|
||||||
|
padding: 8px 12px; margin: 10px 0 0; font-size: 13px;
|
||||||
|
}
|
||||||
|
.recovery .hint { overflow-wrap: anywhere; }
|
||||||
|
|
||||||
/* ------------------------------------------------------------ 面板 */
|
/* ------------------------------------------------------------ 面板 */
|
||||||
.panel {
|
.panel {
|
||||||
border: 1px solid var(--line); border-radius: 12px; background: var(--panel);
|
border: 1px solid var(--line); border-radius: 12px; background: var(--panel);
|
||||||
|
|||||||
@@ -62,7 +62,7 @@ class BayesLoopAcceptanceTests(unittest.TestCase):
|
|||||||
|
|
||||||
def test_sixteen_sources_stop_by_count_bound(self):
|
def test_sixteen_sources_stop_by_count_bound(self):
|
||||||
sources = random_omni_scene(seed=6, count=16, config=CONFIG)
|
sources = random_omni_scene(seed=6, count=16, config=CONFIG)
|
||||||
summary, report, simulator = _run_scene(sources)
|
summary, report, _ = _run_scene(sources)
|
||||||
truth = sorted(s.channel for s in sources)
|
truth = sorted(s.channel for s in sources)
|
||||||
self.assertTrue(summary["completed"])
|
self.assertTrue(summary["completed"])
|
||||||
self.assertEqual(summary["stop_reason"], "sixteen_cleared")
|
self.assertEqual(summary["stop_reason"], "sixteen_cleared")
|
||||||
@@ -72,7 +72,7 @@ class BayesLoopAcceptanceTests(unittest.TestCase):
|
|||||||
|
|
||||||
def test_empty_arena_all_absent(self):
|
def test_empty_arena_all_absent(self):
|
||||||
"""0 源:每个频道都必须由 PF 排除盘证明缺席后正常退出。"""
|
"""0 源:每个频道都必须由 PF 排除盘证明缺席后正常退出。"""
|
||||||
summary, report, simulator = _run_scene([], config=NO_PERSIST)
|
summary, report, _ = _run_scene([], config=NO_PERSIST)
|
||||||
self.assertTrue(summary["completed"])
|
self.assertTrue(summary["completed"])
|
||||||
self.assertEqual(summary["stop_reason"], "all_resolved")
|
self.assertEqual(summary["stop_reason"], "all_resolved")
|
||||||
self.assertEqual(sorted(summary["absent_channels"]),
|
self.assertEqual(sorted(summary["absent_channels"]),
|
||||||
@@ -89,7 +89,7 @@ class BayesLoopAcceptanceTests(unittest.TestCase):
|
|||||||
sources = [Source(c + 1, 1800.0 * math.cos(math.pi / 6 + 2 * math.pi * c / 4),
|
sources = [Source(c + 1, 1800.0 * math.cos(math.pi / 6 + 2 * math.pi * c / 4),
|
||||||
1800.0 * math.sin(math.pi / 6 + 2 * math.pi * c / 4),
|
1800.0 * math.sin(math.pi / 6 + 2 * math.pi * c / 4),
|
||||||
1000.0) for c in range(4)]
|
1000.0) for c in range(4)]
|
||||||
summary, report, simulator = _run_scene(sources, config=NO_PERSIST)
|
summary, report, _ = _run_scene(sources, config=NO_PERSIST)
|
||||||
truth = sorted(s.channel for s in sources)
|
truth = sorted(s.channel for s in sources)
|
||||||
self.assertTrue(summary["completed"])
|
self.assertTrue(summary["completed"])
|
||||||
self.assertEqual(sorted(summary["cleared_channels"]), truth)
|
self.assertEqual(sorted(summary["cleared_channels"]), truth)
|
||||||
@@ -112,7 +112,7 @@ class BayesLoopAcceptanceTests(unittest.TestCase):
|
|||||||
def test_transport_fault_retries_keep_session_clean(self):
|
def test_transport_fault_retries_keep_session_clean(self):
|
||||||
sources = random_omni_scene(seed=5, count=10, config=CONFIG)
|
sources = random_omni_scene(seed=5, count=10, config=CONFIG)
|
||||||
script = [("drop_before", 1)] # 首个请求丢弃一次:幂等重试路径
|
script = [("drop_before", 1)] # 首个请求丢弃一次:幂等重试路径
|
||||||
summary, report, simulator = _run_scene(sources, fault_script=script)
|
summary, report, _ = _run_scene(sources, fault_script=script)
|
||||||
truth = sorted(s.channel for s in sources)
|
truth = sorted(s.channel for s in sources)
|
||||||
self.assertTrue(summary["completed"])
|
self.assertTrue(summary["completed"])
|
||||||
self.assertEqual(sorted(summary["cleared_channels"]), truth)
|
self.assertEqual(sorted(summary["cleared_channels"]), truth)
|
||||||
@@ -141,9 +141,9 @@ class BayesLoopAcceptanceTests(unittest.TestCase):
|
|||||||
|
|
||||||
def test_cli_strategy_registry(self):
|
def test_cli_strategy_registry(self):
|
||||||
"""CLI --strategy 注册表:bayes/seven 用七站,grid 用 49 点网格。"""
|
"""CLI --strategy 注册表:bayes/seven 用七站,grid 用 49 点网格。"""
|
||||||
import mcm26b.cli as cli
|
from mcm26b import cli
|
||||||
from mcm26b.coverage import coverage_points
|
from mcm26b.coverage import coverage_points
|
||||||
from mcm26b.planner.seven import SevenBatch, seven_points
|
from mcm26b.planner.seven import seven_points
|
||||||
self.assertIs(cli.Q3_STRATEGIES["bayes"], BayesLoop)
|
self.assertIs(cli.Q3_STRATEGIES["bayes"], BayesLoop)
|
||||||
for name, loop_cls in cli.Q3_STRATEGIES.items():
|
for name, loop_cls in cli.Q3_STRATEGIES.items():
|
||||||
simulator = LocalSimulator([], robot_id="T001", config=CONFIG)
|
simulator = LocalSimulator([], robot_id="T001", config=CONFIG)
|
||||||
|
|||||||
@@ -132,7 +132,7 @@ class BeliefDirectionTests(TestCase):
|
|||||||
def test_near_shrinks_to_five_meters(self):
|
def test_near_shrinks_to_five_meters(self):
|
||||||
belief = BeliefState(CONFIG)
|
belief = BeliefState(CONFIG)
|
||||||
belief.update_measure(2, (100.0, 100.0), NEAR, None)
|
belief.update_measure(2, (100.0, 100.0), NEAR, None)
|
||||||
center, radius = belief.candidate_set(2).mec()
|
_, radius = belief.candidate_set(2).mec()
|
||||||
self.assertLessEqual(radius, 5.1)
|
self.assertLessEqual(radius, 5.1)
|
||||||
self.assertTrue(belief.candidate_set(2).contains((101.0, 101.0)))
|
self.assertTrue(belief.candidate_set(2).contains((101.0, 101.0)))
|
||||||
self.assertEqual(belief.channels[2].status, ChannelStatus.DETECTED)
|
self.assertEqual(belief.channels[2].status, ChannelStatus.DETECTED)
|
||||||
|
|||||||
@@ -16,7 +16,8 @@ class GeometryTests(unittest.TestCase):
|
|||||||
self.assertEqual(result['status'], 'polygon')
|
self.assertEqual(result['status'], 'polygon')
|
||||||
self.assertEqual(len(result['vertices']), 3)
|
self.assertEqual(len(result['vertices']), 3)
|
||||||
self.assertAlmostEqual(result['diameter_m'], 40, places=7)
|
self.assertAlmostEqual(result['diameter_m'], 40, places=7)
|
||||||
self.assertAlmostEqual(result['minimum_enclosing_circle']['radius_m'], 40/math.sqrt(3), places=7)
|
self.assertAlmostEqual(result['minimum_enclosing_circle']['radius_m'],
|
||||||
|
40/math.sqrt(3), places=7)
|
||||||
self.assertFalse(result['diameter_circle']['covers'])
|
self.assertFalse(result['diameter_circle']['covers'])
|
||||||
self.assertFalse(result['guaranteed_clearable_at_mec_center'])
|
self.assertFalse(result['guaranteed_clearable_at_mec_center'])
|
||||||
|
|
||||||
@@ -84,7 +85,7 @@ class GeometryTests(unittest.TestCase):
|
|||||||
c, r = smallest_enclosing_circle([(0,0),(1,0),(3,0)])
|
c, r = smallest_enclosing_circle([(0,0),(1,0),(3,0)])
|
||||||
self.assertAlmostEqual(r, 1.5)
|
self.assertAlmostEqual(r, 1.5)
|
||||||
|
|
||||||
def test_random_bounded_truth_containment_and_monotonicity(self):
|
def test_random_truth_containment_and_monotonic(self):
|
||||||
rng = random.Random(20260910)
|
rng = random.Random(20260910)
|
||||||
for _ in range(100):
|
for _ in range(100):
|
||||||
target = (rng.uniform(-300,300), rng.uniform(-300,300))
|
target = (rng.uniform(-300,300), rng.uniform(-300,300))
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ ROOT = Path(__file__).resolve().parents[1]
|
|||||||
|
|
||||||
|
|
||||||
class CertificateTests(unittest.TestCase):
|
class CertificateTests(unittest.TestCase):
|
||||||
def test_incremental_circle_against_exhaustive_reference(self):
|
def test_incremental_circle_matches_reference(self):
|
||||||
rng = random.Random(271828)
|
rng = random.Random(271828)
|
||||||
for _ in range(250):
|
for _ in range(250):
|
||||||
points = [(rng.uniform(-20, 20), rng.uniform(-20, 20))
|
points = [(rng.uniform(-20, 20), rng.uniform(-20, 20))
|
||||||
@@ -25,7 +25,7 @@ class CertificateTests(unittest.TestCase):
|
|||||||
self.assertLess(distance(center, expected_c), 1e-6)
|
self.assertLess(distance(center, expected_c), 1e-6)
|
||||||
self.assertTrue(all(distance(center, p) <= radius for p in points))
|
self.assertTrue(all(distance(center, p) <= radius for p in points))
|
||||||
|
|
||||||
def test_circle_large_cocircular_duplicate_and_collinear_sets(self):
|
def test_circle_cocircular_duplicate_and_collinear(self):
|
||||||
# More than Python's recursion limit, with analytically known answers.
|
# More than Python's recursion limit, with analytically known answers.
|
||||||
points = [(1e6+30*math.cos(i*math.tau/1500),
|
points = [(1e6+30*math.cos(i*math.tau/1500),
|
||||||
-2e6+30*math.sin(i*math.tau/1500)) for i in range(1500)]
|
-2e6+30*math.sin(i*math.tau/1500)) for i in range(1500)]
|
||||||
@@ -37,15 +37,16 @@ class CertificateTests(unittest.TestCase):
|
|||||||
self.assertEqual(c, (749.5, 1499))
|
self.assertEqual(c, (749.5, 1499))
|
||||||
self.assertEqual(smallest_enclosing_circle([(3, 4)]*10), ((3, 4), 0))
|
self.assertEqual(smallest_enclosing_circle([(3, 4)]*10), ((3, 4), 0))
|
||||||
|
|
||||||
def test_witness_and_clearance_margin_from_saved_observations(self):
|
def test_witness_and_clearance_margin_from_saved(self):
|
||||||
path = next((ROOT/'output/q1/practice').glob('*_observations.json'))
|
path = next((ROOT/'output/q1/practice').glob('*_observations.json'))
|
||||||
rows = json.loads(path.read_text(encoding='utf-8'))['measurements']
|
rows = json.loads(path.read_text(encoding='utf-8'))['measurements']
|
||||||
result = solve([Measurement(**m) for m in rows])
|
result = solve([Measurement(**m) for m in rows])
|
||||||
circle = result['diameter_circle']
|
circle = result['diameter_circle']
|
||||||
self.assertFalse(circle['covers'])
|
self.assertFalse(circle['covers'])
|
||||||
self.assertIn(circle['witness_vertex'], result['vertices'])
|
self.assertIn(circle['witness_vertex'], result['vertices'])
|
||||||
self.assertAlmostEqual(distance(circle['witness_vertex'], circle['center'])-circle['radius_m'],
|
self.assertAlmostEqual(
|
||||||
circle['max_excess_m'])
|
distance(circle['witness_vertex'], circle['center'])-circle['radius_m'],
|
||||||
|
circle['max_excess_m'])
|
||||||
for i in circle['outside_vertex_indices']:
|
for i in circle['outside_vertex_indices']:
|
||||||
self.assertGreater(distance(result['vertices'][i], circle['center']),
|
self.assertGreater(distance(result['vertices'][i], circle['center']),
|
||||||
circle['radius_m']+circle['coverage_tolerance_m'])
|
circle['radius_m']+circle['coverage_tolerance_m'])
|
||||||
@@ -61,7 +62,7 @@ class CertificateTests(unittest.TestCase):
|
|||||||
self.assertTrue(r['diameter_circle']['covers'])
|
self.assertTrue(r['diameter_circle']['covers'])
|
||||||
self.assertIsNone(r['diameter_circle']['witness_vertex'])
|
self.assertIsNone(r['diameter_circle']['witness_vertex'])
|
||||||
|
|
||||||
def test_first_sector_disk_covers_all_angles_and_distances(self):
|
def test_first_sector_disk_covers_all_angles(self):
|
||||||
for angle in [0, 359.9, -180]:
|
for angle in [0, 359.9, -180]:
|
||||||
for delta in [0, 1, 1.005, 45]:
|
for delta in [0, 1, 1.005, 45]:
|
||||||
m = Measurement(-800, 1200, angle, delta)
|
m = Measurement(-800, 1200, angle, delta)
|
||||||
@@ -72,11 +73,15 @@ class CertificateTests(unittest.TestCase):
|
|||||||
for k in range(11):
|
for k in range(11):
|
||||||
phi = math.radians(angle-delta+2*delta*j/20)
|
phi = math.radians(angle-delta+2*delta*j/20)
|
||||||
p = (m.x+150*k*math.cos(phi), m.y+150*k*math.sin(phi))
|
p = (m.x+150*k*math.cos(phi), m.y+150*k*math.sin(phi))
|
||||||
self.assertLessEqual(distance(p, enclosing['center']), enclosing['radius_m']+1e-9)
|
self.assertLessEqual(
|
||||||
|
distance(p, enclosing['center']),
|
||||||
|
enclosing['radius_m']+1e-9)
|
||||||
if candidate:
|
if candidate:
|
||||||
for t in range(8):
|
for t in range(8):
|
||||||
q = (candidate['center'][0]+candidate['radius_m']*math.cos(t*math.tau/8),
|
q = (candidate['center'][0]
|
||||||
candidate['center'][1]+candidate['radius_m']*math.sin(t*math.tau/8))
|
+ candidate['radius_m']*math.cos(t*math.tau/8),
|
||||||
|
candidate['center'][1]
|
||||||
|
+ candidate['radius_m']*math.sin(t*math.tau/8))
|
||||||
self.assertLessEqual(distance(p, q), 1000+1e-9)
|
self.assertLessEqual(distance(p, q), 1000+1e-9)
|
||||||
if delta <= 1.005:
|
if delta <= 1.005:
|
||||||
self.assertTrue(cert['candidate_nonempty_certified'])
|
self.assertTrue(cert['candidate_nonempty_certified'])
|
||||||
@@ -100,7 +105,8 @@ class CertificateTests(unittest.TestCase):
|
|||||||
self.assertEqual(outputs[0]['error_policy']['effective_half_angles_deg'], [1, 1, 1])
|
self.assertEqual(outputs[0]['error_policy']['effective_half_angles_deg'], [1, 1, 1])
|
||||||
self.assertEqual(outputs[1]['error_policy']['effective_half_angles_deg'], [1.005]*3)
|
self.assertEqual(outputs[1]['error_policy']['effective_half_angles_deg'], [1.005]*3)
|
||||||
self.assertGreater(outputs[1]['diameter_m'], outputs[0]['diameter_m'])
|
self.assertGreater(outputs[1]['diameter_m'], outputs[0]['diameter_m'])
|
||||||
bad = subprocess.run(base+['--rounding-margin-deg', '-1'], cwd=ROOT, capture_output=True)
|
bad = subprocess.run(base+['--rounding-margin-deg', '-1'], cwd=ROOT,
|
||||||
|
capture_output=True, check=False)
|
||||||
self.assertNotEqual(bad.returncode, 0)
|
self.assertNotEqual(bad.returncode, 0)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+14
-14
@@ -90,7 +90,7 @@ class LayoutTests(unittest.TestCase):
|
|||||||
again = env._loop.hrl_observation(env._current_candidates)
|
again = env._loop.hrl_observation(env._current_candidates)
|
||||||
self.assertTrue(numpy.array_equal(obs, again))
|
self.assertTrue(numpy.array_equal(obs, again))
|
||||||
|
|
||||||
def test_mask_admits_at_least_one_action_and_aligns_with_candidates(self):
|
def test_mask_nonempty_and_matches_candidate_slots(self):
|
||||||
env = _env(problem=3)
|
env = _env(problem=3)
|
||||||
env.reset(seed=7)
|
env.reset(seed=7)
|
||||||
for _ in range(6):
|
for _ in range(6):
|
||||||
@@ -98,7 +98,7 @@ class LayoutTests(unittest.TestCase):
|
|||||||
self.assertTrue(mask.any())
|
self.assertTrue(mask.any())
|
||||||
slots = {cand.slot for cand in env._current_candidates}
|
slots = {cand.slot for cand in env._current_candidates}
|
||||||
self.assertEqual({int(i) for i in numpy.flatnonzero(mask)}, slots)
|
self.assertEqual({int(i) for i in numpy.flatnonzero(mask)}, slots)
|
||||||
obs, _r, term, trunc, _info = env.step(
|
_obs, _r, term, trunc, _info = env.step(
|
||||||
int(env._loop.heuristic_action(env._current_candidates)))
|
int(env._loop.heuristic_action(env._current_candidates)))
|
||||||
if term or trunc:
|
if term or trunc:
|
||||||
break
|
break
|
||||||
@@ -114,7 +114,7 @@ class LayoutTests(unittest.TestCase):
|
|||||||
self.assertEqual(FALLBACK_OFFSET, STATION_OFFSET + STATION_SLOTS)
|
self.assertEqual(FALLBACK_OFFSET, STATION_OFFSET + STATION_SLOTS)
|
||||||
self.assertEqual(N_CANDIDATES - (FALLBACK_OFFSET + FALLBACK_SLOTS), 11)
|
self.assertEqual(N_CANDIDATES - (FALLBACK_OFFSET + FALLBACK_SLOTS), 11)
|
||||||
|
|
||||||
def test_build_candidates_keeps_clear_17_to_20_and_locate_1_to_4(self):
|
def test_build_candidates_keeps_clear_and_locate(self):
|
||||||
host = _CandidateHost(detected=[1, 2, 3, 4, 17, 18, 19, 20],
|
host = _CandidateHost(detected=[1, 2, 3, 4, 17, 18, 19, 20],
|
||||||
certified=[17, 18, 19, 20])
|
certified=[17, 18, 19, 20])
|
||||||
candidates = host.build_candidates()
|
candidates = host.build_candidates()
|
||||||
@@ -130,7 +130,7 @@ class LayoutTests(unittest.TestCase):
|
|||||||
selected = host.candidate_by_slot(candidates, CLEAR_OFFSET + channel - 1)
|
selected = host.candidate_by_slot(candidates, CLEAR_OFFSET + channel - 1)
|
||||||
self.assertEqual((selected.kind, selected.channel), ("clear", channel))
|
self.assertEqual((selected.kind, selected.channel), ("clear", channel))
|
||||||
|
|
||||||
def test_build_candidates_fallback_includes_channel_20_within_its_slots(self):
|
def test_build_candidates_fallback_covers_slots(self):
|
||||||
channels = list(range(17, 21))
|
channels = list(range(17, 21))
|
||||||
host = _CandidateHost(detected=channels, blocked=channels)
|
host = _CandidateHost(detected=channels, blocked=channels)
|
||||||
candidates = host.build_candidates()
|
candidates = host.build_candidates()
|
||||||
@@ -151,11 +151,11 @@ class LayoutTests(unittest.TestCase):
|
|||||||
class HeuristicLoopTests(unittest.TestCase):
|
class HeuristicLoopTests(unittest.TestCase):
|
||||||
def test_q3_full_episode_clears_all_sources(self):
|
def test_q3_full_episode_clears_all_sources(self):
|
||||||
env = _env(problem=3, stage_index=3, max_episode_steps=500)
|
env = _env(problem=3, stage_index=3, max_episode_steps=500)
|
||||||
obs, _ = env.reset(seed=11)
|
_, _ = env.reset(seed=11)
|
||||||
steps = 0
|
steps = 0
|
||||||
while True:
|
while True:
|
||||||
action = env._loop.heuristic_action(env._current_candidates)
|
action = env._loop.heuristic_action(env._current_candidates)
|
||||||
obs, r, term, trunc, info = env.step(action)
|
_, _, term, trunc, info = env.step(action)
|
||||||
steps += 1
|
steps += 1
|
||||||
if term or trunc:
|
if term or trunc:
|
||||||
break
|
break
|
||||||
@@ -170,7 +170,7 @@ class HeuristicLoopTests(unittest.TestCase):
|
|||||||
env.reset(seed=21)
|
env.reset(seed=21)
|
||||||
while True:
|
while True:
|
||||||
action = env._loop.heuristic_action(env._current_candidates)
|
action = env._loop.heuristic_action(env._current_candidates)
|
||||||
obs, r, term, trunc, info = env.step(action)
|
_, _, term, trunc, info = env.step(action)
|
||||||
if term or trunc:
|
if term or trunc:
|
||||||
break
|
break
|
||||||
self.assertTrue(term)
|
self.assertTrue(term)
|
||||||
@@ -185,8 +185,8 @@ class HeuristicLoopTests(unittest.TestCase):
|
|||||||
reward_cfg = env.reward
|
reward_cfg = env.reward
|
||||||
for _ in range(8):
|
for _ in range(8):
|
||||||
action = env._loop.heuristic_action(env._current_candidates)
|
action = env._loop.heuristic_action(env._current_candidates)
|
||||||
virtual_before = env._client.state.virtual_time_s
|
_virtual_before = env._client.state.virtual_time_s
|
||||||
obs, r, term, trunc, info = env.step(action)
|
_, r, term, trunc, info = env.step(action)
|
||||||
outcome = env._last_outcome
|
outcome = env._last_outcome
|
||||||
if outcome.invalid:
|
if outcome.invalid:
|
||||||
self.assertAlmostEqual(r, -reward_cfg.invalid, places=6)
|
self.assertAlmostEqual(r, -reward_cfg.invalid, places=6)
|
||||||
@@ -206,13 +206,13 @@ class HeuristicLoopTests(unittest.TestCase):
|
|||||||
if term or trunc:
|
if term or trunc:
|
||||||
break
|
break
|
||||||
|
|
||||||
def test_invalid_action_penalized_without_simulator_interaction(self):
|
def test_invalid_action_penalized_without_sim_call(self):
|
||||||
env = _env(problem=3, stage_index=0, max_episode_steps=50)
|
env = _env(problem=3, stage_index=0, max_episode_steps=50)
|
||||||
env.reset(seed=3)
|
env.reset(seed=3)
|
||||||
mask = env.action_masks()
|
mask = env.action_masks()
|
||||||
invalid_action = int(numpy.argmin(mask))
|
invalid_action = int(numpy.argmin(mask))
|
||||||
virtual_before = env._client.state.virtual_time_s
|
virtual_before = env._client.state.virtual_time_s
|
||||||
obs, r, term, trunc, info = env.step(invalid_action)
|
_, r, _, _, _ = env.step(invalid_action)
|
||||||
self.assertAlmostEqual(r, -env.reward.invalid, places=6)
|
self.assertAlmostEqual(r, -env.reward.invalid, places=6)
|
||||||
self.assertAlmostEqual(env._client.state.virtual_time_s,
|
self.assertAlmostEqual(env._client.state.virtual_time_s,
|
||||||
virtual_before, places=6)
|
virtual_before, places=6)
|
||||||
@@ -228,7 +228,7 @@ class LowLevelTests(unittest.TestCase):
|
|||||||
goal = tuple(info["goal"])
|
goal = tuple(info["goal"])
|
||||||
total = 0
|
total = 0
|
||||||
for _ in range(60):
|
for _ in range(60):
|
||||||
obs, r, term, trunc, info = env.step(env.expert_action())
|
obs, _, term, trunc, _ = env.step(env.expert_action())
|
||||||
total += 1
|
total += 1
|
||||||
if term or trunc:
|
if term or trunc:
|
||||||
break
|
break
|
||||||
@@ -237,7 +237,7 @@ class LowLevelTests(unittest.TestCase):
|
|||||||
efficiency = straight / (total * env.step_m)
|
efficiency = straight / (total * env.step_m)
|
||||||
self.assertGreater(efficiency, 0.7, "专家路径效率应接近直线")
|
self.assertGreater(efficiency, 0.7, "专家路径效率应接近直线")
|
||||||
|
|
||||||
def test_executor_straightline_is_goal_only_and_policy_path_snaps(self):
|
def test_executor_straightline_is_goal_only(self):
|
||||||
executor = LowLevelExecutor(policy=None)
|
executor = LowLevelExecutor(policy=None)
|
||||||
self.assertEqual(executor.roll((0.0, 0.0), (500.0, 200.0)),
|
self.assertEqual(executor.roll((0.0, 0.0), (500.0, 200.0)),
|
||||||
[(500.0, 200.0)])
|
[(500.0, 200.0)])
|
||||||
@@ -257,7 +257,7 @@ class CurriculumTests(unittest.TestCase):
|
|||||||
self.assertLessEqual(prev.bearing_error_half_deg,
|
self.assertLessEqual(prev.bearing_error_half_deg,
|
||||||
nxt.bearing_error_half_deg)
|
nxt.bearing_error_half_deg)
|
||||||
|
|
||||||
def test_training_config_defaults_to_production_fidelity(self):
|
def test_training_config_defaults_to_production(self):
|
||||||
config = training_config(CONFIG, STAGES[3])
|
config = training_config(CONFIG, STAGES[3])
|
||||||
self.assertEqual(config.field.channel_max, 20)
|
self.assertEqual(config.field.channel_max, 20)
|
||||||
self.assertEqual(config.field.bearing_error_half_deg, 1.0)
|
self.assertEqual(config.field.bearing_error_half_deg, 1.0)
|
||||||
|
|||||||
+244
-16
@@ -1,10 +1,13 @@
|
|||||||
"""高层训练落盘保证:checkpoint、SIGTERM 优雅保存、worker 信号屏蔽。
|
"""高层训练落盘保证:checkpoint、控制文件优雅停止、信号兜底、worker 信号屏蔽。
|
||||||
|
|
||||||
背景:训练前端的「停止」向整个进程组发 SIGTERM,而旧脚本只在 ``learn()``
|
背景:训练前端的「停止」以前直接向进程组发 SIGTERM,而旧脚本只在 ``learn()``
|
||||||
正常返回后才保存,导致"停一下就全丢"。这里把三条保证固定成回归测试:
|
正常返回后才保存,导致"停一下就全丢";在 Windows 上 ``os.killpg`` 更是直接
|
||||||
|
``AttributeError``。现在停止/暂停走**控制文件**(跨平台),信号只作兜底。这里把
|
||||||
|
保证固定成回归测试:
|
||||||
1. 定期 checkpoint 可续训(zip + 同名元数据);
|
1. 定期 checkpoint 可续训(zip + 同名元数据);
|
||||||
2. 进程组收到 SIGTERM 后仍能保存最终模型与元数据;
|
2. 控制文件发出 stop 后仍能保存最终模型与元数据(Windows 同样成立);
|
||||||
3. subproc worker 忽略停止信号,不抢在主进程收尾之前退出。
|
3. POSIX 下进程组 SIGTERM 也能优雅收尾;
|
||||||
|
4. subproc worker 忽略停止信号,不抢在主进程收尾之前退出。
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -15,11 +18,14 @@ import os
|
|||||||
import signal
|
import signal
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
|
import threading
|
||||||
import time
|
import time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
|
from mcm26b.process_control import spawn_kwargs, terminate_process_tree, write_control
|
||||||
|
|
||||||
|
|
||||||
def _train_module():
|
def _train_module():
|
||||||
"""按路径加载训练脚本(不触发 torch 导入,torch 只在函数内部惰性导入)。"""
|
"""按路径加载训练脚本(不触发 torch 导入,torch 只在函数内部惰性导入)。"""
|
||||||
@@ -39,13 +45,20 @@ def _require_rl():
|
|||||||
|
|
||||||
|
|
||||||
class _Run:
|
class _Run:
|
||||||
"""一次子进程训练:stdout 走文件,避免管道写满把子进程卡死。"""
|
"""一次子进程训练:stdout 走文件,避免管道写满把子进程卡死。
|
||||||
|
|
||||||
|
与训练前端一致:用控制文件请求暂停/停止,进程组隔离走
|
||||||
|
:func:`mcm26b.process_control.spawn_kwargs`(POSIX ``setsid`` /
|
||||||
|
Windows ``CREATE_NEW_PROCESS_GROUP``)。
|
||||||
|
"""
|
||||||
|
|
||||||
def __init__(self, run_dir: Path, *extra: str):
|
def __init__(self, run_dir: Path, *extra: str):
|
||||||
script = Path(__file__).parents[1] / "scripts" / "train_hrl_high.py"
|
script = Path(__file__).parents[1] / "scripts" / "train_hrl_high.py"
|
||||||
self.run_dir = run_dir
|
self.run_dir = run_dir
|
||||||
self.log_path = run_dir / "train_stdout.log"
|
self.log_path = run_dir / "train_stdout.log"
|
||||||
self.log_file = self.log_path.open("w", encoding="utf-8")
|
self.log_file = self.log_path.open("w", encoding="utf-8")
|
||||||
|
self.control_path = run_dir / "control.json"
|
||||||
|
write_control(self.control_path)
|
||||||
cmd = [sys.executable, str(script),
|
cmd = [sys.executable, str(script),
|
||||||
"--problem", "3", "--stage", "1", "--timesteps", "200000",
|
"--problem", "3", "--stage", "1", "--timesteps", "200000",
|
||||||
"--seed", "7", "--envs", "2", "--vec-env", "subproc",
|
"--seed", "7", "--envs", "2", "--vec-env", "subproc",
|
||||||
@@ -53,13 +66,28 @@ class _Run:
|
|||||||
"--reward-profile", "time", "--eval-episodes", "1",
|
"--reward-profile", "time", "--eval-episodes", "1",
|
||||||
"--rollout-steps", "32", "--batch-size", "32", "--n-epochs", "1",
|
"--rollout-steps", "32", "--batch-size", "32", "--n-epochs", "1",
|
||||||
"--max-wall-s", "600", "--force", "--verbose", "0",
|
"--max-wall-s", "600", "--force", "--verbose", "0",
|
||||||
|
"--control-file", str(self.control_path),
|
||||||
"--model", str(run_dir / "q3.zip"), *extra]
|
"--model", str(run_dir / "q3.zip"), *extra]
|
||||||
self.proc = subprocess.Popen(
|
self.proc = subprocess.Popen( # pylint: disable=consider-using-with
|
||||||
cmd, stdout=self.log_file, stderr=subprocess.STDOUT, text=True,
|
cmd, stdout=self.log_file, stderr=subprocess.STDOUT, text=True,
|
||||||
start_new_session=True)
|
**spawn_kwargs())
|
||||||
|
|
||||||
def checkpoints(self) -> list[Path]:
|
def checkpoints(self) -> list[Path]:
|
||||||
return sorted((self.run_dir / "checkpoints").glob("q3_ckpt_*.zip"))
|
"""历史 checkpoint(``<stem>_<步数>.zip``)。
|
||||||
|
|
||||||
|
刻意排除滚动恢复点 ``<stem>_latest.zip``:它随写随覆盖,不是"第几个
|
||||||
|
checkpoint",混进来会让序号断言失去意义。
|
||||||
|
"""
|
||||||
|
found = (self.run_dir / "checkpoints").glob("q3_ckpt_[0-9]*.zip")
|
||||||
|
return sorted(path for path in found
|
||||||
|
if path.stem.rsplit("_", 1)[-1].isdigit())
|
||||||
|
|
||||||
|
def recovery(self) -> Path:
|
||||||
|
return self.run_dir / "checkpoints" / "q3_ckpt_latest.zip"
|
||||||
|
|
||||||
|
def resume_hint(self) -> dict:
|
||||||
|
return json.loads((self.run_dir / "resume.json").read_text(
|
||||||
|
encoding="utf-8"))
|
||||||
|
|
||||||
def wait_for(self, predicate, timeout_s: float) -> bool:
|
def wait_for(self, predicate, timeout_s: float) -> bool:
|
||||||
deadline = time.monotonic() + timeout_s
|
deadline = time.monotonic() + timeout_s
|
||||||
@@ -71,8 +99,35 @@ class _Run:
|
|||||||
time.sleep(0.4)
|
time.sleep(0.4)
|
||||||
return predicate()
|
return predicate()
|
||||||
|
|
||||||
|
def logs(self) -> str:
|
||||||
|
try:
|
||||||
|
return self.log_path.read_text(encoding="utf-8", errors="replace")
|
||||||
|
except OSError: # pragma: no cover - 文件被并发写时的极端情况
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def pause(self) -> None:
|
||||||
|
write_control(self.control_path, paused=True)
|
||||||
|
|
||||||
|
def resume(self) -> None:
|
||||||
|
write_control(self.control_path, paused=False)
|
||||||
|
|
||||||
|
def _finish(self) -> str:
|
||||||
|
self.log_file.close()
|
||||||
|
return self.logs()
|
||||||
|
|
||||||
def stop(self) -> str:
|
def stop(self) -> str:
|
||||||
"""复现训练前端的停止语义:对整个进程组发 SIGTERM。"""
|
"""复现训练前端的停止语义:写控制文件请求优雅收尾(跨平台)。"""
|
||||||
|
write_control(self.control_path, stop=True)
|
||||||
|
forced = terminate_process_tree(self.proc, grace_s=240.0)
|
||||||
|
if forced:
|
||||||
|
self.log_file.close()
|
||||||
|
pytest.fail("控制文件停止请求后训练进程未在 240s 内退出(优雅停止失败)")
|
||||||
|
return self._finish()
|
||||||
|
|
||||||
|
def stop_by_signal(self) -> str:
|
||||||
|
"""POSIX 兜底路径:对整个进程组发 SIGTERM(Windows 不支持 killpg)。"""
|
||||||
|
if os.name == "nt": # pragma: no cover - 仅在 Windows 上走到
|
||||||
|
pytest.skip("Windows 无 os.killpg,信号路径不适用")
|
||||||
try:
|
try:
|
||||||
os.killpg(os.getpgid(self.proc.pid), signal.SIGTERM)
|
os.killpg(os.getpgid(self.proc.pid), signal.SIGTERM)
|
||||||
except (ProcessLookupError, PermissionError):
|
except (ProcessLookupError, PermissionError):
|
||||||
@@ -84,8 +139,12 @@ class _Run:
|
|||||||
self.proc.wait(timeout=30)
|
self.proc.wait(timeout=30)
|
||||||
self.log_file.close()
|
self.log_file.close()
|
||||||
pytest.fail("SIGTERM 后训练进程未在 240s 内退出(优雅停止失败)")
|
pytest.fail("SIGTERM 后训练进程未在 240s 内退出(优雅停止失败)")
|
||||||
|
return self._finish()
|
||||||
|
|
||||||
|
def hard_kill(self) -> None:
|
||||||
|
"""模拟"急停":立即强杀整棵进程树,不给训练脚本任何收尾机会。"""
|
||||||
|
terminate_process_tree(self.proc, grace_s=0.0)
|
||||||
self.log_file.close()
|
self.log_file.close()
|
||||||
return self.log_path.read_text(encoding="utf-8", errors="replace")
|
|
||||||
|
|
||||||
|
|
||||||
def _run_dir(tmp_path: Path) -> Path:
|
def _run_dir(tmp_path: Path) -> Path:
|
||||||
@@ -119,7 +178,7 @@ def test_worker_ignores_stop_signals_but_main_does_not(monkeypatch):
|
|||||||
class _Worker:
|
class _Worker:
|
||||||
name = "SpawnProcess-3"
|
name = "SpawnProcess-3"
|
||||||
|
|
||||||
monkeypatch.setattr(multiprocessing, "current_process", lambda: _Worker())
|
monkeypatch.setattr(multiprocessing, "current_process", _Worker)
|
||||||
try:
|
try:
|
||||||
module._ignore_stop_signals_in_worker()
|
module._ignore_stop_signals_in_worker()
|
||||||
assert signal.getsignal(signal.SIGTERM) == signal.SIG_IGN
|
assert signal.getsignal(signal.SIGTERM) == signal.SIG_IGN
|
||||||
@@ -131,7 +190,7 @@ def test_worker_ignores_stop_signals_but_main_does_not(monkeypatch):
|
|||||||
class _Main:
|
class _Main:
|
||||||
name = "MainProcess"
|
name = "MainProcess"
|
||||||
|
|
||||||
monkeypatch.setattr(multiprocessing, "current_process", lambda: _Main())
|
monkeypatch.setattr(multiprocessing, "current_process", _Main)
|
||||||
module._ignore_stop_signals_in_worker()
|
module._ignore_stop_signals_in_worker()
|
||||||
assert signal.getsignal(signal.SIGTERM) == original
|
assert signal.getsignal(signal.SIGTERM) == original
|
||||||
|
|
||||||
@@ -168,14 +227,139 @@ def test_policy_hyperparams_records_effective_values():
|
|||||||
assert module._policy_hyperparams(object())["n_steps"] == 0
|
assert module._policy_hyperparams(object())["n_steps"] == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_control_poll_state_machine(tmp_path):
|
||||||
|
"""控制文件轮询状态机:抽样读盘、暂停阻塞、stop 打断、暂停时长累计。"""
|
||||||
|
module = _train_module()
|
||||||
|
path = tmp_path / "control.json"
|
||||||
|
write_control(path, paused=True)
|
||||||
|
poll = module._ControlPoll(path, poll_every=1, poll_s=0.05)
|
||||||
|
assert poll.poll().paused is True
|
||||||
|
|
||||||
|
events = []
|
||||||
|
|
||||||
|
def _resume_later():
|
||||||
|
time.sleep(0.15)
|
||||||
|
write_control(path, paused=False)
|
||||||
|
|
||||||
|
threading.Thread(target=_resume_later, daemon=True).start()
|
||||||
|
stopped = poll.wait_while_paused(lambda: events.append("pause"),
|
||||||
|
lambda: events.append("resume"))
|
||||||
|
assert stopped is False
|
||||||
|
assert events == ["pause", "resume"]
|
||||||
|
assert poll.paused_total_s >= 0.1
|
||||||
|
|
||||||
|
# 暂停期间收到 stop:wait 必须返回 True,让调用方转入收尾
|
||||||
|
write_control(path, paused=True)
|
||||||
|
|
||||||
|
def _stop_later():
|
||||||
|
time.sleep(0.15)
|
||||||
|
write_control(path, stop=True)
|
||||||
|
|
||||||
|
threading.Thread(target=_stop_later, daemon=True).start()
|
||||||
|
assert poll.wait_while_paused(lambda: None, lambda: None) is True
|
||||||
|
assert poll.poll().stop is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_atomic_model_save_keeps_previous_snapshot_on_failure(tmp_path):
|
||||||
|
"""保存途中失败/被杀:目标路径仍是上一份完整快照,不会出现半截 zip。"""
|
||||||
|
module = _train_module()
|
||||||
|
target = tmp_path / "q3_ckpt_latest.zip"
|
||||||
|
target.write_bytes(b"OLD-COMPLETE")
|
||||||
|
|
||||||
|
class _FailingModel:
|
||||||
|
def save(self, path):
|
||||||
|
Path(path).write_bytes(b"PARTIAL")
|
||||||
|
raise RuntimeError("磁盘写满")
|
||||||
|
|
||||||
|
with pytest.raises(RuntimeError):
|
||||||
|
module._save_model_atomic(_FailingModel(), target)
|
||||||
|
assert target.read_bytes() == b"OLD-COMPLETE"
|
||||||
|
leftovers = [p.name for p in tmp_path.iterdir() if p.name != target.name]
|
||||||
|
assert leftovers and all(name.startswith(".tmp-") for name in leftovers)
|
||||||
|
|
||||||
|
|
||||||
|
def test_write_json_atomic_replaces_in_one_step(tmp_path):
|
||||||
|
module = _train_module()
|
||||||
|
path = tmp_path / "resume.json"
|
||||||
|
module._write_json_atomic(path, {"model": "a.zip", "timesteps": 1})
|
||||||
|
assert json.loads(path.read_text(encoding="utf-8"))["timesteps"] == 1
|
||||||
|
module._write_json_atomic(path, {"model": "b.zip", "timesteps": 2})
|
||||||
|
assert json.loads(path.read_text(encoding="utf-8"))["model"] == "b.zip"
|
||||||
|
assert not list(tmp_path.glob(".tmp-*"))
|
||||||
|
|
||||||
|
|
||||||
|
def test_prune_checkpoints_never_deletes_recovery_point(tmp_path):
|
||||||
|
"""滚动恢复点与临时文件不参与历史清理。"""
|
||||||
|
module = _train_module()
|
||||||
|
directory = tmp_path / "checkpoints"
|
||||||
|
directory.mkdir()
|
||||||
|
latest = directory / "q3_ckpt_latest.zip"
|
||||||
|
latest.write_bytes(b"RECOVERY")
|
||||||
|
(directory / ".tmp-q3_ckpt_000000009.zip").write_bytes(b"PARTIAL")
|
||||||
|
for index in range(5):
|
||||||
|
(directory / f"q3_ckpt_{index:09d}.zip").write_bytes(b"x")
|
||||||
|
module._prune_checkpoints(directory, "q3_ckpt", 1)
|
||||||
|
assert latest.exists() and latest.read_bytes() == b"RECOVERY"
|
||||||
|
assert (directory / ".tmp-q3_ckpt_000000009.zip").exists()
|
||||||
|
|
||||||
|
|
||||||
# --------------------------------------------------------------- 集成回归
|
# --------------------------------------------------------------- 集成回归
|
||||||
|
def test_recovery_point_exists_before_training_and_is_loadable(tmp_path):
|
||||||
|
"""起步基线:训练刚起就有可加载、可续训的产物(--checkpoint-every 0 也成立)。"""
|
||||||
|
_require_rl()
|
||||||
|
from sb3_contrib import MaskablePPO
|
||||||
|
|
||||||
|
run = _Run(_run_dir(tmp_path), "--checkpoint-every", "0")
|
||||||
|
try:
|
||||||
|
assert run.wait_for(lambda: run.recovery().exists(), 120.0), \
|
||||||
|
"起步恢复点未落盘"
|
||||||
|
loaded = MaskablePPO.load(str(run.recovery()))
|
||||||
|
assert loaded.num_timesteps >= 0
|
||||||
|
hint = run.resume_hint()
|
||||||
|
assert Path(hint["model"]) == run.recovery()
|
||||||
|
assert hint["timesteps_semantics"] == "additional"
|
||||||
|
finally:
|
||||||
|
run.stop()
|
||||||
|
|
||||||
|
|
||||||
|
def test_hard_kill_still_leaves_resumable_recovery_point(tmp_path):
|
||||||
|
"""急停(强杀、无收尾)后仍能续训:滚动恢复点 + resume.json 都在且可加载。"""
|
||||||
|
_require_rl()
|
||||||
|
from sb3_contrib import MaskablePPO
|
||||||
|
|
||||||
|
run = _Run(_run_dir(tmp_path), "--checkpoint-every", "0",
|
||||||
|
"--recovery-every", "32")
|
||||||
|
try:
|
||||||
|
assert run.wait_for(lambda: run.recovery().exists(), 120.0), \
|
||||||
|
"起步恢复点未落盘"
|
||||||
|
first = run.recovery().stat().st_mtime_ns
|
||||||
|
assert run.wait_for(
|
||||||
|
lambda: run.recovery().stat().st_mtime_ns != first, 300.0), \
|
||||||
|
"滚动恢复点未按 --recovery-every 刷新"
|
||||||
|
assert run.proc.poll() is None, "训练已自行结束,无法验证急停"
|
||||||
|
finally:
|
||||||
|
run.hard_kill() # 急停:强杀整棵进程树,没有任何 Python 收尾
|
||||||
|
|
||||||
|
hint = run.resume_hint()
|
||||||
|
assert Path(hint["model"]).is_file(), hint
|
||||||
|
assert Path(hint["model"]) == run.recovery(), hint
|
||||||
|
loaded = MaskablePPO.load(str(run.recovery()))
|
||||||
|
assert loaded.num_timesteps >= 32, loaded.num_timesteps
|
||||||
|
metadata = json.loads(run.recovery().with_suffix(".json").read_text(
|
||||||
|
encoding="utf-8"))
|
||||||
|
assert metadata["recovery"] is True
|
||||||
|
assert metadata["checkpoint_reason"] in ("baseline", "interval")
|
||||||
|
# 急停没有 graceful 收尾,所以不会有"stopped_by_signal"的最终模型
|
||||||
|
assert not (run.run_dir / "q3.zip").exists()
|
||||||
|
|
||||||
|
|
||||||
def test_checkpoint_is_written_and_loadable(tmp_path):
|
def test_checkpoint_is_written_and_loadable(tmp_path):
|
||||||
_require_rl()
|
_require_rl()
|
||||||
from sb3_contrib import MaskablePPO
|
from sb3_contrib import MaskablePPO
|
||||||
|
|
||||||
run = _Run(_run_dir(tmp_path), "--checkpoint-every", "64")
|
run = _Run(_run_dir(tmp_path), "--checkpoint-every", "64")
|
||||||
try:
|
try:
|
||||||
assert run.wait_for(lambda: bool(run.checkpoints()), 300.0), \
|
assert run.wait_for(run.checkpoints, 300.0), \
|
||||||
"训练未在 300s 内产出 checkpoint"
|
"训练未在 300s 内产出 checkpoint"
|
||||||
finally:
|
finally:
|
||||||
run.stop()
|
run.stop()
|
||||||
@@ -193,16 +377,60 @@ def test_checkpoint_is_written_and_loadable(tmp_path):
|
|||||||
assert (hyper["n_steps"], hyper["batch_size"], hyper["n_epochs"]) == (32, 32, 1)
|
assert (hyper["n_steps"], hyper["batch_size"], hyper["n_epochs"]) == (32, 32, 1)
|
||||||
|
|
||||||
|
|
||||||
|
def test_control_file_stop_saves_model(tmp_path):
|
||||||
|
"""控制文件 stop 是训练前端的默认停止路径,Windows 上同样必须成立。"""
|
||||||
|
_require_rl()
|
||||||
|
run = _Run(_run_dir(tmp_path), "--checkpoint-every", "64")
|
||||||
|
model_path = run.run_dir / "q3.zip"
|
||||||
|
try:
|
||||||
|
assert run.wait_for(run.checkpoints, 300.0), \
|
||||||
|
"训练未在 300s 内进入 rollout 阶段"
|
||||||
|
assert run.proc.poll() is None, "训练进程提前退出,无法验证优雅停止"
|
||||||
|
finally:
|
||||||
|
output = run.stop()
|
||||||
|
|
||||||
|
assert run.proc.returncode == 0, \
|
||||||
|
f"控制文件停止应正常退出,实际 {run.proc.returncode}\n{output}"
|
||||||
|
assert model_path.exists(), "控制文件停止后最终模型未落盘"
|
||||||
|
metadata = json.loads(model_path.with_suffix(".json").read_text(
|
||||||
|
encoding="utf-8"))
|
||||||
|
assert metadata["stopped_by_signal"] is True
|
||||||
|
assert metadata["stop_signal"] == "control_file"
|
||||||
|
assert metadata["checkpoint_files"], "元数据应记录已产出的 checkpoint"
|
||||||
|
assert not (run.run_dir / "training_failure.json").exists()
|
||||||
|
assert "saved_after_stop" in output
|
||||||
|
|
||||||
|
|
||||||
|
def test_control_file_pause_then_resume_keeps_training_alive(tmp_path):
|
||||||
|
"""暂停是协作式的:进程存活、停在 step 边界,continue 后照常收尾。"""
|
||||||
|
_require_rl()
|
||||||
|
run = _Run(_run_dir(tmp_path), "--checkpoint-every", "64")
|
||||||
|
try:
|
||||||
|
assert run.wait_for(run.checkpoints, 300.0), \
|
||||||
|
"训练未在 300s 内进入 rollout 阶段"
|
||||||
|
run.pause()
|
||||||
|
assert run.wait_for(lambda: '"event": "paused"' in run.logs(), 120.0), \
|
||||||
|
"训练脚本未在 120s 内进入暂停"
|
||||||
|
assert run.proc.poll() is None, "暂停不应结束进程"
|
||||||
|
run.resume()
|
||||||
|
assert run.wait_for(lambda: '"event": "resumed"' in run.logs(), 120.0), \
|
||||||
|
"训练脚本未在 120s 内恢复"
|
||||||
|
finally:
|
||||||
|
run.stop()
|
||||||
|
assert run.proc.returncode == 0
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.skipif(os.name == "nt", reason="Windows 没有 os.killpg,信号兜底路径不适用")
|
||||||
def test_sigterm_saves_model_instead_of_losing_the_run(tmp_path):
|
def test_sigterm_saves_model_instead_of_losing_the_run(tmp_path):
|
||||||
_require_rl()
|
_require_rl()
|
||||||
run = _Run(_run_dir(tmp_path), "--checkpoint-every", "64")
|
run = _Run(_run_dir(tmp_path), "--checkpoint-every", "64")
|
||||||
model_path = run.run_dir / "q3.zip"
|
model_path = run.run_dir / "q3.zip"
|
||||||
try:
|
try:
|
||||||
assert run.wait_for(lambda: bool(run.checkpoints()), 300.0), \
|
assert run.wait_for(run.checkpoints, 300.0), \
|
||||||
"训练未在 300s 内进入 rollout 阶段"
|
"训练未在 300s 内进入 rollout 阶段"
|
||||||
assert run.proc.poll() is None, "训练进程提前退出,无法验证优雅停止"
|
assert run.proc.poll() is None, "训练进程提前退出,无法验证优雅停止"
|
||||||
finally:
|
finally:
|
||||||
output = run.stop()
|
output = run.stop_by_signal()
|
||||||
|
|
||||||
assert run.proc.returncode == 0, \
|
assert run.proc.returncode == 0, \
|
||||||
f"优雅停止应正常退出,实际 {run.proc.returncode}\n{output}"
|
f"优雅停止应正常退出,实际 {run.proc.returncode}\n{output}"
|
||||||
|
|||||||
@@ -1,7 +1,6 @@
|
|||||||
"""低层导航运动学与训练入口的行为回归;不验证策略收益。"""
|
"""低层导航运动学与训练入口的行为回归;不验证策略收益。"""
|
||||||
|
|
||||||
import importlib.util
|
import importlib.util
|
||||||
import math
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
@@ -59,7 +58,7 @@ def test_expert_reaches_all_200_fixed_seed_targets():
|
|||||||
|
|
||||||
def test_failed_executor_reports_failure_without_free_terminal_move():
|
def test_failed_executor_reports_failure_without_free_terminal_move():
|
||||||
class WestPolicy:
|
class WestPolicy:
|
||||||
def select(self, observation):
|
def select(self, _observation):
|
||||||
return 4
|
return 4
|
||||||
|
|
||||||
executor = LowLevelExecutor(WestPolicy(), max_steps=3)
|
executor = LowLevelExecutor(WestPolicy(), max_steps=3)
|
||||||
@@ -73,7 +72,7 @@ def test_failed_executor_reports_failure_without_free_terminal_move():
|
|||||||
|
|
||||||
def test_env_and_executor_share_step_transitions():
|
def test_env_and_executor_share_step_transitions():
|
||||||
class EastPolicy:
|
class EastPolicy:
|
||||||
def select(self, observation):
|
def select(self, _observation):
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
env = LowLevelEnv(max_steps=3)
|
env = LowLevelEnv(max_steps=3)
|
||||||
|
|||||||
@@ -75,6 +75,9 @@ def test_time_only_reward_has_no_per_source_shaping():
|
|||||||
reason="RL 依赖组未安装")
|
reason="RL 依赖组未安装")
|
||||||
def test_project_compat_layer_reexports_real_classes():
|
def test_project_compat_layer_reexports_real_classes():
|
||||||
from mcm26b.hrl.hierarchical_env import HierarchicalEnv
|
from mcm26b.hrl.hierarchical_env import HierarchicalEnv
|
||||||
|
# project 是仓库根部的兼容层:pytest 经 rootdir 把仓库根放进 sys.path,
|
||||||
|
# 而单独 lint tests/ 时 pylint 的 sys.path 不含仓库根,故此处局部豁免。
|
||||||
|
# pylint: disable=import-error
|
||||||
from project.env.local_env import HierarchicalEnv as CompatEnv
|
from project.env.local_env import HierarchicalEnv as CompatEnv
|
||||||
from project.env.simulator_client import SimulatorClient
|
from project.env.simulator_client import SimulatorClient
|
||||||
from project.policies.high_level_ppo import HighLevelPolicy
|
from project.policies.high_level_ppo import HighLevelPolicy
|
||||||
|
|||||||
@@ -43,21 +43,21 @@ def test_fallback_failure_then_success_records_actual_time_position_and_clear():
|
|||||||
assert loop.client.state.channel == 20
|
assert loop.client.state.channel == 20
|
||||||
assert loop.fallback_log[-1]["attempts"] == 2
|
assert loop.fallback_log[-1]["attempts"] == 2
|
||||||
assert 20 not in loop.blocked_probes
|
assert 20 not in loop.blocked_probes
|
||||||
assert loop.anomalies == []
|
assert not loop.anomalies
|
||||||
finally:
|
finally:
|
||||||
loop.client.exit()
|
loop.client.exit()
|
||||||
|
|
||||||
|
|
||||||
def test_invalid_policy_output_is_logged_and_replaced_by_valid_action():
|
def test_invalid_policy_output_is_logged_and_replaced_by_valid_action():
|
||||||
class InvalidPolicy:
|
class InvalidPolicy:
|
||||||
def select(self, observation, mask):
|
def select(self, _observation, _mask):
|
||||||
return N_CANDIDATES + 1
|
return N_CANDIDATES + 1
|
||||||
|
|
||||||
loop = make_loop(problem=3, high_policy=InvalidPolicy())
|
loop = make_loop(problem=3, high_policy=InvalidPolicy())
|
||||||
candidate = Candidate(STATION_OFFSET, "station", (0.0, 0.0), station_index=0)
|
candidate = Candidate(STATION_OFFSET, "station", (0.0, 0.0), station_index=0)
|
||||||
executed = []
|
executed = []
|
||||||
|
|
||||||
def execute(action, candidates):
|
def execute(action, candidates): # pylint: disable=unused-argument
|
||||||
executed.append(action)
|
executed.append(action)
|
||||||
loop.stop_reason = "test_complete"
|
loop.stop_reason = "test_complete"
|
||||||
|
|
||||||
@@ -78,8 +78,10 @@ def test_opportunistic_measure_precedes_candidate_snapshot():
|
|||||||
order = []
|
order = []
|
||||||
try:
|
try:
|
||||||
assert loop.config.bayes.opportunistic_measure
|
assert loop.config.bayes.opportunistic_measure
|
||||||
with patch.object(loop, "_measure_at_stop", side_effect=lambda: order.append("measure")), \
|
with patch.object(loop, "_measure_at_stop",
|
||||||
patch.object(loop, "build_candidates", side_effect=lambda: order.append("snapshot") or []):
|
side_effect=lambda: order.append("measure")), \
|
||||||
|
patch.object(loop, "build_candidates",
|
||||||
|
side_effect=lambda: order.append("snapshot") or []):
|
||||||
loop._coverage_phase()
|
loop._coverage_phase()
|
||||||
assert order == ["measure", "snapshot"]
|
assert order == ["measure", "snapshot"]
|
||||||
finally:
|
finally:
|
||||||
|
|||||||
@@ -134,7 +134,7 @@ class StepTimingTests(unittest.TestCase):
|
|||||||
|
|
||||||
def test_steps_recorded_with_virtual_breakdown(self):
|
def test_steps_recorded_with_virtual_breakdown(self):
|
||||||
sources = random_omni_scene(seed=5, count=2)
|
sources = random_omni_scene(seed=5, count=2)
|
||||||
summary, report, _ = _run_scene(sources)
|
summary, _, _ = _run_scene(sources)
|
||||||
steps = summary["steps"]
|
steps = summary["steps"]
|
||||||
self.assertEqual(steps[0]["action"], "enter")
|
self.assertEqual(steps[0]["action"], "enter")
|
||||||
self.assertEqual(steps[-1]["action"], "exit")
|
self.assertEqual(steps[-1]["action"], "exit")
|
||||||
@@ -228,7 +228,7 @@ class SimulatorVirtualCapTests(unittest.TestCase):
|
|||||||
class AbortFinalizationTests(unittest.TestCase):
|
class AbortFinalizationTests(unittest.TestCase):
|
||||||
"""断连超过重试上限:仍须 /exit、产出未完成 summary,不得静默崩溃。"""
|
"""断连超过重试上限:仍须 /exit、产出未完成 summary,不得静默崩溃。"""
|
||||||
|
|
||||||
def test_transport_failure_aborts_with_summary_and_exit(self):
|
def test_transport_failure_aborts_with_summary(self):
|
||||||
"""首个检测动作三次断连:run() 正常返回,/exit 成功,stop_reason 留档。"""
|
"""首个检测动作三次断连:run() 正常返回,/exit 成功,stop_reason 留档。"""
|
||||||
summary, _, _ = _run_scene(
|
summary, _, _ = _run_scene(
|
||||||
random_omni_scene(seed=5, count=2),
|
random_omni_scene(seed=5, count=2),
|
||||||
|
|||||||
@@ -97,7 +97,7 @@ class ProbeLadderGeometryTests(unittest.TestCase):
|
|||||||
loop.blocked_probes[7] = [blocked_point]
|
loop.blocked_probes[7] = [blocked_point]
|
||||||
return loop._next_probe(region, 7)
|
return loop._next_probe(region, 7)
|
||||||
|
|
||||||
def test_antipode_reflects_blocked_probe_through_center(self):
|
def test_antipode_reflects_blocked_probe_in_center(self):
|
||||||
"""圆心为原点、原探点 (200,0):对跖复核应返回 (-200,0)。"""
|
"""圆心为原点、原探点 (200,0):对跖复核应返回 (-200,0)。"""
|
||||||
region = CandidateRegion.disk((0.0, 0.0), 100.0, 192) # safe≈900>=200
|
region = CandidateRegion.disk((0.0, 0.0), 100.0, 192) # safe≈900>=200
|
||||||
probe = self._probe_after_block(region, (200.0, 0.0))
|
probe = self._probe_after_block(region, (200.0, 0.0))
|
||||||
@@ -113,7 +113,7 @@ class ProbeLadderGeometryTests(unittest.TestCase):
|
|||||||
self.assertAlmostEqual(math.hypot(probe[0], probe[1]), safe, places=6)
|
self.assertAlmostEqual(math.hypot(probe[0], probe[1]), safe, places=6)
|
||||||
self.assertLess(probe[0], 0.0, "应位于原探点的对跖方向")
|
self.assertLess(probe[0], 0.0, "应位于原探点的对跖方向")
|
||||||
|
|
||||||
def test_degenerate_center_block_walks_full_safe_radius(self):
|
def test_degenerate_center_block_walks_safe_radius(self):
|
||||||
"""被遮挡探点恰在圆心:沿 +x 走满安全盘半径,不再钳到 1m。"""
|
"""被遮挡探点恰在圆心:沿 +x 走满安全盘半径,不再钳到 1m。"""
|
||||||
region = CandidateRegion.disk((0.0, 0.0), 100.0, 192)
|
region = CandidateRegion.disk((0.0, 0.0), 100.0, 192)
|
||||||
probe = self._probe_after_block(region, (0.0, 0.0))
|
probe = self._probe_after_block(region, (0.0, 0.0))
|
||||||
@@ -125,7 +125,7 @@ class ProbeLadderGeometryTests(unittest.TestCase):
|
|||||||
class MixedLoopAcceptanceTests(unittest.TestCase):
|
class MixedLoopAcceptanceTests(unittest.TestCase):
|
||||||
def test_mixed_ten_sources_all_cleared(self):
|
def test_mixed_ten_sources_all_cleared(self):
|
||||||
sources = mixed_scene(seed=5, count=10, directional=3)
|
sources = mixed_scene(seed=5, count=10, directional=3)
|
||||||
summary, report, simulator, _, loop = _run_scene(sources)
|
summary, report, simulator, _, _ = _run_scene(sources)
|
||||||
truth = sorted(s.channel for s in sources)
|
truth = sorted(s.channel for s in sources)
|
||||||
self.assertTrue(summary["completed"])
|
self.assertTrue(summary["completed"])
|
||||||
self.assertEqual(summary["stop_reason"], "all_resolved")
|
self.assertEqual(summary["stop_reason"], "all_resolved")
|
||||||
@@ -155,7 +155,7 @@ class MixedLoopAcceptanceTests(unittest.TestCase):
|
|||||||
self.assertEqual(sorted(simulator.cleared), truth)
|
self.assertEqual(sorted(simulator.cleared), truth)
|
||||||
self.assertEqual(report["mismatches"], [])
|
self.assertEqual(report["mismatches"], [])
|
||||||
|
|
||||||
def test_outward_boundary_directional_source_not_missed(self):
|
def test_outward_boundary_directional_not_missed(self):
|
||||||
"""朝向背离圆心、r_eff=1000 的近边界源:覆盖保证仍发现并清除。"""
|
"""朝向背离圆心、r_eff=1000 的近边界源:覆盖保证仍发现并清除。"""
|
||||||
source = Source(channel=7, x=1790.0, y=0.0, r_eff_m=1000.0,
|
source = Source(channel=7, x=1790.0, y=0.0, r_eff_m=1000.0,
|
||||||
directional=True, heading_deg=0.0)
|
directional=True, heading_deg=0.0)
|
||||||
@@ -169,8 +169,8 @@ class MixedLoopAcceptanceTests(unittest.TestCase):
|
|||||||
"""固定批量种子回归:任何种子下不得全清失败或破坏守恒性。"""
|
"""固定批量种子回归:任何种子下不得全清失败或破坏守恒性。"""
|
||||||
for seed in range(5, 10):
|
for seed in range(5, 10):
|
||||||
sources = mixed_scene(seed=seed, count=10, directional=4)
|
sources = mixed_scene(seed=seed, count=10, directional=4)
|
||||||
summary, report, simulator, _, _ = _run_scene(sources,
|
summary, report, _, _, _ = _run_scene(sources,
|
||||||
config=SMALL_CAP)
|
config=SMALL_CAP)
|
||||||
truth = sorted(s.channel for s in sources)
|
truth = sorted(s.channel for s in sources)
|
||||||
self.assertTrue(summary["completed"], f"seed={seed}")
|
self.assertTrue(summary["completed"], f"seed={seed}")
|
||||||
self.assertEqual(sorted(summary["cleared_channels"]), truth,
|
self.assertEqual(sorted(summary["cleared_channels"]), truth,
|
||||||
@@ -194,7 +194,7 @@ class JointConstraintBaselineTests(unittest.TestCase):
|
|||||||
for witness in region.witnesses(300.0):
|
for witness in region.witnesses(300.0):
|
||||||
self.assertTrue(region.contains(witness, tol_m=0.05))
|
self.assertTrue(region.contains(witness, tol_m=0.05))
|
||||||
|
|
||||||
def test_directional_source_retained_through_blocked_probes(self):
|
def test_directional_source_kept_after_blocked_probes(self):
|
||||||
"""端到端保留性:定向源被遮挡若干次后仍被清除,不因 no_signal 误删。"""
|
"""端到端保留性:定向源被遮挡若干次后仍被清除,不因 no_signal 误删。"""
|
||||||
source = Source(channel=3, x=600.0, y=800.0, r_eff_m=1500.0,
|
source = Source(channel=3, x=600.0, y=800.0, r_eff_m=1500.0,
|
||||||
directional=True, heading_deg=200.0)
|
directional=True, heading_deg=200.0)
|
||||||
@@ -261,7 +261,7 @@ class FallbackDisciplineTests(unittest.TestCase):
|
|||||||
cells = fallback_clear_cells(region.polygon, CONFIG)
|
cells = fallback_clear_cells(region.polygon, CONFIG)
|
||||||
self.assertLessEqual(len(cells), CONFIG.search.fallback_max_cells)
|
self.assertLessEqual(len(cells), CONFIG.search.fallback_max_cells)
|
||||||
|
|
||||||
def test_fallback_exhaustion_records_anomaly_without_deletion(self):
|
def test_fallback_exhaustion_records_anomaly(self):
|
||||||
"""后备全失败(空域):守恒性异常留痕,且不删除空间候选。"""
|
"""后备全失败(空域):守恒性异常留痕,且不删除空间候选。"""
|
||||||
simulator = LocalSimulator([], robot_id="T001", config=CONFIG)
|
simulator = LocalSimulator([], robot_id="T001", config=CONFIG)
|
||||||
client = SimulatorClient("T001", config=CONFIG,
|
client = SimulatorClient("T001", config=CONFIG,
|
||||||
|
|||||||
@@ -13,7 +13,6 @@ from dataclasses import replace
|
|||||||
from random import Random
|
from random import Random
|
||||||
|
|
||||||
from mcm26b.config import ProblemConfig
|
from mcm26b.config import ProblemConfig
|
||||||
from mcm26b.geometry.localization import smallest_enclosing_circle
|
|
||||||
from mcm26b.particles import ChannelParticles, angular_gap
|
from mcm26b.particles import ChannelParticles, angular_gap
|
||||||
from mcm26b.protocol.rules import DIRECTION, NEAR, NO_SIGNAL
|
from mcm26b.protocol.rules import DIRECTION, NEAR, NO_SIGNAL
|
||||||
from mcm26b.planner.seven import seven_points
|
from mcm26b.planner.seven import seven_points
|
||||||
|
|||||||
@@ -31,7 +31,7 @@ def _k1_region() -> CandidateRegion:
|
|||||||
|
|
||||||
|
|
||||||
class SelectionGuaranteeTests(TestCase):
|
class SelectionGuaranteeTests(TestCase):
|
||||||
def test_selected_point_receives_every_sector_candidate(self):
|
def test_selected_point_gets_every_sector_candidate(self):
|
||||||
"""安全盘内选点对解析角域 K1 内任意候选源的距离 <= 1000m。
|
"""安全盘内选点对解析角域 K1 内任意候选源的距离 <= 1000m。
|
||||||
|
|
||||||
直接对截断角域 K1 = Ω ∩ W(s1, θ±δ) ∩ B(s1,1500) 稠密采样验证,
|
直接对截断角域 K1 = Ω ∩ W(s1, θ±δ) ∩ B(s1,1500) 稠密采样验证,
|
||||||
@@ -113,7 +113,7 @@ class ClearCertificateTests(TestCase):
|
|||||||
|
|
||||||
|
|
||||||
class SelectSecondPointTests(TestCase):
|
class SelectSecondPointTests(TestCase):
|
||||||
def test_plan_lies_in_analytic_safe_disk_or_is_equivalent(self):
|
def test_plan_in_analytic_safe_disk_is_equivalent(self):
|
||||||
"""P3 入口输出与 geometry 解析证书一致(或提供同等保证)。"""
|
"""P3 入口输出与 geometry 解析证书一致(或提供同等保证)。"""
|
||||||
simulator = LocalSimulator([], robot_id="T001")
|
simulator = LocalSimulator([], robot_id="T001")
|
||||||
client = SimulatorClient("T001", transport=LocalSimTransport(simulator))
|
client = SimulatorClient("T001", transport=LocalSimTransport(simulator))
|
||||||
|
|||||||
@@ -38,7 +38,7 @@ def test_seven_stations_cannot_prove_directional_absence():
|
|||||||
|
|
||||||
|
|
||||||
def test_stopped_opportunistic_measure_is_serial_and_never_repeats_location():
|
def test_stopped_opportunistic_measure_is_serial_and_never_repeats_location():
|
||||||
sim, client, loop = setup([Source(1, 800, 200, 1500)])
|
_, client, loop = setup([Source(1, 800, 200, 1500)])
|
||||||
client.enter()
|
client.enter()
|
||||||
loop._record_measure(1, (0, 0), client.measure((0, 0), 1))
|
loop._record_measure(1, (0, 0), client.measure((0, 0), 1))
|
||||||
client.measure((600, 0), 2)
|
client.measure((600, 0), 2)
|
||||||
|
|||||||
@@ -40,8 +40,10 @@ class PracticeReplayTests(unittest.TestCase):
|
|||||||
self.assertEqual(data['source'],'official_practice')
|
self.assertEqual(data['source'],'official_practice')
|
||||||
self.assertIsNone(data['ground_truth'])
|
self.assertIsNone(data['ground_truth'])
|
||||||
directions=[e for e in events if e['response'].get('measure_result')=='direction']
|
directions=[e for e in events if e['response'].get('measure_result')=='direction']
|
||||||
self.assertEqual(directions[0]['request']['position'],directions[1]['request']['position'])
|
self.assertEqual(directions[0]['request']['position'],
|
||||||
self.assertEqual(directions[0]['response']['svd_deg'],directions[1]['response']['svd_deg'])
|
directions[1]['request']['position'])
|
||||||
|
self.assertEqual(directions[0]['response']['svd_deg'],
|
||||||
|
directions[1]['response']['svd_deg'])
|
||||||
|
|
||||||
|
|
||||||
if __name__=='__main__': unittest.main()
|
if __name__=='__main__': unittest.main()
|
||||||
|
|||||||
@@ -45,7 +45,7 @@ class FaultRecoveryTests(unittest.TestCase):
|
|||||||
outcome = client.measure((0, 0), 1)
|
outcome = client.measure((0, 0), 1)
|
||||||
self.assertEqual(outcome.virtual_time_s, 5.0)
|
self.assertEqual(outcome.virtual_time_s, 5.0)
|
||||||
# measure 只被服务器执行了一次:请求发出两次,但虚拟时刻只推进一次。
|
# measure 只被服务器执行了一次:请求发出两次,但虚拟时刻只推进一次。
|
||||||
measure_bodies = [body for body in faults.sent_requests[1:]]
|
measure_bodies = list(faults.sent_requests[1:])
|
||||||
self.assertEqual(len(measure_bodies), 2, "应发出原始请求+一次重试")
|
self.assertEqual(len(measure_bodies), 2, "应发出原始请求+一次重试")
|
||||||
first, retry = (json.loads(b) for b in measure_bodies)
|
first, retry = (json.loads(b) for b in measure_bodies)
|
||||||
self.assertEqual(first["request_id"], retry["request_id"],
|
self.assertEqual(first["request_id"], retry["request_id"],
|
||||||
@@ -56,22 +56,22 @@ class FaultRecoveryTests(unittest.TestCase):
|
|||||||
|
|
||||||
def test_drop_before_apply_is_safe_to_retry(self):
|
def test_drop_before_apply_is_safe_to_retry(self):
|
||||||
"""连接失败(未送达)后重试成功,无残留效果。"""
|
"""连接失败(未送达)后重试成功,无残留效果。"""
|
||||||
client, faults, simulator = self._client(["drop_before", None])
|
client, _, simulator = self._client(["drop_before", None])
|
||||||
client.enter()
|
client.enter()
|
||||||
client.measure((100, 0), 1)
|
client.measure((100, 0), 1)
|
||||||
self.assertEqual(simulator.virtual_time_s, 25.0) # 100/5 移动 + 5 检测
|
self.assertEqual(simulator.virtual_time_s, 25.0) # 100/5 移动 + 5 检测
|
||||||
self.assertEqual(client.state.virtual_time_s, 25.0)
|
self.assertEqual(client.state.virtual_time_s, 25.0)
|
||||||
|
|
||||||
def test_server_500_retried_with_same_id(self):
|
def test_server_500_retried_with_same_id(self):
|
||||||
client, faults, simulator = self._client([None, ("status", 500), None, None])
|
client, _, _ = self._client([None, ("status", 500), None, None])
|
||||||
client.enter()
|
client.enter()
|
||||||
client.measure((0, 0), 1)
|
client.measure((0, 0), 1)
|
||||||
self.assertEqual(client.state.virtual_time_s, 5.0)
|
self.assertEqual(client.state.virtual_time_s, 5.0)
|
||||||
|
|
||||||
def test_exhausted_retries_raise_and_leave_state_clean(self):
|
def test_exhausted_retries_raise_and_leave_state_clean(self):
|
||||||
# enter 放行后,measure 连续注入断连直至超过 max_attempts=3。
|
# enter 放行后,measure 连续注入断连直至超过 max_attempts=3。
|
||||||
client, faults, simulator = self._client([None, "drop_before", "drop_before",
|
client, _, simulator = self._client([None, "drop_before", "drop_before",
|
||||||
"drop_before"])
|
"drop_before"])
|
||||||
client.enter()
|
client.enter()
|
||||||
with self.assertRaises(TransportError):
|
with self.assertRaises(TransportError):
|
||||||
client.measure((0, 0), 1)
|
client.measure((0, 0), 1)
|
||||||
@@ -83,7 +83,7 @@ class FaultRecoveryTests(unittest.TestCase):
|
|||||||
self.assertEqual(outcome.virtual_time_s, 5.0)
|
self.assertEqual(outcome.virtual_time_s, 5.0)
|
||||||
|
|
||||||
def test_accepted_false_raises_and_state_untouched(self):
|
def test_accepted_false_raises_and_state_untouched(self):
|
||||||
client, faults, simulator = self._client([None])
|
client, _, _ = self._client([None])
|
||||||
with self.assertRaises(ProtocolError) as ctx:
|
with self.assertRaises(ProtocolError) as ctx:
|
||||||
client.measure((0, 0), 1) # 未 enter -> accepted=false
|
client.measure((0, 0), 1) # 未 enter -> accepted=false
|
||||||
self.assertEqual(ctx.exception.status, 200)
|
self.assertEqual(ctx.exception.status, 200)
|
||||||
@@ -103,7 +103,7 @@ class DeadlineBudgetTests(unittest.TestCase):
|
|||||||
now[0] += 300.0
|
now[0] += 300.0
|
||||||
self.assertAlmostEqual(client.real_remaining_s, 900.0)
|
self.assertAlmostEqual(client.real_remaining_s, 900.0)
|
||||||
|
|
||||||
def test_action_blocked_when_reserve_margin_cannot_be_kept(self):
|
def test_action_blocked_when_reserve_margin_low(self):
|
||||||
now = [0.0]
|
now = [0.0]
|
||||||
simulator = LocalSimulator([], robot_id="T001")
|
simulator = LocalSimulator([], robot_id="T001")
|
||||||
client = SimulatorClient("T001", transport=LocalSimTransport(simulator),
|
client = SimulatorClient("T001", transport=LocalSimTransport(simulator),
|
||||||
@@ -172,8 +172,8 @@ class PracticeReplayTests(unittest.TestCase):
|
|||||||
events = [dict(path=e["path"], request=e["request"], response=e["response"])
|
events = [dict(path=e["path"], request=e["request"], response=e["response"])
|
||||||
for e in self._events()]
|
for e in self._events()]
|
||||||
stats = summarize(events)
|
stats = summarize(events)
|
||||||
directions = [e for e in events
|
_directions = [e for e in events
|
||||||
if e["response"].get("measure_result") == "direction"]
|
if e["response"].get("measure_result") == "direction"]
|
||||||
self.assertGreater(stats["measures"], 0)
|
self.assertGreater(stats["measures"], 0)
|
||||||
self.assertEqual(stats["clears"], 0, "该次演练只采集观测,未调用 /clear")
|
self.assertEqual(stats["clears"], 0, "该次演练只采集观测,未调用 /clear")
|
||||||
|
|
||||||
|
|||||||
@@ -29,7 +29,7 @@ def test_runtime_code_never_imports_referee():
|
|||||||
"print('clean')\n"
|
"print('clean')\n"
|
||||||
)
|
)
|
||||||
result = subprocess.run([sys.executable, "-c", code], capture_output=True,
|
result = subprocess.run([sys.executable, "-c", code], capture_output=True,
|
||||||
text=True, cwd=ROOT)
|
text=True, cwd=ROOT, check=False)
|
||||||
assert result.returncode == 0, result.stderr
|
assert result.returncode == 0, result.stderr
|
||||||
assert "clean" in result.stdout
|
assert "clean" in result.stdout
|
||||||
|
|
||||||
|
|||||||
+3
-4
@@ -4,7 +4,6 @@ RL 依赖属于可选组 [dependency-groups] rl;未安装 numpy/gymnasium 时
|
|||||||
全部跳过(运行闭环不依赖它们)。
|
全部跳过(运行闭环不依赖它们)。
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import math
|
|
||||||
import unittest
|
import unittest
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
@@ -74,13 +73,13 @@ class TrainingEnvironmentTests(unittest.TestCase):
|
|||||||
self.assertIsInstance(reward, float)
|
self.assertIsInstance(reward, float)
|
||||||
self.assertFalse(terminated and truncated)
|
self.assertFalse(terminated and truncated)
|
||||||
|
|
||||||
def test_q4_directional_episode_reports_blocked_or_fallback(self):
|
def test_q4_directional_reports_blocked_or_fallback(self):
|
||||||
env = LocalizationEnv(problem=4, directional_probability=1.0)
|
env = LocalizationEnv(problem=4, directional_probability=1.0)
|
||||||
obs, info = env.reset(seed=7)
|
_, info = env.reset(seed=7)
|
||||||
self.assertTrue(info["source"]["directional"])
|
self.assertTrue(info["source"]["directional"])
|
||||||
terminated = truncated = False
|
terminated = truncated = False
|
||||||
for _ in range(env.max_episode_steps + 2):
|
for _ in range(env.max_episode_steps + 2):
|
||||||
obs, reward, terminated, truncated, info = env.step(
|
_, _, terminated, truncated, info = env.step(
|
||||||
env.action_space.sample())
|
env.action_space.sample())
|
||||||
if terminated or truncated:
|
if terminated or truncated:
|
||||||
break
|
break
|
||||||
|
|||||||
@@ -9,7 +9,7 @@ np = pytest.importorskip('numpy')
|
|||||||
pytest.importorskip('gymnasium')
|
pytest.importorskip('gymnasium')
|
||||||
|
|
||||||
from mcm26b.rl.env import LocalizationEnv
|
from mcm26b.rl.env import LocalizationEnv
|
||||||
from mcm26b.rl.features import action_mask, build_observation, candidate_actions
|
from mcm26b.rl.features import build_observation, candidate_actions
|
||||||
|
|
||||||
|
|
||||||
def test_training_observation_matches_runtime_channel_belief():
|
def test_training_observation_matches_runtime_channel_belief():
|
||||||
@@ -85,10 +85,12 @@ def test_bc_and_ppo_forward_mapping_agree():
|
|||||||
spec.loader.exec_module(module)
|
spec.loader.exec_module(module)
|
||||||
torch.set_num_threads(1)
|
torch.set_num_threads(1)
|
||||||
env = LocalizationEnv()
|
env = LocalizationEnv()
|
||||||
ppo = PPO('MlpPolicy',env,policy_kwargs={'net_arch':[128,128]},device='cpu',n_steps=16,batch_size=8)
|
ppo = PPO('MlpPolicy',env,policy_kwargs={'net_arch':[128,128]},device='cpu',
|
||||||
|
n_steps=16,batch_size=8)
|
||||||
bc = module.BCMlp()
|
bc = module.BCMlp()
|
||||||
state = bc.net.state_dict()
|
state = bc.net.state_dict()
|
||||||
ppo.policy.mlp_extractor.policy_net.load_state_dict({k:v for k,v in state.items() if not k.startswith('4.')})
|
ppo.policy.mlp_extractor.policy_net.load_state_dict(
|
||||||
|
{k:v for k,v in state.items() if not k.startswith('4.')})
|
||||||
ppo.policy.action_net.load_state_dict({'weight':state['4.weight'],'bias':state['4.bias']})
|
ppo.policy.action_net.load_state_dict({'weight':state['4.weight'],'bias':state['4.bias']})
|
||||||
x = torch.randn(10,401)
|
x = torch.randn(10,401)
|
||||||
torch.testing.assert_close(bc(x),ppo.policy.action_net(ppo.policy.mlp_extractor.policy_net(x)))
|
torch.testing.assert_close(bc(x),ppo.policy.action_net(ppo.policy.mlp_extractor.policy_net(x)))
|
||||||
|
|||||||
+290
-16
@@ -6,11 +6,16 @@
|
|||||||
- SB3 表格解析必须给每一行打上 ``total_timesteps`` 横轴(曾把表格行当独立
|
- SB3 表格解析必须给每一行打上 ``total_timesteps`` 横轴(曾把表格行当独立
|
||||||
点,曲线上出现重复的幽灵点);
|
点,曲线上出现重复的幽灵点);
|
||||||
- ``metric_offset``/``metric_end`` 游标契约(前端增量拉取依赖它);
|
- ``metric_offset``/``metric_end`` 游标契约(前端增量拉取依赖它);
|
||||||
- 用一段桩训练脚本真起子进程,验证启动/轮询/停止/退出码全链路。
|
- 用一段桩训练脚本真起子进程,验证启动/轮询/暂停/继续/停止/退出码全链路。
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
# pytest 用参数名注入 fixture:`stub_repo`/`client` 这类参数在函数体里不直接
|
||||||
|
# 引用也是正常用法(它们的作用是 setup/teardown),这里统一放行。
|
||||||
|
# pylint: disable=unused-argument
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import dataclasses
|
||||||
import http.client
|
import http.client
|
||||||
import json
|
import json
|
||||||
import sys
|
import sys
|
||||||
@@ -22,6 +27,7 @@ import pytest
|
|||||||
|
|
||||||
pytest.importorskip("mcm26b")
|
pytest.importorskip("mcm26b")
|
||||||
|
|
||||||
|
from mcm26b import process_control as pc
|
||||||
from mcm26b.training_ui import demo, server as td
|
from mcm26b.training_ui import demo, server as td
|
||||||
from mcm26b.training_ui.schema import (CONFIG_FIELDS, config_schema,
|
from mcm26b.training_ui.schema import (CONFIG_FIELDS, config_schema,
|
||||||
preset_payloads)
|
preset_payloads)
|
||||||
@@ -32,7 +38,7 @@ REPO = Path(__file__).resolve().parents[1]
|
|||||||
# 最后打印一行元数据。全部行为由参数控制,不依赖 torch/SB3。
|
# 最后打印一行元数据。全部行为由参数控制,不依赖 torch/SB3。
|
||||||
STUB_SCRIPT = r'''
|
STUB_SCRIPT = r'''
|
||||||
"""训练前端测试桩:模拟 SB3 verbose=1 表格 + --progress-json 输出。"""
|
"""训练前端测试桩:模拟 SB3 verbose=1 表格 + --progress-json 输出。"""
|
||||||
import argparse, json, signal, sys, time
|
import argparse, json, os, signal, sys, time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
parser = argparse.ArgumentParser()
|
parser = argparse.ArgumentParser()
|
||||||
@@ -41,12 +47,17 @@ parser.add_argument("--stage", type=int, default=1)
|
|||||||
parser.add_argument("--timesteps", type=int, default=96)
|
parser.add_argument("--timesteps", type=int, default=96)
|
||||||
parser.add_argument("--model", default="out/model.zip")
|
parser.add_argument("--model", default="out/model.zip")
|
||||||
parser.add_argument("--rollout-steps", type=int, default=32)
|
parser.add_argument("--rollout-steps", type=int, default=32)
|
||||||
parser.add_argument("--sleep", type=float, default=0.0)
|
# 默认 sleep 可由环境变量控制:HTTP 端到端用例无法直接追加命令行参数,
|
||||||
|
# 用小 sleep 拉长运行窗口,才能稳定地测"暂停/继续"。
|
||||||
|
parser.add_argument("--sleep", type=float,
|
||||||
|
default=float(os.environ.get("STUB_TRAIN_SLEEP", "0")))
|
||||||
parser.add_argument("--lines", type=int, default=4)
|
parser.add_argument("--lines", type=int, default=4)
|
||||||
parser.add_argument("--table-first", action="store_true")
|
parser.add_argument("--table-first", action="store_true")
|
||||||
parser.add_argument("--progress-json", action="store_true")
|
parser.add_argument("--progress-json", action="store_true")
|
||||||
parser.add_argument("--crash", action="store_true")
|
parser.add_argument("--crash", action="store_true")
|
||||||
parser.add_argument("--ignore-sigterm", action="store_true")
|
parser.add_argument("--ignore-sigterm", action="store_true")
|
||||||
|
parser.add_argument("--ignore-control", action="store_true")
|
||||||
|
parser.add_argument("--control-file", default=None)
|
||||||
args, _extra = parser.parse_known_args()
|
args, _extra = parser.parse_known_args()
|
||||||
|
|
||||||
model = Path(args.model)
|
model = Path(args.model)
|
||||||
@@ -65,6 +76,49 @@ print("NORMAL-LOG-LINE")
|
|||||||
sys.stdout.flush()
|
sys.stdout.flush()
|
||||||
|
|
||||||
|
|
||||||
|
def read_control():
|
||||||
|
if not args.control_file or args.ignore_control:
|
||||||
|
return {}
|
||||||
|
try:
|
||||||
|
with open(args.control_file, encoding="utf-8") as handle:
|
||||||
|
return json.load(handle)
|
||||||
|
except Exception:
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
def control_pause_or_stop(step):
|
||||||
|
"""返回 True 表示应退出;暂停时阻塞到继续或停止(模拟协作式暂停)。"""
|
||||||
|
state = read_control()
|
||||||
|
if state.get("stop"):
|
||||||
|
print(json.dumps({"event": "stop_requested", "signal": "control_file"}), flush=True)
|
||||||
|
return True
|
||||||
|
if state.get("paused"):
|
||||||
|
print(json.dumps({"event": "paused", "timesteps": step}), flush=True)
|
||||||
|
while True:
|
||||||
|
time.sleep(0.05)
|
||||||
|
state = read_control()
|
||||||
|
if state.get("stop"):
|
||||||
|
print(json.dumps({"event": "stop_requested", "signal": "control_file"}),
|
||||||
|
flush=True)
|
||||||
|
return True
|
||||||
|
if not state.get("paused"):
|
||||||
|
break
|
||||||
|
print(json.dumps({"event": "resumed", "timesteps": step}), flush=True)
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def sleep_with_control(seconds, step):
|
||||||
|
"""把长 sleep 切成 0.1s 片,保证停止/暂停请求能被及时响应。"""
|
||||||
|
deadline = time.monotonic() + max(0.0, seconds)
|
||||||
|
while True:
|
||||||
|
if control_pause_or_stop(step):
|
||||||
|
raise SystemExit(0)
|
||||||
|
remaining = deadline - time.monotonic()
|
||||||
|
if remaining <= 0:
|
||||||
|
return
|
||||||
|
time.sleep(min(0.1, remaining))
|
||||||
|
|
||||||
|
|
||||||
def render_table(step, fps):
|
def render_table(step, fps):
|
||||||
return (
|
return (
|
||||||
"-----------------------------\n"
|
"-----------------------------\n"
|
||||||
@@ -72,16 +126,15 @@ def render_table(step, fps):
|
|||||||
"| ep_len_mean | 9.67 |\n"
|
"| ep_len_mean | 9.67 |\n"
|
||||||
"| ep_rew_mean | -3.24 |\n"
|
"| ep_rew_mean | -3.24 |\n"
|
||||||
"| time/ | |\n"
|
"| time/ | |\n"
|
||||||
"| fps | %-8d |\n"
|
f"| fps | {fps:<8d} |\n"
|
||||||
"| total_timesteps | %-8d |\n"
|
f"| total_timesteps | {step:<8d} |\n"
|
||||||
"-----------------------------\n" % (fps, step)
|
"-----------------------------\n"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
for index in range(args.lines):
|
for index in range(args.lines):
|
||||||
if args.sleep:
|
|
||||||
time.sleep(args.sleep)
|
|
||||||
step = 32 * (index + 1)
|
step = 32 * (index + 1)
|
||||||
|
sleep_with_control(args.sleep, step)
|
||||||
fps = 17 + index
|
fps = 17 + index
|
||||||
progress = {
|
progress = {
|
||||||
"event": "progress", "timesteps": step, "n_updates": index + 1,
|
"event": "progress", "timesteps": step, "n_updates": index + 1,
|
||||||
@@ -153,7 +206,7 @@ def test_schema_matches_train_config_defaults():
|
|||||||
assert field["default"] == getattr(defaults, field["key"]), \
|
assert field["default"] == getattr(defaults, field["key"]), \
|
||||||
f"{field['key']} 默认值与后端不一致"
|
f"{field['key']} 默认值与后端不一致"
|
||||||
exposed = {field["key"] for field in schema["fields"]}
|
exposed = {field["key"] for field in schema["fields"]}
|
||||||
missing = set(defaults.__dataclass_fields__) - exposed
|
missing = {field.name for field in dataclasses.fields(defaults)} - exposed
|
||||||
assert not missing, f"TrainConfig 字段未暴露给界面: {sorted(missing)}"
|
assert not missing, f"TrainConfig 字段未暴露给界面: {sorted(missing)}"
|
||||||
groups = {group["key"] for group in schema["groups"]}
|
groups = {group["key"] for group in schema["groups"]}
|
||||||
for field in schema["fields"]:
|
for field in schema["fields"]:
|
||||||
@@ -227,9 +280,11 @@ class _FakeJob:
|
|||||||
self._last_table_steps = None
|
self._last_table_steps = None
|
||||||
self._last_json_steps = None
|
self._last_json_steps = None
|
||||||
self._last_monitor_sig = None
|
self._last_monitor_sig = None
|
||||||
|
self.status = "running"
|
||||||
self.config = make_config()
|
self.config = make_config()
|
||||||
|
|
||||||
_on_line = td.TrainingJob._on_line
|
_on_line = td.TrainingJob._on_line
|
||||||
|
_sync_control_events = td.TrainingJob._sync_control_events
|
||||||
_parse_progress_json = td.TrainingJob._parse_progress_json
|
_parse_progress_json = td.TrainingJob._parse_progress_json
|
||||||
_parse_sb3_table = td.TrainingJob._parse_sb3_table
|
_parse_sb3_table = td.TrainingJob._parse_sb3_table
|
||||||
_flush_table = td.TrainingJob._flush_table
|
_flush_table = td.TrainingJob._flush_table
|
||||||
@@ -274,9 +329,9 @@ def test_sb3_table_with_separators_emits_one_point_per_table():
|
|||||||
for index, steps in enumerate((32, 64, 96)):
|
for index, steps in enumerate((32, 64, 96)):
|
||||||
job._on_line("-----------------------------")
|
job._on_line("-----------------------------")
|
||||||
job._on_line("| rollout/ | |")
|
job._on_line("| rollout/ | |")
|
||||||
job._on_line("| ep_rew_mean | %.2f |" % (-3.0 + index))
|
job._on_line(f"| ep_rew_mean | {-3.0 + index:.2f} |")
|
||||||
job._on_line("| time/ | |")
|
job._on_line("| time/ | |")
|
||||||
job._on_line("| total_timesteps | %d |" % steps)
|
job._on_line(f"| total_timesteps | {steps} |")
|
||||||
job._on_line("-----------------------------")
|
job._on_line("-----------------------------")
|
||||||
job._flush_table()
|
job._flush_table()
|
||||||
assert [point["timesteps"] for point in job.metrics] == [32, 64, 96], list(job.metrics)
|
assert [point["timesteps"] for point in job.metrics] == [32, 64, 96], list(job.metrics)
|
||||||
@@ -476,7 +531,7 @@ def test_job_lifecycle_with_stub_script(stub_repo):
|
|||||||
def test_job_reports_nonzero_exit_code(stub_repo):
|
def test_job_reports_nonzero_exit_code(stub_repo):
|
||||||
config = make_config(model="output/hrl/ui/test/crash/q3.zip")
|
config = make_config(model="output/hrl/ui/test/crash/q3.zip")
|
||||||
job = td.TrainingJob(config)
|
job = td.TrainingJob(config)
|
||||||
job.command = td.build_command(config) + ["--crash"]
|
job.command = list(job.command) + ["--crash"]
|
||||||
job.start()
|
job.start()
|
||||||
assert wait_until(lambda: job.status in ("exited", "failed"), timeout=30)
|
assert wait_until(lambda: job.status in ("exited", "failed"), timeout=30)
|
||||||
assert job.status == "failed"
|
assert job.status == "failed"
|
||||||
@@ -484,15 +539,200 @@ def test_job_reports_nonzero_exit_code(stub_repo):
|
|||||||
assert job.snapshot(log_tail=5)["process"]["tone"] == "bad"
|
assert job.snapshot(log_tail=5)["process"]["tone"] == "bad"
|
||||||
|
|
||||||
|
|
||||||
def test_stop_sends_sigterm_and_marks_stopped(stub_repo):
|
def test_job_command_carries_control_file(stub_repo):
|
||||||
|
"""控制文件路径必须随命令下发,否则暂停/停止退化成超时强杀。"""
|
||||||
|
config = make_config(model="output/hrl/ui/test/control/q3.zip")
|
||||||
|
job = td.TrainingJob(config, control_dir=Path(stub_repo).parent)
|
||||||
|
assert "--control-file" in job.command
|
||||||
|
assert str(job.control_path) in job.command
|
||||||
|
assert job.control_path.parent == Path(stub_repo).parent
|
||||||
|
|
||||||
|
|
||||||
|
def test_stop_gracefully_marks_stopped_without_forced_kill(stub_repo):
|
||||||
config = make_config(model="output/hrl/ui/test/stop/q3.zip")
|
config = make_config(model="output/hrl/ui/test/stop/q3.zip")
|
||||||
job = td.TrainingJob(config)
|
job = td.TrainingJob(config)
|
||||||
job.command = td.build_command(config) + ["--sleep", "5", "--lines", "50"]
|
job.command = list(job.command) + ["--sleep", "5", "--lines", "50"]
|
||||||
job.start()
|
job.start()
|
||||||
assert wait_until(lambda: job.status == "running", timeout=10)
|
assert wait_until(lambda: job.status == "running", timeout=10)
|
||||||
assert job.stop() is True
|
assert job.stop() is True
|
||||||
assert wait_until(lambda: job.status in ("stopped", "failed"), timeout=15)
|
assert wait_until(lambda: job.status in ("stopped", "failed"), timeout=15)
|
||||||
assert job.status == "stopped"
|
assert job.status == "stopped"
|
||||||
|
# 桩脚本读控制文件后主动退出:不应走到强杀分支
|
||||||
|
assert job.forced_kill is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_pause_and_resume_via_control_file(stub_repo):
|
||||||
|
config = make_config(model="output/hrl/ui/test/pause/q3.zip")
|
||||||
|
job = td.TrainingJob(config)
|
||||||
|
job.command = list(job.command) + ["--sleep", "0.2", "--lines", "60"]
|
||||||
|
job.start()
|
||||||
|
assert wait_until(lambda: job.status == "running", timeout=10)
|
||||||
|
assert job.pause() is True
|
||||||
|
# 训练脚本回显 paused 事件;状态由 _sync_control_events 校正
|
||||||
|
assert wait_until(
|
||||||
|
lambda: any('"event": "paused"' in line for line in job.snapshot()["log_lines"]),
|
||||||
|
timeout=10), "训练脚本未回显 paused 事件"
|
||||||
|
assert job.status == "paused"
|
||||||
|
paused_metrics = len(job.snapshot()["metrics"])
|
||||||
|
time.sleep(0.6)
|
||||||
|
assert len(job.snapshot()["metrics"]) <= paused_metrics + 1, "暂停后仍在产出指标"
|
||||||
|
assert job.resume() is True
|
||||||
|
assert wait_until(
|
||||||
|
lambda: any('"event": "resumed"' in line for line in job.snapshot()["log_lines"]),
|
||||||
|
timeout=10), "训练脚本未回显 resumed 事件"
|
||||||
|
assert wait_until(lambda: job.status == "running", timeout=5)
|
||||||
|
assert job.stop() is True
|
||||||
|
assert wait_until(lambda: job.status in ("stopped", "failed"), timeout=15)
|
||||||
|
assert job.status == "stopped"
|
||||||
|
|
||||||
|
|
||||||
|
def test_stop_forced_kill_when_control_ignored(stub_repo, monkeypatch):
|
||||||
|
"""训练进程不读控制文件时必须超时强杀,并留下恢复指引。"""
|
||||||
|
monkeypatch.setattr(td, "STOP_GRACE_S", 0.5)
|
||||||
|
config = make_config(model="output/hrl/ui/test/stubborn/q3.zip")
|
||||||
|
job = td.TrainingJob(config)
|
||||||
|
job.command = list(job.command) + ["--ignore-control", "--sleep", "5", "--lines", "50"]
|
||||||
|
job.start()
|
||||||
|
assert wait_until(lambda: job.status == "running", timeout=10)
|
||||||
|
# 造一份"磁盘上已有滚动恢复点"的现场,强杀后应被记录进恢复指引
|
||||||
|
ckpt_dir = job.config.model_path.parent / "checkpoints"
|
||||||
|
ckpt_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
latest = ckpt_dir / "q3_ckpt_latest.zip"
|
||||||
|
latest.write_bytes(b"PK\x03\x04fake")
|
||||||
|
assert job.stop() is True
|
||||||
|
assert wait_until(lambda: job.status in ("stopped", "failed"), timeout=20)
|
||||||
|
assert job.forced_kill is True
|
||||||
|
assert job.snapshot(log_tail=5)["forced_kill"] is True
|
||||||
|
marker = json.loads(
|
||||||
|
(job.config.model_path.parent / "forced_stop.json").read_text(encoding="utf-8"))
|
||||||
|
assert marker["event"] == "forced_stop"
|
||||||
|
assert marker["reason"] == "grace_timeout"
|
||||||
|
assert marker["recovery"]["model"] == str(latest)
|
||||||
|
assert marker["recovery"]["can_resume"] is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_discover_recovery_prefers_resume_hint(tmp_path):
|
||||||
|
run_dir = tmp_path / "run"
|
||||||
|
(run_dir / "checkpoints").mkdir(parents=True)
|
||||||
|
latest = run_dir / "checkpoints" / "q3_ckpt_latest.zip"
|
||||||
|
latest.write_bytes(b"PK\x03\x04fake")
|
||||||
|
final = run_dir / "q3.zip"
|
||||||
|
final.write_bytes(b"PK\x03\x04fake")
|
||||||
|
(run_dir / "resume.json").write_text(json.dumps({
|
||||||
|
"model": str(latest), "timesteps": 128, "reason": "interval",
|
||||||
|
"created_unix_s": 5.0}), encoding="utf-8")
|
||||||
|
state = td.discover_recovery(run_dir, final)
|
||||||
|
assert state["can_resume"] is True
|
||||||
|
assert state["model"] == str(latest)
|
||||||
|
assert state["timesteps"] == 128
|
||||||
|
assert state["source"] == "resume.json"
|
||||||
|
|
||||||
|
|
||||||
|
def test_discover_recovery_falls_back_to_scan_on_dangling_hint(tmp_path):
|
||||||
|
run_dir = tmp_path / "run"
|
||||||
|
(run_dir / "checkpoints").mkdir(parents=True)
|
||||||
|
latest = run_dir / "checkpoints" / "q3_ckpt_latest.zip"
|
||||||
|
latest.write_bytes(b"PK\x03\x04fake")
|
||||||
|
# 悬空指针(目标文件已不在)必须回退到扫描,而不是给出一个打不开的路径
|
||||||
|
(run_dir / "resume.json").write_text(
|
||||||
|
json.dumps({"model": str(run_dir / "gone.zip")}), encoding="utf-8")
|
||||||
|
state = td.discover_recovery(run_dir, run_dir / "q3.zip")
|
||||||
|
assert state["can_resume"] is True
|
||||||
|
assert state["model"] == str(latest)
|
||||||
|
assert state["source"] == "scan"
|
||||||
|
|
||||||
|
|
||||||
|
def test_discover_recovery_with_nothing_on_disk(tmp_path):
|
||||||
|
state = td.discover_recovery(tmp_path, tmp_path / "q3.zip")
|
||||||
|
assert state["can_resume"] is False and state["model"] is None
|
||||||
|
assert state["reason"] == "none"
|
||||||
|
|
||||||
|
|
||||||
|
def test_command_carries_checkpoint_and_recovery_flags():
|
||||||
|
config = make_config(model="output/x/q3.zip", recovery_every=100,
|
||||||
|
checkpoint_every=500, checkpoint_keep=2)
|
||||||
|
joined = " ".join(td.build_command(config))
|
||||||
|
assert "--recovery-every 100" in joined
|
||||||
|
assert "--checkpoint-every 500" in joined
|
||||||
|
assert "--checkpoint-keep 2" in joined
|
||||||
|
|
||||||
|
|
||||||
|
def test_config_rejects_bad_recovery_settings():
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
make_config(recovery_every=-1)
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
make_config(checkpoint_every=-1)
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
make_config(checkpoint_keep=0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_pause_resume_return_false_without_live_process(stub_repo):
|
||||||
|
config = make_config(model="output/hrl/ui/test/nolive/q3.zip")
|
||||||
|
job = td.TrainingJob(config)
|
||||||
|
assert job.pause() is False and job.resume() is False
|
||||||
|
job.start()
|
||||||
|
assert wait_until(lambda: job.status in ("exited", "failed"), timeout=30)
|
||||||
|
assert job.pause() is False and job.resume() is False
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------- 跨平台进程控制
|
||||||
|
def test_control_file_roundtrip_and_corruption(tmp_path):
|
||||||
|
path = tmp_path / "control.json"
|
||||||
|
assert pc.read_control(path) == pc.ControlState() # 缺失 = 无指令
|
||||||
|
pc.write_control(path, paused=True)
|
||||||
|
assert pc.read_control(path).paused is True
|
||||||
|
pc.write_control(path, stop=True)
|
||||||
|
state = pc.read_control(path)
|
||||||
|
assert state.stop is True and state.paused is False
|
||||||
|
path.write_text("{ 半截 JSON", encoding="utf-8")
|
||||||
|
assert pc.read_control(path) == pc.ControlState() # 损坏 = 无指令
|
||||||
|
|
||||||
|
|
||||||
|
def test_spawn_kwargs_isolates_child_process_group(monkeypatch):
|
||||||
|
monkeypatch.setattr(pc, "IS_WINDOWS", False)
|
||||||
|
assert pc.spawn_kwargs() == {"start_new_session": True}
|
||||||
|
monkeypatch.setattr(pc, "IS_WINDOWS", True)
|
||||||
|
kwargs = pc.spawn_kwargs()
|
||||||
|
# Windows 上 start_new_session 被 CPython 静默忽略,绝不能再用它
|
||||||
|
assert "start_new_session" not in kwargs
|
||||||
|
if kwargs:
|
||||||
|
assert kwargs["creationflags"] == pc._CREATE_NEW_PROCESS_GROUP
|
||||||
|
|
||||||
|
|
||||||
|
def test_hard_kill_on_windows_uses_taskkill(monkeypatch):
|
||||||
|
commands = []
|
||||||
|
|
||||||
|
class _FakeProcess:
|
||||||
|
pid = 4242
|
||||||
|
|
||||||
|
def kill(self): # pragma: no cover - Windows 分支不应走到这里
|
||||||
|
commands.append("Popen.kill")
|
||||||
|
|
||||||
|
monkeypatch.setattr(pc, "IS_WINDOWS", True)
|
||||||
|
monkeypatch.setattr(pc.subprocess, "run",
|
||||||
|
lambda argv, **kwargs: commands.append(argv))
|
||||||
|
pc._hard_kill_tree(_FakeProcess())
|
||||||
|
assert commands and commands[0][0] == "taskkill"
|
||||||
|
assert "/PID" in commands[0] and "4242" in commands[0]
|
||||||
|
|
||||||
|
|
||||||
|
def test_hard_kill_without_killpg_does_not_raise(monkeypatch):
|
||||||
|
"""回归:Windows 上 os 没有 killpg/getpgid,旧代码会抛 AttributeError。"""
|
||||||
|
killed = []
|
||||||
|
|
||||||
|
class _FakeProcess:
|
||||||
|
pid = 4242
|
||||||
|
|
||||||
|
def kill(self):
|
||||||
|
killed.append(True)
|
||||||
|
|
||||||
|
class _PosixlessOS:
|
||||||
|
"""模拟 Windows:只有 getpid 之类,没有 killpg/getpgid。"""
|
||||||
|
|
||||||
|
monkeypatch.setattr(pc, "IS_WINDOWS", False)
|
||||||
|
monkeypatch.setattr(pc, "os", _PosixlessOS())
|
||||||
|
pc._hard_kill_tree(_FakeProcess())
|
||||||
|
assert killed == [True]
|
||||||
|
|
||||||
|
|
||||||
def test_reader_flushes_truncated_table_on_exit(stub_repo):
|
def test_reader_flushes_truncated_table_on_exit(stub_repo):
|
||||||
@@ -628,8 +868,42 @@ def test_http_start_poll_stop(client):
|
|||||||
|
|
||||||
status, body = call(port, "POST", "/api/train/stop", {})
|
status, body = call(port, "POST", "/api/train/stop", {})
|
||||||
assert status == 200 and body["stopped"] is True
|
assert status == 200 and body["stopped"] is True
|
||||||
|
# 等训练进程真正退出后再点一次:应返回 False 而不是报错
|
||||||
|
assert wait_until(
|
||||||
|
lambda: call(port, "GET", "/api/train/state?log_tail=5")[1]["job"]["status"]
|
||||||
|
in ("stopped", "failed"), timeout=15)
|
||||||
status, body = call(port, "POST", "/api/train/stop", {})
|
status, body = call(port, "POST", "/api/train/stop", {})
|
||||||
assert status == 200 and body["stopped"] is False # 已结束,返回 False 而不是报错
|
assert status == 200 and body["stopped"] is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_http_pause_resume_and_state(client, monkeypatch):
|
||||||
|
monkeypatch.setenv("STUB_TRAIN_SLEEP", "0.6")
|
||||||
|
_ui, port = client
|
||||||
|
status, body = call(port, "POST", "/api/train/start",
|
||||||
|
{"model": "output/hrl/ui/test/http-pause/q3.zip",
|
||||||
|
"timesteps": 4096, "rollout_steps": 32, "batch_size": 32})
|
||||||
|
assert status == 200
|
||||||
|
|
||||||
|
def status_now():
|
||||||
|
return call(port, "GET", "/api/train/state?log_tail=20")[1]["job"]["status"]
|
||||||
|
|
||||||
|
def is_running():
|
||||||
|
return status_now() == "running"
|
||||||
|
|
||||||
|
assert wait_until(is_running, timeout=10)
|
||||||
|
status, body = call(port, "POST", "/api/train/pause", {})
|
||||||
|
assert status == 200 and body["paused"] is True
|
||||||
|
assert wait_until(lambda: status_now() == "paused", timeout=10), status_now()
|
||||||
|
# 暂停中仍视为"有任务在跑":不允许再开一局
|
||||||
|
status, body = call(port, "POST", "/api/train/start",
|
||||||
|
{"model": "output/hrl/ui/test/http-pause2/q3.zip"})
|
||||||
|
assert status == 409 and "已有训练任务" in body["error"]
|
||||||
|
status, body = call(port, "POST", "/api/train/resume", {})
|
||||||
|
assert status == 200 and body["resumed"] is True
|
||||||
|
assert wait_until(is_running, timeout=10), status_now()
|
||||||
|
status, body = call(port, "POST", "/api/train/stop", {})
|
||||||
|
assert status == 200 and body["stopped"] is True
|
||||||
|
assert wait_until(lambda: status_now() in ("stopped", "failed"), timeout=15)
|
||||||
|
|
||||||
|
|
||||||
def test_http_rejects_second_job_while_running(client):
|
def test_http_rejects_second_job_while_running(client):
|
||||||
@@ -661,7 +935,7 @@ def test_http_binds_loopback_only(client):
|
|||||||
|
|
||||||
|
|
||||||
def test_port_conflict_reports_actionable_error(client):
|
def test_port_conflict_reports_actionable_error(client):
|
||||||
ui, port = client
|
_ui, port = client
|
||||||
with pytest.raises(OSError) as excinfo:
|
with pytest.raises(OSError) as excinfo:
|
||||||
td.TrainingUI(ui_port=port)
|
td.TrainingUI(ui_port=port)
|
||||||
assert "ui-port" in str(excinfo.value)
|
assert "ui-port" in str(excinfo.value)
|
||||||
|
|||||||
@@ -36,6 +36,7 @@ def test_training_ui_frontend_logic(tmp_path):
|
|||||||
|
|
||||||
result = subprocess.run(
|
result = subprocess.run(
|
||||||
["node", str(HARNESS)], capture_output=True, text=True, timeout=120,
|
["node", str(HARNESS)], capture_output=True, text=True, timeout=120,
|
||||||
|
check=False,
|
||||||
env={**os.environ, "REPO": str(REPO),
|
env={**os.environ, "REPO": str(REPO),
|
||||||
"DEMO_STATE": str(scenarios_path), "SCHEMA": str(schema_path)})
|
"DEMO_STATE": str(scenarios_path), "SCHEMA": str(schema_path)})
|
||||||
assert result.returncode == 0, \
|
assert result.returncode == 0, \
|
||||||
|
|||||||
Reference in New Issue
Block a user