holy fuking shit
This commit is contained in:
@@ -0,0 +1,261 @@
|
||||
# 训练前端 Windows 兼容性与「暂停」审计(结论 + 修复方案)
|
||||
|
||||
> 审计对象:`src/mcm26b/training_ui/`、`src/mcm26b/process_control.py`、
|
||||
> `scripts/train_hrl_high.py`。
|
||||
> 触发问题:「无法暂停」「频繁出现 os 无 killpg 等」。
|
||||
> 结论一句话:**这两个现象同源——训练前端的停止/暂停路径是纯 POSIX 设计;
|
||||
> Windows 上 `os.killpg` 根本不存在,异常又没被兜住,所以点了停止等于没点。**
|
||||
|
||||
---
|
||||
|
||||
## 0. 结论摘要
|
||||
|
||||
| # | 现象 | 根因 | 严重度 |
|
||||
| --- | --- | --- | --- |
|
||||
| 1 | 点「停止训练」返回 500 / 无效果,训练照跑,于是「无法暂停」 | `TrainingJob.stop()` 直接调用 `os.killpg(os.getpgid(...))`;这两个符号**只存在于 POSIX**,Windows 上抛 `AttributeError`,而旧 `except` 只接 `ProcessLookupError/PermissionError` | 阻断 |
|
||||
| 2 | 控制台/日志反复出现 `module 'os' has no attribute 'killpg'` | 同上:错误最终落到 `_UIHandler.do_POST` 的兜底 `except Exception`,返回 500 并带出 `AttributeError` 文本;每点一次复现一次 | 阻断 |
|
||||
| 3 | 即使把 killpg 换成 `terminate()`,停止后模型/元数据仍然丢失(Windows 上) | Windows 的 `os.kill(pid, SIGTERM)` 语义是 `TerminateProcess` **硬杀**,训练脚本里"收到信号后在 step 边界保存"的处理器不会执行 | 高 |
|
||||
| 4 | 界面根本没有「暂停」能力(POSIX 也没有) | 原设计只有 start/stop;且 `SIGSTOP/SIGCONT` 在 Windows 上不存在,**原生暂停在 Windows 上无法实现**,必须改成协作式 | 高 |
|
||||
| 5 | 子进程进程组隔离在 Windows 上失效 | `Popen(start_new_session=True)` 在 Windows 被**静默忽略**(CPython 的 Windows `_execute_child` 形参名就是 `unused_start_new_session`) | 中 |
|
||||
| 6 | subproc worker 在 Windows 下可能抢在主进程收尾前退出 | worker 只忽略 `SIGTERM/SIGINT`,没处理 Windows 的 `SIGBREAK`(`CTRL_BREAK_EVENT` 的落点) | 中 |
|
||||
|
||||
**修复原则**:把"暂停/继续/停止"从**信号语义**改成**控制文件语义**。
|
||||
控制文件在 POSIX 与 Windows 上都只是普通文件 IO,行为完全一致;信号与强杀退居
|
||||
兜底。这样 Windows 上暂停、继续、优雅停止、落盘保证全部成立。
|
||||
|
||||
---
|
||||
|
||||
## 1. 失败链复现(旧代码)
|
||||
|
||||
```
|
||||
用户点「停止训练」
|
||||
└─ POST /api/train/stop
|
||||
└─ TrainingUI.stop() → TrainingJob.stop()
|
||||
└─ os.killpg(os.getpgid(process.pid), signal.SIGTERM)
|
||||
├─ POSIX :成功,进程组收到 SIGTERM,脚本优雅收尾
|
||||
└─ Windows:AttributeError: module 'os' has no attribute 'killpg'
|
||||
↑ 只 except (ProcessLookupError, PermissionError),接不住
|
||||
└─ _UIHandler.do_POST 兜底 except Exception → HTTP 500
|
||||
└─ 训练子进程完全没收到任何指令,继续跑 → 「无法暂停 / 停止没反应」
|
||||
```
|
||||
|
||||
关键代码(修复前,`src/mcm26b/training_ui/server.py`):
|
||||
|
||||
```python
|
||||
def stop(self) -> bool:
|
||||
with self._lock:
|
||||
process = self.process
|
||||
if process is None or process.poll() is not None:
|
||||
return False
|
||||
try:
|
||||
os.killpg(os.getpgid(process.pid), signal.SIGTERM)
|
||||
except (ProcessLookupError, PermissionError):
|
||||
process.terminate()
|
||||
self.status = "stopping"
|
||||
return True
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. 根因逐条(含证据)
|
||||
|
||||
### 2.1 `os.killpg` / `os.getpgid` 是 POSIX-only
|
||||
|
||||
CPython 只在 `posix` 构建里注册这两个函数;Windows 的 `os` 模块没有它们。
|
||||
访问会得到 `AttributeError`(不是 `OSError` 子类),所以旧 `except` 接不住。
|
||||
|
||||
### 2.2 `start_new_session=True` 在 Windows 被静默忽略
|
||||
|
||||
`subprocess.Popen` 接受该参数,但 Windows 的 `_execute_child` 形参名是
|
||||
`unused_start_new_session`,也就是**根本不读**。要隔离进程组,Windows 必须传
|
||||
`creationflags=CREATE_NEW_PROCESS_GROUP`。
|
||||
|
||||
### 2.3 Windows 的 `SIGTERM` 是硬杀
|
||||
|
||||
`os.kill(pid, sig)` 在 Windows 上只有 `CTRL_C_EVENT` / `CTRL_BREAK_EVENT` 是
|
||||
"软"信号;**任何其它值**(含 `SIGTERM`)都会走 `TerminateProcess`。因此
|
||||
`scripts/train_hrl_high.py` 里的:
|
||||
|
||||
```python
|
||||
for _signum in (signal.SIGTERM, signal.SIGINT): # 旧代码
|
||||
signal.signal(_signum, _on_stop_signal)
|
||||
```
|
||||
|
||||
在 Windows 上形同虚设——处理器注册了,但没人会用它。这也解释了为什么"停止后
|
||||
模型没保存"。
|
||||
|
||||
### 2.4 Windows 没有 `SIGSTOP`,暂停无法用信号实现
|
||||
|
||||
暂停(挂起/继续)在 POSIX 上可以用 `SIGSTOP/SIGCONT`,但 Windows 既无这两个
|
||||
信号,也无等价的"无侵入挂起"用户态 API(真正的挂起要 `SuspendThread`/调试 API)。
|
||||
所以**任何跨平台暂停都必须是协作式的**:由训练进程自己在 step 边界检查标志并
|
||||
让出执行。本项目原先没有这条通道,界面也没有暂停按钮。
|
||||
|
||||
### 2.5 `SIGBREAK` 未处理
|
||||
|
||||
Windows 的 `CTRL_BREAK_EVENT`(发给进程组的软信号)在 Python 里落到
|
||||
`signal.SIGBREAK`。旧的 handler 安装与 worker 屏蔽列表都只含
|
||||
`SIGTERM/SIGINT`,Windows 下等于裸奔。
|
||||
|
||||
### 2.6 异常类型没兜住
|
||||
|
||||
`except (ProcessLookupError, PermissionError)` 漏掉 `AttributeError`、以及
|
||||
更一般的 `OSError`;`_hard_kill` 里 `signal.SIGKILL` 在 Windows 上也不存在
|
||||
(`getattr(signal, "SIGKILL", None) is None`),同样会 `AttributeError`。
|
||||
|
||||
---
|
||||
|
||||
## 3. 修复方案(已实施)
|
||||
|
||||
### 3.1 新增 `src/mcm26b/process_control.py`(零第三方依赖)
|
||||
|
||||
| 能力 | 实现 | 作用 |
|
||||
| --- | --- | --- |
|
||||
| 子进程隔离 | `spawn_kwargs()`:POSIX `start_new_session=True`;Windows `creationflags=CREATE_NEW_PROCESS_GROUP` | 前端 Ctrl+C 不误伤训练;Windows 也能组杀进程树 |
|
||||
| 控制协议读写 | `write_control()` / `read_control()`:JSON + 同目录临时文件 + `os.replace` 原子替换;损坏/缺失一律视为"无指令" | 两端统一的暂停/停止通道 |
|
||||
| 强杀兜底 | `terminate_process_tree(process, grace_s)`:先等宽限期;超时后 POSIX 走 `killpg(SIGKILL)`、Windows 走 `taskkill /F /T /PID`;所有 POSIX 符号用 `getattr` 探测 | 彻底移除 `os.killpg` 无条件调用;返回"是否动用了强杀" |
|
||||
|
||||
`IS_WINDOWS` 是运行期常量而非导入期分支,便于测试直接 monkeypatch 模拟 Windows。
|
||||
|
||||
### 3.2 控制文件协议
|
||||
|
||||
路径由前端生成并随命令下发:`--control-file <abs>`(默认落在
|
||||
`output/hrl/ui/control/<job_id>.control.json`,`output/` 已 gitignore)。
|
||||
|
||||
```json
|
||||
{"paused": false, "stop": false, "seq": 1712345678901234567, "ts": 1712345678.9}
|
||||
```
|
||||
|
||||
- 前端**只写**:`pause()` → `paused=true`;`resume()` → `paused=false`;
|
||||
`stop()` → `stop=true`(同时清零 paused,保证阻塞中的暂停循环能被唤醒)。
|
||||
- 训练脚本**只读**:`_ControlPoll` 每 32 个 step 抽样读一次盘(避免每 step 一次
|
||||
文件 IO);暂停时在 `_on_step` 内阻塞轮询(`poll_s=0.2`),进程存活、显存仍占用,
|
||||
但不消耗算力。
|
||||
- 墙钟上限 `--max-wall-s` **扣除累计暂停时长**,避免"挂起一小时回来直接判超时"。
|
||||
|
||||
### 3.3 训练脚本 `scripts/train_hrl_high.py`
|
||||
|
||||
- 新增 `--control-file`;`_ControlPoll`(纯标准库、可单测)+ 薄薄的
|
||||
`_ControlChannel(BaseCallback)` 接到 SB3 回调链**最前面**。
|
||||
- 收尾信号集合扩为 `_stop_signals()`:POSIX `SIGTERM/SIGINT`;Windows 追加
|
||||
`SIGBREAK`。worker 屏蔽同一集合,保证主进程 `vec_env.close()` 之前 worker 不先退。
|
||||
- 落盘保证不变:停止后仍写最终模型 + 元数据,`stopped_by_signal=true`,
|
||||
`stop_signal` 记 `control_file` 或信号名;异常仍落 `*_emergency.zip`。
|
||||
|
||||
### 3.4 训练前端 `server.py` / `static/`
|
||||
|
||||
- 新增 `POST /api/train/pause`、`POST /api/train/resume`;`stop` 语义改为
|
||||
"写控制文件 → 宽限期内自行退出;超时强杀",强杀在**独立看门狗线程**里做,
|
||||
HTTP 请求不会被 15s 等待卡住。
|
||||
- 状态机新增 `paused`;`_sync_control_events()` 用训练脚本回显的
|
||||
`paused/resumed/stop_requested` 事件校正状态。
|
||||
- 快照新增 `forced_kill`,前端横幅区分"优雅停止"与"超时强杀"。
|
||||
- 前端新增「暂停训练 / 继续训练」按钮、暂停横幅;停止确认文案更新。
|
||||
|
||||
### 3.5 停止/暂停时序
|
||||
|
||||
```
|
||||
pause: UI 写 {paused:true} ──► 脚本 _on_step 读到 ──► 回显 paused,阻塞轮询
|
||||
resume: UI 写 {paused:false} ─► 暂停循环退出 ────────► 回显 resumed,继续训练
|
||||
stop: UI 写 {stop:true} ────► 脚本 step 边界收尾落盘 ──► 退出码 0
|
||||
│
|
||||
└─(宽限 15s 未退)─► terminate_process_tree 强杀;forced_kill=true
|
||||
```
|
||||
|
||||
### 3.6 急停(强杀)也必须可恢复
|
||||
|
||||
宽限期一到就强杀,训练脚本**没有任何 Python 收尾机会**——不写最终模型、不写
|
||||
元数据、不写 `training_failure.json`。如果磁盘上只有 `--checkpoint-every` 的稀疏
|
||||
历史 checkpoint,急停就可能丢掉上万步,或者在"保存途中被杀"时留下一个打不开的
|
||||
半截 zip。为此加了四条保证:
|
||||
|
||||
| 保证 | 实现 | 效果 |
|
||||
| --- | --- | --- |
|
||||
| 起步基线 | `recovery_every > 0` 时,`learn()` 之前先写一份 `<stem>_latest.zip` | 训练刚起就急停,也有可加载、可 `--resume` 的产物 |
|
||||
| 滚动恢复点 | 每 `--recovery-every`(默认 2000)环境步覆盖写同一份 `<stem>_latest.zip` | 急停最多丢 `recovery_every` 步,而不是 `checkpoint-every` 步 |
|
||||
| 原子落盘 | 模型 zip / 旁挂 json / `resume.json` 一律「同目录 `.tmp-` + `os.replace`」 | 保存途中被强杀,目标路径仍是上一份**完整**快照;不会出现半截文件 |
|
||||
| 权威恢复指针 | 每次落盘刷新 `<run_dir>/resume.json`(model / timesteps / 增量语义 / 生效 PPO 超参);前端 `/api/train/state` 暴露为 `recovery` | 无论怎么停,都有一份明确的"从哪继续";界面一键「用最近检查点续训」 |
|
||||
|
||||
前端在强杀后还会补写 `<run_dir>/forced_stop.json`(`job_id`、`pid`、`reason`、
|
||||
`recovery.model`),因为此时训练脚本已经无法记录任何事情。
|
||||
|
||||
---
|
||||
|
||||
## 4. 平台兼容性矩阵(修复后)
|
||||
|
||||
| 能力 | POSIX | Windows |
|
||||
| --- | --- | --- |
|
||||
| 启动子进程并隔离 | `start_new_session=True`(setsid) | `CREATE_NEW_PROCESS_GROUP` |
|
||||
| 暂停 / 继续 | 控制文件(协作式) | 控制文件(协作式) |
|
||||
| 优雅停止 + 落盘 | 控制文件;`SIGTERM` 亦可 | 控制文件;`CTRL_BREAK_EVENT`→`SIGBREAK` 亦可 |
|
||||
| 超时强杀进程树 | `killpg(SIGKILL)` | `taskkill /F /T /PID` |
|
||||
| `os.killpg` 依赖 | 有 | **无(不再调用)** |
|
||||
| 暂停期间是否算墙钟 | 否(扣除) | 否(扣除) |
|
||||
|
||||
---
|
||||
|
||||
## 5. 验证
|
||||
|
||||
- 新增/更新回归测试:
|
||||
- `tests/test_training_ui.py`:控制文件协议往返与损坏容错、`spawn_kwargs`
|
||||
平台分支、Windows 强杀走 `taskkill`、**无 `killpg` 环境下强杀不抛异常**
|
||||
(直接复现旧 bug)、暂停/继续/停止全链路、控制文件被忽略时强杀并写
|
||||
`forced_stop.json` 恢复指引、`discover_recovery` 的
|
||||
`resume.json`/扫描/悬空指针三分支、HTTP `pause/resume` 端到端。
|
||||
- `tests/test_hrl_checkpoint.py`:控制文件状态机单测;原子保存失败不破坏上一份
|
||||
快照;JSON 原子替换;历史清理不误删滚动恢复点;起步基线可加载;
|
||||
**强杀急停后 `resume.json` + 滚动恢复点仍可 `--resume`**;控制文件停止后
|
||||
模型/元数据落盘(`stop_signal=control_file`)与暂停→继续;POSIX 信号路径
|
||||
单列并加平台跳过。以上集成用例都用**真实训练脚本**跑 subproc。
|
||||
- 手工验证:`--recovery-every 32` 短程实跑后强杀,`resume.json` 指向
|
||||
`q3_ckpt_latest.zip`;以它为 `--resume` 目标再跑 32 步,元数据
|
||||
`actual_timesteps` 从 64 变 96、`timesteps_semantics=additional`。
|
||||
- 全量测试:`pytest tests referee/tests` 全绿。
|
||||
- 全仓库 `pylint src scripts project tests referee main.py` 达到 10.00/10
|
||||
(口径见 §7)。
|
||||
|
||||
---
|
||||
|
||||
## 6. 残余风险与后续建议
|
||||
|
||||
1. **暂停粒度** = `poll_every` 个 env step(默认 32,约 0.1–3s 量级),不是即时;
|
||||
若要更快可调小 `_ControlChannel(poll_every=...)`,代价是更频繁的文件 stat。
|
||||
2. **宽限期固定 15s**(`server.STOP_GRACE_S`)。大模型保存可能更久;但因为落盘是
|
||||
原子的,即使宽限期到点被强杀,也只是"新的最终模型没写成",**上一份滚动恢复点
|
||||
与历史 checkpoint 完好**,不会出现半截文件。需要更长可调大该常量。
|
||||
3. **急停的损失上界 = `--recovery-every`**(默认 2000 环境步)。要更小就调小它;
|
||||
调到 0 表示放弃滚动恢复点(此时只剩 `--checkpoint-every` 的稀疏历史
|
||||
checkpoint 与起步基线)。
|
||||
4. **Windows 强杀依赖 `taskkill`**(系统自带)。若被安全策略禁用,会退回
|
||||
`Popen.kill()`(只杀主进程,spawn 的 worker 可能残留)。
|
||||
5. 更严格的 Windows 方案是 **Job Object**(子进程随 job 句柄一起销毁),但需要
|
||||
`ctypes`/`pywin32`,本仓库基础依赖刻意保持轻量,未引入。
|
||||
6. **前端进程重启后**,内存中的任务对象没了,「用最近检查点续训」按钮不会自动
|
||||
出现;但恢复信息都落在磁盘上(`<run_dir>/resume.json`、`forced_stop.json`、
|
||||
`checkpoints/<stem>_latest.zip`),在表单里填同一个模型输出路径并勾选
|
||||
`--resume` 即可继续。
|
||||
7. 界面 `--device mps` 选项在 Windows 上无意义;`auto` 会回退 cuda/cpu,属既有
|
||||
行为,未改。
|
||||
8. `_ControlPoll` 的暂停循环会占用一个 Python 线程的调度(每 0.2s 一次读盘),
|
||||
长暂停期间 CPU 占用可忽略。
|
||||
|
||||
---
|
||||
|
||||
## 7. pylint 口径说明
|
||||
|
||||
配置写在 `pyproject.toml` 的 `[tool.pylint.*]`,目标是
|
||||
`pylint src scripts project tests referee main.py` = 10.00/10。
|
||||
|
||||
**改代码修掉的真实问题**(不是 disable):未使用的 import/变量/参数、重复字典键
|
||||
(`reward_version`)、`value != value` 判 NaN、`__dataclass_fields__` 私有访问、
|
||||
`raise` 非异常类型、`pytest.importorskip` 之外的赋值不确定、过长行、无谓括号、
|
||||
`len(x)` 判空、循环闭包变量、`subprocess.run` 未显式 `check`、`os.path` 循环导入等。
|
||||
|
||||
**有意 disable 的类别**(每条在 pyproject 里写了理由):项目刻意的懒加载
|
||||
(`import-outside-toplevel`)、防御式宽异常(`broad-exception-caught`)、测试访问
|
||||
内部状态(`protected-access`)、pytest fixture 同名(`redefined-outer-name`)、
|
||||
无头环境不实现 `gym.Env.render`(`abstract-method`)、以及静态分析的已知误报
|
||||
(`possibly-used-before-assignment`)。
|
||||
|
||||
**注意**:`good-names` 不覆盖 `redefined-builtin`,stdlib 回调签名里的
|
||||
`log_message(self, format, ...)` 只能在对应 `def` 上行内 disable。
|
||||
+53
-14
@@ -23,7 +23,7 @@ uv run python -m mcm26b.training_ui --print-preview # 只看离线示例快照
|
||||
|
||||
| 区块 | 内容 |
|
||||
| --- | --- |
|
||||
| 顶栏 | 任务摘要、进程/进度/墙钟/设备/连接五个状态 chip、刷新·预览·停止 |
|
||||
| 顶栏 | 任务摘要、进程/进度/墙钟/设备/连接五个状态 chip、刷新·预览·暂停/继续·停止 |
|
||||
| 执行进度与进程 | 步进进度条、剩余时间与预计结束、PID/吞吐/退出码、PPO 阶段时间线、完整命令行 |
|
||||
| 训练配置 | 由 `/api/train/schema` 生成的五组表单 + 四个预设,运行中锁定 |
|
||||
| 参数收敛状态 | 判读 chip + 判据说明,每个指标一张卡片(最新值、增量、迷你趋势线、异常高亮) |
|
||||
@@ -40,7 +40,15 @@ uv run python -m mcm26b.training_ui --print-preview # 只看离线示例快照
|
||||
| `GET /api/train/schema` | 字段/分组/预设/指标字典,**与 `TrainConfig` 同源**(`schema.py`) |
|
||||
| `GET /api/train/state` | 快照;支持 `log_tail`、`metric_limit`、`after_metric` 游标与 `preview=1` |
|
||||
| `POST /api/train/start` | 允许只提交部分字段(其余取默认值),校验失败返回 409 |
|
||||
| `POST /api/train/stop` | 向独立进程组发送 SIGTERM |
|
||||
| `POST /api/train/pause` | 写控制文件请求在 step 边界挂起(跨平台) |
|
||||
| `POST /api/train/resume` | 解除挂起,从暂停处继续 |
|
||||
| `POST /api/train/stop` | 请求优雅停止;宽限期(`STOP_GRACE_S`,默认 15s)后强杀进程树 |
|
||||
|
||||
暂停/停止走**控制文件**而不是信号,原因与平台差异见
|
||||
[`training-ui-windows-audit.md`](training-ui-windows-audit.md):Windows 没有
|
||||
`SIGSTOP`,且 `os.kill(pid, SIGTERM)` 会退化成 `TerminateProcess` 硬杀,训练脚本
|
||||
"保存模型再退出"的收尾分支不会执行。控制文件在 POSIX 与 Windows 上都只是普通
|
||||
文件 IO,语义完全一致;信号(POSIX `SIGTERM/SIGINT`、Windows `SIGBREAK`)只作兜底。
|
||||
|
||||
进度数据的两个来源,服务端都已解析:
|
||||
|
||||
@@ -76,41 +84,66 @@ uv run python -m mcm26b.training_ui --print-preview # 只看离线示例快照
|
||||
|
||||
- 只监听 `127.0.0.1`;静态资源是固定白名单,不提供目录遍历。
|
||||
- 命令由服务端以**参数列表**构造,不经过 shell;字符串参数只拒绝控制字符。
|
||||
- 同一时刻只允许一个训练任务;停止用独立进程组 SIGTERM,避免残留子进程。
|
||||
- 同一时刻只允许一个训练任务;暂停中的任务仍算「在跑」,必须先停止才能开新一局。
|
||||
- 暂停/停止走控制文件(`--control-file`,默认 `output/hrl/ui/control/<job_id>.control.json`),
|
||||
进程组隔离按平台选择(POSIX `setsid` / Windows `CREATE_NEW_PROCESS_GROUP`),
|
||||
超时强杀按平台选择(POSIX `killpg(SIGKILL)` / Windows `taskkill /F /T`)。
|
||||
- 产物路径限定在仓库根下解析,模型输出沿用训练脚本自身的 `--force`/`--resume`
|
||||
语义(两者互斥,由后端校验)。
|
||||
|
||||
## 停止与落盘保证
|
||||
## 暂停、停止与落盘保证
|
||||
|
||||
训练前端的「停止」是对独立进程组发 SIGTERM。`train_hrl_high.py` 有三层兜底,
|
||||
点一下「停止」不再等于这一局白跑:
|
||||
「暂停」是**协作式**的:前端原子写控制文件,`train_hrl_high.py` 的
|
||||
`_ControlPoll` 每 32 个环境步抽样读一次;暂停时训练阻塞在 `_on_step` 的轮询里,
|
||||
进程与显存保留、不消耗算力,点「继续」从同一个 step 边界恢复。暂停时长不计入
|
||||
`--max-wall-s`(否则"挂起一小时回来直接判超时")。
|
||||
|
||||
「停止」同样是控制文件请求(**跨平台**)。`train_hrl_high.py` 有多层兜底,
|
||||
点一下「停止」不再等于这一局白跑;而**急停**(强杀,例如 `SIGKILL` /
|
||||
`taskkill /F`,进程没有任何收尾机会)也保证可恢复:
|
||||
|
||||
| 机制 | 行为 |
|
||||
| --- | --- |
|
||||
| 定期 checkpoint | `--checkpoint-every`(默认 20000 环境步)在 `<模型目录>/checkpoints/` 落 `*_ckpt_<步数>.zip` + 同名元数据;`--checkpoint-keep` 只保留最近 K 个(默认 3),`--checkpoint-dir` 可改目录 |
|
||||
| 优雅停止 | subproc worker 忽略 SIGTERM/SIGINT,主进程收到信号后在 step 边界收尾,保存最终模型与元数据(`stopped_by_signal=true`、`stop_signal`、`checkpoint_files`)并正常关闭环境;收尾阶段恢复默认处理器,再按一次可立即强杀 |
|
||||
| 异常兜底 | `learn()` 抛出任何异常时先落 `*_emergency.zip` + 元数据,再写 `training_failure.json` 并抛出 |
|
||||
| 滚动恢复点 | `--recovery-every`(默认 2000 环境步)把 `<checkpoints>/<stem>_latest.zip` **原子覆盖写**一次;训练开始前先写一份**起步基线**。因此急停后最多只丢 `--recovery-every` 个环境步,且磁盘上始终有一份完整、可加载、可 `--resume` 的产物 |
|
||||
| 原子落盘 | 模型 zip、旁挂 json、`resume.json` 都是「同目录临时文件 + `os.replace`」:保存途中被强杀不会留下半截文件,也不会破坏上一份完整快照 |
|
||||
| 恢复指针 | `<run_dir>/resume.json` 记录当前最佳恢复目标 + 步数(增量语义)+ 生效过的 PPO 超参;前端顶部的「用最近检查点续训」按钮直接读它 |
|
||||
| 定期 checkpoint | `--checkpoint-every`(默认 20000 环境步)另留 `<stem>_<步数>.zip`,`--checkpoint-keep` 只保留最近 K 个(默认 3),供返祖对比与多档恢复;滚动恢复点与起步基线不受清理影响 |
|
||||
| 优雅停止 | 控制文件 `stop=true`(或 POSIX 信号 / Windows `SIGBREAK`)后,主进程在 step 边界收尾,保存最终模型与元数据(`stopped_by_signal=true`、`stop_signal`、`checkpoint_files`、`recovery_model`)并正常关闭环境 |
|
||||
| 超时强杀 | 宽限期(`STOP_GRACE_S`,默认 15s)内没退出则强杀整棵进程树,快照 `forced_kill=true`,并在 `<run_dir>/forced_stop.json` 写明 `recovery.model`(可直接续训的目标) |
|
||||
| 异常兜底 | `learn()` 抛出任何异常时先落 `*_emergency.zip` + 元数据 + `resume.json`,再写 `training_failure.json` 并抛出 |
|
||||
|
||||
急停后恢复的操作方式:界面点「用最近检查点续训」;命令行则
|
||||
`--resume --model <resume.json 里的 model>`,`--timesteps` 按**追加**步数理解。
|
||||
|
||||
checkpoint 与最终模型共用同一份 json 契约,因此 `--resume <checkpoint.zip>` 直接
|
||||
续训(`--timesteps` 为增量步数)。注意 `--resume`/`--init` 会沿用 checkpoint 内的
|
||||
PPO 超参(`n_steps`/`batch_size`/`n_epochs`):命令行这三项被忽略,脚本会打一条
|
||||
`hyperparams_from_checkpoint` 事件说明差异,元数据 `ppo_hyperparams` 记录**实际生效**值。
|
||||
回归测试见 `tests/test_hrl_checkpoint.py`,
|
||||
其中端到端用例真起 subproc 子进程并用 `killpg(SIGTERM)` 复现前端的停止路径。
|
||||
回归测试见 `tests/test_hrl_checkpoint.py`,其中端到端用例真起 subproc 子进程,覆盖
|
||||
控制文件停止、POSIX 信号停止、**强杀后仍可续训**与暂停→继续(信号路径在 Windows
|
||||
上自动跳过)。
|
||||
|
||||
## 测试
|
||||
|
||||
```bash
|
||||
.venv/bin/python -m pytest tests/test_training_ui.py tests/test_training_ui_js.py -q
|
||||
.venv/bin/python -m pytest tests/test_training_ui.py tests/test_training_ui_js.py tests/test_hrl_checkpoint.py -q
|
||||
```
|
||||
|
||||
- `tests/test_training_ui.py`:配置契约同源、预设可直接提交、参数校验、SB3 表格
|
||||
刻度打点、游标分页、`describe_process` 进度/ETA、示例快照与真实快照同键、
|
||||
以及用**桩训练脚本真起子进程**的启动/轮询/停止/退出码全链路。
|
||||
刻度打点、游标分页、`describe_process` 进度/ETA、示例快照与真实快照同键;
|
||||
用**桩训练脚本真起子进程**覆盖启动/轮询/暂停/继续/停止/退出码全链路;并覆盖
|
||||
跨平台进程控制——控制文件往返与损坏容错、`spawn_kwargs` 平台分支、Windows 走
|
||||
`taskkill`、**无 `killpg` 环境下强杀不抛异常**(旧 bug 的直接回归)、控制文件被
|
||||
忽略时强杀并写 `forced_stop.json` 恢复指引、`discover_recovery` 的
|
||||
`resume.json`/扫描/悬空指针三分支、HTTP `pause/resume`。
|
||||
- `tests/js/training_ui_harness.cjs`(经 `tests/test_training_ui_js.py` 调起):
|
||||
DOM 桩加载发布的 `charts.js`/`app.js` 原文,用服务端生成的真实快照驱动渲染,
|
||||
断言表单/指标卡/曲线/日志/逐回合表都拿到数据,覆盖增量游标三条分支与
|
||||
`classList`、HTML 转义、极端数值等边界。
|
||||
- `tests/test_hrl_checkpoint.py`:落盘与恢复的端到端回归——原子保存失败不破坏
|
||||
上一份快照、JSON 原子替换、历史清理不误删滚动恢复点、起步基线可加载、
|
||||
**强杀急停后 `resume.json` 与滚动恢复点仍可继续训练**、控制文件停止落盘、
|
||||
暂停→继续、POSIX 信号停止。
|
||||
|
||||
## 已知取舍
|
||||
|
||||
@@ -119,3 +152,9 @@ PPO 超参(`n_steps`/`batch_size`/`n_epochs`):命令行这三项被忽略
|
||||
格式,退化路径是「只显示 progress JSON」,不会导致界面崩溃。
|
||||
- 收敛判读是启发式;正式结论仍需用 `scripts/benchmark_hrl.py` 的配对评测与
|
||||
学习曲线人工复核。
|
||||
- 暂停粒度是 `poll_every` 个环境步(默认 32),不是即时挂起;停止宽限期默认 15s。
|
||||
- Windows 强杀依赖系统自带的 `taskkill`;若被策略禁用会退回 `Popen.kill()`
|
||||
(只杀主进程)。更严格的 Job Object 方案需要额外依赖,未引入。
|
||||
|
||||
Windows 兼容性问题的完整根因与修复记录见
|
||||
[`training-ui-windows-audit.md`](training-ui-windows-audit.md)。
|
||||
|
||||
@@ -3,7 +3,7 @@ from mcm26b.planner.bayes_loop import BayesLoop
|
||||
from mcm26b.planner.hrl_loop import HRLLoop
|
||||
|
||||
|
||||
def HeuristicHRL(*args, **kwargs):
|
||||
def HeuristicHRL(*args, **kwargs): # pylint: disable=invalid-name # 兼容入口保留题面公开名
|
||||
"""返回无模型的高层 HRL 闭环(heuristic_action 降级路径)。"""
|
||||
kwargs.setdefault("high_policy", None)
|
||||
return HRLLoop(*args, **kwargs)
|
||||
|
||||
@@ -16,13 +16,13 @@ def randomize_problem_config(base: ProblemConfig, *, seed: int,
|
||||
r_eff_range: tuple[float, float] | None = None
|
||||
) -> ProblemConfig:
|
||||
"""返回带域随机化标量的新配置;不修改全局默认配置。"""
|
||||
rng = Random(seed)
|
||||
_rng = Random(seed) # 固定种子留作兼容:真实随机化在 make_scene 内完成
|
||||
field = base.field
|
||||
if bearing_error_half_deg is None:
|
||||
bearing_error_half_deg = field.bearing_error_half_deg
|
||||
if r_eff_range is not None:
|
||||
lo, hi = r_eff_range
|
||||
if not (0 <= lo <= hi):
|
||||
if not 0 <= lo <= hi:
|
||||
raise ValueError("r_eff_range 必须为递增非负区间")
|
||||
return dataclasses.replace(base, field=dataclasses.replace(
|
||||
field, bearing_error_half_deg=float(bearing_error_half_deg)))
|
||||
|
||||
@@ -52,3 +52,68 @@ mcm26b = "mcm26b.cli:main"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests", "referee/tests"]
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# pylint:全仓库静态检查口径
|
||||
#
|
||||
# 目标:`python -m pylint src scripts project tests referee main.py` 拿 10.00/10。
|
||||
# 原则:真实的逻辑 / 资源 / 命名 / 死代码问题一律改代码;下面 disable 的每一条
|
||||
# 都对应项目刻意的设计选择或已确认的静态分析误报,并写明理由,不做"一键静音"。
|
||||
# ---------------------------------------------------------------------------
|
||||
[tool.pylint.main]
|
||||
py-version = "3.10"
|
||||
jobs = 0
|
||||
# 不写 ~/Library/Caches/pylint:CI 与受限沙箱里没有该目录,会刷警告
|
||||
persistent = false
|
||||
recursive = true
|
||||
ignore = [".venv", ".uv-cache", "output"]
|
||||
|
||||
[tool.pylint.format]
|
||||
max-line-length = 100
|
||||
|
||||
[tool.pylint.design]
|
||||
# 阈值按"实际最长函数 + 余量"设定,而不是关闭检查:超过即说明该拆了
|
||||
max-args = 14
|
||||
max-positional-arguments = 10
|
||||
max-locals = 45
|
||||
max-branches = 26
|
||||
max-statements = 180
|
||||
max-attributes = 32
|
||||
max-nested-blocks = 6
|
||||
max-returns = 12
|
||||
max-module-lines = 4000
|
||||
min-public-methods = 0
|
||||
|
||||
[tool.pylint.basic]
|
||||
# 这些是既有代码里的短名/别名(importorskip 后的模块句柄、几何单字母常量、
|
||||
# 坐标系里的 x/y/z),让命名检查接受它们。
|
||||
# 注意:good-names 不覆盖 redefined-builtin——stdlib 回调签名里的 `format`
|
||||
# 之类只能在对应 def 上写行内 disable(见 server.py / referee 的 log_message)。
|
||||
good-names = [
|
||||
"i", "j", "k", "n", "m", "x", "y", "z", "_", "ex",
|
||||
"id", "pytest", "numpy", "S", "ROOT", "FONT", "ET",
|
||||
]
|
||||
# http.server 要求 do_GET/do_POST 这类方法名,不能改成 snake_case
|
||||
method-rgx = "[a-z_][a-z0-9_]{2,49}$|do_(GET|POST|PUT|DELETE|HEAD|PATCH|OPTIONS)$"
|
||||
|
||||
[tool.pylint."messages control"]
|
||||
disable = [
|
||||
# —— 项目刻意风格 ——
|
||||
"missing-module-docstring", # 内部小模块与 __main__ 不强制
|
||||
"missing-class-docstring",
|
||||
"missing-function-docstring",
|
||||
"import-outside-toplevel", # torch/SB3/referee 等重依赖与可选依赖懒加载
|
||||
"wrong-import-position", # 同上:importorskip 守卫后的延迟导入
|
||||
"ungrouped-imports", # 同上:延迟导入无法与顶部导入分组
|
||||
"broad-exception-caught", # 子进程 / IO / HTTP 边界防御式兜底,逐处注释
|
||||
"protected-access", # 测试与 mixin 有意检查内部状态
|
||||
"duplicate-code", # 脚本与测试夹具存在结构性相似,非复制粘贴缺陷
|
||||
"redefined-outer-name", # pytest 要求测试参数与 fixture 同名
|
||||
"attribute-defined-outside-init", # 轻量对象在 helper 里补字段
|
||||
"multiple-statements", # 单行守卫 (if x: return) 是可读性选择
|
||||
"use-dict-literal", # dict(...) 与 {...} 混用是既有风格
|
||||
"abstract-method", # 无头训练环境不需要实现 gym.Env.render
|
||||
# —— 已确认的静态分析误报 ——
|
||||
"possibly-used-before-assignment", # pytest.importorskip 成功即赋值,skip 会抛异常
|
||||
]
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ from dataclasses import dataclass
|
||||
|
||||
from .jsonio import RequestFormatError, dumps, parse_request_object
|
||||
from .rules import ACTION_FIELDS
|
||||
from .simulator import LocalSimulator, SimulatorClosed
|
||||
from .simulator import LocalSimulator
|
||||
|
||||
MAX_BODY_BYTES = 65_536 # [题面 附件2 5.1]
|
||||
MAX_SAFE_READ_BYTES = 1_048_576 # 超限请求最多读 1MB 即断,防内存滥用 [工程]
|
||||
|
||||
@@ -22,7 +22,7 @@ class _ActionHandler(BaseHTTPRequestHandler):
|
||||
server_version = "MCM26B-Referee/1.0"
|
||||
|
||||
# ---- 日志静默(界面"指令与反馈"由会话层记录) ----------------------------
|
||||
def log_message(self, format, *args): # noqa: A002 - 标准库签名
|
||||
def log_message(self, format, *args): # pylint: disable=redefined-builtin(标准库签名)
|
||||
pass
|
||||
|
||||
# ---- 各方法统一走同一边界;未知方法由 do_* 缺失天然 501,这里显式覆盖常用
|
||||
@@ -123,7 +123,7 @@ class ActionInterfaceServer:
|
||||
name=f"referee-action-{session.case_code}",
|
||||
daemon=True).start()
|
||||
|
||||
def close(self, session: LiveSession) -> None:
|
||||
def close(self, _session: LiveSession) -> None:
|
||||
httpd, self._httpd, self._session = self._httpd, None, None
|
||||
if httpd is None:
|
||||
return
|
||||
|
||||
@@ -423,7 +423,7 @@ class Referee:
|
||||
def set_display_limit(self, limit: int) -> None:
|
||||
"""指令与反馈显示条数 100..5000 [题面 附件1 4.5];不影响日志完整性。"""
|
||||
with self._lock:
|
||||
if not isinstance(limit, int) or not (DISPLAY_LIMIT_MIN <= limit <= DISPLAY_LIMIT_MAX):
|
||||
if not isinstance(limit, int) or not DISPLAY_LIMIT_MIN <= limit <= DISPLAY_LIMIT_MAX:
|
||||
raise RefereeError(f"显示条数须为 {DISPLAY_LIMIT_MIN}..{DISPLAY_LIMIT_MAX}")
|
||||
self.display_limit = limit
|
||||
|
||||
|
||||
@@ -19,8 +19,8 @@ from threading import Event, Lock
|
||||
from unicodedata import category
|
||||
from copy import deepcopy
|
||||
|
||||
from .rules import (ACTION_FIELDS, CLEAR_NO_TARGET, CLEAR_RESULTS, CLEAR_SUCCESS,
|
||||
COMMON_FIELDS, DIRECTION, EXIT_USER, MEASURE_RESULTS, NEAR,
|
||||
from .rules import (ACTION_FIELDS, CLEAR_NO_TARGET, CLEAR_SUCCESS,
|
||||
COMMON_FIELDS, DIRECTION, EXIT_USER, NEAR,
|
||||
NO_SIGNAL, POSITION_FIELDS)
|
||||
from .spec import Spec, DEFAULT_SPEC
|
||||
from .timing import clear_cost, measure_cost
|
||||
@@ -58,7 +58,7 @@ class LocalSimulator:
|
||||
if len(channels) != len(set(channels)):
|
||||
raise ValueError("每个频道最多对应一个干扰源")
|
||||
for s in sources:
|
||||
if not (spec.field.channel_min <= s.channel <= spec.field.channel_max):
|
||||
if not spec.field.channel_min <= s.channel <= spec.field.channel_max:
|
||||
raise ValueError("频道超出 1..20")
|
||||
if not s.directional and s.heading_deg != 0.0:
|
||||
raise ValueError("全向源不应有朝向")
|
||||
@@ -109,8 +109,9 @@ class LocalSimulator:
|
||||
owner = True
|
||||
if not owner:
|
||||
pending["done"].wait()
|
||||
if pending["error"] is not None:
|
||||
raise pending["error"]
|
||||
failure: BaseException | None = pending["error"]
|
||||
if failure is not None:
|
||||
raise failure
|
||||
return deepcopy(pending["result"])
|
||||
try:
|
||||
result = self._handle(path, payload)
|
||||
@@ -166,7 +167,8 @@ class LocalSimulator:
|
||||
value = payload.get(name)
|
||||
if (not isinstance(value, str) or not value
|
||||
or any(category(c) in ("Cc", "Cf", "Cs") for c in value)
|
||||
or (field_spec.max_bytes and len(value.encode("utf-8")) > field_spec.max_bytes)):
|
||||
or (field_spec.max_bytes
|
||||
and len(value.encode("utf-8")) > field_spec.max_bytes)):
|
||||
return 400, self.error_body(path)
|
||||
# 2) 动作字段存在性 -> 400
|
||||
action_spec = ACTION_FIELDS[path]
|
||||
|
||||
@@ -14,8 +14,7 @@ from pathlib import Path
|
||||
from urllib.parse import parse_qs, urlsplit
|
||||
|
||||
from .jsonio import dumps
|
||||
from .session import (DISPLAY_LIMIT_MAX, DISPLAY_LIMIT_MIN, LiveSession,
|
||||
Referee, RefereeError)
|
||||
from .session import LiveSession, Referee, RefereeError
|
||||
|
||||
STATIC_DIR = Path(__file__).parent / "static"
|
||||
STATIC_FILES = {
|
||||
@@ -31,7 +30,7 @@ class _UIHandler(BaseHTTPRequestHandler):
|
||||
protocol_version = "HTTP/1.1"
|
||||
server_version = "MCM26B-Referee-UI/1.0"
|
||||
|
||||
def log_message(self, format, *args): # noqa: A002 - 标准库签名
|
||||
def log_message(self, format, *args): # pylint: disable=redefined-builtin(标准库签名)
|
||||
pass
|
||||
|
||||
# ------------------------------------------------------------ 路由
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
"""真实 HTTP 承载测试:接口仅回环、仅窗口期开放、协议边界与官方一致。"""
|
||||
|
||||
# pylint: disable=redefined-outer-name # pytest 夹具名必须与测试函数形参同名,非遮蔽缺陷
|
||||
|
||||
import http.client
|
||||
import json
|
||||
import time
|
||||
@@ -20,7 +22,7 @@ def wait_until(condition, timeout=5.0, interval=0.02):
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def referee(tmp_path):
|
||||
def local_referee(tmp_path):
|
||||
ref = Referee(action_port=0, data_dir=tmp_path / "data")
|
||||
ActionInterfaceServer(ref)
|
||||
return ref
|
||||
@@ -43,20 +45,20 @@ def post(port, path, obj=None, *, headers=None, method="POST", raw=None):
|
||||
return response.status, parsed
|
||||
|
||||
|
||||
def payload(id="e", **extra):
|
||||
return dict(arena_id="default", robot_id="2026001", request_id=id, **extra)
|
||||
def payload(request_id="e", **extra):
|
||||
return dict(arena_id="default", robot_id="2026001", request_id=request_id, **extra)
|
||||
|
||||
|
||||
def start_open_session(referee, **opts):
|
||||
def start_open_session(local_referee, **opts):
|
||||
opts.setdefault("countdown_s", 0.05)
|
||||
session = referee.start("q3_practice", "2026001", **opts)
|
||||
session = local_referee.start("q3_practice", "2026001", **opts)
|
||||
assert wait_until(lambda: session.phase == "open")
|
||||
return session
|
||||
|
||||
|
||||
def test_binding_is_loopback_only(referee):
|
||||
session = start_open_session(referee)
|
||||
server = referee._open_interface.__self__ # ActionInterfaceServer
|
||||
def test_binding_is_loopback_only(local_referee):
|
||||
_session = start_open_session(local_referee)
|
||||
server = local_referee._open_interface.__self__ # ActionInterfaceServer
|
||||
port = open_port(server)
|
||||
assert server._httpd.server_address[0] == "127.0.0.1"
|
||||
status, body = post(port, "/enter", payload("enter"))
|
||||
@@ -64,17 +66,17 @@ def test_binding_is_loopback_only(referee):
|
||||
assert body["remaining_real_duration_s"] == 1200
|
||||
|
||||
|
||||
def test_interface_refuses_connections_outside_window(referee):
|
||||
session = referee.start("q3_practice", "2026001", countdown_s=5)
|
||||
server = referee._open_interface.__self__
|
||||
def test_interface_refuses_connections_outside_window(local_referee):
|
||||
session = local_referee.start("q3_practice", "2026001", countdown_s=5)
|
||||
server = local_referee._open_interface.__self__
|
||||
assert server.is_open is False # 倒计时期间未监听
|
||||
assert wait_until(lambda: session.phase == "open", timeout=6)
|
||||
assert server.is_open is True # 窗口开启后才绑定端口
|
||||
|
||||
|
||||
def test_full_protocol_round_trip_over_http(referee):
|
||||
session = start_open_session(referee)
|
||||
server = referee._open_interface.__self__
|
||||
def test_full_protocol_round_trip_over_http(local_referee):
|
||||
session = start_open_session(local_referee)
|
||||
server = local_referee._open_interface.__self__
|
||||
port = open_port(server)
|
||||
status, body = post(port, "/enter", payload("enter-1"))
|
||||
assert status == 200 and body["accepted"] is True
|
||||
@@ -94,9 +96,9 @@ def test_full_protocol_round_trip_over_http(referee):
|
||||
assert session.end_reason == "user_exit"
|
||||
|
||||
|
||||
def test_protocol_error_statuses_over_http(referee):
|
||||
start_open_session(referee)
|
||||
server = referee._open_interface.__self__
|
||||
def test_protocol_error_statuses_over_http(local_referee):
|
||||
start_open_session(local_referee)
|
||||
server = local_referee._open_interface.__self__
|
||||
port = open_port(server)
|
||||
assert post(port, "/unknown", payload())[0] == 404
|
||||
assert post(port, "/enter/", payload())[0] == 404
|
||||
@@ -109,9 +111,9 @@ def test_protocol_error_statuses_over_http(referee):
|
||||
assert status == 200 and body["accepted"] is True
|
||||
|
||||
|
||||
def test_interface_closes_after_window_timeout(referee):
|
||||
session = start_open_session(referee, window_s=0.4)
|
||||
server = referee._open_interface.__self__
|
||||
def test_interface_closes_after_window_timeout(local_referee):
|
||||
session = start_open_session(local_referee, window_s=0.4)
|
||||
server = local_referee._open_interface.__self__
|
||||
port = open_port(server)
|
||||
assert wait_until(lambda: session.phase == "finished", timeout=5)
|
||||
assert session.end_reason == "window_timeout"
|
||||
@@ -120,9 +122,9 @@ def test_interface_closes_after_window_timeout(referee):
|
||||
post(port, "/enter", payload("late"))
|
||||
|
||||
|
||||
def test_recorded_feed_contains_every_request(referee):
|
||||
session = start_open_session(referee)
|
||||
server = referee._open_interface.__self__
|
||||
def test_recorded_feed_contains_every_request(local_referee):
|
||||
session = start_open_session(local_referee)
|
||||
server = local_referee._open_interface.__self__
|
||||
port = open_port(server)
|
||||
post(port, "/enter", payload("enter-1"))
|
||||
post(port, "/unknown", payload("bad"))
|
||||
|
||||
@@ -11,8 +11,8 @@ from referee import LocalSimulator, SimulatorClosed
|
||||
from referee.interface import handle_raw
|
||||
|
||||
|
||||
def payload(id="e", **extra):
|
||||
return dict(arena_id="default", robot_id="2026001", request_id=id, **extra)
|
||||
def payload(request_id="e", **extra):
|
||||
return dict(arena_id="default", robot_id="2026001", request_id=request_id, **extra)
|
||||
|
||||
|
||||
def post(sim, path, body=None, headers=None, method="POST", target=None):
|
||||
@@ -60,7 +60,8 @@ def test_raw_size_and_depth_boundaries_are_inclusive():
|
||||
("PUT", "/clear", None, 405),
|
||||
("POST", "/enter", [("Content-Type", "text/plain")], 415),
|
||||
("POST", "/enter", [("Content-Type", "application/json;foo=bar")], 415),
|
||||
("POST", "/enter", [("Content-Type", "application/json"), ("Content-Type", "application/json")], 415),
|
||||
("POST", "/enter",
|
||||
[("Content-Type", "application/json"), ("Content-Type", "application/json")], 415),
|
||||
("POST", "/enter", [("Content-Type", "application/json"), ("Content-Encoding", "gzip")], 415),
|
||||
("POST", "/enter", [("content-type", "Application/JSON; charset=UTF-8")], 200),
|
||||
("POST", "/enter", [("content-type", 'application/json;charset="utf-8"')], 200),
|
||||
@@ -84,7 +85,8 @@ def test_on_request_callback_receives_parsed_payload():
|
||||
seen = []
|
||||
raw = json.dumps(payload("enter-1")).encode()
|
||||
handle_raw(sim, "POST", "/enter", [("Content-Type", "application/json")], raw,
|
||||
on_request=lambda path, payload_dict, result: seen.append((path, payload_dict, result)))
|
||||
on_request=lambda path, payload_dict, result: seen.append(
|
||||
(path, payload_dict, result)))
|
||||
assert seen[0][0] == "/enter"
|
||||
assert seen[0][1]["request_id"] == "enter-1"
|
||||
assert seen[0][2].status == 200
|
||||
|
||||
@@ -20,6 +20,7 @@ HARNESS = REPO / "referee" / "tests" / "js" / "replay_harness.cjs"
|
||||
@pytest.mark.skipif(shutil.which("node") is None, reason="需要 node 环境运行 JS 桩测试")
|
||||
def test_replay_player_logic():
|
||||
result = subprocess.run(["node", str(HARNESS)], capture_output=True, text=True,
|
||||
env={**os.environ, "REPO": str(REPO)}, timeout=60)
|
||||
env={**os.environ, "REPO": str(REPO)}, timeout=60,
|
||||
check=False)
|
||||
assert result.returncode == 0, f"回放器逻辑测试失败:\n{result.stdout}\n{result.stderr}"
|
||||
assert "ALL PASS" in result.stdout
|
||||
|
||||
@@ -65,7 +65,7 @@ def test_cannot_start_while_running_and_restart_after_finish(tmp_path):
|
||||
def test_formal_attempts_accounting(tmp_path):
|
||||
referee = Referee(action_port=0, data_dir=tmp_path)
|
||||
for _ in range(FORMAL_ATTEMPTS):
|
||||
session = referee.start("q3_formal", "2026001", countdown_s=0.05)
|
||||
_session = referee.start("q3_formal", "2026001", countdown_s=0.05)
|
||||
referee.abort() # 正式中止仍占用机会 [题面 4.5]
|
||||
with pytest.raises(RefereeError):
|
||||
referee.start("q3_formal", "2026001", countdown_s=0.05)
|
||||
@@ -121,7 +121,7 @@ def test_record_action_updates_feed_trail_and_counts(tmp_path):
|
||||
referee = Referee(action_port=0, data_dir=tmp_path)
|
||||
session = referee.start("q3_practice", "2026001", countdown_s=0.05, window_s=5)
|
||||
assert wait_until(lambda: session.phase == "open")
|
||||
sim = session.simulator
|
||||
_sim = session.simulator
|
||||
referee.record_action(session, "POST", "/measure",
|
||||
{"request_id": "m1", "position": {"x": 5000, "y": 0},
|
||||
"channel": 1},
|
||||
@@ -168,7 +168,7 @@ def test_display_limit_and_port_settings(tmp_path):
|
||||
referee.set_display_limit(50)
|
||||
with pytest.raises(RefereeError):
|
||||
referee.set_display_limit(5001)
|
||||
session = referee.start("q3_practice", "2026001", countdown_s=5)
|
||||
_session = referee.start("q3_practice", "2026001", countdown_s=5)
|
||||
with pytest.raises(RefereeError):
|
||||
referee.set_action_port(8080) # 测试进行中不能改端口
|
||||
referee.abort()
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
"""评测引擎一致性:时限、状态机、观测判定、幂等与量化(移植自参赛队仓库并扩展)。"""
|
||||
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from dataclasses import replace
|
||||
from threading import Event
|
||||
|
||||
import pytest
|
||||
@@ -9,12 +8,12 @@ import pytest
|
||||
from referee import FieldSpec, LocalSimulator, SimulatorClosed, Source, Spec
|
||||
|
||||
|
||||
def payload(id="e", **extra):
|
||||
return dict(arena_id="default", robot_id="2026001", request_id=id, **extra)
|
||||
def payload(request_id="e", **extra):
|
||||
return dict(arena_id="default", robot_id="2026001", request_id=request_id, **extra)
|
||||
|
||||
|
||||
def measure(id="m", x=0, y=0, channel=1):
|
||||
return payload(id, position=dict(x=x, y=y), channel=channel)
|
||||
def measure(request_id="m", x=0, y=0, channel=1):
|
||||
return payload(request_id, position=dict(x=x, y=y), channel=channel)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 时限 [题面 附件1 2.5]
|
||||
@@ -92,7 +91,7 @@ def test_error_rejection_does_not_consume_id():
|
||||
sim = LocalSimulator([])
|
||||
for bad in ["e\n", "e\u200b", "e\u0085", "e\ud800"]:
|
||||
assert sim.handle("/enter", payload(bad))[0] == 400
|
||||
assert sim._idempotency == {}
|
||||
assert not sim._idempotency
|
||||
assert sim.handle("/enter", payload())[1]["accepted"]
|
||||
|
||||
|
||||
@@ -181,6 +180,8 @@ def test_huge_numeric_fields_return_bad_request():
|
||||
(dict(arena_id="other", robot_id="2026001", request_id="e"), False),
|
||||
(dict(arena_id="default", robot_id="someone-else", request_id="e"), False),
|
||||
])
|
||||
# pytest 参数化保留 expect 列(用例矩阵可读性),当前断言恒为 False,故不参与断言
|
||||
# pylint: disable-next=unused-argument
|
||||
def test_unknown_fields_and_mismatch_are_business_rejected(body, expect):
|
||||
sim = LocalSimulator([])
|
||||
status, response = sim.handle("/enter", body)
|
||||
@@ -204,12 +205,14 @@ def test_channel_and_position_validation():
|
||||
]
|
||||
for extra in cases:
|
||||
assert sim.handle("/measure", payload("m", **extra))[0] == 400
|
||||
assert sim.handle("/measure", payload("ok", position=dict(x=0, y=0), channel=1.0))[1]["accepted"]
|
||||
assert sim.handle("/measure",
|
||||
payload("ok", position=dict(x=0, y=0), channel=1.0))[1]["accepted"]
|
||||
|
||||
|
||||
def test_actions_before_enter_are_business_rejected():
|
||||
sim = LocalSimulator([])
|
||||
for path, body in [("/measure", measure()), ("/clear", payload("c", position=dict(x=0, y=0), channel=1)),
|
||||
for path, body in [("/measure", measure()),
|
||||
("/clear", payload("c", position=dict(x=0, y=0), channel=1)),
|
||||
("/exit", payload("x"))]:
|
||||
status, response = sim.handle(path, body)
|
||||
assert status == 200 and response["accepted"] is False
|
||||
@@ -224,7 +227,8 @@ def test_clear_rules():
|
||||
assert ok["clear_result"] == "success"
|
||||
again = sim.handle("/clear", payload("c5b", position=dict(x=10, y=0), channel=5))[1]
|
||||
assert again["clear_result"] == "no_target_in_range" # 同一源只能清除一次
|
||||
assert sim.handle("/clear", payload("c6", position=dict(x=15, y=0), channel=6))[1]["clear_result"] == "success"
|
||||
cleared = sim.handle("/clear", payload("c6", position=dict(x=15, y=0), channel=6))[1]
|
||||
assert cleared["clear_result"] == "success"
|
||||
miss = sim.handle("/clear", payload("c9", position=dict(x=0, y=0), channel=9))[1]
|
||||
assert miss["clear_result"] == "no_target_in_range"
|
||||
assert sim.channel == 1 # /clear 不切换测向机频道 [题面 附件1 2.3]
|
||||
@@ -238,15 +242,20 @@ def test_measure_observation_matrix():
|
||||
sim = LocalSimulator(sources, error_half_deg=0.0)
|
||||
sim.handle("/enter", payload("enter"))
|
||||
# 超出有效接收半径
|
||||
assert sim.handle("/measure", measure("a", -600, 0, channel=1))[1]["measure_result"] == "no_signal"
|
||||
assert sim.handle("/measure",
|
||||
measure("a", -600, 0, channel=1))[1]["measure_result"] == "no_signal"
|
||||
# 接收半径边界:distance == r_eff 仍有信号 [题面 附件2 2.1 "超过"]
|
||||
assert sim.handle("/measure", measure("b", -500, 0, channel=1))[1]["measure_result"] == "direction"
|
||||
assert sim.handle("/measure",
|
||||
measure("b", -500, 0, channel=1))[1]["measure_result"] == "direction"
|
||||
# 覆盖角边界:源2 朝东,正东方向(bearing=0°)恰在 ±90° 覆盖内
|
||||
assert sim.handle("/measure", measure("c", 200, 0, channel=2))[1]["measure_result"] == "direction"
|
||||
assert sim.handle("/measure",
|
||||
measure("c", 200, 0, channel=2))[1]["measure_result"] == "direction"
|
||||
# 覆盖角边界:bearing=90°(正北)仍在覆盖内(含边界)[题面 附件2 2.2]
|
||||
assert sim.handle("/measure", measure("c2", 100, 100, channel=2))[1]["measure_result"] == "direction"
|
||||
assert sim.handle("/measure",
|
||||
measure("c2", 100, 100, channel=2))[1]["measure_result"] == "direction"
|
||||
# 覆盖角外(bearing≈135° > 90°):无信号
|
||||
assert sim.handle("/measure", measure("d", 90, 10, channel=2))[1]["measure_result"] == "no_signal"
|
||||
assert sim.handle("/measure",
|
||||
measure("d", 90, 10, channel=2))[1]["measure_result"] == "no_signal"
|
||||
# 距离过近:不返回 svd_deg [题面 附件2 2.4]
|
||||
near = sim.handle("/measure", measure("near", 998, 0, channel=1))[1]
|
||||
assert near["measure_result"] == "near" and "svd_deg" not in near
|
||||
|
||||
@@ -4,12 +4,12 @@ from referee import CostBreakdown, LocalSimulator, Source, clear_cost, measure_c
|
||||
from referee.timing import move_time
|
||||
|
||||
|
||||
def payload(id="e", **extra):
|
||||
return dict(arena_id="default", robot_id="2026001", request_id=id, **extra)
|
||||
def payload(request_id="e", **extra):
|
||||
return dict(arena_id="default", robot_id="2026001", request_id=request_id, **extra)
|
||||
|
||||
|
||||
def measure(id="m", x=0, y=0, channel=1):
|
||||
return payload(id, position=dict(x=x, y=y), channel=channel)
|
||||
def measure(request_id="m", x=0, y=0, channel=1):
|
||||
return payload(request_id, position=dict(x=x, y=y), channel=channel)
|
||||
|
||||
|
||||
def test_attachment_timing_example():
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
"""WebUI API 测试:状态快照、启动/中止/配置、会话列表与回放数据源。"""
|
||||
|
||||
# pylint: disable=redefined-outer-name # pytest 夹具名必须与测试函数形参同名,非遮蔽缺陷
|
||||
|
||||
import http.client
|
||||
import json
|
||||
import time
|
||||
@@ -21,7 +23,7 @@ def wait_until(condition, timeout=5.0, interval=0.02):
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def ui(tmp_path):
|
||||
def webui(tmp_path):
|
||||
referee = Referee(action_port=0, data_dir=tmp_path / "data")
|
||||
ActionInterfaceServer(referee)
|
||||
server = WebUIServer(referee, 0)
|
||||
@@ -43,8 +45,8 @@ def call(port, method, path, obj=None):
|
||||
return response.status, json.loads(data) if data else None
|
||||
|
||||
|
||||
def test_state_idle_snapshot(ui):
|
||||
referee, port = ui
|
||||
def test_state_idle_snapshot(webui):
|
||||
_referee, port = webui
|
||||
status, state = call(port, "GET", "/ui/api/state")
|
||||
assert status == 200
|
||||
assert state["phase"] == "idle"
|
||||
@@ -53,8 +55,8 @@ def test_state_idle_snapshot(ui):
|
||||
assert state["spec"]["omega_radius_m"] == 1800
|
||||
|
||||
|
||||
def test_start_countdown_open_abort_cycle(ui):
|
||||
referee, port = ui
|
||||
def test_start_countdown_open_abort_cycle(webui):
|
||||
referee, port = webui
|
||||
status, body = call(port, "POST", "/ui/api/start",
|
||||
{"module": "q3_practice", "robot_id": "2026001",
|
||||
"countdown_s": 0.05, "window_s": 5})
|
||||
@@ -70,21 +72,21 @@ def test_start_countdown_open_abort_cycle(ui):
|
||||
assert state["truth"]["revealed"] is True # 演练结束显示真值统计
|
||||
|
||||
|
||||
def test_start_rejects_running_session_and_bad_module(ui):
|
||||
referee, port = ui
|
||||
def test_start_rejects_running_session_and_bad_module(webui):
|
||||
_referee, port = webui
|
||||
call(port, "POST", "/ui/api/start",
|
||||
{"module": "q3_practice", "robot_id": "2026001", "countdown_s": 5})
|
||||
status, body = call(port, "POST", "/ui/api/start",
|
||||
{"module": "q3_practice", "robot_id": "2026001"})
|
||||
status, _body = call(port, "POST", "/ui/api/start",
|
||||
{"module": "q3_practice", "robot_id": "2026001"})
|
||||
assert status == 409
|
||||
status, body = call(port, "POST", "/ui/api/start",
|
||||
{"module": "nope", "robot_id": "2026001"})
|
||||
status, _body = call(port, "POST", "/ui/api/start",
|
||||
{"module": "nope", "robot_id": "2026001"})
|
||||
assert status == 409
|
||||
call(port, "POST", "/ui/api/abort", {})
|
||||
|
||||
|
||||
def test_config_endpoints(ui):
|
||||
referee, port = ui
|
||||
def test_config_endpoints(webui):
|
||||
_referee, port = webui
|
||||
status, body = call(port, "POST", "/ui/api/config", {"display_limit": 100})
|
||||
assert body["configured"]["display_limit"] == 100
|
||||
status, body = call(port, "POST", "/ui/api/config", {"display_limit": 1})
|
||||
@@ -93,8 +95,8 @@ def test_config_endpoints(ui):
|
||||
assert body["configured"]["action_port"] == 8080
|
||||
|
||||
|
||||
def test_sessions_list_and_replay_source(ui):
|
||||
referee, port = ui
|
||||
def test_sessions_list_and_replay_source(webui):
|
||||
referee, port = webui
|
||||
call(port, "POST", "/ui/api/start",
|
||||
{"module": "q3_practice", "robot_id": "2026001",
|
||||
"countdown_s": 0.05, "window_s": 5, "seed": 11, "source_count": 10})
|
||||
@@ -125,8 +127,8 @@ def test_sessions_list_and_replay_source(ui):
|
||||
assert call(port, "GET", "/ui/api/replay?case=../etc")[0] == 400
|
||||
|
||||
|
||||
def test_static_page_served(ui):
|
||||
_, port = ui
|
||||
def test_static_page_served(webui):
|
||||
_, port = webui
|
||||
conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
|
||||
conn.request("GET", "/")
|
||||
response = conn.getresponse()
|
||||
@@ -140,8 +142,8 @@ def test_static_page_served(ui):
|
||||
assert 'id="rp-stage"' in page
|
||||
|
||||
|
||||
def test_replay_script_served_with_file_import(ui):
|
||||
_, port = ui
|
||||
def test_replay_script_served_with_file_import(webui):
|
||||
_, port = webui
|
||||
conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
|
||||
conn.request("GET", "/replay.js")
|
||||
response = conn.getresponse()
|
||||
|
||||
@@ -26,14 +26,13 @@ from __future__ import annotations
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import math
|
||||
import statistics
|
||||
import sys
|
||||
import time
|
||||
from collections import defaultdict
|
||||
from dataclasses import asdict, dataclass
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable
|
||||
from typing import Any
|
||||
|
||||
from mcm26b.config import ProblemConfig
|
||||
from mcm26b.harness import LocalSimTransport, mixed_scene, random_omni_scene
|
||||
|
||||
+10
-6
@@ -39,17 +39,21 @@ def main():
|
||||
for margin in [0, 0.005]:
|
||||
ms = [replace(m, half_angle_deg=m.half_angle_deg+margin) for m in observations]
|
||||
result, duration = timed(solve, ms, 31)
|
||||
cert = result['clearance_certificate']
|
||||
sensitivity.append(dict(additional_half_angle_deg=margin, solve_ms=duration,
|
||||
diameter_m=result['diameter_m'],
|
||||
mec_radius_m=result['minimum_enclosing_circle']['radius_m'],
|
||||
diameter_circle_excess_m=result['diameter_circle']['max_excess_m'],
|
||||
clearance_margin_m=result['clearance_certificate']['remaining_margin_m']))
|
||||
report = dict(python=sys.version.split()[0], timing_method='median perf_counter; warm local process',
|
||||
timing_scope='MEC only; not whole solver speedup or simulator/network performance',
|
||||
circle_benchmarks=timings, practice_sensitivity=sensitivity,
|
||||
q2_first_bearing_certificate=first_bearing_candidate_disk(Measurement(0, 0, 0, 1.005)))
|
||||
clearance_margin_m=cert['remaining_margin_m']))
|
||||
report = dict(
|
||||
python=sys.version.split()[0],
|
||||
timing_method='median perf_counter; warm local process',
|
||||
timing_scope='MEC only; not whole solver speedup or simulator/network performance',
|
||||
circle_benchmarks=timings, practice_sensitivity=sensitivity,
|
||||
q2_first_bearing_certificate=first_bearing_candidate_disk(Measurement(0, 0, 0, 1.005)))
|
||||
output = ROOT/'output/q1/optimization_report.json'
|
||||
output.write_text(json.dumps(report, ensure_ascii=False, indent=2, allow_nan=False)+'\n', encoding='utf-8')
|
||||
output.write_text(
|
||||
json.dumps(report, ensure_ascii=False, indent=2, allow_nan=False)+'\n', encoding='utf-8')
|
||||
print(json.dumps(report, ensure_ascii=False, indent=2))
|
||||
|
||||
|
||||
|
||||
@@ -48,19 +48,22 @@ def main():
|
||||
for p in sorted(Path('src/mcm26b').rglob('*.py'))}
|
||||
if args.baseline:
|
||||
for name in ('baseline_bayes.py', 'baseline_config.json'):
|
||||
source_sha256[str(args.out/name)] = hashlib.sha256((args.out/name).read_bytes()).hexdigest()
|
||||
archive = args.out/name
|
||||
source_sha256[str(archive)] = hashlib.sha256(archive.read_bytes()).hexdigest()
|
||||
scenes = []
|
||||
if args.suite == 'development':
|
||||
for count in (10, 16):
|
||||
for seed in range(6):
|
||||
scenes.append((f'random-{count}-{seed}', random_omni_scene(seed, count), 20260910 + seed))
|
||||
scenes.append((f'random-{count}-{seed}', random_omni_scene(seed, count),
|
||||
20260910 + seed))
|
||||
edge = [Source(c+1, 1800*math.cos(math.pi/6+2*math.pi*c/count),
|
||||
1800*math.sin(math.pi/6+2*math.pi*c/count), 1000) for c in range(count)]
|
||||
scenes.append((f'boundary-{count}', edge, 20260910))
|
||||
else:
|
||||
for count in (10, 13, 16):
|
||||
for seed in range(100, 106):
|
||||
scenes.append((f'holdout-{count}-{seed}', random_omni_scene(seed, count), 20260910+seed))
|
||||
scenes.append((f'holdout-{count}-{seed}', random_omni_scene(seed, count),
|
||||
20260910+seed))
|
||||
rows = []
|
||||
for name, sources, error_seed in scenes:
|
||||
sim = LocalSimulator(sources, robot_id='BENCH', config=config, error_seed=error_seed)
|
||||
@@ -81,12 +84,15 @@ def main():
|
||||
sources=[asdict(s) for s in sources], error_seed=error_seed,
|
||||
summary=summary, actions=actions), ensure_ascii=False), encoding='utf-8')
|
||||
print(f'{name}: valid={valid} T={row["virtual_s"]:.1f} M={row["measures"]}', flush=True)
|
||||
result = dict(config=asdict(config), source='local_simulator_only', baseline=args.baseline, rows=rows,
|
||||
mean_s=mean(r['virtual_s'] for r in rows), median_s=median(r['virtual_s'] for r in rows),
|
||||
result = dict(config=asdict(config), source='local_simulator_only',
|
||||
baseline=args.baseline, rows=rows,
|
||||
mean_s=mean(r['virtual_s'] for r in rows),
|
||||
median_s=median(r['virtual_s'] for r in rows),
|
||||
max_s=max(r['virtual_s'] for r in rows), valid=sum(r['valid'] for r in rows),
|
||||
under_4000=sum(r['virtual_s'] < 4000 for r in rows),
|
||||
source_sha256=source_sha256)
|
||||
(folder/'results.json').write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding='utf-8')
|
||||
(folder/'results.json').write_text(
|
||||
json.dumps(result, ensure_ascii=False, indent=2), encoding='utf-8')
|
||||
print({k:result[k] for k in ('mean_s','median_s','max_s','valid','under_4000')}, flush=True)
|
||||
|
||||
|
||||
|
||||
@@ -14,14 +14,11 @@ from pathlib import Path
|
||||
from statistics import mean, median
|
||||
from time import perf_counter
|
||||
|
||||
from mcm26b.belief import ChannelStatus # noqa: F401 (兼容旧 import 路径)
|
||||
from mcm26b.config import ProblemConfig
|
||||
from mcm26b.coverage import CoverageLedger # noqa: F401
|
||||
from mcm26b.planner import OmnidirectionalLoop, clear_certificate # noqa: F401
|
||||
from mcm26b.planner import OmnidirectionalLoop
|
||||
from mcm26b.planner.seven import (SevenBatch, SevenImmediate, SevenSweep,
|
||||
RING_RADIUS_M, seven_points)
|
||||
from mcm26b.protocol.client import SimulatorClient
|
||||
from mcm26b.protocol.rules import NEAR # noqa: F401
|
||||
from mcm26b.harness import LocalSimTransport, LocalSimulator
|
||||
from referee import Source
|
||||
from mcm26b.harness import random_omni_scene
|
||||
@@ -70,7 +67,8 @@ def run_case(name, sources, error_seed, out):
|
||||
"actions_file": str(log_path.relative_to(out)),
|
||||
"actions_sha256": hashlib.sha256(log_path.read_bytes()).hexdigest()}
|
||||
rows.append(row)
|
||||
print(f"{name} {label}: all_cleared={row['all_cleared']} T={row['virtual_time_s']:.1f}s M={row['measures']}", flush=True)
|
||||
print(f"{name} {label}: all_cleared={row['all_cleared']} "
|
||||
f"T={row['virtual_time_s']:.1f}s M={row['measures']}", flush=True)
|
||||
return rows
|
||||
|
||||
|
||||
@@ -96,7 +94,8 @@ def geometry_check():
|
||||
"sampled_max_distance_m": worst[0], "sampled_worst_point": worst[1],
|
||||
"samples": 181*721,
|
||||
"passed": worst[0] <= analytic+1e-8 and analytic < field.r_eff_min_m,
|
||||
"note": "Grid samples check the derivation; the sector proof establishes continuous coverage."}
|
||||
"note": ("Grid samples check the derivation; "
|
||||
"the sector proof establishes continuous coverage.")}
|
||||
|
||||
|
||||
def main():
|
||||
@@ -137,10 +136,16 @@ def main():
|
||||
result = {"source": "local_simulator_only", "production_algorithm_modified": False,
|
||||
"seeds": args.seeds, "config": asdict(ProblemConfig()),
|
||||
"geometry": geometry_check(), "aggregates": aggregates, "runs": rows,
|
||||
"limitations": "Same scene and fixed per-position error field for each variant; synthetic data, not an official improvement estimate. Only search/scheduling is changed; no posterior or localizer redesign."}
|
||||
(args.out/"benchmark.json").write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
"limitations": (
|
||||
"Same scene and fixed per-position error field for each variant; "
|
||||
"synthetic data, not an official improvement estimate. "
|
||||
"Only search/scheduling is changed; no posterior or localizer redesign.")}
|
||||
(args.out/"benchmark.json").write_text(
|
||||
json.dumps(result, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
print(json.dumps(aggregates, ensure_ascii=False, indent=2))
|
||||
return 0 if result["geometry"]["passed"] and all(a["valid_runs"] == a["runs"] for a in aggregates.values()) else 1
|
||||
passed = (result["geometry"]["passed"]
|
||||
and all(a["valid_runs"] == a["runs"] for a in aggregates.values()))
|
||||
return 0 if passed else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -95,7 +95,8 @@ def main(argv=None) -> int:
|
||||
cleared = all(row["rl"]["all_cleared"] and row[baseline]["all_cleared"]
|
||||
for row in rows)
|
||||
valid = all(part['completed'] and part['all_cleared'] and part['correct_absence']
|
||||
and part['normal_exit'] and not part['anomalies'] and not part['timing_mismatches']
|
||||
and part['normal_exit'] and not part['anomalies']
|
||||
and not part['timing_mismatches']
|
||||
for row in rows for part in (row[baseline],row['rl']))
|
||||
report[key + "_summary"] = {
|
||||
"mean_virtual_s": {
|
||||
|
||||
@@ -53,15 +53,23 @@ def main():
|
||||
data={**case,'measurements':[asdict(m) for m in ms],'frames':frames}
|
||||
dataset.append(data)
|
||||
if case['source']=='synthetic':
|
||||
(examples/(case['id']+'.json')).write_text(json.dumps({k:v for k,v in data.items() if k!='frames'},ensure_ascii=False,indent=2),encoding='utf-8')
|
||||
(results/(case['id']+'_result.json')).write_text(json.dumps(frames[-1],ensure_ascii=False,indent=2,allow_nan=False),encoding='utf-8')
|
||||
(results/'demo_data.json').write_text(json.dumps(dataset,ensure_ascii=False,indent=2,allow_nan=False),encoding='utf-8')
|
||||
(examples/(case['id']+'.json')).write_text(
|
||||
json.dumps({k:v for k,v in data.items() if k!='frames'},
|
||||
ensure_ascii=False,indent=2),encoding='utf-8')
|
||||
(results/(case['id']+'_result.json')).write_text(
|
||||
json.dumps(frames[-1],ensure_ascii=False,indent=2,allow_nan=False),encoding='utf-8')
|
||||
(results/'demo_data.json').write_text(
|
||||
json.dumps(dataset,ensure_ascii=False,indent=2,allow_nan=False),encoding='utf-8')
|
||||
template=(ASSETS / 'q1.template.html').read_text(encoding='utf-8')
|
||||
fragment=template.replace('/*__Q1_DATA__*/',json.dumps(dataset,ensure_ascii=False,allow_nan=False))
|
||||
fragment=template.replace('/*__Q1_DATA__*/',
|
||||
json.dumps(dataset,ensure_ascii=False,allow_nan=False))
|
||||
(ROOT/'output'/'q1'/'question1_full.html').write_text(fragment,encoding='utf-8')
|
||||
# A standalone companion uses the same fragment and computation results.
|
||||
theme=(ASSETS / 'standalone.css').read_text(encoding='utf-8')
|
||||
page='<!doctype html><html lang="zh-CN"><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>B题问题一交会定位</title><style>'+theme+'</style><body>'+fragment+'</body></html>'
|
||||
page=('<!doctype html><html lang="zh-CN"><meta charset="utf-8">'
|
||||
'<meta name="viewport" content="width=device-width,initial-scale=1">'
|
||||
'<title>B题问题一交会定位</title>'
|
||||
'<style>'+theme+'</style><body>'+fragment+'</body></html>')
|
||||
(results/'question1.html').write_text(page,encoding='utf-8')
|
||||
print(json.dumps({'cases':len(dataset),'fragment_bytes':len(fragment.encode('utf-8')),
|
||||
'standalone':str(results/'question1.html')},ensure_ascii=False))
|
||||
|
||||
@@ -43,8 +43,10 @@ class PracticeClient:
|
||||
status = response.status
|
||||
body = json.load(response)
|
||||
# A retried action is recorded/consumed only once.
|
||||
self.events.append(dict(path=path, request=payload, response=body, retries=attempt))
|
||||
self.output.write_text(json.dumps(self.events,ensure_ascii=False,indent=2),encoding='utf-8')
|
||||
self.events.append(dict(path=path, request=payload, response=body,
|
||||
retries=attempt))
|
||||
self.output.write_text(
|
||||
json.dumps(self.events,ensure_ascii=False,indent=2),encoding='utf-8')
|
||||
if status != 200 or body.get('accepted') is not True:
|
||||
raise RuntimeError(f'动作未被接受: {status}, {body}')
|
||||
if path == '/enter':
|
||||
@@ -54,6 +56,7 @@ class PracticeClient:
|
||||
if attempt == 2:
|
||||
raise
|
||||
time.sleep(.25)
|
||||
raise URLError('网络重试次数已用尽')
|
||||
|
||||
def measure(self, p, channel):
|
||||
return self.action('/measure',position={'x':p[0],'y':p[1]},channel=channel)
|
||||
@@ -106,12 +109,15 @@ def main():
|
||||
if len(observations)>=3:
|
||||
break
|
||||
data={'name':'官方问题3演练观测','source':'official_practice','case_code':args.case_code,
|
||||
'channel':channel,'ground_truth':None,'measurements':[m.__dict__ for m in observations],
|
||||
'channel':channel,'ground_truth':None,
|
||||
'measurements':[m.__dict__ for m in observations],
|
||||
'repeated_station_response':repeat,'result':solve(observations),
|
||||
'scope':'仅采集问题一观测,不执行问题三全清除任务;没有调用clear'}
|
||||
path=args.output/f'{args.case_code}_observations.json'
|
||||
path.write_text(json.dumps(data,ensure_ascii=False,indent=2,allow_nan=False),encoding='utf-8')
|
||||
print(json.dumps({'observations':str(path),'count':len(observations),'result':data['result']},ensure_ascii=False,indent=2))
|
||||
path.write_text(json.dumps(data,ensure_ascii=False,indent=2,allow_nan=False),
|
||||
encoding='utf-8')
|
||||
print(json.dumps({'observations':str(path),'count':len(observations),
|
||||
'result':data['result']},ensure_ascii=False,indent=2))
|
||||
finally:
|
||||
if entered:
|
||||
client.action('/exit')
|
||||
|
||||
@@ -22,7 +22,8 @@ from referee import LocalSimulator, Source
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--split', choices=('development','validation','stress'), default='development')
|
||||
parser.add_argument('--split', choices=('development','validation','stress'),
|
||||
default='development')
|
||||
parser.add_argument('--variants', default='baseline,legacy,revised')
|
||||
parser.add_argument('--problems', default='3,4')
|
||||
parser.add_argument('--model-dir', type=Path, default=Path('output/rl/review-20260911'))
|
||||
@@ -49,7 +50,8 @@ def main():
|
||||
repeated_measures=sum(r['repeated_measures'] for r in part))
|
||||
result = dict(source='local_referee_only', config=asdict(config), split=args.split,
|
||||
rows=rows, aggregates=aggregates)
|
||||
(output/'results.json').write_text(json.dumps(result,ensure_ascii=False,indent=2),encoding='utf-8')
|
||||
(output/'results.json').write_text(
|
||||
json.dumps(result,ensure_ascii=False,indent=2),encoding='utf-8')
|
||||
return aggregates
|
||||
for problem in map(int, args.problems.split(',')):
|
||||
scenes = []
|
||||
@@ -71,7 +73,8 @@ def main():
|
||||
scenes.append((f'{count}-{seed}',count,seed,sources))
|
||||
for name,count,seed,sources in scenes:
|
||||
scene_data = dict(sources=[asdict(s) for s in sources], error_seed=seed+2000000)
|
||||
(output/f'q{problem}-{name}-scene.json').write_text(json.dumps(scene_data),encoding='utf-8')
|
||||
scene_path = output/f'q{problem}-{name}-scene.json'
|
||||
scene_path.write_text(json.dumps(scene_data),encoding='utf-8')
|
||||
for variant in variants:
|
||||
model = None
|
||||
cls = BayesLoop if problem==3 else DirectionalExtension
|
||||
@@ -81,13 +84,15 @@ def main():
|
||||
model = (Path(f'output/rl/q{problem}_ppo.zip') if variant=='legacy'
|
||||
else args.model_dir/f'q{problem}_masked.zip')
|
||||
kwargs = dict(model_path=model, device='cpu')
|
||||
sim = LocalSimulator(sources,robot_id='RL-AUDIT',spec=referee_spec(config),error_seed=seed+2000000)
|
||||
sim = LocalSimulator(sources,robot_id='RL-AUDIT',spec=referee_spec(config),
|
||||
error_seed=seed+2000000)
|
||||
client = SimulatorClient('RL-AUDIT',config=config,transport=LocalSimTransport(sim))
|
||||
start = perf_counter()
|
||||
summary = cls(client,config,**kwargs).run()
|
||||
events = client.log.action_events()
|
||||
truth = {s.channel for s in sources}
|
||||
successful = {e['request']['channel'] for e in events if e['response'].get('clear_result')=='success'}
|
||||
successful = {e['request']['channel'] for e in events
|
||||
if e['response'].get('clear_result')=='success'}
|
||||
seen = set()
|
||||
repeats = 0
|
||||
for e in events:
|
||||
@@ -102,12 +107,15 @@ def main():
|
||||
and summary['exit_reason']=='user_exit' and not mismatches)
|
||||
row = dict(case=name,problem=problem,variant=variant,valid=valid,
|
||||
virtual_s=client.state.virtual_time_s,all_cleared=successful==truth,
|
||||
measures=summary['measures'],failed_clears=summary['clears']-len(successful),
|
||||
measures=summary['measures'],
|
||||
failed_clears=summary['clears']-len(successful),
|
||||
repeated_measures=repeats,wall_s=perf_counter()-start,
|
||||
model_sha256=hashlib.sha256(model.read_bytes()).hexdigest() if model else None)
|
||||
model_sha256=(hashlib.sha256(model.read_bytes()).hexdigest()
|
||||
if model else None))
|
||||
rows.append(row)
|
||||
(output/f'q{problem}-{name}-{variant}.json').write_text(json.dumps(
|
||||
dict(row=row,summary=summary,events=events,timing_mismatches=mismatches),ensure_ascii=False),encoding='utf-8')
|
||||
dict(row=row,summary=summary,events=events,timing_mismatches=mismatches),
|
||||
ensure_ascii=False),encoding='utf-8')
|
||||
save()
|
||||
print(json.dumps(row),flush=True)
|
||||
print(json.dumps(save(),indent=2),flush=True)
|
||||
|
||||
@@ -88,7 +88,7 @@ def collect(problem: int, episodes: int, seed: int, config: ProblemConfig):
|
||||
max_episode_steps=16)
|
||||
observations, actions, groups = [], [], []
|
||||
for episode in range(episodes):
|
||||
obs, info = env.reset(seed=seed + episode)
|
||||
obs, _info = env.reset(seed=seed + episode)
|
||||
for _step in range(env.max_episode_steps + 2):
|
||||
action = expert_action(env)
|
||||
observations.append(obs)
|
||||
@@ -108,7 +108,8 @@ def train_bc(x, y, *, epochs: int, batch_size: int, lr: float,
|
||||
episodes = np.unique(groups)
|
||||
if len(episodes) < 2:
|
||||
raise ValueError('行为克隆至少需要两个独立episode才能划分验证集')
|
||||
valid_groups = np.random.default_rng(0).permutation(episodes)[:max(1, int(len(episodes)*valid_frac))]
|
||||
valid_groups = np.random.default_rng(0).permutation(episodes)
|
||||
valid_groups = valid_groups[:max(1, int(len(episodes)*valid_frac))]
|
||||
valid = torch.as_tensor(np.isin(groups, valid_groups), device=device)
|
||||
x_t, y_t = torch.as_tensor(x, device=device), torch.as_tensor(y, device=device)
|
||||
x_v, y_v = x_t[valid], y_t[valid]
|
||||
|
||||
@@ -6,17 +6,17 @@ import math
|
||||
from pathlib import Path
|
||||
import re
|
||||
import xml.etree.ElementTree as ET
|
||||
from PIL import Image,ImageDraw,ImageFont
|
||||
from PIL import Image,ImageDraw,ImageFont # pylint: disable=import-error # 可选依赖
|
||||
|
||||
ROOT=Path(__file__).resolve().parents[1]
|
||||
S=2
|
||||
FONT=ImageFont.truetype('C:/Windows/Fonts/msyh.ttc',12*S)
|
||||
|
||||
|
||||
def render(path):
|
||||
root=ET.fromstring(path.read_text(encoding='utf-8'))
|
||||
def render(svg_path):
|
||||
root=ET.fromstring(svg_path.read_text(encoding='utf-8'))
|
||||
width,height=map(float,(root.get('width'),root.get('height')))
|
||||
bg='#191c21' if 'dark' in path.stem else '#ffffff'
|
||||
bg='#191c21' if 'dark' in svg_path.stem else '#ffffff'
|
||||
image=Image.new('RGBA',(round(width*S),round(height*S)),bg)
|
||||
clip=next(el for el in root.iter() if el.tag.endswith('clipPath'))[0]
|
||||
clipbox=tuple(float(clip.get(k))*S for k in ('x','y','width','height'))
|
||||
@@ -34,7 +34,8 @@ def render(path):
|
||||
for child in el:draw(child,clipped or bool(el.get('clip-path')))
|
||||
return
|
||||
layer=Image.new('RGBA',image.size,(0,0,0,0));d=ImageDraw.Draw(layer)
|
||||
num=lambda key,default=0:float(el.get(key,default))*S
|
||||
def num(key,default=0):
|
||||
return float(el.get(key,default))*S
|
||||
fill=color(el.get('fill'),float(el.get('fill-opacity',1)))
|
||||
stroke=color(el.get('stroke'),float(el.get('stroke-opacity',1)))
|
||||
sw=max(1,round(num('stroke-width',1)))
|
||||
@@ -43,13 +44,15 @@ def render(path):
|
||||
dash=el.get('stroke-dasharray','none')
|
||||
if dash=='none': d.line(xy,fill=stroke,width=sw)
|
||||
else:
|
||||
a,b=xy;dist=math.dist(a,b);unit=((b[0]-a[0])/max(dist,1e-15),(b[1]-a[1])/max(dist,1e-15))
|
||||
a,b=xy;dist=math.dist(a,b)
|
||||
unit=((b[0]-a[0])/max(dist,1e-15),(b[1]-a[1])/max(dist,1e-15))
|
||||
on,off=[float(v)*S for v in dash.split()]
|
||||
# Bound offscreen ray work to the finite exported viewport vicinity.
|
||||
segments=min(100000,int(dist/(on+off))+1)
|
||||
for i in range(segments):
|
||||
lo=i*(on+off);hi=min(lo+on,dist)
|
||||
d.line([(a[0]+lo*unit[0],a[1]+lo*unit[1]),(a[0]+hi*unit[0],a[1]+hi*unit[1])],fill=stroke,width=sw)
|
||||
d.line([(a[0]+lo*unit[0],a[1]+lo*unit[1]),
|
||||
(a[0]+hi*unit[0],a[1]+hi*unit[1])],fill=stroke,width=sw)
|
||||
elif tag=='rect':
|
||||
box=(num('x'),num('y'),num('x')+num('width'),num('y')+num('height'))
|
||||
d.rectangle(box,fill=fill,outline=stroke,width=sw)
|
||||
@@ -63,7 +66,8 @@ def render(path):
|
||||
d.polygon(points,fill=fill)
|
||||
if stroke:d.line(points+[points[0]],fill=stroke,width=sw,joint='curve')
|
||||
elif tag=='path':
|
||||
for x,y,axis,length in re.findall(r'M([\d.e+-]+),([\d.e+-]+)([hv])([\d.e+-]+)',el.get('d')):
|
||||
pattern=r'M([\d.e+-]+),([\d.e+-]+)([hv])([\d.e+-]+)'
|
||||
for x,y,axis,length in re.findall(pattern,el.get('d')):
|
||||
x,y,length=float(x)*S,float(y)*S,float(length)*S
|
||||
d.line([(x,y),(x+length,y) if axis=='h' else (x,y+length)],fill=stroke,width=sw)
|
||||
elif tag=='text':
|
||||
@@ -86,7 +90,7 @@ def render(path):
|
||||
image=Image.alpha_composite(image,layer)
|
||||
|
||||
draw(root)
|
||||
output=path.with_suffix('.png');image.convert('RGB').save(output)
|
||||
output=svg_path.with_suffix('.png');image.convert('RGB').save(output)
|
||||
return str(output.relative_to(ROOT))
|
||||
|
||||
|
||||
|
||||
@@ -32,8 +32,9 @@ def main():
|
||||
metadata = {"source": "official_practice", "problem": 3,
|
||||
"strategy": "seven_batch", "started_local": datetime.now().astimezone().isoformat(),
|
||||
"config": asdict(config), "coverage_points": seven_points(),
|
||||
"source_sha256": {str(p.relative_to(root)): hashlib.sha256(p.read_bytes()).hexdigest()
|
||||
for p in paths}}
|
||||
"source_sha256": {
|
||||
str(p.relative_to(root)): hashlib.sha256(p.read_bytes()).hexdigest()
|
||||
for p in paths}}
|
||||
def save(name, value):
|
||||
(args.output / name).write_text(json.dumps(value, ensure_ascii=False, indent=2,
|
||||
allow_nan=False) + "\n", encoding="utf-8")
|
||||
|
||||
@@ -12,7 +12,8 @@ from mcm26b.rl.features import build_observation
|
||||
|
||||
def main():
|
||||
root = Path('output/rl/review-20260911')
|
||||
records = json.loads((root/'evaluation/validation/final/results.json').read_text(encoding='utf-8'))
|
||||
records = json.loads((root/'evaluation/validation/final/results.json')
|
||||
.read_text(encoding='utf-8'))
|
||||
rng = np.random.default_rng(20260911)
|
||||
comparisons = {}
|
||||
for problem in (3,4):
|
||||
@@ -46,13 +47,16 @@ def main():
|
||||
bc = {}
|
||||
for problem in (3,4):
|
||||
# 使用旧检查点和原始专家输入,比对原ReLU网络与拷贝后的Tanh网络。
|
||||
checkpoint = torch.load(f'output/rl/bc/q{problem}_bc.pt',map_location='cpu',weights_only=True)
|
||||
checkpoint = torch.load(f'output/rl/bc/q{problem}_bc.pt',map_location='cpu',
|
||||
weights_only=True)
|
||||
data = np.load(f'output/rl/bc/q{problem}_expert.npz',allow_pickle=False)
|
||||
x = torch.from_numpy(data['obs'][::max(1,len(data['obs'])//1000)].copy())
|
||||
def network(activation):
|
||||
|
||||
def network(activation, checkpoint=checkpoint):
|
||||
net = torch.nn.Sequential(torch.nn.Linear(401,128),activation(),
|
||||
torch.nn.Linear(128,128),activation(),torch.nn.Linear(128,60))
|
||||
net.load_state_dict({k.removeprefix('net.'):v for k,v in checkpoint['state_dict'].items()})
|
||||
state = {k.removeprefix('net.'):v for k,v in checkpoint['state_dict'].items()}
|
||||
net.load_state_dict(state)
|
||||
return net
|
||||
with torch.no_grad():
|
||||
before,after=network(torch.nn.ReLU)(x),network(torch.nn.Tanh)(x)
|
||||
@@ -63,9 +67,11 @@ def main():
|
||||
limitations='置信区间仅重采样固定模型的场景,不覆盖训练随机种子不确定性。',
|
||||
code_sha256={str(p):hashlib.sha256(p.read_bytes()).hexdigest()
|
||||
for p in sorted(Path('src/mcm26b/rl').glob('*.py'))})
|
||||
(root/'review_evidence.json').write_text(json.dumps(report,ensure_ascii=False,indent=2),encoding='utf-8')
|
||||
(root/'review_evidence.json').write_text(
|
||||
json.dumps(report,ensure_ascii=False,indent=2),encoding='utf-8')
|
||||
print(json.dumps(dict(comparisons=comparisons,feature_probe=features,bc_probe=bc,
|
||||
official=[dict(case=r['case_code'],passed=r['passed']) for r in official]),indent=2))
|
||||
official=[dict(case=r['case_code'],passed=r['passed'])
|
||||
for r in official]),indent=2))
|
||||
|
||||
|
||||
if __name__=='__main__':
|
||||
|
||||
+382
-107
@@ -8,13 +8,31 @@
|
||||
当前 HTTP 执行路径仍由高层宏动作终点决定,不把八方向推演当正式收益。
|
||||
|
||||
落盘保证(长训必需):
|
||||
- ``--checkpoint-every`` 每 N 个环境步在 ``<模型目录>/checkpoints/`` 落一个可续训
|
||||
的 checkpoint(zip + 同名 json 元数据),只保留最近 ``--checkpoint-keep`` 个;
|
||||
- 训练前端的「停止」是对**进程组**发 SIGTERM,因此 subproc worker 在 :func:`_make_env`
|
||||
里显式忽略 SIGTERM/SIGINT,主进程收到信号后在 step 边界收尾并保存模型,
|
||||
不会被 worker 先行退出打断;
|
||||
- 任何异常(含被强杀之外的失败)都会先落一份 ``*_emergency.zip`` 再抛出,
|
||||
不再出现"跑了很久什么都没有"。
|
||||
- **急停可恢复**:``--recovery-every``(默认 2000 环境步)把一份滚动恢复点
|
||||
``<模型目录>/checkpoints/<stem>_latest.zip`` 原子覆盖写;训练开始前先写一份
|
||||
起步基线。因此即使进程被 SIGKILL / ``taskkill /F`` 强杀(没有任何 Python
|
||||
收尾机会),也**最多只丢 ``--recovery-every`` 个环境步**,且磁盘上始终有一份
|
||||
完整、可加载、可 ``--resume`` 的产物;
|
||||
- **原子落盘**:模型 zip、旁挂 json 与 ``<run_dir>/resume.json`` 都是"临时文件 +
|
||||
``os.replace``",保存途中被杀不会留下半截文件,也不会破坏上一份完整快照;
|
||||
- ``--checkpoint-every`` 每 N 个环境步在 ``<模型目录>/checkpoints/`` 另留一份历史
|
||||
checkpoint(``<stem>_<步数>.zip``),只保留最近 ``--checkpoint-keep`` 个,供
|
||||
返祖对比与多档恢复;
|
||||
- ``<run_dir>/resume.json`` 是恢复的**权威指针**:无论上一刻是滚动恢复点、历史
|
||||
checkpoint、优雅收尾还是异常兜底,它总指向一份可续训产物,并记录增量语义
|
||||
(``timesteps`` 是追加步数)与生效过的 PPO 超参;
|
||||
- ``--control-file`` 是训练前端的**跨平台**控制通道:前端原子写 JSON
|
||||
(``paused`` / ``stop``),本脚本每个 step 轮询。暂停时训练在 step 边界挂起,
|
||||
停止时先保存模型与元数据再退出;
|
||||
- 信号(SIGTERM/SIGINT,Windows 上另有 SIGBREAK)仍可用:直接 kill 本脚本或
|
||||
前端发进程组信号时同样走优雅收尾;subproc worker 在 :func:`_make_env` 里忽略
|
||||
这些信号,保证它们活到主进程 ``vec_env.close()`` 之后再退出;
|
||||
- 任何异常都会先落一份 ``*_emergency.zip`` + ``resume.json`` 再抛出,不再出现
|
||||
"跑了很久什么都没有"。
|
||||
|
||||
为什么暂停/停止不靠信号:Windows 没有 SIGSTOP,且 ``os.kill(pid, SIGTERM)`` 会
|
||||
退化成 ``TerminateProcess`` 硬杀——收尾分支根本不会执行。控制文件两端都只是
|
||||
普通文件 IO,POSIX 与 Windows 行为一致(见 :mod:`mcm26b.process_control`)。
|
||||
|
||||
P1 smoke 示例:
|
||||
python scripts/train_hrl_high.py \
|
||||
@@ -29,6 +47,7 @@ from __future__ import annotations
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import platform
|
||||
import random
|
||||
@@ -39,9 +58,10 @@ from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from mcm26b.config import ProblemConfig
|
||||
from mcm26b.hrl.curriculum import STAGES, training_config
|
||||
from mcm26b.hrl.curriculum import STAGES
|
||||
from mcm26b.hrl.constants import (ACTION_LAYOUT_VERSION, FEATURE_VERSION,
|
||||
HIGH_OBS_DIM, N_CANDIDATES, REWARD_VERSION)
|
||||
from mcm26b.process_control import ControlState, read_control
|
||||
|
||||
|
||||
def _stage_from_index(index: int):
|
||||
@@ -68,24 +88,86 @@ def _reward_from_profile(profile: str):
|
||||
raise ValueError(f"未知 reward-profile: {profile}")
|
||||
|
||||
|
||||
def _ignore_stop_signals_in_worker() -> None:
|
||||
"""subproc worker 忽略 SIGTERM/SIGINT,保证主进程能优雅收尾落盘。
|
||||
def _stop_signals() -> list[int]:
|
||||
"""本平台可用于"请求收尾"的信号集合。
|
||||
|
||||
训练前端的「停止」向整个进程组发 SIGTERM;若 worker 同时退出,主进程会在
|
||||
保存模型之前就拿到断管/EOF 错误,整局训练白跑。worker 活在管道后面,主进程
|
||||
收尾时通过 ``vec_env.close()`` 的 ``close`` 指令正常结束它们。
|
||||
Windows 上没有 SIGTERM 的软语义(``os.kill`` 直接 TerminateProcess),但
|
||||
``SIGBREAK`` 可以由 ``CTRL_BREAK_EVENT`` 触发并进入 Python 处理器;POSIX 上
|
||||
SIGTERM/SIGINT 均可捕获。控制文件才是主通道,这里只是让"直接 kill 脚本"
|
||||
也尽量走优雅收尾。
|
||||
"""
|
||||
signals = [signal.SIGTERM, signal.SIGINT]
|
||||
sigbreak = getattr(signal, "SIGBREAK", None)
|
||||
if sigbreak is not None:
|
||||
signals.append(sigbreak)
|
||||
return signals
|
||||
|
||||
|
||||
def _ignore_stop_signals_in_worker() -> None:
|
||||
"""subproc worker 忽略收尾信号,保证主进程能优雅落盘。
|
||||
|
||||
无论信号来自进程组 SIGTERM 还是 Windows ``CTRL_BREAK_EVENT``,若 worker
|
||||
同时退出,主进程会在保存模型之前就拿到断管/EOF 错误,整局训练白跑。worker
|
||||
活在管道后面,主进程收尾时通过 ``vec_env.close()`` 的 ``close`` 指令正常
|
||||
结束它们。
|
||||
"""
|
||||
try:
|
||||
import multiprocessing
|
||||
if multiprocessing.current_process().name == "MainProcess":
|
||||
return
|
||||
for signum in (signal.SIGTERM, signal.SIGINT):
|
||||
for signum in _stop_signals():
|
||||
signal.signal(signum, signal.SIG_IGN)
|
||||
except (ImportError, ValueError, OSError): # 非主线程/平台不支持:不影响训练
|
||||
return
|
||||
|
||||
|
||||
def _make_env(problem: int, stage_index: int, seed: int,
|
||||
class _ControlPoll:
|
||||
"""控制文件轮询状态机(纯标准库,可脱离 SB3 单测)。
|
||||
|
||||
``poll()`` 每 ``poll_every`` 次调用才真正读盘,避免每个 env step 都做一次
|
||||
文件 IO;``wait_while_paused()`` 阻塞到暂停解除或收到 stop,并累计暂停秒数
|
||||
(``--max-wall-s`` 要把这段扣掉,否则"挂起一小时回来就判超时")。
|
||||
"""
|
||||
|
||||
def __init__(self, path: Path, *, poll_every: int = 32,
|
||||
poll_s: float = 0.2):
|
||||
self.path = Path(path)
|
||||
self.poll_every = max(1, int(poll_every))
|
||||
self.poll_s = max(0.02, float(poll_s))
|
||||
self.paused_total_s = 0.0
|
||||
self._calls = 0
|
||||
self._state = ControlState()
|
||||
|
||||
def poll(self) -> ControlState:
|
||||
"""按抽样间隔返回最近一次控制状态。"""
|
||||
self._calls += 1
|
||||
if self._calls % self.poll_every:
|
||||
return self._state
|
||||
self._state = read_control(self.path)
|
||||
return self._state
|
||||
|
||||
def wait_while_paused(self, on_pause, on_resume) -> bool:
|
||||
"""阻塞到继续或停止;返回 True 表示期间收到了 stop。
|
||||
|
||||
``on_pause`` / ``on_resume`` 用来向前端回显事件(训练脚本的 JSON 行)。
|
||||
"""
|
||||
on_pause()
|
||||
started = time.monotonic()
|
||||
stopped = False
|
||||
while True:
|
||||
time.sleep(self.poll_s)
|
||||
self._state = read_control(self.path)
|
||||
if self._state.stop:
|
||||
stopped = True
|
||||
break
|
||||
if not self._state.paused:
|
||||
break
|
||||
self.paused_total_s += time.monotonic() - started
|
||||
on_resume()
|
||||
return stopped
|
||||
|
||||
|
||||
def _make_env(problem: int, stage_index: int,
|
||||
monitor_path: str | None, profile: str,
|
||||
reward_profile: str):
|
||||
"""顶层工厂:供 DummyVecEnv / SubprocVecEnv 调用(必须是可 pickle 的)。"""
|
||||
@@ -112,7 +194,7 @@ def _build_vec_env(args, monitor_paths: list[str]):
|
||||
from functools import partial
|
||||
|
||||
factories = [
|
||||
partial(_make_env, args.problem, args.stage, args.seed + rank,
|
||||
partial(_make_env, args.problem, args.stage,
|
||||
monitor_paths[rank], args.config_profile, args.reward_profile)
|
||||
for rank in range(args.envs)
|
||||
]
|
||||
@@ -161,6 +243,52 @@ def _policy_hyperparams(model) -> dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
def _progress_payload(model, start: int) -> dict[str, Any]:
|
||||
"""组装 ``--progress-json`` 的单行负载。
|
||||
|
||||
纯函数(只读 model 的 logger / ep_info_buffer),便于脱离训练循环测试;
|
||||
NaN / ±inf 一律折算成 None,保证 ``json.dumps(allow_nan=False)`` 不会炸。
|
||||
"""
|
||||
values = getattr(model.logger, "name_to_value", {}) or {}
|
||||
|
||||
def _num(key, default=None):
|
||||
value = values.get(key)
|
||||
if value is None:
|
||||
return default
|
||||
try:
|
||||
value = float(value)
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
if not math.isfinite(value):
|
||||
return default
|
||||
return value
|
||||
|
||||
ep_rewards: list[float] = []
|
||||
ep_lengths: list[float] = []
|
||||
for item in getattr(model, "ep_info_buffer", []) or []:
|
||||
try:
|
||||
ep_rewards.append(float(item.get("r")))
|
||||
ep_lengths.append(float(item.get("l")))
|
||||
except (TypeError, ValueError, AttributeError):
|
||||
continue
|
||||
return {
|
||||
"event": "progress",
|
||||
"timesteps": int(model.num_timesteps),
|
||||
"start_timesteps": int(start),
|
||||
"n_updates": int(getattr(model, "_n_updates", 0) or 0),
|
||||
"ep_rew_mean": (sum(ep_rewards) / len(ep_rewards)
|
||||
if ep_rewards else None),
|
||||
"ep_len_mean": (sum(ep_lengths) / len(ep_lengths)
|
||||
if ep_lengths else None),
|
||||
"value_loss": _num("train/value_loss"),
|
||||
"policy_gradient_loss": _num("train/policy_gradient_loss"),
|
||||
"approx_kl": _num("train/approx_kl"),
|
||||
"entropy_loss": _num("train/entropy_loss"),
|
||||
"explained_variance": _num("train/explained_variance"),
|
||||
"fps": _num("time/fps"),
|
||||
}
|
||||
|
||||
|
||||
def _model_metadata(*, args, wall_s: float, n_envs: int,
|
||||
actual_timesteps: int) -> dict[str, Any]:
|
||||
import numpy
|
||||
@@ -190,12 +318,12 @@ def _model_metadata(*, args, wall_s: float, n_envs: int,
|
||||
"config_profile": args.config_profile,
|
||||
"fidelity": _profile_fidelity(args.config_profile),
|
||||
"reward_profile": args.reward_profile,
|
||||
"reward_version": REWARD_VERSION,
|
||||
"local_selector": args.local_selector,
|
||||
"local_model": args.local_model,
|
||||
"checkpoint_every": int(getattr(args, "checkpoint_every", 0) or 0),
|
||||
"checkpoint_keep": int(getattr(args, "checkpoint_keep", 0) or 0),
|
||||
"checkpoint_dir": getattr(args, "checkpoint_dir", None),
|
||||
"recovery_every": int(getattr(args, "recovery_every", 0) or 0),
|
||||
"device": args.device,
|
||||
"wall_seconds": round(wall_s, 3),
|
||||
"max_wall_s": float(args.max_wall_s),
|
||||
@@ -211,6 +339,68 @@ def _model_metadata(*, args, wall_s: float, n_envs: int,
|
||||
}
|
||||
|
||||
|
||||
def _write_json_atomic(path: Path, payload: dict[str, Any]) -> None:
|
||||
"""原子写 JSON:同目录 ``.tmp-`` 前缀临时文件 + ``os.replace``。
|
||||
|
||||
急停(SIGKILL / ``taskkill /F``)会绕过所有 Python 清理逻辑,保存途中被杀
|
||||
是完全可能的。原子替换保证**真实文件名下要么是旧的完整文件、要么是新的
|
||||
完整文件**,不会出现半截 JSON 让续训读到一半就崩。
|
||||
"""
|
||||
path = Path(path)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = path.with_name(f".tmp-{path.name}")
|
||||
tmp.write_text(json.dumps(payload, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8")
|
||||
os.replace(tmp, path)
|
||||
|
||||
|
||||
def _save_model_atomic(model, target: Path) -> Path:
|
||||
"""把 SB3 模型原子地写到 ``target``,返回实际落盘路径。
|
||||
|
||||
先写 ``.tmp-<name>``(同样以 ``.zip`` 结尾,SB3 不会再加后缀),完整写完后
|
||||
``os.replace`` 覆盖目标。于是:
|
||||
|
||||
- 保存途中被强杀 → 目标路径仍是上一份**完整**的快照,可继续复训;
|
||||
- 保存成功 → 目标路径立即变成新快照,不存在"看起来存在但打不开"的窗口。
|
||||
|
||||
临时名带 ``.tmp-`` 前缀,因此不会被 ``*_ckpt_*.zip`` 的清理逻辑误当成正式
|
||||
checkpoint。
|
||||
"""
|
||||
target = Path(target)
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = target.with_name(f".tmp-{target.name}")
|
||||
model.save(str(tmp))
|
||||
if not tmp.exists(): # SB3 对未知后缀可能自行补 .zip
|
||||
candidate = tmp.with_name(tmp.name + ".zip")
|
||||
if candidate.exists():
|
||||
tmp = candidate
|
||||
os.replace(tmp, target)
|
||||
return target
|
||||
|
||||
|
||||
def _write_resume_hint(*, run_dir: Path, model_path: Path, timesteps: int,
|
||||
reason: str, model=None) -> dict[str, Any]:
|
||||
"""把"从哪继续"写进 ``<run_dir>/resume.json``。
|
||||
|
||||
这是急停恢复的**权威指针**:无论上一刻是周期 checkpoint、滚动恢复点、
|
||||
优雅收尾还是异常兜底,这个文件总指向一份完整、可 ``--resume`` 的产物,
|
||||
并带上续训语义(``timesteps`` 是增量步数)与生效过的 PPO 超参。
|
||||
"""
|
||||
hint: dict[str, Any] = {
|
||||
"event": "resume_hint",
|
||||
"model": str(model_path),
|
||||
"timesteps": int(timesteps),
|
||||
"timesteps_semantics": "additional",
|
||||
"reason": str(reason),
|
||||
"created_unix_s": time.time(),
|
||||
"resume_args": ["--resume", "--model", str(model_path)],
|
||||
}
|
||||
if model is not None:
|
||||
hint["ppo_hyperparams"] = _policy_hyperparams(model)
|
||||
_write_json_atomic(run_dir / "resume.json", hint)
|
||||
return hint
|
||||
|
||||
|
||||
def _write_metadata(*, model_path: Path, args, wall_s: float, n_envs: int,
|
||||
actual_timesteps: int, monitor_paths: list[str],
|
||||
model=None,
|
||||
@@ -227,14 +417,18 @@ def _write_metadata(*, model_path: Path, args, wall_s: float, n_envs: int,
|
||||
"reward_profile=prompt 使用原始 §5.2 塑形。模型元数据已固化口径。")
|
||||
if extra:
|
||||
metadata.update(extra)
|
||||
model_path.with_suffix(".json").write_text(
|
||||
json.dumps(metadata, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
_write_json_atomic(model_path.with_suffix(".json"), metadata)
|
||||
return metadata
|
||||
|
||||
|
||||
def _prune_checkpoints(directory: Path, stem: str, keep: int) -> list[str]:
|
||||
"""只保留最近 ``keep`` 个 checkpoint(zip+json 同删),返回被删的 zip 路径。"""
|
||||
files = sorted(directory.glob(f"{stem}_*.zip"),
|
||||
"""只保留最近 ``keep`` 个 checkpoint(zip+json 同删),返回被删的 zip 路径。
|
||||
|
||||
只匹配 ``<stem>_<步数>.zip``;滚动恢复点 ``<stem>_latest.zip`` 与临时文件
|
||||
``.tmp-*.zip`` 都不匹配,因此永远不会被历史清理误删。
|
||||
"""
|
||||
files = sorted((path for path in directory.glob(f"{stem}_[0-9]*.zip")
|
||||
if path.stem.rsplit("_", 1)[-1].isdigit()),
|
||||
key=lambda path: path.stat().st_mtime, reverse=True)
|
||||
removed: list[str] = []
|
||||
for stale in files[max(1, int(keep)):]:
|
||||
@@ -267,7 +461,7 @@ def _evaluate_env(model, *, problem: int, stage_index: int,
|
||||
while not done:
|
||||
action, _state = model.predict(
|
||||
obs, action_masks=env.action_masks(), deterministic=True)
|
||||
obs, reward, terminated, truncated, info = env.step(int(action))
|
||||
obs, _, terminated, truncated, info = env.step(int(action))
|
||||
done = bool(terminated or truncated)
|
||||
steps += 1
|
||||
# 课程阶段的小场景只需证明全部频道 accounted;完整阶段至少应清掉
|
||||
@@ -299,7 +493,7 @@ def _evaluate_env(model, *, problem: int, stage_index: int,
|
||||
|
||||
|
||||
def _evaluate_manifest(model_path: Path, manifest_path: str, *,
|
||||
limit: int, problem: int) -> dict[str, Any]:
|
||||
limit: int) -> dict[str, Any]:
|
||||
"""可选:用 benchmark_hrl 的同一套真值隔离评测在固定 manifest 上跑已训模型。"""
|
||||
scripts_dir = Path(__file__).resolve().parent
|
||||
if str(scripts_dir) not in sys.path:
|
||||
@@ -404,26 +598,72 @@ def run_training(args) -> int:
|
||||
"命令行 rollout-steps/batch-size/n-epochs 被忽略")},
|
||||
ensure_ascii=False), file=sys.stderr, flush=True)
|
||||
|
||||
# ------------------------------------------------------------ 优雅停止
|
||||
# 训练前端「停止」= killpg(SIGTERM):worker 已在 _make_env 里忽略该信号,
|
||||
# 主进程在此装处理器,等当前 step 结束后收尾落盘。
|
||||
# ------------------------------------------------------ 优雅暂停 / 停止
|
||||
# 主通道是控制文件(跨平台);信号是兜底:直接 kill 本进程时也走同一段收尾。
|
||||
stop_state: dict[str, Any] = {"requested": False, "signal": None}
|
||||
previous_handlers: dict[int, Any] = {}
|
||||
|
||||
def _on_stop_signal(signum, _frame):
|
||||
def _request_stop(source: str) -> None:
|
||||
if stop_state["requested"]:
|
||||
return
|
||||
stop_state["requested"] = True
|
||||
stop_state["signal"] = signal.Signals(signum).name
|
||||
print(json.dumps({"event": "stop_requested",
|
||||
"signal": signal.Signals(signum).name}), flush=True)
|
||||
stop_state["signal"] = source
|
||||
print(json.dumps({"event": "stop_requested", "signal": source}), flush=True)
|
||||
|
||||
for _signum in (signal.SIGTERM, signal.SIGINT):
|
||||
def _on_stop_signal(signum, _frame):
|
||||
_request_stop(signal.Signals(signum).name)
|
||||
|
||||
for _signum in _stop_signals():
|
||||
try:
|
||||
previous_handlers[_signum] = signal.signal(_signum, _on_stop_signal)
|
||||
except (ValueError, OSError): # 非主线程/平台不支持:不阻塞训练
|
||||
pass
|
||||
|
||||
class _ControlChannel(BaseCallback):
|
||||
"""把 :class:`_ControlPoll` 接到 SB3 的 step 回调上。
|
||||
|
||||
为什么不用信号:Windows 没有 SIGSTOP,且 ``os.kill(SIGTERM)`` 会退化成
|
||||
``TerminateProcess`` 硬杀,收尾分支根本不会执行。控制文件在 POSIX 与
|
||||
Windows 上都只是普通文件 IO,语义完全一致。
|
||||
|
||||
暂停是**协作式**的:在 ``_on_step`` 内阻塞轮询,训练因此停在 step 边界,
|
||||
进程仍存活、显存仍占用,但不消耗算力;收到 stop 立刻转入收尾落盘。
|
||||
"""
|
||||
|
||||
def __init__(self, path: Path, *, poll_every: int = 32,
|
||||
poll_s: float = 0.2):
|
||||
super().__init__()
|
||||
self.poller = _ControlPoll(path, poll_every=poll_every,
|
||||
poll_s=poll_s)
|
||||
|
||||
def paused_seconds(self) -> float:
|
||||
return self.poller.paused_total_s
|
||||
|
||||
def _event(self, name: str) -> None:
|
||||
print(json.dumps({"event": name,
|
||||
"timesteps": int(self.model.num_timesteps)}),
|
||||
flush=True)
|
||||
|
||||
def _on_step(self) -> bool:
|
||||
state = self.poller.poll()
|
||||
if state.stop:
|
||||
_request_stop("control_file")
|
||||
return False
|
||||
if state.paused:
|
||||
stopped = self.poller.wait_while_paused(
|
||||
lambda: self._event("paused"),
|
||||
lambda: self._event("resumed"))
|
||||
if stopped:
|
||||
_request_stop("control_file")
|
||||
return False
|
||||
return not stop_state["requested"]
|
||||
return True
|
||||
|
||||
control_path = getattr(args, "control_file", None)
|
||||
control_cb = _ControlChannel(Path(control_path)) if control_path else None
|
||||
|
||||
class _GracefulStop(BaseCallback):
|
||||
"""收到停止信号后在最近一次 step 边界结束 rollout,让模型能落盘。"""
|
||||
"""收到停止请求后在最近一次 step 边界结束 rollout,让模型能落盘。"""
|
||||
|
||||
def _on_step(self) -> bool:
|
||||
if stop_state["requested"]:
|
||||
@@ -435,15 +675,22 @@ def run_training(args) -> int:
|
||||
return True
|
||||
|
||||
class _WallClockStop(BaseCallback):
|
||||
def __init__(self, max_wall_s: float):
|
||||
"""墙钟硬上限;暂停时间不计入,避免"挂起一小时回来就判超时"。"""
|
||||
|
||||
def __init__(self, max_wall_s: float, paused_clock=None):
|
||||
super().__init__()
|
||||
self.max_wall_s = max_wall_s
|
||||
self.start = time.monotonic()
|
||||
self.paused_clock = paused_clock
|
||||
|
||||
def paused_seconds(self) -> float:
|
||||
return float(self.paused_clock()) if self.paused_clock else 0.0
|
||||
|
||||
def _on_step(self) -> bool:
|
||||
if time.monotonic() - self.start >= self.max_wall_s:
|
||||
elapsed = time.monotonic() - self.start - self.paused_seconds()
|
||||
if elapsed >= self.max_wall_s:
|
||||
print(json.dumps({"event": "wall_clock_stop",
|
||||
"elapsed_s": time.monotonic() - self.start}))
|
||||
"elapsed_s": elapsed}))
|
||||
return False
|
||||
return True
|
||||
|
||||
@@ -473,44 +720,7 @@ def run_training(args) -> int:
|
||||
return True
|
||||
self.last_emit = current
|
||||
self.emitted += 1
|
||||
values = getattr(self.model.logger, "name_to_value", {}) or {}
|
||||
|
||||
def _num(key, default=None):
|
||||
value = values.get(key)
|
||||
if value is None:
|
||||
return default
|
||||
try:
|
||||
value = float(value)
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
if value != value or value in (float("inf"), float("-inf")):
|
||||
return default
|
||||
return value
|
||||
|
||||
ep_rewards = []
|
||||
ep_lengths = []
|
||||
for item in getattr(self.model, "ep_info_buffer", []) or []:
|
||||
try:
|
||||
ep_rewards.append(float(item.get("r")))
|
||||
ep_lengths.append(float(item.get("l")))
|
||||
except (TypeError, ValueError, AttributeError):
|
||||
continue
|
||||
payload = {
|
||||
"event": "progress",
|
||||
"timesteps": current,
|
||||
"start_timesteps": self.start,
|
||||
"n_updates": int(getattr(self.model, "_n_updates", 0) or 0),
|
||||
"ep_rew_mean": (sum(ep_rewards) / len(ep_rewards)
|
||||
if ep_rewards else None),
|
||||
"ep_len_mean": (sum(ep_lengths) / len(ep_lengths)
|
||||
if ep_lengths else None),
|
||||
"value_loss": _num("train/value_loss"),
|
||||
"policy_gradient_loss": _num("train/policy_gradient_loss"),
|
||||
"approx_kl": _num("train/approx_kl"),
|
||||
"entropy_loss": _num("train/entropy_loss"),
|
||||
"explained_variance": _num("train/explained_variance"),
|
||||
"fps": _num("time/fps"),
|
||||
}
|
||||
payload = _progress_payload(self.model, self.start)
|
||||
print(json.dumps(payload, ensure_ascii=False, allow_nan=False),
|
||||
flush=True)
|
||||
return True
|
||||
@@ -521,38 +731,79 @@ def run_training(args) -> int:
|
||||
checkpoint_stem = f"{model_path.stem}_ckpt"
|
||||
|
||||
class _CheckpointJSON(BaseCallback):
|
||||
"""每 N 个环境步落一个可续训 checkpoint(zip + 同名元数据)。
|
||||
"""可续训落盘:滚动恢复点 + 周期 checkpoint + resume.json 指针。
|
||||
|
||||
只保留最近 ``keep`` 个;checkpoint 与最终模型共用同名 json 契约,
|
||||
因此 ``--resume <checkpoint.zip>`` 直接可用。
|
||||
- **滚动恢复点** ``<stem>_latest.zip``:每 ``recovery_every`` 个环境步
|
||||
原子覆盖写一次,训练开始前也先写一份。它保证"急停"(SIGKILL /
|
||||
``taskkill /F``,没有任何 Python 清理机会)之后**最多只丢
|
||||
``recovery_every`` 个环境步**,而且文件始终完整可加载。
|
||||
- **周期 checkpoint** ``<stem>_<步数>.zip``:每 ``every_steps`` 步留一份,
|
||||
只保留最近 ``keep`` 个,用于返祖对比与多档恢复。
|
||||
- 每次落盘都刷新 ``<run_dir>/resume.json``:无论怎么停,都有一份权威的
|
||||
"从哪继续"指针(含增量语义与生效过的 PPO 超参)。
|
||||
"""
|
||||
|
||||
def __init__(self, every_steps: int, keep: int):
|
||||
def __init__(self, every_steps: int, keep: int, recovery_every: int):
|
||||
super().__init__()
|
||||
self.every_steps = max(1, int(every_steps))
|
||||
self.every_steps = max(0, int(every_steps))
|
||||
self.keep = max(1, int(keep))
|
||||
self.recovery_every = max(0, int(recovery_every))
|
||||
self.last_saved = int(model.num_timesteps)
|
||||
self.last_recovery = int(model.num_timesteps)
|
||||
self.saved: list[str] = []
|
||||
self.latest_path = checkpoints_dir / f"{checkpoint_stem}_latest.zip"
|
||||
|
||||
def write_recovery(self, reason: str) -> None:
|
||||
"""立刻写一份滚动恢复点;训练开始前调用即"起步基线"。
|
||||
|
||||
用闭包里的 ``model`` 而不是 ``self.model``:``BaseCallback.model``
|
||||
要等 ``learn()`` 时才被赋值,而起步基线恰恰在 ``learn()`` 之前写。
|
||||
"""
|
||||
current = int(model.num_timesteps)
|
||||
self.last_recovery = current
|
||||
self._write_recovery(current, reason)
|
||||
|
||||
def _on_step(self) -> bool:
|
||||
current = int(self.model.num_timesteps)
|
||||
if current - self.last_saved < self.every_steps:
|
||||
return True
|
||||
self.last_saved = current
|
||||
self._write(current)
|
||||
current = int(model.num_timesteps)
|
||||
if (self.recovery_every > 0
|
||||
and current - self.last_recovery >= self.recovery_every):
|
||||
self.last_recovery = current
|
||||
self._write_recovery(current, "interval")
|
||||
if (self.every_steps > 0
|
||||
and current - self.last_saved >= self.every_steps):
|
||||
self.last_saved = current
|
||||
self._write_interval(current)
|
||||
return True
|
||||
|
||||
def _write(self, current: int) -> None:
|
||||
checkpoints_dir.mkdir(parents=True, exist_ok=True)
|
||||
target = checkpoints_dir / f"{checkpoint_stem}_{current:09d}.zip"
|
||||
self.model.save(str(target))
|
||||
def _persist(self, target: Path, current: int, reason: str,
|
||||
extra: dict[str, Any]) -> None:
|
||||
"""原子落盘 zip + 元数据,并刷新 resume.json 指针。"""
|
||||
_save_model_atomic(model, target)
|
||||
_write_metadata(
|
||||
model_path=target, args=args,
|
||||
wall_s=time.monotonic() - started, n_envs=args.envs,
|
||||
actual_timesteps=current, monitor_paths=monitor_paths,
|
||||
model=self.model,
|
||||
extra={"checkpoint": True, "checkpoint_timesteps": current,
|
||||
"checkpoint_of": str(model_path)})
|
||||
model=model,
|
||||
extra={"checkpoint_timesteps": current,
|
||||
"checkpoint_reason": reason,
|
||||
"checkpoint_of": str(model_path), **extra})
|
||||
_write_resume_hint(run_dir=run_dir, model_path=target,
|
||||
timesteps=current, reason=reason,
|
||||
model=model)
|
||||
|
||||
def _write_recovery(self, current: int, reason: str) -> None:
|
||||
checkpoints_dir.mkdir(parents=True, exist_ok=True)
|
||||
self._persist(self.latest_path, current, reason,
|
||||
{"recovery": True})
|
||||
print(json.dumps({"event": "recovery_checkpoint",
|
||||
"reason": reason, "timesteps": current,
|
||||
"model": str(self.latest_path)},
|
||||
ensure_ascii=False), flush=True)
|
||||
|
||||
def _write_interval(self, current: int) -> None:
|
||||
checkpoints_dir.mkdir(parents=True, exist_ok=True)
|
||||
target = checkpoints_dir / f"{checkpoint_stem}_{current:09d}.zip"
|
||||
self._persist(target, current, "interval", {"checkpoint": True})
|
||||
removed = _prune_checkpoints(checkpoints_dir, checkpoint_stem,
|
||||
self.keep)
|
||||
self.saved.append(str(target))
|
||||
@@ -562,15 +813,24 @@ def run_training(args) -> int:
|
||||
|
||||
progress_every = (args.progress_every if args.progress_every > 0
|
||||
else args.rollout_steps * args.envs)
|
||||
checkpoint_cb = None
|
||||
if args.checkpoint_every > 0:
|
||||
checkpoint_cb = _CheckpointJSON(args.checkpoint_every,
|
||||
args.checkpoint_keep)
|
||||
callbacks = [_GracefulStop(), _WallClockStop(args.max_wall_s)]
|
||||
if checkpoint_cb is not None:
|
||||
callbacks.append(checkpoint_cb)
|
||||
checkpoint_cb = _CheckpointJSON(args.checkpoint_every, args.checkpoint_keep,
|
||||
args.recovery_every)
|
||||
if args.recovery_every > 0:
|
||||
# 起步基线:训练还没跑就急停,也至少有一份可加载、可续训的产物
|
||||
checkpoint_cb.write_recovery("baseline")
|
||||
# 顺序有语义:控制通道最先(暂停/停止要在本步立刻生效),进度次之,
|
||||
# 墙钟上限要能读到控制通道累计的暂停秒数,checkpoint 放最后。
|
||||
callbacks: list[Any] = []
|
||||
if control_cb is not None:
|
||||
callbacks.append(control_cb)
|
||||
if args.progress_json:
|
||||
callbacks.insert(0, _ProgressJSON(progress_every))
|
||||
callbacks.append(_ProgressJSON(progress_every))
|
||||
callbacks.append(_GracefulStop())
|
||||
callbacks.append(_WallClockStop(
|
||||
args.max_wall_s,
|
||||
paused_clock=(control_cb.paused_seconds if control_cb is not None
|
||||
else None)))
|
||||
callbacks.append(checkpoint_cb)
|
||||
|
||||
actual_timesteps = int(model.num_timesteps)
|
||||
try:
|
||||
@@ -587,7 +847,7 @@ def run_training(args) -> int:
|
||||
}
|
||||
emergency = run_dir / f"{model_path.stem}_emergency.zip"
|
||||
try:
|
||||
model.save(str(emergency))
|
||||
_save_model_atomic(model, emergency)
|
||||
_write_metadata(model_path=emergency, args=args,
|
||||
wall_s=time.monotonic() - started,
|
||||
n_envs=args.envs,
|
||||
@@ -595,27 +855,34 @@ def run_training(args) -> int:
|
||||
monitor_paths=monitor_paths, model=model,
|
||||
extra={"emergency": True,
|
||||
"error": failure["error"]})
|
||||
_write_resume_hint(run_dir=run_dir, model_path=emergency,
|
||||
timesteps=int(model.num_timesteps),
|
||||
reason="emergency", model=model)
|
||||
failure["emergency_model"] = str(emergency)
|
||||
except Exception as save_exc: # 落盘失败不能再掩盖原始异常
|
||||
failure["emergency_save_error"] = (
|
||||
f"{type(save_exc).__name__}: {save_exc}")
|
||||
(run_dir / "training_failure.json").write_text(
|
||||
json.dumps(failure, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
_write_json_atomic(run_dir / "training_failure.json", failure)
|
||||
print(json.dumps(failure, ensure_ascii=False), file=sys.stderr,
|
||||
flush=True)
|
||||
raise
|
||||
wall_s = time.monotonic() - started
|
||||
actual_timesteps = int(model.num_timesteps)
|
||||
stopped_by_signal = bool(stop_state["requested"])
|
||||
model.save(str(model_path))
|
||||
_save_model_atomic(model, model_path)
|
||||
metadata = _write_metadata(
|
||||
model_path=model_path, args=args, wall_s=wall_s, n_envs=args.envs,
|
||||
actual_timesteps=actual_timesteps, monitor_paths=monitor_paths,
|
||||
model=model,
|
||||
extra={"stopped_by_signal": stopped_by_signal,
|
||||
"stop_signal": stop_state["signal"],
|
||||
"checkpoint_files": (checkpoint_cb.saved
|
||||
if checkpoint_cb is not None else [])})
|
||||
"checkpoint_files": checkpoint_cb.saved,
|
||||
"recovery_model": str(checkpoint_cb.latest_path)})
|
||||
# 最终模型也刷新一次 resume.json:优雅收尾后"从哪继续"指向最终模型
|
||||
_write_resume_hint(run_dir=run_dir, model_path=model_path,
|
||||
timesteps=actual_timesteps,
|
||||
reason=("stopped" if stopped_by_signal else "finished"),
|
||||
model=model)
|
||||
metadata_path = model_path.with_suffix(".json")
|
||||
vec_env.close()
|
||||
# 收尾阶段恢复默认处理器:再次 Ctrl+C / 停止可立即强杀,不再等待评估。
|
||||
@@ -637,8 +904,7 @@ def run_training(args) -> int:
|
||||
if args.eval_manifest:
|
||||
try:
|
||||
manifest_eval = _evaluate_manifest(
|
||||
model_path, args.eval_manifest, limit=args.eval_manifest_limit,
|
||||
problem=args.problem)
|
||||
model_path, args.eval_manifest, limit=args.eval_manifest_limit)
|
||||
except Exception as exc:
|
||||
manifest_eval = {"error": f"{type(exc).__name__}: {exc}"}
|
||||
metadata["evaluation"] = evaluation
|
||||
@@ -685,10 +951,17 @@ def main(argv=None) -> int:
|
||||
help="time=P0.4 纯时间口径;prompt=原始 §5.2 塑形对照")
|
||||
parser.add_argument("--progress-json", action="store_true",
|
||||
help="向训练前端输出单行 JSON 进度(UI 自动启用)")
|
||||
parser.add_argument("--control-file", default=None,
|
||||
help="训练前端控制文件(JSON:paused/stop);"
|
||||
"跨平台暂停与优雅停止的主通道")
|
||||
parser.add_argument("--progress-every", type=int, default=0,
|
||||
help="JSON 进度间隔环境步数;0=每个 rollout")
|
||||
parser.add_argument("--checkpoint-every", type=int, default=20000,
|
||||
help="每 N 个环境步落一个 checkpoint;0=关闭(默认 20000)")
|
||||
help="每 N 个环境步留一个历史 checkpoint;0=关闭(默认 20000)")
|
||||
parser.add_argument("--recovery-every", type=int, default=2000,
|
||||
help="每 N 个环境步原子覆盖写一份滚动恢复点 "
|
||||
"<模型目录>/checkpoints/<stem>_latest.zip;"
|
||||
"急停后最多丢 N 步。0=关闭(默认 2000)")
|
||||
parser.add_argument("--checkpoint-keep", type=int, default=3,
|
||||
help="checkpoint 保留最近 K 个(默认 3)")
|
||||
parser.add_argument("--checkpoint-dir", default=None,
|
||||
@@ -703,6 +976,8 @@ def main(argv=None) -> int:
|
||||
parser.error("eval-episodes 不能为负")
|
||||
if args.checkpoint_every < 0:
|
||||
parser.error("checkpoint-every 不能为负")
|
||||
if args.recovery_every < 0:
|
||||
parser.error("recovery-every 不能为负")
|
||||
if args.checkpoint_keep < 1:
|
||||
parser.error("checkpoint-keep 至少为 1")
|
||||
return run_training(args)
|
||||
|
||||
@@ -82,7 +82,6 @@ def bc_validation_mask(episode_ids, *, seed: int):
|
||||
|
||||
def train_bc(x, y, episode_ids, *, epochs: int, batch_size: int, lr: float,
|
||||
device: str, seed: int):
|
||||
import numpy as np
|
||||
import torch
|
||||
BCMlp = _bc_module(LOW_OBS_DIM, LOW_NUM_ACTIONS, HIDDEN)
|
||||
model = BCMlp().to(device)
|
||||
|
||||
+2
-1
@@ -130,7 +130,8 @@ def main(argv=None) -> int:
|
||||
for rank in range(max(1, args.envs))])
|
||||
if args.init:
|
||||
metadata = json.loads(Path(args.init).with_suffix('.json').read_text(encoding='utf-8'))
|
||||
if metadata.get('algorithm', 'ppo') != args.algorithm or metadata.get('feature_version') != 2:
|
||||
if (metadata.get('algorithm', 'ppo') != args.algorithm
|
||||
or metadata.get('feature_version') != 2):
|
||||
parser.error('初始化模型的算法/特征版本与当前训练不符,请重训或提供v2初始化模型')
|
||||
model = model_cls.load(args.init, env=vec_env, device=device)
|
||||
else:
|
||||
|
||||
@@ -44,7 +44,7 @@ def evaluate(bayes_params: dict) -> tuple[float, dict]:
|
||||
config = replace(BASE, bayes=replace(BASE.bayes, **bayes_params))
|
||||
times = []
|
||||
failures = 0
|
||||
for name, sources, error_seed in SCENES:
|
||||
for _name, sources, error_seed in SCENES:
|
||||
sim = LocalSimulator(sources, robot_id="TUNE", config=config,
|
||||
error_seed=error_seed)
|
||||
client = SimulatorClient("TUNE", config=config,
|
||||
|
||||
@@ -33,7 +33,10 @@ class StrictPlayback(PlaybackTransport):
|
||||
raise AssertionError("Replay attempted an extra action")
|
||||
expected = self.events[self.index]
|
||||
actual_request = json.loads(body)
|
||||
without_id = lambda value: {k: v for k, v in value.items() if k != "request_id"}
|
||||
|
||||
def without_id(value):
|
||||
return {k: v for k, v in value.items() if k != "request_id"}
|
||||
|
||||
if (urlsplit(url).path != expected["path"] or
|
||||
without_id(actual_request) != without_id(expected["request"])):
|
||||
raise AssertionError(f"Replay action differs at index {self.index}")
|
||||
@@ -66,6 +69,47 @@ def save_json(path, value):
|
||||
allow_nan=False) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
def timing_rows(events):
|
||||
"""按官方附件 2 第 4.1 节逐动作累计虚拟时间(微秒),返回明细与汇总。"""
|
||||
position, channel, virtual_us = (0.0, 0.0), 1, 0
|
||||
distance_total, switch_total, move_total_us = 0.0, 0, 0
|
||||
rows = []
|
||||
previous_recorded = 0.0
|
||||
max_local_gap = 0.0
|
||||
for i, event in enumerate(events):
|
||||
request, response = event["request"], event["response"]
|
||||
path = event["path"]
|
||||
distance, switches, action_s = 0.0, 0, 0
|
||||
if path in ("/measure", "/clear"):
|
||||
target = (request["position"]["x"], request["position"]["y"])
|
||||
distance = math.dist(position, target)
|
||||
position = target
|
||||
if path == "/measure":
|
||||
switches = int(channel != request["channel"])
|
||||
channel = request["channel"]
|
||||
action_s = 5
|
||||
else:
|
||||
action_s = 5 if response["clear_result"] == "success" else 3
|
||||
move_us = round(distance / 5 * 1_000_000)
|
||||
virtual_us += move_us + (switches + action_s) * 1_000_000
|
||||
distance_total += distance
|
||||
switch_total += switches
|
||||
move_total_us += move_us
|
||||
recorded = response["virtual_time_s"]
|
||||
gap_us = round(recorded * 1_000_000) - virtual_us
|
||||
max_local_gap = max(max_local_gap,
|
||||
abs(recorded - previous_recorded - distance/5 - switches - action_s))
|
||||
previous_recorded = recorded
|
||||
rows.append({"index": i, "path": path, "request_id": request["request_id"],
|
||||
"channel": request.get("channel", ""), "x": position[0], "y": position[1],
|
||||
"result": response.get("measure_result", response.get("clear_result", "")),
|
||||
"svd_deg": response.get("svd_deg", ""), "distance_m": distance,
|
||||
"move_us": move_us, "switch_s": switches, "action_s": action_s,
|
||||
"recomputed_us": virtual_us, "recorded_s": recorded, "gap_us": gap_us})
|
||||
return rows, virtual_us, distance_total, move_total_us, switch_total, previous_recorded, \
|
||||
max_local_gap
|
||||
|
||||
|
||||
def audit(folder: Path) -> dict:
|
||||
records = [json.loads(line) for line in
|
||||
(folder / "session.jsonl").read_text(encoding="utf-8").splitlines()]
|
||||
@@ -130,40 +174,8 @@ def audit(folder: Path) -> dict:
|
||||
|
||||
# Official attachment 2 section 4.1: accumulate virtual time in microseconds.
|
||||
# Round each move to the nearest microsecond, then add integer action costs.
|
||||
position, channel, virtual_us = (0.0, 0.0), 1, 0
|
||||
distance_total, switch_total, move_total_us = 0.0, 0, 0
|
||||
rows = []
|
||||
previous_recorded = 0.0
|
||||
max_local_gap = 0.0
|
||||
for i, event in enumerate(events):
|
||||
request, response, path = event["request"], event["response"], event["path"]
|
||||
distance, switches, action_s = 0.0, 0, 0
|
||||
if path in ("/measure", "/clear"):
|
||||
target = (request["position"]["x"], request["position"]["y"])
|
||||
distance = math.dist(position, target)
|
||||
position = target
|
||||
if path == "/measure":
|
||||
switches = int(channel != request["channel"])
|
||||
channel = request["channel"]
|
||||
action_s = 5
|
||||
else:
|
||||
action_s = 5 if response["clear_result"] == "success" else 3
|
||||
move_us = round(distance / 5 * 1_000_000)
|
||||
virtual_us += move_us + (switches + action_s) * 1_000_000
|
||||
distance_total += distance
|
||||
switch_total += switches
|
||||
move_total_us += move_us
|
||||
recorded = response["virtual_time_s"]
|
||||
gap_us = round(recorded * 1_000_000) - virtual_us
|
||||
max_local_gap = max(max_local_gap,
|
||||
abs(recorded - previous_recorded - distance/5 - switches - action_s))
|
||||
previous_recorded = recorded
|
||||
rows.append({"index": i, "path": path, "request_id": request["request_id"],
|
||||
"channel": request.get("channel", ""), "x": position[0], "y": position[1],
|
||||
"result": response.get("measure_result", response.get("clear_result", "")),
|
||||
"svd_deg": response.get("svd_deg", ""), "distance_m": distance,
|
||||
"move_us": move_us, "switch_s": switches, "action_s": action_s,
|
||||
"recomputed_us": virtual_us, "recorded_s": recorded, "gap_us": gap_us})
|
||||
rows, virtual_us, distance_total, move_total_us, switch_total, previous_recorded, \
|
||||
max_local_gap = timing_rows(events)
|
||||
checks["all_timestamps_match_microsecond_recomputation"] = all(r["gap_us"] == 0 for r in rows)
|
||||
real_s = (events[-1]["response"]["real_timestamp_ms"] -
|
||||
events[0]["response"]["real_timestamp_ms"]) / 1000
|
||||
@@ -206,7 +218,8 @@ def audit(folder: Path) -> dict:
|
||||
"clear_time_s": sum(r["action_s"] for r in rows if r["path"] == "/clear"),
|
||||
"timing_audit": {
|
||||
"original_unquantized_mismatch_count": len(raw_timing["mismatches"]),
|
||||
"original_unquantized_final_gap_s": previous_recorded - raw_timing["final_virtual_time_s"],
|
||||
"original_unquantized_final_gap_s":
|
||||
previous_recorded - raw_timing["final_virtual_time_s"],
|
||||
"max_single_action_unquantized_gap_s": max_local_gap,
|
||||
"microsecond_mismatch_count": sum(r["gap_us"] != 0 for r in rows),
|
||||
"explanation": "原复算累计未量化浮点耗时;按动作移动耗时取最近整数微秒后,所有官方时刻完全一致。",
|
||||
|
||||
@@ -116,8 +116,8 @@ if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("folder", type=Path)
|
||||
args = parser.parse_args()
|
||||
report = audit(args.folder)
|
||||
print(json.dumps({"passed": report["passed"], "case_code": report["case_code"],
|
||||
"problem": report["problem"], "checks": report["checks"]},
|
||||
result = audit(args.folder)
|
||||
print(json.dumps({"passed": result["passed"], "case_code": result["case_code"],
|
||||
"problem": result["problem"], "checks": result["checks"]},
|
||||
ensure_ascii=False, indent=2))
|
||||
raise SystemExit(0 if report["passed"] else 1)
|
||||
raise SystemExit(0 if result["passed"] else 1)
|
||||
|
||||
@@ -126,10 +126,10 @@ class CandidateRegion:
|
||||
@classmethod
|
||||
def initial(cls, station: Vec2, config: ProblemConfig) -> "CandidateRegion":
|
||||
"""首个 direction 后的基础集:Ω ∩ B(station, r_max) 的保守外包。"""
|
||||
field, sides = config.field, config.search.polygon_sides
|
||||
slack = field.omega_radius_m * (1 - math.cos(math.pi / sides))
|
||||
region = cls.disk((0.0, 0.0), field.omega_radius_m, sides, slack)
|
||||
region.clip_disk(station, field.r_eff_max_m, sides)
|
||||
field_cfg, sides = config.field, config.search.polygon_sides
|
||||
slack = field_cfg.omega_radius_m * (1 - math.cos(math.pi / sides))
|
||||
region = cls.disk((0.0, 0.0), field_cfg.omega_radius_m, sides, slack)
|
||||
region.clip_disk(station, field_cfg.r_eff_max_m, sides)
|
||||
return region
|
||||
|
||||
# ------------------------------------------------------------ 收缩
|
||||
@@ -198,8 +198,7 @@ class CandidateRegion:
|
||||
if len(self.polygon) == 2:
|
||||
p, q = self.polygon
|
||||
return _point_segment_distance(point, p, q) <= tol_m + self.slack_m
|
||||
for index in range(len(self.polygon)):
|
||||
p = self.polygon[index]
|
||||
for index, p in enumerate(self.polygon):
|
||||
q = self.polygon[(index + 1) % len(self.polygon)]
|
||||
if (q[0] - p[0]) * (point[1] - p[1]) - (q[1] - p[1]) * (point[0] - p[0]) \
|
||||
< -tol_m - self.slack_m:
|
||||
|
||||
@@ -294,6 +294,13 @@ def _run(args) -> int:
|
||||
if judge is not None:
|
||||
_settle_local_judge(judge)
|
||||
summary["robot_id"] = args.robot_id
|
||||
return _finalize_run(args, summary, strategy, truth=truth, judge=judge,
|
||||
judge_session=judge_session, real_budget_s=real_budget_s)
|
||||
|
||||
|
||||
def _finalize_run(args, summary, strategy, *, truth, judge, judge_session,
|
||||
real_budget_s) -> int:
|
||||
"""补齐 summary 元信息、核对真值并输出;返回进程退出码。"""
|
||||
if args.problem == 4:
|
||||
summary["strategy"] = "rl_q4" if strategy == "rl" else strategy
|
||||
else:
|
||||
|
||||
@@ -32,7 +32,7 @@ class CoverageLedger:
|
||||
"""逐频道覆盖账本:(频道, 网格点) -> 是否已测。"""
|
||||
|
||||
config: ProblemConfig = field(default_factory=ProblemConfig)
|
||||
points: list[Vec2] = field(default_factory=lambda: coverage_points())
|
||||
points: list[Vec2] = field(default_factory=coverage_points)
|
||||
measured: set = field(default_factory=set) # {(channel, point_index)}
|
||||
skipped: dict = field(default_factory=dict) # channel -> 原因
|
||||
|
||||
|
||||
@@ -23,8 +23,10 @@ def main(argv=None):
|
||||
measurements = [replace(m, half_angle_deg=m.half_angle_deg+args.rounding_margin_deg)
|
||||
for m in measurements]
|
||||
result = solve(measurements)
|
||||
result["error_policy"] = {"additional_half_angle_deg": args.rounding_margin_deg,
|
||||
"effective_half_angles_deg": [m.half_angle_deg for m in measurements]}
|
||||
result["error_policy"] = {
|
||||
"additional_half_angle_deg": args.rounding_margin_deg,
|
||||
"effective_half_angles_deg": [m.half_angle_deg for m in measurements],
|
||||
}
|
||||
except (ValueError, TypeError, KeyError, OSError) as exc:
|
||||
parser.error(str(exc))
|
||||
text = json.dumps(result, ensure_ascii=False, indent=2, allow_nan=False)
|
||||
|
||||
@@ -133,6 +133,13 @@ def _enclosing_circle_reference(points: list[Point], eps: float = EPS):
|
||||
return best_c, best_r
|
||||
|
||||
|
||||
def _circle_covers(circle, point: Point, tolerance: float) -> bool:
|
||||
"""Whether the circle already covers point; None means no circle yet."""
|
||||
if circle is None:
|
||||
return False
|
||||
return distance(circle[0], point) <= circle[1] + tolerance
|
||||
|
||||
|
||||
def smallest_enclosing_circle(points: list[Point], eps: float = EPS):
|
||||
"""Welzl's randomized incremental algorithm, using an explicit call stack.
|
||||
|
||||
@@ -169,8 +176,9 @@ def smallest_enclosing_circle(points: list[Point], eps: float = EPS):
|
||||
elif not returning:
|
||||
stack.append((n, boundary, True))
|
||||
stack.append((n-1, boundary, False))
|
||||
elif circle is None or distance(circle[0], local[n-1]) > circle[1] + tolerance:
|
||||
elif not _circle_covers(circle, local[n-1], tolerance):
|
||||
stack.append((n-1, boundary + (local[n-1],), False))
|
||||
assert circle is not None
|
||||
center = (origin[0]+circle[0][0]*scale, origin[1]+circle[0][1]*scale)
|
||||
# Recompute after transforming back: returned circle contains all input points
|
||||
# to distance() precision, including any roundoff from the large translation.
|
||||
|
||||
@@ -29,7 +29,7 @@ class LocalSimTransport(Transport):
|
||||
def post(self, url, body, headers, timeout):
|
||||
return self.request("POST", url, body, headers, timeout)
|
||||
|
||||
def request(self, method, url, body, headers, timeout=None):
|
||||
def request(self, method, url, body, headers, _timeout=None):
|
||||
"""验证原始请求边界;内存回环不模拟 TCP、UI 或连接限流。"""
|
||||
target = urlsplit(url)
|
||||
raw_target = target.path + (f"?{target.query}" if target.query else "")
|
||||
@@ -57,5 +57,6 @@ class ScriptedFaultTransport(Transport):
|
||||
self.inner.post(url, body, headers, timeout) # 服务器已受理
|
||||
raise TransportError("injected: read timeout after server applied")
|
||||
if isinstance(fault, tuple) and fault and fault[0] == "status":
|
||||
return TransportResponse(int(fault[1]), b'{"accepted": false, "real_timestamp_ms": 0, "virtual_time_s": 0}')
|
||||
body = b'{"accepted": false, "real_timestamp_ms": 0, "virtual_time_s": 0}'
|
||||
return TransportResponse(int(fault[1]), body)
|
||||
return self.inner.post(url, body, headers, timeout)
|
||||
|
||||
@@ -17,7 +17,7 @@ import math
|
||||
import numpy as np
|
||||
|
||||
from ..config import Vec2
|
||||
from .constants import (LOW_MAX_STEPS, LOW_NUM_ACTIONS, LOW_OBS_DIM,
|
||||
from .constants import (LOW_MAX_STEPS, LOW_NUM_ACTIONS,
|
||||
LOW_REACH_M, LOW_STEP_M, POS_SCALE_M)
|
||||
|
||||
__all__ = ["build_low_observation", "LowLevelExecutor", "action_bearing",
|
||||
|
||||
@@ -144,7 +144,6 @@ class HierarchicalEnv(gym.Env):
|
||||
problem=self.problem)
|
||||
|
||||
def _training_spec(self):
|
||||
from ..harness import referee_spec
|
||||
from ..hrl.curriculum import training_spec
|
||||
return training_spec(self.train_config, self.stage,
|
||||
max_real_duration_s=self.real_budget_s)
|
||||
|
||||
@@ -19,7 +19,7 @@ from gymnasium import spaces
|
||||
|
||||
from ..config import ProblemConfig
|
||||
from .constants import (LOW_MAX_STEPS, LOW_NUM_ACTIONS, LOW_OBS_DIM,
|
||||
LOW_REACH_M, LOW_STEP_M, POS_SCALE_M)
|
||||
LOW_REACH_M, LOW_STEP_M)
|
||||
from .executor import (action_bearing, build_low_observation,
|
||||
navigation_transition)
|
||||
|
||||
|
||||
@@ -48,7 +48,7 @@ def _load_maskable(model_path: Path, device: str, *,
|
||||
if meta_path.exists() else {}
|
||||
# 高层模型必须携带完整语义元数据;缺字段不能默认兼容,防止旧槽位/
|
||||
# 旧奖励语义模型被静默加载。低层导航模型只校验基础特征版本。
|
||||
strict = (act_dim == N_CANDIDATES)
|
||||
strict = act_dim == N_CANDIDATES
|
||||
if strict:
|
||||
required = ("feature_version", "action_layout_version",
|
||||
"reward_version", "algorithm", "problem",
|
||||
|
||||
@@ -29,7 +29,7 @@ from typing import NamedTuple
|
||||
import numpy as np
|
||||
|
||||
from ..belief import ChannelStatus
|
||||
from ..config import ProblemConfig, Vec2
|
||||
from ..config import Vec2
|
||||
from ..protocol.rules import CLEAR_SUCCESS, DIRECTION, NEAR, NO_SIGNAL
|
||||
from ..protocol.timing import move_time
|
||||
from .constants import (CHANNEL_COUNT, CLEAR_OFFSET, COST_SCALE_S,
|
||||
@@ -84,6 +84,8 @@ class HRLControllerMixin:
|
||||
"""
|
||||
|
||||
problem: int
|
||||
# 由宿主 BayesLoop.__init__ 初始化;此处声明让 pylint 可见该诊断计数。
|
||||
no_shrink_measures: int = 0
|
||||
|
||||
# ------------------------------------------------------------ 候选构建
|
||||
def build_candidates(self) -> list[Candidate]:
|
||||
@@ -220,7 +222,7 @@ class HRLControllerMixin:
|
||||
def hrl_observation(self, candidates: list[Candidate]) -> np.ndarray:
|
||||
"""构造高层观测(shape=(HIGH_OBS_DIM,),训练/推理逐位一致)。"""
|
||||
config = self.config
|
||||
field = config.field
|
||||
field_cfg = config.field
|
||||
position = self.client.state.position
|
||||
belief = self.belief
|
||||
|
||||
@@ -232,24 +234,24 @@ class HRLControllerMixin:
|
||||
glob = [
|
||||
position[0] / POS_SCALE_M,
|
||||
position[1] / POS_SCALE_M,
|
||||
min(self.client.state.virtual_time_s, field.max_virtual_duration_s)
|
||||
/ field.max_virtual_duration_s,
|
||||
min(self.client.state.virtual_time_s, field_cfg.max_virtual_duration_s)
|
||||
/ field_cfg.max_virtual_duration_s,
|
||||
cleared / 16.0,
|
||||
accounted / CHANNEL_COUNT,
|
||||
(self.client.state.channel - field.channel_min)
|
||||
/ max(field.channel_max - field.channel_min, 1),
|
||||
(self.client.state.channel - field_cfg.channel_min)
|
||||
/ max(field_cfg.channel_max - field_cfg.channel_min, 1),
|
||||
float(self.problem == 4),
|
||||
detected_count / 16.0,
|
||||
]
|
||||
assert len(glob) == HIGH_GLOBAL_DIM
|
||||
|
||||
pending_station = np.zeros(CHANNEL_COUNT + 1, dtype=bool)
|
||||
for index, channels in getattr(self, "_station_pending_map", {}).items():
|
||||
for channels in getattr(self, "_station_pending_map", {}).values():
|
||||
for channel in channels:
|
||||
pending_station[channel] = True
|
||||
|
||||
channel_features: list[float] = []
|
||||
for channel in range(field.channel_min, field.channel_max + 1):
|
||||
for channel in range(field_cfg.channel_min, field_cfg.channel_max + 1):
|
||||
ch = belief.channels[channel]
|
||||
status6 = [0.0] * 6
|
||||
status6[self._hrl_channel_status_index(channel)] = 1.0
|
||||
@@ -363,7 +365,7 @@ class HRLControllerMixin:
|
||||
outcome = StepOutcome(action=action, kind=cand.kind,
|
||||
channel=cand.channel,
|
||||
position_reached=tuple(self.client.state.position))
|
||||
outcome.delta_virtual_s = (self.client.state.virtual_time_s - old_virtual)
|
||||
outcome.delta_virtual_s = self.client.state.virtual_time_s - old_virtual
|
||||
new_detections: list[int] = []
|
||||
info_gains: dict[int, float] = {}
|
||||
redundant = False
|
||||
|
||||
@@ -237,10 +237,11 @@ class ChannelParticles:
|
||||
base = i * self.width
|
||||
for j in range(j0, min(j0 + _COARSE_FACTOR, self.width)):
|
||||
index = base + j
|
||||
if index in self.alive:
|
||||
dist = math.dist(self.cell_center(index), point)
|
||||
if dist <= limit_m and (best is None or dist < best):
|
||||
best = dist
|
||||
if index not in self.alive:
|
||||
continue
|
||||
dist = math.dist(self.cell_center(index), point)
|
||||
if dist <= limit_m and (best is None or dist < best):
|
||||
best = dist
|
||||
return best
|
||||
|
||||
def guaranteed_no_signal(self, station: Vec2) -> bool:
|
||||
|
||||
@@ -621,7 +621,8 @@ class DirectionalExtension(OmnidirectionalLoop):
|
||||
self.ledger.mark_measured(channel, index)
|
||||
self.belief.update_measure(channel, point, outcome.result, outcome.svd_deg)
|
||||
if outcome.result == NEAR:
|
||||
cert = clear_certificate(self.belief.candidate_set(channel), channel, self.config)
|
||||
cert = clear_certificate(
|
||||
self.belief.candidate_set(channel), channel, self.config)
|
||||
if cert.certified:
|
||||
self._attempt_clear(channel, point, cert)
|
||||
if self._count_limit_reached():
|
||||
@@ -629,7 +630,8 @@ class DirectionalExtension(OmnidirectionalLoop):
|
||||
queue = self.belief.channels_by_status(ChannelStatus.DETECTED)
|
||||
while queue and self.stop_reason is None:
|
||||
channel = min(queue, key=lambda c: (
|
||||
math.dist(self.client.state.position, self.belief.candidate_set(c).mec()[0]), c))
|
||||
math.dist(self.client.state.position,
|
||||
self.belief.candidate_set(c).mec()[0]), c))
|
||||
queue.remove(channel)
|
||||
self._localize_and_clear(channel)
|
||||
if self._count_limit_reached():
|
||||
@@ -672,8 +674,7 @@ class DirectionalExtension(OmnidirectionalLoop):
|
||||
self.directional_observed.append(channel)
|
||||
self.blocked_probes.setdefault(channel, []).append(point)
|
||||
continue # 盘外无信号也可能是距离不足,不据此断言定向
|
||||
else:
|
||||
self.blocked_probes.pop(channel, None)
|
||||
self.blocked_probes.pop(channel, None)
|
||||
steps += 1
|
||||
if self.belief.channels[channel].status is ChannelStatus.CLEARED:
|
||||
return True
|
||||
|
||||
@@ -109,13 +109,15 @@ class BayesLoop(OmnidirectionalLoop):
|
||||
if cert is not None and cert.certified:
|
||||
continue
|
||||
region = self.belief.candidate_set(channel)
|
||||
if max(math.dist(point, v) for v in region.polygon) > self.config.field.r_eff_min_m - 1e-3:
|
||||
farthest = max(math.dist(point, v) for v in region.polygon)
|
||||
if farthest > self.config.field.r_eff_min_m - 1e-3:
|
||||
continue
|
||||
radius = region.mec()[1]
|
||||
vertices = region.polygon
|
||||
stride = max(1, len(vertices) // self.config.bayes.mc_representatives)
|
||||
radii = []
|
||||
delta = self.config.field.bearing_error_half_deg + self.config.search.bearing_rounding_margin_deg
|
||||
delta = (self.config.field.bearing_error_half_deg
|
||||
+ self.config.search.bearing_rounding_margin_deg)
|
||||
for target in vertices[::stride]:
|
||||
theta = math.degrees(math.atan2(target[1]-point[1], target[0]-point[0]))
|
||||
probe = CandidateRegion(list(vertices), region.slack_m)
|
||||
@@ -146,7 +148,7 @@ class BayesLoop(OmnidirectionalLoop):
|
||||
def _skip_station_channel(self, channel: int, index: int, point: Vec2) -> bool:
|
||||
"""跳测规则(按 pf.version 缓存;幸存集单调收缩,判定单调)。"""
|
||||
pf = self.particles[channel]
|
||||
if not len(pf):
|
||||
if not pf:
|
||||
return True
|
||||
cached = self._skip_cache.get((channel, index))
|
||||
if cached is not None and cached[0] == pf.version:
|
||||
@@ -219,7 +221,9 @@ class BayesLoop(OmnidirectionalLoop):
|
||||
total += self.config.cost.switch_s * (channel != task.channel)
|
||||
channel = task.channel
|
||||
elif task.kind == "station" and task.station_channels:
|
||||
pending = sorted(task.station_channels, key=lambda c: (c != channel, c))
|
||||
pending = sorted(
|
||||
task.station_channels,
|
||||
key=lambda c, channel=channel: (c != channel, c))
|
||||
total += self.config.cost.measure_s * len(pending)
|
||||
total += self.config.cost.switch_s * (len(pending) - (channel in pending))
|
||||
channel = pending[-1]
|
||||
@@ -328,7 +332,7 @@ class BayesLoop(OmnidirectionalLoop):
|
||||
return # 已证毕:清除/缺席后的 no_signal 与距离无关,不进 PF
|
||||
pf = self.particles[channel]
|
||||
pf.apply_measure(position, outcome.result, outcome.svd_deg)
|
||||
if not len(pf):
|
||||
if not pf:
|
||||
if belief.status is ChannelStatus.UNKNOWN:
|
||||
self.belief.mark_absent(channel)
|
||||
if channel not in self.pf_absent_channels:
|
||||
@@ -350,7 +354,8 @@ class BayesLoop(OmnidirectionalLoop):
|
||||
limit = (self.config.field.clear_radius_m - pf.margin_m
|
||||
- self.config.search.clear_margin_m)
|
||||
if radius <= limit:
|
||||
return self._near_clear_target(ClearCertificate(channel, center, radius, limit, True))
|
||||
return self._near_clear_target(
|
||||
ClearCertificate(channel, center, radius, limit, True))
|
||||
return cert
|
||||
|
||||
def _near_clear_target(self, cert: ClearCertificate) -> ClearCertificate:
|
||||
@@ -368,7 +373,7 @@ class BayesLoop(OmnidirectionalLoop):
|
||||
"""定位测量点:期望 log 幸存占比 + λ·移动时间 最小的候选。"""
|
||||
bayes, search = self.config.bayes, self.config.search
|
||||
pf = self.particles[channel]
|
||||
if not len(pf):
|
||||
if not pf:
|
||||
return None
|
||||
region = self.belief.candidate_set(channel)
|
||||
center, radius = pf.mec()
|
||||
|
||||
@@ -16,8 +16,6 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from ..belief import ChannelStatus
|
||||
from ..config import ProblemConfig, Vec2
|
||||
from ..coverage import CoverageLedger, coverage_points
|
||||
@@ -116,7 +114,7 @@ class HRLLoop(HRLControllerMixin, BayesLoop):
|
||||
if belief.status not in (ChannelStatus.CLEARED, ChannelStatus.ABSENT):
|
||||
if not (self.problem == 4 and outcome.result == NO_SIGNAL):
|
||||
pf.apply_measure(position, outcome.result, outcome.svd_deg)
|
||||
if not len(pf):
|
||||
if not pf:
|
||||
if belief.status is ChannelStatus.UNKNOWN:
|
||||
self.belief.mark_absent(channel)
|
||||
if channel not in self.pf_absent_channels:
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
"""跨平台子进程控制:启动隔离、协作式暂停/停止、必要时强杀整棵进程树。
|
||||
|
||||
训练前端要在 POSIX 与 Windows 上表现一致,而两边的进程模型差别正好卡在
|
||||
「停止训练」这条路径上:
|
||||
|
||||
- ``os.killpg`` / ``os.getpgid`` **只存在于 POSIX**;Windows 上访问它们会抛
|
||||
``AttributeError``。旧实现只捕获 ``ProcessLookupError`` / ``PermissionError``,
|
||||
于是 ``POST /api/train/stop`` 在 Windows 上直接 500,训练进程照跑——表现就是
|
||||
「点停止没反应」。
|
||||
- ``subprocess.Popen(start_new_session=True)`` 在 Windows 上被**静默忽略**:
|
||||
CPython 的 Windows ``_execute_child`` 形参名就是 ``unused_start_new_session``,
|
||||
要隔离进程组只能传 ``CREATE_NEW_PROCESS_GROUP``。
|
||||
- POSIX 的 SIGTERM 在 Windows 上经 ``os.kill`` 会退化成 ``TerminateProcess``:
|
||||
硬杀,训练脚本里"收到信号后保存模型"的处理器根本不会执行。
|
||||
|
||||
所以「优雅暂停 / 优雅停止」不依赖信号量语义,而是走一个**控制文件**:前端原子
|
||||
写 JSON,训练脚本每步轮询。两端都只是普通文件 IO,Windows 与 POSIX 行为一致;
|
||||
信号与强杀退居兜底,只在训练进程不响应控制文件时使用。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import signal
|
||||
import subprocess
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
__all__ = [
|
||||
"IS_WINDOWS", "ControlState", "spawn_kwargs", "write_control",
|
||||
"read_control", "write_json_atomic", "terminate_process_tree",
|
||||
]
|
||||
|
||||
# 运行期常量而非"导入期分支":调用方(含测试)可以 monkeypatch 它来模拟 Windows
|
||||
IS_WINDOWS = os.name == "nt"
|
||||
|
||||
# 只有 Windows 定义 CREATE_NEW_PROCESS_GROUP;POSIX 上取 0 表示不使用 creationflags
|
||||
_CREATE_NEW_PROCESS_GROUP = getattr(subprocess, "CREATE_NEW_PROCESS_GROUP", 0)
|
||||
|
||||
# 发出「停止」到强杀之间留给训练脚本收尾(保存模型 + 写元数据)的秒数
|
||||
DEFAULT_GRACE_S = 15.0
|
||||
|
||||
|
||||
def spawn_kwargs() -> dict[str, Any]:
|
||||
"""返回 ``Popen`` 的平台相关参数:让训练进程独立于前端进程组。
|
||||
|
||||
POSIX 用 ``setsid``(``start_new_session``);Windows 上用
|
||||
``CREATE_NEW_PROCESS_GROUP``,否则前端所在控制台的 Ctrl+C 会连带打到训练进程。
|
||||
"""
|
||||
if IS_WINDOWS:
|
||||
if _CREATE_NEW_PROCESS_GROUP:
|
||||
return {"creationflags": _CREATE_NEW_PROCESS_GROUP}
|
||||
return {}
|
||||
return {"start_new_session": True}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ControlState:
|
||||
"""控制文件的一次快照;缺失或损坏时字段全为默认(即"无指令")。"""
|
||||
|
||||
paused: bool = False
|
||||
stop: bool = False
|
||||
seq: int = 0
|
||||
ts: float = 0.0
|
||||
|
||||
|
||||
def write_control(path: Path, *, paused: bool = False, stop: bool = False,
|
||||
seq: int | None = None) -> ControlState:
|
||||
"""原子写控制文件(同目录临时文件 + ``os.replace``)。
|
||||
|
||||
训练侧随时可能来读,必须避免读到写了一半的 JSON。``os.replace`` 在
|
||||
Windows 与 POSIX 上都是原子的(同目录、同卷)。
|
||||
"""
|
||||
path = Path(path)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
state = ControlState(
|
||||
paused=bool(paused),
|
||||
stop=bool(stop),
|
||||
seq=int(seq) if seq is not None else time.time_ns(),
|
||||
ts=time.time(),
|
||||
)
|
||||
payload = {"paused": state.paused, "stop": state.stop,
|
||||
"seq": state.seq, "ts": state.ts}
|
||||
tmp = path.with_name(f".{path.name}.{os.getpid()}.tmp")
|
||||
tmp.write_text(json.dumps(payload, ensure_ascii=False), encoding="utf-8")
|
||||
os.replace(tmp, path)
|
||||
return state
|
||||
|
||||
|
||||
def read_control(path: Path) -> ControlState:
|
||||
"""读控制文件;缺失 / 半截 / 损坏一律当作"无指令",绝不让训练崩在控制面上。"""
|
||||
try:
|
||||
raw = Path(path).read_text(encoding="utf-8")
|
||||
payload = json.loads(raw)
|
||||
except (OSError, ValueError, UnicodeDecodeError):
|
||||
return ControlState()
|
||||
if not isinstance(payload, dict):
|
||||
return ControlState()
|
||||
return ControlState(
|
||||
paused=bool(payload.get("paused", False)),
|
||||
stop=bool(payload.get("stop", False)),
|
||||
seq=int(payload.get("seq") or 0),
|
||||
ts=float(payload.get("ts") or 0.0),
|
||||
)
|
||||
|
||||
|
||||
def write_json_atomic(path: Path, payload: Any) -> None:
|
||||
"""原子写 JSON 文件(同目录临时文件 + ``os.replace``)。
|
||||
|
||||
用于"急停恢复"相关的标记文件(``resume.json`` / ``forced_stop.json`` 等):
|
||||
读取方随时可能来读,必须避免看到写了一半的内容。
|
||||
"""
|
||||
path = Path(path)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = path.with_name(f".{path.name}.{os.getpid()}.tmp")
|
||||
tmp.write_text(json.dumps(payload, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8")
|
||||
os.replace(tmp, path)
|
||||
|
||||
|
||||
def terminate_process_tree(process: subprocess.Popen | None, *,
|
||||
grace_s: float = DEFAULT_GRACE_S,
|
||||
poll_s: float = 0.2) -> bool:
|
||||
"""等待 ``grace_s`` 让协作式停止生效,超时则强杀整棵进程树。
|
||||
|
||||
返回**是否动用了强杀**:``False`` 表示进程在宽限期内自行退出(模型有机会
|
||||
落盘),``True`` 表示进程被强杀、最后一段产物可能缺失。调用方应先通过控制
|
||||
文件请求停止,再调用本函数;本函数不负责写控制文件。
|
||||
"""
|
||||
if process is None:
|
||||
return False
|
||||
if process.poll() is not None:
|
||||
return False
|
||||
deadline = time.monotonic() + max(0.0, grace_s)
|
||||
while time.monotonic() < deadline:
|
||||
if process.poll() is not None:
|
||||
return False
|
||||
time.sleep(min(poll_s, max(0.0, deadline - time.monotonic())))
|
||||
if process.poll() is not None:
|
||||
return False
|
||||
_hard_kill_tree(process)
|
||||
try:
|
||||
process.wait(timeout=10.0)
|
||||
except subprocess.TimeoutExpired:
|
||||
pass
|
||||
return True
|
||||
|
||||
|
||||
def _hard_kill_tree(process: subprocess.Popen) -> None:
|
||||
"""强杀进程树:POSIX 走进程组 SIGKILL,Windows 走 ``taskkill /T``。
|
||||
|
||||
这里所有 POSIX 专属调用都走 ``getattr`` 探测,Windows 上拿不到分支就退回
|
||||
``Popen.kill()``——不会再出现 ``module 'os' has no attribute 'killpg'``。
|
||||
"""
|
||||
if IS_WINDOWS:
|
||||
try:
|
||||
subprocess.run(["taskkill", "/F", "/T", "/PID", str(process.pid)],
|
||||
capture_output=True, check=False)
|
||||
return
|
||||
except OSError:
|
||||
process.kill()
|
||||
return
|
||||
killpg = getattr(os, "killpg", None)
|
||||
getpgid = getattr(os, "getpgid", None)
|
||||
sigkill = getattr(signal, "SIGKILL", None)
|
||||
if killpg is not None and getpgid is not None and sigkill is not None:
|
||||
try:
|
||||
killpg(getpgid(process.pid), sigkill)
|
||||
return
|
||||
except (ProcessLookupError, PermissionError, OSError):
|
||||
pass
|
||||
process.kill()
|
||||
@@ -13,6 +13,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import math
|
||||
import uuid
|
||||
from dataclasses import dataclass
|
||||
from time import monotonic, sleep
|
||||
@@ -91,7 +92,7 @@ def _validate_position(position: dict, limit: float) -> None:
|
||||
value = position[key]
|
||||
if isinstance(value, bool) or not isinstance(value, (int, float)):
|
||||
raise ValueError(f"position.{key} 必须是数值")
|
||||
if value != value or value in (float("inf"), float("-inf")):
|
||||
if not math.isfinite(value):
|
||||
raise ValueError(f"position.{key} 必须是有限数值")
|
||||
if abs(value) > limit:
|
||||
raise ValueError(f"position.{key} 绝对值不得超过 {limit} 米")
|
||||
@@ -234,7 +235,7 @@ class SimulatorClient:
|
||||
f"剩余现实时间 {self.real_remaining_s:.1f}s 不足以预留 {margin:.1f}s 退出余量")
|
||||
|
||||
# ---------------------------------------------------------------- 内部机制
|
||||
def _payload(self, path: str, *, position: Vec2 | None = None,
|
||||
def _payload(self, _path: str, *, position: Vec2 | None = None,
|
||||
channel: int | None = None) -> dict:
|
||||
payload = {"arena_id": self.session.arena_id,
|
||||
"robot_id": self.robot_id,
|
||||
|
||||
+14
-12
@@ -27,7 +27,7 @@ from ..config import ProblemConfig
|
||||
from ..planner import clear_certificate, fallback_clear_cells, safe_disk_radius
|
||||
from ..protocol.client import SimulatorClient
|
||||
from ..protocol.log import SessionLog
|
||||
from ..protocol.rules import CLEAR_SUCCESS, DIRECTION, NEAR, NO_SIGNAL
|
||||
from ..protocol.rules import CLEAR_SUCCESS, DIRECTION, NO_SIGNAL
|
||||
from .features import NUM_ACTIONS, OBS_DIM, action_mask, build_observation, candidate_actions
|
||||
|
||||
|
||||
@@ -65,9 +65,9 @@ class LocalizationEnv(gym.Env):
|
||||
from referee import LocalSimulator as _RefereeLocalSimulator
|
||||
from referee import Source
|
||||
from ..harness import LocalSimTransport, referee_spec
|
||||
self._Source = Source
|
||||
self._LocalSimulator = _RefereeLocalSimulator
|
||||
self._LocalSimTransport = LocalSimTransport
|
||||
self._source_cls = Source
|
||||
self._local_simulator_cls = _RefereeLocalSimulator
|
||||
self._local_sim_transport_cls = LocalSimTransport
|
||||
self._referee_spec = referee_spec
|
||||
|
||||
self._scene_rng = Random()
|
||||
@@ -106,23 +106,22 @@ class LocalizationEnv(gym.Env):
|
||||
# 首次检测点:距离 [300, min(1450, r_eff-20)] 内随机,定向源取朝向
|
||||
# ±89.9°(接近但不跨越90°边界,保证direction)。机器狗可到Ω外 [题面]。
|
||||
max_distance = max(310.0, min(1450.0, r_eff - 20.0))
|
||||
if max_distance < 310.0:
|
||||
max_distance = 310.0
|
||||
max_distance = max(max_distance, 310.0)
|
||||
probe_distance = rng.uniform(300.0, max_distance)
|
||||
probe_bearing = (heading + rng.uniform(-89.9, 89.9)
|
||||
if directional else rng.uniform(0.0, 360.0))
|
||||
probe_radians = math.radians(probe_bearing)
|
||||
probe = (x + probe_distance * math.cos(probe_radians),
|
||||
y + probe_distance * math.sin(probe_radians))
|
||||
source = self._Source(channel=channel, x=x, y=y, r_eff_m=r_eff,
|
||||
directional=directional, heading_deg=heading)
|
||||
source = self._source_cls(channel=channel, x=x, y=y, r_eff_m=r_eff,
|
||||
directional=directional, heading_deg=heading)
|
||||
return source, probe
|
||||
|
||||
def _make_episode(self) -> bool:
|
||||
"""重建模拟器并完成首次测向;返回 False 表示连续未得到 direction。"""
|
||||
for _ in range(20):
|
||||
self._source, probe = self._make_source_and_probe()
|
||||
self._simulator = self._LocalSimulator(
|
||||
self._simulator = self._local_simulator_cls(
|
||||
[self._source], robot_id="rl-train",
|
||||
spec=self._referee_spec(
|
||||
self.config, max_real_duration_s=self.real_budget_s),
|
||||
@@ -131,7 +130,7 @@ class LocalizationEnv(gym.Env):
|
||||
stress_mode=True)
|
||||
self._client = SimulatorClient(
|
||||
"rl-train", config=self.config,
|
||||
transport=self._LocalSimTransport(self._simulator),
|
||||
transport=self._local_sim_transport_cls(self._simulator),
|
||||
log=SessionLog())
|
||||
self._client.enter()
|
||||
outcome = self._client.measure(probe, self._source.channel)
|
||||
@@ -149,7 +148,9 @@ class LocalizationEnv(gym.Env):
|
||||
if self._scene_rng.random() < self.handoff_probability:
|
||||
angle = self._scene_rng.uniform(0, 2*math.pi)
|
||||
reach = self.config.field.omega_radius_m * math.sqrt(self._scene_rng.random())
|
||||
other = self.config.field.channel_min if self._channel != self.config.field.channel_min else self.config.field.channel_max
|
||||
other = (self.config.field.channel_min
|
||||
if self._channel != self.config.field.channel_min
|
||||
else self.config.field.channel_max)
|
||||
self._client.measure((reach*math.cos(angle), reach*math.sin(angle)), other)
|
||||
self._start_virtual_s = self._client.state.virtual_time_s
|
||||
return True
|
||||
@@ -325,5 +326,6 @@ class LocalizationEnv(gym.Env):
|
||||
return observation, float(reward), self._terminated, self._truncated, info
|
||||
|
||||
def close(self):
|
||||
if self._client is not None and self._client.state.entered and not self._client.state.exited:
|
||||
if (self._client is not None and self._client.state.entered
|
||||
and not self._client.state.exited):
|
||||
self._client.exit()
|
||||
|
||||
@@ -19,7 +19,7 @@ import math
|
||||
import numpy as np
|
||||
|
||||
from ..config import ProblemConfig, Vec2
|
||||
from ..planner import candidate_points, safe_disk_radius
|
||||
from ..planner import safe_disk_radius
|
||||
from ._constants import GLOBAL_DIM, LADDER_SLOTS, NUM_ACTIONS, OBS_DIM, PER_ACTION_DIM
|
||||
|
||||
__all__ = [
|
||||
@@ -100,7 +100,8 @@ def action_mask(region, position: Vec2, config: ProblemConfig, problem: int,
|
||||
Q3有距离保证盘时屏蔽盘外探点;Q4允许盘外探点,但绝不把距离保证
|
||||
当作朝向保证。训练与推理共用此函数,不读取源位置或朝向。
|
||||
"""
|
||||
key = lambda p: (round(p[0]*2), round(p[1]*2))
|
||||
def key(p):
|
||||
return (round(p[0]*2), round(p[1]*2))
|
||||
visited = {key(r['position']) for r in getattr(belief, 'measurements', [])}
|
||||
seen = set()
|
||||
center, _ = region.mec()
|
||||
|
||||
@@ -40,7 +40,8 @@ class PointSelectionPolicy:
|
||||
device = "cpu"
|
||||
self.device = device
|
||||
meta_path = path.with_suffix('.json')
|
||||
self.metadata = json.loads(meta_path.read_text(encoding='utf-8')) if meta_path.exists() else {}
|
||||
self.metadata = (json.loads(meta_path.read_text(encoding='utf-8'))
|
||||
if meta_path.exists() else {})
|
||||
self.feature_version = self.metadata.get('feature_version', 1)
|
||||
if self.feature_version not in (1, 2):
|
||||
raise RuntimeError('不支持的RL特征版本')
|
||||
@@ -78,7 +79,9 @@ class PointSelectionPolicy:
|
||||
region, position, channel, config, problem, blocked, belief,
|
||||
virtual_time_s=virtual_time_s,
|
||||
receiver_channel=receiver_channel if self.feature_version >= 2 else None)
|
||||
kwargs = {'action_masks': action_mask(region, position, config, problem, blocked, belief)} if self.masked else {}
|
||||
kwargs = ({'action_masks': action_mask(region, position, config, problem,
|
||||
blocked, belief)}
|
||||
if self.masked else {})
|
||||
action, _state = self.model.predict(observation, deterministic=True, **kwargs)
|
||||
action = int(action)
|
||||
candidates = candidate_actions(region, position, config, blocked)
|
||||
|
||||
@@ -0,0 +1,397 @@
|
||||
"""训练快照的纯函数派生:曲线统计、收敛判读、进程视图。
|
||||
|
||||
从 ``server.py`` 拆出来的两个理由:
|
||||
|
||||
1. 断开 ``demo`` ↔ ``server`` 的循环导入——离线预览(``demo.py``)只想复用
|
||||
这两个纯函数,没必要反向 import 整个 HTTP 服务器模块;
|
||||
2. 让"数值判读"可以脱离 HTTP 与子进程单独测试。
|
||||
|
||||
本模块只依赖标准库,不做任何 IO、不持有任何状态。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
import re
|
||||
import subprocess
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
__all__ = ["MAX_LOG_LINES", "MAX_METRICS", "finite_float",
|
||||
"summarize_convergence", "describe_process", "process_pid"]
|
||||
|
||||
MAX_LOG_LINES = 4000
|
||||
MAX_METRICS = 5000
|
||||
|
||||
_NUM_RE = re.compile(r"^[+-]?(?:\d+\.?\d*|\.\d+)(?:[eE][+-]?\d+)?$")
|
||||
|
||||
|
||||
def finite_float(value: Any) -> float | None:
|
||||
"""把任意输入转成有限浮点数;NaN / ±inf / 非数字一律返回 None。
|
||||
|
||||
旧实现用 ``value != value`` 判 NaN,pylint 报 ``comparison-with-itself``;
|
||||
这里用 ``math.isfinite`` 表达同一语义,意图更直白。
|
||||
"""
|
||||
if value is None or isinstance(value, bool):
|
||||
return None
|
||||
if isinstance(value, (int, float)):
|
||||
value = float(value)
|
||||
if not math.isfinite(value):
|
||||
return None
|
||||
return value
|
||||
text = str(value).strip()
|
||||
if not _NUM_RE.match(text):
|
||||
return None
|
||||
try:
|
||||
return float(text)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def _series(metrics: list[dict[str, Any]], key: str) -> list[tuple[float, float]]:
|
||||
points: list[tuple[float, float]] = []
|
||||
for point in metrics:
|
||||
value = finite_float(point.get(key))
|
||||
steps = finite_float(point.get("timesteps")) or finite_float(
|
||||
point.get("total_timesteps"))
|
||||
if value is None:
|
||||
continue
|
||||
points.append((steps if steps is not None else float(len(points) + 1), value))
|
||||
return points
|
||||
|
||||
|
||||
def _last_value(points: list[tuple[float, float]]) -> float | None:
|
||||
return points[-1][1] if points else None
|
||||
|
||||
|
||||
def _spread(points: list[tuple[float, float]], window: int = 10
|
||||
) -> tuple[float, float] | None:
|
||||
"""最近 window 个点的 (极差, 起点绝对值),用于判断"是否还在动"。"""
|
||||
sample = points[-window:]
|
||||
if len(sample) < 3:
|
||||
return None
|
||||
ys = [item[1] for item in sample]
|
||||
return max(ys) - min(ys), abs(ys[0])
|
||||
|
||||
|
||||
def _cv(points: list[tuple[float, float]], window: int = 10) -> float | None:
|
||||
"""最近 window 个点的变异系数(标准差/|均值|),越小越平稳。"""
|
||||
sample = [item[1] for item in points[-window:]]
|
||||
if len(sample) < 3:
|
||||
return None
|
||||
mean = sum(sample) / len(sample)
|
||||
if abs(mean) < 1e-12:
|
||||
return None
|
||||
var = sum((value - mean) ** 2 for value in sample) / len(sample)
|
||||
return (var ** 0.5) / abs(mean)
|
||||
|
||||
|
||||
def _slope(points: list[tuple[float, float]], window: int = 8) -> float | None:
|
||||
sample = points[-window:]
|
||||
if len(sample) < 3:
|
||||
return None
|
||||
xs = [item[0] for item in sample]
|
||||
ys = [item[1] for item in sample]
|
||||
n = len(xs)
|
||||
mean_x = sum(xs) / n
|
||||
mean_y = sum(ys) / n
|
||||
denom = sum((x - mean_x) ** 2 for x in xs)
|
||||
if abs(denom) < 1e-12:
|
||||
return None
|
||||
return sum((x - mean_x) * (y - mean_y) for x, y in zip(xs, ys)) / denom
|
||||
|
||||
|
||||
def _rate_per_1k(reward_points: list[tuple[float, float]],
|
||||
window: int = 8) -> float | None:
|
||||
"""最近 window 个点的奖励增益换算成"每 1000 环境步",便于读趋势。"""
|
||||
slope = _slope(reward_points, window)
|
||||
if slope is None:
|
||||
return None
|
||||
return slope * 1000.0
|
||||
|
||||
|
||||
def _fmt_num(value: float | None, digits: int = 4) -> str:
|
||||
if value is None:
|
||||
return "—"
|
||||
if value != 0 and abs(value) < 1e-3:
|
||||
return f"{value:.3e}"
|
||||
return f"{value:.{digits}g}"
|
||||
|
||||
|
||||
def _flatness(points: list[tuple[float, float]], window: int = 10
|
||||
) -> tuple[float, float] | None:
|
||||
"""窗口内"趋势位移 / 逐点噪声"比值:<1 表示看不出系统性变化。
|
||||
|
||||
判定收敛时只看斜率绝对值会把"很慢但持续的提升"误判为平台;这里把窗口
|
||||
对半切,用两半均值之差当趋势位移,用相邻点平均绝对差当噪声尺度。返回
|
||||
(比值, 噪声尺度),两者都小才是真的"又平又稳"。
|
||||
"""
|
||||
sample = points[-window:]
|
||||
if len(sample) < 4:
|
||||
return None
|
||||
ys = [item[1] for item in sample]
|
||||
half = len(ys) // 2
|
||||
shift = abs(sum(ys[half:]) / len(ys[half:]) - sum(ys[:half]) / len(ys[:half]))
|
||||
noise = sum(abs(b - a) for a, b in zip(ys, ys[1:])) / (len(ys) - 1)
|
||||
if noise <= 0:
|
||||
return (0.0, 0.0) if shift <= 0 else (float("inf"), 0.0)
|
||||
return shift / noise, noise
|
||||
|
||||
|
||||
def summarize_convergence(metrics: list[dict[str, Any]],
|
||||
monitor: list[dict[str, Any]]) -> dict[str, Any]:
|
||||
"""给前端返回可解释的收敛状态与各参数最近值。
|
||||
|
||||
这是**启发式判读**,不替代正式学习曲线分析:只依据最近窗口的斜率、
|
||||
极差与变异系数给出"平坦/上升/未上升"的分类,并把同一组数字交给前端
|
||||
展示,便于人工复核。
|
||||
"""
|
||||
reward_points = _series(metrics, "ep_rew_mean")
|
||||
if not reward_points and monitor:
|
||||
reward_points = [(float(index), float(row.get("r", 0.0)))
|
||||
for index, row in enumerate(monitor)
|
||||
if finite_float(row.get("r")) is not None]
|
||||
kl_points = _series(metrics, "approx_kl")
|
||||
ev_points = _series(metrics, "explained_variance")
|
||||
vf_points = _series(metrics, "value_loss")
|
||||
pg_points = _series(metrics, "policy_gradient_loss")
|
||||
entropy_points = _series(metrics, "entropy_loss")
|
||||
fps_points = _series(metrics, "fps")
|
||||
|
||||
reward_slope = _slope(reward_points)
|
||||
reward_rate = _rate_per_1k(reward_points)
|
||||
spread = _spread(reward_points)
|
||||
cv = _cv(reward_points)
|
||||
latest_kl = _last_value(kl_points)
|
||||
latest_ev = _last_value(ev_points)
|
||||
latest_vf = _last_value(vf_points)
|
||||
latest_pg = _last_value(pg_points)
|
||||
latest_entropy = _last_value(entropy_points)
|
||||
latest_fps = _last_value(fps_points)
|
||||
latest_timesteps = metrics[-1].get("timesteps") if metrics else None
|
||||
if latest_timesteps is None and metrics:
|
||||
latest_timesteps = metrics[-1].get("total_timesteps")
|
||||
latest_reward = _last_value(reward_points)
|
||||
latest_len = _last_value(_series(metrics, "ep_len_mean"))
|
||||
latest_updates = _last_value(_series(metrics, "n_updates"))
|
||||
window = min(10, max(3, len(reward_points)))
|
||||
reward_window = [round(value, 6) for _, value in reward_points[-window:]]
|
||||
best_reward = max((value for _, value in reward_points), default=None)
|
||||
|
||||
notes: list[str] = []
|
||||
if not metrics and not monitor:
|
||||
status, tone = "等待首个 rollout", "gray"
|
||||
notes.append("训练进程启动后先输出环境初始化与首个 rollout 的进度行。")
|
||||
elif not metrics:
|
||||
status, tone = "已收到回合奖励,等待进度行", "warn"
|
||||
notes.append("Monitor CSV 已有回合记录,但尚未收到 --progress-json 进度行;"
|
||||
"首个 rollout 结束后会出现完整指标。")
|
||||
elif len(metrics) < 3:
|
||||
status, tone = "预热阶段", "warn"
|
||||
notes.append("已收到初始指标,需要更多 rollout 才能判断趋势。")
|
||||
else:
|
||||
tone, status = "ok", "收敛趋势良好"
|
||||
# 平台判据(三条同时成立):对半窗口看不出位移、窗口内波动小、
|
||||
# 逐点噪声本身也小。慢速但持续的提升会因第一条被排除。
|
||||
flat = _flatness(reward_points)
|
||||
plateau = False
|
||||
if flat is not None and cv is not None and latest_reward is not None:
|
||||
flat_ratio, flat_noise = flat
|
||||
scale = max(abs(latest_reward), 1e-9)
|
||||
plateau = (flat_ratio <= 1.0 and cv <= 0.15
|
||||
and flat_noise <= 0.02 * scale
|
||||
and (spread is None or spread[1] < 1e-12
|
||||
or spread[0] <= 0.15 * spread[1]))
|
||||
if latest_kl is not None and latest_kl > 0.05:
|
||||
status, tone = "KL 偏大,策略更新可能过猛", "bad"
|
||||
notes.append(f"approx_kl={latest_kl:.4f} > 0.05")
|
||||
elif latest_ev is not None and latest_ev < 0.0:
|
||||
status, tone = "价值函数尚未拟合", "warn"
|
||||
notes.append(f"explained_variance={latest_ev:.3f} < 0")
|
||||
elif reward_slope is not None and reward_slope < 0:
|
||||
status, tone = "奖励未形成上升趋势", "warn"
|
||||
notes.append(f"最近 {window} 点奖励斜率为负({_fmt_num(reward_slope)}/step)")
|
||||
elif plateau:
|
||||
status, tone = "已趋于平稳(疑似收敛)", "ok"
|
||||
notes.append("窗口内对半位移小于逐点噪声,且波动很小"
|
||||
f"(变异系数 {_fmt_num(cv, 3)})")
|
||||
else:
|
||||
notes.append(f"最近 {window} 点奖励斜率为正({_fmt_num(reward_slope)}/step)")
|
||||
if latest_kl is not None and latest_kl <= 0.02:
|
||||
notes.append(f"approx_kl={latest_kl:.4f} 较稳定")
|
||||
if latest_ev is not None:
|
||||
notes.append(f"explained_variance={latest_ev:.3f}")
|
||||
if cv is not None:
|
||||
notes.append(f"奖励变异系数={_fmt_num(cv, 3)}")
|
||||
if latest_entropy is not None and abs(latest_entropy) < 1e-3:
|
||||
notes.append("策略熵接近 0,探索可能过早停止")
|
||||
return {
|
||||
"status": status,
|
||||
"tone": tone,
|
||||
"notes": notes,
|
||||
"window": window,
|
||||
"latest_timesteps": latest_timesteps,
|
||||
"reward_slope": reward_slope,
|
||||
"reward_rate_per_1k": reward_rate,
|
||||
"reward_cv": cv,
|
||||
"reward_spread": spread[0] if spread else None,
|
||||
"reward_window": reward_window,
|
||||
"best_reward": best_reward,
|
||||
"latest_reward": latest_reward,
|
||||
"latest_ep_len": latest_len,
|
||||
"latest_updates": latest_updates,
|
||||
"latest_kl": latest_kl,
|
||||
"latest_explained_variance": latest_ev,
|
||||
"latest_value_loss": latest_vf,
|
||||
"latest_policy_loss": latest_pg,
|
||||
"latest_entropy_loss": latest_entropy,
|
||||
"latest_fps": latest_fps,
|
||||
"reward_points": len(reward_points),
|
||||
}
|
||||
|
||||
|
||||
# 快照状态 -> 界面展示用的进程语义(阶段名称、色调、是否终态)
|
||||
_STATUS_VIEW = {
|
||||
"created": ("已创建", "gray", False, "命令已构造,等待 spawn"),
|
||||
"running": ("运行中", "ok", False, "子进程持有控制台输出与独立进程组"),
|
||||
"paused": ("已暂停", "warn", False, "进程存活但停在 step 边界;点「继续」从暂停处恢复"),
|
||||
"stopping": ("停止中", "warn", False, "已请求优雅停止,等待子进程保存并退出"),
|
||||
"stopped": ("已手动停止", "warn", True, "收到停止请求后退出;脚本会在 step 边界保存模型与元数据"),
|
||||
"exited": ("正常结束", "ok", True, "退出码 0:训练与评估流程走完"),
|
||||
"failed": ("异常结束", "bad", True, "非零退出码或启动失败,见日志尾部"),
|
||||
}
|
||||
|
||||
_PHASES = [
|
||||
("created", "构造命令"),
|
||||
("spawn", "启动子进程"),
|
||||
("rollout", "采集 rollout"),
|
||||
("update", "PPO 更新"),
|
||||
("evaluate", "评估 / 保存"),
|
||||
("done", "收敛归档"),
|
||||
]
|
||||
|
||||
# 状态 -> 当前所处的阶段节点;节点之前为已完成,之后为未开始。
|
||||
_STATUS_PHASE = {
|
||||
"created": "created",
|
||||
"running": "update",
|
||||
"paused": "update",
|
||||
"stopping": "update",
|
||||
"stopped": "done",
|
||||
"exited": "done",
|
||||
"failed": "done",
|
||||
}
|
||||
|
||||
|
||||
def process_pid(process: subprocess.Popen | None) -> int | None:
|
||||
"""已退出进程的 pid 访问会抛 OSError;这里统一吞掉,只用于展示。"""
|
||||
if process is None:
|
||||
return None
|
||||
try:
|
||||
return int(process.pid)
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def describe_process(snapshot: dict[str, Any]) -> dict[str, Any]:
|
||||
"""把 snapshot 派生为界面直接可渲染的进程对象。
|
||||
|
||||
包含:状态语义词、进度(步/百分比)、墙钟与剩余时间估计、最近吞吐、
|
||||
进程时间线节点。数字都取自 snapshot,前端不再重复累加,避免两个刷新
|
||||
频率不同的计数器互相打架。
|
||||
"""
|
||||
status = str(snapshot.get("status") or "created")
|
||||
label, tone, terminal, hint = _STATUS_VIEW.get(
|
||||
status, (status, "gray", False, ""))
|
||||
config = snapshot.get("config") or {}
|
||||
order = [key for key, _ in _PHASES]
|
||||
current = _STATUS_PHASE.get(status, "created")
|
||||
timeline = []
|
||||
for key, title in _PHASES:
|
||||
index = order.index(key)
|
||||
if index < order.index(current):
|
||||
state = "done"
|
||||
elif index == order.index(current):
|
||||
state = "active"
|
||||
else:
|
||||
state = "pending"
|
||||
timeline.append({"key": key, "label": title, "state": state})
|
||||
|
||||
metrics = snapshot.get("metrics") or []
|
||||
latest: dict[str, Any] = metrics[-1] if metrics else {}
|
||||
convergence = snapshot.get("convergence") or {}
|
||||
total_steps = finite_float(config.get("timesteps")) or 0.0
|
||||
# SB3 表格行可能没有 steps 字段,因此进度取"所有进度点的最大步数",
|
||||
# 而不是只看最后一个点——否则末行恰好缺字段时进度会回退到 0。
|
||||
done_steps = 0.0
|
||||
for point in metrics:
|
||||
value = (finite_float(point.get("timesteps"))
|
||||
or finite_float(point.get("total_timesteps")))
|
||||
if value is not None and value > done_steps:
|
||||
done_steps = value
|
||||
if done_steps == 0.0:
|
||||
done_steps = finite_float(convergence.get("latest_timesteps")) or 0.0
|
||||
# init/resume(课程迁移)时 num_timesteps 从上一阶段继续,进度必须按本次
|
||||
# 增量算;start_timesteps 由训练脚本的进度 JSON 提供,缺省按 0 处理。
|
||||
start_steps = 0.0
|
||||
for point in metrics:
|
||||
value = finite_float(point.get("start_timesteps"))
|
||||
if value is not None:
|
||||
start_steps = value
|
||||
break
|
||||
trained_steps = max(0.0, done_steps - start_steps)
|
||||
percent = (100.0 * trained_steps / total_steps) if total_steps > 0 else None
|
||||
if percent is not None:
|
||||
percent = max(0.0, min(100.0, percent))
|
||||
elapsed = float(snapshot.get("elapsed_s") or 0.0)
|
||||
rate = _latest_rate(metrics)
|
||||
remaining: float | None = None
|
||||
eta_ts: float | None = None
|
||||
if rate and rate > 0 and total_steps > 0 and trained_steps < total_steps:
|
||||
remaining = (total_steps - trained_steps) / rate
|
||||
eta_ts = float(snapshot.get("finished_at") or time.time()) + remaining
|
||||
if not terminal and remaining is None:
|
||||
cap = finite_float(config.get("max_wall_s")) or 0.0
|
||||
if cap > 0:
|
||||
remaining = max(0.0, cap - elapsed)
|
||||
return {
|
||||
"label": label,
|
||||
"tone": tone,
|
||||
"terminal": terminal,
|
||||
"hint": hint,
|
||||
"phase": current,
|
||||
"timeline": timeline,
|
||||
"percent": percent,
|
||||
"done_steps": done_steps,
|
||||
"start_steps": start_steps,
|
||||
"trained_steps": trained_steps,
|
||||
"total_steps": total_steps,
|
||||
"elapsed_s": round(elapsed, 3),
|
||||
"rate_per_s": rate,
|
||||
"remaining_s": remaining,
|
||||
"eta_epoch_s": eta_ts,
|
||||
"log_total": snapshot.get("log_total"),
|
||||
"log_capacity": MAX_LOG_LINES,
|
||||
"metric_total": snapshot.get("metric_total"),
|
||||
"latest": latest,
|
||||
}
|
||||
|
||||
|
||||
def _latest_rate(metrics: list[dict[str, Any]], window: int = 6) -> float | None:
|
||||
"""最近若干个**带 fps 的**进度点的平均吞吐(步/秒),用于剩余时间估计。
|
||||
|
||||
progress JSON 行不含 fps,只有 SB3 表格行有;若只看最后 window 个点,
|
||||
恰好落在若干 JSON 行之间时会得到 None,剩余时间就会退化成墙钟上限的
|
||||
粗糙估计。这里向前扫描直到拿到至少 1 个 fps(最多看 3*window 个点)。
|
||||
"""
|
||||
if not metrics:
|
||||
return None
|
||||
usable: list[float] = []
|
||||
for point in reversed(metrics[-window * 3:]):
|
||||
value = finite_float(point.get("fps"))
|
||||
if value and value > 0:
|
||||
usable.append(value)
|
||||
if len(usable) >= window:
|
||||
break
|
||||
if not usable:
|
||||
return None
|
||||
return sum(usable) / len(usable)
|
||||
@@ -15,6 +15,8 @@ import random
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
from .analysis import describe_process, summarize_convergence
|
||||
|
||||
__all__ = ["DEMO_CONFIG", "DEMO_SCENARIOS", "demo_metrics", "demo_monitor",
|
||||
"demo_log_lines", "demo_snapshot"]
|
||||
|
||||
@@ -44,7 +46,7 @@ DEMO_STEPS = 48
|
||||
|
||||
|
||||
def _finite(value: float) -> float | None:
|
||||
if value != value or value in (float("inf"), float("-inf")):
|
||||
if not math.isfinite(value):
|
||||
return None
|
||||
return round(value, 8)
|
||||
|
||||
@@ -171,8 +173,6 @@ def demo_snapshot(*, log_tail: int = 400, scenario: str = "rising",
|
||||
(而不是刚好 100%);``scenario`` 选择收敛判读分支;``fps`` 决定剩余
|
||||
时间估计的吞吐量级。
|
||||
"""
|
||||
from .server import describe_process, summarize_convergence
|
||||
|
||||
now = time.time()
|
||||
ratio = min(1.0, max(0.05, float(done_ratio)))
|
||||
metrics = demo_metrics(count=max(6, int(round(DEMO_STEPS * ratio))),
|
||||
@@ -193,6 +193,7 @@ def demo_snapshot(*, log_tail: int = 400, scenario: str = "rising",
|
||||
"elapsed_s": 184.0,
|
||||
"returncode": None,
|
||||
"error": None,
|
||||
"forced_kill": False,
|
||||
"command": list(DEMO_COMMAND),
|
||||
"model": str(config["model"]),
|
||||
"config": config,
|
||||
@@ -203,6 +204,17 @@ def demo_snapshot(*, log_tail: int = 400, scenario: str = "rising",
|
||||
"metric_end": len(metrics),
|
||||
"metric_total": len(metrics),
|
||||
"monitor": monitor,
|
||||
"recovery": {
|
||||
"can_resume": True,
|
||||
"model": str(config["model"]),
|
||||
"timesteps": int(DEMO_STEPS * ratio * 12),
|
||||
"reason": "recovery",
|
||||
"updated_unix_s": now - 24.0,
|
||||
"source": "resume.json",
|
||||
"hint_file": str(config["model"]).rsplit("/", 1)[0] + "/resume.json",
|
||||
"candidates": 3,
|
||||
"preview": True,
|
||||
},
|
||||
"preview": True,
|
||||
"preview_scenario": scenario,
|
||||
}
|
||||
|
||||
@@ -163,6 +163,24 @@ CONFIG_FIELDS: list[dict[str, Any]] = [
|
||||
"label": "覆盖已有模型 --force", "default": False,
|
||||
"help": "输出路径已存在时必须勾选,否则训练脚本会拒绝覆盖。与续训互斥。",
|
||||
},
|
||||
# ---------------------------------------------------------- 恢复与落盘
|
||||
{
|
||||
"key": "recovery_every", "kind": "integer", "group": "recovery",
|
||||
"label": "滚动恢复点间隔 --recovery-every", "default": 2000,
|
||||
"help": "每 N 个环境步原子覆盖写 <checkpoints>/<stem>_latest.zip,"
|
||||
"训练开始前先写起步基线;急停(强杀)后最多丢 N 步。0=关闭。",
|
||||
},
|
||||
{
|
||||
"key": "checkpoint_every", "kind": "integer", "group": "recovery",
|
||||
"label": "历史 checkpoint 间隔 --checkpoint-every", "default": 20000,
|
||||
"help": "每 N 个环境步另留一份 <stem>_<步数>.zip 供返祖对比;"
|
||||
"0=只保留滚动恢复点。",
|
||||
},
|
||||
{
|
||||
"key": "checkpoint_keep", "kind": "integer", "group": "recovery",
|
||||
"label": "历史 checkpoint 保留数 --checkpoint-keep", "default": 3,
|
||||
"help": "只保留最近 K 个历史 checkpoint;滚动恢复点与起步基线不受影响。",
|
||||
},
|
||||
]
|
||||
|
||||
CONFIG_GROUPS: list[dict[str, str]] = [
|
||||
@@ -170,6 +188,7 @@ CONFIG_GROUPS: list[dict[str, str]] = [
|
||||
{"key": "ppo", "label": "PPO 超参数", "hint": "采样宽度与更新强度"},
|
||||
{"key": "runtime", "label": "运行环境", "hint": "并行、设备与低层执行器"},
|
||||
{"key": "scale", "label": "训练规模", "hint": "步数、墙钟上限与评估"},
|
||||
{"key": "recovery", "label": "恢复与落盘", "hint": "急停恢复点与历史 checkpoint"},
|
||||
{"key": "paths", "label": "模型与迁移", "hint": "产物路径、续训与课程迁移"},
|
||||
]
|
||||
|
||||
|
||||
+285
-401
@@ -4,7 +4,8 @@
|
||||
设计边界:
|
||||
- 只监听 127.0.0.1,只允许白名单静态文件;
|
||||
- 只允许一个训练任务,命令由服务端用参数列表构造,不经过 shell;
|
||||
- 支持启动/停止/状态查询;停止时发送 SIGTERM 到独立进程组;
|
||||
- 支持启动/暂停/继续/停止/状态查询;暂停与停止通过**控制文件**下发,
|
||||
POSIX 与 Windows 行为一致(详见 :mod:`mcm26b.process_control`);
|
||||
- 训练进程通过 ``--progress-json`` 输出单行 JSON 进度,服务端同时兼容
|
||||
Stable-Baselines3 的表格输出和 Monitor CSV。
|
||||
|
||||
@@ -12,15 +13,17 @@
|
||||
- ``GET /api/train/schema`` 字段/分组/预设/指标字典(``schema.py`` 单一出处)
|
||||
- ``GET /api/train/state`` 快照;``after_metric=N`` 只回增量指标
|
||||
- ``POST /api/train/start`` 以部分字段启动(其余取默认值)
|
||||
- ``POST /api/train/stop`` SIGTERM 独立进程组
|
||||
- ``POST /api/train/pause`` 请求训练进程在 step 边界挂起
|
||||
- ``POST /api/train/resume`` 解除挂起
|
||||
- ``POST /api/train/stop`` 请求优雅停止,超时后强杀进程树
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import signal
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
@@ -32,15 +35,24 @@ from pathlib import Path
|
||||
from typing import Any
|
||||
from urllib.parse import parse_qs, urlsplit
|
||||
|
||||
from ..process_control import (DEFAULT_GRACE_S, spawn_kwargs,
|
||||
terminate_process_tree, write_control,
|
||||
write_json_atomic)
|
||||
from .analysis import (MAX_LOG_LINES, MAX_METRICS, describe_process, finite_float,
|
||||
process_pid, summarize_convergence)
|
||||
from .demo import DEMO_SCENARIOS, demo_snapshot
|
||||
from .schema import config_schema
|
||||
|
||||
__all__ = ["TrainingJob", "TrainingUI", "run_training_ui", "build_command",
|
||||
"summarize_convergence", "describe_process"]
|
||||
"summarize_convergence", "describe_process", "discover_recovery"]
|
||||
|
||||
STATIC_DIR = Path(__file__).resolve().parent / "static"
|
||||
REPO_ROOT = Path(__file__).resolve().parents[3]
|
||||
TRAIN_SCRIPT = REPO_ROOT / "scripts" / "train_hrl_high.py"
|
||||
# 控制文件集中放在 output/ 下(已 gitignore),job 之间互不干扰
|
||||
CONTROL_DIR = REPO_ROOT / "output" / "hrl" / "ui" / "control"
|
||||
# 从发出停止请求到强杀之间留给训练脚本收尾的秒数
|
||||
STOP_GRACE_S = DEFAULT_GRACE_S
|
||||
STATIC_FILES = {
|
||||
"/": ("index.html", "text/html; charset=utf-8"),
|
||||
"/index.html": ("index.html", "text/html; charset=utf-8"),
|
||||
@@ -49,10 +61,6 @@ STATIC_FILES = {
|
||||
"/style.css": ("style.css", "text/css; charset=utf-8"),
|
||||
}
|
||||
|
||||
MAX_LOG_LINES = 4000
|
||||
MAX_METRICS = 5000
|
||||
|
||||
_NUM_RE = re.compile(r"^[+-]?(?:\d+\.?\d*|\.\d+)(?:[eE][+-]?\d+)?$")
|
||||
_SB3_SECTIONS = ("rollout/", "time/", "train/", "system/")
|
||||
# 路径/字符串参数只拒绝控制字符(含 \r\n\t 与 DEL),不做 shell 元字符过滤:
|
||||
# 命令走参数列表、不经过 shell,这里只是防日志与命令行被注入控制序列。
|
||||
@@ -61,23 +69,6 @@ _SB3_SECTIONS = ("rollout/", "time/", "train/", "system/")
|
||||
_SAFE_ARG_RE = re.compile(r"^[^\x00-\x1f\x7f]{1,512}$")
|
||||
|
||||
|
||||
def _finite_float(value: Any) -> float | None:
|
||||
if value is None or isinstance(value, bool):
|
||||
return None
|
||||
if isinstance(value, (int, float)):
|
||||
value = float(value)
|
||||
if value != value or value in (float("inf"), float("-inf")):
|
||||
return None
|
||||
return value
|
||||
text = str(value).strip()
|
||||
if not _NUM_RE.match(text):
|
||||
return None
|
||||
try:
|
||||
return float(text)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def _normalize_metric(key: str) -> str:
|
||||
"""把 SB3 的 ``rollout/ep_rew_mean`` 与进度 JSON 的 ``ep_rew_mean``
|
||||
归一化到同一短名,便于前端绘制。"""
|
||||
@@ -109,6 +100,10 @@ class TrainConfig:
|
||||
max_wall_s: float = 1200.0
|
||||
local_selector: str = "geometry"
|
||||
local_model: str | None = None
|
||||
# 落盘/恢复:默认值与 scripts/train_hrl_high.py 保持一致
|
||||
checkpoint_every: int = 20000
|
||||
checkpoint_keep: int = 3
|
||||
recovery_every: int = 2000
|
||||
verbose: int = 1
|
||||
|
||||
@classmethod
|
||||
@@ -117,9 +112,11 @@ class TrainConfig:
|
||||
raise ValueError("请求体必须是 JSON 对象")
|
||||
defaults = cls()
|
||||
data: dict[str, Any] = {}
|
||||
for field in cls.__dataclass_fields__:
|
||||
if field in payload and payload[field] is not None:
|
||||
data[field] = payload[field]
|
||||
# 用 dataclasses.fields 而不是 cls.__dataclass_fields__:后者是内部
|
||||
# 实现细节,pylint 看不到它,也无法保证未来版本仍存在。
|
||||
for field in dataclasses.fields(cls):
|
||||
if field.name in payload and payload[field.name] is not None:
|
||||
data[field.name] = payload[field.name]
|
||||
cfg = cls(**{**defaults.__dict__, **data})
|
||||
cfg.validate()
|
||||
return cfg
|
||||
@@ -153,6 +150,12 @@ class TrainConfig:
|
||||
raise ValueError("max_wall_s 必须在 0..7200 秒")
|
||||
if int(self.eval_episodes) < 0:
|
||||
raise ValueError("eval_episodes 不能为负")
|
||||
if not 0 <= int(self.checkpoint_every) <= 10_000_000:
|
||||
raise ValueError("checkpoint_every 必须在 0..10,000,000")
|
||||
if not 1 <= int(self.checkpoint_keep) <= 100:
|
||||
raise ValueError("checkpoint_keep 必须在 1..100")
|
||||
if not 0 <= int(self.recovery_every) <= 10_000_000:
|
||||
raise ValueError("recovery_every 必须在 0..10,000,000")
|
||||
for label, value in (("model", self.model), ("init", self.init),
|
||||
("local_model", self.local_model)):
|
||||
if value is None:
|
||||
@@ -207,6 +210,9 @@ def build_command(config: TrainConfig) -> list[str]:
|
||||
"--n-epochs", str(config.n_epochs),
|
||||
"--max-wall-s", str(config.max_wall_s),
|
||||
"--local-selector", config.local_selector,
|
||||
"--checkpoint-every", str(config.checkpoint_every),
|
||||
"--checkpoint-keep", str(config.checkpoint_keep),
|
||||
"--recovery-every", str(config.recovery_every),
|
||||
"--progress-json",
|
||||
"--verbose", str(config.verbose),
|
||||
]
|
||||
@@ -223,25 +229,118 @@ def build_command(config: TrainConfig) -> list[str]:
|
||||
return cmd
|
||||
|
||||
|
||||
class TrainingJob:
|
||||
"""单个训练子进程的生命周期、日志和指标缓存。"""
|
||||
def discover_recovery(run_dir: Path, model_path: Path) -> dict[str, Any]:
|
||||
"""找出"现在就能继续训练"的那份产物。
|
||||
|
||||
def __init__(self, config: TrainConfig):
|
||||
优先级:
|
||||
1. ``<run_dir>/resume.json``——训练脚本写的权威指针(校验目标文件真实存在);
|
||||
2. 否则按 mtime 取最新:最终模型 / 滚动恢复点 / 历史 checkpoint / 异常兜底。
|
||||
|
||||
只认**已完整落盘**的文件:模型的 zip 与 json 都是原子替换写的,所以不会
|
||||
出现"文件在、但打不开"的半截快照。
|
||||
"""
|
||||
run_dir = Path(run_dir)
|
||||
model_path = Path(model_path)
|
||||
hint_file = run_dir / "resume.json"
|
||||
hint = _read_json_file(hint_file)
|
||||
|
||||
candidates: list[tuple[float, str, str]] = []
|
||||
|
||||
def _consider(path: Path, reason: str) -> None:
|
||||
try:
|
||||
stat = path.stat()
|
||||
except OSError:
|
||||
return
|
||||
if stat.st_size > 0:
|
||||
candidates.append((stat.st_mtime, str(path), reason))
|
||||
|
||||
_consider(model_path, "final_model")
|
||||
ckpt_dir = run_dir / "checkpoints"
|
||||
if ckpt_dir.is_dir():
|
||||
_consider(ckpt_dir / f"{model_path.stem}_ckpt_latest.zip", "recovery")
|
||||
for path in ckpt_dir.glob(f"{model_path.stem}_ckpt_[0-9]*.zip"):
|
||||
if path.stem.rsplit("_", 1)[-1].isdigit():
|
||||
_consider(path, "checkpoint")
|
||||
_consider(run_dir / f"{model_path.stem}_emergency.zip", "emergency")
|
||||
|
||||
state: dict[str, Any] = {
|
||||
"can_resume": False, "model": None, "timesteps": None,
|
||||
"reason": "none", "updated_unix_s": None, "source": "none",
|
||||
"hint_file": str(hint_file), "candidates": len(candidates),
|
||||
}
|
||||
if hint and hint.get("model") and Path(str(hint["model"])).is_file():
|
||||
state.update({
|
||||
"model": str(hint["model"]),
|
||||
"timesteps": _int_or_none(hint.get("timesteps")),
|
||||
"reason": str(hint.get("reason") or "resume_hint"),
|
||||
"updated_unix_s": _float_or_none(hint.get("created_unix_s")),
|
||||
"source": "resume.json",
|
||||
})
|
||||
elif candidates:
|
||||
mtime, path, reason = max(candidates, key=lambda item: item[0])
|
||||
state.update({
|
||||
"model": path, "reason": reason,
|
||||
"updated_unix_s": mtime, "source": "scan",
|
||||
})
|
||||
if state["model"]:
|
||||
state["can_resume"] = Path(str(state["model"])).is_file()
|
||||
return state
|
||||
|
||||
|
||||
def _read_json_file(path: Path) -> dict[str, Any] | None:
|
||||
"""读 JSON 文件;缺失或损坏返回 None(恢复指针不是关键路径,不能因此崩)。"""
|
||||
try:
|
||||
payload = json.loads(Path(path).read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError, UnicodeDecodeError):
|
||||
return None
|
||||
return payload if isinstance(payload, dict) else None
|
||||
|
||||
|
||||
def _int_or_none(value: Any) -> int | None:
|
||||
try:
|
||||
return int(value)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _float_or_none(value: Any) -> float | None:
|
||||
try:
|
||||
return float(value)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
class TrainingJob:
|
||||
"""单个训练子进程的生命周期、日志和指标缓存。
|
||||
|
||||
``control_dir`` 是控制文件的落地目录;默认与模型同目录,服务端则统一放到
|
||||
``output/hrl/ui/control``(已被 gitignore)。
|
||||
"""
|
||||
|
||||
def __init__(self, config: TrainConfig, control_dir: Path | None = None):
|
||||
self.config = config
|
||||
self.job_id = f"job-{int(time.time() * 1000)}"
|
||||
self.command = config.command()
|
||||
self.control_path = (Path(control_dir) if control_dir is not None
|
||||
else config.model_path.parent) / \
|
||||
f"{self.job_id}.control.json"
|
||||
# 控制文件是暂停/停止的唯一可靠通道:Windows 上没有 SIGSTOP,
|
||||
# 且 os.kill(SIGTERM) 会退化成硬杀,信号无法承载"优雅收尾"语义。
|
||||
self.command = config.command() + ["--control-file", str(self.control_path)]
|
||||
self.created_at = time.time()
|
||||
self.started_at: float | None = None
|
||||
self.finished_at: float | None = None
|
||||
self.returncode: int | None = None
|
||||
self.status = "created" # created/running/exited/failed/stopped
|
||||
# created/running/paused/stopping/stopped/exited/failed
|
||||
self.status = "created"
|
||||
self.error: str | None = None
|
||||
self.forced_kill = False # 停止请求超时后是否动用了强杀
|
||||
self.process: subprocess.Popen | None = None
|
||||
self.logs: deque[str] = deque(maxlen=MAX_LOG_LINES)
|
||||
self.metrics: deque[dict[str, Any]] = deque(maxlen=MAX_METRICS)
|
||||
self.monitor: list[dict[str, Any]] = []
|
||||
self._lock = threading.RLock()
|
||||
self._reader: threading.Thread | None = None
|
||||
self._stopper: threading.Thread | None = None
|
||||
self._table_sections: list[tuple[str, dict[str, Any]]] = []
|
||||
self._table_steps: float | None = None # 当前表的 total_timesteps
|
||||
self._pending_table: dict[str, Any] | None = None # 未见刻度的半张表
|
||||
@@ -254,10 +353,13 @@ class TrainingJob:
|
||||
with self._lock:
|
||||
if self.process is not None and self.process.poll() is None:
|
||||
raise RuntimeError("已有训练任务在运行")
|
||||
# 先落一份"不暂停、不停止"的初始控制文件,训练脚本首步就能读到;
|
||||
# 否则脚本要等到第一次轮询才区分"文件缺失"与"无指令"。
|
||||
write_control(self.control_path, paused=False, stop=False)
|
||||
env = os.environ.copy()
|
||||
env["PYTHONUNBUFFERED"] = "1"
|
||||
env["HRL_UI_JOB_ID"] = self.job_id
|
||||
self.process = subprocess.Popen(
|
||||
self.process = subprocess.Popen( # pylint: disable=consider-using-with
|
||||
self.command,
|
||||
cwd=str(REPO_ROOT),
|
||||
stdout=subprocess.PIPE,
|
||||
@@ -265,7 +367,7 @@ class TrainingJob:
|
||||
text=True,
|
||||
bufsize=1,
|
||||
env=env,
|
||||
start_new_session=True,
|
||||
**spawn_kwargs(),
|
||||
)
|
||||
self.started_at = time.time()
|
||||
self.status = "running"
|
||||
@@ -273,18 +375,101 @@ class TrainingJob:
|
||||
target=self._read_loop, name=f"{self.job_id}-reader", daemon=True)
|
||||
self._reader.start()
|
||||
|
||||
def stop(self) -> bool:
|
||||
def pause(self) -> bool:
|
||||
"""请求训练进程在下一个 step 边界挂起(跨平台:控制文件)。"""
|
||||
with self._lock:
|
||||
process = self.process
|
||||
if process is None or process.poll() is not None:
|
||||
return False
|
||||
try:
|
||||
os.killpg(os.getpgid(process.pid), signal.SIGTERM)
|
||||
except (ProcessLookupError, PermissionError):
|
||||
process.terminate()
|
||||
self.status = "stopping"
|
||||
if self.status not in ("running", "paused"):
|
||||
return False
|
||||
write_control(self.control_path, paused=True, stop=False)
|
||||
self.status = "paused"
|
||||
return True
|
||||
|
||||
def resume(self) -> bool:
|
||||
"""解除挂起;训练从暂停的那个 step 继续。"""
|
||||
with self._lock:
|
||||
process = self.process
|
||||
if process is None or process.poll() is not None:
|
||||
return False
|
||||
if self.status not in ("paused", "running"):
|
||||
return False
|
||||
write_control(self.control_path, paused=False, stop=False)
|
||||
self.status = "running"
|
||||
return True
|
||||
|
||||
def stop(self) -> bool:
|
||||
"""请求优雅停止;超时后由看门狗强杀整棵进程树。
|
||||
|
||||
控制文件先于信号:Windows 上 SIGTERM 等价于 TerminateProcess,训练
|
||||
脚本"保存模型 + 写元数据"的收尾分支不会执行;控制文件则两个平台都能
|
||||
命中同一段协作式收尾。强杀放到独立线程,避免 HTTP 请求被 grace 等待卡住。
|
||||
"""
|
||||
with self._lock:
|
||||
process = self.process
|
||||
if process is None or process.poll() is not None:
|
||||
return False
|
||||
write_control(self.control_path, paused=False, stop=True)
|
||||
self.status = "stopping"
|
||||
if self._stopper is None or not self._stopper.is_alive():
|
||||
self._stopper = threading.Thread(
|
||||
target=self._stop_watchdog, args=(process,),
|
||||
name=f"{self.job_id}-stopper", daemon=True)
|
||||
self._stopper.start()
|
||||
return True
|
||||
|
||||
def _stop_watchdog(self, process: subprocess.Popen) -> None:
|
||||
"""等 grace 让协作式停止生效,超时强杀并记下恢复点。"""
|
||||
forced = terminate_process_tree(process, grace_s=STOP_GRACE_S)
|
||||
with self._lock:
|
||||
self.forced_kill = forced
|
||||
if forced and not self.error:
|
||||
self.error = (f"训练进程未在 {STOP_GRACE_S:.0f}s 内响应停止请求,"
|
||||
f"已强制结束进程树")
|
||||
if forced:
|
||||
self._write_forced_stop(process.pid)
|
||||
|
||||
def _write_forced_stop(self, pid: int) -> None:
|
||||
"""强杀后落一份 `<模型目录>/forced_stop.json`,写明"从哪继续"。
|
||||
|
||||
强杀没有给训练脚本任何收尾机会,所以这里由前端补上审计与恢复指引:
|
||||
磁盘上仍有原子落盘的滚动恢复点 / 历史 checkpoint,``recovery.model``
|
||||
就是可直接 ``--resume`` 的目标。
|
||||
"""
|
||||
run_dir = self.config.model_path.parent
|
||||
try:
|
||||
recovery = discover_recovery(run_dir, self.config.model_path)
|
||||
write_json_atomic(run_dir / "forced_stop.json", {
|
||||
"event": "forced_stop",
|
||||
"job_id": self.job_id,
|
||||
"pid": int(pid),
|
||||
"created_unix_s": time.time(),
|
||||
"reason": "grace_timeout",
|
||||
"grace_s": STOP_GRACE_S,
|
||||
"recovery": recovery,
|
||||
"note": ("训练进程未响应控制文件,已强杀整棵进程树;"
|
||||
"把 recovery.model 作为 --resume 目标即可继续"
|
||||
"(timesteps 为追加步数)"),
|
||||
})
|
||||
except OSError as exc: # 标记文件写不了也不能影响停止流程
|
||||
with self._lock:
|
||||
if not self.error:
|
||||
self.error = f"强杀后写恢复标记失败:{exc}"
|
||||
|
||||
def wait_for_exit(self, timeout: float | None = None) -> bool:
|
||||
"""等待停止流程收尾(优雅退出或强杀);供前端进程退出前调用。
|
||||
|
||||
训练子进程在独立进程组里,前端先退出并不会带走它。若不在这里等,
|
||||
前端一退出,子进程会成为孤儿——虽然控制文件仍会让它自行收尾,但
|
||||
日志与退出码就没人接了。
|
||||
"""
|
||||
thread = self._stopper
|
||||
if thread is not None:
|
||||
thread.join(timeout)
|
||||
process = self.process
|
||||
return process is None or process.poll() is not None
|
||||
|
||||
def _read_loop(self) -> None:
|
||||
process = self.process
|
||||
assert process is not None and process.stdout is not None
|
||||
@@ -315,10 +500,38 @@ class TrainingJob:
|
||||
def _on_line(self, line: str) -> None:
|
||||
with self._lock:
|
||||
self.logs.append(line)
|
||||
self._sync_control_events(line)
|
||||
self._parse_progress_json(line)
|
||||
self._parse_sb3_table(line)
|
||||
self._refresh_monitor()
|
||||
|
||||
def _sync_control_events(self, line: str) -> None:
|
||||
"""用训练脚本的回显校正前端状态。
|
||||
|
||||
正常情况下状态由前端自己设置;但外部直接改控制文件、或脚本比前端更早
|
||||
感知到指令时,以脚本回显的 ``paused`` / ``resumed`` / ``stop_requested``
|
||||
为准更真实。
|
||||
"""
|
||||
stripped = line.strip()
|
||||
if not stripped.startswith("{") or '"event"' not in stripped:
|
||||
return
|
||||
try:
|
||||
payload = json.loads(stripped)
|
||||
except ValueError:
|
||||
return
|
||||
if not isinstance(payload, dict):
|
||||
return
|
||||
event = payload.get("event")
|
||||
if event not in ("paused", "resumed", "stop_requested"):
|
||||
return
|
||||
with self._lock:
|
||||
if event == "paused" and self.status == "running":
|
||||
self.status = "paused"
|
||||
elif event == "resumed" and self.status == "paused":
|
||||
self.status = "running"
|
||||
elif event == "stop_requested" and self.status in ("running", "paused"):
|
||||
self.status = "stopping"
|
||||
|
||||
# ------------------------------------------------------------ parsers
|
||||
def _parse_progress_json(self, line: str) -> None:
|
||||
stripped = line.strip()
|
||||
@@ -332,9 +545,9 @@ class TrainingJob:
|
||||
return
|
||||
point = {_normalize_metric(str(key)): value
|
||||
for key, value in payload.items() if key != "event"}
|
||||
steps = _finite_float(point.get("timesteps"))
|
||||
steps = finite_float(point.get("timesteps"))
|
||||
if steps is None:
|
||||
steps = _finite_float(point.get("total_timesteps"))
|
||||
steps = finite_float(point.get("total_timesteps"))
|
||||
if steps is not None:
|
||||
with self._lock:
|
||||
self._last_json_steps = steps
|
||||
@@ -371,7 +584,7 @@ class TrainingJob:
|
||||
self._table_sections.append((raw_key, {}))
|
||||
return
|
||||
key = _normalize_metric(raw_key)
|
||||
value = _finite_float(raw_value)
|
||||
value = finite_float(raw_value)
|
||||
if not key or value is None:
|
||||
return
|
||||
with self._lock:
|
||||
@@ -401,7 +614,7 @@ class TrainingJob:
|
||||
for _name, items in sections:
|
||||
table.update(items)
|
||||
if steps is None:
|
||||
steps = _finite_float(table.get("total_timesteps"))
|
||||
steps = finite_float(table.get("total_timesteps"))
|
||||
if steps is None:
|
||||
# 半张表:交给调用方决定是否已确认结束(终止时按末尾兜底提交)
|
||||
with self._lock:
|
||||
@@ -432,7 +645,7 @@ class TrainingJob:
|
||||
def _append_metric(self, point: dict[str, Any]) -> None:
|
||||
clean: dict[str, Any] = {}
|
||||
for key, value in point.items():
|
||||
number = _finite_float(value)
|
||||
number = finite_float(value)
|
||||
clean[key] = number if number is not None else value
|
||||
# 补齐 timesteps:progress JSON 优先,SB3 表用 total_timesteps。
|
||||
if "timesteps" not in clean and "total_timesteps" in clean:
|
||||
@@ -471,7 +684,7 @@ class TrainingJob:
|
||||
row = {}
|
||||
valid = True
|
||||
for key, raw in zip(header, values):
|
||||
number = _finite_float(raw)
|
||||
number = finite_float(raw)
|
||||
if number is None:
|
||||
valid = False
|
||||
break
|
||||
@@ -505,17 +718,17 @@ class TrainingJob:
|
||||
config = dict(self.config.__dict__)
|
||||
command = list(self.command)
|
||||
model_path = str(self.config.model_path)
|
||||
timesteps = int(self.config.timesteps)
|
||||
payload = {
|
||||
"job_id": self.job_id,
|
||||
"status": self.status,
|
||||
"pid": _pid(self.process),
|
||||
"pid": process_pid(self.process),
|
||||
"created_at": self.created_at,
|
||||
"started_at": self.started_at,
|
||||
"finished_at": self.finished_at,
|
||||
"elapsed_s": round(elapsed, 3),
|
||||
"returncode": self.returncode,
|
||||
"error": self.error,
|
||||
"forced_kill": self.forced_kill,
|
||||
"command": command,
|
||||
"model": model_path,
|
||||
"config": config,
|
||||
@@ -529,356 +742,12 @@ class TrainingJob:
|
||||
}
|
||||
payload["convergence"] = summarize_convergence(metrics, monitor)
|
||||
payload["process"] = describe_process(payload)
|
||||
# 恢复指针在锁外做文件扫描:即使训练被强杀,这里也能给出可续训的产物
|
||||
payload["recovery"] = discover_recovery(
|
||||
self.config.model_path.parent, self.config.model_path)
|
||||
return payload
|
||||
|
||||
|
||||
def _series(metrics: list[dict[str, Any]], key: str) -> list[tuple[float, float]]:
|
||||
points: list[tuple[float, float]] = []
|
||||
for point in metrics:
|
||||
value = _finite_float(point.get(key))
|
||||
steps = _finite_float(point.get("timesteps")) or _finite_float(
|
||||
point.get("total_timesteps"))
|
||||
if value is None:
|
||||
continue
|
||||
points.append((steps if steps is not None else float(len(points) + 1), value))
|
||||
return points
|
||||
|
||||
|
||||
def _last_value(points: list[tuple[float, float]]) -> float | None:
|
||||
return points[-1][1] if points else None
|
||||
|
||||
|
||||
def _spread(points: list[tuple[float, float]], window: int = 10
|
||||
) -> tuple[float, float] | None:
|
||||
"""最近 window 个点的 (极差, 起点绝对值),用于判断"是否还在动"。"""
|
||||
sample = points[-window:]
|
||||
if len(sample) < 3:
|
||||
return None
|
||||
ys = [item[1] for item in sample]
|
||||
return max(ys) - min(ys), abs(ys[0])
|
||||
|
||||
|
||||
def _cv(points: list[tuple[float, float]], window: int = 10) -> float | None:
|
||||
"""最近 window 个点的变异系数(标准差/|均值|),越小越平稳。"""
|
||||
sample = [item[1] for item in points[-window:]]
|
||||
if len(sample) < 3:
|
||||
return None
|
||||
mean = sum(sample) / len(sample)
|
||||
if abs(mean) < 1e-12:
|
||||
return None
|
||||
var = sum((value - mean) ** 2 for value in sample) / len(sample)
|
||||
return (var ** 0.5) / abs(mean)
|
||||
|
||||
|
||||
def _slope(points: list[tuple[float, float]], window: int = 8) -> float | None:
|
||||
sample = points[-window:]
|
||||
if len(sample) < 3:
|
||||
return None
|
||||
xs = [item[0] for item in sample]
|
||||
ys = [item[1] for item in sample]
|
||||
n = len(xs)
|
||||
mean_x = sum(xs) / n
|
||||
mean_y = sum(ys) / n
|
||||
denom = sum((x - mean_x) ** 2 for x in xs)
|
||||
if abs(denom) < 1e-12:
|
||||
return None
|
||||
return sum((x - mean_x) * (y - mean_y) for x, y in zip(xs, ys)) / denom
|
||||
|
||||
|
||||
def _rate_per_1k(reward_points: list[tuple[float, float]],
|
||||
window: int = 8) -> float | None:
|
||||
"""最近 window 个点的奖励增益换算成"每 1000 环境步",便于读趋势。"""
|
||||
slope = _slope(reward_points, window)
|
||||
if slope is None:
|
||||
return None
|
||||
return slope * 1000.0
|
||||
|
||||
|
||||
def _fmt_num(value: float | None, digits: int = 4) -> str:
|
||||
if value is None:
|
||||
return "—"
|
||||
if value != 0 and abs(value) < 1e-3:
|
||||
return f"{value:.3e}"
|
||||
return f"{value:.{digits}g}"
|
||||
|
||||
|
||||
def _flatness(points: list[tuple[float, float]], window: int = 10
|
||||
) -> tuple[float, float] | None:
|
||||
"""窗口内"趋势位移 / 逐点噪声"比值:<1 表示看不出系统性变化。
|
||||
|
||||
判定收敛时只看斜率绝对值会把"很慢但持续的提升"误判为平台;这里把窗口
|
||||
对半切,用两半均值之差当趋势位移,用相邻点平均绝对差当噪声尺度。返回
|
||||
(比值, 噪声尺度),两者都小才是真的"又平又稳"。
|
||||
"""
|
||||
sample = points[-window:]
|
||||
if len(sample) < 4:
|
||||
return None
|
||||
ys = [item[1] for item in sample]
|
||||
half = len(ys) // 2
|
||||
shift = abs(sum(ys[half:]) / len(ys[half:]) - sum(ys[:half]) / len(ys[:half]))
|
||||
noise = sum(abs(b - a) for a, b in zip(ys, ys[1:])) / (len(ys) - 1)
|
||||
if noise <= 0:
|
||||
return (0.0, 0.0) if shift <= 0 else (float("inf"), 0.0)
|
||||
return shift / noise, noise
|
||||
|
||||
|
||||
def summarize_convergence(metrics: list[dict[str, Any]],
|
||||
monitor: list[dict[str, Any]]) -> dict[str, Any]:
|
||||
"""给前端返回可解释的收敛状态与各参数最近值。
|
||||
|
||||
这是**启发式判读**,不替代正式学习曲线分析:只依据最近窗口的斜率、
|
||||
极差与变异系数给出"平坦/上升/未上升"的分类,并把同一组数字交给前端
|
||||
展示,便于人工复核。
|
||||
"""
|
||||
reward_points = _series(metrics, "ep_rew_mean")
|
||||
if not reward_points and monitor:
|
||||
reward_points = [(float(index), float(row.get("r", 0.0)))
|
||||
for index, row in enumerate(monitor)
|
||||
if _finite_float(row.get("r")) is not None]
|
||||
kl_points = _series(metrics, "approx_kl")
|
||||
ev_points = _series(metrics, "explained_variance")
|
||||
vf_points = _series(metrics, "value_loss")
|
||||
pg_points = _series(metrics, "policy_gradient_loss")
|
||||
entropy_points = _series(metrics, "entropy_loss")
|
||||
fps_points = _series(metrics, "fps")
|
||||
|
||||
reward_slope = _slope(reward_points)
|
||||
reward_rate = _rate_per_1k(reward_points)
|
||||
spread = _spread(reward_points)
|
||||
cv = _cv(reward_points)
|
||||
latest_kl = _last_value(kl_points)
|
||||
latest_ev = _last_value(ev_points)
|
||||
latest_vf = _last_value(vf_points)
|
||||
latest_pg = _last_value(pg_points)
|
||||
latest_entropy = _last_value(entropy_points)
|
||||
latest_fps = _last_value(fps_points)
|
||||
latest_timesteps = metrics[-1].get("timesteps") if metrics else None
|
||||
if latest_timesteps is None and metrics:
|
||||
latest_timesteps = metrics[-1].get("total_timesteps")
|
||||
latest_reward = _last_value(reward_points)
|
||||
latest_len = _last_value(_series(metrics, "ep_len_mean"))
|
||||
latest_updates = _last_value(_series(metrics, "n_updates"))
|
||||
window = min(10, max(3, len(reward_points)))
|
||||
reward_window = [round(value, 6) for _, value in reward_points[-window:]]
|
||||
best_reward = max((value for _, value in reward_points), default=None)
|
||||
|
||||
notes: list[str] = []
|
||||
if not metrics and not monitor:
|
||||
status, tone = "等待首个 rollout", "gray"
|
||||
notes.append("训练进程启动后先输出环境初始化与首个 rollout 的进度行。")
|
||||
elif not metrics:
|
||||
status, tone = "已收到回合奖励,等待进度行", "warn"
|
||||
notes.append("Monitor CSV 已有回合记录,但尚未收到 --progress-json 进度行;"
|
||||
"首个 rollout 结束后会出现完整指标。")
|
||||
elif len(metrics) < 3:
|
||||
status, tone = "预热阶段", "warn"
|
||||
notes.append("已收到初始指标,需要更多 rollout 才能判断趋势。")
|
||||
else:
|
||||
tone, status = "ok", "收敛趋势良好"
|
||||
# 平台判据(三条同时成立):对半窗口看不出位移、窗口内波动小、
|
||||
# 逐点噪声本身也小。慢速但持续的提升会因第一条被排除。
|
||||
flat = _flatness(reward_points)
|
||||
plateau = False
|
||||
if flat is not None and cv is not None and latest_reward is not None:
|
||||
flat_ratio, flat_noise = flat
|
||||
scale = max(abs(latest_reward), 1e-9)
|
||||
plateau = (flat_ratio <= 1.0 and cv <= 0.15
|
||||
and flat_noise <= 0.02 * scale
|
||||
and (spread is None or spread[1] < 1e-12
|
||||
or spread[0] <= 0.15 * spread[1]))
|
||||
if latest_kl is not None and latest_kl > 0.05:
|
||||
status, tone = "KL 偏大,策略更新可能过猛", "bad"
|
||||
notes.append(f"approx_kl={latest_kl:.4f} > 0.05")
|
||||
elif latest_ev is not None and latest_ev < 0.0:
|
||||
status, tone = "价值函数尚未拟合", "warn"
|
||||
notes.append(f"explained_variance={latest_ev:.3f} < 0")
|
||||
elif reward_slope is not None and reward_slope < 0:
|
||||
status, tone = "奖励未形成上升趋势", "warn"
|
||||
notes.append(f"最近 {window} 点奖励斜率为负({_fmt_num(reward_slope)}/step)")
|
||||
elif plateau:
|
||||
status, tone = "已趋于平稳(疑似收敛)", "ok"
|
||||
notes.append("窗口内对半位移小于逐点噪声,且波动很小"
|
||||
f"(变异系数 {_fmt_num(cv, 3)})")
|
||||
else:
|
||||
notes.append(f"最近 {window} 点奖励斜率为正({_fmt_num(reward_slope)}/step)")
|
||||
if latest_kl is not None and latest_kl <= 0.02:
|
||||
notes.append(f"approx_kl={latest_kl:.4f} 较稳定")
|
||||
if latest_ev is not None:
|
||||
notes.append(f"explained_variance={latest_ev:.3f}")
|
||||
if cv is not None:
|
||||
notes.append(f"奖励变异系数={_fmt_num(cv, 3)}")
|
||||
if latest_entropy is not None and abs(latest_entropy) < 1e-3:
|
||||
notes.append("策略熵接近 0,探索可能过早停止")
|
||||
return {
|
||||
"status": status,
|
||||
"tone": tone,
|
||||
"notes": notes,
|
||||
"window": window,
|
||||
"latest_timesteps": latest_timesteps,
|
||||
"reward_slope": reward_slope,
|
||||
"reward_rate_per_1k": reward_rate,
|
||||
"reward_cv": cv,
|
||||
"reward_spread": spread[0] if spread else None,
|
||||
"reward_window": reward_window,
|
||||
"best_reward": best_reward,
|
||||
"latest_reward": latest_reward,
|
||||
"latest_ep_len": latest_len,
|
||||
"latest_updates": latest_updates,
|
||||
"latest_kl": latest_kl,
|
||||
"latest_explained_variance": latest_ev,
|
||||
"latest_value_loss": latest_vf,
|
||||
"latest_policy_loss": latest_pg,
|
||||
"latest_entropy_loss": latest_entropy,
|
||||
"latest_fps": latest_fps,
|
||||
"reward_points": len(reward_points),
|
||||
}
|
||||
|
||||
|
||||
# 快照状态 -> 界面展示用的进程语义(阶段名称、色调、是否终态)
|
||||
_STATUS_VIEW = {
|
||||
"created": ("已创建", "gray", False, "命令已构造,等待 spawn"),
|
||||
"running": ("运行中", "ok", False, "子进程持有控制台输出与独立进程组"),
|
||||
"stopping": ("停止中", "warn", False, "已发送 SIGTERM,等待子进程退出"),
|
||||
"stopped": ("已手动停止", "warn", True, "收到停止信号后退出;脚本会在 step 边界保存模型与元数据"),
|
||||
"exited": ("正常结束", "ok", True, "退出码 0:训练与评估流程走完"),
|
||||
"failed": ("异常结束", "bad", True, "非零退出码或启动失败,见日志尾部"),
|
||||
}
|
||||
|
||||
_PHASES = [
|
||||
("created", "构造命令"),
|
||||
("spawn", "启动子进程"),
|
||||
("rollout", "采集 rollout"),
|
||||
("update", "PPO 更新"),
|
||||
("evaluate", "评估 / 保存"),
|
||||
("done", "收敛归档"),
|
||||
]
|
||||
|
||||
# 状态 -> 当前所处的阶段节点;节点之前为已完成,之后为未开始。
|
||||
_STATUS_PHASE = {
|
||||
"created": "created",
|
||||
"running": "update",
|
||||
"stopping": "update",
|
||||
"stopped": "done",
|
||||
"exited": "done",
|
||||
"failed": "done",
|
||||
}
|
||||
|
||||
|
||||
def _pid(process: subprocess.Popen | None) -> int | None:
|
||||
"""已退出进程的 pid 访问会抛 OSError;这里统一吞掉,只用于展示。"""
|
||||
if process is None:
|
||||
return None
|
||||
try:
|
||||
return int(process.pid)
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def describe_process(snapshot: dict[str, Any]) -> dict[str, Any]:
|
||||
"""把 snapshot 派生为界面直接可渲染的进程对象。
|
||||
|
||||
包含:状态语义词、进度(步/百分比)、墙钟与剩余时间估计、最近吞吐、
|
||||
进程时间线节点。数字都取自 snapshot,前端不再重复累加,避免两个刷新
|
||||
频率不同的计数器互相打架。
|
||||
"""
|
||||
status = str(snapshot.get("status") or "created")
|
||||
label, tone, terminal, hint = _STATUS_VIEW.get(
|
||||
status, (status, "gray", False, ""))
|
||||
config = snapshot.get("config") or {}
|
||||
order = [key for key, _ in _PHASES]
|
||||
current = _STATUS_PHASE.get(status, "created")
|
||||
timeline = []
|
||||
for key, title in _PHASES:
|
||||
index = order.index(key)
|
||||
if index < order.index(current):
|
||||
state = "done"
|
||||
elif index == order.index(current):
|
||||
state = "active"
|
||||
else:
|
||||
state = "pending"
|
||||
timeline.append({"key": key, "label": title, "state": state})
|
||||
|
||||
metrics = snapshot.get("metrics") or []
|
||||
latest: dict[str, Any] = metrics[-1] if metrics else {}
|
||||
convergence = snapshot.get("convergence") or {}
|
||||
total_steps = _finite_float(config.get("timesteps")) or 0.0
|
||||
# SB3 表格行可能没有 steps 字段,因此进度取"所有进度点的最大步数",
|
||||
# 而不是只看最后一个点——否则末行恰好缺字段时进度会回退到 0。
|
||||
done_steps = 0.0
|
||||
for point in metrics:
|
||||
value = (_finite_float(point.get("timesteps"))
|
||||
or _finite_float(point.get("total_timesteps")))
|
||||
if value is not None and value > done_steps:
|
||||
done_steps = value
|
||||
if done_steps == 0.0:
|
||||
done_steps = _finite_float(convergence.get("latest_timesteps")) or 0.0
|
||||
# init/resume(课程迁移)时 num_timesteps 从上一阶段继续,进度必须按本次
|
||||
# 增量算;start_timesteps 由训练脚本的进度 JSON 提供,缺省按 0 处理。
|
||||
start_steps = 0.0
|
||||
for point in metrics:
|
||||
value = _finite_float(point.get("start_timesteps"))
|
||||
if value is not None:
|
||||
start_steps = value
|
||||
break
|
||||
trained_steps = max(0.0, done_steps - start_steps)
|
||||
percent = (100.0 * trained_steps / total_steps) if total_steps > 0 else None
|
||||
if percent is not None:
|
||||
percent = max(0.0, min(100.0, percent))
|
||||
elapsed = float(snapshot.get("elapsed_s") or 0.0)
|
||||
rate = _latest_rate(metrics)
|
||||
remaining: float | None = None
|
||||
eta_ts: float | None = None
|
||||
if rate and rate > 0 and total_steps > 0 and trained_steps < total_steps:
|
||||
remaining = (total_steps - trained_steps) / rate
|
||||
eta_ts = float(snapshot.get("finished_at") or time.time()) + remaining
|
||||
if not terminal and remaining is None:
|
||||
cap = _finite_float(config.get("max_wall_s")) or 0.0
|
||||
if cap > 0:
|
||||
remaining = max(0.0, cap - elapsed)
|
||||
return {
|
||||
"label": label,
|
||||
"tone": tone,
|
||||
"terminal": terminal,
|
||||
"hint": hint,
|
||||
"phase": current,
|
||||
"timeline": timeline,
|
||||
"percent": percent,
|
||||
"done_steps": done_steps,
|
||||
"start_steps": start_steps,
|
||||
"trained_steps": trained_steps,
|
||||
"total_steps": total_steps,
|
||||
"elapsed_s": round(elapsed, 3),
|
||||
"rate_per_s": rate,
|
||||
"remaining_s": remaining,
|
||||
"eta_epoch_s": eta_ts,
|
||||
"log_total": snapshot.get("log_total"),
|
||||
"log_capacity": MAX_LOG_LINES,
|
||||
"metric_total": snapshot.get("metric_total"),
|
||||
"latest": latest,
|
||||
}
|
||||
|
||||
|
||||
def _latest_rate(metrics: list[dict[str, Any]], window: int = 6) -> float | None:
|
||||
"""最近若干个**带 fps 的**进度点的平均吞吐(步/秒),用于剩余时间估计。
|
||||
|
||||
progress JSON 行不含 fps,只有 SB3 表格行有;若只看最后 window 个点,
|
||||
恰好落在若干 JSON 行之间时会得到 None,剩余时间就会退化成墙钟上限的
|
||||
粗糙估计。这里向前扫描直到拿到至少 1 个 fps(最多看 3*window 个点)。
|
||||
"""
|
||||
if not metrics:
|
||||
return None
|
||||
usable: list[float] = []
|
||||
for point in reversed(metrics[-window * 3:]):
|
||||
value = _finite_float(point.get("fps"))
|
||||
if value and value > 0:
|
||||
usable.append(value)
|
||||
if len(usable) >= window:
|
||||
break
|
||||
if not usable:
|
||||
return None
|
||||
return sum(usable) / len(usable)
|
||||
|
||||
|
||||
class TrainingUI:
|
||||
"""训练前端管理器:持有唯一 TrainingJob 与 HTTP 服务器。"""
|
||||
|
||||
@@ -901,9 +770,9 @@ class TrainingUI:
|
||||
config = TrainConfig.from_payload(payload)
|
||||
with self._lock:
|
||||
if self.job is not None and self.job.status in (
|
||||
"created", "running", "stopping"):
|
||||
"created", "running", "paused", "stopping"):
|
||||
raise RuntimeError("已有训练任务在运行,请先停止")
|
||||
self.job = TrainingJob(config)
|
||||
self.job = TrainingJob(config, control_dir=CONTROL_DIR)
|
||||
try:
|
||||
self.job.start()
|
||||
except Exception:
|
||||
@@ -913,6 +782,14 @@ class TrainingUI:
|
||||
# 与 snapshot() 同形:HTTP 层直接把返回值当响应体,不再二次拆包
|
||||
return {"job": self.job.snapshot(log_tail=1)}
|
||||
|
||||
def pause(self) -> bool:
|
||||
with self._lock:
|
||||
return bool(self.job and self.job.pause())
|
||||
|
||||
def resume(self) -> bool:
|
||||
with self._lock:
|
||||
return bool(self.job and self.job.resume())
|
||||
|
||||
def stop(self) -> bool:
|
||||
with self._lock:
|
||||
return bool(self.job and self.job.stop())
|
||||
@@ -938,7 +815,10 @@ class TrainingUI:
|
||||
self._serving = False
|
||||
|
||||
def shutdown(self) -> None:
|
||||
self.stop()
|
||||
job = self.job
|
||||
if job is not None and job.stop():
|
||||
# 留出收尾落盘的时间,避免前端先退出把训练子进程留成孤儿
|
||||
job.wait_for_exit(timeout=STOP_GRACE_S + 5.0)
|
||||
if self._serving:
|
||||
self.httpd.shutdown()
|
||||
self.httpd.server_close()
|
||||
@@ -948,8 +828,8 @@ class _UIHandler(BaseHTTPRequestHandler):
|
||||
protocol_version = "HTTP/1.1"
|
||||
server_version = "MCM26B-Training-UI/1.0"
|
||||
|
||||
def log_message(self, format, *args): # noqa: A002
|
||||
pass
|
||||
def log_message(self, format, *args): # noqa: A002 # pylint: disable=redefined-builtin
|
||||
"""覆写 stdlib 签名以吞掉访问日志;参数名沿用 ``format`` 不能改。"""
|
||||
|
||||
@property
|
||||
def ui(self) -> TrainingUI:
|
||||
@@ -999,6 +879,10 @@ class _UIHandler(BaseHTTPRequestHandler):
|
||||
try:
|
||||
if parts.path == "/api/train/start":
|
||||
self._send_json(200, self.ui.start(payload))
|
||||
elif parts.path == "/api/train/pause":
|
||||
self._send_json(200, {"paused": self.ui.pause()})
|
||||
elif parts.path == "/api/train/resume":
|
||||
self._send_json(200, {"resumed": self.ui.resume()})
|
||||
elif parts.path == "/api/train/stop":
|
||||
self._send_json(200, {"stopped": self.ui.stop()})
|
||||
else:
|
||||
|
||||
@@ -23,6 +23,8 @@ const state = {
|
||||
chartKey: "ep_rew_mean",
|
||||
jobKey: null, // job_id 变化时清空本地缓存
|
||||
lastLogSignature: "",
|
||||
lastStatus: null, // 最近一次快照的状态:决定暂停按钮是"暂停"还是"继续"
|
||||
lastJob: null, // 最近一次快照:恢复按钮需要读 job.recovery
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------- 工具
|
||||
@@ -285,9 +287,11 @@ function absorbMetrics(job) {
|
||||
// ---------------------------------------------------------------- 渲染
|
||||
function render(job) {
|
||||
if (!job) {
|
||||
state.lastJob = null;
|
||||
renderIdle();
|
||||
return;
|
||||
}
|
||||
state.lastJob = job;
|
||||
if (job.job_id !== state.jobKey) {
|
||||
state.jobKey = job.job_id;
|
||||
state.metrics = [];
|
||||
@@ -302,6 +306,7 @@ function render(job) {
|
||||
renderLog(job);
|
||||
renderEpisodes(job);
|
||||
renderBanner(job);
|
||||
renderRecovery(job);
|
||||
renderJobLine(job);
|
||||
}
|
||||
|
||||
@@ -319,6 +324,10 @@ function renderIdle() {
|
||||
setText("stat-logs", "0");
|
||||
$("timeline").innerHTML = "";
|
||||
setText("command-preview", "");
|
||||
state.lastStatus = null;
|
||||
const pauseBtn = $("btn-pause");
|
||||
pauseBtn.disabled = true;
|
||||
pauseBtn.textContent = "暂停训练";
|
||||
chip("chip-converge", "判读", "等待数据", "gray");
|
||||
$("converge-notes").innerHTML = "";
|
||||
$("metric-grid").innerHTML = '<p class="hint">尚未启动训练,暂无参数。</p>';
|
||||
@@ -370,8 +379,14 @@ function renderProcess(job) {
|
||||
|
||||
renderTimeline(process);
|
||||
setText("command-preview", job.command ? `$ ${job.command.join(" ")}` : "");
|
||||
setFormLocked(job.status === "running" || job.status === "created" || job.status === "stopping");
|
||||
$("btn-stop").disabled = !(job.status === "running" || job.status === "created");
|
||||
state.lastStatus = job.status;
|
||||
setFormLocked(job.status === "running" || job.status === "created"
|
||||
|| job.status === "paused" || job.status === "stopping");
|
||||
$("btn-stop").disabled = !(job.status === "running" || job.status === "created"
|
||||
|| job.status === "paused");
|
||||
const pauseBtn = $("btn-pause");
|
||||
pauseBtn.disabled = !(job.status === "running" || job.status === "paused");
|
||||
pauseBtn.textContent = job.status === "paused" ? "继续训练" : "暂停训练";
|
||||
}
|
||||
|
||||
function renderTimeline(process) {
|
||||
@@ -427,8 +442,12 @@ function renderBanner(job) {
|
||||
}
|
||||
if (job.status === "failed") {
|
||||
showBanner("bad", `训练异常结束(退出码 <b>${job.returncode === null ? "?" : job.returncode}</b>):${escapeHtml(job.error || "见日志尾部")}。`);
|
||||
} else if (job.status === "paused") {
|
||||
showBanner("warn", "训练已暂停:进程与显存仍在,但停在 step 边界不再推进。点「继续训练」从暂停处恢复,或点「停止训练」保存当前模型后退出。");
|
||||
} else if (job.status === "stopped") {
|
||||
showBanner("warn", "已按请求停止训练(SIGTERM 独立进程组),脚本会在 step 边界保存当前模型与元数据。");
|
||||
showBanner("warn", job.forced_kill
|
||||
? "停止请求超时,已强制结束进程树;最后一段模型可能未保存,请检查 checkpoint 目录。"
|
||||
: "已按请求停止训练(控制文件协作式停止),脚本已在 step 边界保存当前模型与元数据。");
|
||||
} else if (job.status === "exited") {
|
||||
showBanner("ok", "训练与评估流程正常结束,模型与元数据已写入输出目录。");
|
||||
} else if (process.remaining_s !== null && process.remaining_s !== undefined
|
||||
@@ -439,6 +458,26 @@ function renderBanner(job) {
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- 恢复
|
||||
function renderRecovery(job) {
|
||||
const recovery = job.recovery || {};
|
||||
const box = $("recovery-box");
|
||||
if (!box) return;
|
||||
if (!recovery.model || recovery.reason === "none") {
|
||||
box.classList.add("hidden");
|
||||
return;
|
||||
}
|
||||
box.classList.remove("hidden");
|
||||
const steps = recovery.timesteps === null || recovery.timesteps === undefined
|
||||
? "步数未知" : `${fmtInt(recovery.timesteps)} 步`;
|
||||
const when = recovery.updated_unix_s ? fmtClock(recovery.updated_unix_s) : "—";
|
||||
setText("recovery-text",
|
||||
`可恢复产物:${steps} · 来源 ${recovery.reason || "?"} · 更新 ${when} · ${recovery.model}`);
|
||||
const button = $("btn-recover");
|
||||
// 离线预览里的恢复信息是合成的,不允许真的去续训
|
||||
button.disabled = !recovery.can_resume || Boolean(state.preview);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- 收敛
|
||||
function renderConvergence(job) {
|
||||
const convergence = job.convergence || {};
|
||||
@@ -679,6 +718,33 @@ function renderEpisodes(job) {
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- 交互
|
||||
async function submitPayload(payload) {
|
||||
const err = $("form-error");
|
||||
err.textContent = "";
|
||||
if (state.saving) return false; // 训练中表单锁定,续训按钮同样不许重复提交
|
||||
if (payload.resume && !payload.model) {
|
||||
err.textContent = "续训需要填写模型输出路径";
|
||||
return false;
|
||||
}
|
||||
const res = await fetch("/api/train/start", {
|
||||
method: "POST", headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(payload),
|
||||
});
|
||||
const body = await res.json().catch(() => ({}));
|
||||
if (!res.ok) {
|
||||
err.textContent = body.error || `启动失败(HTTP ${res.status})`;
|
||||
return false;
|
||||
}
|
||||
state.preview = false;
|
||||
state.jobKey = null;
|
||||
state.metrics = [];
|
||||
state.metricCache = [];
|
||||
state.metricNext = 0;
|
||||
state.metricTotal = 0;
|
||||
await poll();
|
||||
return true;
|
||||
}
|
||||
|
||||
$("config-form").addEventListener("submit", async (event) => {
|
||||
event.preventDefault();
|
||||
if (state.saving) return;
|
||||
@@ -691,24 +757,43 @@ $("config-form").addEventListener("submit", async (event) => {
|
||||
for (const [key, value] of Object.entries(values)) {
|
||||
if (value !== null) payload[key] = value;
|
||||
}
|
||||
if (payload.resume && !payload.model) { err.textContent = "续训需要填写模型输出路径"; return; }
|
||||
const res = await fetch("/api/train/start", {
|
||||
method: "POST", headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(payload),
|
||||
await submitPayload(payload);
|
||||
});
|
||||
|
||||
$("btn-recover").addEventListener("click", async () => {
|
||||
const recovery = (state.lastJob && state.lastJob.recovery) || {};
|
||||
if (!recovery.can_resume || !recovery.model) return;
|
||||
if (!confirm(`用最近的可恢复产物续训?\n${recovery.model}\n`
|
||||
+ "当前表单里的 timesteps 会作为**追加**步数。")) return;
|
||||
// 同步回表单,用户随后可微调步数再手动点「开始训练」
|
||||
const modelField = $("cfg-model");
|
||||
if (modelField) modelField.value = recovery.model;
|
||||
const resumeField = $("cfg-resume");
|
||||
if (resumeField) resumeField.checked = true;
|
||||
const values = collectValues();
|
||||
values.model = recovery.model;
|
||||
values.resume = true;
|
||||
values.force = false; // --resume 与 --force 互斥
|
||||
await submitPayload(values);
|
||||
});
|
||||
|
||||
$("btn-pause").addEventListener("click", async () => {
|
||||
const resuming = state.lastStatus === "paused";
|
||||
const res = await fetch(resuming ? "/api/train/resume" : "/api/train/pause", {
|
||||
method: "POST", headers: { "Content-Type": "application/json" }, body: "{}",
|
||||
});
|
||||
const body = await res.json().catch(() => ({}));
|
||||
if (!res.ok) { err.textContent = body.error || `启动失败(HTTP ${res.status})`; return; }
|
||||
state.preview = false;
|
||||
state.jobKey = null;
|
||||
state.metrics = [];
|
||||
state.metricCache = [];
|
||||
state.metricNext = 0;
|
||||
state.metricTotal = 0;
|
||||
if (!res.ok) {
|
||||
showBanner("bad", `${resuming ? "继续" : "暂停"}失败:${escapeHtml(body.error || res.status)}`);
|
||||
} else if (!resuming && body.paused === false) {
|
||||
showBanner("warn", "当前没有可暂停的训练进程(可能已结束)。");
|
||||
}
|
||||
await poll();
|
||||
});
|
||||
|
||||
$("btn-stop").addEventListener("click", async () => {
|
||||
if (!confirm("确认停止当前训练?\n将向独立进程组发送 SIGTERM,脚本会先保存当前模型与元数据再退出。")) return;
|
||||
if (!confirm("确认停止当前训练?\n将通过控制文件请求优雅停止,脚本先保存当前模型与元数据再退出;"
|
||||
+ "若超时未响应则强制结束进程树。")) return;
|
||||
const res = await fetch("/api/train/stop", {
|
||||
method: "POST", headers: { "Content-Type": "application/json" }, body: "{}",
|
||||
});
|
||||
|
||||
@@ -30,6 +30,7 @@
|
||||
</select>
|
||||
</label>
|
||||
<button id="btn-preview" type="button" title="渲染示例快照,不启动任何训练进程">离线预览</button>
|
||||
<button id="btn-pause" type="button" disabled>暂停训练</button>
|
||||
<button id="btn-stop" class="danger" type="button" disabled>停止训练</button>
|
||||
</div>
|
||||
</header>
|
||||
@@ -37,6 +38,11 @@
|
||||
<main>
|
||||
<div id="banner" class="banner hidden"></div>
|
||||
|
||||
<div id="recovery-box" class="recovery hidden">
|
||||
<span class="hint" id="recovery-text">—</span>
|
||||
<button id="btn-recover" type="button" title="把最近一次完整落盘的产物作为 --resume 目标继续训练">用最近检查点续训</button>
|
||||
</div>
|
||||
|
||||
<!-- =============================================== 1. 执行进度与进程 -->
|
||||
<section class="panel" id="panel-process">
|
||||
<div class="panel-head">
|
||||
|
||||
@@ -64,6 +64,15 @@ button.danger:hover:enabled { background: rgba(248,113,113,.12); }
|
||||
.banner.warn { border-color: var(--warn); border-left-color: var(--warn); background: rgba(245,158,11,.08); }
|
||||
.banner b { font-family: var(--mono); }
|
||||
|
||||
/* 恢复条:显示最近一份完整落盘的产物,并提供一键续训 */
|
||||
.recovery {
|
||||
display: flex; align-items: center; justify-content: space-between; gap: 12px;
|
||||
border: 1px solid var(--cyan, #22d3ee); border-left: 4px solid var(--cyan, #22d3ee);
|
||||
background: rgba(34,211,238,.07); border-radius: 8px;
|
||||
padding: 8px 12px; margin: 10px 0 0; font-size: 13px;
|
||||
}
|
||||
.recovery .hint { overflow-wrap: anywhere; }
|
||||
|
||||
/* ------------------------------------------------------------ 面板 */
|
||||
.panel {
|
||||
border: 1px solid var(--line); border-radius: 12px; background: var(--panel);
|
||||
|
||||
@@ -62,7 +62,7 @@ class BayesLoopAcceptanceTests(unittest.TestCase):
|
||||
|
||||
def test_sixteen_sources_stop_by_count_bound(self):
|
||||
sources = random_omni_scene(seed=6, count=16, config=CONFIG)
|
||||
summary, report, simulator = _run_scene(sources)
|
||||
summary, report, _ = _run_scene(sources)
|
||||
truth = sorted(s.channel for s in sources)
|
||||
self.assertTrue(summary["completed"])
|
||||
self.assertEqual(summary["stop_reason"], "sixteen_cleared")
|
||||
@@ -72,7 +72,7 @@ class BayesLoopAcceptanceTests(unittest.TestCase):
|
||||
|
||||
def test_empty_arena_all_absent(self):
|
||||
"""0 源:每个频道都必须由 PF 排除盘证明缺席后正常退出。"""
|
||||
summary, report, simulator = _run_scene([], config=NO_PERSIST)
|
||||
summary, report, _ = _run_scene([], config=NO_PERSIST)
|
||||
self.assertTrue(summary["completed"])
|
||||
self.assertEqual(summary["stop_reason"], "all_resolved")
|
||||
self.assertEqual(sorted(summary["absent_channels"]),
|
||||
@@ -89,7 +89,7 @@ class BayesLoopAcceptanceTests(unittest.TestCase):
|
||||
sources = [Source(c + 1, 1800.0 * math.cos(math.pi / 6 + 2 * math.pi * c / 4),
|
||||
1800.0 * math.sin(math.pi / 6 + 2 * math.pi * c / 4),
|
||||
1000.0) for c in range(4)]
|
||||
summary, report, simulator = _run_scene(sources, config=NO_PERSIST)
|
||||
summary, report, _ = _run_scene(sources, config=NO_PERSIST)
|
||||
truth = sorted(s.channel for s in sources)
|
||||
self.assertTrue(summary["completed"])
|
||||
self.assertEqual(sorted(summary["cleared_channels"]), truth)
|
||||
@@ -112,7 +112,7 @@ class BayesLoopAcceptanceTests(unittest.TestCase):
|
||||
def test_transport_fault_retries_keep_session_clean(self):
|
||||
sources = random_omni_scene(seed=5, count=10, config=CONFIG)
|
||||
script = [("drop_before", 1)] # 首个请求丢弃一次:幂等重试路径
|
||||
summary, report, simulator = _run_scene(sources, fault_script=script)
|
||||
summary, report, _ = _run_scene(sources, fault_script=script)
|
||||
truth = sorted(s.channel for s in sources)
|
||||
self.assertTrue(summary["completed"])
|
||||
self.assertEqual(sorted(summary["cleared_channels"]), truth)
|
||||
@@ -141,9 +141,9 @@ class BayesLoopAcceptanceTests(unittest.TestCase):
|
||||
|
||||
def test_cli_strategy_registry(self):
|
||||
"""CLI --strategy 注册表:bayes/seven 用七站,grid 用 49 点网格。"""
|
||||
import mcm26b.cli as cli
|
||||
from mcm26b import cli
|
||||
from mcm26b.coverage import coverage_points
|
||||
from mcm26b.planner.seven import SevenBatch, seven_points
|
||||
from mcm26b.planner.seven import seven_points
|
||||
self.assertIs(cli.Q3_STRATEGIES["bayes"], BayesLoop)
|
||||
for name, loop_cls in cli.Q3_STRATEGIES.items():
|
||||
simulator = LocalSimulator([], robot_id="T001", config=CONFIG)
|
||||
|
||||
@@ -132,7 +132,7 @@ class BeliefDirectionTests(TestCase):
|
||||
def test_near_shrinks_to_five_meters(self):
|
||||
belief = BeliefState(CONFIG)
|
||||
belief.update_measure(2, (100.0, 100.0), NEAR, None)
|
||||
center, radius = belief.candidate_set(2).mec()
|
||||
_, radius = belief.candidate_set(2).mec()
|
||||
self.assertLessEqual(radius, 5.1)
|
||||
self.assertTrue(belief.candidate_set(2).contains((101.0, 101.0)))
|
||||
self.assertEqual(belief.channels[2].status, ChannelStatus.DETECTED)
|
||||
|
||||
@@ -16,7 +16,8 @@ class GeometryTests(unittest.TestCase):
|
||||
self.assertEqual(result['status'], 'polygon')
|
||||
self.assertEqual(len(result['vertices']), 3)
|
||||
self.assertAlmostEqual(result['diameter_m'], 40, places=7)
|
||||
self.assertAlmostEqual(result['minimum_enclosing_circle']['radius_m'], 40/math.sqrt(3), places=7)
|
||||
self.assertAlmostEqual(result['minimum_enclosing_circle']['radius_m'],
|
||||
40/math.sqrt(3), places=7)
|
||||
self.assertFalse(result['diameter_circle']['covers'])
|
||||
self.assertFalse(result['guaranteed_clearable_at_mec_center'])
|
||||
|
||||
@@ -84,7 +85,7 @@ class GeometryTests(unittest.TestCase):
|
||||
c, r = smallest_enclosing_circle([(0,0),(1,0),(3,0)])
|
||||
self.assertAlmostEqual(r, 1.5)
|
||||
|
||||
def test_random_bounded_truth_containment_and_monotonicity(self):
|
||||
def test_random_truth_containment_and_monotonic(self):
|
||||
rng = random.Random(20260910)
|
||||
for _ in range(100):
|
||||
target = (rng.uniform(-300,300), rng.uniform(-300,300))
|
||||
|
||||
@@ -14,7 +14,7 @@ ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
class CertificateTests(unittest.TestCase):
|
||||
def test_incremental_circle_against_exhaustive_reference(self):
|
||||
def test_incremental_circle_matches_reference(self):
|
||||
rng = random.Random(271828)
|
||||
for _ in range(250):
|
||||
points = [(rng.uniform(-20, 20), rng.uniform(-20, 20))
|
||||
@@ -25,7 +25,7 @@ class CertificateTests(unittest.TestCase):
|
||||
self.assertLess(distance(center, expected_c), 1e-6)
|
||||
self.assertTrue(all(distance(center, p) <= radius for p in points))
|
||||
|
||||
def test_circle_large_cocircular_duplicate_and_collinear_sets(self):
|
||||
def test_circle_cocircular_duplicate_and_collinear(self):
|
||||
# More than Python's recursion limit, with analytically known answers.
|
||||
points = [(1e6+30*math.cos(i*math.tau/1500),
|
||||
-2e6+30*math.sin(i*math.tau/1500)) for i in range(1500)]
|
||||
@@ -37,15 +37,16 @@ class CertificateTests(unittest.TestCase):
|
||||
self.assertEqual(c, (749.5, 1499))
|
||||
self.assertEqual(smallest_enclosing_circle([(3, 4)]*10), ((3, 4), 0))
|
||||
|
||||
def test_witness_and_clearance_margin_from_saved_observations(self):
|
||||
def test_witness_and_clearance_margin_from_saved(self):
|
||||
path = next((ROOT/'output/q1/practice').glob('*_observations.json'))
|
||||
rows = json.loads(path.read_text(encoding='utf-8'))['measurements']
|
||||
result = solve([Measurement(**m) for m in rows])
|
||||
circle = result['diameter_circle']
|
||||
self.assertFalse(circle['covers'])
|
||||
self.assertIn(circle['witness_vertex'], result['vertices'])
|
||||
self.assertAlmostEqual(distance(circle['witness_vertex'], circle['center'])-circle['radius_m'],
|
||||
circle['max_excess_m'])
|
||||
self.assertAlmostEqual(
|
||||
distance(circle['witness_vertex'], circle['center'])-circle['radius_m'],
|
||||
circle['max_excess_m'])
|
||||
for i in circle['outside_vertex_indices']:
|
||||
self.assertGreater(distance(result['vertices'][i], circle['center']),
|
||||
circle['radius_m']+circle['coverage_tolerance_m'])
|
||||
@@ -61,7 +62,7 @@ class CertificateTests(unittest.TestCase):
|
||||
self.assertTrue(r['diameter_circle']['covers'])
|
||||
self.assertIsNone(r['diameter_circle']['witness_vertex'])
|
||||
|
||||
def test_first_sector_disk_covers_all_angles_and_distances(self):
|
||||
def test_first_sector_disk_covers_all_angles(self):
|
||||
for angle in [0, 359.9, -180]:
|
||||
for delta in [0, 1, 1.005, 45]:
|
||||
m = Measurement(-800, 1200, angle, delta)
|
||||
@@ -72,11 +73,15 @@ class CertificateTests(unittest.TestCase):
|
||||
for k in range(11):
|
||||
phi = math.radians(angle-delta+2*delta*j/20)
|
||||
p = (m.x+150*k*math.cos(phi), m.y+150*k*math.sin(phi))
|
||||
self.assertLessEqual(distance(p, enclosing['center']), enclosing['radius_m']+1e-9)
|
||||
self.assertLessEqual(
|
||||
distance(p, enclosing['center']),
|
||||
enclosing['radius_m']+1e-9)
|
||||
if candidate:
|
||||
for t in range(8):
|
||||
q = (candidate['center'][0]+candidate['radius_m']*math.cos(t*math.tau/8),
|
||||
candidate['center'][1]+candidate['radius_m']*math.sin(t*math.tau/8))
|
||||
q = (candidate['center'][0]
|
||||
+ candidate['radius_m']*math.cos(t*math.tau/8),
|
||||
candidate['center'][1]
|
||||
+ candidate['radius_m']*math.sin(t*math.tau/8))
|
||||
self.assertLessEqual(distance(p, q), 1000+1e-9)
|
||||
if delta <= 1.005:
|
||||
self.assertTrue(cert['candidate_nonempty_certified'])
|
||||
@@ -100,7 +105,8 @@ class CertificateTests(unittest.TestCase):
|
||||
self.assertEqual(outputs[0]['error_policy']['effective_half_angles_deg'], [1, 1, 1])
|
||||
self.assertEqual(outputs[1]['error_policy']['effective_half_angles_deg'], [1.005]*3)
|
||||
self.assertGreater(outputs[1]['diameter_m'], outputs[0]['diameter_m'])
|
||||
bad = subprocess.run(base+['--rounding-margin-deg', '-1'], cwd=ROOT, capture_output=True)
|
||||
bad = subprocess.run(base+['--rounding-margin-deg', '-1'], cwd=ROOT,
|
||||
capture_output=True, check=False)
|
||||
self.assertNotEqual(bad.returncode, 0)
|
||||
|
||||
|
||||
|
||||
+14
-14
@@ -90,7 +90,7 @@ class LayoutTests(unittest.TestCase):
|
||||
again = env._loop.hrl_observation(env._current_candidates)
|
||||
self.assertTrue(numpy.array_equal(obs, again))
|
||||
|
||||
def test_mask_admits_at_least_one_action_and_aligns_with_candidates(self):
|
||||
def test_mask_nonempty_and_matches_candidate_slots(self):
|
||||
env = _env(problem=3)
|
||||
env.reset(seed=7)
|
||||
for _ in range(6):
|
||||
@@ -98,7 +98,7 @@ class LayoutTests(unittest.TestCase):
|
||||
self.assertTrue(mask.any())
|
||||
slots = {cand.slot for cand in env._current_candidates}
|
||||
self.assertEqual({int(i) for i in numpy.flatnonzero(mask)}, slots)
|
||||
obs, _r, term, trunc, _info = env.step(
|
||||
_obs, _r, term, trunc, _info = env.step(
|
||||
int(env._loop.heuristic_action(env._current_candidates)))
|
||||
if term or trunc:
|
||||
break
|
||||
@@ -114,7 +114,7 @@ class LayoutTests(unittest.TestCase):
|
||||
self.assertEqual(FALLBACK_OFFSET, STATION_OFFSET + STATION_SLOTS)
|
||||
self.assertEqual(N_CANDIDATES - (FALLBACK_OFFSET + FALLBACK_SLOTS), 11)
|
||||
|
||||
def test_build_candidates_keeps_clear_17_to_20_and_locate_1_to_4(self):
|
||||
def test_build_candidates_keeps_clear_and_locate(self):
|
||||
host = _CandidateHost(detected=[1, 2, 3, 4, 17, 18, 19, 20],
|
||||
certified=[17, 18, 19, 20])
|
||||
candidates = host.build_candidates()
|
||||
@@ -130,7 +130,7 @@ class LayoutTests(unittest.TestCase):
|
||||
selected = host.candidate_by_slot(candidates, CLEAR_OFFSET + channel - 1)
|
||||
self.assertEqual((selected.kind, selected.channel), ("clear", channel))
|
||||
|
||||
def test_build_candidates_fallback_includes_channel_20_within_its_slots(self):
|
||||
def test_build_candidates_fallback_covers_slots(self):
|
||||
channels = list(range(17, 21))
|
||||
host = _CandidateHost(detected=channels, blocked=channels)
|
||||
candidates = host.build_candidates()
|
||||
@@ -151,11 +151,11 @@ class LayoutTests(unittest.TestCase):
|
||||
class HeuristicLoopTests(unittest.TestCase):
|
||||
def test_q3_full_episode_clears_all_sources(self):
|
||||
env = _env(problem=3, stage_index=3, max_episode_steps=500)
|
||||
obs, _ = env.reset(seed=11)
|
||||
_, _ = env.reset(seed=11)
|
||||
steps = 0
|
||||
while True:
|
||||
action = env._loop.heuristic_action(env._current_candidates)
|
||||
obs, r, term, trunc, info = env.step(action)
|
||||
_, _, term, trunc, info = env.step(action)
|
||||
steps += 1
|
||||
if term or trunc:
|
||||
break
|
||||
@@ -170,7 +170,7 @@ class HeuristicLoopTests(unittest.TestCase):
|
||||
env.reset(seed=21)
|
||||
while True:
|
||||
action = env._loop.heuristic_action(env._current_candidates)
|
||||
obs, r, term, trunc, info = env.step(action)
|
||||
_, _, term, trunc, info = env.step(action)
|
||||
if term or trunc:
|
||||
break
|
||||
self.assertTrue(term)
|
||||
@@ -185,8 +185,8 @@ class HeuristicLoopTests(unittest.TestCase):
|
||||
reward_cfg = env.reward
|
||||
for _ in range(8):
|
||||
action = env._loop.heuristic_action(env._current_candidates)
|
||||
virtual_before = env._client.state.virtual_time_s
|
||||
obs, r, term, trunc, info = env.step(action)
|
||||
_virtual_before = env._client.state.virtual_time_s
|
||||
_, r, term, trunc, info = env.step(action)
|
||||
outcome = env._last_outcome
|
||||
if outcome.invalid:
|
||||
self.assertAlmostEqual(r, -reward_cfg.invalid, places=6)
|
||||
@@ -206,13 +206,13 @@ class HeuristicLoopTests(unittest.TestCase):
|
||||
if term or trunc:
|
||||
break
|
||||
|
||||
def test_invalid_action_penalized_without_simulator_interaction(self):
|
||||
def test_invalid_action_penalized_without_sim_call(self):
|
||||
env = _env(problem=3, stage_index=0, max_episode_steps=50)
|
||||
env.reset(seed=3)
|
||||
mask = env.action_masks()
|
||||
invalid_action = int(numpy.argmin(mask))
|
||||
virtual_before = env._client.state.virtual_time_s
|
||||
obs, r, term, trunc, info = env.step(invalid_action)
|
||||
_, r, _, _, _ = env.step(invalid_action)
|
||||
self.assertAlmostEqual(r, -env.reward.invalid, places=6)
|
||||
self.assertAlmostEqual(env._client.state.virtual_time_s,
|
||||
virtual_before, places=6)
|
||||
@@ -228,7 +228,7 @@ class LowLevelTests(unittest.TestCase):
|
||||
goal = tuple(info["goal"])
|
||||
total = 0
|
||||
for _ in range(60):
|
||||
obs, r, term, trunc, info = env.step(env.expert_action())
|
||||
obs, _, term, trunc, _ = env.step(env.expert_action())
|
||||
total += 1
|
||||
if term or trunc:
|
||||
break
|
||||
@@ -237,7 +237,7 @@ class LowLevelTests(unittest.TestCase):
|
||||
efficiency = straight / (total * env.step_m)
|
||||
self.assertGreater(efficiency, 0.7, "专家路径效率应接近直线")
|
||||
|
||||
def test_executor_straightline_is_goal_only_and_policy_path_snaps(self):
|
||||
def test_executor_straightline_is_goal_only(self):
|
||||
executor = LowLevelExecutor(policy=None)
|
||||
self.assertEqual(executor.roll((0.0, 0.0), (500.0, 200.0)),
|
||||
[(500.0, 200.0)])
|
||||
@@ -257,7 +257,7 @@ class CurriculumTests(unittest.TestCase):
|
||||
self.assertLessEqual(prev.bearing_error_half_deg,
|
||||
nxt.bearing_error_half_deg)
|
||||
|
||||
def test_training_config_defaults_to_production_fidelity(self):
|
||||
def test_training_config_defaults_to_production(self):
|
||||
config = training_config(CONFIG, STAGES[3])
|
||||
self.assertEqual(config.field.channel_max, 20)
|
||||
self.assertEqual(config.field.bearing_error_half_deg, 1.0)
|
||||
|
||||
+244
-16
@@ -1,10 +1,13 @@
|
||||
"""高层训练落盘保证:checkpoint、SIGTERM 优雅保存、worker 信号屏蔽。
|
||||
"""高层训练落盘保证:checkpoint、控制文件优雅停止、信号兜底、worker 信号屏蔽。
|
||||
|
||||
背景:训练前端的「停止」向整个进程组发 SIGTERM,而旧脚本只在 ``learn()``
|
||||
正常返回后才保存,导致"停一下就全丢"。这里把三条保证固定成回归测试:
|
||||
背景:训练前端的「停止」以前直接向进程组发 SIGTERM,而旧脚本只在 ``learn()``
|
||||
正常返回后才保存,导致"停一下就全丢";在 Windows 上 ``os.killpg`` 更是直接
|
||||
``AttributeError``。现在停止/暂停走**控制文件**(跨平台),信号只作兜底。这里把
|
||||
保证固定成回归测试:
|
||||
1. 定期 checkpoint 可续训(zip + 同名元数据);
|
||||
2. 进程组收到 SIGTERM 后仍能保存最终模型与元数据;
|
||||
3. subproc worker 忽略停止信号,不抢在主进程收尾之前退出。
|
||||
2. 控制文件发出 stop 后仍能保存最终模型与元数据(Windows 同样成立);
|
||||
3. POSIX 下进程组 SIGTERM 也能优雅收尾;
|
||||
4. subproc worker 忽略停止信号,不抢在主进程收尾之前退出。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -15,11 +18,14 @@ import os
|
||||
import signal
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from mcm26b.process_control import spawn_kwargs, terminate_process_tree, write_control
|
||||
|
||||
|
||||
def _train_module():
|
||||
"""按路径加载训练脚本(不触发 torch 导入,torch 只在函数内部惰性导入)。"""
|
||||
@@ -39,13 +45,20 @@ def _require_rl():
|
||||
|
||||
|
||||
class _Run:
|
||||
"""一次子进程训练:stdout 走文件,避免管道写满把子进程卡死。"""
|
||||
"""一次子进程训练:stdout 走文件,避免管道写满把子进程卡死。
|
||||
|
||||
与训练前端一致:用控制文件请求暂停/停止,进程组隔离走
|
||||
:func:`mcm26b.process_control.spawn_kwargs`(POSIX ``setsid`` /
|
||||
Windows ``CREATE_NEW_PROCESS_GROUP``)。
|
||||
"""
|
||||
|
||||
def __init__(self, run_dir: Path, *extra: str):
|
||||
script = Path(__file__).parents[1] / "scripts" / "train_hrl_high.py"
|
||||
self.run_dir = run_dir
|
||||
self.log_path = run_dir / "train_stdout.log"
|
||||
self.log_file = self.log_path.open("w", encoding="utf-8")
|
||||
self.control_path = run_dir / "control.json"
|
||||
write_control(self.control_path)
|
||||
cmd = [sys.executable, str(script),
|
||||
"--problem", "3", "--stage", "1", "--timesteps", "200000",
|
||||
"--seed", "7", "--envs", "2", "--vec-env", "subproc",
|
||||
@@ -53,13 +66,28 @@ class _Run:
|
||||
"--reward-profile", "time", "--eval-episodes", "1",
|
||||
"--rollout-steps", "32", "--batch-size", "32", "--n-epochs", "1",
|
||||
"--max-wall-s", "600", "--force", "--verbose", "0",
|
||||
"--control-file", str(self.control_path),
|
||||
"--model", str(run_dir / "q3.zip"), *extra]
|
||||
self.proc = subprocess.Popen(
|
||||
self.proc = subprocess.Popen( # pylint: disable=consider-using-with
|
||||
cmd, stdout=self.log_file, stderr=subprocess.STDOUT, text=True,
|
||||
start_new_session=True)
|
||||
**spawn_kwargs())
|
||||
|
||||
def checkpoints(self) -> list[Path]:
|
||||
return sorted((self.run_dir / "checkpoints").glob("q3_ckpt_*.zip"))
|
||||
"""历史 checkpoint(``<stem>_<步数>.zip``)。
|
||||
|
||||
刻意排除滚动恢复点 ``<stem>_latest.zip``:它随写随覆盖,不是"第几个
|
||||
checkpoint",混进来会让序号断言失去意义。
|
||||
"""
|
||||
found = (self.run_dir / "checkpoints").glob("q3_ckpt_[0-9]*.zip")
|
||||
return sorted(path for path in found
|
||||
if path.stem.rsplit("_", 1)[-1].isdigit())
|
||||
|
||||
def recovery(self) -> Path:
|
||||
return self.run_dir / "checkpoints" / "q3_ckpt_latest.zip"
|
||||
|
||||
def resume_hint(self) -> dict:
|
||||
return json.loads((self.run_dir / "resume.json").read_text(
|
||||
encoding="utf-8"))
|
||||
|
||||
def wait_for(self, predicate, timeout_s: float) -> bool:
|
||||
deadline = time.monotonic() + timeout_s
|
||||
@@ -71,8 +99,35 @@ class _Run:
|
||||
time.sleep(0.4)
|
||||
return predicate()
|
||||
|
||||
def logs(self) -> str:
|
||||
try:
|
||||
return self.log_path.read_text(encoding="utf-8", errors="replace")
|
||||
except OSError: # pragma: no cover - 文件被并发写时的极端情况
|
||||
return ""
|
||||
|
||||
def pause(self) -> None:
|
||||
write_control(self.control_path, paused=True)
|
||||
|
||||
def resume(self) -> None:
|
||||
write_control(self.control_path, paused=False)
|
||||
|
||||
def _finish(self) -> str:
|
||||
self.log_file.close()
|
||||
return self.logs()
|
||||
|
||||
def stop(self) -> str:
|
||||
"""复现训练前端的停止语义:对整个进程组发 SIGTERM。"""
|
||||
"""复现训练前端的停止语义:写控制文件请求优雅收尾(跨平台)。"""
|
||||
write_control(self.control_path, stop=True)
|
||||
forced = terminate_process_tree(self.proc, grace_s=240.0)
|
||||
if forced:
|
||||
self.log_file.close()
|
||||
pytest.fail("控制文件停止请求后训练进程未在 240s 内退出(优雅停止失败)")
|
||||
return self._finish()
|
||||
|
||||
def stop_by_signal(self) -> str:
|
||||
"""POSIX 兜底路径:对整个进程组发 SIGTERM(Windows 不支持 killpg)。"""
|
||||
if os.name == "nt": # pragma: no cover - 仅在 Windows 上走到
|
||||
pytest.skip("Windows 无 os.killpg,信号路径不适用")
|
||||
try:
|
||||
os.killpg(os.getpgid(self.proc.pid), signal.SIGTERM)
|
||||
except (ProcessLookupError, PermissionError):
|
||||
@@ -84,8 +139,12 @@ class _Run:
|
||||
self.proc.wait(timeout=30)
|
||||
self.log_file.close()
|
||||
pytest.fail("SIGTERM 后训练进程未在 240s 内退出(优雅停止失败)")
|
||||
return self._finish()
|
||||
|
||||
def hard_kill(self) -> None:
|
||||
"""模拟"急停":立即强杀整棵进程树,不给训练脚本任何收尾机会。"""
|
||||
terminate_process_tree(self.proc, grace_s=0.0)
|
||||
self.log_file.close()
|
||||
return self.log_path.read_text(encoding="utf-8", errors="replace")
|
||||
|
||||
|
||||
def _run_dir(tmp_path: Path) -> Path:
|
||||
@@ -119,7 +178,7 @@ def test_worker_ignores_stop_signals_but_main_does_not(monkeypatch):
|
||||
class _Worker:
|
||||
name = "SpawnProcess-3"
|
||||
|
||||
monkeypatch.setattr(multiprocessing, "current_process", lambda: _Worker())
|
||||
monkeypatch.setattr(multiprocessing, "current_process", _Worker)
|
||||
try:
|
||||
module._ignore_stop_signals_in_worker()
|
||||
assert signal.getsignal(signal.SIGTERM) == signal.SIG_IGN
|
||||
@@ -131,7 +190,7 @@ def test_worker_ignores_stop_signals_but_main_does_not(monkeypatch):
|
||||
class _Main:
|
||||
name = "MainProcess"
|
||||
|
||||
monkeypatch.setattr(multiprocessing, "current_process", lambda: _Main())
|
||||
monkeypatch.setattr(multiprocessing, "current_process", _Main)
|
||||
module._ignore_stop_signals_in_worker()
|
||||
assert signal.getsignal(signal.SIGTERM) == original
|
||||
|
||||
@@ -168,14 +227,139 @@ def test_policy_hyperparams_records_effective_values():
|
||||
assert module._policy_hyperparams(object())["n_steps"] == 0
|
||||
|
||||
|
||||
def test_control_poll_state_machine(tmp_path):
|
||||
"""控制文件轮询状态机:抽样读盘、暂停阻塞、stop 打断、暂停时长累计。"""
|
||||
module = _train_module()
|
||||
path = tmp_path / "control.json"
|
||||
write_control(path, paused=True)
|
||||
poll = module._ControlPoll(path, poll_every=1, poll_s=0.05)
|
||||
assert poll.poll().paused is True
|
||||
|
||||
events = []
|
||||
|
||||
def _resume_later():
|
||||
time.sleep(0.15)
|
||||
write_control(path, paused=False)
|
||||
|
||||
threading.Thread(target=_resume_later, daemon=True).start()
|
||||
stopped = poll.wait_while_paused(lambda: events.append("pause"),
|
||||
lambda: events.append("resume"))
|
||||
assert stopped is False
|
||||
assert events == ["pause", "resume"]
|
||||
assert poll.paused_total_s >= 0.1
|
||||
|
||||
# 暂停期间收到 stop:wait 必须返回 True,让调用方转入收尾
|
||||
write_control(path, paused=True)
|
||||
|
||||
def _stop_later():
|
||||
time.sleep(0.15)
|
||||
write_control(path, stop=True)
|
||||
|
||||
threading.Thread(target=_stop_later, daemon=True).start()
|
||||
assert poll.wait_while_paused(lambda: None, lambda: None) is True
|
||||
assert poll.poll().stop is True
|
||||
|
||||
|
||||
def test_atomic_model_save_keeps_previous_snapshot_on_failure(tmp_path):
|
||||
"""保存途中失败/被杀:目标路径仍是上一份完整快照,不会出现半截 zip。"""
|
||||
module = _train_module()
|
||||
target = tmp_path / "q3_ckpt_latest.zip"
|
||||
target.write_bytes(b"OLD-COMPLETE")
|
||||
|
||||
class _FailingModel:
|
||||
def save(self, path):
|
||||
Path(path).write_bytes(b"PARTIAL")
|
||||
raise RuntimeError("磁盘写满")
|
||||
|
||||
with pytest.raises(RuntimeError):
|
||||
module._save_model_atomic(_FailingModel(), target)
|
||||
assert target.read_bytes() == b"OLD-COMPLETE"
|
||||
leftovers = [p.name for p in tmp_path.iterdir() if p.name != target.name]
|
||||
assert leftovers and all(name.startswith(".tmp-") for name in leftovers)
|
||||
|
||||
|
||||
def test_write_json_atomic_replaces_in_one_step(tmp_path):
|
||||
module = _train_module()
|
||||
path = tmp_path / "resume.json"
|
||||
module._write_json_atomic(path, {"model": "a.zip", "timesteps": 1})
|
||||
assert json.loads(path.read_text(encoding="utf-8"))["timesteps"] == 1
|
||||
module._write_json_atomic(path, {"model": "b.zip", "timesteps": 2})
|
||||
assert json.loads(path.read_text(encoding="utf-8"))["model"] == "b.zip"
|
||||
assert not list(tmp_path.glob(".tmp-*"))
|
||||
|
||||
|
||||
def test_prune_checkpoints_never_deletes_recovery_point(tmp_path):
|
||||
"""滚动恢复点与临时文件不参与历史清理。"""
|
||||
module = _train_module()
|
||||
directory = tmp_path / "checkpoints"
|
||||
directory.mkdir()
|
||||
latest = directory / "q3_ckpt_latest.zip"
|
||||
latest.write_bytes(b"RECOVERY")
|
||||
(directory / ".tmp-q3_ckpt_000000009.zip").write_bytes(b"PARTIAL")
|
||||
for index in range(5):
|
||||
(directory / f"q3_ckpt_{index:09d}.zip").write_bytes(b"x")
|
||||
module._prune_checkpoints(directory, "q3_ckpt", 1)
|
||||
assert latest.exists() and latest.read_bytes() == b"RECOVERY"
|
||||
assert (directory / ".tmp-q3_ckpt_000000009.zip").exists()
|
||||
|
||||
|
||||
# --------------------------------------------------------------- 集成回归
|
||||
def test_recovery_point_exists_before_training_and_is_loadable(tmp_path):
|
||||
"""起步基线:训练刚起就有可加载、可续训的产物(--checkpoint-every 0 也成立)。"""
|
||||
_require_rl()
|
||||
from sb3_contrib import MaskablePPO
|
||||
|
||||
run = _Run(_run_dir(tmp_path), "--checkpoint-every", "0")
|
||||
try:
|
||||
assert run.wait_for(lambda: run.recovery().exists(), 120.0), \
|
||||
"起步恢复点未落盘"
|
||||
loaded = MaskablePPO.load(str(run.recovery()))
|
||||
assert loaded.num_timesteps >= 0
|
||||
hint = run.resume_hint()
|
||||
assert Path(hint["model"]) == run.recovery()
|
||||
assert hint["timesteps_semantics"] == "additional"
|
||||
finally:
|
||||
run.stop()
|
||||
|
||||
|
||||
def test_hard_kill_still_leaves_resumable_recovery_point(tmp_path):
|
||||
"""急停(强杀、无收尾)后仍能续训:滚动恢复点 + resume.json 都在且可加载。"""
|
||||
_require_rl()
|
||||
from sb3_contrib import MaskablePPO
|
||||
|
||||
run = _Run(_run_dir(tmp_path), "--checkpoint-every", "0",
|
||||
"--recovery-every", "32")
|
||||
try:
|
||||
assert run.wait_for(lambda: run.recovery().exists(), 120.0), \
|
||||
"起步恢复点未落盘"
|
||||
first = run.recovery().stat().st_mtime_ns
|
||||
assert run.wait_for(
|
||||
lambda: run.recovery().stat().st_mtime_ns != first, 300.0), \
|
||||
"滚动恢复点未按 --recovery-every 刷新"
|
||||
assert run.proc.poll() is None, "训练已自行结束,无法验证急停"
|
||||
finally:
|
||||
run.hard_kill() # 急停:强杀整棵进程树,没有任何 Python 收尾
|
||||
|
||||
hint = run.resume_hint()
|
||||
assert Path(hint["model"]).is_file(), hint
|
||||
assert Path(hint["model"]) == run.recovery(), hint
|
||||
loaded = MaskablePPO.load(str(run.recovery()))
|
||||
assert loaded.num_timesteps >= 32, loaded.num_timesteps
|
||||
metadata = json.loads(run.recovery().with_suffix(".json").read_text(
|
||||
encoding="utf-8"))
|
||||
assert metadata["recovery"] is True
|
||||
assert metadata["checkpoint_reason"] in ("baseline", "interval")
|
||||
# 急停没有 graceful 收尾,所以不会有"stopped_by_signal"的最终模型
|
||||
assert not (run.run_dir / "q3.zip").exists()
|
||||
|
||||
|
||||
def test_checkpoint_is_written_and_loadable(tmp_path):
|
||||
_require_rl()
|
||||
from sb3_contrib import MaskablePPO
|
||||
|
||||
run = _Run(_run_dir(tmp_path), "--checkpoint-every", "64")
|
||||
try:
|
||||
assert run.wait_for(lambda: bool(run.checkpoints()), 300.0), \
|
||||
assert run.wait_for(run.checkpoints, 300.0), \
|
||||
"训练未在 300s 内产出 checkpoint"
|
||||
finally:
|
||||
run.stop()
|
||||
@@ -193,16 +377,60 @@ def test_checkpoint_is_written_and_loadable(tmp_path):
|
||||
assert (hyper["n_steps"], hyper["batch_size"], hyper["n_epochs"]) == (32, 32, 1)
|
||||
|
||||
|
||||
def test_control_file_stop_saves_model(tmp_path):
|
||||
"""控制文件 stop 是训练前端的默认停止路径,Windows 上同样必须成立。"""
|
||||
_require_rl()
|
||||
run = _Run(_run_dir(tmp_path), "--checkpoint-every", "64")
|
||||
model_path = run.run_dir / "q3.zip"
|
||||
try:
|
||||
assert run.wait_for(run.checkpoints, 300.0), \
|
||||
"训练未在 300s 内进入 rollout 阶段"
|
||||
assert run.proc.poll() is None, "训练进程提前退出,无法验证优雅停止"
|
||||
finally:
|
||||
output = run.stop()
|
||||
|
||||
assert run.proc.returncode == 0, \
|
||||
f"控制文件停止应正常退出,实际 {run.proc.returncode}\n{output}"
|
||||
assert model_path.exists(), "控制文件停止后最终模型未落盘"
|
||||
metadata = json.loads(model_path.with_suffix(".json").read_text(
|
||||
encoding="utf-8"))
|
||||
assert metadata["stopped_by_signal"] is True
|
||||
assert metadata["stop_signal"] == "control_file"
|
||||
assert metadata["checkpoint_files"], "元数据应记录已产出的 checkpoint"
|
||||
assert not (run.run_dir / "training_failure.json").exists()
|
||||
assert "saved_after_stop" in output
|
||||
|
||||
|
||||
def test_control_file_pause_then_resume_keeps_training_alive(tmp_path):
|
||||
"""暂停是协作式的:进程存活、停在 step 边界,continue 后照常收尾。"""
|
||||
_require_rl()
|
||||
run = _Run(_run_dir(tmp_path), "--checkpoint-every", "64")
|
||||
try:
|
||||
assert run.wait_for(run.checkpoints, 300.0), \
|
||||
"训练未在 300s 内进入 rollout 阶段"
|
||||
run.pause()
|
||||
assert run.wait_for(lambda: '"event": "paused"' in run.logs(), 120.0), \
|
||||
"训练脚本未在 120s 内进入暂停"
|
||||
assert run.proc.poll() is None, "暂停不应结束进程"
|
||||
run.resume()
|
||||
assert run.wait_for(lambda: '"event": "resumed"' in run.logs(), 120.0), \
|
||||
"训练脚本未在 120s 内恢复"
|
||||
finally:
|
||||
run.stop()
|
||||
assert run.proc.returncode == 0
|
||||
|
||||
|
||||
@pytest.mark.skipif(os.name == "nt", reason="Windows 没有 os.killpg,信号兜底路径不适用")
|
||||
def test_sigterm_saves_model_instead_of_losing_the_run(tmp_path):
|
||||
_require_rl()
|
||||
run = _Run(_run_dir(tmp_path), "--checkpoint-every", "64")
|
||||
model_path = run.run_dir / "q3.zip"
|
||||
try:
|
||||
assert run.wait_for(lambda: bool(run.checkpoints()), 300.0), \
|
||||
assert run.wait_for(run.checkpoints, 300.0), \
|
||||
"训练未在 300s 内进入 rollout 阶段"
|
||||
assert run.proc.poll() is None, "训练进程提前退出,无法验证优雅停止"
|
||||
finally:
|
||||
output = run.stop()
|
||||
output = run.stop_by_signal()
|
||||
|
||||
assert run.proc.returncode == 0, \
|
||||
f"优雅停止应正常退出,实际 {run.proc.returncode}\n{output}"
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
"""低层导航运动学与训练入口的行为回归;不验证策略收益。"""
|
||||
|
||||
import importlib.util
|
||||
import math
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
@@ -59,7 +58,7 @@ def test_expert_reaches_all_200_fixed_seed_targets():
|
||||
|
||||
def test_failed_executor_reports_failure_without_free_terminal_move():
|
||||
class WestPolicy:
|
||||
def select(self, observation):
|
||||
def select(self, _observation):
|
||||
return 4
|
||||
|
||||
executor = LowLevelExecutor(WestPolicy(), max_steps=3)
|
||||
@@ -73,7 +72,7 @@ def test_failed_executor_reports_failure_without_free_terminal_move():
|
||||
|
||||
def test_env_and_executor_share_step_transitions():
|
||||
class EastPolicy:
|
||||
def select(self, observation):
|
||||
def select(self, _observation):
|
||||
return 0
|
||||
|
||||
env = LowLevelEnv(max_steps=3)
|
||||
|
||||
@@ -75,6 +75,9 @@ def test_time_only_reward_has_no_per_source_shaping():
|
||||
reason="RL 依赖组未安装")
|
||||
def test_project_compat_layer_reexports_real_classes():
|
||||
from mcm26b.hrl.hierarchical_env import HierarchicalEnv
|
||||
# project 是仓库根部的兼容层:pytest 经 rootdir 把仓库根放进 sys.path,
|
||||
# 而单独 lint tests/ 时 pylint 的 sys.path 不含仓库根,故此处局部豁免。
|
||||
# pylint: disable=import-error
|
||||
from project.env.local_env import HierarchicalEnv as CompatEnv
|
||||
from project.env.simulator_client import SimulatorClient
|
||||
from project.policies.high_level_ppo import HighLevelPolicy
|
||||
|
||||
@@ -43,21 +43,21 @@ def test_fallback_failure_then_success_records_actual_time_position_and_clear():
|
||||
assert loop.client.state.channel == 20
|
||||
assert loop.fallback_log[-1]["attempts"] == 2
|
||||
assert 20 not in loop.blocked_probes
|
||||
assert loop.anomalies == []
|
||||
assert not loop.anomalies
|
||||
finally:
|
||||
loop.client.exit()
|
||||
|
||||
|
||||
def test_invalid_policy_output_is_logged_and_replaced_by_valid_action():
|
||||
class InvalidPolicy:
|
||||
def select(self, observation, mask):
|
||||
def select(self, _observation, _mask):
|
||||
return N_CANDIDATES + 1
|
||||
|
||||
loop = make_loop(problem=3, high_policy=InvalidPolicy())
|
||||
candidate = Candidate(STATION_OFFSET, "station", (0.0, 0.0), station_index=0)
|
||||
executed = []
|
||||
|
||||
def execute(action, candidates):
|
||||
def execute(action, candidates): # pylint: disable=unused-argument
|
||||
executed.append(action)
|
||||
loop.stop_reason = "test_complete"
|
||||
|
||||
@@ -78,8 +78,10 @@ def test_opportunistic_measure_precedes_candidate_snapshot():
|
||||
order = []
|
||||
try:
|
||||
assert loop.config.bayes.opportunistic_measure
|
||||
with patch.object(loop, "_measure_at_stop", side_effect=lambda: order.append("measure")), \
|
||||
patch.object(loop, "build_candidates", side_effect=lambda: order.append("snapshot") or []):
|
||||
with patch.object(loop, "_measure_at_stop",
|
||||
side_effect=lambda: order.append("measure")), \
|
||||
patch.object(loop, "build_candidates",
|
||||
side_effect=lambda: order.append("snapshot") or []):
|
||||
loop._coverage_phase()
|
||||
assert order == ["measure", "snapshot"]
|
||||
finally:
|
||||
|
||||
@@ -134,7 +134,7 @@ class StepTimingTests(unittest.TestCase):
|
||||
|
||||
def test_steps_recorded_with_virtual_breakdown(self):
|
||||
sources = random_omni_scene(seed=5, count=2)
|
||||
summary, report, _ = _run_scene(sources)
|
||||
summary, _, _ = _run_scene(sources)
|
||||
steps = summary["steps"]
|
||||
self.assertEqual(steps[0]["action"], "enter")
|
||||
self.assertEqual(steps[-1]["action"], "exit")
|
||||
@@ -228,7 +228,7 @@ class SimulatorVirtualCapTests(unittest.TestCase):
|
||||
class AbortFinalizationTests(unittest.TestCase):
|
||||
"""断连超过重试上限:仍须 /exit、产出未完成 summary,不得静默崩溃。"""
|
||||
|
||||
def test_transport_failure_aborts_with_summary_and_exit(self):
|
||||
def test_transport_failure_aborts_with_summary(self):
|
||||
"""首个检测动作三次断连:run() 正常返回,/exit 成功,stop_reason 留档。"""
|
||||
summary, _, _ = _run_scene(
|
||||
random_omni_scene(seed=5, count=2),
|
||||
|
||||
@@ -97,7 +97,7 @@ class ProbeLadderGeometryTests(unittest.TestCase):
|
||||
loop.blocked_probes[7] = [blocked_point]
|
||||
return loop._next_probe(region, 7)
|
||||
|
||||
def test_antipode_reflects_blocked_probe_through_center(self):
|
||||
def test_antipode_reflects_blocked_probe_in_center(self):
|
||||
"""圆心为原点、原探点 (200,0):对跖复核应返回 (-200,0)。"""
|
||||
region = CandidateRegion.disk((0.0, 0.0), 100.0, 192) # safe≈900>=200
|
||||
probe = self._probe_after_block(region, (200.0, 0.0))
|
||||
@@ -113,7 +113,7 @@ class ProbeLadderGeometryTests(unittest.TestCase):
|
||||
self.assertAlmostEqual(math.hypot(probe[0], probe[1]), safe, places=6)
|
||||
self.assertLess(probe[0], 0.0, "应位于原探点的对跖方向")
|
||||
|
||||
def test_degenerate_center_block_walks_full_safe_radius(self):
|
||||
def test_degenerate_center_block_walks_safe_radius(self):
|
||||
"""被遮挡探点恰在圆心:沿 +x 走满安全盘半径,不再钳到 1m。"""
|
||||
region = CandidateRegion.disk((0.0, 0.0), 100.0, 192)
|
||||
probe = self._probe_after_block(region, (0.0, 0.0))
|
||||
@@ -125,7 +125,7 @@ class ProbeLadderGeometryTests(unittest.TestCase):
|
||||
class MixedLoopAcceptanceTests(unittest.TestCase):
|
||||
def test_mixed_ten_sources_all_cleared(self):
|
||||
sources = mixed_scene(seed=5, count=10, directional=3)
|
||||
summary, report, simulator, _, loop = _run_scene(sources)
|
||||
summary, report, simulator, _, _ = _run_scene(sources)
|
||||
truth = sorted(s.channel for s in sources)
|
||||
self.assertTrue(summary["completed"])
|
||||
self.assertEqual(summary["stop_reason"], "all_resolved")
|
||||
@@ -155,7 +155,7 @@ class MixedLoopAcceptanceTests(unittest.TestCase):
|
||||
self.assertEqual(sorted(simulator.cleared), truth)
|
||||
self.assertEqual(report["mismatches"], [])
|
||||
|
||||
def test_outward_boundary_directional_source_not_missed(self):
|
||||
def test_outward_boundary_directional_not_missed(self):
|
||||
"""朝向背离圆心、r_eff=1000 的近边界源:覆盖保证仍发现并清除。"""
|
||||
source = Source(channel=7, x=1790.0, y=0.0, r_eff_m=1000.0,
|
||||
directional=True, heading_deg=0.0)
|
||||
@@ -169,8 +169,8 @@ class MixedLoopAcceptanceTests(unittest.TestCase):
|
||||
"""固定批量种子回归:任何种子下不得全清失败或破坏守恒性。"""
|
||||
for seed in range(5, 10):
|
||||
sources = mixed_scene(seed=seed, count=10, directional=4)
|
||||
summary, report, simulator, _, _ = _run_scene(sources,
|
||||
config=SMALL_CAP)
|
||||
summary, report, _, _, _ = _run_scene(sources,
|
||||
config=SMALL_CAP)
|
||||
truth = sorted(s.channel for s in sources)
|
||||
self.assertTrue(summary["completed"], f"seed={seed}")
|
||||
self.assertEqual(sorted(summary["cleared_channels"]), truth,
|
||||
@@ -194,7 +194,7 @@ class JointConstraintBaselineTests(unittest.TestCase):
|
||||
for witness in region.witnesses(300.0):
|
||||
self.assertTrue(region.contains(witness, tol_m=0.05))
|
||||
|
||||
def test_directional_source_retained_through_blocked_probes(self):
|
||||
def test_directional_source_kept_after_blocked_probes(self):
|
||||
"""端到端保留性:定向源被遮挡若干次后仍被清除,不因 no_signal 误删。"""
|
||||
source = Source(channel=3, x=600.0, y=800.0, r_eff_m=1500.0,
|
||||
directional=True, heading_deg=200.0)
|
||||
@@ -261,7 +261,7 @@ class FallbackDisciplineTests(unittest.TestCase):
|
||||
cells = fallback_clear_cells(region.polygon, CONFIG)
|
||||
self.assertLessEqual(len(cells), CONFIG.search.fallback_max_cells)
|
||||
|
||||
def test_fallback_exhaustion_records_anomaly_without_deletion(self):
|
||||
def test_fallback_exhaustion_records_anomaly(self):
|
||||
"""后备全失败(空域):守恒性异常留痕,且不删除空间候选。"""
|
||||
simulator = LocalSimulator([], robot_id="T001", config=CONFIG)
|
||||
client = SimulatorClient("T001", config=CONFIG,
|
||||
|
||||
@@ -13,7 +13,6 @@ from dataclasses import replace
|
||||
from random import Random
|
||||
|
||||
from mcm26b.config import ProblemConfig
|
||||
from mcm26b.geometry.localization import smallest_enclosing_circle
|
||||
from mcm26b.particles import ChannelParticles, angular_gap
|
||||
from mcm26b.protocol.rules import DIRECTION, NEAR, NO_SIGNAL
|
||||
from mcm26b.planner.seven import seven_points
|
||||
|
||||
@@ -31,7 +31,7 @@ def _k1_region() -> CandidateRegion:
|
||||
|
||||
|
||||
class SelectionGuaranteeTests(TestCase):
|
||||
def test_selected_point_receives_every_sector_candidate(self):
|
||||
def test_selected_point_gets_every_sector_candidate(self):
|
||||
"""安全盘内选点对解析角域 K1 内任意候选源的距离 <= 1000m。
|
||||
|
||||
直接对截断角域 K1 = Ω ∩ W(s1, θ±δ) ∩ B(s1,1500) 稠密采样验证,
|
||||
@@ -113,7 +113,7 @@ class ClearCertificateTests(TestCase):
|
||||
|
||||
|
||||
class SelectSecondPointTests(TestCase):
|
||||
def test_plan_lies_in_analytic_safe_disk_or_is_equivalent(self):
|
||||
def test_plan_in_analytic_safe_disk_is_equivalent(self):
|
||||
"""P3 入口输出与 geometry 解析证书一致(或提供同等保证)。"""
|
||||
simulator = LocalSimulator([], robot_id="T001")
|
||||
client = SimulatorClient("T001", transport=LocalSimTransport(simulator))
|
||||
|
||||
@@ -38,7 +38,7 @@ def test_seven_stations_cannot_prove_directional_absence():
|
||||
|
||||
|
||||
def test_stopped_opportunistic_measure_is_serial_and_never_repeats_location():
|
||||
sim, client, loop = setup([Source(1, 800, 200, 1500)])
|
||||
_, client, loop = setup([Source(1, 800, 200, 1500)])
|
||||
client.enter()
|
||||
loop._record_measure(1, (0, 0), client.measure((0, 0), 1))
|
||||
client.measure((600, 0), 2)
|
||||
|
||||
@@ -40,8 +40,10 @@ class PracticeReplayTests(unittest.TestCase):
|
||||
self.assertEqual(data['source'],'official_practice')
|
||||
self.assertIsNone(data['ground_truth'])
|
||||
directions=[e for e in events if e['response'].get('measure_result')=='direction']
|
||||
self.assertEqual(directions[0]['request']['position'],directions[1]['request']['position'])
|
||||
self.assertEqual(directions[0]['response']['svd_deg'],directions[1]['response']['svd_deg'])
|
||||
self.assertEqual(directions[0]['request']['position'],
|
||||
directions[1]['request']['position'])
|
||||
self.assertEqual(directions[0]['response']['svd_deg'],
|
||||
directions[1]['response']['svd_deg'])
|
||||
|
||||
|
||||
if __name__=='__main__': unittest.main()
|
||||
|
||||
@@ -45,7 +45,7 @@ class FaultRecoveryTests(unittest.TestCase):
|
||||
outcome = client.measure((0, 0), 1)
|
||||
self.assertEqual(outcome.virtual_time_s, 5.0)
|
||||
# measure 只被服务器执行了一次:请求发出两次,但虚拟时刻只推进一次。
|
||||
measure_bodies = [body for body in faults.sent_requests[1:]]
|
||||
measure_bodies = list(faults.sent_requests[1:])
|
||||
self.assertEqual(len(measure_bodies), 2, "应发出原始请求+一次重试")
|
||||
first, retry = (json.loads(b) for b in measure_bodies)
|
||||
self.assertEqual(first["request_id"], retry["request_id"],
|
||||
@@ -56,22 +56,22 @@ class FaultRecoveryTests(unittest.TestCase):
|
||||
|
||||
def test_drop_before_apply_is_safe_to_retry(self):
|
||||
"""连接失败(未送达)后重试成功,无残留效果。"""
|
||||
client, faults, simulator = self._client(["drop_before", None])
|
||||
client, _, simulator = self._client(["drop_before", None])
|
||||
client.enter()
|
||||
client.measure((100, 0), 1)
|
||||
self.assertEqual(simulator.virtual_time_s, 25.0) # 100/5 移动 + 5 检测
|
||||
self.assertEqual(client.state.virtual_time_s, 25.0)
|
||||
|
||||
def test_server_500_retried_with_same_id(self):
|
||||
client, faults, simulator = self._client([None, ("status", 500), None, None])
|
||||
client, _, _ = self._client([None, ("status", 500), None, None])
|
||||
client.enter()
|
||||
client.measure((0, 0), 1)
|
||||
self.assertEqual(client.state.virtual_time_s, 5.0)
|
||||
|
||||
def test_exhausted_retries_raise_and_leave_state_clean(self):
|
||||
# enter 放行后,measure 连续注入断连直至超过 max_attempts=3。
|
||||
client, faults, simulator = self._client([None, "drop_before", "drop_before",
|
||||
"drop_before"])
|
||||
client, _, simulator = self._client([None, "drop_before", "drop_before",
|
||||
"drop_before"])
|
||||
client.enter()
|
||||
with self.assertRaises(TransportError):
|
||||
client.measure((0, 0), 1)
|
||||
@@ -83,7 +83,7 @@ class FaultRecoveryTests(unittest.TestCase):
|
||||
self.assertEqual(outcome.virtual_time_s, 5.0)
|
||||
|
||||
def test_accepted_false_raises_and_state_untouched(self):
|
||||
client, faults, simulator = self._client([None])
|
||||
client, _, _ = self._client([None])
|
||||
with self.assertRaises(ProtocolError) as ctx:
|
||||
client.measure((0, 0), 1) # 未 enter -> accepted=false
|
||||
self.assertEqual(ctx.exception.status, 200)
|
||||
@@ -103,7 +103,7 @@ class DeadlineBudgetTests(unittest.TestCase):
|
||||
now[0] += 300.0
|
||||
self.assertAlmostEqual(client.real_remaining_s, 900.0)
|
||||
|
||||
def test_action_blocked_when_reserve_margin_cannot_be_kept(self):
|
||||
def test_action_blocked_when_reserve_margin_low(self):
|
||||
now = [0.0]
|
||||
simulator = LocalSimulator([], robot_id="T001")
|
||||
client = SimulatorClient("T001", transport=LocalSimTransport(simulator),
|
||||
@@ -172,8 +172,8 @@ class PracticeReplayTests(unittest.TestCase):
|
||||
events = [dict(path=e["path"], request=e["request"], response=e["response"])
|
||||
for e in self._events()]
|
||||
stats = summarize(events)
|
||||
directions = [e for e in events
|
||||
if e["response"].get("measure_result") == "direction"]
|
||||
_directions = [e for e in events
|
||||
if e["response"].get("measure_result") == "direction"]
|
||||
self.assertGreater(stats["measures"], 0)
|
||||
self.assertEqual(stats["clears"], 0, "该次演练只采集观测,未调用 /clear")
|
||||
|
||||
|
||||
@@ -29,7 +29,7 @@ def test_runtime_code_never_imports_referee():
|
||||
"print('clean')\n"
|
||||
)
|
||||
result = subprocess.run([sys.executable, "-c", code], capture_output=True,
|
||||
text=True, cwd=ROOT)
|
||||
text=True, cwd=ROOT, check=False)
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert "clean" in result.stdout
|
||||
|
||||
|
||||
+3
-4
@@ -4,7 +4,6 @@ RL 依赖属于可选组 [dependency-groups] rl;未安装 numpy/gymnasium 时
|
||||
全部跳过(运行闭环不依赖它们)。
|
||||
"""
|
||||
|
||||
import math
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
@@ -74,13 +73,13 @@ class TrainingEnvironmentTests(unittest.TestCase):
|
||||
self.assertIsInstance(reward, float)
|
||||
self.assertFalse(terminated and truncated)
|
||||
|
||||
def test_q4_directional_episode_reports_blocked_or_fallback(self):
|
||||
def test_q4_directional_reports_blocked_or_fallback(self):
|
||||
env = LocalizationEnv(problem=4, directional_probability=1.0)
|
||||
obs, info = env.reset(seed=7)
|
||||
_, info = env.reset(seed=7)
|
||||
self.assertTrue(info["source"]["directional"])
|
||||
terminated = truncated = False
|
||||
for _ in range(env.max_episode_steps + 2):
|
||||
obs, reward, terminated, truncated, info = env.step(
|
||||
_, _, terminated, truncated, info = env.step(
|
||||
env.action_space.sample())
|
||||
if terminated or truncated:
|
||||
break
|
||||
|
||||
@@ -9,7 +9,7 @@ np = pytest.importorskip('numpy')
|
||||
pytest.importorskip('gymnasium')
|
||||
|
||||
from mcm26b.rl.env import LocalizationEnv
|
||||
from mcm26b.rl.features import action_mask, build_observation, candidate_actions
|
||||
from mcm26b.rl.features import build_observation, candidate_actions
|
||||
|
||||
|
||||
def test_training_observation_matches_runtime_channel_belief():
|
||||
@@ -85,10 +85,12 @@ def test_bc_and_ppo_forward_mapping_agree():
|
||||
spec.loader.exec_module(module)
|
||||
torch.set_num_threads(1)
|
||||
env = LocalizationEnv()
|
||||
ppo = PPO('MlpPolicy',env,policy_kwargs={'net_arch':[128,128]},device='cpu',n_steps=16,batch_size=8)
|
||||
ppo = PPO('MlpPolicy',env,policy_kwargs={'net_arch':[128,128]},device='cpu',
|
||||
n_steps=16,batch_size=8)
|
||||
bc = module.BCMlp()
|
||||
state = bc.net.state_dict()
|
||||
ppo.policy.mlp_extractor.policy_net.load_state_dict({k:v for k,v in state.items() if not k.startswith('4.')})
|
||||
ppo.policy.mlp_extractor.policy_net.load_state_dict(
|
||||
{k:v for k,v in state.items() if not k.startswith('4.')})
|
||||
ppo.policy.action_net.load_state_dict({'weight':state['4.weight'],'bias':state['4.bias']})
|
||||
x = torch.randn(10,401)
|
||||
torch.testing.assert_close(bc(x),ppo.policy.action_net(ppo.policy.mlp_extractor.policy_net(x)))
|
||||
|
||||
+290
-16
@@ -6,11 +6,16 @@
|
||||
- SB3 表格解析必须给每一行打上 ``total_timesteps`` 横轴(曾把表格行当独立
|
||||
点,曲线上出现重复的幽灵点);
|
||||
- ``metric_offset``/``metric_end`` 游标契约(前端增量拉取依赖它);
|
||||
- 用一段桩训练脚本真起子进程,验证启动/轮询/停止/退出码全链路。
|
||||
- 用一段桩训练脚本真起子进程,验证启动/轮询/暂停/继续/停止/退出码全链路。
|
||||
"""
|
||||
|
||||
# pytest 用参数名注入 fixture:`stub_repo`/`client` 这类参数在函数体里不直接
|
||||
# 引用也是正常用法(它们的作用是 setup/teardown),这里统一放行。
|
||||
# pylint: disable=unused-argument
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import http.client
|
||||
import json
|
||||
import sys
|
||||
@@ -22,6 +27,7 @@ import pytest
|
||||
|
||||
pytest.importorskip("mcm26b")
|
||||
|
||||
from mcm26b import process_control as pc
|
||||
from mcm26b.training_ui import demo, server as td
|
||||
from mcm26b.training_ui.schema import (CONFIG_FIELDS, config_schema,
|
||||
preset_payloads)
|
||||
@@ -32,7 +38,7 @@ REPO = Path(__file__).resolve().parents[1]
|
||||
# 最后打印一行元数据。全部行为由参数控制,不依赖 torch/SB3。
|
||||
STUB_SCRIPT = r'''
|
||||
"""训练前端测试桩:模拟 SB3 verbose=1 表格 + --progress-json 输出。"""
|
||||
import argparse, json, signal, sys, time
|
||||
import argparse, json, os, signal, sys, time
|
||||
from pathlib import Path
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
@@ -41,12 +47,17 @@ parser.add_argument("--stage", type=int, default=1)
|
||||
parser.add_argument("--timesteps", type=int, default=96)
|
||||
parser.add_argument("--model", default="out/model.zip")
|
||||
parser.add_argument("--rollout-steps", type=int, default=32)
|
||||
parser.add_argument("--sleep", type=float, default=0.0)
|
||||
# 默认 sleep 可由环境变量控制:HTTP 端到端用例无法直接追加命令行参数,
|
||||
# 用小 sleep 拉长运行窗口,才能稳定地测"暂停/继续"。
|
||||
parser.add_argument("--sleep", type=float,
|
||||
default=float(os.environ.get("STUB_TRAIN_SLEEP", "0")))
|
||||
parser.add_argument("--lines", type=int, default=4)
|
||||
parser.add_argument("--table-first", action="store_true")
|
||||
parser.add_argument("--progress-json", action="store_true")
|
||||
parser.add_argument("--crash", action="store_true")
|
||||
parser.add_argument("--ignore-sigterm", action="store_true")
|
||||
parser.add_argument("--ignore-control", action="store_true")
|
||||
parser.add_argument("--control-file", default=None)
|
||||
args, _extra = parser.parse_known_args()
|
||||
|
||||
model = Path(args.model)
|
||||
@@ -65,6 +76,49 @@ print("NORMAL-LOG-LINE")
|
||||
sys.stdout.flush()
|
||||
|
||||
|
||||
def read_control():
|
||||
if not args.control_file or args.ignore_control:
|
||||
return {}
|
||||
try:
|
||||
with open(args.control_file, encoding="utf-8") as handle:
|
||||
return json.load(handle)
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
def control_pause_or_stop(step):
|
||||
"""返回 True 表示应退出;暂停时阻塞到继续或停止(模拟协作式暂停)。"""
|
||||
state = read_control()
|
||||
if state.get("stop"):
|
||||
print(json.dumps({"event": "stop_requested", "signal": "control_file"}), flush=True)
|
||||
return True
|
||||
if state.get("paused"):
|
||||
print(json.dumps({"event": "paused", "timesteps": step}), flush=True)
|
||||
while True:
|
||||
time.sleep(0.05)
|
||||
state = read_control()
|
||||
if state.get("stop"):
|
||||
print(json.dumps({"event": "stop_requested", "signal": "control_file"}),
|
||||
flush=True)
|
||||
return True
|
||||
if not state.get("paused"):
|
||||
break
|
||||
print(json.dumps({"event": "resumed", "timesteps": step}), flush=True)
|
||||
return False
|
||||
|
||||
|
||||
def sleep_with_control(seconds, step):
|
||||
"""把长 sleep 切成 0.1s 片,保证停止/暂停请求能被及时响应。"""
|
||||
deadline = time.monotonic() + max(0.0, seconds)
|
||||
while True:
|
||||
if control_pause_or_stop(step):
|
||||
raise SystemExit(0)
|
||||
remaining = deadline - time.monotonic()
|
||||
if remaining <= 0:
|
||||
return
|
||||
time.sleep(min(0.1, remaining))
|
||||
|
||||
|
||||
def render_table(step, fps):
|
||||
return (
|
||||
"-----------------------------\n"
|
||||
@@ -72,16 +126,15 @@ def render_table(step, fps):
|
||||
"| ep_len_mean | 9.67 |\n"
|
||||
"| ep_rew_mean | -3.24 |\n"
|
||||
"| time/ | |\n"
|
||||
"| fps | %-8d |\n"
|
||||
"| total_timesteps | %-8d |\n"
|
||||
"-----------------------------\n" % (fps, step)
|
||||
f"| fps | {fps:<8d} |\n"
|
||||
f"| total_timesteps | {step:<8d} |\n"
|
||||
"-----------------------------\n"
|
||||
)
|
||||
|
||||
|
||||
for index in range(args.lines):
|
||||
if args.sleep:
|
||||
time.sleep(args.sleep)
|
||||
step = 32 * (index + 1)
|
||||
sleep_with_control(args.sleep, step)
|
||||
fps = 17 + index
|
||||
progress = {
|
||||
"event": "progress", "timesteps": step, "n_updates": index + 1,
|
||||
@@ -153,7 +206,7 @@ def test_schema_matches_train_config_defaults():
|
||||
assert field["default"] == getattr(defaults, field["key"]), \
|
||||
f"{field['key']} 默认值与后端不一致"
|
||||
exposed = {field["key"] for field in schema["fields"]}
|
||||
missing = set(defaults.__dataclass_fields__) - exposed
|
||||
missing = {field.name for field in dataclasses.fields(defaults)} - exposed
|
||||
assert not missing, f"TrainConfig 字段未暴露给界面: {sorted(missing)}"
|
||||
groups = {group["key"] for group in schema["groups"]}
|
||||
for field in schema["fields"]:
|
||||
@@ -227,9 +280,11 @@ class _FakeJob:
|
||||
self._last_table_steps = None
|
||||
self._last_json_steps = None
|
||||
self._last_monitor_sig = None
|
||||
self.status = "running"
|
||||
self.config = make_config()
|
||||
|
||||
_on_line = td.TrainingJob._on_line
|
||||
_sync_control_events = td.TrainingJob._sync_control_events
|
||||
_parse_progress_json = td.TrainingJob._parse_progress_json
|
||||
_parse_sb3_table = td.TrainingJob._parse_sb3_table
|
||||
_flush_table = td.TrainingJob._flush_table
|
||||
@@ -274,9 +329,9 @@ def test_sb3_table_with_separators_emits_one_point_per_table():
|
||||
for index, steps in enumerate((32, 64, 96)):
|
||||
job._on_line("-----------------------------")
|
||||
job._on_line("| rollout/ | |")
|
||||
job._on_line("| ep_rew_mean | %.2f |" % (-3.0 + index))
|
||||
job._on_line(f"| ep_rew_mean | {-3.0 + index:.2f} |")
|
||||
job._on_line("| time/ | |")
|
||||
job._on_line("| total_timesteps | %d |" % steps)
|
||||
job._on_line(f"| total_timesteps | {steps} |")
|
||||
job._on_line("-----------------------------")
|
||||
job._flush_table()
|
||||
assert [point["timesteps"] for point in job.metrics] == [32, 64, 96], list(job.metrics)
|
||||
@@ -476,7 +531,7 @@ def test_job_lifecycle_with_stub_script(stub_repo):
|
||||
def test_job_reports_nonzero_exit_code(stub_repo):
|
||||
config = make_config(model="output/hrl/ui/test/crash/q3.zip")
|
||||
job = td.TrainingJob(config)
|
||||
job.command = td.build_command(config) + ["--crash"]
|
||||
job.command = list(job.command) + ["--crash"]
|
||||
job.start()
|
||||
assert wait_until(lambda: job.status in ("exited", "failed"), timeout=30)
|
||||
assert job.status == "failed"
|
||||
@@ -484,15 +539,200 @@ def test_job_reports_nonzero_exit_code(stub_repo):
|
||||
assert job.snapshot(log_tail=5)["process"]["tone"] == "bad"
|
||||
|
||||
|
||||
def test_stop_sends_sigterm_and_marks_stopped(stub_repo):
|
||||
def test_job_command_carries_control_file(stub_repo):
|
||||
"""控制文件路径必须随命令下发,否则暂停/停止退化成超时强杀。"""
|
||||
config = make_config(model="output/hrl/ui/test/control/q3.zip")
|
||||
job = td.TrainingJob(config, control_dir=Path(stub_repo).parent)
|
||||
assert "--control-file" in job.command
|
||||
assert str(job.control_path) in job.command
|
||||
assert job.control_path.parent == Path(stub_repo).parent
|
||||
|
||||
|
||||
def test_stop_gracefully_marks_stopped_without_forced_kill(stub_repo):
|
||||
config = make_config(model="output/hrl/ui/test/stop/q3.zip")
|
||||
job = td.TrainingJob(config)
|
||||
job.command = td.build_command(config) + ["--sleep", "5", "--lines", "50"]
|
||||
job.command = list(job.command) + ["--sleep", "5", "--lines", "50"]
|
||||
job.start()
|
||||
assert wait_until(lambda: job.status == "running", timeout=10)
|
||||
assert job.stop() is True
|
||||
assert wait_until(lambda: job.status in ("stopped", "failed"), timeout=15)
|
||||
assert job.status == "stopped"
|
||||
# 桩脚本读控制文件后主动退出:不应走到强杀分支
|
||||
assert job.forced_kill is False
|
||||
|
||||
|
||||
def test_pause_and_resume_via_control_file(stub_repo):
|
||||
config = make_config(model="output/hrl/ui/test/pause/q3.zip")
|
||||
job = td.TrainingJob(config)
|
||||
job.command = list(job.command) + ["--sleep", "0.2", "--lines", "60"]
|
||||
job.start()
|
||||
assert wait_until(lambda: job.status == "running", timeout=10)
|
||||
assert job.pause() is True
|
||||
# 训练脚本回显 paused 事件;状态由 _sync_control_events 校正
|
||||
assert wait_until(
|
||||
lambda: any('"event": "paused"' in line for line in job.snapshot()["log_lines"]),
|
||||
timeout=10), "训练脚本未回显 paused 事件"
|
||||
assert job.status == "paused"
|
||||
paused_metrics = len(job.snapshot()["metrics"])
|
||||
time.sleep(0.6)
|
||||
assert len(job.snapshot()["metrics"]) <= paused_metrics + 1, "暂停后仍在产出指标"
|
||||
assert job.resume() is True
|
||||
assert wait_until(
|
||||
lambda: any('"event": "resumed"' in line for line in job.snapshot()["log_lines"]),
|
||||
timeout=10), "训练脚本未回显 resumed 事件"
|
||||
assert wait_until(lambda: job.status == "running", timeout=5)
|
||||
assert job.stop() is True
|
||||
assert wait_until(lambda: job.status in ("stopped", "failed"), timeout=15)
|
||||
assert job.status == "stopped"
|
||||
|
||||
|
||||
def test_stop_forced_kill_when_control_ignored(stub_repo, monkeypatch):
|
||||
"""训练进程不读控制文件时必须超时强杀,并留下恢复指引。"""
|
||||
monkeypatch.setattr(td, "STOP_GRACE_S", 0.5)
|
||||
config = make_config(model="output/hrl/ui/test/stubborn/q3.zip")
|
||||
job = td.TrainingJob(config)
|
||||
job.command = list(job.command) + ["--ignore-control", "--sleep", "5", "--lines", "50"]
|
||||
job.start()
|
||||
assert wait_until(lambda: job.status == "running", timeout=10)
|
||||
# 造一份"磁盘上已有滚动恢复点"的现场,强杀后应被记录进恢复指引
|
||||
ckpt_dir = job.config.model_path.parent / "checkpoints"
|
||||
ckpt_dir.mkdir(parents=True, exist_ok=True)
|
||||
latest = ckpt_dir / "q3_ckpt_latest.zip"
|
||||
latest.write_bytes(b"PK\x03\x04fake")
|
||||
assert job.stop() is True
|
||||
assert wait_until(lambda: job.status in ("stopped", "failed"), timeout=20)
|
||||
assert job.forced_kill is True
|
||||
assert job.snapshot(log_tail=5)["forced_kill"] is True
|
||||
marker = json.loads(
|
||||
(job.config.model_path.parent / "forced_stop.json").read_text(encoding="utf-8"))
|
||||
assert marker["event"] == "forced_stop"
|
||||
assert marker["reason"] == "grace_timeout"
|
||||
assert marker["recovery"]["model"] == str(latest)
|
||||
assert marker["recovery"]["can_resume"] is True
|
||||
|
||||
|
||||
def test_discover_recovery_prefers_resume_hint(tmp_path):
|
||||
run_dir = tmp_path / "run"
|
||||
(run_dir / "checkpoints").mkdir(parents=True)
|
||||
latest = run_dir / "checkpoints" / "q3_ckpt_latest.zip"
|
||||
latest.write_bytes(b"PK\x03\x04fake")
|
||||
final = run_dir / "q3.zip"
|
||||
final.write_bytes(b"PK\x03\x04fake")
|
||||
(run_dir / "resume.json").write_text(json.dumps({
|
||||
"model": str(latest), "timesteps": 128, "reason": "interval",
|
||||
"created_unix_s": 5.0}), encoding="utf-8")
|
||||
state = td.discover_recovery(run_dir, final)
|
||||
assert state["can_resume"] is True
|
||||
assert state["model"] == str(latest)
|
||||
assert state["timesteps"] == 128
|
||||
assert state["source"] == "resume.json"
|
||||
|
||||
|
||||
def test_discover_recovery_falls_back_to_scan_on_dangling_hint(tmp_path):
|
||||
run_dir = tmp_path / "run"
|
||||
(run_dir / "checkpoints").mkdir(parents=True)
|
||||
latest = run_dir / "checkpoints" / "q3_ckpt_latest.zip"
|
||||
latest.write_bytes(b"PK\x03\x04fake")
|
||||
# 悬空指针(目标文件已不在)必须回退到扫描,而不是给出一个打不开的路径
|
||||
(run_dir / "resume.json").write_text(
|
||||
json.dumps({"model": str(run_dir / "gone.zip")}), encoding="utf-8")
|
||||
state = td.discover_recovery(run_dir, run_dir / "q3.zip")
|
||||
assert state["can_resume"] is True
|
||||
assert state["model"] == str(latest)
|
||||
assert state["source"] == "scan"
|
||||
|
||||
|
||||
def test_discover_recovery_with_nothing_on_disk(tmp_path):
|
||||
state = td.discover_recovery(tmp_path, tmp_path / "q3.zip")
|
||||
assert state["can_resume"] is False and state["model"] is None
|
||||
assert state["reason"] == "none"
|
||||
|
||||
|
||||
def test_command_carries_checkpoint_and_recovery_flags():
|
||||
config = make_config(model="output/x/q3.zip", recovery_every=100,
|
||||
checkpoint_every=500, checkpoint_keep=2)
|
||||
joined = " ".join(td.build_command(config))
|
||||
assert "--recovery-every 100" in joined
|
||||
assert "--checkpoint-every 500" in joined
|
||||
assert "--checkpoint-keep 2" in joined
|
||||
|
||||
|
||||
def test_config_rejects_bad_recovery_settings():
|
||||
with pytest.raises(ValueError):
|
||||
make_config(recovery_every=-1)
|
||||
with pytest.raises(ValueError):
|
||||
make_config(checkpoint_every=-1)
|
||||
with pytest.raises(ValueError):
|
||||
make_config(checkpoint_keep=0)
|
||||
|
||||
|
||||
def test_pause_resume_return_false_without_live_process(stub_repo):
|
||||
config = make_config(model="output/hrl/ui/test/nolive/q3.zip")
|
||||
job = td.TrainingJob(config)
|
||||
assert job.pause() is False and job.resume() is False
|
||||
job.start()
|
||||
assert wait_until(lambda: job.status in ("exited", "failed"), timeout=30)
|
||||
assert job.pause() is False and job.resume() is False
|
||||
|
||||
|
||||
# ---------------------------------------------------- 跨平台进程控制
|
||||
def test_control_file_roundtrip_and_corruption(tmp_path):
|
||||
path = tmp_path / "control.json"
|
||||
assert pc.read_control(path) == pc.ControlState() # 缺失 = 无指令
|
||||
pc.write_control(path, paused=True)
|
||||
assert pc.read_control(path).paused is True
|
||||
pc.write_control(path, stop=True)
|
||||
state = pc.read_control(path)
|
||||
assert state.stop is True and state.paused is False
|
||||
path.write_text("{ 半截 JSON", encoding="utf-8")
|
||||
assert pc.read_control(path) == pc.ControlState() # 损坏 = 无指令
|
||||
|
||||
|
||||
def test_spawn_kwargs_isolates_child_process_group(monkeypatch):
|
||||
monkeypatch.setattr(pc, "IS_WINDOWS", False)
|
||||
assert pc.spawn_kwargs() == {"start_new_session": True}
|
||||
monkeypatch.setattr(pc, "IS_WINDOWS", True)
|
||||
kwargs = pc.spawn_kwargs()
|
||||
# Windows 上 start_new_session 被 CPython 静默忽略,绝不能再用它
|
||||
assert "start_new_session" not in kwargs
|
||||
if kwargs:
|
||||
assert kwargs["creationflags"] == pc._CREATE_NEW_PROCESS_GROUP
|
||||
|
||||
|
||||
def test_hard_kill_on_windows_uses_taskkill(monkeypatch):
|
||||
commands = []
|
||||
|
||||
class _FakeProcess:
|
||||
pid = 4242
|
||||
|
||||
def kill(self): # pragma: no cover - Windows 分支不应走到这里
|
||||
commands.append("Popen.kill")
|
||||
|
||||
monkeypatch.setattr(pc, "IS_WINDOWS", True)
|
||||
monkeypatch.setattr(pc.subprocess, "run",
|
||||
lambda argv, **kwargs: commands.append(argv))
|
||||
pc._hard_kill_tree(_FakeProcess())
|
||||
assert commands and commands[0][0] == "taskkill"
|
||||
assert "/PID" in commands[0] and "4242" in commands[0]
|
||||
|
||||
|
||||
def test_hard_kill_without_killpg_does_not_raise(monkeypatch):
|
||||
"""回归:Windows 上 os 没有 killpg/getpgid,旧代码会抛 AttributeError。"""
|
||||
killed = []
|
||||
|
||||
class _FakeProcess:
|
||||
pid = 4242
|
||||
|
||||
def kill(self):
|
||||
killed.append(True)
|
||||
|
||||
class _PosixlessOS:
|
||||
"""模拟 Windows:只有 getpid 之类,没有 killpg/getpgid。"""
|
||||
|
||||
monkeypatch.setattr(pc, "IS_WINDOWS", False)
|
||||
monkeypatch.setattr(pc, "os", _PosixlessOS())
|
||||
pc._hard_kill_tree(_FakeProcess())
|
||||
assert killed == [True]
|
||||
|
||||
|
||||
def test_reader_flushes_truncated_table_on_exit(stub_repo):
|
||||
@@ -628,8 +868,42 @@ def test_http_start_poll_stop(client):
|
||||
|
||||
status, body = call(port, "POST", "/api/train/stop", {})
|
||||
assert status == 200 and body["stopped"] is True
|
||||
# 等训练进程真正退出后再点一次:应返回 False 而不是报错
|
||||
assert wait_until(
|
||||
lambda: call(port, "GET", "/api/train/state?log_tail=5")[1]["job"]["status"]
|
||||
in ("stopped", "failed"), timeout=15)
|
||||
status, body = call(port, "POST", "/api/train/stop", {})
|
||||
assert status == 200 and body["stopped"] is False # 已结束,返回 False 而不是报错
|
||||
assert status == 200 and body["stopped"] is False
|
||||
|
||||
|
||||
def test_http_pause_resume_and_state(client, monkeypatch):
|
||||
monkeypatch.setenv("STUB_TRAIN_SLEEP", "0.6")
|
||||
_ui, port = client
|
||||
status, body = call(port, "POST", "/api/train/start",
|
||||
{"model": "output/hrl/ui/test/http-pause/q3.zip",
|
||||
"timesteps": 4096, "rollout_steps": 32, "batch_size": 32})
|
||||
assert status == 200
|
||||
|
||||
def status_now():
|
||||
return call(port, "GET", "/api/train/state?log_tail=20")[1]["job"]["status"]
|
||||
|
||||
def is_running():
|
||||
return status_now() == "running"
|
||||
|
||||
assert wait_until(is_running, timeout=10)
|
||||
status, body = call(port, "POST", "/api/train/pause", {})
|
||||
assert status == 200 and body["paused"] is True
|
||||
assert wait_until(lambda: status_now() == "paused", timeout=10), status_now()
|
||||
# 暂停中仍视为"有任务在跑":不允许再开一局
|
||||
status, body = call(port, "POST", "/api/train/start",
|
||||
{"model": "output/hrl/ui/test/http-pause2/q3.zip"})
|
||||
assert status == 409 and "已有训练任务" in body["error"]
|
||||
status, body = call(port, "POST", "/api/train/resume", {})
|
||||
assert status == 200 and body["resumed"] is True
|
||||
assert wait_until(is_running, timeout=10), status_now()
|
||||
status, body = call(port, "POST", "/api/train/stop", {})
|
||||
assert status == 200 and body["stopped"] is True
|
||||
assert wait_until(lambda: status_now() in ("stopped", "failed"), timeout=15)
|
||||
|
||||
|
||||
def test_http_rejects_second_job_while_running(client):
|
||||
@@ -661,7 +935,7 @@ def test_http_binds_loopback_only(client):
|
||||
|
||||
|
||||
def test_port_conflict_reports_actionable_error(client):
|
||||
ui, port = client
|
||||
_ui, port = client
|
||||
with pytest.raises(OSError) as excinfo:
|
||||
td.TrainingUI(ui_port=port)
|
||||
assert "ui-port" in str(excinfo.value)
|
||||
|
||||
@@ -36,6 +36,7 @@ def test_training_ui_frontend_logic(tmp_path):
|
||||
|
||||
result = subprocess.run(
|
||||
["node", str(HARNESS)], capture_output=True, text=True, timeout=120,
|
||||
check=False,
|
||||
env={**os.environ, "REPO": str(REPO),
|
||||
"DEMO_STATE": str(scenarios_path), "SCHEMA": str(schema_path)})
|
||||
assert result.returncode == 0, \
|
||||
|
||||
Reference in New Issue
Block a user