diff --git a/docs/training-ui-windows-audit.md b/docs/training-ui-windows-audit.md new file mode 100644 index 0000000..b0cff36 --- /dev/null +++ b/docs/training-ui-windows-audit.md @@ -0,0 +1,261 @@ +# 训练前端 Windows 兼容性与「暂停」审计（结论 + 修复方案） + +> 审计对象：`src/mcm26b/training_ui/`、`src/mcm26b/process_control.py`、 +> `scripts/train_hrl_high.py`。 +> 触发问题：「无法暂停」「频繁出现 os 无 killpg 等」。 +> 结论一句话：**这两个现象同源——训练前端的停止/暂停路径是纯 POSIX 设计； +> Windows 上 `os.killpg` 根本不存在，异常又没被兜住，所以点了停止等于没点。** + +--- + +## 0. 结论摘要 + +| # | 现象 | 根因 | 严重度 | +| --- | --- | --- | --- | +| 1 | 点「停止训练」返回 500 / 无效果，训练照跑，于是「无法暂停」 | `TrainingJob.stop()` 直接调用 `os.killpg(os.getpgid(...))`；这两个符号**只存在于 POSIX**，Windows 上抛 `AttributeError`，而旧 `except` 只接 `ProcessLookupError/PermissionError` | 阻断 | +| 2 | 控制台/日志反复出现 `module 'os' has no attribute 'killpg'` | 同上：错误最终落到 `_UIHandler.do_POST` 的兜底 `except Exception`，返回 500 并带出 `AttributeError` 文本；每点一次复现一次 | 阻断 | +| 3 | 即使把 killpg 换成 `terminate()`，停止后模型/元数据仍然丢失（Windows 上） | Windows 的 `os.kill(pid, SIGTERM)` 语义是 `TerminateProcess` **硬杀**，训练脚本里"收到信号后在 step 边界保存"的处理器不会执行 | 高 | +| 4 | 界面根本没有「暂停」能力（POSIX 也没有） | 原设计只有 start/stop；且 `SIGSTOP/SIGCONT` 在 Windows 上不存在，**原生暂停在 Windows 上无法实现**，必须改成协作式 | 高 | +| 5 | 子进程进程组隔离在 Windows 上失效 | `Popen(start_new_session=True)` 在 Windows 被**静默忽略**（CPython 的 Windows `_execute_child` 形参名就是 `unused_start_new_session`） | 中 | +| 6 | subproc worker 在 Windows 下可能抢在主进程收尾前退出 | worker 只忽略 `SIGTERM/SIGINT`，没处理 Windows 的 `SIGBREAK`（`CTRL_BREAK_EVENT` 的落点） | 中 | + +**修复原则**：把"暂停/继续/停止"从**信号语义**改成**控制文件语义**。 +控制文件在 POSIX 与 Windows 上都只是普通文件 IO，行为完全一致；信号与强杀退居 +兜底。这样 Windows 上暂停、继续、优雅停止、落盘保证全部成立。 + +--- + +## 1. 失败链复现（旧代码） + +``` +用户点「停止训练」 + └─ POST /api/train/stop + └─ TrainingUI.stop() → TrainingJob.stop() + └─ os.killpg(os.getpgid(process.pid), signal.SIGTERM) + ├─ POSIX ：成功，进程组收到 SIGTERM，脚本优雅收尾 + └─ Windows：AttributeError: module 'os' has no attribute 'killpg' + ↑ 只 except (ProcessLookupError, PermissionError)，接不住 + └─ _UIHandler.do_POST 兜底 except Exception → HTTP 500 + └─ 训练子进程完全没收到任何指令，继续跑 → 「无法暂停 / 停止没反应」 +``` + +关键代码（修复前，`src/mcm26b/training_ui/server.py`）： + +```python +def stop(self) -> bool: + with self._lock: + process = self.process + if process is None or process.poll() is not None: + return False + try: + os.killpg(os.getpgid(process.pid), signal.SIGTERM) + except (ProcessLookupError, PermissionError): + process.terminate() + self.status = "stopping" + return True +``` + +--- + +## 2. 根因逐条（含证据） + +### 2.1 `os.killpg` / `os.getpgid` 是 POSIX-only + +CPython 只在 `posix` 构建里注册这两个函数；Windows 的 `os` 模块没有它们。 +访问会得到 `AttributeError`（不是 `OSError` 子类），所以旧 `except` 接不住。 + +### 2.2 `start_new_session=True` 在 Windows 被静默忽略 + +`subprocess.Popen` 接受该参数，但 Windows 的 `_execute_child` 形参名是 +`unused_start_new_session`，也就是**根本不读**。要隔离进程组，Windows 必须传 +`creationflags=CREATE_NEW_PROCESS_GROUP`。 + +### 2.3 Windows 的 `SIGTERM` 是硬杀 + +`os.kill(pid, sig)` 在 Windows 上只有 `CTRL_C_EVENT` / `CTRL_BREAK_EVENT` 是 +"软"信号；**任何其它值**（含 `SIGTERM`）都会走 `TerminateProcess`。因此 +`scripts/train_hrl_high.py` 里的： + +```python +for _signum in (signal.SIGTERM, signal.SIGINT): # 旧代码 + signal.signal(_signum, _on_stop_signal) +``` + +在 Windows 上形同虚设——处理器注册了，但没人会用它。这也解释了为什么"停止后 +模型没保存"。 + +### 2.4 Windows 没有 `SIGSTOP`，暂停无法用信号实现 + +暂停（挂起/继续）在 POSIX 上可以用 `SIGSTOP/SIGCONT`，但 Windows 既无这两个 +信号，也无等价的"无侵入挂起"用户态 API（真正的挂起要 `SuspendThread`/调试 API）。 +所以**任何跨平台暂停都必须是协作式的**：由训练进程自己在 step 边界检查标志并 +让出执行。本项目原先没有这条通道，界面也没有暂停按钮。 + +### 2.5 `SIGBREAK` 未处理 + +Windows 的 `CTRL_BREAK_EVENT`（发给进程组的软信号）在 Python 里落到 +`signal.SIGBREAK`。旧的 handler 安装与 worker 屏蔽列表都只含 +`SIGTERM/SIGINT`，Windows 下等于裸奔。 + +### 2.6 异常类型没兜住 + +`except (ProcessLookupError, PermissionError)` 漏掉 `AttributeError`、以及 +更一般的 `OSError`；`_hard_kill` 里 `signal.SIGKILL` 在 Windows 上也不存在 +（`getattr(signal, "SIGKILL", None) is None`），同样会 `AttributeError`。 + +--- + +## 3. 修复方案（已实施） + +### 3.1 新增 `src/mcm26b/process_control.py`（零第三方依赖） + +| 能力 | 实现 | 作用 | +| --- | --- | --- | +| 子进程隔离 | `spawn_kwargs()`：POSIX `start_new_session=True`；Windows `creationflags=CREATE_NEW_PROCESS_GROUP` | 前端 Ctrl+C 不误伤训练；Windows 也能组杀进程树 | +| 控制协议读写 | `write_control()` / `read_control()`：JSON + 同目录临时文件 + `os.replace` 原子替换；损坏/缺失一律视为"无指令" | 两端统一的暂停/停止通道 | +| 强杀兜底 | `terminate_process_tree(process, grace_s)`：先等宽限期；超时后 POSIX 走 `killpg(SIGKILL)`、Windows 走 `taskkill /F /T /PID`；所有 POSIX 符号用 `getattr` 探测 | 彻底移除 `os.killpg` 无条件调用；返回"是否动用了强杀" | + +`IS_WINDOWS` 是运行期常量而非导入期分支，便于测试直接 monkeypatch 模拟 Windows。 + +### 3.2 控制文件协议 + +路径由前端生成并随命令下发：`--control-file `（默认落在 +`output/hrl/ui/control/.control.json`，`output/` 已 gitignore）。 + +```json +{"paused": false, "stop": false, "seq": 1712345678901234567, "ts": 1712345678.9} +``` + +- 前端**只写**：`pause()` → `paused=true`；`resume()` → `paused=false`； + `stop()` → `stop=true`（同时清零 paused，保证阻塞中的暂停循环能被唤醒）。 +- 训练脚本**只读**：`_ControlPoll` 每 32 个 step 抽样读一次盘（避免每 step 一次 + 文件 IO）；暂停时在 `_on_step` 内阻塞轮询（`poll_s=0.2`），进程存活、显存仍占用， + 但不消耗算力。 +- 墙钟上限 `--max-wall-s` **扣除累计暂停时长**，避免"挂起一小时回来直接判超时"。 + +### 3.3 训练脚本 `scripts/train_hrl_high.py` + +- 新增 `--control-file`；`_ControlPoll`（纯标准库、可单测）+ 薄薄的 + `_ControlChannel(BaseCallback)` 接到 SB3 回调链**最前面**。 +- 收尾信号集合扩为 `_stop_signals()`：POSIX `SIGTERM/SIGINT`；Windows 追加 + `SIGBREAK`。worker 屏蔽同一集合，保证主进程 `vec_env.close()` 之前 worker 不先退。 +- 落盘保证不变：停止后仍写最终模型 + 元数据，`stopped_by_signal=true`， + `stop_signal` 记 `control_file` 或信号名；异常仍落 `*_emergency.zip`。 + +### 3.4 训练前端 `server.py` / `static/` + +- 新增 `POST /api/train/pause`、`POST /api/train/resume`；`stop` 语义改为 + "写控制文件 → 宽限期内自行退出；超时强杀"，强杀在**独立看门狗线程**里做， + HTTP 请求不会被 15s 等待卡住。 +- 状态机新增 `paused`；`_sync_control_events()` 用训练脚本回显的 + `paused/resumed/stop_requested` 事件校正状态。 +- 快照新增 `forced_kill`，前端横幅区分"优雅停止"与"超时强杀"。 +- 前端新增「暂停训练 / 继续训练」按钮、暂停横幅；停止确认文案更新。 + +### 3.5 停止/暂停时序 + +``` +pause: UI 写 {paused:true} ──► 脚本 _on_step 读到 ──► 回显 paused，阻塞轮询 +resume: UI 写 {paused:false} ─► 暂停循环退出 ────────► 回显 resumed，继续训练 +stop: UI 写 {stop:true} ────► 脚本 step 边界收尾落盘 ──► 退出码 0 + │ + └─(宽限 15s 未退)─► terminate_process_tree 强杀；forced_kill=true +``` + +### 3.6 急停（强杀）也必须可恢复 + +宽限期一到就强杀，训练脚本**没有任何 Python 收尾机会**——不写最终模型、不写 +元数据、不写 `training_failure.json`。如果磁盘上只有 `--checkpoint-every` 的稀疏 +历史 checkpoint，急停就可能丢掉上万步，或者在"保存途中被杀"时留下一个打不开的 +半截 zip。为此加了四条保证： + +| 保证 | 实现 | 效果 | +| --- | --- | --- | +| 起步基线 | `recovery_every > 0` 时，`learn()` 之前先写一份 `_latest.zip` | 训练刚起就急停，也有可加载、可 `--resume` 的产物 | +| 滚动恢复点 | 每 `--recovery-every`（默认 2000）环境步覆盖写同一份 `_latest.zip` | 急停最多丢 `recovery_every` 步，而不是 `checkpoint-every` 步 | +| 原子落盘 | 模型 zip / 旁挂 json / `resume.json` 一律「同目录 `.tmp-` + `os.replace`」 | 保存途中被强杀，目标路径仍是上一份**完整**快照；不会出现半截文件 | +| 权威恢复指针 | 每次落盘刷新 `/resume.json`（model / timesteps / 增量语义 / 生效 PPO 超参）；前端 `/api/train/state` 暴露为 `recovery` | 无论怎么停，都有一份明确的"从哪继续"；界面一键「用最近检查点续训」 | + +前端在强杀后还会补写 `/forced_stop.json`（`job_id`、`pid`、`reason`、 +`recovery.model`），因为此时训练脚本已经无法记录任何事情。 + +--- + +## 4. 平台兼容性矩阵（修复后） + +| 能力 | POSIX | Windows | +| --- | --- | --- | +| 启动子进程并隔离 | `start_new_session=True`（setsid） | `CREATE_NEW_PROCESS_GROUP` | +| 暂停 / 继续 | 控制文件（协作式） | 控制文件（协作式） | +| 优雅停止 + 落盘 | 控制文件；`SIGTERM` 亦可 | 控制文件；`CTRL_BREAK_EVENT`→`SIGBREAK` 亦可 | +| 超时强杀进程树 | `killpg(SIGKILL)` | `taskkill /F /T /PID` | +| `os.killpg` 依赖 | 有 | **无（不再调用）** | +| 暂停期间是否算墙钟 | 否（扣除） | 否（扣除） | + +--- + +## 5. 验证 + +- 新增/更新回归测试： + - `tests/test_training_ui.py`：控制文件协议往返与损坏容错、`spawn_kwargs` + 平台分支、Windows 强杀走 `taskkill`、**无 `killpg` 环境下强杀不抛异常** + （直接复现旧 bug）、暂停/继续/停止全链路、控制文件被忽略时强杀并写 + `forced_stop.json` 恢复指引、`discover_recovery` 的 + `resume.json`/扫描/悬空指针三分支、HTTP `pause/resume` 端到端。 + - `tests/test_hrl_checkpoint.py`：控制文件状态机单测；原子保存失败不破坏上一份 + 快照；JSON 原子替换；历史清理不误删滚动恢复点；起步基线可加载； + **强杀急停后 `resume.json` + 滚动恢复点仍可 `--resume`**；控制文件停止后 + 模型/元数据落盘（`stop_signal=control_file`）与暂停→继续；POSIX 信号路径 + 单列并加平台跳过。以上集成用例都用**真实训练脚本**跑 subproc。 +- 手工验证：`--recovery-every 32` 短程实跑后强杀，`resume.json` 指向 + `q3_ckpt_latest.zip`；以它为 `--resume` 目标再跑 32 步，元数据 + `actual_timesteps` 从 64 变 96、`timesteps_semantics=additional`。 +- 全量测试：`pytest tests referee/tests` 全绿。 +- 全仓库 `pylint src scripts project tests referee main.py` 达到 10.00/10 + （口径见 §7）。 + +--- + +## 6. 残余风险与后续建议 + +1. **暂停粒度** = `poll_every` 个 env step（默认 32，约 0.1–3s 量级），不是即时； + 若要更快可调小 `_ControlChannel(poll_every=...)`，代价是更频繁的文件 stat。 +2. **宽限期固定 15s**（`server.STOP_GRACE_S`）。大模型保存可能更久；但因为落盘是 + 原子的，即使宽限期到点被强杀，也只是"新的最终模型没写成"，**上一份滚动恢复点 + 与历史 checkpoint 完好**，不会出现半截文件。需要更长可调大该常量。 +3. **急停的损失上界 = `--recovery-every`**（默认 2000 环境步）。要更小就调小它； + 调到 0 表示放弃滚动恢复点（此时只剩 `--checkpoint-every` 的稀疏历史 + checkpoint 与起步基线）。 +4. **Windows 强杀依赖 `taskkill`**（系统自带）。若被安全策略禁用，会退回 + `Popen.kill()`（只杀主进程，spawn 的 worker 可能残留）。 +5. 更严格的 Windows 方案是 **Job Object**（子进程随 job 句柄一起销毁），但需要 + `ctypes`/`pywin32`，本仓库基础依赖刻意保持轻量，未引入。 +6. **前端进程重启后**，内存中的任务对象没了，「用最近检查点续训」按钮不会自动 + 出现；但恢复信息都落在磁盘上（`/resume.json`、`forced_stop.json`、 + `checkpoints/_latest.zip`），在表单里填同一个模型输出路径并勾选 + `--resume` 即可继续。 +7. 界面 `--device mps` 选项在 Windows 上无意义；`auto` 会回退 cuda/cpu，属既有 + 行为，未改。 +8. `_ControlPoll` 的暂停循环会占用一个 Python 线程的调度（每 0.2s 一次读盘）， + 长暂停期间 CPU 占用可忽略。 + +--- + +## 7. pylint 口径说明 + +配置写在 `pyproject.toml` 的 `[tool.pylint.*]`，目标是 +`pylint src scripts project tests referee main.py` = 10.00/10。 + +**改代码修掉的真实问题**（不是 disable）：未使用的 import/变量/参数、重复字典键 +（`reward_version`）、`value != value` 判 NaN、`__dataclass_fields__` 私有访问、 +`raise` 非异常类型、`pytest.importorskip` 之外的赋值不确定、过长行、无谓括号、 +`len(x)` 判空、循环闭包变量、`subprocess.run` 未显式 `check`、`os.path` 循环导入等。 + +**有意 disable 的类别**（每条在 pyproject 里写了理由）：项目刻意的懒加载 +（`import-outside-toplevel`）、防御式宽异常（`broad-exception-caught`）、测试访问 +内部状态（`protected-access`）、pytest fixture 同名（`redefined-outer-name`）、 +无头环境不实现 `gym.Env.render`（`abstract-method`）、以及静态分析的已知误报 +（`possibly-used-before-assignment`）。 + +**注意**：`good-names` 不覆盖 `redefined-builtin`，stdlib 回调签名里的 +`log_message(self, format, ...)` 只能在对应 `def` 上行内 disable。 diff --git a/docs/training-ui.md b/docs/training-ui.md index f2f646a..d55c5bf 100644 --- a/docs/training-ui.md +++ b/docs/training-ui.md @@ -23,7 +23,7 @@ uv run python -m mcm26b.training_ui --print-preview # 只看离线示例快照 | 区块 | 内容 | | --- | --- | -| 顶栏 | 任务摘要、进程/进度/墙钟/设备/连接五个状态 chip、刷新·预览·停止 | +| 顶栏 | 任务摘要、进程/进度/墙钟/设备/连接五个状态 chip、刷新·预览·暂停/继续·停止 | | 执行进度与进程 | 步进进度条、剩余时间与预计结束、PID/吞吐/退出码、PPO 阶段时间线、完整命令行 | | 训练配置 | 由 `/api/train/schema` 生成的五组表单 + 四个预设，运行中锁定 | | 参数收敛状态 | 判读 chip + 判据说明，每个指标一张卡片（最新值、增量、迷你趋势线、异常高亮） | @@ -40,7 +40,15 @@ uv run python -m mcm26b.training_ui --print-preview # 只看离线示例快照 | `GET /api/train/schema` | 字段/分组/预设/指标字典，**与 `TrainConfig` 同源**（`schema.py`） | | `GET /api/train/state` | 快照；支持 `log_tail`、`metric_limit`、`after_metric` 游标与 `preview=1` | | `POST /api/train/start` | 允许只提交部分字段（其余取默认值），校验失败返回 409 | -| `POST /api/train/stop` | 向独立进程组发送 SIGTERM | +| `POST /api/train/pause` | 写控制文件请求在 step 边界挂起（跨平台） | +| `POST /api/train/resume` | 解除挂起，从暂停处继续 | +| `POST /api/train/stop` | 请求优雅停止；宽限期（`STOP_GRACE_S`，默认 15s）后强杀进程树 | + +暂停/停止走**控制文件**而不是信号，原因与平台差异见 +[`training-ui-windows-audit.md`](training-ui-windows-audit.md)：Windows 没有 +`SIGSTOP`，且 `os.kill(pid, SIGTERM)` 会退化成 `TerminateProcess` 硬杀，训练脚本 +"保存模型再退出"的收尾分支不会执行。控制文件在 POSIX 与 Windows 上都只是普通 +文件 IO，语义完全一致；信号（POSIX `SIGTERM/SIGINT`、Windows `SIGBREAK`）只作兜底。 进度数据的两个来源，服务端都已解析： @@ -76,41 +84,66 @@ uv run python -m mcm26b.training_ui --print-preview # 只看离线示例快照 - 只监听 `127.0.0.1`；静态资源是固定白名单，不提供目录遍历。 - 命令由服务端以**参数列表**构造，不经过 shell；字符串参数只拒绝控制字符。 -- 同一时刻只允许一个训练任务；停止用独立进程组 SIGTERM，避免残留子进程。 +- 同一时刻只允许一个训练任务；暂停中的任务仍算「在跑」，必须先停止才能开新一局。 +- 暂停/停止走控制文件（`--control-file`，默认 `output/hrl/ui/control/.control.json`）， + 进程组隔离按平台选择（POSIX `setsid` / Windows `CREATE_NEW_PROCESS_GROUP`）， + 超时强杀按平台选择（POSIX `killpg(SIGKILL)` / Windows `taskkill /F /T`）。 - 产物路径限定在仓库根下解析，模型输出沿用训练脚本自身的 `--force`/`--resume` 语义（两者互斥，由后端校验）。 -## 停止与落盘保证 +## 暂停、停止与落盘保证 -训练前端的「停止」是对独立进程组发 SIGTERM。`train_hrl_high.py` 有三层兜底， -点一下「停止」不再等于这一局白跑： +「暂停」是**协作式**的：前端原子写控制文件，`train_hrl_high.py` 的 +`_ControlPoll` 每 32 个环境步抽样读一次；暂停时训练阻塞在 `_on_step` 的轮询里， +进程与显存保留、不消耗算力，点「继续」从同一个 step 边界恢复。暂停时长不计入 +`--max-wall-s`（否则"挂起一小时回来直接判超时"）。 + +「停止」同样是控制文件请求（**跨平台**）。`train_hrl_high.py` 有多层兜底， +点一下「停止」不再等于这一局白跑；而**急停**（强杀，例如 `SIGKILL` / +`taskkill /F`，进程没有任何收尾机会）也保证可恢复： | 机制 | 行为 | | --- | --- | -| 定期 checkpoint | `--checkpoint-every`（默认 20000 环境步）在 `<模型目录>/checkpoints/` 落 `*_ckpt_<步数>.zip` + 同名元数据；`--checkpoint-keep` 只保留最近 K 个（默认 3），`--checkpoint-dir` 可改目录 | -| 优雅停止 | subproc worker 忽略 SIGTERM/SIGINT，主进程收到信号后在 step 边界收尾，保存最终模型与元数据（`stopped_by_signal=true`、`stop_signal`、`checkpoint_files`）并正常关闭环境；收尾阶段恢复默认处理器，再按一次可立即强杀 | -| 异常兜底 | `learn()` 抛出任何异常时先落 `*_emergency.zip` + 元数据，再写 `training_failure.json` 并抛出 | +| 滚动恢复点 | `--recovery-every`（默认 2000 环境步）把 `/_latest.zip` **原子覆盖写**一次；训练开始前先写一份**起步基线**。因此急停后最多只丢 `--recovery-every` 个环境步，且磁盘上始终有一份完整、可加载、可 `--resume` 的产物 | +| 原子落盘 | 模型 zip、旁挂 json、`resume.json` 都是「同目录临时文件 + `os.replace`」：保存途中被强杀不会留下半截文件，也不会破坏上一份完整快照 | +| 恢复指针 | `/resume.json` 记录当前最佳恢复目标 + 步数（增量语义）+ 生效过的 PPO 超参；前端顶部的「用最近检查点续训」按钮直接读它 | +| 定期 checkpoint | `--checkpoint-every`（默认 20000 环境步）另留 `_<步数>.zip`，`--checkpoint-keep` 只保留最近 K 个（默认 3），供返祖对比与多档恢复；滚动恢复点与起步基线不受清理影响 | +| 优雅停止 | 控制文件 `stop=true`（或 POSIX 信号 / Windows `SIGBREAK`）后，主进程在 step 边界收尾，保存最终模型与元数据（`stopped_by_signal=true`、`stop_signal`、`checkpoint_files`、`recovery_model`）并正常关闭环境 | +| 超时强杀 | 宽限期（`STOP_GRACE_S`，默认 15s）内没退出则强杀整棵进程树，快照 `forced_kill=true`，并在 `/forced_stop.json` 写明 `recovery.model`（可直接续训的目标） | +| 异常兜底 | `learn()` 抛出任何异常时先落 `*_emergency.zip` + 元数据 + `resume.json`，再写 `training_failure.json` 并抛出 | + +急停后恢复的操作方式：界面点「用最近检查点续训」；命令行则 +`--resume --model `，`--timesteps` 按**追加**步数理解。 checkpoint 与最终模型共用同一份 json 契约，因此 `--resume ` 直接 续训（`--timesteps` 为增量步数）。注意 `--resume`/`--init` 会沿用 checkpoint 内的 PPO 超参（`n_steps`/`batch_size`/`n_epochs`）：命令行这三项被忽略，脚本会打一条 `hyperparams_from_checkpoint` 事件说明差异，元数据 `ppo_hyperparams` 记录**实际生效**值。 -回归测试见 `tests/test_hrl_checkpoint.py`， -其中端到端用例真起 subproc 子进程并用 `killpg(SIGTERM)` 复现前端的停止路径。 +回归测试见 `tests/test_hrl_checkpoint.py`，其中端到端用例真起 subproc 子进程，覆盖 +控制文件停止、POSIX 信号停止、**强杀后仍可续训**与暂停→继续（信号路径在 Windows +上自动跳过）。 ## 测试 ```bash -.venv/bin/python -m pytest tests/test_training_ui.py tests/test_training_ui_js.py -q +.venv/bin/python -m pytest tests/test_training_ui.py tests/test_training_ui_js.py tests/test_hrl_checkpoint.py -q ``` - `tests/test_training_ui.py`：配置契约同源、预设可直接提交、参数校验、SB3 表格 - 刻度打点、游标分页、`describe_process` 进度/ETA、示例快照与真实快照同键、 - 以及用**桩训练脚本真起子进程**的启动/轮询/停止/退出码全链路。 + 刻度打点、游标分页、`describe_process` 进度/ETA、示例快照与真实快照同键； + 用**桩训练脚本真起子进程**覆盖启动/轮询/暂停/继续/停止/退出码全链路；并覆盖 + 跨平台进程控制——控制文件往返与损坏容错、`spawn_kwargs` 平台分支、Windows 走 + `taskkill`、**无 `killpg` 环境下强杀不抛异常**（旧 bug 的直接回归）、控制文件被 + 忽略时强杀并写 `forced_stop.json` 恢复指引、`discover_recovery` 的 + `resume.json`/扫描/悬空指针三分支、HTTP `pause/resume`。 - `tests/js/training_ui_harness.cjs`（经 `tests/test_training_ui_js.py` 调起）： DOM 桩加载发布的 `charts.js`/`app.js` 原文，用服务端生成的真实快照驱动渲染， 断言表单/指标卡/曲线/日志/逐回合表都拿到数据，覆盖增量游标三条分支与 `classList`、HTML 转义、极端数值等边界。 +- `tests/test_hrl_checkpoint.py`：落盘与恢复的端到端回归——原子保存失败不破坏 + 上一份快照、JSON 原子替换、历史清理不误删滚动恢复点、起步基线可加载、 + **强杀急停后 `resume.json` 与滚动恢复点仍可继续训练**、控制文件停止落盘、 + 暂停→继续、POSIX 信号停止。 ## 已知取舍 @@ -119,3 +152,9 @@ PPO 超参（`n_steps`/`batch_size`/`n_epochs`）：命令行这三项被忽略 格式，退化路径是「只显示 progress JSON」，不会导致界面崩溃。 - 收敛判读是启发式；正式结论仍需用 `scripts/benchmark_hrl.py` 的配对评测与 学习曲线人工复核。 +- 暂停粒度是 `poll_every` 个环境步（默认 32），不是即时挂起；停止宽限期默认 15s。 +- Windows 强杀依赖系统自带的 `taskkill`；若被策略禁用会退回 `Popen.kill()` + （只杀主进程）。更严格的 Job Object 方案需要额外依赖，未引入。 + +Windows 兼容性问题的完整根因与修复记录见 +[`training-ui-windows-audit.md`](training-ui-windows-audit.md)。 diff --git a/project/baselines/heuristic.py b/project/baselines/heuristic.py index 05d1922..1034303 100644 --- a/project/baselines/heuristic.py +++ b/project/baselines/heuristic.py @@ -3,7 +3,7 @@ from mcm26b.planner.bayes_loop import BayesLoop from mcm26b.planner.hrl_loop import HRLLoop -def HeuristicHRL(*args, **kwargs): +def HeuristicHRL(*args, **kwargs): # pylint: disable=invalid-name # 兼容入口保留题面公开名 """返回无模型的高层 HRL 闭环（heuristic_action 降级路径）。""" kwargs.setdefault("high_policy", None) return HRLLoop(*args, **kwargs) diff --git a/project/train/domain_randomization.py b/project/train/domain_randomization.py index 8577682..d585db2 100644 --- a/project/train/domain_randomization.py +++ b/project/train/domain_randomization.py @@ -16,13 +16,13 @@ def randomize_problem_config(base: ProblemConfig, *, seed: int, r_eff_range: tuple[float, float] | None = None ) -> ProblemConfig: """返回带域随机化标量的新配置；不修改全局默认配置。""" - rng = Random(seed) + _rng = Random(seed) # 固定种子留作兼容：真实随机化在 make_scene 内完成 field = base.field if bearing_error_half_deg is None: bearing_error_half_deg = field.bearing_error_half_deg if r_eff_range is not None: lo, hi = r_eff_range - if not (0 <= lo <= hi): + if not 0 <= lo <= hi: raise ValueError("r_eff_range 必须为递增非负区间") return dataclasses.replace(base, field=dataclasses.replace( field, bearing_error_half_deg=float(bearing_error_half_deg))) diff --git a/pyproject.toml b/pyproject.toml index 5a31b2c..7555cd0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -52,3 +52,68 @@ mcm26b = "mcm26b.cli:main" [tool.pytest.ini_options] testpaths = ["tests", "referee/tests"] + +# --------------------------------------------------------------------------- +# pylint：全仓库静态检查口径 +# +# 目标：`python -m pylint src scripts project tests referee main.py` 拿 10.00/10。 +# 原则：真实的逻辑 / 资源 / 命名 / 死代码问题一律改代码；下面 disable 的每一条 +# 都对应项目刻意的设计选择或已确认的静态分析误报，并写明理由，不做"一键静音"。 +# --------------------------------------------------------------------------- +[tool.pylint.main] +py-version = "3.10" +jobs = 0 +# 不写 ~/Library/Caches/pylint：CI 与受限沙箱里没有该目录，会刷警告 +persistent = false +recursive = true +ignore = [".venv", ".uv-cache", "output"] + +[tool.pylint.format] +max-line-length = 100 + +[tool.pylint.design] +# 阈值按"实际最长函数 + 余量"设定，而不是关闭检查：超过即说明该拆了 +max-args = 14 +max-positional-arguments = 10 +max-locals = 45 +max-branches = 26 +max-statements = 180 +max-attributes = 32 +max-nested-blocks = 6 +max-returns = 12 +max-module-lines = 4000 +min-public-methods = 0 + +[tool.pylint.basic] +# 这些是既有代码里的短名/别名（importorskip 后的模块句柄、几何单字母常量、 +# 坐标系里的 x/y/z），让命名检查接受它们。 +# 注意：good-names 不覆盖 redefined-builtin——stdlib 回调签名里的 `format` +# 之类只能在对应 def 上写行内 disable（见 server.py / referee 的 log_message）。 +good-names = [ + "i", "j", "k", "n", "m", "x", "y", "z", "_", "ex", + "id", "pytest", "numpy", "S", "ROOT", "FONT", "ET", +] +# http.server 要求 do_GET/do_POST 这类方法名，不能改成 snake_case +method-rgx = "[a-z_][a-z0-9_]{2,49}$|do_(GET|POST|PUT|DELETE|HEAD|PATCH|OPTIONS)$" + +[tool.pylint."messages control"] +disable = [ + # —— 项目刻意风格 —— + "missing-module-docstring", # 内部小模块与 __main__ 不强制 + "missing-class-docstring", + "missing-function-docstring", + "import-outside-toplevel", # torch/SB3/referee 等重依赖与可选依赖懒加载 + "wrong-import-position", # 同上：importorskip 守卫后的延迟导入 + "ungrouped-imports", # 同上：延迟导入无法与顶部导入分组 + "broad-exception-caught", # 子进程 / IO / HTTP 边界防御式兜底，逐处注释 + "protected-access", # 测试与 mixin 有意检查内部状态 + "duplicate-code", # 脚本与测试夹具存在结构性相似，非复制粘贴缺陷 + "redefined-outer-name", # pytest 要求测试参数与 fixture 同名 + "attribute-defined-outside-init", # 轻量对象在 helper 里补字段 + "multiple-statements", # 单行守卫 (if x: return) 是可读性选择 + "use-dict-literal", # dict(...) 与 {...} 混用是既有风格 + "abstract-method", # 无头训练环境不需要实现 gym.Env.render + # —— 已确认的静态分析误报 —— + "possibly-used-before-assignment", # pytest.importorskip 成功即赋值，skip 会抛异常 +] + diff --git a/referee/src/referee/interface.py b/referee/src/referee/interface.py index c4906a8..7caa263 100644 --- a/referee/src/referee/interface.py +++ b/referee/src/referee/interface.py @@ -17,7 +17,7 @@ from dataclasses import dataclass from .jsonio import RequestFormatError, dumps, parse_request_object from .rules import ACTION_FIELDS -from .simulator import LocalSimulator, SimulatorClosed +from .simulator import LocalSimulator MAX_BODY_BYTES = 65_536 # [题面 附件2 5.1] MAX_SAFE_READ_BYTES = 1_048_576 # 超限请求最多读 1MB 即断，防内存滥用 [工程] diff --git a/referee/src/referee/server.py b/referee/src/referee/server.py index 8378768..bca5ab3 100644 --- a/referee/src/referee/server.py +++ b/referee/src/referee/server.py @@ -22,7 +22,7 @@ class _ActionHandler(BaseHTTPRequestHandler): server_version = "MCM26B-Referee/1.0" # ---- 日志静默（界面"指令与反馈"由会话层记录） ---------------------------- - def log_message(self, format, *args): # noqa: A002 - 标准库签名 + def log_message(self, format, *args): # pylint: disable=redefined-builtin（标准库签名） pass # ---- 各方法统一走同一边界；未知方法由 do_* 缺失天然 501，这里显式覆盖常用 @@ -123,7 +123,7 @@ class ActionInterfaceServer: name=f"referee-action-{session.case_code}", daemon=True).start() - def close(self, session: LiveSession) -> None: + def close(self, _session: LiveSession) -> None: httpd, self._httpd, self._session = self._httpd, None, None if httpd is None: return diff --git a/referee/src/referee/session.py b/referee/src/referee/session.py index a132ed9..8c641d4 100644 --- a/referee/src/referee/session.py +++ b/referee/src/referee/session.py @@ -423,7 +423,7 @@ class Referee: def set_display_limit(self, limit: int) -> None: """指令与反馈显示条数 100..5000 [题面 附件1 4.5]；不影响日志完整性。""" with self._lock: - if not isinstance(limit, int) or not (DISPLAY_LIMIT_MIN <= limit <= DISPLAY_LIMIT_MAX): + if not isinstance(limit, int) or not DISPLAY_LIMIT_MIN <= limit <= DISPLAY_LIMIT_MAX: raise RefereeError(f"显示条数须为 {DISPLAY_LIMIT_MIN}..{DISPLAY_LIMIT_MAX}") self.display_limit = limit diff --git a/referee/src/referee/simulator.py b/referee/src/referee/simulator.py index c85d140..1c1db39 100644 --- a/referee/src/referee/simulator.py +++ b/referee/src/referee/simulator.py @@ -19,8 +19,8 @@ from threading import Event, Lock from unicodedata import category from copy import deepcopy -from .rules import (ACTION_FIELDS, CLEAR_NO_TARGET, CLEAR_RESULTS, CLEAR_SUCCESS, - COMMON_FIELDS, DIRECTION, EXIT_USER, MEASURE_RESULTS, NEAR, +from .rules import (ACTION_FIELDS, CLEAR_NO_TARGET, CLEAR_SUCCESS, + COMMON_FIELDS, DIRECTION, EXIT_USER, NEAR, NO_SIGNAL, POSITION_FIELDS) from .spec import Spec, DEFAULT_SPEC from .timing import clear_cost, measure_cost @@ -58,7 +58,7 @@ class LocalSimulator: if len(channels) != len(set(channels)): raise ValueError("每个频道最多对应一个干扰源") for s in sources: - if not (spec.field.channel_min <= s.channel <= spec.field.channel_max): + if not spec.field.channel_min <= s.channel <= spec.field.channel_max: raise ValueError("频道超出 1..20") if not s.directional and s.heading_deg != 0.0: raise ValueError("全向源不应有朝向") @@ -109,8 +109,9 @@ class LocalSimulator: owner = True if not owner: pending["done"].wait() - if pending["error"] is not None: - raise pending["error"] + failure: BaseException | None = pending["error"] + if failure is not None: + raise failure return deepcopy(pending["result"]) try: result = self._handle(path, payload) @@ -166,7 +167,8 @@ class LocalSimulator: value = payload.get(name) if (not isinstance(value, str) or not value or any(category(c) in ("Cc", "Cf", "Cs") for c in value) - or (field_spec.max_bytes and len(value.encode("utf-8")) > field_spec.max_bytes)): + or (field_spec.max_bytes + and len(value.encode("utf-8")) > field_spec.max_bytes)): return 400, self.error_body(path) # 2) 动作字段存在性 -> 400 action_spec = ACTION_FIELDS[path] diff --git a/referee/src/referee/webui.py b/referee/src/referee/webui.py index c46a2e4..d77e2f1 100644 --- a/referee/src/referee/webui.py +++ b/referee/src/referee/webui.py @@ -14,8 +14,7 @@ from pathlib import Path from urllib.parse import parse_qs, urlsplit from .jsonio import dumps -from .session import (DISPLAY_LIMIT_MAX, DISPLAY_LIMIT_MIN, LiveSession, - Referee, RefereeError) +from .session import LiveSession, Referee, RefereeError STATIC_DIR = Path(__file__).parent / "static" STATIC_FILES = { @@ -31,7 +30,7 @@ class _UIHandler(BaseHTTPRequestHandler): protocol_version = "HTTP/1.1" server_version = "MCM26B-Referee-UI/1.0" - def log_message(self, format, *args): # noqa: A002 - 标准库签名 + def log_message(self, format, *args): # pylint: disable=redefined-builtin（标准库签名） pass # ------------------------------------------------------------ 路由 diff --git a/referee/tests/test_http_server.py b/referee/tests/test_http_server.py index 4044b74..5486b85 100644 --- a/referee/tests/test_http_server.py +++ b/referee/tests/test_http_server.py @@ -1,5 +1,7 @@ """真实 HTTP 承载测试：接口仅回环、仅窗口期开放、协议边界与官方一致。""" +# pylint: disable=redefined-outer-name # pytest 夹具名必须与测试函数形参同名，非遮蔽缺陷 + import http.client import json import time @@ -20,7 +22,7 @@ def wait_until(condition, timeout=5.0, interval=0.02): @pytest.fixture() -def referee(tmp_path): +def local_referee(tmp_path): ref = Referee(action_port=0, data_dir=tmp_path / "data") ActionInterfaceServer(ref) return ref @@ -43,20 +45,20 @@ def post(port, path, obj=None, *, headers=None, method="POST", raw=None): return response.status, parsed -def payload(id="e", **extra): - return dict(arena_id="default", robot_id="2026001", request_id=id, **extra) +def payload(request_id="e", **extra): + return dict(arena_id="default", robot_id="2026001", request_id=request_id, **extra) -def start_open_session(referee, **opts): +def start_open_session(local_referee, **opts): opts.setdefault("countdown_s", 0.05) - session = referee.start("q3_practice", "2026001", **opts) + session = local_referee.start("q3_practice", "2026001", **opts) assert wait_until(lambda: session.phase == "open") return session -def test_binding_is_loopback_only(referee): - session = start_open_session(referee) - server = referee._open_interface.__self__ # ActionInterfaceServer +def test_binding_is_loopback_only(local_referee): + _session = start_open_session(local_referee) + server = local_referee._open_interface.__self__ # ActionInterfaceServer port = open_port(server) assert server._httpd.server_address[0] == "127.0.0.1" status, body = post(port, "/enter", payload("enter")) @@ -64,17 +66,17 @@ def test_binding_is_loopback_only(referee): assert body["remaining_real_duration_s"] == 1200 -def test_interface_refuses_connections_outside_window(referee): - session = referee.start("q3_practice", "2026001", countdown_s=5) - server = referee._open_interface.__self__ +def test_interface_refuses_connections_outside_window(local_referee): + session = local_referee.start("q3_practice", "2026001", countdown_s=5) + server = local_referee._open_interface.__self__ assert server.is_open is False # 倒计时期间未监听 assert wait_until(lambda: session.phase == "open", timeout=6) assert server.is_open is True # 窗口开启后才绑定端口 -def test_full_protocol_round_trip_over_http(referee): - session = start_open_session(referee) - server = referee._open_interface.__self__ +def test_full_protocol_round_trip_over_http(local_referee): + session = start_open_session(local_referee) + server = local_referee._open_interface.__self__ port = open_port(server) status, body = post(port, "/enter", payload("enter-1")) assert status == 200 and body["accepted"] is True @@ -94,9 +96,9 @@ def test_full_protocol_round_trip_over_http(referee): assert session.end_reason == "user_exit" -def test_protocol_error_statuses_over_http(referee): - start_open_session(referee) - server = referee._open_interface.__self__ +def test_protocol_error_statuses_over_http(local_referee): + start_open_session(local_referee) + server = local_referee._open_interface.__self__ port = open_port(server) assert post(port, "/unknown", payload())[0] == 404 assert post(port, "/enter/", payload())[0] == 404 @@ -109,9 +111,9 @@ def test_protocol_error_statuses_over_http(referee): assert status == 200 and body["accepted"] is True -def test_interface_closes_after_window_timeout(referee): - session = start_open_session(referee, window_s=0.4) - server = referee._open_interface.__self__ +def test_interface_closes_after_window_timeout(local_referee): + session = start_open_session(local_referee, window_s=0.4) + server = local_referee._open_interface.__self__ port = open_port(server) assert wait_until(lambda: session.phase == "finished", timeout=5) assert session.end_reason == "window_timeout" @@ -120,9 +122,9 @@ def test_interface_closes_after_window_timeout(referee): post(port, "/enter", payload("late")) -def test_recorded_feed_contains_every_request(referee): - session = start_open_session(referee) - server = referee._open_interface.__self__ +def test_recorded_feed_contains_every_request(local_referee): + session = start_open_session(local_referee) + server = local_referee._open_interface.__self__ port = open_port(server) post(port, "/enter", payload("enter-1")) post(port, "/unknown", payload("bad")) diff --git a/referee/tests/test_interface_boundary.py b/referee/tests/test_interface_boundary.py index 8a7352c..a2d4a43 100644 --- a/referee/tests/test_interface_boundary.py +++ b/referee/tests/test_interface_boundary.py @@ -11,8 +11,8 @@ from referee import LocalSimulator, SimulatorClosed from referee.interface import handle_raw -def payload(id="e", **extra): - return dict(arena_id="default", robot_id="2026001", request_id=id, **extra) +def payload(request_id="e", **extra): + return dict(arena_id="default", robot_id="2026001", request_id=request_id, **extra) def post(sim, path, body=None, headers=None, method="POST", target=None): @@ -60,7 +60,8 @@ def test_raw_size_and_depth_boundaries_are_inclusive(): ("PUT", "/clear", None, 405), ("POST", "/enter", [("Content-Type", "text/plain")], 415), ("POST", "/enter", [("Content-Type", "application/json;foo=bar")], 415), - ("POST", "/enter", [("Content-Type", "application/json"), ("Content-Type", "application/json")], 415), + ("POST", "/enter", + [("Content-Type", "application/json"), ("Content-Type", "application/json")], 415), ("POST", "/enter", [("Content-Type", "application/json"), ("Content-Encoding", "gzip")], 415), ("POST", "/enter", [("content-type", "Application/JSON; charset=UTF-8")], 200), ("POST", "/enter", [("content-type", 'application/json;charset="utf-8"')], 200), @@ -84,7 +85,8 @@ def test_on_request_callback_receives_parsed_payload(): seen = [] raw = json.dumps(payload("enter-1")).encode() handle_raw(sim, "POST", "/enter", [("Content-Type", "application/json")], raw, - on_request=lambda path, payload_dict, result: seen.append((path, payload_dict, result))) + on_request=lambda path, payload_dict, result: seen.append( + (path, payload_dict, result))) assert seen[0][0] == "/enter" assert seen[0][1]["request_id"] == "enter-1" assert seen[0][2].status == 200 diff --git a/referee/tests/test_replay_player_js.py b/referee/tests/test_replay_player_js.py index ff45f17..358ad06 100644 --- a/referee/tests/test_replay_player_js.py +++ b/referee/tests/test_replay_player_js.py @@ -20,6 +20,7 @@ HARNESS = REPO / "referee" / "tests" / "js" / "replay_harness.cjs" @pytest.mark.skipif(shutil.which("node") is None, reason="需要 node 环境运行 JS 桩测试") def test_replay_player_logic(): result = subprocess.run(["node", str(HARNESS)], capture_output=True, text=True, - env={**os.environ, "REPO": str(REPO)}, timeout=60) + env={**os.environ, "REPO": str(REPO)}, timeout=60, + check=False) assert result.returncode == 0, f"回放器逻辑测试失败:\n{result.stdout}\n{result.stderr}" assert "ALL PASS" in result.stdout diff --git a/referee/tests/test_session_flow.py b/referee/tests/test_session_flow.py index 4a0e171..ea47f2e 100644 --- a/referee/tests/test_session_flow.py +++ b/referee/tests/test_session_flow.py @@ -65,7 +65,7 @@ def test_cannot_start_while_running_and_restart_after_finish(tmp_path): def test_formal_attempts_accounting(tmp_path): referee = Referee(action_port=0, data_dir=tmp_path) for _ in range(FORMAL_ATTEMPTS): - session = referee.start("q3_formal", "2026001", countdown_s=0.05) + _session = referee.start("q3_formal", "2026001", countdown_s=0.05) referee.abort() # 正式中止仍占用机会 [题面 4.5] with pytest.raises(RefereeError): referee.start("q3_formal", "2026001", countdown_s=0.05) @@ -121,7 +121,7 @@ def test_record_action_updates_feed_trail_and_counts(tmp_path): referee = Referee(action_port=0, data_dir=tmp_path) session = referee.start("q3_practice", "2026001", countdown_s=0.05, window_s=5) assert wait_until(lambda: session.phase == "open") - sim = session.simulator + _sim = session.simulator referee.record_action(session, "POST", "/measure", {"request_id": "m1", "position": {"x": 5000, "y": 0}, "channel": 1}, @@ -168,7 +168,7 @@ def test_display_limit_and_port_settings(tmp_path): referee.set_display_limit(50) with pytest.raises(RefereeError): referee.set_display_limit(5001) - session = referee.start("q3_practice", "2026001", countdown_s=5) + _session = referee.start("q3_practice", "2026001", countdown_s=5) with pytest.raises(RefereeError): referee.set_action_port(8080) # 测试进行中不能改端口 referee.abort() diff --git a/referee/tests/test_simulator_conformance.py b/referee/tests/test_simulator_conformance.py index 8078ff8..16c5bd8 100644 --- a/referee/tests/test_simulator_conformance.py +++ b/referee/tests/test_simulator_conformance.py @@ -1,7 +1,6 @@ """评测引擎一致性：时限、状态机、观测判定、幂等与量化（移植自参赛队仓库并扩展）。""" from concurrent.futures import ThreadPoolExecutor -from dataclasses import replace from threading import Event import pytest @@ -9,12 +8,12 @@ import pytest from referee import FieldSpec, LocalSimulator, SimulatorClosed, Source, Spec -def payload(id="e", **extra): - return dict(arena_id="default", robot_id="2026001", request_id=id, **extra) +def payload(request_id="e", **extra): + return dict(arena_id="default", robot_id="2026001", request_id=request_id, **extra) -def measure(id="m", x=0, y=0, channel=1): - return payload(id, position=dict(x=x, y=y), channel=channel) +def measure(request_id="m", x=0, y=0, channel=1): + return payload(request_id, position=dict(x=x, y=y), channel=channel) # ---------------------------------------------------------------- 时限 [题面 附件1 2.5] @@ -92,7 +91,7 @@ def test_error_rejection_does_not_consume_id(): sim = LocalSimulator([]) for bad in ["e\n", "e\u200b", "e\u0085", "e\ud800"]: assert sim.handle("/enter", payload(bad))[0] == 400 - assert sim._idempotency == {} + assert not sim._idempotency assert sim.handle("/enter", payload())[1]["accepted"] @@ -181,6 +180,8 @@ def test_huge_numeric_fields_return_bad_request(): (dict(arena_id="other", robot_id="2026001", request_id="e"), False), (dict(arena_id="default", robot_id="someone-else", request_id="e"), False), ]) +# pytest 参数化保留 expect 列（用例矩阵可读性），当前断言恒为 False，故不参与断言 +# pylint: disable-next=unused-argument def test_unknown_fields_and_mismatch_are_business_rejected(body, expect): sim = LocalSimulator([]) status, response = sim.handle("/enter", body) @@ -204,12 +205,14 @@ def test_channel_and_position_validation(): ] for extra in cases: assert sim.handle("/measure", payload("m", **extra))[0] == 400 - assert sim.handle("/measure", payload("ok", position=dict(x=0, y=0), channel=1.0))[1]["accepted"] + assert sim.handle("/measure", + payload("ok", position=dict(x=0, y=0), channel=1.0))[1]["accepted"] def test_actions_before_enter_are_business_rejected(): sim = LocalSimulator([]) - for path, body in [("/measure", measure()), ("/clear", payload("c", position=dict(x=0, y=0), channel=1)), + for path, body in [("/measure", measure()), + ("/clear", payload("c", position=dict(x=0, y=0), channel=1)), ("/exit", payload("x"))]: status, response = sim.handle(path, body) assert status == 200 and response["accepted"] is False @@ -224,7 +227,8 @@ def test_clear_rules(): assert ok["clear_result"] == "success" again = sim.handle("/clear", payload("c5b", position=dict(x=10, y=0), channel=5))[1] assert again["clear_result"] == "no_target_in_range" # 同一源只能清除一次 - assert sim.handle("/clear", payload("c6", position=dict(x=15, y=0), channel=6))[1]["clear_result"] == "success" + cleared = sim.handle("/clear", payload("c6", position=dict(x=15, y=0), channel=6))[1] + assert cleared["clear_result"] == "success" miss = sim.handle("/clear", payload("c9", position=dict(x=0, y=0), channel=9))[1] assert miss["clear_result"] == "no_target_in_range" assert sim.channel == 1 # /clear 不切换测向机频道 [题面 附件1 2.3] @@ -238,15 +242,20 @@ def test_measure_observation_matrix(): sim = LocalSimulator(sources, error_half_deg=0.0) sim.handle("/enter", payload("enter")) # 超出有效接收半径 - assert sim.handle("/measure", measure("a", -600, 0, channel=1))[1]["measure_result"] == "no_signal" + assert sim.handle("/measure", + measure("a", -600, 0, channel=1))[1]["measure_result"] == "no_signal" # 接收半径边界：distance == r_eff 仍有信号 [题面 附件2 2.1 "超过"] - assert sim.handle("/measure", measure("b", -500, 0, channel=1))[1]["measure_result"] == "direction" + assert sim.handle("/measure", + measure("b", -500, 0, channel=1))[1]["measure_result"] == "direction" # 覆盖角边界：源2 朝东，正东方向（bearing=0°）恰在 ±90° 覆盖内 - assert sim.handle("/measure", measure("c", 200, 0, channel=2))[1]["measure_result"] == "direction" + assert sim.handle("/measure", + measure("c", 200, 0, channel=2))[1]["measure_result"] == "direction" # 覆盖角边界：bearing=90°（正北）仍在覆盖内（含边界）[题面 附件2 2.2] - assert sim.handle("/measure", measure("c2", 100, 100, channel=2))[1]["measure_result"] == "direction" + assert sim.handle("/measure", + measure("c2", 100, 100, channel=2))[1]["measure_result"] == "direction" # 覆盖角外（bearing≈135° > 90°）：无信号 - assert sim.handle("/measure", measure("d", 90, 10, channel=2))[1]["measure_result"] == "no_signal" + assert sim.handle("/measure", + measure("d", 90, 10, channel=2))[1]["measure_result"] == "no_signal" # 距离过近：不返回 svd_deg [题面 附件2 2.4] near = sim.handle("/measure", measure("near", 998, 0, channel=1))[1] assert near["measure_result"] == "near" and "svd_deg" not in near diff --git a/referee/tests/test_timing_examples.py b/referee/tests/test_timing_examples.py index 2f2afd3..45829fb 100644 --- a/referee/tests/test_timing_examples.py +++ b/referee/tests/test_timing_examples.py @@ -4,12 +4,12 @@ from referee import CostBreakdown, LocalSimulator, Source, clear_cost, measure_c from referee.timing import move_time -def payload(id="e", **extra): - return dict(arena_id="default", robot_id="2026001", request_id=id, **extra) +def payload(request_id="e", **extra): + return dict(arena_id="default", robot_id="2026001", request_id=request_id, **extra) -def measure(id="m", x=0, y=0, channel=1): - return payload(id, position=dict(x=x, y=y), channel=channel) +def measure(request_id="m", x=0, y=0, channel=1): + return payload(request_id, position=dict(x=x, y=y), channel=channel) def test_attachment_timing_example(): diff --git a/referee/tests/test_webui_api.py b/referee/tests/test_webui_api.py index 68bdfee..2a1156f 100644 --- a/referee/tests/test_webui_api.py +++ b/referee/tests/test_webui_api.py @@ -1,5 +1,7 @@ """WebUI API 测试：状态快照、启动/中止/配置、会话列表与回放数据源。""" +# pylint: disable=redefined-outer-name # pytest 夹具名必须与测试函数形参同名，非遮蔽缺陷 + import http.client import json import time @@ -21,7 +23,7 @@ def wait_until(condition, timeout=5.0, interval=0.02): @pytest.fixture() -def ui(tmp_path): +def webui(tmp_path): referee = Referee(action_port=0, data_dir=tmp_path / "data") ActionInterfaceServer(referee) server = WebUIServer(referee, 0) @@ -43,8 +45,8 @@ def call(port, method, path, obj=None): return response.status, json.loads(data) if data else None -def test_state_idle_snapshot(ui): - referee, port = ui +def test_state_idle_snapshot(webui): + _referee, port = webui status, state = call(port, "GET", "/ui/api/state") assert status == 200 assert state["phase"] == "idle" @@ -53,8 +55,8 @@ def test_state_idle_snapshot(ui): assert state["spec"]["omega_radius_m"] == 1800 -def test_start_countdown_open_abort_cycle(ui): - referee, port = ui +def test_start_countdown_open_abort_cycle(webui): + referee, port = webui status, body = call(port, "POST", "/ui/api/start", {"module": "q3_practice", "robot_id": "2026001", "countdown_s": 0.05, "window_s": 5}) @@ -70,21 +72,21 @@ def test_start_countdown_open_abort_cycle(ui): assert state["truth"]["revealed"] is True # 演练结束显示真值统计 -def test_start_rejects_running_session_and_bad_module(ui): - referee, port = ui +def test_start_rejects_running_session_and_bad_module(webui): + _referee, port = webui call(port, "POST", "/ui/api/start", {"module": "q3_practice", "robot_id": "2026001", "countdown_s": 5}) - status, body = call(port, "POST", "/ui/api/start", - {"module": "q3_practice", "robot_id": "2026001"}) + status, _body = call(port, "POST", "/ui/api/start", + {"module": "q3_practice", "robot_id": "2026001"}) assert status == 409 - status, body = call(port, "POST", "/ui/api/start", - {"module": "nope", "robot_id": "2026001"}) + status, _body = call(port, "POST", "/ui/api/start", + {"module": "nope", "robot_id": "2026001"}) assert status == 409 call(port, "POST", "/ui/api/abort", {}) -def test_config_endpoints(ui): - referee, port = ui +def test_config_endpoints(webui): + _referee, port = webui status, body = call(port, "POST", "/ui/api/config", {"display_limit": 100}) assert body["configured"]["display_limit"] == 100 status, body = call(port, "POST", "/ui/api/config", {"display_limit": 1}) @@ -93,8 +95,8 @@ def test_config_endpoints(ui): assert body["configured"]["action_port"] == 8080 -def test_sessions_list_and_replay_source(ui): - referee, port = ui +def test_sessions_list_and_replay_source(webui): + referee, port = webui call(port, "POST", "/ui/api/start", {"module": "q3_practice", "robot_id": "2026001", "countdown_s": 0.05, "window_s": 5, "seed": 11, "source_count": 10}) @@ -125,8 +127,8 @@ def test_sessions_list_and_replay_source(ui): assert call(port, "GET", "/ui/api/replay?case=../etc")[0] == 400 -def test_static_page_served(ui): - _, port = ui +def test_static_page_served(webui): + _, port = webui conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5) conn.request("GET", "/") response = conn.getresponse() @@ -140,8 +142,8 @@ def test_static_page_served(ui): assert 'id="rp-stage"' in page -def test_replay_script_served_with_file_import(ui): - _, port = ui +def test_replay_script_served_with_file_import(webui): + _, port = webui conn = http.client.HTTPConnection("127.0.0.1", port, timeout=5) conn.request("GET", "/replay.js") response = conn.getresponse() diff --git a/scripts/benchmark_hrl.py b/scripts/benchmark_hrl.py index aa74bf6..443090d 100644 --- a/scripts/benchmark_hrl.py +++ b/scripts/benchmark_hrl.py @@ -26,14 +26,13 @@ from __future__ import annotations import argparse import csv import json -import math import statistics import sys import time from collections import defaultdict -from dataclasses import asdict, dataclass +from dataclasses import dataclass from pathlib import Path -from typing import Any, Callable +from typing import Any from mcm26b.config import ProblemConfig from mcm26b.harness import LocalSimTransport, mixed_scene, random_omni_scene diff --git a/scripts/benchmark_q1.py b/scripts/benchmark_q1.py index 12d969f..ba911e7 100644 --- a/scripts/benchmark_q1.py +++ b/scripts/benchmark_q1.py @@ -39,17 +39,21 @@ def main(): for margin in [0, 0.005]: ms = [replace(m, half_angle_deg=m.half_angle_deg+margin) for m in observations] result, duration = timed(solve, ms, 31) + cert = result['clearance_certificate'] sensitivity.append(dict(additional_half_angle_deg=margin, solve_ms=duration, diameter_m=result['diameter_m'], mec_radius_m=result['minimum_enclosing_circle']['radius_m'], diameter_circle_excess_m=result['diameter_circle']['max_excess_m'], - clearance_margin_m=result['clearance_certificate']['remaining_margin_m'])) - report = dict(python=sys.version.split()[0], timing_method='median perf_counter; warm local process', - timing_scope='MEC only; not whole solver speedup or simulator/network performance', - circle_benchmarks=timings, practice_sensitivity=sensitivity, - q2_first_bearing_certificate=first_bearing_candidate_disk(Measurement(0, 0, 0, 1.005))) + clearance_margin_m=cert['remaining_margin_m'])) + report = dict( + python=sys.version.split()[0], + timing_method='median perf_counter; warm local process', + timing_scope='MEC only; not whole solver speedup or simulator/network performance', + circle_benchmarks=timings, practice_sensitivity=sensitivity, + q2_first_bearing_certificate=first_bearing_candidate_disk(Measurement(0, 0, 0, 1.005))) output = ROOT/'output/q1/optimization_report.json' - output.write_text(json.dumps(report, ensure_ascii=False, indent=2, allow_nan=False)+'\n', encoding='utf-8') + output.write_text( + json.dumps(report, ensure_ascii=False, indent=2, allow_nan=False)+'\n', encoding='utf-8') print(json.dumps(report, ensure_ascii=False, indent=2)) diff --git a/scripts/benchmark_q3_revision.py b/scripts/benchmark_q3_revision.py index 80b2208..dc73335 100644 --- a/scripts/benchmark_q3_revision.py +++ b/scripts/benchmark_q3_revision.py @@ -48,19 +48,22 @@ def main(): for p in sorted(Path('src/mcm26b').rglob('*.py'))} if args.baseline: for name in ('baseline_bayes.py', 'baseline_config.json'): - source_sha256[str(args.out/name)] = hashlib.sha256((args.out/name).read_bytes()).hexdigest() + archive = args.out/name + source_sha256[str(archive)] = hashlib.sha256(archive.read_bytes()).hexdigest() scenes = [] if args.suite == 'development': for count in (10, 16): for seed in range(6): - scenes.append((f'random-{count}-{seed}', random_omni_scene(seed, count), 20260910 + seed)) + scenes.append((f'random-{count}-{seed}', random_omni_scene(seed, count), + 20260910 + seed)) edge = [Source(c+1, 1800*math.cos(math.pi/6+2*math.pi*c/count), 1800*math.sin(math.pi/6+2*math.pi*c/count), 1000) for c in range(count)] scenes.append((f'boundary-{count}', edge, 20260910)) else: for count in (10, 13, 16): for seed in range(100, 106): - scenes.append((f'holdout-{count}-{seed}', random_omni_scene(seed, count), 20260910+seed)) + scenes.append((f'holdout-{count}-{seed}', random_omni_scene(seed, count), + 20260910+seed)) rows = [] for name, sources, error_seed in scenes: sim = LocalSimulator(sources, robot_id='BENCH', config=config, error_seed=error_seed) @@ -81,12 +84,15 @@ def main(): sources=[asdict(s) for s in sources], error_seed=error_seed, summary=summary, actions=actions), ensure_ascii=False), encoding='utf-8') print(f'{name}: valid={valid} T={row["virtual_s"]:.1f} M={row["measures"]}', flush=True) - result = dict(config=asdict(config), source='local_simulator_only', baseline=args.baseline, rows=rows, - mean_s=mean(r['virtual_s'] for r in rows), median_s=median(r['virtual_s'] for r in rows), + result = dict(config=asdict(config), source='local_simulator_only', + baseline=args.baseline, rows=rows, + mean_s=mean(r['virtual_s'] for r in rows), + median_s=median(r['virtual_s'] for r in rows), max_s=max(r['virtual_s'] for r in rows), valid=sum(r['valid'] for r in rows), under_4000=sum(r['virtual_s'] < 4000 for r in rows), source_sha256=source_sha256) - (folder/'results.json').write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding='utf-8') + (folder/'results.json').write_text( + json.dumps(result, ensure_ascii=False, indent=2), encoding='utf-8') print({k:result[k] for k in ('mean_s','median_s','max_s','valid','under_4000')}, flush=True) diff --git a/scripts/benchmark_q3_search.py b/scripts/benchmark_q3_search.py index 643de6b..439cfc3 100644 --- a/scripts/benchmark_q3_search.py +++ b/scripts/benchmark_q3_search.py @@ -14,14 +14,11 @@ from pathlib import Path from statistics import mean, median from time import perf_counter -from mcm26b.belief import ChannelStatus # noqa: F401 (兼容旧 import 路径) from mcm26b.config import ProblemConfig -from mcm26b.coverage import CoverageLedger # noqa: F401 -from mcm26b.planner import OmnidirectionalLoop, clear_certificate # noqa: F401 +from mcm26b.planner import OmnidirectionalLoop from mcm26b.planner.seven import (SevenBatch, SevenImmediate, SevenSweep, RING_RADIUS_M, seven_points) from mcm26b.protocol.client import SimulatorClient -from mcm26b.protocol.rules import NEAR # noqa: F401 from mcm26b.harness import LocalSimTransport, LocalSimulator from referee import Source from mcm26b.harness import random_omni_scene @@ -70,7 +67,8 @@ def run_case(name, sources, error_seed, out): "actions_file": str(log_path.relative_to(out)), "actions_sha256": hashlib.sha256(log_path.read_bytes()).hexdigest()} rows.append(row) - print(f"{name} {label}: all_cleared={row['all_cleared']} T={row['virtual_time_s']:.1f}s M={row['measures']}", flush=True) + print(f"{name} {label}: all_cleared={row['all_cleared']} " + f"T={row['virtual_time_s']:.1f}s M={row['measures']}", flush=True) return rows @@ -96,7 +94,8 @@ def geometry_check(): "sampled_max_distance_m": worst[0], "sampled_worst_point": worst[1], "samples": 181*721, "passed": worst[0] <= analytic+1e-8 and analytic < field.r_eff_min_m, - "note": "Grid samples check the derivation; the sector proof establishes continuous coverage."} + "note": ("Grid samples check the derivation; " + "the sector proof establishes continuous coverage.")} def main(): @@ -137,10 +136,16 @@ def main(): result = {"source": "local_simulator_only", "production_algorithm_modified": False, "seeds": args.seeds, "config": asdict(ProblemConfig()), "geometry": geometry_check(), "aggregates": aggregates, "runs": rows, - "limitations": "Same scene and fixed per-position error field for each variant; synthetic data, not an official improvement estimate. Only search/scheduling is changed; no posterior or localizer redesign."} - (args.out/"benchmark.json").write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding="utf-8") + "limitations": ( + "Same scene and fixed per-position error field for each variant; " + "synthetic data, not an official improvement estimate. " + "Only search/scheduling is changed; no posterior or localizer redesign.")} + (args.out/"benchmark.json").write_text( + json.dumps(result, ensure_ascii=False, indent=2), encoding="utf-8") print(json.dumps(aggregates, ensure_ascii=False, indent=2)) - return 0 if result["geometry"]["passed"] and all(a["valid_runs"] == a["runs"] for a in aggregates.values()) else 1 + passed = (result["geometry"]["passed"] + and all(a["valid_runs"] == a["runs"] for a in aggregates.values())) + return 0 if passed else 1 if __name__ == "__main__": diff --git a/scripts/benchmark_rl.py b/scripts/benchmark_rl.py index fa6fa1c..fbfd055 100644 --- a/scripts/benchmark_rl.py +++ b/scripts/benchmark_rl.py @@ -95,7 +95,8 @@ def main(argv=None) -> int: cleared = all(row["rl"]["all_cleared"] and row[baseline]["all_cleared"] for row in rows) valid = all(part['completed'] and part['all_cleared'] and part['correct_absence'] - and part['normal_exit'] and not part['anomalies'] and not part['timing_mismatches'] + and part['normal_exit'] and not part['anomalies'] + and not part['timing_mismatches'] for row in rows for part in (row[baseline],row['rl'])) report[key + "_summary"] = { "mean_virtual_s": { diff --git a/scripts/build_q1_demo.py b/scripts/build_q1_demo.py index ed0b416..4857407 100644 --- a/scripts/build_q1_demo.py +++ b/scripts/build_q1_demo.py @@ -53,15 +53,23 @@ def main(): data={**case,'measurements':[asdict(m) for m in ms],'frames':frames} dataset.append(data) if case['source']=='synthetic': - (examples/(case['id']+'.json')).write_text(json.dumps({k:v for k,v in data.items() if k!='frames'},ensure_ascii=False,indent=2),encoding='utf-8') - (results/(case['id']+'_result.json')).write_text(json.dumps(frames[-1],ensure_ascii=False,indent=2,allow_nan=False),encoding='utf-8') - (results/'demo_data.json').write_text(json.dumps(dataset,ensure_ascii=False,indent=2,allow_nan=False),encoding='utf-8') + (examples/(case['id']+'.json')).write_text( + json.dumps({k:v for k,v in data.items() if k!='frames'}, + ensure_ascii=False,indent=2),encoding='utf-8') + (results/(case['id']+'_result.json')).write_text( + json.dumps(frames[-1],ensure_ascii=False,indent=2,allow_nan=False),encoding='utf-8') + (results/'demo_data.json').write_text( + json.dumps(dataset,ensure_ascii=False,indent=2,allow_nan=False),encoding='utf-8') template=(ASSETS / 'q1.template.html').read_text(encoding='utf-8') - fragment=template.replace('/*__Q1_DATA__*/',json.dumps(dataset,ensure_ascii=False,allow_nan=False)) + fragment=template.replace('/*__Q1_DATA__*/', + json.dumps(dataset,ensure_ascii=False,allow_nan=False)) (ROOT/'output'/'q1'/'question1_full.html').write_text(fragment,encoding='utf-8') # A standalone companion uses the same fragment and computation results. theme=(ASSETS / 'standalone.css').read_text(encoding='utf-8') - page='B题问题一交会定位'+fragment+'' + page=('' + '' + 'B题问题一交会定位' + ''+fragment+'') (results/'question1.html').write_text(page,encoding='utf-8') print(json.dumps({'cases':len(dataset),'fragment_bytes':len(fragment.encode('utf-8')), 'standalone':str(results/'question1.html')},ensure_ascii=False)) diff --git a/scripts/capture_practice.py b/scripts/capture_practice.py index 0bb6c0e..9cfd83e 100644 --- a/scripts/capture_practice.py +++ b/scripts/capture_practice.py @@ -43,8 +43,10 @@ class PracticeClient: status = response.status body = json.load(response) # A retried action is recorded/consumed only once. - self.events.append(dict(path=path, request=payload, response=body, retries=attempt)) - self.output.write_text(json.dumps(self.events,ensure_ascii=False,indent=2),encoding='utf-8') + self.events.append(dict(path=path, request=payload, response=body, + retries=attempt)) + self.output.write_text( + json.dumps(self.events,ensure_ascii=False,indent=2),encoding='utf-8') if status != 200 or body.get('accepted') is not True: raise RuntimeError(f'动作未被接受: {status}, {body}') if path == '/enter': @@ -54,6 +56,7 @@ class PracticeClient: if attempt == 2: raise time.sleep(.25) + raise URLError('网络重试次数已用尽') def measure(self, p, channel): return self.action('/measure',position={'x':p[0],'y':p[1]},channel=channel) @@ -106,12 +109,15 @@ def main(): if len(observations)>=3: break data={'name':'官方问题3演练观测','source':'official_practice','case_code':args.case_code, - 'channel':channel,'ground_truth':None,'measurements':[m.__dict__ for m in observations], + 'channel':channel,'ground_truth':None, + 'measurements':[m.__dict__ for m in observations], 'repeated_station_response':repeat,'result':solve(observations), 'scope':'仅采集问题一观测，不执行问题三全清除任务；没有调用clear'} path=args.output/f'{args.case_code}_observations.json' - path.write_text(json.dumps(data,ensure_ascii=False,indent=2,allow_nan=False),encoding='utf-8') - print(json.dumps({'observations':str(path),'count':len(observations),'result':data['result']},ensure_ascii=False,indent=2)) + path.write_text(json.dumps(data,ensure_ascii=False,indent=2,allow_nan=False), + encoding='utf-8') + print(json.dumps({'observations':str(path),'count':len(observations), + 'result':data['result']},ensure_ascii=False,indent=2)) finally: if entered: client.action('/exit') diff --git a/scripts/evaluate_rl_revision.py b/scripts/evaluate_rl_revision.py index 36d0dcf..209b151 100644 --- a/scripts/evaluate_rl_revision.py +++ b/scripts/evaluate_rl_revision.py @@ -22,7 +22,8 @@ from referee import LocalSimulator, Source def main(): parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument('--split', choices=('development','validation','stress'), default='development') + parser.add_argument('--split', choices=('development','validation','stress'), + default='development') parser.add_argument('--variants', default='baseline,legacy,revised') parser.add_argument('--problems', default='3,4') parser.add_argument('--model-dir', type=Path, default=Path('output/rl/review-20260911')) @@ -49,7 +50,8 @@ def main(): repeated_measures=sum(r['repeated_measures'] for r in part)) result = dict(source='local_referee_only', config=asdict(config), split=args.split, rows=rows, aggregates=aggregates) - (output/'results.json').write_text(json.dumps(result,ensure_ascii=False,indent=2),encoding='utf-8') + (output/'results.json').write_text( + json.dumps(result,ensure_ascii=False,indent=2),encoding='utf-8') return aggregates for problem in map(int, args.problems.split(',')): scenes = [] @@ -71,7 +73,8 @@ def main(): scenes.append((f'{count}-{seed}',count,seed,sources)) for name,count,seed,sources in scenes: scene_data = dict(sources=[asdict(s) for s in sources], error_seed=seed+2000000) - (output/f'q{problem}-{name}-scene.json').write_text(json.dumps(scene_data),encoding='utf-8') + scene_path = output/f'q{problem}-{name}-scene.json' + scene_path.write_text(json.dumps(scene_data),encoding='utf-8') for variant in variants: model = None cls = BayesLoop if problem==3 else DirectionalExtension @@ -81,13 +84,15 @@ def main(): model = (Path(f'output/rl/q{problem}_ppo.zip') if variant=='legacy' else args.model_dir/f'q{problem}_masked.zip') kwargs = dict(model_path=model, device='cpu') - sim = LocalSimulator(sources,robot_id='RL-AUDIT',spec=referee_spec(config),error_seed=seed+2000000) + sim = LocalSimulator(sources,robot_id='RL-AUDIT',spec=referee_spec(config), + error_seed=seed+2000000) client = SimulatorClient('RL-AUDIT',config=config,transport=LocalSimTransport(sim)) start = perf_counter() summary = cls(client,config,**kwargs).run() events = client.log.action_events() truth = {s.channel for s in sources} - successful = {e['request']['channel'] for e in events if e['response'].get('clear_result')=='success'} + successful = {e['request']['channel'] for e in events + if e['response'].get('clear_result')=='success'} seen = set() repeats = 0 for e in events: @@ -102,12 +107,15 @@ def main(): and summary['exit_reason']=='user_exit' and not mismatches) row = dict(case=name,problem=problem,variant=variant,valid=valid, virtual_s=client.state.virtual_time_s,all_cleared=successful==truth, - measures=summary['measures'],failed_clears=summary['clears']-len(successful), + measures=summary['measures'], + failed_clears=summary['clears']-len(successful), repeated_measures=repeats,wall_s=perf_counter()-start, - model_sha256=hashlib.sha256(model.read_bytes()).hexdigest() if model else None) + model_sha256=(hashlib.sha256(model.read_bytes()).hexdigest() + if model else None)) rows.append(row) (output/f'q{problem}-{name}-{variant}.json').write_text(json.dumps( - dict(row=row,summary=summary,events=events,timing_mismatches=mismatches),ensure_ascii=False),encoding='utf-8') + dict(row=row,summary=summary,events=events,timing_mismatches=mismatches), + ensure_ascii=False),encoding='utf-8') save() print(json.dumps(row),flush=True) print(json.dumps(save(),indent=2),flush=True) diff --git a/scripts/pretrain_rl_expert.py b/scripts/pretrain_rl_expert.py index 4b62c55..7a3507c 100644 --- a/scripts/pretrain_rl_expert.py +++ b/scripts/pretrain_rl_expert.py @@ -88,7 +88,7 @@ def collect(problem: int, episodes: int, seed: int, config: ProblemConfig): max_episode_steps=16) observations, actions, groups = [], [], [] for episode in range(episodes): - obs, info = env.reset(seed=seed + episode) + obs, _info = env.reset(seed=seed + episode) for _step in range(env.max_episode_steps + 2): action = expert_action(env) observations.append(obs) @@ -108,7 +108,8 @@ def train_bc(x, y, *, epochs: int, batch_size: int, lr: float, episodes = np.unique(groups) if len(episodes) < 2: raise ValueError('行为克隆至少需要两个独立episode才能划分验证集') - valid_groups = np.random.default_rng(0).permutation(episodes)[:max(1, int(len(episodes)*valid_frac))] + valid_groups = np.random.default_rng(0).permutation(episodes) + valid_groups = valid_groups[:max(1, int(len(episodes)*valid_frac))] valid = torch.as_tensor(np.isin(groups, valid_groups), device=device) x_t, y_t = torch.as_tensor(x, device=device), torch.as_tensor(y, device=device) x_v, y_v = x_t[valid], y_t[valid] diff --git a/scripts/render_q1_figures.py b/scripts/render_q1_figures.py index 1b3b298..c97d820 100644 --- a/scripts/render_q1_figures.py +++ b/scripts/render_q1_figures.py @@ -6,17 +6,17 @@ import math from pathlib import Path import re import xml.etree.ElementTree as ET -from PIL import Image,ImageDraw,ImageFont +from PIL import Image,ImageDraw,ImageFont # pylint: disable=import-error # 可选依赖 ROOT=Path(__file__).resolve().parents[1] S=2 FONT=ImageFont.truetype('C:/Windows/Fonts/msyh.ttc',12*S) -def render(path): - root=ET.fromstring(path.read_text(encoding='utf-8')) +def render(svg_path): + root=ET.fromstring(svg_path.read_text(encoding='utf-8')) width,height=map(float,(root.get('width'),root.get('height'))) - bg='#191c21' if 'dark' in path.stem else '#ffffff' + bg='#191c21' if 'dark' in svg_path.stem else '#ffffff' image=Image.new('RGBA',(round(width*S),round(height*S)),bg) clip=next(el for el in root.iter() if el.tag.endswith('clipPath'))[0] clipbox=tuple(float(clip.get(k))*S for k in ('x','y','width','height')) @@ -34,7 +34,8 @@ def render(path): for child in el:draw(child,clipped or bool(el.get('clip-path'))) return layer=Image.new('RGBA',image.size,(0,0,0,0));d=ImageDraw.Draw(layer) - num=lambda key,default=0:float(el.get(key,default))*S + def num(key,default=0): + return float(el.get(key,default))*S fill=color(el.get('fill'),float(el.get('fill-opacity',1))) stroke=color(el.get('stroke'),float(el.get('stroke-opacity',1))) sw=max(1,round(num('stroke-width',1))) @@ -43,13 +44,15 @@ def render(path): dash=el.get('stroke-dasharray','none') if dash=='none': d.line(xy,fill=stroke,width=sw) else: - a,b=xy;dist=math.dist(a,b);unit=((b[0]-a[0])/max(dist,1e-15),(b[1]-a[1])/max(dist,1e-15)) + a,b=xy;dist=math.dist(a,b) + unit=((b[0]-a[0])/max(dist,1e-15),(b[1]-a[1])/max(dist,1e-15)) on,off=[float(v)*S for v in dash.split()] # Bound offscreen ray work to the finite exported viewport vicinity. segments=min(100000,int(dist/(on+off))+1) for i in range(segments): lo=i*(on+off);hi=min(lo+on,dist) - d.line([(a[0]+lo*unit[0],a[1]+lo*unit[1]),(a[0]+hi*unit[0],a[1]+hi*unit[1])],fill=stroke,width=sw) + d.line([(a[0]+lo*unit[0],a[1]+lo*unit[1]), + (a[0]+hi*unit[0],a[1]+hi*unit[1])],fill=stroke,width=sw) elif tag=='rect': box=(num('x'),num('y'),num('x')+num('width'),num('y')+num('height')) d.rectangle(box,fill=fill,outline=stroke,width=sw) @@ -63,7 +66,8 @@ def render(path): d.polygon(points,fill=fill) if stroke:d.line(points+[points[0]],fill=stroke,width=sw,joint='curve') elif tag=='path': - for x,y,axis,length in re.findall(r'M([\d.e+-]+),([\d.e+-]+)([hv])([\d.e+-]+)',el.get('d')): + pattern=r'M([\d.e+-]+),([\d.e+-]+)([hv])([\d.e+-]+)' + for x,y,axis,length in re.findall(pattern,el.get('d')): x,y,length=float(x)*S,float(y)*S,float(length)*S d.line([(x,y),(x+length,y) if axis=='h' else (x,y+length)],fill=stroke,width=sw) elif tag=='text': @@ -86,7 +90,7 @@ def render(path): image=Image.alpha_composite(image,layer) draw(root) - output=path.with_suffix('.png');image.convert('RGB').save(output) + output=svg_path.with_suffix('.png');image.convert('RGB').save(output) return str(output.relative_to(ROOT)) diff --git a/scripts/run_q3_seven_practice.py b/scripts/run_q3_seven_practice.py index f48ce6d..2f461e7 100644 --- a/scripts/run_q3_seven_practice.py +++ b/scripts/run_q3_seven_practice.py @@ -32,8 +32,9 @@ def main(): metadata = {"source": "official_practice", "problem": 3, "strategy": "seven_batch", "started_local": datetime.now().astimezone().isoformat(), "config": asdict(config), "coverage_points": seven_points(), - "source_sha256": {str(p.relative_to(root)): hashlib.sha256(p.read_bytes()).hexdigest() - for p in paths}} + "source_sha256": { + str(p.relative_to(root)): hashlib.sha256(p.read_bytes()).hexdigest() + for p in paths}} def save(name, value): (args.output / name).write_text(json.dumps(value, ensure_ascii=False, indent=2, allow_nan=False) + "\n", encoding="utf-8") diff --git a/scripts/summarize_rl_revision.py b/scripts/summarize_rl_revision.py index f13c936..60ba546 100644 --- a/scripts/summarize_rl_revision.py +++ b/scripts/summarize_rl_revision.py @@ -12,7 +12,8 @@ from mcm26b.rl.features import build_observation def main(): root = Path('output/rl/review-20260911') - records = json.loads((root/'evaluation/validation/final/results.json').read_text(encoding='utf-8')) + records = json.loads((root/'evaluation/validation/final/results.json') + .read_text(encoding='utf-8')) rng = np.random.default_rng(20260911) comparisons = {} for problem in (3,4): @@ -46,13 +47,16 @@ def main(): bc = {} for problem in (3,4): # 使用旧检查点和原始专家输入，比对原ReLU网络与拷贝后的Tanh网络。 - checkpoint = torch.load(f'output/rl/bc/q{problem}_bc.pt',map_location='cpu',weights_only=True) + checkpoint = torch.load(f'output/rl/bc/q{problem}_bc.pt',map_location='cpu', + weights_only=True) data = np.load(f'output/rl/bc/q{problem}_expert.npz',allow_pickle=False) x = torch.from_numpy(data['obs'][::max(1,len(data['obs'])//1000)].copy()) - def network(activation): + + def network(activation, checkpoint=checkpoint): net = torch.nn.Sequential(torch.nn.Linear(401,128),activation(), torch.nn.Linear(128,128),activation(),torch.nn.Linear(128,60)) - net.load_state_dict({k.removeprefix('net.'):v for k,v in checkpoint['state_dict'].items()}) + state = {k.removeprefix('net.'):v for k,v in checkpoint['state_dict'].items()} + net.load_state_dict(state) return net with torch.no_grad(): before,after=network(torch.nn.ReLU)(x),network(torch.nn.Tanh)(x) @@ -63,9 +67,11 @@ def main(): limitations='置信区间仅重采样固定模型的场景，不覆盖训练随机种子不确定性。', code_sha256={str(p):hashlib.sha256(p.read_bytes()).hexdigest() for p in sorted(Path('src/mcm26b/rl').glob('*.py'))}) - (root/'review_evidence.json').write_text(json.dumps(report,ensure_ascii=False,indent=2),encoding='utf-8') + (root/'review_evidence.json').write_text( + json.dumps(report,ensure_ascii=False,indent=2),encoding='utf-8') print(json.dumps(dict(comparisons=comparisons,feature_probe=features,bc_probe=bc, - official=[dict(case=r['case_code'],passed=r['passed']) for r in official]),indent=2)) + official=[dict(case=r['case_code'],passed=r['passed']) + for r in official]),indent=2)) if __name__=='__main__': diff --git a/scripts/train_hrl_high.py b/scripts/train_hrl_high.py index 711160a..b67a769 100644 --- a/scripts/train_hrl_high.py +++ b/scripts/train_hrl_high.py @@ -8,13 +8,31 @@ 当前 HTTP 执行路径仍由高层宏动作终点决定，不把八方向推演当正式收益。 落盘保证（长训必需）： -- ``--checkpoint-every`` 每 N 个环境步在 ``<模型目录>/checkpoints/`` 落一个可续训 - 的 checkpoint（zip + 同名 json 元数据），只保留最近 ``--checkpoint-keep`` 个； -- 训练前端的「停止」是对**进程组**发 SIGTERM，因此 subproc worker 在 :func:`_make_env` - 里显式忽略 SIGTERM/SIGINT，主进程收到信号后在 step 边界收尾并保存模型， - 不会被 worker 先行退出打断； -- 任何异常（含被强杀之外的失败）都会先落一份 ``*_emergency.zip`` 再抛出， - 不再出现"跑了很久什么都没有"。 +- **急停可恢复**：``--recovery-every``（默认 2000 环境步）把一份滚动恢复点 + ``<模型目录>/checkpoints/_latest.zip`` 原子覆盖写；训练开始前先写一份 + 起步基线。因此即使进程被 SIGKILL / ``taskkill /F`` 强杀（没有任何 Python + 收尾机会），也**最多只丢 ``--recovery-every`` 个环境步**，且磁盘上始终有一份 + 完整、可加载、可 ``--resume`` 的产物； +- **原子落盘**：模型 zip、旁挂 json 与 ``/resume.json`` 都是"临时文件 + + ``os.replace``"，保存途中被杀不会留下半截文件，也不会破坏上一份完整快照； +- ``--checkpoint-every`` 每 N 个环境步在 ``<模型目录>/checkpoints/`` 另留一份历史 + checkpoint（``_<步数>.zip``），只保留最近 ``--checkpoint-keep`` 个，供 + 返祖对比与多档恢复； +- ``/resume.json`` 是恢复的**权威指针**：无论上一刻是滚动恢复点、历史 + checkpoint、优雅收尾还是异常兜底，它总指向一份可续训产物，并记录增量语义 + （``timesteps`` 是追加步数）与生效过的 PPO 超参； +- ``--control-file`` 是训练前端的**跨平台**控制通道：前端原子写 JSON + （``paused`` / ``stop``），本脚本每个 step 轮询。暂停时训练在 step 边界挂起， + 停止时先保存模型与元数据再退出； +- 信号（SIGTERM/SIGINT，Windows 上另有 SIGBREAK）仍可用：直接 kill 本脚本或 + 前端发进程组信号时同样走优雅收尾；subproc worker 在 :func:`_make_env` 里忽略 + 这些信号，保证它们活到主进程 ``vec_env.close()`` 之后再退出； +- 任何异常都会先落一份 ``*_emergency.zip`` + ``resume.json`` 再抛出，不再出现 + "跑了很久什么都没有"。 + +为什么暂停/停止不靠信号：Windows 没有 SIGSTOP，且 ``os.kill(pid, SIGTERM)`` 会 +退化成 ``TerminateProcess`` 硬杀——收尾分支根本不会执行。控制文件两端都只是 +普通文件 IO，POSIX 与 Windows 行为一致（见 :mod:`mcm26b.process_control`）。 P1 smoke 示例： python scripts/train_hrl_high.py \ @@ -29,6 +47,7 @@ from __future__ import annotations import argparse import hashlib import json +import math import os import platform import random @@ -39,9 +58,10 @@ from pathlib import Path from typing import Any from mcm26b.config import ProblemConfig -from mcm26b.hrl.curriculum import STAGES, training_config +from mcm26b.hrl.curriculum import STAGES from mcm26b.hrl.constants import (ACTION_LAYOUT_VERSION, FEATURE_VERSION, HIGH_OBS_DIM, N_CANDIDATES, REWARD_VERSION) +from mcm26b.process_control import ControlState, read_control def _stage_from_index(index: int): @@ -68,24 +88,86 @@ def _reward_from_profile(profile: str): raise ValueError(f"未知 reward-profile: {profile}") -def _ignore_stop_signals_in_worker() -> None: - """subproc worker 忽略 SIGTERM/SIGINT，保证主进程能优雅收尾落盘。 +def _stop_signals() -> list[int]: + """本平台可用于"请求收尾"的信号集合。 - 训练前端的「停止」向整个进程组发 SIGTERM；若 worker 同时退出，主进程会在 - 保存模型之前就拿到断管/EOF 错误，整局训练白跑。worker 活在管道后面，主进程 - 收尾时通过 ``vec_env.close()`` 的 ``close`` 指令正常结束它们。 + Windows 上没有 SIGTERM 的软语义（``os.kill`` 直接 TerminateProcess），但 + ``SIGBREAK`` 可以由 ``CTRL_BREAK_EVENT`` 触发并进入 Python 处理器；POSIX 上 + SIGTERM/SIGINT 均可捕获。控制文件才是主通道，这里只是让"直接 kill 脚本" + 也尽量走优雅收尾。 + """ + signals = [signal.SIGTERM, signal.SIGINT] + sigbreak = getattr(signal, "SIGBREAK", None) + if sigbreak is not None: + signals.append(sigbreak) + return signals + + +def _ignore_stop_signals_in_worker() -> None: + """subproc worker 忽略收尾信号，保证主进程能优雅落盘。 + + 无论信号来自进程组 SIGTERM 还是 Windows ``CTRL_BREAK_EVENT``，若 worker + 同时退出，主进程会在保存模型之前就拿到断管/EOF 错误，整局训练白跑。worker + 活在管道后面，主进程收尾时通过 ``vec_env.close()`` 的 ``close`` 指令正常 + 结束它们。 """ try: import multiprocessing if multiprocessing.current_process().name == "MainProcess": return - for signum in (signal.SIGTERM, signal.SIGINT): + for signum in _stop_signals(): signal.signal(signum, signal.SIG_IGN) except (ImportError, ValueError, OSError): # 非主线程/平台不支持：不影响训练 return -def _make_env(problem: int, stage_index: int, seed: int, +class _ControlPoll: + """控制文件轮询状态机（纯标准库，可脱离 SB3 单测）。 + + ``poll()`` 每 ``poll_every`` 次调用才真正读盘，避免每个 env step 都做一次 + 文件 IO；``wait_while_paused()`` 阻塞到暂停解除或收到 stop，并累计暂停秒数 + （``--max-wall-s`` 要把这段扣掉，否则"挂起一小时回来就判超时"）。 + """ + + def __init__(self, path: Path, *, poll_every: int = 32, + poll_s: float = 0.2): + self.path = Path(path) + self.poll_every = max(1, int(poll_every)) + self.poll_s = max(0.02, float(poll_s)) + self.paused_total_s = 0.0 + self._calls = 0 + self._state = ControlState() + + def poll(self) -> ControlState: + """按抽样间隔返回最近一次控制状态。""" + self._calls += 1 + if self._calls % self.poll_every: + return self._state + self._state = read_control(self.path) + return self._state + + def wait_while_paused(self, on_pause, on_resume) -> bool: + """阻塞到继续或停止；返回 True 表示期间收到了 stop。 + + ``on_pause`` / ``on_resume`` 用来向前端回显事件（训练脚本的 JSON 行）。 + """ + on_pause() + started = time.monotonic() + stopped = False + while True: + time.sleep(self.poll_s) + self._state = read_control(self.path) + if self._state.stop: + stopped = True + break + if not self._state.paused: + break + self.paused_total_s += time.monotonic() - started + on_resume() + return stopped + + +def _make_env(problem: int, stage_index: int, monitor_path: str | None, profile: str, reward_profile: str): """顶层工厂：供 DummyVecEnv / SubprocVecEnv 调用（必须是可 pickle 的）。""" @@ -112,7 +194,7 @@ def _build_vec_env(args, monitor_paths: list[str]): from functools import partial factories = [ - partial(_make_env, args.problem, args.stage, args.seed + rank, + partial(_make_env, args.problem, args.stage, monitor_paths[rank], args.config_profile, args.reward_profile) for rank in range(args.envs) ] @@ -161,6 +243,52 @@ def _policy_hyperparams(model) -> dict[str, Any]: } +def _progress_payload(model, start: int) -> dict[str, Any]: + """组装 ``--progress-json`` 的单行负载。 + + 纯函数（只读 model 的 logger / ep_info_buffer），便于脱离训练循环测试； + NaN / ±inf 一律折算成 None，保证 ``json.dumps(allow_nan=False)`` 不会炸。 + """ + values = getattr(model.logger, "name_to_value", {}) or {} + + def _num(key, default=None): + value = values.get(key) + if value is None: + return default + try: + value = float(value) + except (TypeError, ValueError): + return default + if not math.isfinite(value): + return default + return value + + ep_rewards: list[float] = [] + ep_lengths: list[float] = [] + for item in getattr(model, "ep_info_buffer", []) or []: + try: + ep_rewards.append(float(item.get("r"))) + ep_lengths.append(float(item.get("l"))) + except (TypeError, ValueError, AttributeError): + continue + return { + "event": "progress", + "timesteps": int(model.num_timesteps), + "start_timesteps": int(start), + "n_updates": int(getattr(model, "_n_updates", 0) or 0), + "ep_rew_mean": (sum(ep_rewards) / len(ep_rewards) + if ep_rewards else None), + "ep_len_mean": (sum(ep_lengths) / len(ep_lengths) + if ep_lengths else None), + "value_loss": _num("train/value_loss"), + "policy_gradient_loss": _num("train/policy_gradient_loss"), + "approx_kl": _num("train/approx_kl"), + "entropy_loss": _num("train/entropy_loss"), + "explained_variance": _num("train/explained_variance"), + "fps": _num("time/fps"), + } + + def _model_metadata(*, args, wall_s: float, n_envs: int, actual_timesteps: int) -> dict[str, Any]: import numpy @@ -190,12 +318,12 @@ def _model_metadata(*, args, wall_s: float, n_envs: int, "config_profile": args.config_profile, "fidelity": _profile_fidelity(args.config_profile), "reward_profile": args.reward_profile, - "reward_version": REWARD_VERSION, "local_selector": args.local_selector, "local_model": args.local_model, "checkpoint_every": int(getattr(args, "checkpoint_every", 0) or 0), "checkpoint_keep": int(getattr(args, "checkpoint_keep", 0) or 0), "checkpoint_dir": getattr(args, "checkpoint_dir", None), + "recovery_every": int(getattr(args, "recovery_every", 0) or 0), "device": args.device, "wall_seconds": round(wall_s, 3), "max_wall_s": float(args.max_wall_s), @@ -211,6 +339,68 @@ def _model_metadata(*, args, wall_s: float, n_envs: int, } +def _write_json_atomic(path: Path, payload: dict[str, Any]) -> None: + """原子写 JSON：同目录 ``.tmp-`` 前缀临时文件 + ``os.replace``。 + + 急停（SIGKILL / ``taskkill /F``）会绕过所有 Python 清理逻辑，保存途中被杀 + 是完全可能的。原子替换保证**真实文件名下要么是旧的完整文件、要么是新的 + 完整文件**，不会出现半截 JSON 让续训读到一半就崩。 + """ + path = Path(path) + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_name(f".tmp-{path.name}") + tmp.write_text(json.dumps(payload, ensure_ascii=False, indent=2), + encoding="utf-8") + os.replace(tmp, path) + + +def _save_model_atomic(model, target: Path) -> Path: + """把 SB3 模型原子地写到 ``target``，返回实际落盘路径。 + + 先写 ``.tmp-``（同样以 ``.zip`` 结尾，SB3 不会再加后缀），完整写完后 + ``os.replace`` 覆盖目标。于是： + + - 保存途中被强杀 → 目标路径仍是上一份**完整**的快照，可继续复训； + - 保存成功 → 目标路径立即变成新快照，不存在"看起来存在但打不开"的窗口。 + + 临时名带 ``.tmp-`` 前缀，因此不会被 ``*_ckpt_*.zip`` 的清理逻辑误当成正式 + checkpoint。 + """ + target = Path(target) + target.parent.mkdir(parents=True, exist_ok=True) + tmp = target.with_name(f".tmp-{target.name}") + model.save(str(tmp)) + if not tmp.exists(): # SB3 对未知后缀可能自行补 .zip + candidate = tmp.with_name(tmp.name + ".zip") + if candidate.exists(): + tmp = candidate + os.replace(tmp, target) + return target + + +def _write_resume_hint(*, run_dir: Path, model_path: Path, timesteps: int, + reason: str, model=None) -> dict[str, Any]: + """把"从哪继续"写进 ``/resume.json``。 + + 这是急停恢复的**权威指针**：无论上一刻是周期 checkpoint、滚动恢复点、 + 优雅收尾还是异常兜底，这个文件总指向一份完整、可 ``--resume`` 的产物， + 并带上续训语义（``timesteps`` 是增量步数）与生效过的 PPO 超参。 + """ + hint: dict[str, Any] = { + "event": "resume_hint", + "model": str(model_path), + "timesteps": int(timesteps), + "timesteps_semantics": "additional", + "reason": str(reason), + "created_unix_s": time.time(), + "resume_args": ["--resume", "--model", str(model_path)], + } + if model is not None: + hint["ppo_hyperparams"] = _policy_hyperparams(model) + _write_json_atomic(run_dir / "resume.json", hint) + return hint + + def _write_metadata(*, model_path: Path, args, wall_s: float, n_envs: int, actual_timesteps: int, monitor_paths: list[str], model=None, @@ -227,14 +417,18 @@ def _write_metadata(*, model_path: Path, args, wall_s: float, n_envs: int, "reward_profile=prompt 使用原始 §5.2 塑形。模型元数据已固化口径。") if extra: metadata.update(extra) - model_path.with_suffix(".json").write_text( - json.dumps(metadata, ensure_ascii=False, indent=2), encoding="utf-8") + _write_json_atomic(model_path.with_suffix(".json"), metadata) return metadata def _prune_checkpoints(directory: Path, stem: str, keep: int) -> list[str]: - """只保留最近 ``keep`` 个 checkpoint（zip+json 同删），返回被删的 zip 路径。""" - files = sorted(directory.glob(f"{stem}_*.zip"), + """只保留最近 ``keep`` 个 checkpoint（zip+json 同删），返回被删的 zip 路径。 + + 只匹配 ``_<步数>.zip``；滚动恢复点 ``_latest.zip`` 与临时文件 + ``.tmp-*.zip`` 都不匹配，因此永远不会被历史清理误删。 + """ + files = sorted((path for path in directory.glob(f"{stem}_[0-9]*.zip") + if path.stem.rsplit("_", 1)[-1].isdigit()), key=lambda path: path.stat().st_mtime, reverse=True) removed: list[str] = [] for stale in files[max(1, int(keep)):]: @@ -267,7 +461,7 @@ def _evaluate_env(model, *, problem: int, stage_index: int, while not done: action, _state = model.predict( obs, action_masks=env.action_masks(), deterministic=True) - obs, reward, terminated, truncated, info = env.step(int(action)) + obs, _, terminated, truncated, info = env.step(int(action)) done = bool(terminated or truncated) steps += 1 # 课程阶段的小场景只需证明全部频道 accounted；完整阶段至少应清掉 @@ -299,7 +493,7 @@ def _evaluate_env(model, *, problem: int, stage_index: int, def _evaluate_manifest(model_path: Path, manifest_path: str, *, - limit: int, problem: int) -> dict[str, Any]: + limit: int) -> dict[str, Any]: """可选：用 benchmark_hrl 的同一套真值隔离评测在固定 manifest 上跑已训模型。""" scripts_dir = Path(__file__).resolve().parent if str(scripts_dir) not in sys.path: @@ -404,26 +598,72 @@ def run_training(args) -> int: "命令行 rollout-steps/batch-size/n-epochs 被忽略")}, ensure_ascii=False), file=sys.stderr, flush=True) - # ------------------------------------------------------------ 优雅停止 - # 训练前端「停止」= killpg(SIGTERM)：worker 已在 _make_env 里忽略该信号， - # 主进程在此装处理器，等当前 step 结束后收尾落盘。 + # ------------------------------------------------------ 优雅暂停 / 停止 + # 主通道是控制文件（跨平台）；信号是兜底：直接 kill 本进程时也走同一段收尾。 stop_state: dict[str, Any] = {"requested": False, "signal": None} previous_handlers: dict[int, Any] = {} - def _on_stop_signal(signum, _frame): + def _request_stop(source: str) -> None: + if stop_state["requested"]: + return stop_state["requested"] = True - stop_state["signal"] = signal.Signals(signum).name - print(json.dumps({"event": "stop_requested", - "signal": signal.Signals(signum).name}), flush=True) + stop_state["signal"] = source + print(json.dumps({"event": "stop_requested", "signal": source}), flush=True) - for _signum in (signal.SIGTERM, signal.SIGINT): + def _on_stop_signal(signum, _frame): + _request_stop(signal.Signals(signum).name) + + for _signum in _stop_signals(): try: previous_handlers[_signum] = signal.signal(_signum, _on_stop_signal) except (ValueError, OSError): # 非主线程/平台不支持：不阻塞训练 pass + class _ControlChannel(BaseCallback): + """把 :class:`_ControlPoll` 接到 SB3 的 step 回调上。 + + 为什么不用信号：Windows 没有 SIGSTOP，且 ``os.kill(SIGTERM)`` 会退化成 + ``TerminateProcess`` 硬杀，收尾分支根本不会执行。控制文件在 POSIX 与 + Windows 上都只是普通文件 IO，语义完全一致。 + + 暂停是**协作式**的：在 ``_on_step`` 内阻塞轮询，训练因此停在 step 边界， + 进程仍存活、显存仍占用，但不消耗算力；收到 stop 立刻转入收尾落盘。 + """ + + def __init__(self, path: Path, *, poll_every: int = 32, + poll_s: float = 0.2): + super().__init__() + self.poller = _ControlPoll(path, poll_every=poll_every, + poll_s=poll_s) + + def paused_seconds(self) -> float: + return self.poller.paused_total_s + + def _event(self, name: str) -> None: + print(json.dumps({"event": name, + "timesteps": int(self.model.num_timesteps)}), + flush=True) + + def _on_step(self) -> bool: + state = self.poller.poll() + if state.stop: + _request_stop("control_file") + return False + if state.paused: + stopped = self.poller.wait_while_paused( + lambda: self._event("paused"), + lambda: self._event("resumed")) + if stopped: + _request_stop("control_file") + return False + return not stop_state["requested"] + return True + + control_path = getattr(args, "control_file", None) + control_cb = _ControlChannel(Path(control_path)) if control_path else None + class _GracefulStop(BaseCallback): - """收到停止信号后在最近一次 step 边界结束 rollout，让模型能落盘。""" + """收到停止请求后在最近一次 step 边界结束 rollout，让模型能落盘。""" def _on_step(self) -> bool: if stop_state["requested"]: @@ -435,15 +675,22 @@ def run_training(args) -> int: return True class _WallClockStop(BaseCallback): - def __init__(self, max_wall_s: float): + """墙钟硬上限；暂停时间不计入，避免"挂起一小时回来就判超时"。""" + + def __init__(self, max_wall_s: float, paused_clock=None): super().__init__() self.max_wall_s = max_wall_s self.start = time.monotonic() + self.paused_clock = paused_clock + + def paused_seconds(self) -> float: + return float(self.paused_clock()) if self.paused_clock else 0.0 def _on_step(self) -> bool: - if time.monotonic() - self.start >= self.max_wall_s: + elapsed = time.monotonic() - self.start - self.paused_seconds() + if elapsed >= self.max_wall_s: print(json.dumps({"event": "wall_clock_stop", - "elapsed_s": time.monotonic() - self.start})) + "elapsed_s": elapsed})) return False return True @@ -473,44 +720,7 @@ def run_training(args) -> int: return True self.last_emit = current self.emitted += 1 - values = getattr(self.model.logger, "name_to_value", {}) or {} - - def _num(key, default=None): - value = values.get(key) - if value is None: - return default - try: - value = float(value) - except (TypeError, ValueError): - return default - if value != value or value in (float("inf"), float("-inf")): - return default - return value - - ep_rewards = [] - ep_lengths = [] - for item in getattr(self.model, "ep_info_buffer", []) or []: - try: - ep_rewards.append(float(item.get("r"))) - ep_lengths.append(float(item.get("l"))) - except (TypeError, ValueError, AttributeError): - continue - payload = { - "event": "progress", - "timesteps": current, - "start_timesteps": self.start, - "n_updates": int(getattr(self.model, "_n_updates", 0) or 0), - "ep_rew_mean": (sum(ep_rewards) / len(ep_rewards) - if ep_rewards else None), - "ep_len_mean": (sum(ep_lengths) / len(ep_lengths) - if ep_lengths else None), - "value_loss": _num("train/value_loss"), - "policy_gradient_loss": _num("train/policy_gradient_loss"), - "approx_kl": _num("train/approx_kl"), - "entropy_loss": _num("train/entropy_loss"), - "explained_variance": _num("train/explained_variance"), - "fps": _num("time/fps"), - } + payload = _progress_payload(self.model, self.start) print(json.dumps(payload, ensure_ascii=False, allow_nan=False), flush=True) return True @@ -521,38 +731,79 @@ def run_training(args) -> int: checkpoint_stem = f"{model_path.stem}_ckpt" class _CheckpointJSON(BaseCallback): - """每 N 个环境步落一个可续训 checkpoint（zip + 同名元数据）。 + """可续训落盘：滚动恢复点 + 周期 checkpoint + resume.json 指针。 - 只保留最近 ``keep`` 个；checkpoint 与最终模型共用同名 json 契约， - 因此 ``--resume `` 直接可用。 + - **滚动恢复点** ``_latest.zip``：每 ``recovery_every`` 个环境步 + 原子覆盖写一次，训练开始前也先写一份。它保证"急停"（SIGKILL / + ``taskkill /F``，没有任何 Python 清理机会）之后**最多只丢 + ``recovery_every`` 个环境步**，而且文件始终完整可加载。 + - **周期 checkpoint** ``_<步数>.zip``：每 ``every_steps`` 步留一份， + 只保留最近 ``keep`` 个，用于返祖对比与多档恢复。 + - 每次落盘都刷新 ``/resume.json``：无论怎么停，都有一份权威的 + "从哪继续"指针（含增量语义与生效过的 PPO 超参）。 """ - def __init__(self, every_steps: int, keep: int): + def __init__(self, every_steps: int, keep: int, recovery_every: int): super().__init__() - self.every_steps = max(1, int(every_steps)) + self.every_steps = max(0, int(every_steps)) self.keep = max(1, int(keep)) + self.recovery_every = max(0, int(recovery_every)) self.last_saved = int(model.num_timesteps) + self.last_recovery = int(model.num_timesteps) self.saved: list[str] = [] + self.latest_path = checkpoints_dir / f"{checkpoint_stem}_latest.zip" + + def write_recovery(self, reason: str) -> None: + """立刻写一份滚动恢复点；训练开始前调用即"起步基线"。 + + 用闭包里的 ``model`` 而不是 ``self.model``：``BaseCallback.model`` + 要等 ``learn()`` 时才被赋值，而起步基线恰恰在 ``learn()`` 之前写。 + """ + current = int(model.num_timesteps) + self.last_recovery = current + self._write_recovery(current, reason) def _on_step(self) -> bool: - current = int(self.model.num_timesteps) - if current - self.last_saved < self.every_steps: - return True - self.last_saved = current - self._write(current) + current = int(model.num_timesteps) + if (self.recovery_every > 0 + and current - self.last_recovery >= self.recovery_every): + self.last_recovery = current + self._write_recovery(current, "interval") + if (self.every_steps > 0 + and current - self.last_saved >= self.every_steps): + self.last_saved = current + self._write_interval(current) return True - def _write(self, current: int) -> None: - checkpoints_dir.mkdir(parents=True, exist_ok=True) - target = checkpoints_dir / f"{checkpoint_stem}_{current:09d}.zip" - self.model.save(str(target)) + def _persist(self, target: Path, current: int, reason: str, + extra: dict[str, Any]) -> None: + """原子落盘 zip + 元数据，并刷新 resume.json 指针。""" + _save_model_atomic(model, target) _write_metadata( model_path=target, args=args, wall_s=time.monotonic() - started, n_envs=args.envs, actual_timesteps=current, monitor_paths=monitor_paths, - model=self.model, - extra={"checkpoint": True, "checkpoint_timesteps": current, - "checkpoint_of": str(model_path)}) + model=model, + extra={"checkpoint_timesteps": current, + "checkpoint_reason": reason, + "checkpoint_of": str(model_path), **extra}) + _write_resume_hint(run_dir=run_dir, model_path=target, + timesteps=current, reason=reason, + model=model) + + def _write_recovery(self, current: int, reason: str) -> None: + checkpoints_dir.mkdir(parents=True, exist_ok=True) + self._persist(self.latest_path, current, reason, + {"recovery": True}) + print(json.dumps({"event": "recovery_checkpoint", + "reason": reason, "timesteps": current, + "model": str(self.latest_path)}, + ensure_ascii=False), flush=True) + + def _write_interval(self, current: int) -> None: + checkpoints_dir.mkdir(parents=True, exist_ok=True) + target = checkpoints_dir / f"{checkpoint_stem}_{current:09d}.zip" + self._persist(target, current, "interval", {"checkpoint": True}) removed = _prune_checkpoints(checkpoints_dir, checkpoint_stem, self.keep) self.saved.append(str(target)) @@ -562,15 +813,24 @@ def run_training(args) -> int: progress_every = (args.progress_every if args.progress_every > 0 else args.rollout_steps * args.envs) - checkpoint_cb = None - if args.checkpoint_every > 0: - checkpoint_cb = _CheckpointJSON(args.checkpoint_every, - args.checkpoint_keep) - callbacks = [_GracefulStop(), _WallClockStop(args.max_wall_s)] - if checkpoint_cb is not None: - callbacks.append(checkpoint_cb) + checkpoint_cb = _CheckpointJSON(args.checkpoint_every, args.checkpoint_keep, + args.recovery_every) + if args.recovery_every > 0: + # 起步基线：训练还没跑就急停，也至少有一份可加载、可续训的产物 + checkpoint_cb.write_recovery("baseline") + # 顺序有语义：控制通道最先（暂停/停止要在本步立刻生效），进度次之， + # 墙钟上限要能读到控制通道累计的暂停秒数，checkpoint 放最后。 + callbacks: list[Any] = [] + if control_cb is not None: + callbacks.append(control_cb) if args.progress_json: - callbacks.insert(0, _ProgressJSON(progress_every)) + callbacks.append(_ProgressJSON(progress_every)) + callbacks.append(_GracefulStop()) + callbacks.append(_WallClockStop( + args.max_wall_s, + paused_clock=(control_cb.paused_seconds if control_cb is not None + else None))) + callbacks.append(checkpoint_cb) actual_timesteps = int(model.num_timesteps) try: @@ -587,7 +847,7 @@ def run_training(args) -> int: } emergency = run_dir / f"{model_path.stem}_emergency.zip" try: - model.save(str(emergency)) + _save_model_atomic(model, emergency) _write_metadata(model_path=emergency, args=args, wall_s=time.monotonic() - started, n_envs=args.envs, @@ -595,27 +855,34 @@ def run_training(args) -> int: monitor_paths=monitor_paths, model=model, extra={"emergency": True, "error": failure["error"]}) + _write_resume_hint(run_dir=run_dir, model_path=emergency, + timesteps=int(model.num_timesteps), + reason="emergency", model=model) failure["emergency_model"] = str(emergency) except Exception as save_exc: # 落盘失败不能再掩盖原始异常 failure["emergency_save_error"] = ( f"{type(save_exc).__name__}: {save_exc}") - (run_dir / "training_failure.json").write_text( - json.dumps(failure, ensure_ascii=False, indent=2), encoding="utf-8") + _write_json_atomic(run_dir / "training_failure.json", failure) print(json.dumps(failure, ensure_ascii=False), file=sys.stderr, flush=True) raise wall_s = time.monotonic() - started actual_timesteps = int(model.num_timesteps) stopped_by_signal = bool(stop_state["requested"]) - model.save(str(model_path)) + _save_model_atomic(model, model_path) metadata = _write_metadata( model_path=model_path, args=args, wall_s=wall_s, n_envs=args.envs, actual_timesteps=actual_timesteps, monitor_paths=monitor_paths, model=model, extra={"stopped_by_signal": stopped_by_signal, "stop_signal": stop_state["signal"], - "checkpoint_files": (checkpoint_cb.saved - if checkpoint_cb is not None else [])}) + "checkpoint_files": checkpoint_cb.saved, + "recovery_model": str(checkpoint_cb.latest_path)}) + # 最终模型也刷新一次 resume.json：优雅收尾后"从哪继续"指向最终模型 + _write_resume_hint(run_dir=run_dir, model_path=model_path, + timesteps=actual_timesteps, + reason=("stopped" if stopped_by_signal else "finished"), + model=model) metadata_path = model_path.with_suffix(".json") vec_env.close() # 收尾阶段恢复默认处理器：再次 Ctrl+C / 停止可立即强杀，不再等待评估。 @@ -637,8 +904,7 @@ def run_training(args) -> int: if args.eval_manifest: try: manifest_eval = _evaluate_manifest( - model_path, args.eval_manifest, limit=args.eval_manifest_limit, - problem=args.problem) + model_path, args.eval_manifest, limit=args.eval_manifest_limit) except Exception as exc: manifest_eval = {"error": f"{type(exc).__name__}: {exc}"} metadata["evaluation"] = evaluation @@ -685,10 +951,17 @@ def main(argv=None) -> int: help="time=P0.4 纯时间口径；prompt=原始 §5.2 塑形对照") parser.add_argument("--progress-json", action="store_true", help="向训练前端输出单行 JSON 进度（UI 自动启用）") + parser.add_argument("--control-file", default=None, + help="训练前端控制文件（JSON：paused/stop）；" + "跨平台暂停与优雅停止的主通道") parser.add_argument("--progress-every", type=int, default=0, help="JSON 进度间隔环境步数；0=每个 rollout") parser.add_argument("--checkpoint-every", type=int, default=20000, - help="每 N 个环境步落一个 checkpoint；0=关闭（默认 20000）") + help="每 N 个环境步留一个历史 checkpoint；0=关闭（默认 20000）") + parser.add_argument("--recovery-every", type=int, default=2000, + help="每 N 个环境步原子覆盖写一份滚动恢复点 " + "<模型目录>/checkpoints/_latest.zip；" + "急停后最多丢 N 步。0=关闭（默认 2000）") parser.add_argument("--checkpoint-keep", type=int, default=3, help="checkpoint 保留最近 K 个（默认 3）") parser.add_argument("--checkpoint-dir", default=None, @@ -703,6 +976,8 @@ def main(argv=None) -> int: parser.error("eval-episodes 不能为负") if args.checkpoint_every < 0: parser.error("checkpoint-every 不能为负") + if args.recovery_every < 0: + parser.error("recovery-every 不能为负") if args.checkpoint_keep < 1: parser.error("checkpoint-keep 至少为 1") return run_training(args) diff --git a/scripts/train_hrl_low.py b/scripts/train_hrl_low.py index ea2a7ca..a1f0291 100644 --- a/scripts/train_hrl_low.py +++ b/scripts/train_hrl_low.py @@ -82,7 +82,6 @@ def bc_validation_mask(episode_ids, *, seed: int): def train_bc(x, y, episode_ids, *, epochs: int, batch_size: int, lr: float, device: str, seed: int): - import numpy as np import torch BCMlp = _bc_module(LOW_OBS_DIM, LOW_NUM_ACTIONS, HIDDEN) model = BCMlp().to(device) diff --git a/scripts/train_rl.py b/scripts/train_rl.py index 6552560..521161a 100644 --- a/scripts/train_rl.py +++ b/scripts/train_rl.py @@ -130,7 +130,8 @@ def main(argv=None) -> int: for rank in range(max(1, args.envs))]) if args.init: metadata = json.loads(Path(args.init).with_suffix('.json').read_text(encoding='utf-8')) - if metadata.get('algorithm', 'ppo') != args.algorithm or metadata.get('feature_version') != 2: + if (metadata.get('algorithm', 'ppo') != args.algorithm + or metadata.get('feature_version') != 2): parser.error('初始化模型的算法/特征版本与当前训练不符，请重训或提供v2初始化模型') model = model_cls.load(args.init, env=vec_env, device=device) else: diff --git a/scripts/tune_q3_bayes.py b/scripts/tune_q3_bayes.py index fd7f3ad..29460e7 100644 --- a/scripts/tune_q3_bayes.py +++ b/scripts/tune_q3_bayes.py @@ -44,7 +44,7 @@ def evaluate(bayes_params: dict) -> tuple[float, dict]: config = replace(BASE, bayes=replace(BASE.bayes, **bayes_params)) times = [] failures = 0 - for name, sources, error_seed in SCENES: + for _name, sources, error_seed in SCENES: sim = LocalSimulator(sources, robot_id="TUNE", config=config, error_seed=error_seed) client = SimulatorClient("TUNE", config=config, diff --git a/scripts/verify_q3_practice.py b/scripts/verify_q3_practice.py index 7933b91..2c73adf 100644 --- a/scripts/verify_q3_practice.py +++ b/scripts/verify_q3_practice.py @@ -33,7 +33,10 @@ class StrictPlayback(PlaybackTransport): raise AssertionError("Replay attempted an extra action") expected = self.events[self.index] actual_request = json.loads(body) - without_id = lambda value: {k: v for k, v in value.items() if k != "request_id"} + + def without_id(value): + return {k: v for k, v in value.items() if k != "request_id"} + if (urlsplit(url).path != expected["path"] or without_id(actual_request) != without_id(expected["request"])): raise AssertionError(f"Replay action differs at index {self.index}") @@ -66,6 +69,47 @@ def save_json(path, value): allow_nan=False) + "\n", encoding="utf-8") +def timing_rows(events): + """按官方附件 2 第 4.1 节逐动作累计虚拟时间（微秒），返回明细与汇总。""" + position, channel, virtual_us = (0.0, 0.0), 1, 0 + distance_total, switch_total, move_total_us = 0.0, 0, 0 + rows = [] + previous_recorded = 0.0 + max_local_gap = 0.0 + for i, event in enumerate(events): + request, response = event["request"], event["response"] + path = event["path"] + distance, switches, action_s = 0.0, 0, 0 + if path in ("/measure", "/clear"): + target = (request["position"]["x"], request["position"]["y"]) + distance = math.dist(position, target) + position = target + if path == "/measure": + switches = int(channel != request["channel"]) + channel = request["channel"] + action_s = 5 + else: + action_s = 5 if response["clear_result"] == "success" else 3 + move_us = round(distance / 5 * 1_000_000) + virtual_us += move_us + (switches + action_s) * 1_000_000 + distance_total += distance + switch_total += switches + move_total_us += move_us + recorded = response["virtual_time_s"] + gap_us = round(recorded * 1_000_000) - virtual_us + max_local_gap = max(max_local_gap, + abs(recorded - previous_recorded - distance/5 - switches - action_s)) + previous_recorded = recorded + rows.append({"index": i, "path": path, "request_id": request["request_id"], + "channel": request.get("channel", ""), "x": position[0], "y": position[1], + "result": response.get("measure_result", response.get("clear_result", "")), + "svd_deg": response.get("svd_deg", ""), "distance_m": distance, + "move_us": move_us, "switch_s": switches, "action_s": action_s, + "recomputed_us": virtual_us, "recorded_s": recorded, "gap_us": gap_us}) + return rows, virtual_us, distance_total, move_total_us, switch_total, previous_recorded, \ + max_local_gap + + def audit(folder: Path) -> dict: records = [json.loads(line) for line in (folder / "session.jsonl").read_text(encoding="utf-8").splitlines()] @@ -130,40 +174,8 @@ def audit(folder: Path) -> dict: # Official attachment 2 section 4.1: accumulate virtual time in microseconds. # Round each move to the nearest microsecond, then add integer action costs. - position, channel, virtual_us = (0.0, 0.0), 1, 0 - distance_total, switch_total, move_total_us = 0.0, 0, 0 - rows = [] - previous_recorded = 0.0 - max_local_gap = 0.0 - for i, event in enumerate(events): - request, response, path = event["request"], event["response"], event["path"] - distance, switches, action_s = 0.0, 0, 0 - if path in ("/measure", "/clear"): - target = (request["position"]["x"], request["position"]["y"]) - distance = math.dist(position, target) - position = target - if path == "/measure": - switches = int(channel != request["channel"]) - channel = request["channel"] - action_s = 5 - else: - action_s = 5 if response["clear_result"] == "success" else 3 - move_us = round(distance / 5 * 1_000_000) - virtual_us += move_us + (switches + action_s) * 1_000_000 - distance_total += distance - switch_total += switches - move_total_us += move_us - recorded = response["virtual_time_s"] - gap_us = round(recorded * 1_000_000) - virtual_us - max_local_gap = max(max_local_gap, - abs(recorded - previous_recorded - distance/5 - switches - action_s)) - previous_recorded = recorded - rows.append({"index": i, "path": path, "request_id": request["request_id"], - "channel": request.get("channel", ""), "x": position[0], "y": position[1], - "result": response.get("measure_result", response.get("clear_result", "")), - "svd_deg": response.get("svd_deg", ""), "distance_m": distance, - "move_us": move_us, "switch_s": switches, "action_s": action_s, - "recomputed_us": virtual_us, "recorded_s": recorded, "gap_us": gap_us}) + rows, virtual_us, distance_total, move_total_us, switch_total, previous_recorded, \ + max_local_gap = timing_rows(events) checks["all_timestamps_match_microsecond_recomputation"] = all(r["gap_us"] == 0 for r in rows) real_s = (events[-1]["response"]["real_timestamp_ms"] - events[0]["response"]["real_timestamp_ms"]) / 1000 @@ -206,7 +218,8 @@ def audit(folder: Path) -> dict: "clear_time_s": sum(r["action_s"] for r in rows if r["path"] == "/clear"), "timing_audit": { "original_unquantized_mismatch_count": len(raw_timing["mismatches"]), - "original_unquantized_final_gap_s": previous_recorded - raw_timing["final_virtual_time_s"], + "original_unquantized_final_gap_s": + previous_recorded - raw_timing["final_virtual_time_s"], "max_single_action_unquantized_gap_s": max_local_gap, "microsecond_mismatch_count": sum(r["gap_us"] != 0 for r in rows), "explanation": "原复算累计未量化浮点耗时；按动作移动耗时取最近整数微秒后，所有官方时刻完全一致。", diff --git a/scripts/verify_rl_practice.py b/scripts/verify_rl_practice.py index b49325f..f29895b 100644 --- a/scripts/verify_rl_practice.py +++ b/scripts/verify_rl_practice.py @@ -116,8 +116,8 @@ if __name__ == "__main__": parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("folder", type=Path) args = parser.parse_args() - report = audit(args.folder) - print(json.dumps({"passed": report["passed"], "case_code": report["case_code"], - "problem": report["problem"], "checks": report["checks"]}, + result = audit(args.folder) + print(json.dumps({"passed": result["passed"], "case_code": result["case_code"], + "problem": result["problem"], "checks": result["checks"]}, ensure_ascii=False, indent=2)) - raise SystemExit(0 if report["passed"] else 1) + raise SystemExit(0 if result["passed"] else 1) diff --git a/src/mcm26b/belief/__init__.py b/src/mcm26b/belief/__init__.py index 6ce25c6..ef99fe5 100644 --- a/src/mcm26b/belief/__init__.py +++ b/src/mcm26b/belief/__init__.py @@ -126,10 +126,10 @@ class CandidateRegion: @classmethod def initial(cls, station: Vec2, config: ProblemConfig) -> "CandidateRegion": """首个 direction 后的基础集：Ω ∩ B(station, r_max) 的保守外包。""" - field, sides = config.field, config.search.polygon_sides - slack = field.omega_radius_m * (1 - math.cos(math.pi / sides)) - region = cls.disk((0.0, 0.0), field.omega_radius_m, sides, slack) - region.clip_disk(station, field.r_eff_max_m, sides) + field_cfg, sides = config.field, config.search.polygon_sides + slack = field_cfg.omega_radius_m * (1 - math.cos(math.pi / sides)) + region = cls.disk((0.0, 0.0), field_cfg.omega_radius_m, sides, slack) + region.clip_disk(station, field_cfg.r_eff_max_m, sides) return region # ------------------------------------------------------------ 收缩 @@ -198,8 +198,7 @@ class CandidateRegion: if len(self.polygon) == 2: p, q = self.polygon return _point_segment_distance(point, p, q) <= tol_m + self.slack_m - for index in range(len(self.polygon)): - p = self.polygon[index] + for index, p in enumerate(self.polygon): q = self.polygon[(index + 1) % len(self.polygon)] if (q[0] - p[0]) * (point[1] - p[1]) - (q[1] - p[1]) * (point[0] - p[0]) \ < -tol_m - self.slack_m: diff --git a/src/mcm26b/cli.py b/src/mcm26b/cli.py index f47038b..fab12d3 100644 --- a/src/mcm26b/cli.py +++ b/src/mcm26b/cli.py @@ -294,6 +294,13 @@ def _run(args) -> int: if judge is not None: _settle_local_judge(judge) summary["robot_id"] = args.robot_id + return _finalize_run(args, summary, strategy, truth=truth, judge=judge, + judge_session=judge_session, real_budget_s=real_budget_s) + + +def _finalize_run(args, summary, strategy, *, truth, judge, judge_session, + real_budget_s) -> int: + """补齐 summary 元信息、核对真值并输出；返回进程退出码。""" if args.problem == 4: summary["strategy"] = "rl_q4" if strategy == "rl" else strategy else: diff --git a/src/mcm26b/coverage/__init__.py b/src/mcm26b/coverage/__init__.py index a2fa11a..eb7fe55 100644 --- a/src/mcm26b/coverage/__init__.py +++ b/src/mcm26b/coverage/__init__.py @@ -32,7 +32,7 @@ class CoverageLedger: """逐频道覆盖账本：(频道, 网格点) -> 是否已测。""" config: ProblemConfig = field(default_factory=ProblemConfig) - points: list[Vec2] = field(default_factory=lambda: coverage_points()) + points: list[Vec2] = field(default_factory=coverage_points) measured: set = field(default_factory=set) # {(channel, point_index)} skipped: dict = field(default_factory=dict) # channel -> 原因 diff --git a/src/mcm26b/geometry/__main__.py b/src/mcm26b/geometry/__main__.py index 1be7c0e..e43a5a2 100644 --- a/src/mcm26b/geometry/__main__.py +++ b/src/mcm26b/geometry/__main__.py @@ -23,8 +23,10 @@ def main(argv=None): measurements = [replace(m, half_angle_deg=m.half_angle_deg+args.rounding_margin_deg) for m in measurements] result = solve(measurements) - result["error_policy"] = {"additional_half_angle_deg": args.rounding_margin_deg, - "effective_half_angles_deg": [m.half_angle_deg for m in measurements]} + result["error_policy"] = { + "additional_half_angle_deg": args.rounding_margin_deg, + "effective_half_angles_deg": [m.half_angle_deg for m in measurements], + } except (ValueError, TypeError, KeyError, OSError) as exc: parser.error(str(exc)) text = json.dumps(result, ensure_ascii=False, indent=2, allow_nan=False) diff --git a/src/mcm26b/geometry/localization.py b/src/mcm26b/geometry/localization.py index 95eb7d0..c7bfde8 100644 --- a/src/mcm26b/geometry/localization.py +++ b/src/mcm26b/geometry/localization.py @@ -133,6 +133,13 @@ def _enclosing_circle_reference(points: list[Point], eps: float = EPS): return best_c, best_r +def _circle_covers(circle, point: Point, tolerance: float) -> bool: + """Whether the circle already covers point; None means no circle yet.""" + if circle is None: + return False + return distance(circle[0], point) <= circle[1] + tolerance + + def smallest_enclosing_circle(points: list[Point], eps: float = EPS): """Welzl's randomized incremental algorithm, using an explicit call stack. @@ -169,8 +176,9 @@ def smallest_enclosing_circle(points: list[Point], eps: float = EPS): elif not returning: stack.append((n, boundary, True)) stack.append((n-1, boundary, False)) - elif circle is None or distance(circle[0], local[n-1]) > circle[1] + tolerance: + elif not _circle_covers(circle, local[n-1], tolerance): stack.append((n-1, boundary + (local[n-1],), False)) + assert circle is not None center = (origin[0]+circle[0][0]*scale, origin[1]+circle[0][1]*scale) # Recompute after transforming back: returned circle contains all input points # to distance() precision, including any roundoff from the large translation. diff --git a/src/mcm26b/harness/transport.py b/src/mcm26b/harness/transport.py index 72b5aeb..08e6075 100644 --- a/src/mcm26b/harness/transport.py +++ b/src/mcm26b/harness/transport.py @@ -29,7 +29,7 @@ class LocalSimTransport(Transport): def post(self, url, body, headers, timeout): return self.request("POST", url, body, headers, timeout) - def request(self, method, url, body, headers, timeout=None): + def request(self, method, url, body, headers, _timeout=None): """验证原始请求边界；内存回环不模拟 TCP、UI 或连接限流。""" target = urlsplit(url) raw_target = target.path + (f"?{target.query}" if target.query else "") @@ -57,5 +57,6 @@ class ScriptedFaultTransport(Transport): self.inner.post(url, body, headers, timeout) # 服务器已受理 raise TransportError("injected: read timeout after server applied") if isinstance(fault, tuple) and fault and fault[0] == "status": - return TransportResponse(int(fault[1]), b'{"accepted": false, "real_timestamp_ms": 0, "virtual_time_s": 0}') + body = b'{"accepted": false, "real_timestamp_ms": 0, "virtual_time_s": 0}' + return TransportResponse(int(fault[1]), body) return self.inner.post(url, body, headers, timeout) diff --git a/src/mcm26b/hrl/executor.py b/src/mcm26b/hrl/executor.py index 4728ef8..96f6dbe 100644 --- a/src/mcm26b/hrl/executor.py +++ b/src/mcm26b/hrl/executor.py @@ -17,7 +17,7 @@ import math import numpy as np from ..config import Vec2 -from .constants import (LOW_MAX_STEPS, LOW_NUM_ACTIONS, LOW_OBS_DIM, +from .constants import (LOW_MAX_STEPS, LOW_NUM_ACTIONS, LOW_REACH_M, LOW_STEP_M, POS_SCALE_M) __all__ = ["build_low_observation", "LowLevelExecutor", "action_bearing", diff --git a/src/mcm26b/hrl/hierarchical_env.py b/src/mcm26b/hrl/hierarchical_env.py index 38da088..92d35bb 100644 --- a/src/mcm26b/hrl/hierarchical_env.py +++ b/src/mcm26b/hrl/hierarchical_env.py @@ -144,7 +144,6 @@ class HierarchicalEnv(gym.Env): problem=self.problem) def _training_spec(self): - from ..harness import referee_spec from ..hrl.curriculum import training_spec return training_spec(self.train_config, self.stage, max_real_duration_s=self.real_budget_s) diff --git a/src/mcm26b/hrl/low_level_env.py b/src/mcm26b/hrl/low_level_env.py index 3c1d236..c8788da 100644 --- a/src/mcm26b/hrl/low_level_env.py +++ b/src/mcm26b/hrl/low_level_env.py @@ -19,7 +19,7 @@ from gymnasium import spaces from ..config import ProblemConfig from .constants import (LOW_MAX_STEPS, LOW_NUM_ACTIONS, LOW_OBS_DIM, - LOW_REACH_M, LOW_STEP_M, POS_SCALE_M) + LOW_REACH_M, LOW_STEP_M) from .executor import (action_bearing, build_low_observation, navigation_transition) diff --git a/src/mcm26b/hrl/policy.py b/src/mcm26b/hrl/policy.py index 086fada..4b71cbd 100644 --- a/src/mcm26b/hrl/policy.py +++ b/src/mcm26b/hrl/policy.py @@ -48,7 +48,7 @@ def _load_maskable(model_path: Path, device: str, *, if meta_path.exists() else {} # 高层模型必须携带完整语义元数据；缺字段不能默认兼容，防止旧槽位/ # 旧奖励语义模型被静默加载。低层导航模型只校验基础特征版本。 - strict = (act_dim == N_CANDIDATES) + strict = act_dim == N_CANDIDATES if strict: required = ("feature_version", "action_layout_version", "reward_version", "algorithm", "problem", diff --git a/src/mcm26b/hrl/scheduler.py b/src/mcm26b/hrl/scheduler.py index 34c45ca..d03a081 100644 --- a/src/mcm26b/hrl/scheduler.py +++ b/src/mcm26b/hrl/scheduler.py @@ -29,7 +29,7 @@ from typing import NamedTuple import numpy as np from ..belief import ChannelStatus -from ..config import ProblemConfig, Vec2 +from ..config import Vec2 from ..protocol.rules import CLEAR_SUCCESS, DIRECTION, NEAR, NO_SIGNAL from ..protocol.timing import move_time from .constants import (CHANNEL_COUNT, CLEAR_OFFSET, COST_SCALE_S, @@ -84,6 +84,8 @@ class HRLControllerMixin: """ problem: int + # 由宿主 BayesLoop.__init__ 初始化；此处声明让 pylint 可见该诊断计数。 + no_shrink_measures: int = 0 # ------------------------------------------------------------ 候选构建 def build_candidates(self) -> list[Candidate]: @@ -220,7 +222,7 @@ class HRLControllerMixin: def hrl_observation(self, candidates: list[Candidate]) -> np.ndarray: """构造高层观测（shape=(HIGH_OBS_DIM,)，训练/推理逐位一致）。""" config = self.config - field = config.field + field_cfg = config.field position = self.client.state.position belief = self.belief @@ -232,24 +234,24 @@ class HRLControllerMixin: glob = [ position[0] / POS_SCALE_M, position[1] / POS_SCALE_M, - min(self.client.state.virtual_time_s, field.max_virtual_duration_s) - / field.max_virtual_duration_s, + min(self.client.state.virtual_time_s, field_cfg.max_virtual_duration_s) + / field_cfg.max_virtual_duration_s, cleared / 16.0, accounted / CHANNEL_COUNT, - (self.client.state.channel - field.channel_min) - / max(field.channel_max - field.channel_min, 1), + (self.client.state.channel - field_cfg.channel_min) + / max(field_cfg.channel_max - field_cfg.channel_min, 1), float(self.problem == 4), detected_count / 16.0, ] assert len(glob) == HIGH_GLOBAL_DIM pending_station = np.zeros(CHANNEL_COUNT + 1, dtype=bool) - for index, channels in getattr(self, "_station_pending_map", {}).items(): + for channels in getattr(self, "_station_pending_map", {}).values(): for channel in channels: pending_station[channel] = True channel_features: list[float] = [] - for channel in range(field.channel_min, field.channel_max + 1): + for channel in range(field_cfg.channel_min, field_cfg.channel_max + 1): ch = belief.channels[channel] status6 = [0.0] * 6 status6[self._hrl_channel_status_index(channel)] = 1.0 @@ -363,7 +365,7 @@ class HRLControllerMixin: outcome = StepOutcome(action=action, kind=cand.kind, channel=cand.channel, position_reached=tuple(self.client.state.position)) - outcome.delta_virtual_s = (self.client.state.virtual_time_s - old_virtual) + outcome.delta_virtual_s = self.client.state.virtual_time_s - old_virtual new_detections: list[int] = [] info_gains: dict[int, float] = {} redundant = False diff --git a/src/mcm26b/particles.py b/src/mcm26b/particles.py index 5831019..49ca95e 100644 --- a/src/mcm26b/particles.py +++ b/src/mcm26b/particles.py @@ -237,10 +237,11 @@ class ChannelParticles: base = i * self.width for j in range(j0, min(j0 + _COARSE_FACTOR, self.width)): index = base + j - if index in self.alive: - dist = math.dist(self.cell_center(index), point) - if dist <= limit_m and (best is None or dist < best): - best = dist + if index not in self.alive: + continue + dist = math.dist(self.cell_center(index), point) + if dist <= limit_m and (best is None or dist < best): + best = dist return best def guaranteed_no_signal(self, station: Vec2) -> bool: diff --git a/src/mcm26b/planner/__init__.py b/src/mcm26b/planner/__init__.py index 15b911b..2b3b20e 100644 --- a/src/mcm26b/planner/__init__.py +++ b/src/mcm26b/planner/__init__.py @@ -621,7 +621,8 @@ class DirectionalExtension(OmnidirectionalLoop): self.ledger.mark_measured(channel, index) self.belief.update_measure(channel, point, outcome.result, outcome.svd_deg) if outcome.result == NEAR: - cert = clear_certificate(self.belief.candidate_set(channel), channel, self.config) + cert = clear_certificate( + self.belief.candidate_set(channel), channel, self.config) if cert.certified: self._attempt_clear(channel, point, cert) if self._count_limit_reached(): @@ -629,7 +630,8 @@ class DirectionalExtension(OmnidirectionalLoop): queue = self.belief.channels_by_status(ChannelStatus.DETECTED) while queue and self.stop_reason is None: channel = min(queue, key=lambda c: ( - math.dist(self.client.state.position, self.belief.candidate_set(c).mec()[0]), c)) + math.dist(self.client.state.position, + self.belief.candidate_set(c).mec()[0]), c)) queue.remove(channel) self._localize_and_clear(channel) if self._count_limit_reached(): @@ -672,8 +674,7 @@ class DirectionalExtension(OmnidirectionalLoop): self.directional_observed.append(channel) self.blocked_probes.setdefault(channel, []).append(point) continue # 盘外无信号也可能是距离不足，不据此断言定向 - else: - self.blocked_probes.pop(channel, None) + self.blocked_probes.pop(channel, None) steps += 1 if self.belief.channels[channel].status is ChannelStatus.CLEARED: return True diff --git a/src/mcm26b/planner/bayes_loop.py b/src/mcm26b/planner/bayes_loop.py index e7872bd..831d7fd 100644 --- a/src/mcm26b/planner/bayes_loop.py +++ b/src/mcm26b/planner/bayes_loop.py @@ -109,13 +109,15 @@ class BayesLoop(OmnidirectionalLoop): if cert is not None and cert.certified: continue region = self.belief.candidate_set(channel) - if max(math.dist(point, v) for v in region.polygon) > self.config.field.r_eff_min_m - 1e-3: + farthest = max(math.dist(point, v) for v in region.polygon) + if farthest > self.config.field.r_eff_min_m - 1e-3: continue radius = region.mec()[1] vertices = region.polygon stride = max(1, len(vertices) // self.config.bayes.mc_representatives) radii = [] - delta = self.config.field.bearing_error_half_deg + self.config.search.bearing_rounding_margin_deg + delta = (self.config.field.bearing_error_half_deg + + self.config.search.bearing_rounding_margin_deg) for target in vertices[::stride]: theta = math.degrees(math.atan2(target[1]-point[1], target[0]-point[0])) probe = CandidateRegion(list(vertices), region.slack_m) @@ -146,7 +148,7 @@ class BayesLoop(OmnidirectionalLoop): def _skip_station_channel(self, channel: int, index: int, point: Vec2) -> bool: """跳测规则（按 pf.version 缓存；幸存集单调收缩，判定单调）。""" pf = self.particles[channel] - if not len(pf): + if not pf: return True cached = self._skip_cache.get((channel, index)) if cached is not None and cached[0] == pf.version: @@ -219,7 +221,9 @@ class BayesLoop(OmnidirectionalLoop): total += self.config.cost.switch_s * (channel != task.channel) channel = task.channel elif task.kind == "station" and task.station_channels: - pending = sorted(task.station_channels, key=lambda c: (c != channel, c)) + pending = sorted( + task.station_channels, + key=lambda c, channel=channel: (c != channel, c)) total += self.config.cost.measure_s * len(pending) total += self.config.cost.switch_s * (len(pending) - (channel in pending)) channel = pending[-1] @@ -328,7 +332,7 @@ class BayesLoop(OmnidirectionalLoop): return # 已证毕：清除/缺席后的 no_signal 与距离无关，不进 PF pf = self.particles[channel] pf.apply_measure(position, outcome.result, outcome.svd_deg) - if not len(pf): + if not pf: if belief.status is ChannelStatus.UNKNOWN: self.belief.mark_absent(channel) if channel not in self.pf_absent_channels: @@ -350,7 +354,8 @@ class BayesLoop(OmnidirectionalLoop): limit = (self.config.field.clear_radius_m - pf.margin_m - self.config.search.clear_margin_m) if radius <= limit: - return self._near_clear_target(ClearCertificate(channel, center, radius, limit, True)) + return self._near_clear_target( + ClearCertificate(channel, center, radius, limit, True)) return cert def _near_clear_target(self, cert: ClearCertificate) -> ClearCertificate: @@ -368,7 +373,7 @@ class BayesLoop(OmnidirectionalLoop): """定位测量点：期望 log 幸存占比 + λ·移动时间 最小的候选。""" bayes, search = self.config.bayes, self.config.search pf = self.particles[channel] - if not len(pf): + if not pf: return None region = self.belief.candidate_set(channel) center, radius = pf.mec() diff --git a/src/mcm26b/planner/hrl_loop.py b/src/mcm26b/planner/hrl_loop.py index 3ae12e9..6918301 100644 --- a/src/mcm26b/planner/hrl_loop.py +++ b/src/mcm26b/planner/hrl_loop.py @@ -16,8 +16,6 @@ from __future__ import annotations -from pathlib import Path - from ..belief import ChannelStatus from ..config import ProblemConfig, Vec2 from ..coverage import CoverageLedger, coverage_points @@ -116,7 +114,7 @@ class HRLLoop(HRLControllerMixin, BayesLoop): if belief.status not in (ChannelStatus.CLEARED, ChannelStatus.ABSENT): if not (self.problem == 4 and outcome.result == NO_SIGNAL): pf.apply_measure(position, outcome.result, outcome.svd_deg) - if not len(pf): + if not pf: if belief.status is ChannelStatus.UNKNOWN: self.belief.mark_absent(channel) if channel not in self.pf_absent_channels: diff --git a/src/mcm26b/process_control.py b/src/mcm26b/process_control.py new file mode 100644 index 0000000..2de7e51 --- /dev/null +++ b/src/mcm26b/process_control.py @@ -0,0 +1,175 @@ +"""跨平台子进程控制：启动隔离、协作式暂停/停止、必要时强杀整棵进程树。 + +训练前端要在 POSIX 与 Windows 上表现一致，而两边的进程模型差别正好卡在 +「停止训练」这条路径上： + +- ``os.killpg`` / ``os.getpgid`` **只存在于 POSIX**；Windows 上访问它们会抛 + ``AttributeError``。旧实现只捕获 ``ProcessLookupError`` / ``PermissionError``， + 于是 ``POST /api/train/stop`` 在 Windows 上直接 500，训练进程照跑——表现就是 + 「点停止没反应」。 +- ``subprocess.Popen(start_new_session=True)`` 在 Windows 上被**静默忽略**： + CPython 的 Windows ``_execute_child`` 形参名就是 ``unused_start_new_session``， + 要隔离进程组只能传 ``CREATE_NEW_PROCESS_GROUP``。 +- POSIX 的 SIGTERM 在 Windows 上经 ``os.kill`` 会退化成 ``TerminateProcess``： + 硬杀，训练脚本里"收到信号后保存模型"的处理器根本不会执行。 + +所以「优雅暂停 / 优雅停止」不依赖信号量语义，而是走一个**控制文件**：前端原子 +写 JSON，训练脚本每步轮询。两端都只是普通文件 IO，Windows 与 POSIX 行为一致； +信号与强杀退居兜底，只在训练进程不响应控制文件时使用。 +""" + +from __future__ import annotations + +import json +import os +import signal +import subprocess +import time +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +__all__ = [ + "IS_WINDOWS", "ControlState", "spawn_kwargs", "write_control", + "read_control", "write_json_atomic", "terminate_process_tree", +] + +# 运行期常量而非"导入期分支"：调用方（含测试）可以 monkeypatch 它来模拟 Windows +IS_WINDOWS = os.name == "nt" + +# 只有 Windows 定义 CREATE_NEW_PROCESS_GROUP；POSIX 上取 0 表示不使用 creationflags +_CREATE_NEW_PROCESS_GROUP = getattr(subprocess, "CREATE_NEW_PROCESS_GROUP", 0) + +# 发出「停止」到强杀之间留给训练脚本收尾（保存模型 + 写元数据）的秒数 +DEFAULT_GRACE_S = 15.0 + + +def spawn_kwargs() -> dict[str, Any]: + """返回 ``Popen`` 的平台相关参数：让训练进程独立于前端进程组。 + + POSIX 用 ``setsid``（``start_new_session``）；Windows 上用 + ``CREATE_NEW_PROCESS_GROUP``，否则前端所在控制台的 Ctrl+C 会连带打到训练进程。 + """ + if IS_WINDOWS: + if _CREATE_NEW_PROCESS_GROUP: + return {"creationflags": _CREATE_NEW_PROCESS_GROUP} + return {} + return {"start_new_session": True} + + +@dataclass(frozen=True) +class ControlState: + """控制文件的一次快照；缺失或损坏时字段全为默认（即"无指令"）。""" + + paused: bool = False + stop: bool = False + seq: int = 0 + ts: float = 0.0 + + +def write_control(path: Path, *, paused: bool = False, stop: bool = False, + seq: int | None = None) -> ControlState: + """原子写控制文件（同目录临时文件 + ``os.replace``）。 + + 训练侧随时可能来读，必须避免读到写了一半的 JSON。``os.replace`` 在 + Windows 与 POSIX 上都是原子的（同目录、同卷）。 + """ + path = Path(path) + path.parent.mkdir(parents=True, exist_ok=True) + state = ControlState( + paused=bool(paused), + stop=bool(stop), + seq=int(seq) if seq is not None else time.time_ns(), + ts=time.time(), + ) + payload = {"paused": state.paused, "stop": state.stop, + "seq": state.seq, "ts": state.ts} + tmp = path.with_name(f".{path.name}.{os.getpid()}.tmp") + tmp.write_text(json.dumps(payload, ensure_ascii=False), encoding="utf-8") + os.replace(tmp, path) + return state + + +def read_control(path: Path) -> ControlState: + """读控制文件；缺失 / 半截 / 损坏一律当作"无指令"，绝不让训练崩在控制面上。""" + try: + raw = Path(path).read_text(encoding="utf-8") + payload = json.loads(raw) + except (OSError, ValueError, UnicodeDecodeError): + return ControlState() + if not isinstance(payload, dict): + return ControlState() + return ControlState( + paused=bool(payload.get("paused", False)), + stop=bool(payload.get("stop", False)), + seq=int(payload.get("seq") or 0), + ts=float(payload.get("ts") or 0.0), + ) + + +def write_json_atomic(path: Path, payload: Any) -> None: + """原子写 JSON 文件（同目录临时文件 + ``os.replace``）。 + + 用于"急停恢复"相关的标记文件（``resume.json`` / ``forced_stop.json`` 等）： + 读取方随时可能来读，必须避免看到写了一半的内容。 + """ + path = Path(path) + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_name(f".{path.name}.{os.getpid()}.tmp") + tmp.write_text(json.dumps(payload, ensure_ascii=False, indent=2), + encoding="utf-8") + os.replace(tmp, path) + + +def terminate_process_tree(process: subprocess.Popen | None, *, + grace_s: float = DEFAULT_GRACE_S, + poll_s: float = 0.2) -> bool: + """等待 ``grace_s`` 让协作式停止生效，超时则强杀整棵进程树。 + + 返回**是否动用了强杀**：``False`` 表示进程在宽限期内自行退出（模型有机会 + 落盘），``True`` 表示进程被强杀、最后一段产物可能缺失。调用方应先通过控制 + 文件请求停止，再调用本函数；本函数不负责写控制文件。 + """ + if process is None: + return False + if process.poll() is not None: + return False + deadline = time.monotonic() + max(0.0, grace_s) + while time.monotonic() < deadline: + if process.poll() is not None: + return False + time.sleep(min(poll_s, max(0.0, deadline - time.monotonic()))) + if process.poll() is not None: + return False + _hard_kill_tree(process) + try: + process.wait(timeout=10.0) + except subprocess.TimeoutExpired: + pass + return True + + +def _hard_kill_tree(process: subprocess.Popen) -> None: + """强杀进程树：POSIX 走进程组 SIGKILL，Windows 走 ``taskkill /T``。 + + 这里所有 POSIX 专属调用都走 ``getattr`` 探测，Windows 上拿不到分支就退回 + ``Popen.kill()``——不会再出现 ``module 'os' has no attribute 'killpg'``。 + """ + if IS_WINDOWS: + try: + subprocess.run(["taskkill", "/F", "/T", "/PID", str(process.pid)], + capture_output=True, check=False) + return + except OSError: + process.kill() + return + killpg = getattr(os, "killpg", None) + getpgid = getattr(os, "getpgid", None) + sigkill = getattr(signal, "SIGKILL", None) + if killpg is not None and getpgid is not None and sigkill is not None: + try: + killpg(getpgid(process.pid), sigkill) + return + except (ProcessLookupError, PermissionError, OSError): + pass + process.kill() diff --git a/src/mcm26b/protocol/client.py b/src/mcm26b/protocol/client.py index 13e4ec3..373f87a 100644 --- a/src/mcm26b/protocol/client.py +++ b/src/mcm26b/protocol/client.py @@ -13,6 +13,7 @@ from __future__ import annotations import json +import math import uuid from dataclasses import dataclass from time import monotonic, sleep @@ -91,7 +92,7 @@ def _validate_position(position: dict, limit: float) -> None: value = position[key] if isinstance(value, bool) or not isinstance(value, (int, float)): raise ValueError(f"position.{key} 必须是数值") - if value != value or value in (float("inf"), float("-inf")): + if not math.isfinite(value): raise ValueError(f"position.{key} 必须是有限数值") if abs(value) > limit: raise ValueError(f"position.{key} 绝对值不得超过 {limit} 米") @@ -234,7 +235,7 @@ class SimulatorClient: f"剩余现实时间 {self.real_remaining_s:.1f}s 不足以预留 {margin:.1f}s 退出余量") # ---------------------------------------------------------------- 内部机制 - def _payload(self, path: str, *, position: Vec2 | None = None, + def _payload(self, _path: str, *, position: Vec2 | None = None, channel: int | None = None) -> dict: payload = {"arena_id": self.session.arena_id, "robot_id": self.robot_id, diff --git a/src/mcm26b/rl/env.py b/src/mcm26b/rl/env.py index ea48e40..dfd5c27 100644 --- a/src/mcm26b/rl/env.py +++ b/src/mcm26b/rl/env.py @@ -27,7 +27,7 @@ from ..config import ProblemConfig from ..planner import clear_certificate, fallback_clear_cells, safe_disk_radius from ..protocol.client import SimulatorClient from ..protocol.log import SessionLog -from ..protocol.rules import CLEAR_SUCCESS, DIRECTION, NEAR, NO_SIGNAL +from ..protocol.rules import CLEAR_SUCCESS, DIRECTION, NO_SIGNAL from .features import NUM_ACTIONS, OBS_DIM, action_mask, build_observation, candidate_actions @@ -65,9 +65,9 @@ class LocalizationEnv(gym.Env): from referee import LocalSimulator as _RefereeLocalSimulator from referee import Source from ..harness import LocalSimTransport, referee_spec - self._Source = Source - self._LocalSimulator = _RefereeLocalSimulator - self._LocalSimTransport = LocalSimTransport + self._source_cls = Source + self._local_simulator_cls = _RefereeLocalSimulator + self._local_sim_transport_cls = LocalSimTransport self._referee_spec = referee_spec self._scene_rng = Random() @@ -106,23 +106,22 @@ class LocalizationEnv(gym.Env): # 首次检测点：距离 [300, min(1450, r_eff-20)] 内随机，定向源取朝向 # ±89.9°（接近但不跨越90°边界，保证direction）。机器狗可到Ω外 [题面]。 max_distance = max(310.0, min(1450.0, r_eff - 20.0)) - if max_distance < 310.0: - max_distance = 310.0 + max_distance = max(max_distance, 310.0) probe_distance = rng.uniform(300.0, max_distance) probe_bearing = (heading + rng.uniform(-89.9, 89.9) if directional else rng.uniform(0.0, 360.0)) probe_radians = math.radians(probe_bearing) probe = (x + probe_distance * math.cos(probe_radians), y + probe_distance * math.sin(probe_radians)) - source = self._Source(channel=channel, x=x, y=y, r_eff_m=r_eff, - directional=directional, heading_deg=heading) + source = self._source_cls(channel=channel, x=x, y=y, r_eff_m=r_eff, + directional=directional, heading_deg=heading) return source, probe def _make_episode(self) -> bool: """重建模拟器并完成首次测向；返回 False 表示连续未得到 direction。""" for _ in range(20): self._source, probe = self._make_source_and_probe() - self._simulator = self._LocalSimulator( + self._simulator = self._local_simulator_cls( [self._source], robot_id="rl-train", spec=self._referee_spec( self.config, max_real_duration_s=self.real_budget_s), @@ -131,7 +130,7 @@ class LocalizationEnv(gym.Env): stress_mode=True) self._client = SimulatorClient( "rl-train", config=self.config, - transport=self._LocalSimTransport(self._simulator), + transport=self._local_sim_transport_cls(self._simulator), log=SessionLog()) self._client.enter() outcome = self._client.measure(probe, self._source.channel) @@ -149,7 +148,9 @@ class LocalizationEnv(gym.Env): if self._scene_rng.random() < self.handoff_probability: angle = self._scene_rng.uniform(0, 2*math.pi) reach = self.config.field.omega_radius_m * math.sqrt(self._scene_rng.random()) - other = self.config.field.channel_min if self._channel != self.config.field.channel_min else self.config.field.channel_max + other = (self.config.field.channel_min + if self._channel != self.config.field.channel_min + else self.config.field.channel_max) self._client.measure((reach*math.cos(angle), reach*math.sin(angle)), other) self._start_virtual_s = self._client.state.virtual_time_s return True @@ -325,5 +326,6 @@ class LocalizationEnv(gym.Env): return observation, float(reward), self._terminated, self._truncated, info def close(self): - if self._client is not None and self._client.state.entered and not self._client.state.exited: + if (self._client is not None and self._client.state.entered + and not self._client.state.exited): self._client.exit() diff --git a/src/mcm26b/rl/features.py b/src/mcm26b/rl/features.py index 183dd86..e30ed18 100644 --- a/src/mcm26b/rl/features.py +++ b/src/mcm26b/rl/features.py @@ -19,7 +19,7 @@ import math import numpy as np from ..config import ProblemConfig, Vec2 -from ..planner import candidate_points, safe_disk_radius +from ..planner import safe_disk_radius from ._constants import GLOBAL_DIM, LADDER_SLOTS, NUM_ACTIONS, OBS_DIM, PER_ACTION_DIM __all__ = [ @@ -100,7 +100,8 @@ def action_mask(region, position: Vec2, config: ProblemConfig, problem: int, Q3有距离保证盘时屏蔽盘外探点；Q4允许盘外探点，但绝不把距离保证 当作朝向保证。训练与推理共用此函数，不读取源位置或朝向。 """ - key = lambda p: (round(p[0]*2), round(p[1]*2)) + def key(p): + return (round(p[0]*2), round(p[1]*2)) visited = {key(r['position']) for r in getattr(belief, 'measurements', [])} seen = set() center, _ = region.mec() diff --git a/src/mcm26b/rl/policy.py b/src/mcm26b/rl/policy.py index 3b7b2bd..5a5ca97 100644 --- a/src/mcm26b/rl/policy.py +++ b/src/mcm26b/rl/policy.py @@ -40,7 +40,8 @@ class PointSelectionPolicy: device = "cpu" self.device = device meta_path = path.with_suffix('.json') - self.metadata = json.loads(meta_path.read_text(encoding='utf-8')) if meta_path.exists() else {} + self.metadata = (json.loads(meta_path.read_text(encoding='utf-8')) + if meta_path.exists() else {}) self.feature_version = self.metadata.get('feature_version', 1) if self.feature_version not in (1, 2): raise RuntimeError('不支持的RL特征版本') @@ -78,7 +79,9 @@ class PointSelectionPolicy: region, position, channel, config, problem, blocked, belief, virtual_time_s=virtual_time_s, receiver_channel=receiver_channel if self.feature_version >= 2 else None) - kwargs = {'action_masks': action_mask(region, position, config, problem, blocked, belief)} if self.masked else {} + kwargs = ({'action_masks': action_mask(region, position, config, problem, + blocked, belief)} + if self.masked else {}) action, _state = self.model.predict(observation, deterministic=True, **kwargs) action = int(action) candidates = candidate_actions(region, position, config, blocked) diff --git a/src/mcm26b/training_ui/analysis.py b/src/mcm26b/training_ui/analysis.py new file mode 100644 index 0000000..a48ff28 --- /dev/null +++ b/src/mcm26b/training_ui/analysis.py @@ -0,0 +1,397 @@ +"""训练快照的纯函数派生：曲线统计、收敛判读、进程视图。 + +从 ``server.py`` 拆出来的两个理由： + +1. 断开 ``demo`` ↔ ``server`` 的循环导入——离线预览（``demo.py``）只想复用 + 这两个纯函数，没必要反向 import 整个 HTTP 服务器模块； +2. 让"数值判读"可以脱离 HTTP 与子进程单独测试。 + +本模块只依赖标准库，不做任何 IO、不持有任何状态。 +""" + +from __future__ import annotations + +import math +import re +import subprocess +import time +from typing import Any + +__all__ = ["MAX_LOG_LINES", "MAX_METRICS", "finite_float", + "summarize_convergence", "describe_process", "process_pid"] + +MAX_LOG_LINES = 4000 +MAX_METRICS = 5000 + +_NUM_RE = re.compile(r"^[+-]?(?:\d+\.?\d*|\.\d+)(?:[eE][+-]?\d+)?$") + + +def finite_float(value: Any) -> float | None: + """把任意输入转成有限浮点数；NaN / ±inf / 非数字一律返回 None。 + + 旧实现用 ``value != value`` 判 NaN，pylint 报 ``comparison-with-itself``； + 这里用 ``math.isfinite`` 表达同一语义，意图更直白。 + """ + if value is None or isinstance(value, bool): + return None + if isinstance(value, (int, float)): + value = float(value) + if not math.isfinite(value): + return None + return value + text = str(value).strip() + if not _NUM_RE.match(text): + return None + try: + return float(text) + except ValueError: + return None + + +def _series(metrics: list[dict[str, Any]], key: str) -> list[tuple[float, float]]: + points: list[tuple[float, float]] = [] + for point in metrics: + value = finite_float(point.get(key)) + steps = finite_float(point.get("timesteps")) or finite_float( + point.get("total_timesteps")) + if value is None: + continue + points.append((steps if steps is not None else float(len(points) + 1), value)) + return points + + +def _last_value(points: list[tuple[float, float]]) -> float | None: + return points[-1][1] if points else None + + +def _spread(points: list[tuple[float, float]], window: int = 10 + ) -> tuple[float, float] | None: + """最近 window 个点的 (极差, 起点绝对值)，用于判断"是否还在动"。""" + sample = points[-window:] + if len(sample) < 3: + return None + ys = [item[1] for item in sample] + return max(ys) - min(ys), abs(ys[0]) + + +def _cv(points: list[tuple[float, float]], window: int = 10) -> float | None: + """最近 window 个点的变异系数（标准差/|均值|），越小越平稳。""" + sample = [item[1] for item in points[-window:]] + if len(sample) < 3: + return None + mean = sum(sample) / len(sample) + if abs(mean) < 1e-12: + return None + var = sum((value - mean) ** 2 for value in sample) / len(sample) + return (var ** 0.5) / abs(mean) + + +def _slope(points: list[tuple[float, float]], window: int = 8) -> float | None: + sample = points[-window:] + if len(sample) < 3: + return None + xs = [item[0] for item in sample] + ys = [item[1] for item in sample] + n = len(xs) + mean_x = sum(xs) / n + mean_y = sum(ys) / n + denom = sum((x - mean_x) ** 2 for x in xs) + if abs(denom) < 1e-12: + return None + return sum((x - mean_x) * (y - mean_y) for x, y in zip(xs, ys)) / denom + + +def _rate_per_1k(reward_points: list[tuple[float, float]], + window: int = 8) -> float | None: + """最近 window 个点的奖励增益换算成"每 1000 环境步"，便于读趋势。""" + slope = _slope(reward_points, window) + if slope is None: + return None + return slope * 1000.0 + + +def _fmt_num(value: float | None, digits: int = 4) -> str: + if value is None: + return "—" + if value != 0 and abs(value) < 1e-3: + return f"{value:.3e}" + return f"{value:.{digits}g}" + + +def _flatness(points: list[tuple[float, float]], window: int = 10 + ) -> tuple[float, float] | None: + """窗口内"趋势位移 / 逐点噪声"比值：<1 表示看不出系统性变化。 + + 判定收敛时只看斜率绝对值会把"很慢但持续的提升"误判为平台；这里把窗口 + 对半切，用两半均值之差当趋势位移，用相邻点平均绝对差当噪声尺度。返回 + (比值, 噪声尺度)，两者都小才是真的"又平又稳"。 + """ + sample = points[-window:] + if len(sample) < 4: + return None + ys = [item[1] for item in sample] + half = len(ys) // 2 + shift = abs(sum(ys[half:]) / len(ys[half:]) - sum(ys[:half]) / len(ys[:half])) + noise = sum(abs(b - a) for a, b in zip(ys, ys[1:])) / (len(ys) - 1) + if noise <= 0: + return (0.0, 0.0) if shift <= 0 else (float("inf"), 0.0) + return shift / noise, noise + + +def summarize_convergence(metrics: list[dict[str, Any]], + monitor: list[dict[str, Any]]) -> dict[str, Any]: + """给前端返回可解释的收敛状态与各参数最近值。 + + 这是**启发式判读**，不替代正式学习曲线分析：只依据最近窗口的斜率、 + 极差与变异系数给出"平坦/上升/未上升"的分类，并把同一组数字交给前端 + 展示，便于人工复核。 + """ + reward_points = _series(metrics, "ep_rew_mean") + if not reward_points and monitor: + reward_points = [(float(index), float(row.get("r", 0.0))) + for index, row in enumerate(monitor) + if finite_float(row.get("r")) is not None] + kl_points = _series(metrics, "approx_kl") + ev_points = _series(metrics, "explained_variance") + vf_points = _series(metrics, "value_loss") + pg_points = _series(metrics, "policy_gradient_loss") + entropy_points = _series(metrics, "entropy_loss") + fps_points = _series(metrics, "fps") + + reward_slope = _slope(reward_points) + reward_rate = _rate_per_1k(reward_points) + spread = _spread(reward_points) + cv = _cv(reward_points) + latest_kl = _last_value(kl_points) + latest_ev = _last_value(ev_points) + latest_vf = _last_value(vf_points) + latest_pg = _last_value(pg_points) + latest_entropy = _last_value(entropy_points) + latest_fps = _last_value(fps_points) + latest_timesteps = metrics[-1].get("timesteps") if metrics else None + if latest_timesteps is None and metrics: + latest_timesteps = metrics[-1].get("total_timesteps") + latest_reward = _last_value(reward_points) + latest_len = _last_value(_series(metrics, "ep_len_mean")) + latest_updates = _last_value(_series(metrics, "n_updates")) + window = min(10, max(3, len(reward_points))) + reward_window = [round(value, 6) for _, value in reward_points[-window:]] + best_reward = max((value for _, value in reward_points), default=None) + + notes: list[str] = [] + if not metrics and not monitor: + status, tone = "等待首个 rollout", "gray" + notes.append("训练进程启动后先输出环境初始化与首个 rollout 的进度行。") + elif not metrics: + status, tone = "已收到回合奖励，等待进度行", "warn" + notes.append("Monitor CSV 已有回合记录，但尚未收到 --progress-json 进度行；" + "首个 rollout 结束后会出现完整指标。") + elif len(metrics) < 3: + status, tone = "预热阶段", "warn" + notes.append("已收到初始指标，需要更多 rollout 才能判断趋势。") + else: + tone, status = "ok", "收敛趋势良好" + # 平台判据（三条同时成立）：对半窗口看不出位移、窗口内波动小、 + # 逐点噪声本身也小。慢速但持续的提升会因第一条被排除。 + flat = _flatness(reward_points) + plateau = False + if flat is not None and cv is not None and latest_reward is not None: + flat_ratio, flat_noise = flat + scale = max(abs(latest_reward), 1e-9) + plateau = (flat_ratio <= 1.0 and cv <= 0.15 + and flat_noise <= 0.02 * scale + and (spread is None or spread[1] < 1e-12 + or spread[0] <= 0.15 * spread[1])) + if latest_kl is not None and latest_kl > 0.05: + status, tone = "KL 偏大，策略更新可能过猛", "bad" + notes.append(f"approx_kl={latest_kl:.4f} > 0.05") + elif latest_ev is not None and latest_ev < 0.0: + status, tone = "价值函数尚未拟合", "warn" + notes.append(f"explained_variance={latest_ev:.3f} < 0") + elif reward_slope is not None and reward_slope < 0: + status, tone = "奖励未形成上升趋势", "warn" + notes.append(f"最近 {window} 点奖励斜率为负（{_fmt_num(reward_slope)}/step）") + elif plateau: + status, tone = "已趋于平稳（疑似收敛）", "ok" + notes.append("窗口内对半位移小于逐点噪声，且波动很小" + f"（变异系数 {_fmt_num(cv, 3)}）") + else: + notes.append(f"最近 {window} 点奖励斜率为正（{_fmt_num(reward_slope)}/step）") + if latest_kl is not None and latest_kl <= 0.02: + notes.append(f"approx_kl={latest_kl:.4f} 较稳定") + if latest_ev is not None: + notes.append(f"explained_variance={latest_ev:.3f}") + if cv is not None: + notes.append(f"奖励变异系数={_fmt_num(cv, 3)}") + if latest_entropy is not None and abs(latest_entropy) < 1e-3: + notes.append("策略熵接近 0，探索可能过早停止") + return { + "status": status, + "tone": tone, + "notes": notes, + "window": window, + "latest_timesteps": latest_timesteps, + "reward_slope": reward_slope, + "reward_rate_per_1k": reward_rate, + "reward_cv": cv, + "reward_spread": spread[0] if spread else None, + "reward_window": reward_window, + "best_reward": best_reward, + "latest_reward": latest_reward, + "latest_ep_len": latest_len, + "latest_updates": latest_updates, + "latest_kl": latest_kl, + "latest_explained_variance": latest_ev, + "latest_value_loss": latest_vf, + "latest_policy_loss": latest_pg, + "latest_entropy_loss": latest_entropy, + "latest_fps": latest_fps, + "reward_points": len(reward_points), + } + + +# 快照状态 -> 界面展示用的进程语义（阶段名称、色调、是否终态） +_STATUS_VIEW = { + "created": ("已创建", "gray", False, "命令已构造，等待 spawn"), + "running": ("运行中", "ok", False, "子进程持有控制台输出与独立进程组"), + "paused": ("已暂停", "warn", False, "进程存活但停在 step 边界；点「继续」从暂停处恢复"), + "stopping": ("停止中", "warn", False, "已请求优雅停止，等待子进程保存并退出"), + "stopped": ("已手动停止", "warn", True, "收到停止请求后退出；脚本会在 step 边界保存模型与元数据"), + "exited": ("正常结束", "ok", True, "退出码 0：训练与评估流程走完"), + "failed": ("异常结束", "bad", True, "非零退出码或启动失败，见日志尾部"), +} + +_PHASES = [ + ("created", "构造命令"), + ("spawn", "启动子进程"), + ("rollout", "采集 rollout"), + ("update", "PPO 更新"), + ("evaluate", "评估 / 保存"), + ("done", "收敛归档"), +] + +# 状态 -> 当前所处的阶段节点；节点之前为已完成，之后为未开始。 +_STATUS_PHASE = { + "created": "created", + "running": "update", + "paused": "update", + "stopping": "update", + "stopped": "done", + "exited": "done", + "failed": "done", +} + + +def process_pid(process: subprocess.Popen | None) -> int | None: + """已退出进程的 pid 访问会抛 OSError；这里统一吞掉，只用于展示。""" + if process is None: + return None + try: + return int(process.pid) + except (OSError, ValueError): + return None + + +def describe_process(snapshot: dict[str, Any]) -> dict[str, Any]: + """把 snapshot 派生为界面直接可渲染的进程对象。 + + 包含：状态语义词、进度（步/百分比）、墙钟与剩余时间估计、最近吞吐、 + 进程时间线节点。数字都取自 snapshot，前端不再重复累加，避免两个刷新 + 频率不同的计数器互相打架。 + """ + status = str(snapshot.get("status") or "created") + label, tone, terminal, hint = _STATUS_VIEW.get( + status, (status, "gray", False, "")) + config = snapshot.get("config") or {} + order = [key for key, _ in _PHASES] + current = _STATUS_PHASE.get(status, "created") + timeline = [] + for key, title in _PHASES: + index = order.index(key) + if index < order.index(current): + state = "done" + elif index == order.index(current): + state = "active" + else: + state = "pending" + timeline.append({"key": key, "label": title, "state": state}) + + metrics = snapshot.get("metrics") or [] + latest: dict[str, Any] = metrics[-1] if metrics else {} + convergence = snapshot.get("convergence") or {} + total_steps = finite_float(config.get("timesteps")) or 0.0 + # SB3 表格行可能没有 steps 字段，因此进度取"所有进度点的最大步数"， + # 而不是只看最后一个点——否则末行恰好缺字段时进度会回退到 0。 + done_steps = 0.0 + for point in metrics: + value = (finite_float(point.get("timesteps")) + or finite_float(point.get("total_timesteps"))) + if value is not None and value > done_steps: + done_steps = value + if done_steps == 0.0: + done_steps = finite_float(convergence.get("latest_timesteps")) or 0.0 + # init/resume（课程迁移）时 num_timesteps 从上一阶段继续，进度必须按本次 + # 增量算；start_timesteps 由训练脚本的进度 JSON 提供，缺省按 0 处理。 + start_steps = 0.0 + for point in metrics: + value = finite_float(point.get("start_timesteps")) + if value is not None: + start_steps = value + break + trained_steps = max(0.0, done_steps - start_steps) + percent = (100.0 * trained_steps / total_steps) if total_steps > 0 else None + if percent is not None: + percent = max(0.0, min(100.0, percent)) + elapsed = float(snapshot.get("elapsed_s") or 0.0) + rate = _latest_rate(metrics) + remaining: float | None = None + eta_ts: float | None = None + if rate and rate > 0 and total_steps > 0 and trained_steps < total_steps: + remaining = (total_steps - trained_steps) / rate + eta_ts = float(snapshot.get("finished_at") or time.time()) + remaining + if not terminal and remaining is None: + cap = finite_float(config.get("max_wall_s")) or 0.0 + if cap > 0: + remaining = max(0.0, cap - elapsed) + return { + "label": label, + "tone": tone, + "terminal": terminal, + "hint": hint, + "phase": current, + "timeline": timeline, + "percent": percent, + "done_steps": done_steps, + "start_steps": start_steps, + "trained_steps": trained_steps, + "total_steps": total_steps, + "elapsed_s": round(elapsed, 3), + "rate_per_s": rate, + "remaining_s": remaining, + "eta_epoch_s": eta_ts, + "log_total": snapshot.get("log_total"), + "log_capacity": MAX_LOG_LINES, + "metric_total": snapshot.get("metric_total"), + "latest": latest, + } + + +def _latest_rate(metrics: list[dict[str, Any]], window: int = 6) -> float | None: + """最近若干个**带 fps 的**进度点的平均吞吐（步/秒），用于剩余时间估计。 + + progress JSON 行不含 fps，只有 SB3 表格行有；若只看最后 window 个点， + 恰好落在若干 JSON 行之间时会得到 None，剩余时间就会退化成墙钟上限的 + 粗糙估计。这里向前扫描直到拿到至少 1 个 fps（最多看 3*window 个点）。 + """ + if not metrics: + return None + usable: list[float] = [] + for point in reversed(metrics[-window * 3:]): + value = finite_float(point.get("fps")) + if value and value > 0: + usable.append(value) + if len(usable) >= window: + break + if not usable: + return None + return sum(usable) / len(usable) diff --git a/src/mcm26b/training_ui/demo.py b/src/mcm26b/training_ui/demo.py index 161c326..914777d 100644 --- a/src/mcm26b/training_ui/demo.py +++ b/src/mcm26b/training_ui/demo.py @@ -15,6 +15,8 @@ import random import time from typing import Any +from .analysis import describe_process, summarize_convergence + __all__ = ["DEMO_CONFIG", "DEMO_SCENARIOS", "demo_metrics", "demo_monitor", "demo_log_lines", "demo_snapshot"] @@ -44,7 +46,7 @@ DEMO_STEPS = 48 def _finite(value: float) -> float | None: - if value != value or value in (float("inf"), float("-inf")): + if not math.isfinite(value): return None return round(value, 8) @@ -171,8 +173,6 @@ def demo_snapshot(*, log_tail: int = 400, scenario: str = "rising", （而不是刚好 100%）；``scenario`` 选择收敛判读分支；``fps`` 决定剩余 时间估计的吞吐量级。 """ - from .server import describe_process, summarize_convergence - now = time.time() ratio = min(1.0, max(0.05, float(done_ratio))) metrics = demo_metrics(count=max(6, int(round(DEMO_STEPS * ratio))), @@ -193,6 +193,7 @@ def demo_snapshot(*, log_tail: int = 400, scenario: str = "rising", "elapsed_s": 184.0, "returncode": None, "error": None, + "forced_kill": False, "command": list(DEMO_COMMAND), "model": str(config["model"]), "config": config, @@ -203,6 +204,17 @@ def demo_snapshot(*, log_tail: int = 400, scenario: str = "rising", "metric_end": len(metrics), "metric_total": len(metrics), "monitor": monitor, + "recovery": { + "can_resume": True, + "model": str(config["model"]), + "timesteps": int(DEMO_STEPS * ratio * 12), + "reason": "recovery", + "updated_unix_s": now - 24.0, + "source": "resume.json", + "hint_file": str(config["model"]).rsplit("/", 1)[0] + "/resume.json", + "candidates": 3, + "preview": True, + }, "preview": True, "preview_scenario": scenario, } diff --git a/src/mcm26b/training_ui/schema.py b/src/mcm26b/training_ui/schema.py index bad3c8b..05940e7 100644 --- a/src/mcm26b/training_ui/schema.py +++ b/src/mcm26b/training_ui/schema.py @@ -163,6 +163,24 @@ CONFIG_FIELDS: list[dict[str, Any]] = [ "label": "覆盖已有模型 --force", "default": False, "help": "输出路径已存在时必须勾选，否则训练脚本会拒绝覆盖。与续训互斥。", }, + # ---------------------------------------------------------- 恢复与落盘 + { + "key": "recovery_every", "kind": "integer", "group": "recovery", + "label": "滚动恢复点间隔 --recovery-every", "default": 2000, + "help": "每 N 个环境步原子覆盖写 /_latest.zip，" + "训练开始前先写起步基线；急停（强杀）后最多丢 N 步。0=关闭。", + }, + { + "key": "checkpoint_every", "kind": "integer", "group": "recovery", + "label": "历史 checkpoint 间隔 --checkpoint-every", "default": 20000, + "help": "每 N 个环境步另留一份 _<步数>.zip 供返祖对比；" + "0=只保留滚动恢复点。", + }, + { + "key": "checkpoint_keep", "kind": "integer", "group": "recovery", + "label": "历史 checkpoint 保留数 --checkpoint-keep", "default": 3, + "help": "只保留最近 K 个历史 checkpoint；滚动恢复点与起步基线不受影响。", + }, ] CONFIG_GROUPS: list[dict[str, str]] = [ @@ -170,6 +188,7 @@ CONFIG_GROUPS: list[dict[str, str]] = [ {"key": "ppo", "label": "PPO 超参数", "hint": "采样宽度与更新强度"}, {"key": "runtime", "label": "运行环境", "hint": "并行、设备与低层执行器"}, {"key": "scale", "label": "训练规模", "hint": "步数、墙钟上限与评估"}, + {"key": "recovery", "label": "恢复与落盘", "hint": "急停恢复点与历史 checkpoint"}, {"key": "paths", "label": "模型与迁移", "hint": "产物路径、续训与课程迁移"}, ] diff --git a/src/mcm26b/training_ui/server.py b/src/mcm26b/training_ui/server.py index ed5e581..eb956d2 100644 --- a/src/mcm26b/training_ui/server.py +++ b/src/mcm26b/training_ui/server.py @@ -4,7 +4,8 @@ 设计边界： - 只监听 127.0.0.1，只允许白名单静态文件； - 只允许一个训练任务，命令由服务端用参数列表构造，不经过 shell； -- 支持启动/停止/状态查询；停止时发送 SIGTERM 到独立进程组； +- 支持启动/暂停/继续/停止/状态查询；暂停与停止通过**控制文件**下发， + POSIX 与 Windows 行为一致（详见 :mod:`mcm26b.process_control`）； - 训练进程通过 ``--progress-json`` 输出单行 JSON 进度，服务端同时兼容 Stable-Baselines3 的表格输出和 Monitor CSV。 @@ -12,15 +13,17 @@ - ``GET /api/train/schema`` 字段/分组/预设/指标字典（``schema.py`` 单一出处） - ``GET /api/train/state`` 快照；``after_metric=N`` 只回增量指标 - ``POST /api/train/start`` 以部分字段启动（其余取默认值） -- ``POST /api/train/stop`` SIGTERM 独立进程组 +- ``POST /api/train/pause`` 请求训练进程在 step 边界挂起 +- ``POST /api/train/resume`` 解除挂起 +- ``POST /api/train/stop`` 请求优雅停止，超时后强杀进程树 """ from __future__ import annotations +import dataclasses import json import os import re -import signal import subprocess import sys import threading @@ -32,15 +35,24 @@ from pathlib import Path from typing import Any from urllib.parse import parse_qs, urlsplit +from ..process_control import (DEFAULT_GRACE_S, spawn_kwargs, + terminate_process_tree, write_control, + write_json_atomic) +from .analysis import (MAX_LOG_LINES, MAX_METRICS, describe_process, finite_float, + process_pid, summarize_convergence) from .demo import DEMO_SCENARIOS, demo_snapshot from .schema import config_schema __all__ = ["TrainingJob", "TrainingUI", "run_training_ui", "build_command", - "summarize_convergence", "describe_process"] + "summarize_convergence", "describe_process", "discover_recovery"] STATIC_DIR = Path(__file__).resolve().parent / "static" REPO_ROOT = Path(__file__).resolve().parents[3] TRAIN_SCRIPT = REPO_ROOT / "scripts" / "train_hrl_high.py" +# 控制文件集中放在 output/ 下（已 gitignore），job 之间互不干扰 +CONTROL_DIR = REPO_ROOT / "output" / "hrl" / "ui" / "control" +# 从发出停止请求到强杀之间留给训练脚本收尾的秒数 +STOP_GRACE_S = DEFAULT_GRACE_S STATIC_FILES = { "/": ("index.html", "text/html; charset=utf-8"), "/index.html": ("index.html", "text/html; charset=utf-8"), @@ -49,10 +61,6 @@ STATIC_FILES = { "/style.css": ("style.css", "text/css; charset=utf-8"), } -MAX_LOG_LINES = 4000 -MAX_METRICS = 5000 - -_NUM_RE = re.compile(r"^[+-]?(?:\d+\.?\d*|\.\d+)(?:[eE][+-]?\d+)?$") _SB3_SECTIONS = ("rollout/", "time/", "train/", "system/") # 路径/字符串参数只拒绝控制字符（含 \r\n\t 与 DEL），不做 shell 元字符过滤： # 命令走参数列表、不经过 shell，这里只是防日志与命令行被注入控制序列。 @@ -61,23 +69,6 @@ _SB3_SECTIONS = ("rollout/", "time/", "train/", "system/") _SAFE_ARG_RE = re.compile(r"^[^\x00-\x1f\x7f]{1,512}$") -def _finite_float(value: Any) -> float | None: - if value is None or isinstance(value, bool): - return None - if isinstance(value, (int, float)): - value = float(value) - if value != value or value in (float("inf"), float("-inf")): - return None - return value - text = str(value).strip() - if not _NUM_RE.match(text): - return None - try: - return float(text) - except ValueError: - return None - - def _normalize_metric(key: str) -> str: """把 SB3 的 ``rollout/ep_rew_mean`` 与进度 JSON 的 ``ep_rew_mean`` 归一化到同一短名，便于前端绘制。""" @@ -109,6 +100,10 @@ class TrainConfig: max_wall_s: float = 1200.0 local_selector: str = "geometry" local_model: str | None = None + # 落盘/恢复：默认值与 scripts/train_hrl_high.py 保持一致 + checkpoint_every: int = 20000 + checkpoint_keep: int = 3 + recovery_every: int = 2000 verbose: int = 1 @classmethod @@ -117,9 +112,11 @@ class TrainConfig: raise ValueError("请求体必须是 JSON 对象") defaults = cls() data: dict[str, Any] = {} - for field in cls.__dataclass_fields__: - if field in payload and payload[field] is not None: - data[field] = payload[field] + # 用 dataclasses.fields 而不是 cls.__dataclass_fields__：后者是内部 + # 实现细节，pylint 看不到它，也无法保证未来版本仍存在。 + for field in dataclasses.fields(cls): + if field.name in payload and payload[field.name] is not None: + data[field.name] = payload[field.name] cfg = cls(**{**defaults.__dict__, **data}) cfg.validate() return cfg @@ -153,6 +150,12 @@ class TrainConfig: raise ValueError("max_wall_s 必须在 0..7200 秒") if int(self.eval_episodes) < 0: raise ValueError("eval_episodes 不能为负") + if not 0 <= int(self.checkpoint_every) <= 10_000_000: + raise ValueError("checkpoint_every 必须在 0..10,000,000") + if not 1 <= int(self.checkpoint_keep) <= 100: + raise ValueError("checkpoint_keep 必须在 1..100") + if not 0 <= int(self.recovery_every) <= 10_000_000: + raise ValueError("recovery_every 必须在 0..10,000,000") for label, value in (("model", self.model), ("init", self.init), ("local_model", self.local_model)): if value is None: @@ -207,6 +210,9 @@ def build_command(config: TrainConfig) -> list[str]: "--n-epochs", str(config.n_epochs), "--max-wall-s", str(config.max_wall_s), "--local-selector", config.local_selector, + "--checkpoint-every", str(config.checkpoint_every), + "--checkpoint-keep", str(config.checkpoint_keep), + "--recovery-every", str(config.recovery_every), "--progress-json", "--verbose", str(config.verbose), ] @@ -223,25 +229,118 @@ def build_command(config: TrainConfig) -> list[str]: return cmd -class TrainingJob: - """单个训练子进程的生命周期、日志和指标缓存。""" +def discover_recovery(run_dir: Path, model_path: Path) -> dict[str, Any]: + """找出"现在就能继续训练"的那份产物。 - def __init__(self, config: TrainConfig): + 优先级： + 1. ``/resume.json``——训练脚本写的权威指针（校验目标文件真实存在）； + 2. 否则按 mtime 取最新：最终模型 / 滚动恢复点 / 历史 checkpoint / 异常兜底。 + + 只认**已完整落盘**的文件：模型的 zip 与 json 都是原子替换写的，所以不会 + 出现"文件在、但打不开"的半截快照。 + """ + run_dir = Path(run_dir) + model_path = Path(model_path) + hint_file = run_dir / "resume.json" + hint = _read_json_file(hint_file) + + candidates: list[tuple[float, str, str]] = [] + + def _consider(path: Path, reason: str) -> None: + try: + stat = path.stat() + except OSError: + return + if stat.st_size > 0: + candidates.append((stat.st_mtime, str(path), reason)) + + _consider(model_path, "final_model") + ckpt_dir = run_dir / "checkpoints" + if ckpt_dir.is_dir(): + _consider(ckpt_dir / f"{model_path.stem}_ckpt_latest.zip", "recovery") + for path in ckpt_dir.glob(f"{model_path.stem}_ckpt_[0-9]*.zip"): + if path.stem.rsplit("_", 1)[-1].isdigit(): + _consider(path, "checkpoint") + _consider(run_dir / f"{model_path.stem}_emergency.zip", "emergency") + + state: dict[str, Any] = { + "can_resume": False, "model": None, "timesteps": None, + "reason": "none", "updated_unix_s": None, "source": "none", + "hint_file": str(hint_file), "candidates": len(candidates), + } + if hint and hint.get("model") and Path(str(hint["model"])).is_file(): + state.update({ + "model": str(hint["model"]), + "timesteps": _int_or_none(hint.get("timesteps")), + "reason": str(hint.get("reason") or "resume_hint"), + "updated_unix_s": _float_or_none(hint.get("created_unix_s")), + "source": "resume.json", + }) + elif candidates: + mtime, path, reason = max(candidates, key=lambda item: item[0]) + state.update({ + "model": path, "reason": reason, + "updated_unix_s": mtime, "source": "scan", + }) + if state["model"]: + state["can_resume"] = Path(str(state["model"])).is_file() + return state + + +def _read_json_file(path: Path) -> dict[str, Any] | None: + """读 JSON 文件；缺失或损坏返回 None（恢复指针不是关键路径，不能因此崩）。""" + try: + payload = json.loads(Path(path).read_text(encoding="utf-8")) + except (OSError, ValueError, UnicodeDecodeError): + return None + return payload if isinstance(payload, dict) else None + + +def _int_or_none(value: Any) -> int | None: + try: + return int(value) + except (TypeError, ValueError): + return None + + +def _float_or_none(value: Any) -> float | None: + try: + return float(value) + except (TypeError, ValueError): + return None + + +class TrainingJob: + """单个训练子进程的生命周期、日志和指标缓存。 + + ``control_dir`` 是控制文件的落地目录；默认与模型同目录，服务端则统一放到 + ``output/hrl/ui/control``（已被 gitignore）。 + """ + + def __init__(self, config: TrainConfig, control_dir: Path | None = None): self.config = config self.job_id = f"job-{int(time.time() * 1000)}" - self.command = config.command() + self.control_path = (Path(control_dir) if control_dir is not None + else config.model_path.parent) / \ + f"{self.job_id}.control.json" + # 控制文件是暂停/停止的唯一可靠通道：Windows 上没有 SIGSTOP， + # 且 os.kill(SIGTERM) 会退化成硬杀，信号无法承载"优雅收尾"语义。 + self.command = config.command() + ["--control-file", str(self.control_path)] self.created_at = time.time() self.started_at: float | None = None self.finished_at: float | None = None self.returncode: int | None = None - self.status = "created" # created/running/exited/failed/stopped + # created/running/paused/stopping/stopped/exited/failed + self.status = "created" self.error: str | None = None + self.forced_kill = False # 停止请求超时后是否动用了强杀 self.process: subprocess.Popen | None = None self.logs: deque[str] = deque(maxlen=MAX_LOG_LINES) self.metrics: deque[dict[str, Any]] = deque(maxlen=MAX_METRICS) self.monitor: list[dict[str, Any]] = [] self._lock = threading.RLock() self._reader: threading.Thread | None = None + self._stopper: threading.Thread | None = None self._table_sections: list[tuple[str, dict[str, Any]]] = [] self._table_steps: float | None = None # 当前表的 total_timesteps self._pending_table: dict[str, Any] | None = None # 未见刻度的半张表 @@ -254,10 +353,13 @@ class TrainingJob: with self._lock: if self.process is not None and self.process.poll() is None: raise RuntimeError("已有训练任务在运行") + # 先落一份"不暂停、不停止"的初始控制文件，训练脚本首步就能读到； + # 否则脚本要等到第一次轮询才区分"文件缺失"与"无指令"。 + write_control(self.control_path, paused=False, stop=False) env = os.environ.copy() env["PYTHONUNBUFFERED"] = "1" env["HRL_UI_JOB_ID"] = self.job_id - self.process = subprocess.Popen( + self.process = subprocess.Popen( # pylint: disable=consider-using-with self.command, cwd=str(REPO_ROOT), stdout=subprocess.PIPE, @@ -265,7 +367,7 @@ class TrainingJob: text=True, bufsize=1, env=env, - start_new_session=True, + **spawn_kwargs(), ) self.started_at = time.time() self.status = "running" @@ -273,18 +375,101 @@ class TrainingJob: target=self._read_loop, name=f"{self.job_id}-reader", daemon=True) self._reader.start() - def stop(self) -> bool: + def pause(self) -> bool: + """请求训练进程在下一个 step 边界挂起（跨平台：控制文件）。""" with self._lock: process = self.process if process is None or process.poll() is not None: return False - try: - os.killpg(os.getpgid(process.pid), signal.SIGTERM) - except (ProcessLookupError, PermissionError): - process.terminate() - self.status = "stopping" + if self.status not in ("running", "paused"): + return False + write_control(self.control_path, paused=True, stop=False) + self.status = "paused" return True + def resume(self) -> bool: + """解除挂起；训练从暂停的那个 step 继续。""" + with self._lock: + process = self.process + if process is None or process.poll() is not None: + return False + if self.status not in ("paused", "running"): + return False + write_control(self.control_path, paused=False, stop=False) + self.status = "running" + return True + + def stop(self) -> bool: + """请求优雅停止；超时后由看门狗强杀整棵进程树。 + + 控制文件先于信号：Windows 上 SIGTERM 等价于 TerminateProcess，训练 + 脚本"保存模型 + 写元数据"的收尾分支不会执行；控制文件则两个平台都能 + 命中同一段协作式收尾。强杀放到独立线程，避免 HTTP 请求被 grace 等待卡住。 + """ + with self._lock: + process = self.process + if process is None or process.poll() is not None: + return False + write_control(self.control_path, paused=False, stop=True) + self.status = "stopping" + if self._stopper is None or not self._stopper.is_alive(): + self._stopper = threading.Thread( + target=self._stop_watchdog, args=(process,), + name=f"{self.job_id}-stopper", daemon=True) + self._stopper.start() + return True + + def _stop_watchdog(self, process: subprocess.Popen) -> None: + """等 grace 让协作式停止生效，超时强杀并记下恢复点。""" + forced = terminate_process_tree(process, grace_s=STOP_GRACE_S) + with self._lock: + self.forced_kill = forced + if forced and not self.error: + self.error = (f"训练进程未在 {STOP_GRACE_S:.0f}s 内响应停止请求，" + f"已强制结束进程树") + if forced: + self._write_forced_stop(process.pid) + + def _write_forced_stop(self, pid: int) -> None: + """强杀后落一份 `<模型目录>/forced_stop.json`，写明"从哪继续"。 + + 强杀没有给训练脚本任何收尾机会，所以这里由前端补上审计与恢复指引： + 磁盘上仍有原子落盘的滚动恢复点 / 历史 checkpoint，``recovery.model`` + 就是可直接 ``--resume`` 的目标。 + """ + run_dir = self.config.model_path.parent + try: + recovery = discover_recovery(run_dir, self.config.model_path) + write_json_atomic(run_dir / "forced_stop.json", { + "event": "forced_stop", + "job_id": self.job_id, + "pid": int(pid), + "created_unix_s": time.time(), + "reason": "grace_timeout", + "grace_s": STOP_GRACE_S, + "recovery": recovery, + "note": ("训练进程未响应控制文件，已强杀整棵进程树；" + "把 recovery.model 作为 --resume 目标即可继续" + "（timesteps 为追加步数）"), + }) + except OSError as exc: # 标记文件写不了也不能影响停止流程 + with self._lock: + if not self.error: + self.error = f"强杀后写恢复标记失败：{exc}" + + def wait_for_exit(self, timeout: float | None = None) -> bool: + """等待停止流程收尾（优雅退出或强杀）；供前端进程退出前调用。 + + 训练子进程在独立进程组里，前端先退出并不会带走它。若不在这里等， + 前端一退出，子进程会成为孤儿——虽然控制文件仍会让它自行收尾，但 + 日志与退出码就没人接了。 + """ + thread = self._stopper + if thread is not None: + thread.join(timeout) + process = self.process + return process is None or process.poll() is not None + def _read_loop(self) -> None: process = self.process assert process is not None and process.stdout is not None @@ -315,10 +500,38 @@ class TrainingJob: def _on_line(self, line: str) -> None: with self._lock: self.logs.append(line) + self._sync_control_events(line) self._parse_progress_json(line) self._parse_sb3_table(line) self._refresh_monitor() + def _sync_control_events(self, line: str) -> None: + """用训练脚本的回显校正前端状态。 + + 正常情况下状态由前端自己设置；但外部直接改控制文件、或脚本比前端更早 + 感知到指令时，以脚本回显的 ``paused`` / ``resumed`` / ``stop_requested`` + 为准更真实。 + """ + stripped = line.strip() + if not stripped.startswith("{") or '"event"' not in stripped: + return + try: + payload = json.loads(stripped) + except ValueError: + return + if not isinstance(payload, dict): + return + event = payload.get("event") + if event not in ("paused", "resumed", "stop_requested"): + return + with self._lock: + if event == "paused" and self.status == "running": + self.status = "paused" + elif event == "resumed" and self.status == "paused": + self.status = "running" + elif event == "stop_requested" and self.status in ("running", "paused"): + self.status = "stopping" + # ------------------------------------------------------------ parsers def _parse_progress_json(self, line: str) -> None: stripped = line.strip() @@ -332,9 +545,9 @@ class TrainingJob: return point = {_normalize_metric(str(key)): value for key, value in payload.items() if key != "event"} - steps = _finite_float(point.get("timesteps")) + steps = finite_float(point.get("timesteps")) if steps is None: - steps = _finite_float(point.get("total_timesteps")) + steps = finite_float(point.get("total_timesteps")) if steps is not None: with self._lock: self._last_json_steps = steps @@ -371,7 +584,7 @@ class TrainingJob: self._table_sections.append((raw_key, {})) return key = _normalize_metric(raw_key) - value = _finite_float(raw_value) + value = finite_float(raw_value) if not key or value is None: return with self._lock: @@ -401,7 +614,7 @@ class TrainingJob: for _name, items in sections: table.update(items) if steps is None: - steps = _finite_float(table.get("total_timesteps")) + steps = finite_float(table.get("total_timesteps")) if steps is None: # 半张表：交给调用方决定是否已确认结束（终止时按末尾兜底提交） with self._lock: @@ -432,7 +645,7 @@ class TrainingJob: def _append_metric(self, point: dict[str, Any]) -> None: clean: dict[str, Any] = {} for key, value in point.items(): - number = _finite_float(value) + number = finite_float(value) clean[key] = number if number is not None else value # 补齐 timesteps：progress JSON 优先，SB3 表用 total_timesteps。 if "timesteps" not in clean and "total_timesteps" in clean: @@ -471,7 +684,7 @@ class TrainingJob: row = {} valid = True for key, raw in zip(header, values): - number = _finite_float(raw) + number = finite_float(raw) if number is None: valid = False break @@ -505,17 +718,17 @@ class TrainingJob: config = dict(self.config.__dict__) command = list(self.command) model_path = str(self.config.model_path) - timesteps = int(self.config.timesteps) payload = { "job_id": self.job_id, "status": self.status, - "pid": _pid(self.process), + "pid": process_pid(self.process), "created_at": self.created_at, "started_at": self.started_at, "finished_at": self.finished_at, "elapsed_s": round(elapsed, 3), "returncode": self.returncode, "error": self.error, + "forced_kill": self.forced_kill, "command": command, "model": model_path, "config": config, @@ -529,356 +742,12 @@ class TrainingJob: } payload["convergence"] = summarize_convergence(metrics, monitor) payload["process"] = describe_process(payload) + # 恢复指针在锁外做文件扫描：即使训练被强杀，这里也能给出可续训的产物 + payload["recovery"] = discover_recovery( + self.config.model_path.parent, self.config.model_path) return payload -def _series(metrics: list[dict[str, Any]], key: str) -> list[tuple[float, float]]: - points: list[tuple[float, float]] = [] - for point in metrics: - value = _finite_float(point.get(key)) - steps = _finite_float(point.get("timesteps")) or _finite_float( - point.get("total_timesteps")) - if value is None: - continue - points.append((steps if steps is not None else float(len(points) + 1), value)) - return points - - -def _last_value(points: list[tuple[float, float]]) -> float | None: - return points[-1][1] if points else None - - -def _spread(points: list[tuple[float, float]], window: int = 10 - ) -> tuple[float, float] | None: - """最近 window 个点的 (极差, 起点绝对值)，用于判断"是否还在动"。""" - sample = points[-window:] - if len(sample) < 3: - return None - ys = [item[1] for item in sample] - return max(ys) - min(ys), abs(ys[0]) - - -def _cv(points: list[tuple[float, float]], window: int = 10) -> float | None: - """最近 window 个点的变异系数（标准差/|均值|），越小越平稳。""" - sample = [item[1] for item in points[-window:]] - if len(sample) < 3: - return None - mean = sum(sample) / len(sample) - if abs(mean) < 1e-12: - return None - var = sum((value - mean) ** 2 for value in sample) / len(sample) - return (var ** 0.5) / abs(mean) - - -def _slope(points: list[tuple[float, float]], window: int = 8) -> float | None: - sample = points[-window:] - if len(sample) < 3: - return None - xs = [item[0] for item in sample] - ys = [item[1] for item in sample] - n = len(xs) - mean_x = sum(xs) / n - mean_y = sum(ys) / n - denom = sum((x - mean_x) ** 2 for x in xs) - if abs(denom) < 1e-12: - return None - return sum((x - mean_x) * (y - mean_y) for x, y in zip(xs, ys)) / denom - - -def _rate_per_1k(reward_points: list[tuple[float, float]], - window: int = 8) -> float | None: - """最近 window 个点的奖励增益换算成"每 1000 环境步"，便于读趋势。""" - slope = _slope(reward_points, window) - if slope is None: - return None - return slope * 1000.0 - - -def _fmt_num(value: float | None, digits: int = 4) -> str: - if value is None: - return "—" - if value != 0 and abs(value) < 1e-3: - return f"{value:.3e}" - return f"{value:.{digits}g}" - - -def _flatness(points: list[tuple[float, float]], window: int = 10 - ) -> tuple[float, float] | None: - """窗口内"趋势位移 / 逐点噪声"比值：<1 表示看不出系统性变化。 - - 判定收敛时只看斜率绝对值会把"很慢但持续的提升"误判为平台；这里把窗口 - 对半切，用两半均值之差当趋势位移，用相邻点平均绝对差当噪声尺度。返回 - (比值, 噪声尺度)，两者都小才是真的"又平又稳"。 - """ - sample = points[-window:] - if len(sample) < 4: - return None - ys = [item[1] for item in sample] - half = len(ys) // 2 - shift = abs(sum(ys[half:]) / len(ys[half:]) - sum(ys[:half]) / len(ys[:half])) - noise = sum(abs(b - a) for a, b in zip(ys, ys[1:])) / (len(ys) - 1) - if noise <= 0: - return (0.0, 0.0) if shift <= 0 else (float("inf"), 0.0) - return shift / noise, noise - - -def summarize_convergence(metrics: list[dict[str, Any]], - monitor: list[dict[str, Any]]) -> dict[str, Any]: - """给前端返回可解释的收敛状态与各参数最近值。 - - 这是**启发式判读**，不替代正式学习曲线分析：只依据最近窗口的斜率、 - 极差与变异系数给出"平坦/上升/未上升"的分类，并把同一组数字交给前端 - 展示，便于人工复核。 - """ - reward_points = _series(metrics, "ep_rew_mean") - if not reward_points and monitor: - reward_points = [(float(index), float(row.get("r", 0.0))) - for index, row in enumerate(monitor) - if _finite_float(row.get("r")) is not None] - kl_points = _series(metrics, "approx_kl") - ev_points = _series(metrics, "explained_variance") - vf_points = _series(metrics, "value_loss") - pg_points = _series(metrics, "policy_gradient_loss") - entropy_points = _series(metrics, "entropy_loss") - fps_points = _series(metrics, "fps") - - reward_slope = _slope(reward_points) - reward_rate = _rate_per_1k(reward_points) - spread = _spread(reward_points) - cv = _cv(reward_points) - latest_kl = _last_value(kl_points) - latest_ev = _last_value(ev_points) - latest_vf = _last_value(vf_points) - latest_pg = _last_value(pg_points) - latest_entropy = _last_value(entropy_points) - latest_fps = _last_value(fps_points) - latest_timesteps = metrics[-1].get("timesteps") if metrics else None - if latest_timesteps is None and metrics: - latest_timesteps = metrics[-1].get("total_timesteps") - latest_reward = _last_value(reward_points) - latest_len = _last_value(_series(metrics, "ep_len_mean")) - latest_updates = _last_value(_series(metrics, "n_updates")) - window = min(10, max(3, len(reward_points))) - reward_window = [round(value, 6) for _, value in reward_points[-window:]] - best_reward = max((value for _, value in reward_points), default=None) - - notes: list[str] = [] - if not metrics and not monitor: - status, tone = "等待首个 rollout", "gray" - notes.append("训练进程启动后先输出环境初始化与首个 rollout 的进度行。") - elif not metrics: - status, tone = "已收到回合奖励，等待进度行", "warn" - notes.append("Monitor CSV 已有回合记录，但尚未收到 --progress-json 进度行；" - "首个 rollout 结束后会出现完整指标。") - elif len(metrics) < 3: - status, tone = "预热阶段", "warn" - notes.append("已收到初始指标，需要更多 rollout 才能判断趋势。") - else: - tone, status = "ok", "收敛趋势良好" - # 平台判据（三条同时成立）：对半窗口看不出位移、窗口内波动小、 - # 逐点噪声本身也小。慢速但持续的提升会因第一条被排除。 - flat = _flatness(reward_points) - plateau = False - if flat is not None and cv is not None and latest_reward is not None: - flat_ratio, flat_noise = flat - scale = max(abs(latest_reward), 1e-9) - plateau = (flat_ratio <= 1.0 and cv <= 0.15 - and flat_noise <= 0.02 * scale - and (spread is None or spread[1] < 1e-12 - or spread[0] <= 0.15 * spread[1])) - if latest_kl is not None and latest_kl > 0.05: - status, tone = "KL 偏大，策略更新可能过猛", "bad" - notes.append(f"approx_kl={latest_kl:.4f} > 0.05") - elif latest_ev is not None and latest_ev < 0.0: - status, tone = "价值函数尚未拟合", "warn" - notes.append(f"explained_variance={latest_ev:.3f} < 0") - elif reward_slope is not None and reward_slope < 0: - status, tone = "奖励未形成上升趋势", "warn" - notes.append(f"最近 {window} 点奖励斜率为负（{_fmt_num(reward_slope)}/step）") - elif plateau: - status, tone = "已趋于平稳（疑似收敛）", "ok" - notes.append("窗口内对半位移小于逐点噪声，且波动很小" - f"（变异系数 {_fmt_num(cv, 3)}）") - else: - notes.append(f"最近 {window} 点奖励斜率为正（{_fmt_num(reward_slope)}/step）") - if latest_kl is not None and latest_kl <= 0.02: - notes.append(f"approx_kl={latest_kl:.4f} 较稳定") - if latest_ev is not None: - notes.append(f"explained_variance={latest_ev:.3f}") - if cv is not None: - notes.append(f"奖励变异系数={_fmt_num(cv, 3)}") - if latest_entropy is not None and abs(latest_entropy) < 1e-3: - notes.append("策略熵接近 0，探索可能过早停止") - return { - "status": status, - "tone": tone, - "notes": notes, - "window": window, - "latest_timesteps": latest_timesteps, - "reward_slope": reward_slope, - "reward_rate_per_1k": reward_rate, - "reward_cv": cv, - "reward_spread": spread[0] if spread else None, - "reward_window": reward_window, - "best_reward": best_reward, - "latest_reward": latest_reward, - "latest_ep_len": latest_len, - "latest_updates": latest_updates, - "latest_kl": latest_kl, - "latest_explained_variance": latest_ev, - "latest_value_loss": latest_vf, - "latest_policy_loss": latest_pg, - "latest_entropy_loss": latest_entropy, - "latest_fps": latest_fps, - "reward_points": len(reward_points), - } - - -# 快照状态 -> 界面展示用的进程语义（阶段名称、色调、是否终态） -_STATUS_VIEW = { - "created": ("已创建", "gray", False, "命令已构造，等待 spawn"), - "running": ("运行中", "ok", False, "子进程持有控制台输出与独立进程组"), - "stopping": ("停止中", "warn", False, "已发送 SIGTERM，等待子进程退出"), - "stopped": ("已手动停止", "warn", True, "收到停止信号后退出；脚本会在 step 边界保存模型与元数据"), - "exited": ("正常结束", "ok", True, "退出码 0：训练与评估流程走完"), - "failed": ("异常结束", "bad", True, "非零退出码或启动失败，见日志尾部"), -} - -_PHASES = [ - ("created", "构造命令"), - ("spawn", "启动子进程"), - ("rollout", "采集 rollout"), - ("update", "PPO 更新"), - ("evaluate", "评估 / 保存"), - ("done", "收敛归档"), -] - -# 状态 -> 当前所处的阶段节点；节点之前为已完成，之后为未开始。 -_STATUS_PHASE = { - "created": "created", - "running": "update", - "stopping": "update", - "stopped": "done", - "exited": "done", - "failed": "done", -} - - -def _pid(process: subprocess.Popen | None) -> int | None: - """已退出进程的 pid 访问会抛 OSError；这里统一吞掉，只用于展示。""" - if process is None: - return None - try: - return int(process.pid) - except (OSError, ValueError): - return None - - -def describe_process(snapshot: dict[str, Any]) -> dict[str, Any]: - """把 snapshot 派生为界面直接可渲染的进程对象。 - - 包含：状态语义词、进度（步/百分比）、墙钟与剩余时间估计、最近吞吐、 - 进程时间线节点。数字都取自 snapshot，前端不再重复累加，避免两个刷新 - 频率不同的计数器互相打架。 - """ - status = str(snapshot.get("status") or "created") - label, tone, terminal, hint = _STATUS_VIEW.get( - status, (status, "gray", False, "")) - config = snapshot.get("config") or {} - order = [key for key, _ in _PHASES] - current = _STATUS_PHASE.get(status, "created") - timeline = [] - for key, title in _PHASES: - index = order.index(key) - if index < order.index(current): - state = "done" - elif index == order.index(current): - state = "active" - else: - state = "pending" - timeline.append({"key": key, "label": title, "state": state}) - - metrics = snapshot.get("metrics") or [] - latest: dict[str, Any] = metrics[-1] if metrics else {} - convergence = snapshot.get("convergence") or {} - total_steps = _finite_float(config.get("timesteps")) or 0.0 - # SB3 表格行可能没有 steps 字段，因此进度取"所有进度点的最大步数"， - # 而不是只看最后一个点——否则末行恰好缺字段时进度会回退到 0。 - done_steps = 0.0 - for point in metrics: - value = (_finite_float(point.get("timesteps")) - or _finite_float(point.get("total_timesteps"))) - if value is not None and value > done_steps: - done_steps = value - if done_steps == 0.0: - done_steps = _finite_float(convergence.get("latest_timesteps")) or 0.0 - # init/resume（课程迁移）时 num_timesteps 从上一阶段继续，进度必须按本次 - # 增量算；start_timesteps 由训练脚本的进度 JSON 提供，缺省按 0 处理。 - start_steps = 0.0 - for point in metrics: - value = _finite_float(point.get("start_timesteps")) - if value is not None: - start_steps = value - break - trained_steps = max(0.0, done_steps - start_steps) - percent = (100.0 * trained_steps / total_steps) if total_steps > 0 else None - if percent is not None: - percent = max(0.0, min(100.0, percent)) - elapsed = float(snapshot.get("elapsed_s") or 0.0) - rate = _latest_rate(metrics) - remaining: float | None = None - eta_ts: float | None = None - if rate and rate > 0 and total_steps > 0 and trained_steps < total_steps: - remaining = (total_steps - trained_steps) / rate - eta_ts = float(snapshot.get("finished_at") or time.time()) + remaining - if not terminal and remaining is None: - cap = _finite_float(config.get("max_wall_s")) or 0.0 - if cap > 0: - remaining = max(0.0, cap - elapsed) - return { - "label": label, - "tone": tone, - "terminal": terminal, - "hint": hint, - "phase": current, - "timeline": timeline, - "percent": percent, - "done_steps": done_steps, - "start_steps": start_steps, - "trained_steps": trained_steps, - "total_steps": total_steps, - "elapsed_s": round(elapsed, 3), - "rate_per_s": rate, - "remaining_s": remaining, - "eta_epoch_s": eta_ts, - "log_total": snapshot.get("log_total"), - "log_capacity": MAX_LOG_LINES, - "metric_total": snapshot.get("metric_total"), - "latest": latest, - } - - -def _latest_rate(metrics: list[dict[str, Any]], window: int = 6) -> float | None: - """最近若干个**带 fps 的**进度点的平均吞吐（步/秒），用于剩余时间估计。 - - progress JSON 行不含 fps，只有 SB3 表格行有；若只看最后 window 个点， - 恰好落在若干 JSON 行之间时会得到 None，剩余时间就会退化成墙钟上限的 - 粗糙估计。这里向前扫描直到拿到至少 1 个 fps（最多看 3*window 个点）。 - """ - if not metrics: - return None - usable: list[float] = [] - for point in reversed(metrics[-window * 3:]): - value = _finite_float(point.get("fps")) - if value and value > 0: - usable.append(value) - if len(usable) >= window: - break - if not usable: - return None - return sum(usable) / len(usable) - - class TrainingUI: """训练前端管理器：持有唯一 TrainingJob 与 HTTP 服务器。""" @@ -901,9 +770,9 @@ class TrainingUI: config = TrainConfig.from_payload(payload) with self._lock: if self.job is not None and self.job.status in ( - "created", "running", "stopping"): + "created", "running", "paused", "stopping"): raise RuntimeError("已有训练任务在运行，请先停止") - self.job = TrainingJob(config) + self.job = TrainingJob(config, control_dir=CONTROL_DIR) try: self.job.start() except Exception: @@ -913,6 +782,14 @@ class TrainingUI: # 与 snapshot() 同形：HTTP 层直接把返回值当响应体，不再二次拆包 return {"job": self.job.snapshot(log_tail=1)} + def pause(self) -> bool: + with self._lock: + return bool(self.job and self.job.pause()) + + def resume(self) -> bool: + with self._lock: + return bool(self.job and self.job.resume()) + def stop(self) -> bool: with self._lock: return bool(self.job and self.job.stop()) @@ -938,7 +815,10 @@ class TrainingUI: self._serving = False def shutdown(self) -> None: - self.stop() + job = self.job + if job is not None and job.stop(): + # 留出收尾落盘的时间，避免前端先退出把训练子进程留成孤儿 + job.wait_for_exit(timeout=STOP_GRACE_S + 5.0) if self._serving: self.httpd.shutdown() self.httpd.server_close() @@ -948,8 +828,8 @@ class _UIHandler(BaseHTTPRequestHandler): protocol_version = "HTTP/1.1" server_version = "MCM26B-Training-UI/1.0" - def log_message(self, format, *args): # noqa: A002 - pass + def log_message(self, format, *args): # noqa: A002 # pylint: disable=redefined-builtin + """覆写 stdlib 签名以吞掉访问日志；参数名沿用 ``format`` 不能改。""" @property def ui(self) -> TrainingUI: @@ -999,6 +879,10 @@ class _UIHandler(BaseHTTPRequestHandler): try: if parts.path == "/api/train/start": self._send_json(200, self.ui.start(payload)) + elif parts.path == "/api/train/pause": + self._send_json(200, {"paused": self.ui.pause()}) + elif parts.path == "/api/train/resume": + self._send_json(200, {"resumed": self.ui.resume()}) elif parts.path == "/api/train/stop": self._send_json(200, {"stopped": self.ui.stop()}) else: diff --git a/src/mcm26b/training_ui/static/app.js b/src/mcm26b/training_ui/static/app.js index ccdd744..9b27cf9 100644 --- a/src/mcm26b/training_ui/static/app.js +++ b/src/mcm26b/training_ui/static/app.js @@ -23,6 +23,8 @@ const state = { chartKey: "ep_rew_mean", jobKey: null, // job_id 变化时清空本地缓存 lastLogSignature: "", + lastStatus: null, // 最近一次快照的状态：决定暂停按钮是"暂停"还是"继续" + lastJob: null, // 最近一次快照：恢复按钮需要读 job.recovery }; // ---------------------------------------------------------------- 工具 @@ -285,9 +287,11 @@ function absorbMetrics(job) { // ---------------------------------------------------------------- 渲染 function render(job) { if (!job) { + state.lastJob = null; renderIdle(); return; } + state.lastJob = job; if (job.job_id !== state.jobKey) { state.jobKey = job.job_id; state.metrics = []; @@ -302,6 +306,7 @@ function render(job) { renderLog(job); renderEpisodes(job); renderBanner(job); + renderRecovery(job); renderJobLine(job); } @@ -319,6 +324,10 @@ function renderIdle() { setText("stat-logs", "0"); $("timeline").innerHTML = ""; setText("command-preview", ""); + state.lastStatus = null; + const pauseBtn = $("btn-pause"); + pauseBtn.disabled = true; + pauseBtn.textContent = "暂停训练"; chip("chip-converge", "判读", "等待数据", "gray"); $("converge-notes").innerHTML = ""; $("metric-grid").innerHTML = '

尚未启动训练，暂无参数。

'; @@ -370,8 +379,14 @@ function renderProcess(job) { renderTimeline(process); setText("command-preview", job.command ? `$ ${job.command.join(" ")}` : ""); - setFormLocked(job.status === "running" || job.status === "created" || job.status === "stopping"); - $("btn-stop").disabled = !(job.status === "running" || job.status === "created"); + state.lastStatus = job.status; + setFormLocked(job.status === "running" || job.status === "created" + || job.status === "paused" || job.status === "stopping"); + $("btn-stop").disabled = !(job.status === "running" || job.status === "created" + || job.status === "paused"); + const pauseBtn = $("btn-pause"); + pauseBtn.disabled = !(job.status === "running" || job.status === "paused"); + pauseBtn.textContent = job.status === "paused" ? "继续训练" : "暂停训练"; } function renderTimeline(process) { @@ -427,8 +442,12 @@ function renderBanner(job) { } if (job.status === "failed") { showBanner("bad", `训练异常结束（退出码 ${job.returncode === null ? "?" : job.returncode}）：${escapeHtml(job.error || "见日志尾部")}。`); + } else if (job.status === "paused") { + showBanner("warn", "训练已暂停：进程与显存仍在，但停在 step 边界不再推进。点「继续训练」从暂停处恢复，或点「停止训练」保存当前模型后退出。"); } else if (job.status === "stopped") { - showBanner("warn", "已按请求停止训练（SIGTERM 独立进程组），脚本会在 step 边界保存当前模型与元数据。"); + showBanner("warn", job.forced_kill + ? "停止请求超时，已强制结束进程树；最后一段模型可能未保存，请检查 checkpoint 目录。" + : "已按请求停止训练（控制文件协作式停止），脚本已在 step 边界保存当前模型与元数据。"); } else if (job.status === "exited") { showBanner("ok", "训练与评估流程正常结束，模型与元数据已写入输出目录。"); } else if (process.remaining_s !== null && process.remaining_s !== undefined @@ -439,6 +458,26 @@ function renderBanner(job) { } } +// ---------------------------------------------------------------- 恢复 +function renderRecovery(job) { + const recovery = job.recovery || {}; + const box = $("recovery-box"); + if (!box) return; + if (!recovery.model || recovery.reason === "none") { + box.classList.add("hidden"); + return; + } + box.classList.remove("hidden"); + const steps = recovery.timesteps === null || recovery.timesteps === undefined + ? "步数未知" : `${fmtInt(recovery.timesteps)} 步`; + const when = recovery.updated_unix_s ? fmtClock(recovery.updated_unix_s) : "—"; + setText("recovery-text", + `可恢复产物：${steps} · 来源 ${recovery.reason || "?"} · 更新 ${when} · ${recovery.model}`); + const button = $("btn-recover"); + // 离线预览里的恢复信息是合成的，不允许真的去续训 + button.disabled = !recovery.can_resume || Boolean(state.preview); +} + // ---------------------------------------------------------------- 收敛 function renderConvergence(job) { const convergence = job.convergence || {}; @@ -679,6 +718,33 @@ function renderEpisodes(job) { } // ---------------------------------------------------------------- 交互 +async function submitPayload(payload) { + const err = $("form-error"); + err.textContent = ""; + if (state.saving) return false; // 训练中表单锁定，续训按钮同样不许重复提交 + if (payload.resume && !payload.model) { + err.textContent = "续训需要填写模型输出路径"; + return false; + } + const res = await fetch("/api/train/start", { + method: "POST", headers: { "Content-Type": "application/json" }, + body: JSON.stringify(payload), + }); + const body = await res.json().catch(() => ({})); + if (!res.ok) { + err.textContent = body.error || `启动失败（HTTP ${res.status}）`; + return false; + } + state.preview = false; + state.jobKey = null; + state.metrics = []; + state.metricCache = []; + state.metricNext = 0; + state.metricTotal = 0; + await poll(); + return true; +} + $("config-form").addEventListener("submit", async (event) => { event.preventDefault(); if (state.saving) return; @@ -691,24 +757,43 @@ $("config-form").addEventListener("submit", async (event) => { for (const [key, value] of Object.entries(values)) { if (value !== null) payload[key] = value; } - if (payload.resume && !payload.model) { err.textContent = "续训需要填写模型输出路径"; return; } - const res = await fetch("/api/train/start", { - method: "POST", headers: { "Content-Type": "application/json" }, - body: JSON.stringify(payload), + await submitPayload(payload); +}); + +$("btn-recover").addEventListener("click", async () => { + const recovery = (state.lastJob && state.lastJob.recovery) || {}; + if (!recovery.can_resume || !recovery.model) return; + if (!confirm(`用最近的可恢复产物续训？\n${recovery.model}\n` + + "当前表单里的 timesteps 会作为**追加**步数。")) return; + // 同步回表单，用户随后可微调步数再手动点「开始训练」 + const modelField = $("cfg-model"); + if (modelField) modelField.value = recovery.model; + const resumeField = $("cfg-resume"); + if (resumeField) resumeField.checked = true; + const values = collectValues(); + values.model = recovery.model; + values.resume = true; + values.force = false; // --resume 与 --force 互斥 + await submitPayload(values); +}); + +$("btn-pause").addEventListener("click", async () => { + const resuming = state.lastStatus === "paused"; + const res = await fetch(resuming ? "/api/train/resume" : "/api/train/pause", { + method: "POST", headers: { "Content-Type": "application/json" }, body: "{}", }); const body = await res.json().catch(() => ({})); - if (!res.ok) { err.textContent = body.error || `启动失败（HTTP ${res.status}）`; return; } - state.preview = false; - state.jobKey = null; - state.metrics = []; - state.metricCache = []; - state.metricNext = 0; - state.metricTotal = 0; + if (!res.ok) { + showBanner("bad", `${resuming ? "继续" : "暂停"}失败：${escapeHtml(body.error || res.status)}`); + } else if (!resuming && body.paused === false) { + showBanner("warn", "当前没有可暂停的训练进程（可能已结束）。"); + } await poll(); }); $("btn-stop").addEventListener("click", async () => { - if (!confirm("确认停止当前训练？\n将向独立进程组发送 SIGTERM，脚本会先保存当前模型与元数据再退出。")) return; + if (!confirm("确认停止当前训练？\n将通过控制文件请求优雅停止，脚本先保存当前模型与元数据再退出；" + + "若超时未响应则强制结束进程树。")) return; const res = await fetch("/api/train/stop", { method: "POST", headers: { "Content-Type": "application/json" }, body: "{}", }); diff --git a/src/mcm26b/training_ui/static/index.html b/src/mcm26b/training_ui/static/index.html index ff6bd4a..c1a93ac 100644 --- a/src/mcm26b/training_ui/static/index.html +++ b/src/mcm26b/training_ui/static/index.html @@ -30,6 +30,7 @@ + @@ -37,6 +38,11 @@
+ +
diff --git a/src/mcm26b/training_ui/static/style.css b/src/mcm26b/training_ui/static/style.css index 0859caf..185d01d 100644 --- a/src/mcm26b/training_ui/static/style.css +++ b/src/mcm26b/training_ui/static/style.css @@ -64,6 +64,15 @@ button.danger:hover:enabled { background: rgba(248,113,113,.12); } .banner.warn { border-color: var(--warn); border-left-color: var(--warn); background: rgba(245,158,11,.08); } .banner b { font-family: var(--mono); } +/* 恢复条：显示最近一份完整落盘的产物，并提供一键续训 */ +.recovery { + display: flex; align-items: center; justify-content: space-between; gap: 12px; + border: 1px solid var(--cyan, #22d3ee); border-left: 4px solid var(--cyan, #22d3ee); + background: rgba(34,211,238,.07); border-radius: 8px; + padding: 8px 12px; margin: 10px 0 0; font-size: 13px; +} +.recovery .hint { overflow-wrap: anywhere; } + /* ------------------------------------------------------------ 面板 */ .panel { border: 1px solid var(--line); border-radius: 12px; background: var(--panel); diff --git a/tests/test_bayes_loop_q3.py b/tests/test_bayes_loop_q3.py index 0edc46c..0c29e21 100644 --- a/tests/test_bayes_loop_q3.py +++ b/tests/test_bayes_loop_q3.py @@ -62,7 +62,7 @@ class BayesLoopAcceptanceTests(unittest.TestCase): def test_sixteen_sources_stop_by_count_bound(self): sources = random_omni_scene(seed=6, count=16, config=CONFIG) - summary, report, simulator = _run_scene(sources) + summary, report, _ = _run_scene(sources) truth = sorted(s.channel for s in sources) self.assertTrue(summary["completed"]) self.assertEqual(summary["stop_reason"], "sixteen_cleared") @@ -72,7 +72,7 @@ class BayesLoopAcceptanceTests(unittest.TestCase): def test_empty_arena_all_absent(self): """0 源：每个频道都必须由 PF 排除盘证明缺席后正常退出。""" - summary, report, simulator = _run_scene([], config=NO_PERSIST) + summary, report, _ = _run_scene([], config=NO_PERSIST) self.assertTrue(summary["completed"]) self.assertEqual(summary["stop_reason"], "all_resolved") self.assertEqual(sorted(summary["absent_channels"]), @@ -89,7 +89,7 @@ class BayesLoopAcceptanceTests(unittest.TestCase): sources = [Source(c + 1, 1800.0 * math.cos(math.pi / 6 + 2 * math.pi * c / 4), 1800.0 * math.sin(math.pi / 6 + 2 * math.pi * c / 4), 1000.0) for c in range(4)] - summary, report, simulator = _run_scene(sources, config=NO_PERSIST) + summary, report, _ = _run_scene(sources, config=NO_PERSIST) truth = sorted(s.channel for s in sources) self.assertTrue(summary["completed"]) self.assertEqual(sorted(summary["cleared_channels"]), truth) @@ -112,7 +112,7 @@ class BayesLoopAcceptanceTests(unittest.TestCase): def test_transport_fault_retries_keep_session_clean(self): sources = random_omni_scene(seed=5, count=10, config=CONFIG) script = [("drop_before", 1)] # 首个请求丢弃一次：幂等重试路径 - summary, report, simulator = _run_scene(sources, fault_script=script) + summary, report, _ = _run_scene(sources, fault_script=script) truth = sorted(s.channel for s in sources) self.assertTrue(summary["completed"]) self.assertEqual(sorted(summary["cleared_channels"]), truth) @@ -141,9 +141,9 @@ class BayesLoopAcceptanceTests(unittest.TestCase): def test_cli_strategy_registry(self): """CLI --strategy 注册表：bayes/seven 用七站，grid 用 49 点网格。""" - import mcm26b.cli as cli + from mcm26b import cli from mcm26b.coverage import coverage_points - from mcm26b.planner.seven import SevenBatch, seven_points + from mcm26b.planner.seven import seven_points self.assertIs(cli.Q3_STRATEGIES["bayes"], BayesLoop) for name, loop_cls in cli.Q3_STRATEGIES.items(): simulator = LocalSimulator([], robot_id="T001", config=CONFIG) diff --git a/tests/test_belief.py b/tests/test_belief.py index bc55d16..3efdebb 100644 --- a/tests/test_belief.py +++ b/tests/test_belief.py @@ -132,7 +132,7 @@ class BeliefDirectionTests(TestCase): def test_near_shrinks_to_five_meters(self): belief = BeliefState(CONFIG) belief.update_measure(2, (100.0, 100.0), NEAR, None) - center, radius = belief.candidate_set(2).mec() + _, radius = belief.candidate_set(2).mec() self.assertLessEqual(radius, 5.1) self.assertTrue(belief.candidate_set(2).contains((101.0, 101.0))) self.assertEqual(belief.channels[2].status, ChannelStatus.DETECTED) diff --git a/tests/test_geometry.py b/tests/test_geometry.py index d869fcc..4f39bfd 100644 --- a/tests/test_geometry.py +++ b/tests/test_geometry.py @@ -16,7 +16,8 @@ class GeometryTests(unittest.TestCase): self.assertEqual(result['status'], 'polygon') self.assertEqual(len(result['vertices']), 3) self.assertAlmostEqual(result['diameter_m'], 40, places=7) - self.assertAlmostEqual(result['minimum_enclosing_circle']['radius_m'], 40/math.sqrt(3), places=7) + self.assertAlmostEqual(result['minimum_enclosing_circle']['radius_m'], + 40/math.sqrt(3), places=7) self.assertFalse(result['diameter_circle']['covers']) self.assertFalse(result['guaranteed_clearable_at_mec_center']) @@ -84,7 +85,7 @@ class GeometryTests(unittest.TestCase): c, r = smallest_enclosing_circle([(0,0),(1,0),(3,0)]) self.assertAlmostEqual(r, 1.5) - def test_random_bounded_truth_containment_and_monotonicity(self): + def test_random_truth_containment_and_monotonic(self): rng = random.Random(20260910) for _ in range(100): target = (rng.uniform(-300,300), rng.uniform(-300,300)) diff --git a/tests/test_geometry_certificates.py b/tests/test_geometry_certificates.py index 346024c..17f7f94 100644 --- a/tests/test_geometry_certificates.py +++ b/tests/test_geometry_certificates.py @@ -14,7 +14,7 @@ ROOT = Path(__file__).resolve().parents[1] class CertificateTests(unittest.TestCase): - def test_incremental_circle_against_exhaustive_reference(self): + def test_incremental_circle_matches_reference(self): rng = random.Random(271828) for _ in range(250): points = [(rng.uniform(-20, 20), rng.uniform(-20, 20)) @@ -25,7 +25,7 @@ class CertificateTests(unittest.TestCase): self.assertLess(distance(center, expected_c), 1e-6) self.assertTrue(all(distance(center, p) <= radius for p in points)) - def test_circle_large_cocircular_duplicate_and_collinear_sets(self): + def test_circle_cocircular_duplicate_and_collinear(self): # More than Python's recursion limit, with analytically known answers. points = [(1e6+30*math.cos(i*math.tau/1500), -2e6+30*math.sin(i*math.tau/1500)) for i in range(1500)] @@ -37,15 +37,16 @@ class CertificateTests(unittest.TestCase): self.assertEqual(c, (749.5, 1499)) self.assertEqual(smallest_enclosing_circle([(3, 4)]*10), ((3, 4), 0)) - def test_witness_and_clearance_margin_from_saved_observations(self): + def test_witness_and_clearance_margin_from_saved(self): path = next((ROOT/'output/q1/practice').glob('*_observations.json')) rows = json.loads(path.read_text(encoding='utf-8'))['measurements'] result = solve([Measurement(**m) for m in rows]) circle = result['diameter_circle'] self.assertFalse(circle['covers']) self.assertIn(circle['witness_vertex'], result['vertices']) - self.assertAlmostEqual(distance(circle['witness_vertex'], circle['center'])-circle['radius_m'], - circle['max_excess_m']) + self.assertAlmostEqual( + distance(circle['witness_vertex'], circle['center'])-circle['radius_m'], + circle['max_excess_m']) for i in circle['outside_vertex_indices']: self.assertGreater(distance(result['vertices'][i], circle['center']), circle['radius_m']+circle['coverage_tolerance_m']) @@ -61,7 +62,7 @@ class CertificateTests(unittest.TestCase): self.assertTrue(r['diameter_circle']['covers']) self.assertIsNone(r['diameter_circle']['witness_vertex']) - def test_first_sector_disk_covers_all_angles_and_distances(self): + def test_first_sector_disk_covers_all_angles(self): for angle in [0, 359.9, -180]: for delta in [0, 1, 1.005, 45]: m = Measurement(-800, 1200, angle, delta) @@ -72,11 +73,15 @@ class CertificateTests(unittest.TestCase): for k in range(11): phi = math.radians(angle-delta+2*delta*j/20) p = (m.x+150*k*math.cos(phi), m.y+150*k*math.sin(phi)) - self.assertLessEqual(distance(p, enclosing['center']), enclosing['radius_m']+1e-9) + self.assertLessEqual( + distance(p, enclosing['center']), + enclosing['radius_m']+1e-9) if candidate: for t in range(8): - q = (candidate['center'][0]+candidate['radius_m']*math.cos(t*math.tau/8), - candidate['center'][1]+candidate['radius_m']*math.sin(t*math.tau/8)) + q = (candidate['center'][0] + + candidate['radius_m']*math.cos(t*math.tau/8), + candidate['center'][1] + + candidate['radius_m']*math.sin(t*math.tau/8)) self.assertLessEqual(distance(p, q), 1000+1e-9) if delta <= 1.005: self.assertTrue(cert['candidate_nonempty_certified']) @@ -100,7 +105,8 @@ class CertificateTests(unittest.TestCase): self.assertEqual(outputs[0]['error_policy']['effective_half_angles_deg'], [1, 1, 1]) self.assertEqual(outputs[1]['error_policy']['effective_half_angles_deg'], [1.005]*3) self.assertGreater(outputs[1]['diameter_m'], outputs[0]['diameter_m']) - bad = subprocess.run(base+['--rounding-margin-deg', '-1'], cwd=ROOT, capture_output=True) + bad = subprocess.run(base+['--rounding-margin-deg', '-1'], cwd=ROOT, + capture_output=True, check=False) self.assertNotEqual(bad.returncode, 0) diff --git a/tests/test_hrl.py b/tests/test_hrl.py index 9fdfc3e..99ea2ae 100644 --- a/tests/test_hrl.py +++ b/tests/test_hrl.py @@ -90,7 +90,7 @@ class LayoutTests(unittest.TestCase): again = env._loop.hrl_observation(env._current_candidates) self.assertTrue(numpy.array_equal(obs, again)) - def test_mask_admits_at_least_one_action_and_aligns_with_candidates(self): + def test_mask_nonempty_and_matches_candidate_slots(self): env = _env(problem=3) env.reset(seed=7) for _ in range(6): @@ -98,7 +98,7 @@ class LayoutTests(unittest.TestCase): self.assertTrue(mask.any()) slots = {cand.slot for cand in env._current_candidates} self.assertEqual({int(i) for i in numpy.flatnonzero(mask)}, slots) - obs, _r, term, trunc, _info = env.step( + _obs, _r, term, trunc, _info = env.step( int(env._loop.heuristic_action(env._current_candidates))) if term or trunc: break @@ -114,7 +114,7 @@ class LayoutTests(unittest.TestCase): self.assertEqual(FALLBACK_OFFSET, STATION_OFFSET + STATION_SLOTS) self.assertEqual(N_CANDIDATES - (FALLBACK_OFFSET + FALLBACK_SLOTS), 11) - def test_build_candidates_keeps_clear_17_to_20_and_locate_1_to_4(self): + def test_build_candidates_keeps_clear_and_locate(self): host = _CandidateHost(detected=[1, 2, 3, 4, 17, 18, 19, 20], certified=[17, 18, 19, 20]) candidates = host.build_candidates() @@ -130,7 +130,7 @@ class LayoutTests(unittest.TestCase): selected = host.candidate_by_slot(candidates, CLEAR_OFFSET + channel - 1) self.assertEqual((selected.kind, selected.channel), ("clear", channel)) - def test_build_candidates_fallback_includes_channel_20_within_its_slots(self): + def test_build_candidates_fallback_covers_slots(self): channels = list(range(17, 21)) host = _CandidateHost(detected=channels, blocked=channels) candidates = host.build_candidates() @@ -151,11 +151,11 @@ class LayoutTests(unittest.TestCase): class HeuristicLoopTests(unittest.TestCase): def test_q3_full_episode_clears_all_sources(self): env = _env(problem=3, stage_index=3, max_episode_steps=500) - obs, _ = env.reset(seed=11) + _, _ = env.reset(seed=11) steps = 0 while True: action = env._loop.heuristic_action(env._current_candidates) - obs, r, term, trunc, info = env.step(action) + _, _, term, trunc, info = env.step(action) steps += 1 if term or trunc: break @@ -170,7 +170,7 @@ class HeuristicLoopTests(unittest.TestCase): env.reset(seed=21) while True: action = env._loop.heuristic_action(env._current_candidates) - obs, r, term, trunc, info = env.step(action) + _, _, term, trunc, info = env.step(action) if term or trunc: break self.assertTrue(term) @@ -185,8 +185,8 @@ class HeuristicLoopTests(unittest.TestCase): reward_cfg = env.reward for _ in range(8): action = env._loop.heuristic_action(env._current_candidates) - virtual_before = env._client.state.virtual_time_s - obs, r, term, trunc, info = env.step(action) + _virtual_before = env._client.state.virtual_time_s + _, r, term, trunc, info = env.step(action) outcome = env._last_outcome if outcome.invalid: self.assertAlmostEqual(r, -reward_cfg.invalid, places=6) @@ -206,13 +206,13 @@ class HeuristicLoopTests(unittest.TestCase): if term or trunc: break - def test_invalid_action_penalized_without_simulator_interaction(self): + def test_invalid_action_penalized_without_sim_call(self): env = _env(problem=3, stage_index=0, max_episode_steps=50) env.reset(seed=3) mask = env.action_masks() invalid_action = int(numpy.argmin(mask)) virtual_before = env._client.state.virtual_time_s - obs, r, term, trunc, info = env.step(invalid_action) + _, r, _, _, _ = env.step(invalid_action) self.assertAlmostEqual(r, -env.reward.invalid, places=6) self.assertAlmostEqual(env._client.state.virtual_time_s, virtual_before, places=6) @@ -228,7 +228,7 @@ class LowLevelTests(unittest.TestCase): goal = tuple(info["goal"]) total = 0 for _ in range(60): - obs, r, term, trunc, info = env.step(env.expert_action()) + obs, _, term, trunc, _ = env.step(env.expert_action()) total += 1 if term or trunc: break @@ -237,7 +237,7 @@ class LowLevelTests(unittest.TestCase): efficiency = straight / (total * env.step_m) self.assertGreater(efficiency, 0.7, "专家路径效率应接近直线") - def test_executor_straightline_is_goal_only_and_policy_path_snaps(self): + def test_executor_straightline_is_goal_only(self): executor = LowLevelExecutor(policy=None) self.assertEqual(executor.roll((0.0, 0.0), (500.0, 200.0)), [(500.0, 200.0)]) @@ -257,7 +257,7 @@ class CurriculumTests(unittest.TestCase): self.assertLessEqual(prev.bearing_error_half_deg, nxt.bearing_error_half_deg) - def test_training_config_defaults_to_production_fidelity(self): + def test_training_config_defaults_to_production(self): config = training_config(CONFIG, STAGES[3]) self.assertEqual(config.field.channel_max, 20) self.assertEqual(config.field.bearing_error_half_deg, 1.0) diff --git a/tests/test_hrl_checkpoint.py b/tests/test_hrl_checkpoint.py index 0d7ddb9..bb558b1 100644 --- a/tests/test_hrl_checkpoint.py +++ b/tests/test_hrl_checkpoint.py @@ -1,10 +1,13 @@ -"""高层训练落盘保证：checkpoint、SIGTERM 优雅保存、worker 信号屏蔽。 +"""高层训练落盘保证：checkpoint、控制文件优雅停止、信号兜底、worker 信号屏蔽。 -背景：训练前端的「停止」向整个进程组发 SIGTERM，而旧脚本只在 ``learn()`` -正常返回后才保存，导致"停一下就全丢"。这里把三条保证固定成回归测试： +背景：训练前端的「停止」以前直接向进程组发 SIGTERM，而旧脚本只在 ``learn()`` +正常返回后才保存，导致"停一下就全丢"；在 Windows 上 ``os.killpg`` 更是直接 +``AttributeError``。现在停止/暂停走**控制文件**（跨平台），信号只作兜底。这里把 +保证固定成回归测试： 1. 定期 checkpoint 可续训（zip + 同名元数据）； -2. 进程组收到 SIGTERM 后仍能保存最终模型与元数据； -3. subproc worker 忽略停止信号，不抢在主进程收尾之前退出。 +2. 控制文件发出 stop 后仍能保存最终模型与元数据（Windows 同样成立）； +3. POSIX 下进程组 SIGTERM 也能优雅收尾； +4. subproc worker 忽略停止信号，不抢在主进程收尾之前退出。 """ from __future__ import annotations @@ -15,11 +18,14 @@ import os import signal import subprocess import sys +import threading import time from pathlib import Path import pytest +from mcm26b.process_control import spawn_kwargs, terminate_process_tree, write_control + def _train_module(): """按路径加载训练脚本（不触发 torch 导入，torch 只在函数内部惰性导入）。""" @@ -39,13 +45,20 @@ def _require_rl(): class _Run: - """一次子进程训练：stdout 走文件，避免管道写满把子进程卡死。""" + """一次子进程训练：stdout 走文件，避免管道写满把子进程卡死。 + + 与训练前端一致：用控制文件请求暂停/停止，进程组隔离走 + :func:`mcm26b.process_control.spawn_kwargs`（POSIX ``setsid`` / + Windows ``CREATE_NEW_PROCESS_GROUP``）。 + """ def __init__(self, run_dir: Path, *extra: str): script = Path(__file__).parents[1] / "scripts" / "train_hrl_high.py" self.run_dir = run_dir self.log_path = run_dir / "train_stdout.log" self.log_file = self.log_path.open("w", encoding="utf-8") + self.control_path = run_dir / "control.json" + write_control(self.control_path) cmd = [sys.executable, str(script), "--problem", "3", "--stage", "1", "--timesteps", "200000", "--seed", "7", "--envs", "2", "--vec-env", "subproc", @@ -53,13 +66,28 @@ class _Run: "--reward-profile", "time", "--eval-episodes", "1", "--rollout-steps", "32", "--batch-size", "32", "--n-epochs", "1", "--max-wall-s", "600", "--force", "--verbose", "0", + "--control-file", str(self.control_path), "--model", str(run_dir / "q3.zip"), *extra] - self.proc = subprocess.Popen( + self.proc = subprocess.Popen( # pylint: disable=consider-using-with cmd, stdout=self.log_file, stderr=subprocess.STDOUT, text=True, - start_new_session=True) + **spawn_kwargs()) def checkpoints(self) -> list[Path]: - return sorted((self.run_dir / "checkpoints").glob("q3_ckpt_*.zip")) + """历史 checkpoint（``_<步数>.zip``）。 + + 刻意排除滚动恢复点 ``_latest.zip``：它随写随覆盖，不是"第几个 + checkpoint"，混进来会让序号断言失去意义。 + """ + found = (self.run_dir / "checkpoints").glob("q3_ckpt_[0-9]*.zip") + return sorted(path for path in found + if path.stem.rsplit("_", 1)[-1].isdigit()) + + def recovery(self) -> Path: + return self.run_dir / "checkpoints" / "q3_ckpt_latest.zip" + + def resume_hint(self) -> dict: + return json.loads((self.run_dir / "resume.json").read_text( + encoding="utf-8")) def wait_for(self, predicate, timeout_s: float) -> bool: deadline = time.monotonic() + timeout_s @@ -71,8 +99,35 @@ class _Run: time.sleep(0.4) return predicate() + def logs(self) -> str: + try: + return self.log_path.read_text(encoding="utf-8", errors="replace") + except OSError: # pragma: no cover - 文件被并发写时的极端情况 + return "" + + def pause(self) -> None: + write_control(self.control_path, paused=True) + + def resume(self) -> None: + write_control(self.control_path, paused=False) + + def _finish(self) -> str: + self.log_file.close() + return self.logs() + def stop(self) -> str: - """复现训练前端的停止语义：对整个进程组发 SIGTERM。""" + """复现训练前端的停止语义：写控制文件请求优雅收尾（跨平台）。""" + write_control(self.control_path, stop=True) + forced = terminate_process_tree(self.proc, grace_s=240.0) + if forced: + self.log_file.close() + pytest.fail("控制文件停止请求后训练进程未在 240s 内退出（优雅停止失败）") + return self._finish() + + def stop_by_signal(self) -> str: + """POSIX 兜底路径：对整个进程组发 SIGTERM（Windows 不支持 killpg）。""" + if os.name == "nt": # pragma: no cover - 仅在 Windows 上走到 + pytest.skip("Windows 无 os.killpg，信号路径不适用") try: os.killpg(os.getpgid(self.proc.pid), signal.SIGTERM) except (ProcessLookupError, PermissionError): @@ -84,8 +139,12 @@ class _Run: self.proc.wait(timeout=30) self.log_file.close() pytest.fail("SIGTERM 后训练进程未在 240s 内退出（优雅停止失败）") + return self._finish() + + def hard_kill(self) -> None: + """模拟"急停"：立即强杀整棵进程树，不给训练脚本任何收尾机会。""" + terminate_process_tree(self.proc, grace_s=0.0) self.log_file.close() - return self.log_path.read_text(encoding="utf-8", errors="replace") def _run_dir(tmp_path: Path) -> Path: @@ -119,7 +178,7 @@ def test_worker_ignores_stop_signals_but_main_does_not(monkeypatch): class _Worker: name = "SpawnProcess-3" - monkeypatch.setattr(multiprocessing, "current_process", lambda: _Worker()) + monkeypatch.setattr(multiprocessing, "current_process", _Worker) try: module._ignore_stop_signals_in_worker() assert signal.getsignal(signal.SIGTERM) == signal.SIG_IGN @@ -131,7 +190,7 @@ def test_worker_ignores_stop_signals_but_main_does_not(monkeypatch): class _Main: name = "MainProcess" - monkeypatch.setattr(multiprocessing, "current_process", lambda: _Main()) + monkeypatch.setattr(multiprocessing, "current_process", _Main) module._ignore_stop_signals_in_worker() assert signal.getsignal(signal.SIGTERM) == original @@ -168,14 +227,139 @@ def test_policy_hyperparams_records_effective_values(): assert module._policy_hyperparams(object())["n_steps"] == 0 +def test_control_poll_state_machine(tmp_path): + """控制文件轮询状态机：抽样读盘、暂停阻塞、stop 打断、暂停时长累计。""" + module = _train_module() + path = tmp_path / "control.json" + write_control(path, paused=True) + poll = module._ControlPoll(path, poll_every=1, poll_s=0.05) + assert poll.poll().paused is True + + events = [] + + def _resume_later(): + time.sleep(0.15) + write_control(path, paused=False) + + threading.Thread(target=_resume_later, daemon=True).start() + stopped = poll.wait_while_paused(lambda: events.append("pause"), + lambda: events.append("resume")) + assert stopped is False + assert events == ["pause", "resume"] + assert poll.paused_total_s >= 0.1 + + # 暂停期间收到 stop：wait 必须返回 True，让调用方转入收尾 + write_control(path, paused=True) + + def _stop_later(): + time.sleep(0.15) + write_control(path, stop=True) + + threading.Thread(target=_stop_later, daemon=True).start() + assert poll.wait_while_paused(lambda: None, lambda: None) is True + assert poll.poll().stop is True + + +def test_atomic_model_save_keeps_previous_snapshot_on_failure(tmp_path): + """保存途中失败/被杀：目标路径仍是上一份完整快照，不会出现半截 zip。""" + module = _train_module() + target = tmp_path / "q3_ckpt_latest.zip" + target.write_bytes(b"OLD-COMPLETE") + + class _FailingModel: + def save(self, path): + Path(path).write_bytes(b"PARTIAL") + raise RuntimeError("磁盘写满") + + with pytest.raises(RuntimeError): + module._save_model_atomic(_FailingModel(), target) + assert target.read_bytes() == b"OLD-COMPLETE" + leftovers = [p.name for p in tmp_path.iterdir() if p.name != target.name] + assert leftovers and all(name.startswith(".tmp-") for name in leftovers) + + +def test_write_json_atomic_replaces_in_one_step(tmp_path): + module = _train_module() + path = tmp_path / "resume.json" + module._write_json_atomic(path, {"model": "a.zip", "timesteps": 1}) + assert json.loads(path.read_text(encoding="utf-8"))["timesteps"] == 1 + module._write_json_atomic(path, {"model": "b.zip", "timesteps": 2}) + assert json.loads(path.read_text(encoding="utf-8"))["model"] == "b.zip" + assert not list(tmp_path.glob(".tmp-*")) + + +def test_prune_checkpoints_never_deletes_recovery_point(tmp_path): + """滚动恢复点与临时文件不参与历史清理。""" + module = _train_module() + directory = tmp_path / "checkpoints" + directory.mkdir() + latest = directory / "q3_ckpt_latest.zip" + latest.write_bytes(b"RECOVERY") + (directory / ".tmp-q3_ckpt_000000009.zip").write_bytes(b"PARTIAL") + for index in range(5): + (directory / f"q3_ckpt_{index:09d}.zip").write_bytes(b"x") + module._prune_checkpoints(directory, "q3_ckpt", 1) + assert latest.exists() and latest.read_bytes() == b"RECOVERY" + assert (directory / ".tmp-q3_ckpt_000000009.zip").exists() + + # --------------------------------------------------------------- 集成回归 +def test_recovery_point_exists_before_training_and_is_loadable(tmp_path): + """起步基线：训练刚起就有可加载、可续训的产物（--checkpoint-every 0 也成立）。""" + _require_rl() + from sb3_contrib import MaskablePPO + + run = _Run(_run_dir(tmp_path), "--checkpoint-every", "0") + try: + assert run.wait_for(lambda: run.recovery().exists(), 120.0), \ + "起步恢复点未落盘" + loaded = MaskablePPO.load(str(run.recovery())) + assert loaded.num_timesteps >= 0 + hint = run.resume_hint() + assert Path(hint["model"]) == run.recovery() + assert hint["timesteps_semantics"] == "additional" + finally: + run.stop() + + +def test_hard_kill_still_leaves_resumable_recovery_point(tmp_path): + """急停（强杀、无收尾）后仍能续训：滚动恢复点 + resume.json 都在且可加载。""" + _require_rl() + from sb3_contrib import MaskablePPO + + run = _Run(_run_dir(tmp_path), "--checkpoint-every", "0", + "--recovery-every", "32") + try: + assert run.wait_for(lambda: run.recovery().exists(), 120.0), \ + "起步恢复点未落盘" + first = run.recovery().stat().st_mtime_ns + assert run.wait_for( + lambda: run.recovery().stat().st_mtime_ns != first, 300.0), \ + "滚动恢复点未按 --recovery-every 刷新" + assert run.proc.poll() is None, "训练已自行结束，无法验证急停" + finally: + run.hard_kill() # 急停：强杀整棵进程树，没有任何 Python 收尾 + + hint = run.resume_hint() + assert Path(hint["model"]).is_file(), hint + assert Path(hint["model"]) == run.recovery(), hint + loaded = MaskablePPO.load(str(run.recovery())) + assert loaded.num_timesteps >= 32, loaded.num_timesteps + metadata = json.loads(run.recovery().with_suffix(".json").read_text( + encoding="utf-8")) + assert metadata["recovery"] is True + assert metadata["checkpoint_reason"] in ("baseline", "interval") + # 急停没有 graceful 收尾，所以不会有"stopped_by_signal"的最终模型 + assert not (run.run_dir / "q3.zip").exists() + + def test_checkpoint_is_written_and_loadable(tmp_path): _require_rl() from sb3_contrib import MaskablePPO run = _Run(_run_dir(tmp_path), "--checkpoint-every", "64") try: - assert run.wait_for(lambda: bool(run.checkpoints()), 300.0), \ + assert run.wait_for(run.checkpoints, 300.0), \ "训练未在 300s 内产出 checkpoint" finally: run.stop() @@ -193,16 +377,60 @@ def test_checkpoint_is_written_and_loadable(tmp_path): assert (hyper["n_steps"], hyper["batch_size"], hyper["n_epochs"]) == (32, 32, 1) +def test_control_file_stop_saves_model(tmp_path): + """控制文件 stop 是训练前端的默认停止路径，Windows 上同样必须成立。""" + _require_rl() + run = _Run(_run_dir(tmp_path), "--checkpoint-every", "64") + model_path = run.run_dir / "q3.zip" + try: + assert run.wait_for(run.checkpoints, 300.0), \ + "训练未在 300s 内进入 rollout 阶段" + assert run.proc.poll() is None, "训练进程提前退出，无法验证优雅停止" + finally: + output = run.stop() + + assert run.proc.returncode == 0, \ + f"控制文件停止应正常退出，实际 {run.proc.returncode}\n{output}" + assert model_path.exists(), "控制文件停止后最终模型未落盘" + metadata = json.loads(model_path.with_suffix(".json").read_text( + encoding="utf-8")) + assert metadata["stopped_by_signal"] is True + assert metadata["stop_signal"] == "control_file" + assert metadata["checkpoint_files"], "元数据应记录已产出的 checkpoint" + assert not (run.run_dir / "training_failure.json").exists() + assert "saved_after_stop" in output + + +def test_control_file_pause_then_resume_keeps_training_alive(tmp_path): + """暂停是协作式的：进程存活、停在 step 边界，continue 后照常收尾。""" + _require_rl() + run = _Run(_run_dir(tmp_path), "--checkpoint-every", "64") + try: + assert run.wait_for(run.checkpoints, 300.0), \ + "训练未在 300s 内进入 rollout 阶段" + run.pause() + assert run.wait_for(lambda: '"event": "paused"' in run.logs(), 120.0), \ + "训练脚本未在 120s 内进入暂停" + assert run.proc.poll() is None, "暂停不应结束进程" + run.resume() + assert run.wait_for(lambda: '"event": "resumed"' in run.logs(), 120.0), \ + "训练脚本未在 120s 内恢复" + finally: + run.stop() + assert run.proc.returncode == 0 + + +@pytest.mark.skipif(os.name == "nt", reason="Windows 没有 os.killpg，信号兜底路径不适用") def test_sigterm_saves_model_instead_of_losing_the_run(tmp_path): _require_rl() run = _Run(_run_dir(tmp_path), "--checkpoint-every", "64") model_path = run.run_dir / "q3.zip" try: - assert run.wait_for(lambda: bool(run.checkpoints()), 300.0), \ + assert run.wait_for(run.checkpoints, 300.0), \ "训练未在 300s 内进入 rollout 阶段" assert run.proc.poll() is None, "训练进程提前退出，无法验证优雅停止" finally: - output = run.stop() + output = run.stop_by_signal() assert run.proc.returncode == 0, \ f"优雅停止应正常退出，实际 {run.proc.returncode}\n{output}" diff --git a/tests/test_hrl_low_revision.py b/tests/test_hrl_low_revision.py index 982971a..b8ae2bb 100644 --- a/tests/test_hrl_low_revision.py +++ b/tests/test_hrl_low_revision.py @@ -1,7 +1,6 @@ """低层导航运动学与训练入口的行为回归；不验证策略收益。""" import importlib.util -import math from pathlib import Path import pytest @@ -59,7 +58,7 @@ def test_expert_reaches_all_200_fixed_seed_targets(): def test_failed_executor_reports_failure_without_free_terminal_move(): class WestPolicy: - def select(self, observation): + def select(self, _observation): return 4 executor = LowLevelExecutor(WestPolicy(), max_steps=3) @@ -73,7 +72,7 @@ def test_failed_executor_reports_failure_without_free_terminal_move(): def test_env_and_executor_share_step_transitions(): class EastPolicy: - def select(self, observation): + def select(self, _observation): return 0 env = LowLevelEnv(max_steps=3) diff --git a/tests/test_hrl_p1.py b/tests/test_hrl_p1.py index a3a7235..128541e 100644 --- a/tests/test_hrl_p1.py +++ b/tests/test_hrl_p1.py @@ -75,6 +75,9 @@ def test_time_only_reward_has_no_per_source_shaping(): reason="RL 依赖组未安装") def test_project_compat_layer_reexports_real_classes(): from mcm26b.hrl.hierarchical_env import HierarchicalEnv + # project 是仓库根部的兼容层：pytest 经 rootdir 把仓库根放进 sys.path， + # 而单独 lint tests/ 时 pylint 的 sys.path 不含仓库根，故此处局部豁免。 + # pylint: disable=import-error from project.env.local_env import HierarchicalEnv as CompatEnv from project.env.simulator_client import SimulatorClient from project.policies.high_level_ppo import HighLevelPolicy diff --git a/tests/test_hrl_scheduler_revision.py b/tests/test_hrl_scheduler_revision.py index b3486ab..a67ad9f 100644 --- a/tests/test_hrl_scheduler_revision.py +++ b/tests/test_hrl_scheduler_revision.py @@ -43,21 +43,21 @@ def test_fallback_failure_then_success_records_actual_time_position_and_clear(): assert loop.client.state.channel == 20 assert loop.fallback_log[-1]["attempts"] == 2 assert 20 not in loop.blocked_probes - assert loop.anomalies == [] + assert not loop.anomalies finally: loop.client.exit() def test_invalid_policy_output_is_logged_and_replaced_by_valid_action(): class InvalidPolicy: - def select(self, observation, mask): + def select(self, _observation, _mask): return N_CANDIDATES + 1 loop = make_loop(problem=3, high_policy=InvalidPolicy()) candidate = Candidate(STATION_OFFSET, "station", (0.0, 0.0), station_index=0) executed = [] - def execute(action, candidates): + def execute(action, candidates): # pylint: disable=unused-argument executed.append(action) loop.stop_reason = "test_complete" @@ -78,8 +78,10 @@ def test_opportunistic_measure_precedes_candidate_snapshot(): order = [] try: assert loop.config.bayes.opportunistic_measure - with patch.object(loop, "_measure_at_stop", side_effect=lambda: order.append("measure")), \ - patch.object(loop, "build_candidates", side_effect=lambda: order.append("snapshot") or []): + with patch.object(loop, "_measure_at_stop", + side_effect=lambda: order.append("measure")), \ + patch.object(loop, "build_candidates", + side_effect=lambda: order.append("snapshot") or []): loop._coverage_phase() assert order == ["measure", "snapshot"] finally: diff --git a/tests/test_loop_q3.py b/tests/test_loop_q3.py index a55fd29..7e6ee55 100644 --- a/tests/test_loop_q3.py +++ b/tests/test_loop_q3.py @@ -134,7 +134,7 @@ class StepTimingTests(unittest.TestCase): def test_steps_recorded_with_virtual_breakdown(self): sources = random_omni_scene(seed=5, count=2) - summary, report, _ = _run_scene(sources) + summary, _, _ = _run_scene(sources) steps = summary["steps"] self.assertEqual(steps[0]["action"], "enter") self.assertEqual(steps[-1]["action"], "exit") @@ -228,7 +228,7 @@ class SimulatorVirtualCapTests(unittest.TestCase): class AbortFinalizationTests(unittest.TestCase): """断连超过重试上限：仍须 /exit、产出未完成 summary，不得静默崩溃。""" - def test_transport_failure_aborts_with_summary_and_exit(self): + def test_transport_failure_aborts_with_summary(self): """首个检测动作三次断连：run() 正常返回，/exit 成功，stop_reason 留档。""" summary, _, _ = _run_scene( random_omni_scene(seed=5, count=2), diff --git a/tests/test_loop_q4.py b/tests/test_loop_q4.py index 68528e9..cffebae 100644 --- a/tests/test_loop_q4.py +++ b/tests/test_loop_q4.py @@ -97,7 +97,7 @@ class ProbeLadderGeometryTests(unittest.TestCase): loop.blocked_probes[7] = [blocked_point] return loop._next_probe(region, 7) - def test_antipode_reflects_blocked_probe_through_center(self): + def test_antipode_reflects_blocked_probe_in_center(self): """圆心为原点、原探点 (200,0)：对跖复核应返回 (-200,0)。""" region = CandidateRegion.disk((0.0, 0.0), 100.0, 192) # safe≈900>=200 probe = self._probe_after_block(region, (200.0, 0.0)) @@ -113,7 +113,7 @@ class ProbeLadderGeometryTests(unittest.TestCase): self.assertAlmostEqual(math.hypot(probe[0], probe[1]), safe, places=6) self.assertLess(probe[0], 0.0, "应位于原探点的对跖方向") - def test_degenerate_center_block_walks_full_safe_radius(self): + def test_degenerate_center_block_walks_safe_radius(self): """被遮挡探点恰在圆心：沿 +x 走满安全盘半径，不再钳到 1m。""" region = CandidateRegion.disk((0.0, 0.0), 100.0, 192) probe = self._probe_after_block(region, (0.0, 0.0)) @@ -125,7 +125,7 @@ class ProbeLadderGeometryTests(unittest.TestCase): class MixedLoopAcceptanceTests(unittest.TestCase): def test_mixed_ten_sources_all_cleared(self): sources = mixed_scene(seed=5, count=10, directional=3) - summary, report, simulator, _, loop = _run_scene(sources) + summary, report, simulator, _, _ = _run_scene(sources) truth = sorted(s.channel for s in sources) self.assertTrue(summary["completed"]) self.assertEqual(summary["stop_reason"], "all_resolved") @@ -155,7 +155,7 @@ class MixedLoopAcceptanceTests(unittest.TestCase): self.assertEqual(sorted(simulator.cleared), truth) self.assertEqual(report["mismatches"], []) - def test_outward_boundary_directional_source_not_missed(self): + def test_outward_boundary_directional_not_missed(self): """朝向背离圆心、r_eff=1000 的近边界源：覆盖保证仍发现并清除。""" source = Source(channel=7, x=1790.0, y=0.0, r_eff_m=1000.0, directional=True, heading_deg=0.0) @@ -169,8 +169,8 @@ class MixedLoopAcceptanceTests(unittest.TestCase): """固定批量种子回归：任何种子下不得全清失败或破坏守恒性。""" for seed in range(5, 10): sources = mixed_scene(seed=seed, count=10, directional=4) - summary, report, simulator, _, _ = _run_scene(sources, - config=SMALL_CAP) + summary, report, _, _, _ = _run_scene(sources, + config=SMALL_CAP) truth = sorted(s.channel for s in sources) self.assertTrue(summary["completed"], f"seed={seed}") self.assertEqual(sorted(summary["cleared_channels"]), truth, @@ -194,7 +194,7 @@ class JointConstraintBaselineTests(unittest.TestCase): for witness in region.witnesses(300.0): self.assertTrue(region.contains(witness, tol_m=0.05)) - def test_directional_source_retained_through_blocked_probes(self): + def test_directional_source_kept_after_blocked_probes(self): """端到端保留性：定向源被遮挡若干次后仍被清除，不因 no_signal 误删。""" source = Source(channel=3, x=600.0, y=800.0, r_eff_m=1500.0, directional=True, heading_deg=200.0) @@ -261,7 +261,7 @@ class FallbackDisciplineTests(unittest.TestCase): cells = fallback_clear_cells(region.polygon, CONFIG) self.assertLessEqual(len(cells), CONFIG.search.fallback_max_cells) - def test_fallback_exhaustion_records_anomaly_without_deletion(self): + def test_fallback_exhaustion_records_anomaly(self): """后备全失败（空域）：守恒性异常留痕，且不删除空间候选。""" simulator = LocalSimulator([], robot_id="T001", config=CONFIG) client = SimulatorClient("T001", config=CONFIG, diff --git a/tests/test_particles.py b/tests/test_particles.py index e3bea1b..10c1678 100644 --- a/tests/test_particles.py +++ b/tests/test_particles.py @@ -13,7 +13,6 @@ from dataclasses import replace from random import Random from mcm26b.config import ProblemConfig -from mcm26b.geometry.localization import smallest_enclosing_circle from mcm26b.particles import ChannelParticles, angular_gap from mcm26b.protocol.rules import DIRECTION, NEAR, NO_SIGNAL from mcm26b.planner.seven import seven_points diff --git a/tests/test_planner.py b/tests/test_planner.py index 92f33c6..693a057 100644 --- a/tests/test_planner.py +++ b/tests/test_planner.py @@ -31,7 +31,7 @@ def _k1_region() -> CandidateRegion: class SelectionGuaranteeTests(TestCase): - def test_selected_point_receives_every_sector_candidate(self): + def test_selected_point_gets_every_sector_candidate(self): """安全盘内选点对解析角域 K1 内任意候选源的距离 <= 1000m。 直接对截断角域 K1 = Ω ∩ W(s1, θ±δ) ∩ B(s1,1500) 稠密采样验证， @@ -113,7 +113,7 @@ class ClearCertificateTests(TestCase): class SelectSecondPointTests(TestCase): - def test_plan_lies_in_analytic_safe_disk_or_is_equivalent(self): + def test_plan_in_analytic_safe_disk_is_equivalent(self): """P3 入口输出与 geometry 解析证书一致（或提供同等保证）。""" simulator = LocalSimulator([], robot_id="T001") client = SimulatorClient("T001", transport=LocalSimTransport(simulator)) diff --git a/tests/test_planner_revision.py b/tests/test_planner_revision.py index 54b95bc..6388b8c 100644 --- a/tests/test_planner_revision.py +++ b/tests/test_planner_revision.py @@ -38,7 +38,7 @@ def test_seven_stations_cannot_prove_directional_absence(): def test_stopped_opportunistic_measure_is_serial_and_never_repeats_location(): - sim, client, loop = setup([Source(1, 800, 200, 1500)]) + _, client, loop = setup([Source(1, 800, 200, 1500)]) client.enter() loop._record_measure(1, (0, 0), client.measure((0, 0), 1)) client.measure((600, 0), 2) diff --git a/tests/test_practice_replay.py b/tests/test_practice_replay.py index e17a260..19000f8 100644 --- a/tests/test_practice_replay.py +++ b/tests/test_practice_replay.py @@ -40,8 +40,10 @@ class PracticeReplayTests(unittest.TestCase): self.assertEqual(data['source'],'official_practice') self.assertIsNone(data['ground_truth']) directions=[e for e in events if e['response'].get('measure_result')=='direction'] - self.assertEqual(directions[0]['request']['position'],directions[1]['request']['position']) - self.assertEqual(directions[0]['response']['svd_deg'],directions[1]['response']['svd_deg']) + self.assertEqual(directions[0]['request']['position'], + directions[1]['request']['position']) + self.assertEqual(directions[0]['response']['svd_deg'], + directions[1]['response']['svd_deg']) if __name__=='__main__': unittest.main() diff --git a/tests/test_protocol_client.py b/tests/test_protocol_client.py index ed20bd1..8ad2d74 100644 --- a/tests/test_protocol_client.py +++ b/tests/test_protocol_client.py @@ -45,7 +45,7 @@ class FaultRecoveryTests(unittest.TestCase): outcome = client.measure((0, 0), 1) self.assertEqual(outcome.virtual_time_s, 5.0) # measure 只被服务器执行了一次：请求发出两次，但虚拟时刻只推进一次。 - measure_bodies = [body for body in faults.sent_requests[1:]] + measure_bodies = list(faults.sent_requests[1:]) self.assertEqual(len(measure_bodies), 2, "应发出原始请求+一次重试") first, retry = (json.loads(b) for b in measure_bodies) self.assertEqual(first["request_id"], retry["request_id"], @@ -56,22 +56,22 @@ class FaultRecoveryTests(unittest.TestCase): def test_drop_before_apply_is_safe_to_retry(self): """连接失败（未送达）后重试成功，无残留效果。""" - client, faults, simulator = self._client(["drop_before", None]) + client, _, simulator = self._client(["drop_before", None]) client.enter() client.measure((100, 0), 1) self.assertEqual(simulator.virtual_time_s, 25.0) # 100/5 移动 + 5 检测 self.assertEqual(client.state.virtual_time_s, 25.0) def test_server_500_retried_with_same_id(self): - client, faults, simulator = self._client([None, ("status", 500), None, None]) + client, _, _ = self._client([None, ("status", 500), None, None]) client.enter() client.measure((0, 0), 1) self.assertEqual(client.state.virtual_time_s, 5.0) def test_exhausted_retries_raise_and_leave_state_clean(self): # enter 放行后，measure 连续注入断连直至超过 max_attempts=3。 - client, faults, simulator = self._client([None, "drop_before", "drop_before", - "drop_before"]) + client, _, simulator = self._client([None, "drop_before", "drop_before", + "drop_before"]) client.enter() with self.assertRaises(TransportError): client.measure((0, 0), 1) @@ -83,7 +83,7 @@ class FaultRecoveryTests(unittest.TestCase): self.assertEqual(outcome.virtual_time_s, 5.0) def test_accepted_false_raises_and_state_untouched(self): - client, faults, simulator = self._client([None]) + client, _, _ = self._client([None]) with self.assertRaises(ProtocolError) as ctx: client.measure((0, 0), 1) # 未 enter -> accepted=false self.assertEqual(ctx.exception.status, 200) @@ -103,7 +103,7 @@ class DeadlineBudgetTests(unittest.TestCase): now[0] += 300.0 self.assertAlmostEqual(client.real_remaining_s, 900.0) - def test_action_blocked_when_reserve_margin_cannot_be_kept(self): + def test_action_blocked_when_reserve_margin_low(self): now = [0.0] simulator = LocalSimulator([], robot_id="T001") client = SimulatorClient("T001", transport=LocalSimTransport(simulator), @@ -172,8 +172,8 @@ class PracticeReplayTests(unittest.TestCase): events = [dict(path=e["path"], request=e["request"], response=e["response"]) for e in self._events()] stats = summarize(events) - directions = [e for e in events - if e["response"].get("measure_result") == "direction"] + _directions = [e for e in events + if e["response"].get("measure_result") == "direction"] self.assertGreater(stats["measures"], 0) self.assertEqual(stats["clears"], 0, "该次演练只采集观测，未调用 /clear") diff --git a/tests/test_referee_decoupling.py b/tests/test_referee_decoupling.py index 7a30b32..789ed89 100644 --- a/tests/test_referee_decoupling.py +++ b/tests/test_referee_decoupling.py @@ -29,7 +29,7 @@ def test_runtime_code_never_imports_referee(): "print('clean')\n" ) result = subprocess.run([sys.executable, "-c", code], capture_output=True, - text=True, cwd=ROOT) + text=True, cwd=ROOT, check=False) assert result.returncode == 0, result.stderr assert "clean" in result.stdout diff --git a/tests/test_rl.py b/tests/test_rl.py index bd98045..f0cd6da 100644 --- a/tests/test_rl.py +++ b/tests/test_rl.py @@ -4,7 +4,6 @@ RL 依赖属于可选组 [dependency-groups] rl；未安装 numpy/gymnasium 时 全部跳过（运行闭环不依赖它们）。 """ -import math import unittest from pathlib import Path @@ -74,13 +73,13 @@ class TrainingEnvironmentTests(unittest.TestCase): self.assertIsInstance(reward, float) self.assertFalse(terminated and truncated) - def test_q4_directional_episode_reports_blocked_or_fallback(self): + def test_q4_directional_reports_blocked_or_fallback(self): env = LocalizationEnv(problem=4, directional_probability=1.0) - obs, info = env.reset(seed=7) + _, info = env.reset(seed=7) self.assertTrue(info["source"]["directional"]) terminated = truncated = False for _ in range(env.max_episode_steps + 2): - obs, reward, terminated, truncated, info = env.step( + _, _, terminated, truncated, info = env.step( env.action_space.sample()) if terminated or truncated: break diff --git a/tests/test_rl_revision.py b/tests/test_rl_revision.py index 3204e5c..129dd12 100644 --- a/tests/test_rl_revision.py +++ b/tests/test_rl_revision.py @@ -9,7 +9,7 @@ np = pytest.importorskip('numpy') pytest.importorskip('gymnasium') from mcm26b.rl.env import LocalizationEnv -from mcm26b.rl.features import action_mask, build_observation, candidate_actions +from mcm26b.rl.features import build_observation, candidate_actions def test_training_observation_matches_runtime_channel_belief(): @@ -85,10 +85,12 @@ def test_bc_and_ppo_forward_mapping_agree(): spec.loader.exec_module(module) torch.set_num_threads(1) env = LocalizationEnv() - ppo = PPO('MlpPolicy',env,policy_kwargs={'net_arch':[128,128]},device='cpu',n_steps=16,batch_size=8) + ppo = PPO('MlpPolicy',env,policy_kwargs={'net_arch':[128,128]},device='cpu', + n_steps=16,batch_size=8) bc = module.BCMlp() state = bc.net.state_dict() - ppo.policy.mlp_extractor.policy_net.load_state_dict({k:v for k,v in state.items() if not k.startswith('4.')}) + ppo.policy.mlp_extractor.policy_net.load_state_dict( + {k:v for k,v in state.items() if not k.startswith('4.')}) ppo.policy.action_net.load_state_dict({'weight':state['4.weight'],'bias':state['4.bias']}) x = torch.randn(10,401) torch.testing.assert_close(bc(x),ppo.policy.action_net(ppo.policy.mlp_extractor.policy_net(x))) diff --git a/tests/test_training_ui.py b/tests/test_training_ui.py index afe76db..bc141aa 100644 --- a/tests/test_training_ui.py +++ b/tests/test_training_ui.py @@ -6,11 +6,16 @@ - SB3 表格解析必须给每一行打上 ``total_timesteps`` 横轴（曾把表格行当独立 点，曲线上出现重复的幽灵点）； - ``metric_offset``/``metric_end`` 游标契约（前端增量拉取依赖它）； -- 用一段桩训练脚本真起子进程，验证启动/轮询/停止/退出码全链路。 +- 用一段桩训练脚本真起子进程，验证启动/轮询/暂停/继续/停止/退出码全链路。 """ +# pytest 用参数名注入 fixture：`stub_repo`/`client` 这类参数在函数体里不直接 +# 引用也是正常用法（它们的作用是 setup/teardown），这里统一放行。 +# pylint: disable=unused-argument + from __future__ import annotations +import dataclasses import http.client import json import sys @@ -22,6 +27,7 @@ import pytest pytest.importorskip("mcm26b") +from mcm26b import process_control as pc from mcm26b.training_ui import demo, server as td from mcm26b.training_ui.schema import (CONFIG_FIELDS, config_schema, preset_payloads) @@ -32,7 +38,7 @@ REPO = Path(__file__).resolve().parents[1] # 最后打印一行元数据。全部行为由参数控制，不依赖 torch/SB3。 STUB_SCRIPT = r''' """训练前端测试桩：模拟 SB3 verbose=1 表格 + --progress-json 输出。""" -import argparse, json, signal, sys, time +import argparse, json, os, signal, sys, time from pathlib import Path parser = argparse.ArgumentParser() @@ -41,12 +47,17 @@ parser.add_argument("--stage", type=int, default=1) parser.add_argument("--timesteps", type=int, default=96) parser.add_argument("--model", default="out/model.zip") parser.add_argument("--rollout-steps", type=int, default=32) -parser.add_argument("--sleep", type=float, default=0.0) +# 默认 sleep 可由环境变量控制：HTTP 端到端用例无法直接追加命令行参数， +# 用小 sleep 拉长运行窗口，才能稳定地测"暂停/继续"。 +parser.add_argument("--sleep", type=float, + default=float(os.environ.get("STUB_TRAIN_SLEEP", "0"))) parser.add_argument("--lines", type=int, default=4) parser.add_argument("--table-first", action="store_true") parser.add_argument("--progress-json", action="store_true") parser.add_argument("--crash", action="store_true") parser.add_argument("--ignore-sigterm", action="store_true") +parser.add_argument("--ignore-control", action="store_true") +parser.add_argument("--control-file", default=None) args, _extra = parser.parse_known_args() model = Path(args.model) @@ -65,6 +76,49 @@ print("NORMAL-LOG-LINE") sys.stdout.flush() +def read_control(): + if not args.control_file or args.ignore_control: + return {} + try: + with open(args.control_file, encoding="utf-8") as handle: + return json.load(handle) + except Exception: + return {} + + +def control_pause_or_stop(step): + """返回 True 表示应退出；暂停时阻塞到继续或停止（模拟协作式暂停）。""" + state = read_control() + if state.get("stop"): + print(json.dumps({"event": "stop_requested", "signal": "control_file"}), flush=True) + return True + if state.get("paused"): + print(json.dumps({"event": "paused", "timesteps": step}), flush=True) + while True: + time.sleep(0.05) + state = read_control() + if state.get("stop"): + print(json.dumps({"event": "stop_requested", "signal": "control_file"}), + flush=True) + return True + if not state.get("paused"): + break + print(json.dumps({"event": "resumed", "timesteps": step}), flush=True) + return False + + +def sleep_with_control(seconds, step): + """把长 sleep 切成 0.1s 片，保证停止/暂停请求能被及时响应。""" + deadline = time.monotonic() + max(0.0, seconds) + while True: + if control_pause_or_stop(step): + raise SystemExit(0) + remaining = deadline - time.monotonic() + if remaining <= 0: + return + time.sleep(min(0.1, remaining)) + + def render_table(step, fps): return ( "-----------------------------\n" @@ -72,16 +126,15 @@ def render_table(step, fps): "| ep_len_mean | 9.67 |\n" "| ep_rew_mean | -3.24 |\n" "| time/ | |\n" - "| fps | %-8d |\n" - "| total_timesteps | %-8d |\n" - "-----------------------------\n" % (fps, step) + f"| fps | {fps:<8d} |\n" + f"| total_timesteps | {step:<8d} |\n" + "-----------------------------\n" ) for index in range(args.lines): - if args.sleep: - time.sleep(args.sleep) step = 32 * (index + 1) + sleep_with_control(args.sleep, step) fps = 17 + index progress = { "event": "progress", "timesteps": step, "n_updates": index + 1, @@ -153,7 +206,7 @@ def test_schema_matches_train_config_defaults(): assert field["default"] == getattr(defaults, field["key"]), \ f"{field['key']} 默认值与后端不一致" exposed = {field["key"] for field in schema["fields"]} - missing = set(defaults.__dataclass_fields__) - exposed + missing = {field.name for field in dataclasses.fields(defaults)} - exposed assert not missing, f"TrainConfig 字段未暴露给界面: {sorted(missing)}" groups = {group["key"] for group in schema["groups"]} for field in schema["fields"]: @@ -227,9 +280,11 @@ class _FakeJob: self._last_table_steps = None self._last_json_steps = None self._last_monitor_sig = None + self.status = "running" self.config = make_config() _on_line = td.TrainingJob._on_line + _sync_control_events = td.TrainingJob._sync_control_events _parse_progress_json = td.TrainingJob._parse_progress_json _parse_sb3_table = td.TrainingJob._parse_sb3_table _flush_table = td.TrainingJob._flush_table @@ -274,9 +329,9 @@ def test_sb3_table_with_separators_emits_one_point_per_table(): for index, steps in enumerate((32, 64, 96)): job._on_line("-----------------------------") job._on_line("| rollout/ | |") - job._on_line("| ep_rew_mean | %.2f |" % (-3.0 + index)) + job._on_line(f"| ep_rew_mean | {-3.0 + index:.2f} |") job._on_line("| time/ | |") - job._on_line("| total_timesteps | %d |" % steps) + job._on_line(f"| total_timesteps | {steps} |") job._on_line("-----------------------------") job._flush_table() assert [point["timesteps"] for point in job.metrics] == [32, 64, 96], list(job.metrics) @@ -476,7 +531,7 @@ def test_job_lifecycle_with_stub_script(stub_repo): def test_job_reports_nonzero_exit_code(stub_repo): config = make_config(model="output/hrl/ui/test/crash/q3.zip") job = td.TrainingJob(config) - job.command = td.build_command(config) + ["--crash"] + job.command = list(job.command) + ["--crash"] job.start() assert wait_until(lambda: job.status in ("exited", "failed"), timeout=30) assert job.status == "failed" @@ -484,15 +539,200 @@ def test_job_reports_nonzero_exit_code(stub_repo): assert job.snapshot(log_tail=5)["process"]["tone"] == "bad" -def test_stop_sends_sigterm_and_marks_stopped(stub_repo): +def test_job_command_carries_control_file(stub_repo): + """控制文件路径必须随命令下发，否则暂停/停止退化成超时强杀。""" + config = make_config(model="output/hrl/ui/test/control/q3.zip") + job = td.TrainingJob(config, control_dir=Path(stub_repo).parent) + assert "--control-file" in job.command + assert str(job.control_path) in job.command + assert job.control_path.parent == Path(stub_repo).parent + + +def test_stop_gracefully_marks_stopped_without_forced_kill(stub_repo): config = make_config(model="output/hrl/ui/test/stop/q3.zip") job = td.TrainingJob(config) - job.command = td.build_command(config) + ["--sleep", "5", "--lines", "50"] + job.command = list(job.command) + ["--sleep", "5", "--lines", "50"] job.start() assert wait_until(lambda: job.status == "running", timeout=10) assert job.stop() is True assert wait_until(lambda: job.status in ("stopped", "failed"), timeout=15) assert job.status == "stopped" + # 桩脚本读控制文件后主动退出：不应走到强杀分支 + assert job.forced_kill is False + + +def test_pause_and_resume_via_control_file(stub_repo): + config = make_config(model="output/hrl/ui/test/pause/q3.zip") + job = td.TrainingJob(config) + job.command = list(job.command) + ["--sleep", "0.2", "--lines", "60"] + job.start() + assert wait_until(lambda: job.status == "running", timeout=10) + assert job.pause() is True + # 训练脚本回显 paused 事件；状态由 _sync_control_events 校正 + assert wait_until( + lambda: any('"event": "paused"' in line for line in job.snapshot()["log_lines"]), + timeout=10), "训练脚本未回显 paused 事件" + assert job.status == "paused" + paused_metrics = len(job.snapshot()["metrics"]) + time.sleep(0.6) + assert len(job.snapshot()["metrics"]) <= paused_metrics + 1, "暂停后仍在产出指标" + assert job.resume() is True + assert wait_until( + lambda: any('"event": "resumed"' in line for line in job.snapshot()["log_lines"]), + timeout=10), "训练脚本未回显 resumed 事件" + assert wait_until(lambda: job.status == "running", timeout=5) + assert job.stop() is True + assert wait_until(lambda: job.status in ("stopped", "failed"), timeout=15) + assert job.status == "stopped" + + +def test_stop_forced_kill_when_control_ignored(stub_repo, monkeypatch): + """训练进程不读控制文件时必须超时强杀，并留下恢复指引。""" + monkeypatch.setattr(td, "STOP_GRACE_S", 0.5) + config = make_config(model="output/hrl/ui/test/stubborn/q3.zip") + job = td.TrainingJob(config) + job.command = list(job.command) + ["--ignore-control", "--sleep", "5", "--lines", "50"] + job.start() + assert wait_until(lambda: job.status == "running", timeout=10) + # 造一份"磁盘上已有滚动恢复点"的现场，强杀后应被记录进恢复指引 + ckpt_dir = job.config.model_path.parent / "checkpoints" + ckpt_dir.mkdir(parents=True, exist_ok=True) + latest = ckpt_dir / "q3_ckpt_latest.zip" + latest.write_bytes(b"PK\x03\x04fake") + assert job.stop() is True + assert wait_until(lambda: job.status in ("stopped", "failed"), timeout=20) + assert job.forced_kill is True + assert job.snapshot(log_tail=5)["forced_kill"] is True + marker = json.loads( + (job.config.model_path.parent / "forced_stop.json").read_text(encoding="utf-8")) + assert marker["event"] == "forced_stop" + assert marker["reason"] == "grace_timeout" + assert marker["recovery"]["model"] == str(latest) + assert marker["recovery"]["can_resume"] is True + + +def test_discover_recovery_prefers_resume_hint(tmp_path): + run_dir = tmp_path / "run" + (run_dir / "checkpoints").mkdir(parents=True) + latest = run_dir / "checkpoints" / "q3_ckpt_latest.zip" + latest.write_bytes(b"PK\x03\x04fake") + final = run_dir / "q3.zip" + final.write_bytes(b"PK\x03\x04fake") + (run_dir / "resume.json").write_text(json.dumps({ + "model": str(latest), "timesteps": 128, "reason": "interval", + "created_unix_s": 5.0}), encoding="utf-8") + state = td.discover_recovery(run_dir, final) + assert state["can_resume"] is True + assert state["model"] == str(latest) + assert state["timesteps"] == 128 + assert state["source"] == "resume.json" + + +def test_discover_recovery_falls_back_to_scan_on_dangling_hint(tmp_path): + run_dir = tmp_path / "run" + (run_dir / "checkpoints").mkdir(parents=True) + latest = run_dir / "checkpoints" / "q3_ckpt_latest.zip" + latest.write_bytes(b"PK\x03\x04fake") + # 悬空指针（目标文件已不在）必须回退到扫描，而不是给出一个打不开的路径 + (run_dir / "resume.json").write_text( + json.dumps({"model": str(run_dir / "gone.zip")}), encoding="utf-8") + state = td.discover_recovery(run_dir, run_dir / "q3.zip") + assert state["can_resume"] is True + assert state["model"] == str(latest) + assert state["source"] == "scan" + + +def test_discover_recovery_with_nothing_on_disk(tmp_path): + state = td.discover_recovery(tmp_path, tmp_path / "q3.zip") + assert state["can_resume"] is False and state["model"] is None + assert state["reason"] == "none" + + +def test_command_carries_checkpoint_and_recovery_flags(): + config = make_config(model="output/x/q3.zip", recovery_every=100, + checkpoint_every=500, checkpoint_keep=2) + joined = " ".join(td.build_command(config)) + assert "--recovery-every 100" in joined + assert "--checkpoint-every 500" in joined + assert "--checkpoint-keep 2" in joined + + +def test_config_rejects_bad_recovery_settings(): + with pytest.raises(ValueError): + make_config(recovery_every=-1) + with pytest.raises(ValueError): + make_config(checkpoint_every=-1) + with pytest.raises(ValueError): + make_config(checkpoint_keep=0) + + +def test_pause_resume_return_false_without_live_process(stub_repo): + config = make_config(model="output/hrl/ui/test/nolive/q3.zip") + job = td.TrainingJob(config) + assert job.pause() is False and job.resume() is False + job.start() + assert wait_until(lambda: job.status in ("exited", "failed"), timeout=30) + assert job.pause() is False and job.resume() is False + + +# ---------------------------------------------------- 跨平台进程控制 +def test_control_file_roundtrip_and_corruption(tmp_path): + path = tmp_path / "control.json" + assert pc.read_control(path) == pc.ControlState() # 缺失 = 无指令 + pc.write_control(path, paused=True) + assert pc.read_control(path).paused is True + pc.write_control(path, stop=True) + state = pc.read_control(path) + assert state.stop is True and state.paused is False + path.write_text("{ 半截 JSON", encoding="utf-8") + assert pc.read_control(path) == pc.ControlState() # 损坏 = 无指令 + + +def test_spawn_kwargs_isolates_child_process_group(monkeypatch): + monkeypatch.setattr(pc, "IS_WINDOWS", False) + assert pc.spawn_kwargs() == {"start_new_session": True} + monkeypatch.setattr(pc, "IS_WINDOWS", True) + kwargs = pc.spawn_kwargs() + # Windows 上 start_new_session 被 CPython 静默忽略，绝不能再用它 + assert "start_new_session" not in kwargs + if kwargs: + assert kwargs["creationflags"] == pc._CREATE_NEW_PROCESS_GROUP + + +def test_hard_kill_on_windows_uses_taskkill(monkeypatch): + commands = [] + + class _FakeProcess: + pid = 4242 + + def kill(self): # pragma: no cover - Windows 分支不应走到这里 + commands.append("Popen.kill") + + monkeypatch.setattr(pc, "IS_WINDOWS", True) + monkeypatch.setattr(pc.subprocess, "run", + lambda argv, **kwargs: commands.append(argv)) + pc._hard_kill_tree(_FakeProcess()) + assert commands and commands[0][0] == "taskkill" + assert "/PID" in commands[0] and "4242" in commands[0] + + +def test_hard_kill_without_killpg_does_not_raise(monkeypatch): + """回归：Windows 上 os 没有 killpg/getpgid，旧代码会抛 AttributeError。""" + killed = [] + + class _FakeProcess: + pid = 4242 + + def kill(self): + killed.append(True) + + class _PosixlessOS: + """模拟 Windows：只有 getpid 之类，没有 killpg/getpgid。""" + + monkeypatch.setattr(pc, "IS_WINDOWS", False) + monkeypatch.setattr(pc, "os", _PosixlessOS()) + pc._hard_kill_tree(_FakeProcess()) + assert killed == [True] def test_reader_flushes_truncated_table_on_exit(stub_repo): @@ -628,8 +868,42 @@ def test_http_start_poll_stop(client): status, body = call(port, "POST", "/api/train/stop", {}) assert status == 200 and body["stopped"] is True + # 等训练进程真正退出后再点一次：应返回 False 而不是报错 + assert wait_until( + lambda: call(port, "GET", "/api/train/state?log_tail=5")[1]["job"]["status"] + in ("stopped", "failed"), timeout=15) status, body = call(port, "POST", "/api/train/stop", {}) - assert status == 200 and body["stopped"] is False # 已结束，返回 False 而不是报错 + assert status == 200 and body["stopped"] is False + + +def test_http_pause_resume_and_state(client, monkeypatch): + monkeypatch.setenv("STUB_TRAIN_SLEEP", "0.6") + _ui, port = client + status, body = call(port, "POST", "/api/train/start", + {"model": "output/hrl/ui/test/http-pause/q3.zip", + "timesteps": 4096, "rollout_steps": 32, "batch_size": 32}) + assert status == 200 + + def status_now(): + return call(port, "GET", "/api/train/state?log_tail=20")[1]["job"]["status"] + + def is_running(): + return status_now() == "running" + + assert wait_until(is_running, timeout=10) + status, body = call(port, "POST", "/api/train/pause", {}) + assert status == 200 and body["paused"] is True + assert wait_until(lambda: status_now() == "paused", timeout=10), status_now() + # 暂停中仍视为"有任务在跑"：不允许再开一局 + status, body = call(port, "POST", "/api/train/start", + {"model": "output/hrl/ui/test/http-pause2/q3.zip"}) + assert status == 409 and "已有训练任务" in body["error"] + status, body = call(port, "POST", "/api/train/resume", {}) + assert status == 200 and body["resumed"] is True + assert wait_until(is_running, timeout=10), status_now() + status, body = call(port, "POST", "/api/train/stop", {}) + assert status == 200 and body["stopped"] is True + assert wait_until(lambda: status_now() in ("stopped", "failed"), timeout=15) def test_http_rejects_second_job_while_running(client): @@ -661,7 +935,7 @@ def test_http_binds_loopback_only(client): def test_port_conflict_reports_actionable_error(client): - ui, port = client + _ui, port = client with pytest.raises(OSError) as excinfo: td.TrainingUI(ui_port=port) assert "ui-port" in str(excinfo.value) diff --git a/tests/test_training_ui_js.py b/tests/test_training_ui_js.py index bcadf63..f8bb906 100644 --- a/tests/test_training_ui_js.py +++ b/tests/test_training_ui_js.py @@ -36,6 +36,7 @@ def test_training_ui_frontend_logic(tmp_path): result = subprocess.run( ["node", str(HARNESS)], capture_output=True, text=True, timeout=120, + check=False, env={**os.environ, "REPO": str(REPO), "DEMO_STATE": str(scenarios_path), "SCHEMA": str(schema_path)}) assert result.returncode == 0, \