feat: 链路控制、延迟校准与长测工具,补全文档
构建客户端 / 检查(类型 + 单元测试) (push) Canceled after 0s
构建客户端 / Linux x64(deb + AppImage) (push) Canceled after 0s
构建客户端 / Windows x64(NSIS 安装包) (push) Canceled after 0s
构建客户端 / macOS 通用(.app + .dmg) (push) Canceled after 0s

- 固件:新增 link_control / stream_control,覆盖运行中 Wi-Fi 断线恢复、取帧节拍、
  温控与质量/帧率减载;UDP 发送增加有界重试预算与帧期限,PSRAM JPEG 直发减少拷贝。
- 协议:重组器严格校验分片布局,统计未收齐、超时与完全缺失的帧,处理帧号回绕与设备重启。
- 桌面端:视频改 latest-value 通道并前置帧时间标记,拒绝积压和慢解码的过期画面;新增
  延迟标记页与 --latency-report 校准,完善录制、服务端与退出流程。
- 文档与脚本:新增硬件、开发、延迟、功耗、实时性、稳定性文档和改动记录,重写根 README;
  新增构建、性能、延迟与串口长测脚本,以及长测客户端脚本。
- 构建:固定 esp-1.97.0.0 工具链,分区表随构建复制,更新 sdkconfig 与忽略规则。
This commit is contained in:
2026-10-19 00:19:07 +08:00
parent 4794bed2cf
commit cc6b82d7ec
60 changed files with 4347 additions and 655 deletions
+47 -12
View File
@@ -12,8 +12,8 @@
```text
desktop/
core/ cam-core:不含任何界面与 Tauri 的东西(80 个测试)
src-tauri/ Tauri 后端:推流、命令、录制(30 个测试)
core/ cam-core:不含任何界面与 Tauri 的东西(83 个测试)
src-tauri/ Tauri 后端:推流、命令、录制(32 个测试)
src/ React 前端:两套页面 + HUD
src/lib/ IPC 封装、WebSocket 客户端、状态存储、格式化
src/components/ 视频窗格、血条、胜负横幅、瞄准器、装甲板表、队名编辑、录制面板、设置抽屉
@@ -27,16 +27,37 @@ cd desktop
pnpm install
pnpm app # = tauri dev:起 vite + 编 Rust + 开窗口
pnpm build # 只构建前端(tsc --noEmit && vite build)
pnpm tauri build # 出 .app 与 .dmg
pnpm tauri build # 默认打包 macOS 的 .app 与 .dmg;Windows 见下文
```
下面两条是 Rust 侧,用它验证不依赖窗口的部分:
```sh
cd desktop/core && cargo test # 80
cd desktop/src-tauri && cargo test # 30,其中几条真的会起 ffmpeg
cd desktop/core && cargo test # 83
cd desktop/src-tauri && cargo test # 32,其中几条真的会起 ffmpeg
```
### Windows x64 构建
需要 Node.js 22.12+(或 24)、pnpm 12、Rust stable MSVC,以及 Visual Studio Build Tools 的 C++ 桌面开发组件和 Windows SDK。应用窗口使用 WebView2 Runtime。
在 `desktop/` 中执行:
```powershell
pnpm install --frozen-lockfile
pnpm tauri build --target x86_64-pc-windows-msvc --bundles nsis --ci -- --locked --config 'unstable.build-std=[]'
```
`--target` 覆盖本目录原来的 macOS target,`--bundles nsis` 选择 Windows 安装包;末尾的 Cargo 参数清空从根固件项目继承的 `build-std`。
- 程序:`src-tauri/target/x86_64-pc-windows-msvc/release/cam-desktop.exe`。
- 安装包:同一 release 目录下的 `bundle/nsis/*-setup.exe`。
- 核心测试:进入 `core/` 后运行 `cargo +stable test --locked --target x86_64-pc-windows-msvc --config 'unstable.build-std=[]'`。
视频录制另需可从 `PATH` 找到的 `ffmpeg`,也可以在配置的 `[recording] ffmpeg` 中指定路径;基础图传显示不依赖它。
设备发现使用板卡的 Hello 广播;客户端除广播订阅外,还会按已发现设备的 IP 定向续订。Windows 开启 VPN / 虚拟网卡时,全局广播可能走虚拟接口;定向续订可避免“能发现并绑定,但始终等待画面”的情况。队伍端的绑定和画面过滤仍然生效。
### 演示模式:没有板子也能把整条链路跑起来
```sh
@@ -410,11 +431,13 @@ open 'http://127.0.0.1:1420/?preview=referee' # 裁判端(密码 1234)
## 测试
```sh
cd desktop/core && cargo test # 80:重组、发现、两道闸门、帧率窗口、槽位、配置、密码、胜负判定
cd desktop/src-tauri && cargo test # 30:帧头布局、录制、截图、演示画面、命令行
cd protocol && cargo test # 15
cd desktop/core && cargo test # 83:重组、发现、两道闸门、帧率窗口、槽位、配置、密码、胜负判定
cd desktop/src-tauri && cargo test # 32:帧头布局、录制、截图、演示画面、命令行
cd protocol && cargo test # 20
```
前端解码队列测试:在 `desktop/` 运行 `pnpm test:stream`,5 项测试覆盖慢解码时保留最新帧、多路独立处理、关闭清理、坏帧恢复与设备重启。它不替代实际 WebView 的画面验证。
三条值得单独说的:
- **录制**那条真的起 ffmpeg、喂 20 帧、停,然后用 `ffprobe` 读回帧数与分辨率。只断言"文件
@@ -444,8 +467,11 @@ RUST_LOG=debug ./target/aarch64-apple-darwin/debug/cam-desktop | grep 'stream:'
| `demo-*`(仅演示) | 合成协议数据报 | 走同一条真实路径 |
| tokio 任务 | WebSocket 服务、5 Hz 快照 | 异步 IO |
跨线程只有两样东西:`Arc` 与广播通道。帧在接收线程里编码一次(16 字节头 + JPEG),
所有界面共用同一份 `Bytes`。
帧在接收线程里编码一次(16 字节头 + JPEG),所有界面共用同一份 `Bytes`。
视频使用每路只保留最新帧的 watch 通道;状态与控制消息使用有序广播通道。
前端每路最多保留一次正在进行的解码和一张最新待解码帧。
每张本地 JPEG 前增加 `frameTiming` 文本消息,包含同机后端发布时刻;JPEG 二进制头仍为 16 字节。前端在解码前后检查本机队列年龄,超过 100 ms 的待解码帧或解码结果直接释放,恢复后优先显示最新画面。后端不重放超过 100 ms 的缓存帧,本地视频 socket 写入超时后重连。旧版没有时间标记时仍支持 JPEG,但只能按 JS 收到时刻判断本地年龄。这些时间不能用于测量 ESP 到电脑的无线传输延迟。
## 已知限制与没做的
@@ -459,11 +485,20 @@ RUST_LOG=debug ./target/aarch64-apple-darwin/debug/cam-desktop | grep 'stream:'
`ws://127.0.0.1:*`(端口是启动时随机分配的,写不死)。验证方式是"前端能不能连上后端"
——见下面"测试"一节最后一条。
- **证书与公证**没做,`.dmg` 只在本机可用。
- 前端没有自动化测试。界面这一侧的验证手段是浏览器预览 + 真机运行;逻辑都在 `core/` 里
有测试。
- 前端已有解码队列测试;布局、实际 WebView 解码和画面显示仍需浏览器预览及真机运行验证。
## 排查
需要测量相机到客户端的整条链路时,使用裁判配置启动校准模式:
```powershell
.\cam-desktop.exe --config .\referee.toml --latency-report .\latency-run.ndjson
```
报告路径必须是新文件。让相机完整拍到左侧四个彩色角块,避开右侧回显,固定相机和窗口位置。标记每 50 ms 更新;程序沿正常接收、解码和 `VideoSurface` 绘制路径记录每个标记的首次有效回显,拒绝曝光混合而无法可靠解码的帧。NDJSON 包含相邻 rAF 夹定的时间、解码完成到画布绘制时间和标记分析耗时,不保存图像或配置。面板扫描和像素响应未测量,精确物理光学延迟需用高速相机复核。正常启动不运行标记识别;校准模式不会额外订阅一份 UDP 视频。
`pnpm test:latency` 验证标记边界、旋转/镜像、混合曝光拒绝及屏幕原色泛白情形。
| 现象 | 原因 |
|---|---|
| 起不来,提示绑不上 UDP 5600 | 同一端口上已经跑着另一个客户端(旧的 Slint 版或另一份本程序)。看 `~/.config/cam-desktop/startup-error.txt` |
+26
View File
@@ -55,6 +55,7 @@ dependencies = [
"rand",
"serde",
"serde_json",
"socket2",
"toml",
]
@@ -297,6 +298,16 @@ dependencies = [
"serde_core",
]
[[package]]
name = "socket2"
version = "0.6.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4"
dependencies = [
"libc",
"windows-sys",
]
[[package]]
name = "subtle"
version = "2.6.1"
@@ -391,6 +402,21 @@ dependencies = [
"wit-bindgen",
]
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-sys"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
dependencies = [
"windows-link",
]
[[package]]
name = "winnow"
version = "0.7.15"
+1
View File
@@ -10,6 +10,7 @@ protocol = { path = "../../protocol" }
anyhow = "1"
log = "0.4"
socket2 = "0.6"
# 配置的序列化。这里与旧客户端(client/)的做法相反:那一版的配置只有十来个标量
# 字段,手写 TOML 换来零依赖是划算的。现在配置里有 HUD 的嵌套结构和密码哈希,手写
+114 -22
View File
@@ -11,7 +11,8 @@
//! 两个方向各司其职:
//!
//! - 板子每 2 秒广播一次 [`protocol::MsgType::Hello`],客户端被动收到就知道有谁在线。
//! - 客户端广播 [`protocol::MsgType::Subscribe`],板子收到后开始定向推流。
//! - 客户端广播 [`protocol::MsgType::Subscribe`],并向已发现的板子单播续订。
//! 多网卡 / VPN 可能把全局广播路由到虚拟网卡;单播按板子 IP 选择局域网接口。
//!
//! 板子靠**源地址**记住客户端,所以客户端只要周期性续订就能维持推流,不存在任何
//! 超时淘汰的竞态。
@@ -59,7 +60,7 @@ use protocol::{
/// 来自网络,而不是来自过小的缓冲。
const RECV_BUFFER: usize = 2048;
/// 重新广播订阅的间隔。
/// 广播及定向续订的间隔。
///
/// 比板子判定"客户端掉线"的超时(6 秒)短得多,所以偶发丢一两个包不会中断推流。
const SUBSCRIBE_INTERVAL: Duration = Duration::from_millis(800);
@@ -291,8 +292,7 @@ impl Discovery {
/// 客户端身份。
///
/// 常量而非 MAC 派生:队伍客户端和裁判终端各自唯一,进程内也不需要区分自己,而一个
/// 确定的常量让日志更好读。
/// 旧客户端使用的固定标识,保留用于兼容判断。
pub const CLIENT_ID: u32 = 0x0C11_0001;
/// 接收循环的可变状态:只由接收线程在跑循环时访问。
@@ -318,6 +318,7 @@ pub struct Receiver {
/// 身份可以运行时切换(同一个进程里从裁判端切到用户端),所以订阅里声明的角色
/// 也得能改——它决定板子要不要给这个客户端推流。
role: Mutex<SubscribeRole>,
client_id: u32,
camera: Ipv4Addr,
stop: Arc<AtomicBool>,
}
@@ -327,6 +328,10 @@ impl Receiver {
///
/// `only_sender` 为 `Some` 时只接受那一块板子的帧。
pub fn new(camera: Ipv4Addr, role: SubscribeRole, only_sender: Option<u32>) -> Self {
let client_id = loop {
let id = rand::random::<u32>();
if id != 0 && id != CLIENT_ID { break id; }
};
Self {
discovery: Arc::new(Discovery::new()),
latest: Arc::new(LatestFrames::default()),
@@ -337,6 +342,7 @@ impl Receiver {
discovery_all: false,
}),
role: Mutex::new(role),
client_id,
camera,
stop: Arc::new(AtomicBool::new(false)),
}
@@ -354,6 +360,15 @@ impl Receiver {
) -> std::io::Result<(Self, UdpSocket)> {
let socket =
UdpSocket::bind(SocketAddrV4::new(Ipv4Addr::UNSPECIFIED, DEFAULT_PORT))?;
// Absorb short scheduler/NIC bursts without losing JPEG fragments. The
// reader still drains continuously; decoded video uses latest frames.
let socket_ref = socket2::SockRef::from(&socket);
if let Err(err) = socket_ref.set_recv_buffer_size(1024 * 1024) {
log::warn!("could not enlarge the video receive buffer: {err}");
}
if let Ok(bytes) = socket_ref.recv_buffer_size() {
log::info!("video UDP receive buffer: {bytes} bytes");
}
socket.set_read_timeout(Some(Duration::from_millis(100)))?;
let broadcast = socket.set_broadcast(true).is_ok();
@@ -473,11 +488,17 @@ impl Receiver {
.entry(id)
.or_insert_with(|| FrameAssembler::new(FRAME_TIMEOUT));
match assembler.push(frag) {
Ok(frame) => {
drop(ingest);
let dropped_before = assembler.dropped_frames();
let result = assembler.push(frag);
let dropped = assembler.dropped_frames() - dropped_before;
drop(ingest);
self.discovery.count(id, |s| {
s.datagrams += 1;
s.frames_dropped += dropped;
});
self.discovery.count(id, |s| s.datagrams += 1);
match result {
Ok(frame) => {
self.discovery
.note_frame(id, frame.width, frame.height, frame.data.len());
@@ -489,17 +510,7 @@ impl Receiver {
sink(frame);
}
}
Err(protocol::DropReason::Superseded) => {
// 被更新的帧追上了。这不是丢帧:半截的那张只是被超越,在健康链路上
// 这是帧被放弃的**正常**方式。
self.discovery.count(id, |s| s.datagrams += 1);
}
Err(protocol::DropReason::TimedOut) => {
self.discovery.count(id, |s| {
s.datagrams += 1;
s.frames_dropped += 1;
});
}
Err(_) => {}
}
}
@@ -521,6 +532,25 @@ impl Receiver {
}
}
/// 全局广播用于发现;已知板子还需按 IP 续订,避免广播被 VPN / 虚拟网卡接走。
fn subscription_targets(&self) -> Vec<SocketAddrV4> {
let mut targets = vec![SocketAddrV4::new(self.camera, DEFAULT_PORT)];
let only_sender = self.sender_limit();
for camera in self.discovery.online_cameras() {
if only_sender.is_some_and(|id| camera.id != id) {
continue;
}
// Hello 的来源端口不一定是订阅监听端口;协议固定使用 DEFAULT_PORT。
let target = SocketAddrV4::new(*camera.addr.ip(), DEFAULT_PORT);
if !targets.contains(&target) {
targets.push(target);
}
}
targets
}
/// 运行接收循环,直到 [`Receiver::shutdown`]。
///
/// 阻塞,所以放在单独的线程上。
@@ -543,11 +573,14 @@ impl Receiver {
if last_subscribe.elapsed() >= SUBSCRIBE_INTERVAL {
let subscribe = Subscribe {
role: self.role(),
client_id: CLIENT_ID,
client_id: self.client_id,
};
if let Err(err) = socket.send_to(&subscribe.encode(), target) {
log::warn!("could not subscribe at {target}: {err}");
let encoded = subscribe.encode();
for target in self.subscription_targets() {
if let Err(err) = socket.send_to(&encoded, target) {
log::warn!("could not subscribe at {target}: {err}");
}
}
last_subscribe = Instant::now();
}
@@ -639,6 +672,48 @@ mod tests {
rx.datagram(&hello.encode(MsgType::Hello), from(from_ip));
}
#[test]
fn discovered_boards_are_subscribed_directly_when_broadcast_uses_another_interface() {
let rx = Receiver::new(Ipv4Addr::BROADCAST, SubscribeRole::Referee, None);
let hello = Hello { sender_id: 1, firmware: 7 };
// Hello 可从其它端口发来,订阅仍需发往板子的固定监听端口。
rx.datagram(&hello.encode(MsgType::Hello), "192.168.1.139:12345".parse().unwrap());
feed_hello(&rx, 2, [192, 168, 1, 140]);
assert_eq!(rx.subscription_targets(), vec![
"255.255.255.255:5600".parse().unwrap(),
"192.168.1.139:5600".parse().unwrap(),
"192.168.1.140:5600".parse().unwrap(),
]);
// DHCP 地址变化后只使用新地址。
feed_hello(&rx, 1, [192, 168, 1, 141]);
assert_eq!(rx.subscription_targets(), vec![
"255.255.255.255:5600".parse().unwrap(),
"192.168.1.141:5600".parse().unwrap(),
"192.168.1.140:5600".parse().unwrap(),
]);
}
#[test]
fn discovery_of_other_teams_does_not_directly_subscribe_to_their_video() {
let rx = Receiver::new(Ipv4Addr::BROADCAST, SubscribeRole::Team, Some(1));
rx.set_discovery_all(true);
feed_hello(&rx, 1, [192, 168, 1, 10]);
feed_hello(&rx, 2, [192, 168, 1, 20]);
assert_eq!(rx.discovery().cameras().len(), 2);
assert_eq!(rx.subscription_targets(), vec![
"255.255.255.255:5600".parse().unwrap(),
"192.168.1.10:5600".parse().unwrap(),
]);
rx.restrict_to(Some(2));
assert_eq!(rx.subscription_targets(), vec![
"255.255.255.255:5600".parse().unwrap(),
"192.168.1.20:5600".parse().unwrap(),
]);
}
#[test]
fn delivers_the_latest_frame_of_the_bound_sender_only() {
let rx = Receiver::new(Ipv4Addr::UNSPECIFIED, SubscribeRole::Team, Some(1));
@@ -761,4 +836,21 @@ mod tests {
assert!(rx.discovery().cameras().is_empty());
}
#[test]
fn loss_statistics_include_superseded_frames_and_entirely_missing_ids() {
let rx = Receiver::new(Ipv4Addr::UNSPECIFIED, SubscribeRole::Referee, None);
feed_hello(&rx, 1, [192, 168, 1, 10]);
feed_frame(&rx, 1, 1, [192, 168, 1, 10]);
let partial = fragment_frame(1, 2, &jpeg(), 640, 480);
rx.datagram(&partial[0].encode(), from([192, 168, 1, 10]));
assert_eq!(rx.discovery().stats(1).frames_dropped, 0, "normal pending fragments are not drops");
feed_frame(&rx, 1, 4, [192, 168, 1, 10]);
let stats = rx.discovery().stats(1);
assert_eq!(stats.frames_shown, 2);
assert_eq!(stats.frames_dropped, 2, "partial 2 and entirely missing 3");
assert_eq!(stats.loss_percent(), 50.0);
for frag in partial { rx.datagram(&frag.encode(), from([192, 168, 1, 10])); }
assert_eq!(rx.discovery().stats(1).frames_dropped, 2);
}
}
+3 -1
View File
@@ -10,7 +10,9 @@
"preview": "vite preview",
"tauri": "tauri",
"app": "tauri dev",
"check": "tsc --noEmit"
"check": "tsc --noEmit",
"test:stream": "node --test scripts/test-stream.mjs",
"test:latency": "node --test scripts/test-latency-marker.mjs"
},
"dependencies": {
"@tauri-apps/api": "^2",
+146
View File
@@ -0,0 +1,146 @@
// Observe the native client's local video channel without adding a UDP peer.
// Read the token internally from a deliberately enabled debug log; never print it.
import { readFile, writeFile, appendFile, mkdir, access } from 'node:fs/promises';
import { dirname } from 'node:path';
const args = Object.fromEntries(process.argv.slice(2).reduce((pairs, value, i, all) => {
if (i % 2 === 0) pairs.push([value.replace(/^--/, ''), all[i+1]]);
return pairs;
}, []));
if (!args.log || !args.out) throw new Error('Use --log <backend-debug-log> --out <output-prefix> [--seconds 1800] [--warmup 30]');
const seconds = Number(args.seconds ?? 300);
const warmup = Number(args.warmup ?? 10);
const sampleEveryMs = 1000*Number(args['sample-every'] ?? 0);
const camera = Number(args.camera ?? '0x3D4D8A9A');
await mkdir(dirname(args.out), { recursive: true });
const log = await readFile(args.log, 'utf8');
const url = [...log.matchAll(/ws:\/\/127\.0\.0\.1:\d+\/\?token=[A-Za-z0-9_-]+/g)].at(-1)?.[0];
if (!url) throw new Error('No local video-channel URL in the debug log');
const ws = new WebSocket(url);
ws.binaryType = 'arraybuffer';
let latest = null, demo = true, diagnostics = false, clients = 0, connected = false;
let binary = 0, invalid = 0, lastArrival = null, gaps = [], bytes = 0;
let firstFrame = null;
let sampleCount = 0, nextSampleAt = 0;
const allGaps = [], windows = [];
let measurementStarted = null, previousAt = null, previousStats = null;
let windowBinary = 0, windowInvalid = 0, windowBytes = 0;
let totalFrames = 0, totalDropped = 0, counterResets = 0, socketCloses = 0;
let offlineSnapshots = 0;
let diagnosticsUnavailableSnapshots = 0;
let finished = false;
ws.addEventListener('close', () => { connected = false; if (!finished) socketCloses++; });
ws.addEventListener('message', event => {
if (typeof event.data === 'string') {
const msg = JSON.parse(event.data);
if (msg.t === 'snapshot') {
latest = msg.stats.find(stat => stat.camera === camera) ?? null;
demo = msg.demo;
diagnostics = msg.diagnostics;
clients = msg.clients;
if (measurementStarted !== null && (!latest || !latest.online)) offlineSnapshots++;
if (measurementStarted !== null && !diagnostics) diagnosticsUnavailableSnapshots++;
}
return;
}
const data = Buffer.from(event.data);
if (data.length < 16 || data.readUInt32LE(0) !== camera) return;
if (measurementStarted === null) return;
const valid = data.length === data.readUInt32LE(12)+16 && data[16] === 255 && data[17] === 216 && data.at(-2) === 255 && data.at(-1) === 217;
if (!valid) { invalid++; windowInvalid++; return; }
const now = performance.now();
if (lastArrival !== null) { gaps.push(now-lastArrival); allGaps.push(now-lastArrival); }
lastArrival = now;
binary++; windowBinary++;
bytes += data.length-16; windowBytes += data.length-16;
if (firstFrame === null) {
firstFrame = Buffer.from(data.subarray(16));
void writeFile(`${args.out}.first.jpg`, firstFrame).catch(error => console.error('Could not save sample JPEG:', error.message));
}
if (sampleEveryMs > 0 && now >= nextSampleAt) {
nextSampleAt = now+sampleEveryMs;
const path = `${args.out}.sample-${String(++sampleCount).padStart(4,'0')}.jpg`;
void writeFile(path, data.subarray(16)).catch(error => console.error('Could not save diagnostic JPEG:', error.message));
}
});
await Promise.race([
new Promise((resolve, reject) => { ws.addEventListener('open', () => { connected = true; resolve(); }, { once:true }); ws.addEventListener('error', reject, { once:true }); }),
new Promise((_, reject) => setTimeout(() => reject(new Error('Video channel open timeout')), 15_000)),
]);
await new Promise(resolve => setTimeout(resolve, warmup*1000));
if (demo || !latest?.online || !latest?.hasFrames || clients < 2) throw new Error('Native GUI and online real hardware video are required before starting the soak');
if (!diagnostics) throw new Error('Receiver counters require referee diagnostics; team release mode hides them');
measurementStarted = previousAt = performance.now();
previousStats = latest;
await writeFile(`${args.out}.windows.ndjson`, '');
function percentile(values, p) {
if (!values.length) return null;
const sorted = [...values].sort((a,b) => a-b);
return Number(sorted[Math.floor((sorted.length-1)*p)].toFixed(3));
}
async function sample() {
const now = performance.now();
const elapsed = (now-previousAt)/1000;
let complete = 0, dropped = 0;
if (latest && previousStats) {
if (latest.framesShown < previousStats.framesShown || latest.framesDropped < previousStats.framesDropped) counterResets++;
else { complete = latest.framesShown-previousStats.framesShown; dropped = latest.framesDropped-previousStats.framesDropped; }
}
totalFrames += complete; totalDropped += dropped;
const row = {
utc:new Date().toISOString(), elapsedSeconds:(now-measurementStarted)/1000, windowSeconds:elapsed,
completeFrames:complete, framesDropped:dropped, receiverFps:complete/elapsed,
frameLossPercent:100*dropped/Math.max(1,complete+dropped),
binaryFrames:windowBinary, channelFps:windowBinary/elapsed, invalidJpegs:windowInvalid,
meanJpegBytes:windowBinary ? windowBytes/windowBinary : 0,
gapP95Ms:percentile(gaps,.95), gapMaxMs:Math.max(0,...gaps, lastArrival===null ? (now-measurementStarted) : now-lastArrival),
cameraOnline:latest?.online ?? false, frontendConnected:clients>=2, observerConnected:connected,
};
windows.push(row);
await appendFile(`${args.out}.windows.ndjson`, JSON.stringify(row)+'\n');
await writeFile(`${args.out}.latest.json`, JSON.stringify(row,null,2)+'\n');
console.log(JSON.stringify(row));
previousAt = now;
previousStats = latest;
windowBinary = windowInvalid = windowBytes = 0;
gaps = [];
}
let stoppedByRequest = false;
while ((performance.now()-measurementStarted)/1000 < seconds) {
if (args['stop-file'] && await access(args['stop-file']).then(() => true, () => false)) {
stoppedByRequest = true;
break;
}
const remaining = seconds-(performance.now()-measurementStarted)/1000;
await new Promise(resolve => setTimeout(resolve, Math.min(1,remaining)*1000));
if (performance.now()-previousAt >= 60_000 || (performance.now()-measurementStarted)/1000 >= seconds) await sample();
}
if (stoppedByRequest && performance.now()-previousAt > 1_000) await sample();
finished = true;
ws.close();
const actualSeconds = (performance.now()-measurementStarted)/1000;
const receiverMetricsValid = counterResets === 0 && diagnosticsUnavailableSnapshots === 0;
const summary = {
actualSeconds, requestedSeconds:seconds, stoppedByRequest, warmupSeconds:warmup, camera, demo, windows, binaryFrames:binary,
invalidJpegs:invalid, receiverMetricsValid,
completeFrames:receiverMetricsValid ? totalFrames : null, framesDropped:receiverMetricsValid ? totalDropped : null,
rawWindowCountedCompleteFrames:totalFrames, rawWindowCountedDroppedFrames:totalDropped,
receiverFps:receiverMetricsValid ? totalFrames/actualSeconds : null, channelFps:binary/actualSeconds,
frameLossPercent:receiverMetricsValid ? 100*totalDropped/Math.max(1,totalFrames+totalDropped) : null,
gapP95Ms:percentile(allGaps,.95), gapMaxMs:Math.max(0,...allGaps),
meanJpegBytes:binary ? bytes/binary : 0, counterResets, socketCloses,
offlineSnapshots,
diagnosticsUnavailableSnapshots,
diagnosticSamplesSaved:sampleCount,
allWindowsOnline:windows.every(window => window.cameraOnline),
zeroVideoWindows:windows.filter(window => window.binaryFrames === 0).length,
noVideoSecondsAtEnd:lastArrival === null ? actualSeconds : (performance.now()-lastArrival)/1000,
maxObservedSilenceMs:Math.max(0,...windows.map(window => window.gapMaxMs)),
frontendConnected:clients>=2,
measurementScope:'Native receiver and local JPEG WebSocket channel; not optical display latency',
};
await writeFile(`${args.out}.summary.json`, JSON.stringify(summary,null,2)+'\n');
if (firstFrame) await writeFile(`${args.out}.first.jpg`, firstFrame);
console.log(JSON.stringify({ ...summary, windows:windows.length }));
if (demo || !binary || invalid || socketCloses || counterResets || offlineSnapshots || diagnosticsUnavailableSnapshots) process.exitCode = 1;
+50
View File
@@ -0,0 +1,50 @@
import {readFileSync} from 'node:fs';
import assert from 'node:assert/strict';
import {test} from 'node:test';
import ts from 'typescript';
const source=readFileSync(new URL('../src/lib/latency-marker.ts',import.meta.url),'utf8');
const code=ts.transpileModule(source,{compilerOptions:{module:ts.ModuleKind.CommonJS,target:ts.ScriptTarget.ES2022}}).outputText;
const exports={};new Function('exports',code)(exports);
const {drawMarker,decodeMarker}=exports;
function image(counter,w=320,h=240) {
const pixels=new Uint8ClampedArray(w*h*4);
const ctx={canvas:{width:w,height:h},fillStyle:'',fillRect(x,y,width,height){
const c=this.fillStyle.startsWith('rgb(')?this.fillStyle.slice(4,-1).split(',').map(Number):
this.fillStyle==='#fff'?[255,255,255]:this.fillStyle==='#000'?[0,0,0]:[60,60,60];
for(let yy=Math.max(0,Math.ceil(y));yy<Math.min(h,y+height);yy++)
for(let xx=Math.max(0,Math.ceil(x));xx<Math.min(w,x+width);xx++)pixels.set([...c,255],(yy*w+xx)*4);
}};
drawMarker(ctx,counter);return pixels;
}
test('screen IDs decode at Gray-code boundaries and across counter wrap',()=>{
for(const id of [0,1,2,3,63,127,1023,2047,2048,4094,4095,4096]) {
assert.equal(decodeMarker(image(id),320,240)?.counter,id&4095);
}
});
test('camera rotation and reflection preserve the observed ID',()=>{
const original=image(1371),rotated=new Uint8ClampedArray(original.length),flipped=new Uint8ClampedArray(original.length);
for(let y=0;y<240;y++)for(let x=0;x<320;x++){
rotated.set(original.subarray((y*320+x)*4,(y*320+x)*4+4),(x*240+239-y)*4);
flipped.set(original.subarray((y*320+x)*4,(y*320+x)*4+4),(y*320+319-x)*4);
}
assert.equal(decodeMarker(rotated,240,320)?.counter,1371);
assert.equal(decodeMarker(flipped,320,240)?.counter,1371);
});
test('missing fiducials and ambiguous rolling transitions do not invent latency samples',()=>{
assert.equal(decodeMarker(new Uint8ClampedArray(320*240*4),320,240),null);
const pixels=image(123);
for(let y=90;y<152;y++)for(let x=44;x<276;x++)pixels.set([128,128,128,255],(y*320+x)*4);
assert.equal(decodeMarker(pixels,320,240),null);
});
test('washed screen primaries survive without mistaking a blue-tinted background',()=>{
const pixels=image(1371);
for(let i=0;i<pixels.length;i+=4){
const r=pixels[i],g=pixels[i+1],b=pixels[i+2];
if(r===32&&g===255&&b===32)pixels.set([190,255,208,255],i);
else if(r===32&&g===96&&b===255)pixels.set([78,205,255,255],i);
else if(r===255&&g===32&&b===255)pixels.set([255,112,255,255],i);
else if(r===60&&g===60&&b===60)pixels.set([165,181,225,255],i);
}
assert.equal(decodeMarker(pixels,320,240)?.counter,1371);
});
+41
View File
@@ -0,0 +1,41 @@
// Exercise the production StreamClient against native hardware video, while
// deliberately stalling only this auxiliary local consumer. Adds no UDP peer.
// Bitmap decoding is stubbed here; actual JPEG decoding is checked separately.
import {readFileSync,writeFileSync} from 'node:fs';
import assert from 'node:assert/strict';
import ts from 'typescript';
const [logPath,outPath]=process.argv.slice(2);
if(!logPath||!outPath)throw new Error('Use <private native debug log> <result JSON>');
const log=readFileSync(logPath,'utf8');
const url=[...log.matchAll(/ws:\/\/127\.0\.0\.1:\d+\/\?token=[A-Za-z0-9_-]+/g)].at(-1)?.[0];
if(!url)throw new Error('No local video URL in the private debug log');
const source=readFileSync(new URL('../src/lib/stream.ts',import.meta.url),'utf8');
const compiled=ts.transpileModule(source,{compilerOptions:{module:ts.ModuleKind.CommonJS,target:ts.ScriptTarget.ES2022}}).outputText;
const exports={};new Function('require','exports',compiled)(()=>({FRAME_HEADER_LEN:16}),exports);
globalThis.window={setTimeout,clearTimeout};
globalThis.createImageBitmap=async()=>({close(){}});
const rx=new exports.StreamClient({onMessage(){},onConnection(){}});
const delay=ms=>new Promise(resolve=>setTimeout(resolve,ms));
try {
rx.connect(url);
const deadline=performance.now()+10_000;
while(rx.counters.frames<10&&performance.now()<deadline)await delay(30);
assert.ok(rx.counters.frames>=10,'Real native JPEG video did not arrive');
const camera=[...rx.frames.keys()][0];
const before=rx.latest(camera)?.frameId;
const staleBefore=rx.counters.staleFrames;
const blockedAt=performance.now();
while(performance.now()-blockedAt<350) { /* synthetic consumer stall */ }
await delay(700);
const latest=rx.latest(camera);
assert.ok(rx.counters.staleFrames>staleBefore,'Stale native-timestamped frames were not rejected');
assert.ok(latest&&latest.frameId!==before,'The consumer did not recover to new video');
assert.ok(performance.now()-latest.receivedAt<150,'Recovered image is still stale locally');
const result={passed:true,stall_ms:350,stale_frames_rejected:rx.counters.staleFrames-staleBefore,
frame_before:before,frame_after:latest.frameId,frames_received:rx.counters.frames,
additional_udp_subscribers:0,jpeg_decoder_stubbed:true,
scope:'Production StreamClient over real native WebSocket; tests queued-frame age and recovery, not physical display latency'};
writeFileSync(outPath,JSON.stringify(result,null,2)+'\n');
console.log(JSON.stringify(result));
} finally {rx.close();}
+143
View File
@@ -0,0 +1,143 @@
import { readFileSync } from 'node:fs';
import assert from 'node:assert/strict';
import { test } from 'node:test';
import ts from 'typescript';
const source = readFileSync(new URL('../src/lib/stream.ts', import.meta.url), 'utf8');
const compiled = ts.transpileModule(source, { compilerOptions: { module: ts.ModuleKind.CommonJS, target: ts.ScriptTarget.ES2022 } }).outputText;
const exports = {};
new Function('require', 'exports', compiled)(name => {
assert.equal(name, './types');
return { FRAME_HEADER_LEN: 16 };
}, exports);
const { StreamClient } = exports;
globalThis.window = { clearTimeout, setTimeout };
const tick = () => new Promise(resolve => setImmediate(resolve));
const bitmap = () => ({ closed: false, close() { this.closed = true; } });
function frame(id, camera = 42) {
const bytes = new ArrayBuffer(20);
const header = new DataView(bytes);
header.setUint32(0, camera, true);
header.setUint32(4, id, true);
header.setUint16(8, 640, true);
header.setUint16(10, 480, true);
header.setUint32(12, 4, true);
return bytes;
}
function client(now, wallNow) { return new StreamClient({ onMessage() {}, onConnection() {} },now,wallNow); }
function deferredDecoder() {
const requests = [];
globalThis.createImageBitmap = () => new Promise((resolve, reject) => requests.push({ resolve, reject }));
return requests;
}
test('a slow decoder keeps only the latest pending frame for each camera', async () => {
const requests = deferredDecoder();
const rx = client();
const finished = rx.onBinary(frame(1));
for (let id = 2; id <= 100; id++) await rx.onBinary(frame(id));
assert.equal(requests.length, 1);
assert.equal(rx.pending.size, 1);
const first = bitmap();
requests[0].resolve(first);
await tick();
assert.equal(requests.length, 2);
requests[1].resolve(bitmap());
await finished;
assert.equal(rx.latest(42).frameId, 100);
assert.equal(rx.counters.skippedDecodes, 98);
assert.equal(first.closed, true);
rx.close();
});
test('two cameras decode independently without queues growing behind either', async () => {
const requests = deferredDecoder();
const rx = client();
const a = rx.onBinary(frame(1, 1));
const b = rx.onBinary(frame(2, 2));
assert.equal(requests.length, 2);
requests[1].resolve(bitmap());
await b;
assert.equal(rx.latest(2).frameId, 2);
assert.equal(rx.latest(1), undefined);
requests[0].resolve(bitmap());
await a;
rx.close();
});
test('closing during decode releases the late bitmap and discards pending work', async () => {
const requests = deferredDecoder();
const rx = client();
const work = rx.onBinary(frame(1));
await rx.onBinary(frame(2));
rx.close();
const late = bitmap();
requests[0].resolve(late);
await work;
assert.equal(late.closed, true);
assert.equal(requests.length, 1);
assert.equal(rx.latest(42), undefined);
});
test('a corrupt JPEG does not stop the next frame from being decoded', async () => {
const requests = deferredDecoder();
const rx = client();
const work = rx.onBinary(frame(1));
await rx.onBinary(frame(2));
requests[0].reject(new Error('bad JPEG'));
await tick();
requests[1].resolve(bitmap());
await work;
assert.equal(rx.counters.decodeFailures, 1);
assert.equal(rx.latest(42).frameId, 2);
rx.close();
});
test('late frames are rejected but a reboot after silence can restart the counter', () => {
const rx = client();
rx.injectFrame(42, 100, 640, 480, bitmap());
const old = bitmap();
rx.injectFrame(42, 99, 640, 480, old);
assert.equal(old.closed, true);
assert.equal(rx.latest(42).frameId, 100);
rx.latest(42).receivedAt -= 2000;
rx.injectFrame(42, 1, 640, 480, bitmap());
assert.equal(rx.latest(42).frameId, 1);
rx.close();
});
test('a WebView backlog is discarded before JPEG decoding using native host timing', async () => {
const requests = deferredDecoder();
const rx = client(() => 0, () => 2000);
rx.onTiming({camera:42,frameId:1,publishedUnixMs:1500});
await rx.onBinary(frame(1));
assert.equal(requests.length,0);
assert.equal(rx.counters.staleFrames,1);
rx.onTiming({camera:42,frameId:2,publishedUnixMs:2000});
const work = rx.onBinary(frame(2));
requests[0].resolve(bitmap());await work;
assert.equal(rx.latest(42).frameId,2);
rx.close();
});
test('a blocked decoder releases its old result and resumes with the fresh pending frame', async () => {
let clock=0;
const requests=deferredDecoder(),rx=client(()=>clock);
const work=rx.onBinary(frame(1));
clock=110;await rx.onBinary(frame(2));
const old=bitmap();requests[0].resolve(old);await tick();
assert.equal(old.closed,true);
assert.equal(rx.latest(42),undefined);
requests[1].resolve(bitmap());await work;
assert.equal(rx.latest(42).frameId,2);
assert.equal(rx.counters.staleFrames,1);rx.close();
});
test('pending frames that expired during a decoder stall never reach the decoder', async () => {
let clock=0;
const requests=deferredDecoder(),rx=client(()=>clock);
const work=rx.onBinary(frame(1));await rx.onBinary(frame(2));
clock=200;const old=bitmap();requests[0].resolve(old);await work;
assert.equal(old.closed,true);assert.equal(requests.length,1);
assert.equal(rx.counters.staleFrames,2);assert.equal(rx.latest(42),undefined);rx.close();
});
+1
View File
@@ -476,6 +476,7 @@ dependencies = [
"protocol",
"rand 0.9.5",
"serde",
"socket2",
"toml 0.9.12+spec-1.1.0",
]
+1 -5
View File
@@ -485,11 +485,7 @@ pub fn clear_config(password: Option<String>, state: Shared<'_>) -> Result<Strin
/// 退出程序。用户端要密码——现场"随手关掉"也是一种需要挡住的误操作。
#[tauri::command]
pub fn quit_app(
password: Option<String>,
app: AppHandle,
state: Shared<'_>,
) -> Result<(), String> {
pub fn quit_app(password: Option<String>, app: AppHandle, state: Shared<'_>) -> Result<(), String> {
state.require_unlocked(password.as_deref())?;
// 先把录制收尾,否则 ffmpeg 会留下一个没有索引的 mp4。
+51 -18
View File
@@ -31,9 +31,7 @@ use std::time::Duration;
use protocol::{fragment_frame, ArmorHealth, MsgType};
use crate::state::{
AppState, DEMO_ARMOR_A, DEMO_ARMOR_B, DEMO_CAMERA_A, DEMO_CAMERA_B,
};
use crate::state::{AppState, DEMO_ARMOR_A, DEMO_ARMOR_B, DEMO_CAMERA_A, DEMO_CAMERA_B};
/// 合成画面的尺寸。与实机一致(VGA),这样界面的缩放/比例在演示下也说得通。
const WIDTH: u32 = 640;
@@ -83,9 +81,7 @@ fn spawn_camera(state: Arc<AppState>, camera: u32, team: u8) {
// 板子每 2 秒广播一次在线状态,与固件一致。
if last_hello.elapsed() >= Duration::from_secs(2) {
state
.receiver
.datagram(&hello.encode(MsgType::Hello), from);
state.receiver.datagram(&hello.encode(MsgType::Hello), from);
last_hello = std::time::Instant::now();
}
@@ -140,10 +136,8 @@ fn spawn_armor(state: Arc<AppState>) {
hp[slot] = hp[slot].saturating_sub(damage);
}
let from = SocketAddr::V4(SocketAddrV4::new(
Ipv4Addr::new(127, 0, 0, 1),
*port,
));
let from =
SocketAddr::V4(SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), *port));
// 主板一次广播四条,整车一条血所以 hp 相同、index 0..3。
for index in 0..4u8 {
@@ -215,10 +209,38 @@ fn render(camera: u32, team: u8, frame_id: u32) -> Vec<u8> {
// 中心的靶心:两个同心圆 + 四臂。瞄准器偏移校准时对着它看。
ring(&mut img, cx, cy, 60, image::Rgb([200, 200, 210]));
ring(&mut img, cx, cy, 28, image::Rgb([150, 150, 160]));
arm(&mut img, cx - 90, cy, cx - 40, cy, image::Rgb([120, 120, 130]));
arm(&mut img, cx + 40, cy, cx + 90, cy, image::Rgb([120, 120, 130]));
arm(&mut img, cx, cy - 90, cx, cy - 40, image::Rgb([120, 120, 130]));
arm(&mut img, cx, cy + 40, cx, cy + 90, image::Rgb([120, 120, 130]));
arm(
&mut img,
cx - 90,
cy,
cx - 40,
cy,
image::Rgb([120, 120, 130]),
);
arm(
&mut img,
cx + 40,
cy,
cx + 90,
cy,
image::Rgb([120, 120, 130]),
);
arm(
&mut img,
cx,
cy - 90,
cx,
cy - 40,
image::Rgb([120, 120, 130]),
);
arm(
&mut img,
cx,
cy + 40,
cx,
cy + 90,
image::Rgb([120, 120, 130]),
);
// 二进制帧号:16 个方块,最左边是低位。
for bit in 0..16i32 {
@@ -241,7 +263,14 @@ fn render(camera: u32, team: u8, frame_id: u32) -> Vec<u8> {
// 底部随时间右移的指示条:截图里能判断这一帧是不是较新的。
let bar = ((t * 60.0) as i32).rem_euclid(WIDTH as i32 - 80);
fill_rect(&mut img, bar, HEIGHT as i32 - 24, 60, 8, image::Rgb([90, 200, 255]));
fill_rect(
&mut img,
bar,
HEIGHT as i32 - 24,
60,
8,
image::Rgb([90, 200, 255]),
);
// 摄像头 id 的最低位用一个角标表示,进一步区分两路。
let corner = if camera == DEMO_CAMERA_A { 0 } else { 1 };
@@ -341,8 +370,8 @@ mod tests {
assert!(jpeg.starts_with(&[0xFF, 0xD8]), "JPEG SOI");
assert!(jpeg.ends_with(&[0xFF, 0xD9]), "JPEG EOI");
let decoded = image::load_from_memory_with_format(&jpeg, image::ImageFormat::Jpeg)
.expect("decodes");
let decoded =
image::load_from_memory_with_format(&jpeg, image::ImageFormat::Jpeg).expect("decodes");
assert_eq!((decoded.width(), decoded.height()), (WIDTH, HEIGHT));
}
@@ -366,7 +395,11 @@ mod tests {
fn a_frame_is_big_enough_to_be_a_realistic_load() {
// 太小说明画的内容退化了(比如全是纯色),那分片/带宽的验证就失去意义。
let jpeg = render(DEMO_CAMERA_A, 0, 1);
assert!(jpeg.len() > 3_000, "suspiciously small: {} bytes", jpeg.len());
assert!(
jpeg.len() > 3_000,
"suspiciously small: {} bytes",
jpeg.len()
);
assert!(
jpeg.len() < 200_000,
"suspiciously large: {} bytes",
+77
View File
@@ -0,0 +1,77 @@
//! Opt-in, local-only calibration reports. No images, tokens or user config.
use serde::Deserialize;
use std::{
fs::{File, OpenOptions},
io::Write,
path::PathBuf,
sync::Mutex,
};
pub struct Recorder {
output: Mutex<Option<File>>,
}
impl Recorder {
pub fn new(path: Option<PathBuf>) -> std::io::Result<Self> {
let output = if let Some(path) = path {
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
Some(OpenOptions::new().write(true).create_new(true).open(path)?)
} else {
None
};
Ok(Self {
output: Mutex::new(output),
})
}
}
#[derive(Deserialize, serde::Serialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct Sample {
frame_id: u32,
counter: u16,
source_draw_ms: f64,
draw_ms: f64,
next_raf_ms: f64,
lower_ms: f64,
upper_ms: f64,
decode_to_draw_ms: f64,
analysis_ms: f64,
}
#[tauri::command]
pub fn latency_enabled(recorder: tauri::State<'_, Recorder>) -> bool {
recorder
.output
.lock()
.expect("latency output mutex")
.is_some()
}
#[tauri::command]
pub fn latency_sample(sample: Sample, recorder: tauri::State<'_, Recorder>) -> Result<(), String> {
if ![
sample.source_draw_ms,
sample.draw_ms,
sample.next_raf_ms,
sample.lower_ms,
sample.upper_ms,
sample.decode_to_draw_ms,
sample.analysis_ms,
]
.iter()
.all(|v| v.is_finite())
|| sample.lower_ms < 0.0
|| sample.upper_ms < sample.lower_ms
|| sample.upper_ms > 5000.0
{
return Err("Invalid latency sample".into());
}
let mut guard = recorder.output.lock().map_err(|_| "latency output mutex")?;
let out = guard.as_mut().ok_or("Latency calibration is disabled")?;
serde_json::to_writer(&mut *out, &sample).map_err(|e| e.to_string())?;
out.write_all(b"\n")
.and_then(|_| out.flush())
.map_err(|e| e.to_string())
}
+18 -3
View File
@@ -23,6 +23,7 @@
mod commands;
mod demo;
mod latency;
mod recording;
mod server;
mod state;
@@ -42,6 +43,7 @@ const USAGE: &str = "\
--role <team|referee> 身份。默认看配置文件;没有配置文件时是 referee
--config <路径> 配置文件位置
--latency-report <路径> 打开裁判端屏幕标记校准,写入新的本地 NDJSON 报告
--demo 演示模式:画面与血量都是本地合成的
--list 打印发现的摄像头与装甲板后退出
-h, --help 显示这段帮助
@@ -78,6 +80,7 @@ fn main() {
struct Args {
role: Option<Role>,
config: Option<PathBuf>,
latency_report: Option<PathBuf>,
demo: bool,
list: bool,
help: bool,
@@ -104,6 +107,9 @@ impl Args {
"--config" => {
args.config = it.next().map(PathBuf::from);
}
"--latency-report" => {
args.latency_report = it.next().map(PathBuf::from);
}
other => {
// 见模块文档:macOS 的 .app 启动参数与 shell 里的笔误都可能出现在这里,
// 但报错的代价(窗口起不来)远大于收益。
@@ -124,7 +130,10 @@ fn run() -> Result<(), Box<dyn Error>> {
return Ok(());
}
let config_path = args.config.clone().unwrap_or_else(config::default_config_path);
let config_path = args
.config
.clone()
.unwrap_or_else(config::default_config_path);
// 首次启动时导入旧客户端的配置,并把旧文件改名成 .bak。
let (mut config, notes) = config::load_with_migration(&config_path)?;
@@ -134,6 +143,10 @@ fn run() -> Result<(), Box<dyn Error>> {
if let Some(role) = args.role {
config.role = role;
}
if args.latency_report.is_some() && config.role != Role::Referee {
return Err("Latency calibration requires a referee test configuration".into());
}
let latency_recorder = latency::Recorder::new(args.latency_report.clone())?;
for note in &notes {
log::info!("{note}");
@@ -150,8 +163,7 @@ fn run() -> Result<(), Box<dyn Error>> {
config_path.display()
);
let (state, video_socket, armor_socket) =
AppState::new(config_path, config, notes, args.demo)?;
let (state, video_socket, armor_socket) = AppState::new(config_path, config, notes, args.demo)?;
// 视频端口绑不上时没有 socket,接收线程也就没有意义——界面会显示那条说明。
if let Some(socket) = video_socket {
@@ -219,7 +231,10 @@ fn run() -> Result<(), Box<dyn Error>> {
let app = tauri::Builder::default()
.plugin(tauri_plugin_opener::init())
.manage(Arc::clone(&state))
.manage(latency_recorder)
.invoke_handler(tauri::generate_handler![
latency::latency_enabled,
latency::latency_sample,
commands::stream_info,
commands::get_snapshot,
commands::get_config,
+39 -19
View File
@@ -99,9 +99,7 @@ impl Recorder {
.stderr(Stdio::piped())
.spawn()
.with_context(|| {
format!(
"could not start ffmpeg ({ffmpeg}); 用 [recording] ffmpeg 指定它的路径"
)
format!("could not start ffmpeg ({ffmpeg}); 用 [recording] ffmpeg 指定它的路径")
})?;
let stdin = child
@@ -124,8 +122,7 @@ impl Recorder {
.spawn(move || {
if let Err(err) = pump(&rx, stdin, &mut child, &frames_written) {
log::error!("recording {} stopped: {err:#}", path.display());
*error.lock().expect("rec error mutex poisoned") =
Some(format!("{err:#}"));
*error.lock().expect("rec error mutex poisoned") = Some(format!("{err:#}"));
}
})?
};
@@ -191,7 +188,10 @@ impl Recorder {
dto.active = false;
if let Some(error) = dto.error.as_deref() {
log::warn!("recording 0x{:08X} finished with an error: {error}", self.camera);
log::warn!(
"recording 0x{:08X} finished with an error: {error}",
self.camera
);
} else {
log::info!(
"recording 0x{:08X} stopped: {} frames -> {}",
@@ -291,7 +291,9 @@ fn pump(
let reason = child_stderr(child);
return Err(anyhow!(
"writing to ffmpeg failed: {err}{}",
reason.map(|r| format!(";ffmpeg 说:{r}")).unwrap_or_default()
reason
.map(|r| format!(";ffmpeg 说:{r}"))
.unwrap_or_default()
));
}
@@ -356,7 +358,11 @@ impl RecordingSet {
// 文件是它启动之后才创建的。先开新的话,新的一段会挑到与上一段相同的名字,然后被
// 它的 `-y` 覆盖掉——而上一段刚要收尾。先停就能保证上一个文件已经落地,`exists()`
// 于是是可信的。
let previous = self.inner.lock().expect("recording mutex poisoned").remove(&camera);
let previous = self
.inner
.lock()
.expect("recording mutex poisoned")
.remove(&camera);
if let Some(previous) = previous {
log::warn!("0x{camera:08X} was already recording; the previous take is being closed");
@@ -399,7 +405,12 @@ impl RecordingSet {
/// 把一帧喂给对应那一路的录制器。在 UDP 接收线程里被调用,所以必须快。
pub fn feed(&self, frame: &Arc<Frame>) {
// 先快速判断有没有在录,避免每帧都抢锁。
if self.inner.lock().expect("recording mutex poisoned").is_empty() {
if self
.inner
.lock()
.expect("recording mutex poisoned")
.is_empty()
{
return;
}
@@ -498,8 +509,7 @@ mod tests {
}
let mut out = Vec::new();
let mut encoder =
image::codecs::jpeg::JpegEncoder::new_with_quality(&mut out, 80);
let mut encoder = image::codecs::jpeg::JpegEncoder::new_with_quality(&mut out, 80);
encoder.encode_image(&img).expect("encodes");
out
}
@@ -525,10 +535,8 @@ mod tests {
}
fn temp_dir(tag: &str) -> PathBuf {
let dir = std::env::temp_dir().join(format!(
"cam-desktop-rec-{tag}-{}",
std::process::id()
));
let dir =
std::env::temp_dir().join(format!("cam-desktop-rec-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).expect("creates the temp dir");
dir
@@ -585,7 +593,10 @@ mod tests {
let path = dto.path.expect("has a path");
let meta = std::fs::metadata(&path).expect("the file exists");
assert!(meta.len() > 0, "an empty container is a failure, not a take");
assert!(
meta.len() > 0,
"an empty container is a failure, not a take"
);
if let Some((frames, width, height)) = ffprobe_frames(&path) {
assert_eq!(frames, 20, "ffmpeg wrote every frame we fed it");
@@ -704,7 +715,10 @@ mod tests {
let decoded = image::open(&path).expect("the PNG is readable");
assert_eq!((decoded.width(), decoded.height()), (64, 48));
assert!(
path.file_name().unwrap().to_string_lossy().starts_with("shot-3D4D8A9A-"),
path.file_name()
.unwrap()
.to_string_lossy()
.starts_with("shot-3D4D8A9A-"),
"文件名要能看出是哪一路截的:{path:?}"
);
@@ -727,7 +741,10 @@ mod tests {
let forced = unique_path(&dir, "shot-00000001-fixed", "png");
std::fs::write(&forced, b"").unwrap();
let forced2 = unique_path(&dir, "shot-00000001-fixed", "png");
assert!(forced2.to_string_lossy().ends_with("-2.png"), "got {forced2:?}");
assert!(
forced2.to_string_lossy().ends_with("-2.png"),
"got {forced2:?}"
);
let _ = std::fs::remove_dir_all(&dir);
}
@@ -755,7 +772,10 @@ mod tests {
assert_eq!(names.len(), 2, "got {names:?}");
assert!(names.contains(&"a.mp4"));
assert!(names.contains(&"b.PNG"), "后缀大小写不该决定它是不是一张截图");
assert!(
names.contains(&"b.PNG"),
"后缀大小写不该决定它是不是一张截图"
);
assert_eq!(files[0].kind, "video");
let _ = std::fs::remove_dir_all(&dir);
+179 -15
View File
@@ -29,14 +29,15 @@
//! 小端。16 字节头,前端切掉它就能拿到一段完整的 JPEG。`sender_id` 让前端知道这一帧是
//! 哪块板子的——用户端只画绑定的那一路,裁判端按槽位分配画两路。
use std::collections::HashMap;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::Arc;
use std::time::Duration;
use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};
use futures_util::{SinkExt, StreamExt};
use serde::Serialize;
use tokio::net::{TcpListener, TcpStream};
use tokio::sync::broadcast;
use tokio::sync::{broadcast, watch};
use tokio_tungstenite::tungstenite::handshake::server::{ErrorResponse, Request, Response};
use tokio_tungstenite::tungstenite::http::StatusCode;
use tokio_tungstenite::tungstenite::{Bytes, Message};
@@ -46,24 +47,43 @@ use cam_core::snapshot::{HitDto, Snapshot};
/// 二进制帧头长度。
pub const FRAME_HEADER_LEN: usize = 16;
/// 广播通道的容量。
///
/// 满了之后 `send` 会挤掉最老的(接收慢的连接会拿到 `Lagged`)——对实时画面这正是想要
/// 的:落后 200 帧的客户端需要的是最新的那一帧,不是补齐历史。按每帧 16+22 kB 算,
/// 256 个槽位约 5.6 MB 上限。
/// Ordered control-message capacity. JPEGs use a separate latest-value channel.
pub const CHANNEL_CAPACITY: usize = 256;
const LOCAL_VIDEO_MAX_AGE: Duration = Duration::from_millis(100);
/// 向界面推送的消息。
pub enum ServerMsg {
/// 一帧原始 JPEG(已经拼好 16 字节头)。
///
/// 载荷用 `Bytes` 而不是 `Arc<Vec<u8>>`:它本身就是引用计数的缓冲,克隆是加一次
/// 引用计数,而且正好是 tungstenite 要的类型——每个连接一次多余的拷贝就此消失。
Binary(Bytes),
/// JSON 文本。
Text(String),
}
#[derive(Clone)]
pub struct VideoPacket {
frame_id: u32,
bytes: Bytes,
published_at: Instant,
published_unix_ms: u64,
}
pub type VideoFrames = HashMap<u32, VideoPacket>;
pub type VideoSender = watch::Sender<VideoFrames>;
/// Video uses latest-value semantics; control messages keep ordered delivery.
pub fn publish_video(tx: &VideoSender, frame: &protocol::Frame) {
let packet = VideoPacket {
frame_id: frame.frame_id,
bytes: encode_frame(frame).into(),
published_at: Instant::now(),
published_unix_ms: SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_millis() as u64,
};
tx.send_modify(|latest| {
latest.insert(frame.sender_id, packet);
});
}
/// JSON 文本消息的信封。
///
/// 用内部标签 `t` 而不是两个字段:前端一个 `switch (msg.t)` 就能分发,且不需要为每种
@@ -82,6 +102,14 @@ pub enum WireMsg {
/// 写成带字段的结构体而不是 `Note(String)`:serde 的**内部标签**(`#[serde(tag)]`)
/// 只支持载荷是 map 的变体,newtype 里放一个字符串会在运行时报错,而不是编译期。
Note { message: String },
/// Timestamp precedes its JPEG so a stalled WebView can reject queued data.
FrameTiming {
camera: u32,
#[serde(rename = "frameId")]
frame_id: u32,
#[serde(rename = "publishedUnixMs")]
published_unix_ms: u64,
},
}
/// 前端连接这条通道需要的信息。
@@ -144,6 +172,7 @@ pub async fn serve(
listener: TcpListener,
token: String,
tx: broadcast::Sender<Arc<ServerMsg>>,
video: VideoSender,
clients: Arc<AtomicUsize>,
) {
loop {
@@ -159,9 +188,10 @@ pub async fn serve(
let tx = tx.clone();
let token = token.clone();
let clients = Arc::clone(&clients);
let video = video.subscribe();
tauri::async_runtime::spawn(async move {
if let Err(err) = handle_client(stream, peer, &token, tx, clients).await {
if let Err(err) = handle_client(stream, peer, &token, tx, video, clients).await {
log::debug!("stream: client {peer} ended: {err}");
}
});
@@ -173,6 +203,7 @@ async fn handle_client(
peer: std::net::SocketAddr,
token: &str,
tx: broadcast::Sender<Arc<ServerMsg>>,
mut video: watch::Receiver<VideoFrames>,
clients: Arc<AtomicUsize>,
) -> anyhow::Result<()> {
stream.set_nodelay(true).ok();
@@ -201,6 +232,8 @@ async fn handle_client(
let (mut sink, mut incoming) = ws.split();
let mut rx = tx.subscribe();
video.mark_changed();
let mut last_sent = HashMap::new();
let mut keepalive = tokio::time::interval(Duration::from_secs(20));
let count = clients.fetch_add(1, Ordering::Relaxed) + 1;
log::info!("stream: interface connected from {peer} ({count} connected)");
@@ -212,12 +245,35 @@ async fn handle_client(
.unwrap_or_else(|_| "{}".into());
let _ = sink.send(Message::text(hello)).await;
let result = loop {
let result = 'messages: loop {
tokio::select! {
changed = video.changed() => {
if changed.is_err() { break Ok(()); }
// Release the watch read lock before awaiting socket writes.
let latest: Vec<_> = video.borrow_and_update().iter()
.map(|(&camera, packet)| (camera, packet.clone())).collect();
for (camera, packet) in latest {
if last_sent.get(&camera) == Some(&packet.frame_id) { continue; }
if packet.published_at.elapsed() >= LOCAL_VIDEO_MAX_AGE { continue; }
let timing = serde_json::to_string(&WireMsg::FrameTiming {
camera, frame_id: packet.frame_id, published_unix_ms: packet.published_unix_ms,
})?;
// Keep the 16-byte JPEG header compatible. This small text
// message carries host queue age through TCP/WebView queues.
let send = async {
sink.send(Message::text(timing)).await?;
sink.send(Message::Binary(packet.bytes)).await
};
match tokio::time::timeout(LOCAL_VIDEO_MAX_AGE, send).await {
Ok(Ok(())) => { last_sent.insert(camera, packet.frame_id); }
Ok(Err(err)) => break 'messages Err(anyhow::anyhow!("video send failed: {err}")),
Err(_) => break 'messages Err(anyhow::anyhow!("video client stalled")),
}
}
},
message = rx.recv() => match message {
Ok(msg) => {
let out = match &*msg {
ServerMsg::Binary(bytes) => Message::Binary(bytes.clone()),
ServerMsg::Text(text) => Message::text(text.clone()),
};
@@ -334,6 +390,114 @@ mod tests {
assert_eq!(random_token().len(), 48);
}
#[test]
fn a_slow_video_consumer_keeps_only_the_latest_frame_of_each_camera() {
let (video, mut reader) = watch::channel(VideoFrames::new());
for id in 1..=1000 {
publish_video(&video, &frame(1, id, 100));
publish_video(&video, &frame(2, id + 10, 100));
}
let latest = reader.borrow_and_update();
assert_eq!(latest.len(), 2);
assert_eq!(latest[&1].frame_id, 1000);
assert_eq!(latest[&2].frame_id, 1010);
}
#[tokio::test]
async fn a_reconnecting_interface_receives_cached_latest_frames_without_waiting_for_capture() {
let (listener, port) = bind().await.unwrap();
let (control, _) = broadcast::channel(CHANNEL_CAPACITY);
let (video, _) = watch::channel(VideoFrames::new());
for id in 1..=100 {
publish_video(&video, &frame(1, id, 100));
publish_video(&video, &frame(2, id + 10, 100));
}
let task = tokio::spawn(serve(
listener,
"test-token".into(),
control,
video,
Arc::new(AtomicUsize::new(0)),
));
let (mut client, _) =
tokio_tungstenite::connect_async(format!("ws://127.0.0.1:{port}/?token=test-token"))
.await
.unwrap();
let mut received = Vec::new();
let mut timings = HashMap::new();
while received.len() < 2 {
let message = tokio::time::timeout(Duration::from_secs(2), client.next())
.await
.unwrap()
.unwrap()
.unwrap();
if let Message::Text(text) = &message {
let value: serde_json::Value = serde_json::from_str(text).unwrap();
if value["t"] == "frameTiming" {
assert!(value["publishedUnixMs"].as_u64().unwrap() > 0);
timings.insert(
value["camera"].as_u64().unwrap() as u32,
value["frameId"].as_u64().unwrap() as u32,
);
}
}
if let Message::Binary(bytes) = message {
let (camera, frame_id, _, _, _) = decode_frame_header(&bytes).unwrap();
assert_eq!(timings.get(&camera), Some(&frame_id));
received.push((camera, frame_id));
}
}
received.sort_unstable();
assert_eq!(received, vec![(1, 100), (2, 110)]);
client.close(None).await.unwrap();
task.abort();
}
#[tokio::test]
async fn reconnect_does_not_replay_expired_cached_video() {
let (listener, port) = bind().await.unwrap();
let (control, _) = broadcast::channel(CHANNEL_CAPACITY);
let (video, _) = watch::channel(VideoFrames::new());
publish_video(&video, &frame(1, 1, 100));
video.send_modify(|latest| {
latest.get_mut(&1).unwrap().published_at = Instant::now() - Duration::from_secs(1)
});
let task = tokio::spawn(serve(
listener,
"test-token".into(),
control,
video.clone(),
Arc::new(AtomicUsize::new(0)),
));
let (mut client, _) =
tokio_tungstenite::connect_async(format!("ws://127.0.0.1:{port}/?token=test-token"))
.await
.unwrap();
let stale = tokio::time::timeout(Duration::from_millis(100), async {
while let Some(message) = client.next().await {
if matches!(message.unwrap(), Message::Binary(_)) {
return;
}
}
})
.await;
assert!(stale.is_err());
publish_video(&video, &frame(1, 2, 100));
let fresh = tokio::time::timeout(Duration::from_secs(2), async {
while let Some(message) = client.next().await {
if let Message::Binary(bytes) = message.unwrap() {
return decode_frame_header(&bytes).unwrap().1;
}
}
panic!("video connection closed");
})
.await
.unwrap();
assert_eq!(fresh, 2);
client.close(None).await.unwrap();
task.abort();
}
#[test]
fn the_wire_envelope_carries_the_message_kind() {
let json = serde_json::to_string(&WireMsg::Hit(HitDto {
+58 -37
View File
@@ -24,14 +24,14 @@ use cam_core::armor::{ArmorHealthStore, HealthTracker};
use cam_core::config::{Config, Role};
use cam_core::password;
use cam_core::receiver::{Discovery, LatestFrames, Receiver};
use cam_core::ArmorSource;
use cam_core::slots::SlotAllocator;
use cam_core::snapshot::{
resolve_outcomes, ArmorBoardDto, CameraDto, CameraStatsDto, ConfigView, HitDto, Snapshot,
SlotsDto, TeamHealthDto,
resolve_outcomes, ArmorBoardDto, CameraDto, CameraStatsDto, ConfigView, HitDto, SlotsDto,
Snapshot, TeamHealthDto,
};
use cam_core::stats::{FpsWindow, DEFAULT_WINDOW};
use tokio::sync::broadcast;
use cam_core::ArmorSource;
use tokio::sync::{broadcast, watch};
use crate::recording::{now_ms, RecordingSet};
use crate::server::{self, ServerMsg, WireMsg};
@@ -67,6 +67,7 @@ pub struct AppState {
pub recordings: Arc<RecordingSet>,
tx: broadcast::Sender<Arc<ServerMsg>>,
video: server::VideoSender,
/// 回环上已经绑好的 WebSocket listener,等 `spawn_background` 取走。
///
/// 先绑再交给后台任务,是为了让"端口绑不上"在启动阶段就报错,而不是变成一个永远
@@ -92,38 +93,43 @@ impl AppState {
config: Config,
mut notes: Vec<String>,
demo: bool,
) -> Result<(Arc<Self>, Option<std::net::UdpSocket>, Option<std::net::UdpSocket>)> {
) -> Result<(
Arc<Self>,
Option<std::net::UdpSocket>,
Option<std::net::UdpSocket>,
)> {
let role = config.role;
let bound = if role == Role::Team { config.camera_id } else { None };
let bound = if role == Role::Team {
config.camera_id
} else {
None
};
// 视频端口绑不上**不**致命:界面照常起来,并把原因写在最显眼的地方。
//
// 一开始这里是 `?`(直接退出)。但那样一来,现场看到的是一次闪退加上一句只出现
// 在 stderr 的话——从访达双击启动时连 stderr 都没有。让窗口起来并说明原因,比
// 静默退出有用得多;而且"另一个客户端已经在跑"恰好是最常见的原因。
let (receiver, video_socket) = match Receiver::start(
protocol::BROADCAST_ADDR,
role.to_protocol(),
bound,
) {
Ok((receiver, socket)) => (Arc::new(receiver), Some(socket)),
Err(err) => {
log::error!("could not bind UDP {}: {err}", protocol::DEFAULT_PORT);
notes.push(format!(
"绑不上 UDP {}:{err}。多半是同一个端口上已经跑着另一个客户端 \
let (receiver, video_socket) =
match Receiver::start(protocol::BROADCAST_ADDR, role.to_protocol(), bound) {
Ok((receiver, socket)) => (Arc::new(receiver), Some(socket)),
Err(err) => {
log::error!("could not bind UDP {}: {err}", protocol::DEFAULT_PORT);
notes.push(format!(
"绑不上 UDP {}:{err}。多半是同一个端口上已经跑着另一个客户端 \
(旧的 Slint 版或另一份本程序)。画面收不到,其它功能照常。",
protocol::DEFAULT_PORT
));
(
Arc::new(Receiver::new(
protocol::BROADCAST_ADDR,
role.to_protocol(),
bound,
)),
None,
)
}
};
protocol::DEFAULT_PORT
));
(
Arc::new(Receiver::new(
protocol::BROADCAST_ADDR,
role.to_protocol(),
bound,
)),
None,
)
}
};
let discovery = receiver.discovery();
let latest = receiver.latest_frames();
@@ -141,6 +147,7 @@ impl AppState {
};
let (tx, _) = broadcast::channel(server::CHANNEL_CAPACITY);
let (video, _) = watch::channel(server::VideoFrames::new());
let (listener, port) = tauri::async_runtime::block_on(server::bind())
.context("绑不上本地 WebSocket 端口(回环地址)")?;
@@ -157,6 +164,7 @@ impl AppState {
slots: Mutex::new(SlotAllocator::new()),
recordings: RecordingSet::new(),
tx,
video,
listener: Mutex::new(Some(listener)),
port,
token: server::random_token(),
@@ -175,15 +183,14 @@ impl AppState {
///
/// 必须在 `Arc<AppState>` 建好之后调用(sink 里要持有 state 的通道)。
pub fn install_sinks(self: &Arc<Self>) {
let tx = self.tx.clone();
let video = self.video.clone();
let recordings = Arc::clone(&self.recordings);
// 帧的 sink:在接收线程里跑,只做"编码一次 + 广播"。这一份字节所有界面共用。
self.receiver
.set_frame_sink(Arc::new(move |frame: Arc<protocol::Frame>| {
recordings.feed(&frame);
let bytes = server::encode_frame(&frame);
let _ = tx.send(Arc::new(ServerMsg::Binary(bytes.into())));
server::publish_video(&video, &frame);
}));
let tx = self.tx.clone();
@@ -240,9 +247,10 @@ impl AppState {
let token = self.token.clone();
let tx = self.tx.clone();
let clients = Arc::clone(&self.clients);
let video = self.video.clone();
tauri::async_runtime::spawn(async move {
server::serve(listener, token, tx, clients).await;
server::serve(listener, token, tx, video, clients).await;
});
let state_for_loop = Arc::clone(&state);
@@ -306,7 +314,9 @@ impl AppState {
// 本来就要看到全场。
let open = match role {
Role::Referee => true,
Role::Team => self.discovery_open.load(Ordering::Relaxed) && self.unlocked.load(Ordering::Relaxed),
Role::Team => {
self.discovery_open.load(Ordering::Relaxed) && self.unlocked.load(Ordering::Relaxed)
}
};
self.receiver.set_discovery_all(open);
@@ -457,7 +467,12 @@ impl AppState {
/// 推进帧率窗口与槽位分配。
pub fn tick(&self) {
let online: Vec<u32> = self.discovery.online_cameras().iter().map(|c| c.id).collect();
let online: Vec<u32> = self
.discovery
.online_cameras()
.iter()
.map(|c| c.id)
.collect();
let online_set: HashSet<u32> = online.iter().copied().collect();
{
@@ -539,8 +554,12 @@ impl AppState {
})
.collect();
let armor_boards: Vec<ArmorBoardDto> =
self.armor.boards().iter().map(ArmorBoardDto::from).collect();
let armor_boards: Vec<ArmorBoardDto> = self
.armor
.boards()
.iter()
.map(ArmorBoardDto::from)
.collect();
let slots = self.slots.lock().expect("slots mutex poisoned").slots();
@@ -627,7 +646,9 @@ impl AppState {
if let Some(board) = board {
return match self.armor.health_of_board(board) {
Some(hp) => TeamHealthDto::present(team, Some(board), [hp; 4], low, &config.team_names),
Some(hp) => {
TeamHealthDto::present(team, Some(board), [hp; 4], low, &config.team_names)
}
None => TeamHealthDto::absent(team, Some(board), &config.team_names),
};
}
+8 -2
View File
@@ -23,6 +23,7 @@ import { useEffect, useRef, type ReactNode } from "react";
import { streamClient } from "../lib/store";
import { withAlpha } from "../lib/format";
import type { CrosshairConfig, GridConfig, HitFeedbackConfig } from "../lib/types";
import type { FrameInfo } from "../lib/stream";
export interface ViewTransform {
zoom: number;
@@ -61,6 +62,8 @@ interface Props {
hitFeedback: HitFeedbackConfig;
showZoom: boolean;
children?: ReactNode;
/** Only used by the opt-in screen/camera calibration page. */
onFramePainted?: (canvas:HTMLCanvasElement,frame:FrameInfo,drawMs:number)=>void;
}
export function VideoSurface({
@@ -77,12 +80,13 @@ export function VideoSurface({
hitFeedback,
showZoom,
children,
onFramePainted,
}: Props) {
const canvasRef = useRef<HTMLCanvasElement | null>(null);
// 绘制循环不随 props 重建,所以最新一份 props 放在 ref 里给它读。
const props = useRef({ camera, mirrored, crosshair, grid, view, hit, hitFeedback });
props.current = { camera, mirrored, crosshair, grid, view, hit, hitFeedback };
const props = useRef({ camera, mirrored, crosshair, grid, view, hit, hitFeedback,onFramePainted });
props.current = { camera, mirrored, crosshair, grid, view, hit, hitFeedback,onFramePainted };
const drag = useRef<{ x: number; y: number; originX: number; originY: number } | null>(null);
const dirty = useRef(true);
@@ -127,6 +131,7 @@ export function VideoSurface({
const current = props.current;
const frame = streamClient().latest(current.camera);
const newFrame=frame && frame.frameId!==lastFrameId.current;
if (frame && frame.frameId !== lastFrameId.current) {
lastFrameId.current = frame.frameId;
@@ -145,6 +150,7 @@ export function VideoSurface({
dirty.current = false;
draw(canvas, current, frame, timestamp);
if(newFrame && current.onFramePainted) current.onFramePainted(canvas,frame,performance.now());
};
raf = requestAnimationFrame(loop);
+101
View File
@@ -0,0 +1,101 @@
/** A screen marker observed through the real camera. No device clock sync. */
export type Point = { x: number; y: number };
export const CORNERS = [[255,32,32],[32,255,32],[32,96,255],[255,32,255]] as const;
export const MARKER_BITS = 16;
export function markerBits(counter: number): number[] {
const gray=(counter&4095)^((counter&4095)>>>1);
return [1,0,1,0,...Array.from({length:12},(_,i)=>(gray>>>(11-i))&1)];
}
export function drawMarker(ctx: CanvasRenderingContext2D, counter:number) {
const w=ctx.canvas.width,h=ctx.canvas.height;
ctx.fillStyle='#30363e';ctx.fillRect(0,0,w,h);
// Static coloured detail keeps the target from being an unrealistically tiny JPEG.
for(let y=0;y<8;y++) for(let x=0;x<12;x++) {
ctx.fillStyle=`hsl(${(x*31+y*47)%360} 10% ${20+(x+y)%3*7}%)`;
ctx.fillRect(x*w/12,y*h/8,w/12,h/8);
}
[[.06,.08],[.94,.08],[.94,.92],[.06,.92]].forEach(([x,y],i)=>{
ctx.fillStyle=`rgb(${CORNERS[i].join(',')})`;
ctx.fillRect(x*w-.035*w,y*h-.045*h,.07*w,.09*h);
});
const bits=markerBits(counter);
ctx.fillStyle='#222';ctx.fillRect(.12*w,.34*h,.76*w,.32*h);
bits.forEach((bit,i)=>{
const x=(.14+.72*i/MARKER_BITS)*w,cw=.72*w/MARKER_BITS;
ctx.fillStyle=bit?'#fff':'#000';ctx.fillRect(x,.38*h,cw,.10*h);
ctx.fillStyle=bit?'#000':'#fff';ctx.fillRect(x,.52*h,cw,.10*h);
});
}
function colour(r:number,g:number,b:number):number {
// Screen primaries can clip and bloom through exposure/white balance.
// Absolute channel separation admits pastel green/cyan while excluding
// the low-saturation background (which can otherwise look more blue).
if(r>150 && b>150 && r-g>40 && b-g>40 && Math.abs(r-b)<100) return 4;
if(r>150 && r-g>40 && r-b>40) return 1;
if(g>150 && g-r>35 && g-b>30) return 2;
if(b>150 && b>r*1.7 && b-g>20) return 3;
return 0;
}
function project(corners:Point[],u:number,v:number):Point|null {
const [p0,p1,p2,p3]=corners;
const dx1=p1.x-p2.x, dx2=p3.x-p2.x, dx3=p0.x-p1.x+p2.x-p3.x;
const dy1=p1.y-p2.y, dy2=p3.y-p2.y, dy3=p0.y-p1.y+p2.y-p3.y;
const det=dx1*dy2-dx2*dy1;
if(Math.abs(det)<1e-6) return null;
const g=(dx3*dy2-dx2*dy3)/det, h=(dx1*dy3-dx3*dy1)/det;
const den=g*u+h*v+1;
if(Math.abs(den)<.01) return null;
return {x:((p1.x-p0.x+g*p1.x)*u+(p3.x-p0.x+h*p3.x)*v+p0.x)/den,
y:((p1.y-p0.y+g*p1.y)*u+(p3.y-p0.y+h*p3.y)*v+p0.y)/den};
}
export function decodeMarker(pixels:Uint8ClampedArray,w:number,h:number):{counter:number;corners:Point[]}|null {
const labels=new Uint8Array(w*h);
for(let i=0;i<labels.length;i++) labels[i]=colour(pixels[i*4],pixels[i*4+1],pixels[i*4+2]);
const best=Array.from({length:4},()=>({count:0,x:0,y:0}));
const queue=new Int32Array(w*h);
for(let start=0;start<labels.length;start++) {
const cls=labels[start]; if(!cls) continue;
let head=0,tail=1,count=0,sx=0,sy=0;
queue[0]=start;labels[start]=0;
while(head<tail) {
const i=queue[head++],x=i%w,y=Math.floor(i/w);
count++;sx+=x;sy+=y;
for(const n of [x>0?i-1:-1,x<w-1?i+1:-1,y>0?i-w:-1,y<h-1?i+w:-1]) {
if(n>=0 && labels[n]===cls) { labels[n]=0;queue[tail++]=n; }
}
}
if(count>=6 && count<w*h/6 && count>best[cls-1].count) best[cls-1]={count,x:sx/count,y:sy/count};
}
if(best.some(b=>!b.count)) return null;
const corners=best.map(({x,y})=>({x,y}));
const area=Math.abs(corners.reduce((a,p,i)=>a+p.x*corners[(i+1)%4].y-p.y*corners[(i+1)%4].x,0))/2;
if(area<1500) return null;
const luma=(x:number,y:number)=>{
let total=0,count=0;
for(let dy=-1;dy<=1;dy++) for(let dx=-1;dx<=1;dx++) {
const xx=Math.round(x)+dx,yy=Math.round(y)+dy;
if(xx<0||xx>=w||yy<0||yy>=h) continue;
const i=(yy*w+xx)*4;total+=(pixels[i]+pixels[i+1]+pixels[i+2])/3;count++;
}
return count?total/count:NaN;
};
const bits:number[]=[];
for(let i=0;i<MARKER_BITS;i++) {
const x=.14+.72*(i+.5)/MARKER_BITS;
const top=project(corners,(x-.06)/.88,(.43-.08)/.84);
const bottom=project(corners,(x-.06)/.88,(.57-.08)/.84);
if(!top||!bottom) return null;
const delta=luma(top.x,top.y)-luma(bottom.x,bottom.y);
if(!Number.isFinite(delta)||Math.abs(delta)<60) return null;
bits.push(delta>0?1:0);
}
if(bits.slice(0,4).join('')!=='1010') return null;
let gray=bits.slice(4).reduce((n,b)=>(n<<1)|b,0),counter=gray;
while(gray>>>=1) counter^=gray;
return {counter,corners};
}
+2
View File
@@ -76,6 +76,8 @@ export function currentStore(): AppStore {
function handleMessage(msg: WireMsg) {
switch (msg.t) {
case "frameTiming":
return; // Consumed by StreamClient before reaching application state.
case "snapshot": {
const { t: _tag, ...snapshot } = msg;
emit({ snapshot: snapshot as Snapshot });
+82 -20
View File
@@ -28,7 +28,7 @@ export interface FrameInfo {
width: number;
height: number;
bitmap: ImageBitmap;
/** 收到的时刻(`performance.now()`),界面用来显示端到端延迟。 */
/** 本地完成解码的时刻(`performance.now()`),用于判断帧的新旧。 */
receivedAt: number;
}
@@ -39,6 +39,13 @@ interface Handlers {
/** 断线重连的退避上限。 */
const MAX_BACKOFF_MS = 3000;
const MAX_LOCAL_VIDEO_AGE_MS = 100;
interface PendingFrame {
bytes: ArrayBuffer;
arrivedAt: number;
publishedUnixMs?: number;
}
export class StreamClient {
private ws: WebSocket | null = null;
@@ -48,14 +55,19 @@ export class StreamClient {
private retryTimer: number | null = null;
private backoff = 250;
private closed = false;
private generation = 0;
private readonly pending = new Map<number, PendingFrame>();
private readonly frameTimes = new Map<number, { frameId: number; publishedUnixMs: number }>();
private readonly decoding = new Map<number, number>();
/** 统计:收到多少帧、解码失败多少张。界面用它区分"链路没数据"和"数据坏了"。 */
readonly counters = { frames: 0, decodeFailures: 0, reconnects: 0 };
readonly counters = { frames: 0, decodeFailures: 0, reconnects: 0, skippedDecodes: 0, staleFrames: 0 };
constructor(handlers: Handlers) {
constructor(handlers: Handlers, private readonly now = () => performance.now(), private readonly wallNow = () => Date.now()) {
this.handlers = handlers;
}
connect(url: string) {
this.close();
this.url = url;
this.closed = false;
this.open();
@@ -76,11 +88,17 @@ export class StreamClient {
this.ws = socket;
socket.onopen = () => {
if (this.ws !== socket) return;
this.backoff = 250;
this.handlers.onConnection(true);
};
socket.onclose = () => {
if (this.ws !== socket) return;
this.generation += 1;
this.pending.clear();
this.frameTimes.clear();
this.decoding.clear();
this.handlers.onConnection(false);
this.scheduleReconnect();
};
@@ -90,9 +108,12 @@ export class StreamClient {
};
socket.onmessage = (event) => {
if (this.ws !== socket) return;
if (typeof event.data === "string") {
try {
this.handlers.onMessage(JSON.parse(event.data) as WireMsg);
const message = JSON.parse(event.data) as WireMsg;
if (message.t === "frameTiming") this.onTiming(message);
else this.handlers.onMessage(message);
} catch {
// 前端解析不了的消息只可能是版本不匹配,忽略比崩掉好。
}
@@ -103,31 +124,66 @@ export class StreamClient {
};
}
private onTiming(message: { camera: number; frameId: number; publishedUnixMs: number }) {
if (Number.isFinite(message.publishedUnixMs) && message.publishedUnixMs > 0) {
this.frameTimes.set(message.camera, message);
}
}
private expired(frame: PendingFrame): boolean {
return this.now() - frame.arrivedAt >= MAX_LOCAL_VIDEO_AGE_MS ||
(frame.publishedUnixMs !== undefined && this.wallNow() - frame.publishedUnixMs >= MAX_LOCAL_VIDEO_AGE_MS);
}
private async onBinary(buffer: ArrayBuffer) {
if (buffer.byteLength < FRAME_HEADER_LEN) return;
if (this.closed || buffer.byteLength < FRAME_HEADER_LEN) return;
const view = new DataView(buffer);
const camera = view.getUint32(0, true);
const frameId = view.getUint32(4, true);
const width = view.getUint16(8, true);
const height = view.getUint16(10, true);
const length = view.getUint32(12, true);
if (buffer.byteLength < FRAME_HEADER_LEN + length) return;
if (buffer.byteLength !== FRAME_HEADER_LEN + length) return;
this.counters.frames += 1;
const timing = this.frameTimes.get(camera);
const received: PendingFrame = { bytes: buffer, arrivedAt: this.now(),
publishedUnixMs: timing?.frameId === view.getUint32(4, true) ? timing.publishedUnixMs : undefined };
if (this.expired(received)) { this.counters.staleFrames += 1; return; }
if (this.pending.has(camera)) this.counters.skippedDecodes += 1;
this.pending.set(camera, received);
if (this.decoding.has(camera)) return;
const jpeg = new Uint8Array(buffer, FRAME_HEADER_LEN, length);
let bitmap: ImageBitmap;
const generation = this.generation;
this.decoding.set(camera, generation);
try {
bitmap = await createImageBitmap(new Blob([jpeg], { type: "image/jpeg" }));
} catch {
this.counters.decodeFailures += 1;
return;
while (!this.closed && this.generation === generation) {
const next = this.pending.get(camera);
if (!next) break;
this.pending.delete(camera);
if (this.expired(next)) { this.counters.staleFrames += 1; continue; }
const header = new DataView(next.bytes);
const jpeg = new Uint8Array(next.bytes, FRAME_HEADER_LEN);
let bitmap: ImageBitmap;
try {
bitmap = await createImageBitmap(new Blob([jpeg], { type: "image/jpeg" }));
} catch {
this.counters.decodeFailures += 1;
continue;
}
if (this.closed || this.generation !== generation) {
bitmap.close();
break;
}
if (this.expired(next)) {
bitmap.close();
this.counters.staleFrames += 1;
continue;
}
this.store(camera, header.getUint32(4, true), header.getUint16(8, true), header.getUint16(10, true), bitmap);
}
} finally {
if (this.decoding.get(camera) === generation) this.decoding.delete(camera);
}
this.store(camera, frameId, width, height, bitmap);
}
/**
@@ -149,7 +205,9 @@ export class StreamClient {
const current = this.frames.get(camera);
// 带符号差处理 u32 回绕。
if (current && ((frameId - current.frameId) | 0) <= 0) {
// After a quiet interval the board may have rebooted and restarted its
// frame counter. An older in-flight decode cannot hold the display forever.
if (current && ((frameId - current.frameId) | 0) <= 0 && this.now() - current.receivedAt < 1000) {
bitmap.close();
return;
}
@@ -162,7 +220,7 @@ export class StreamClient {
width,
height,
bitmap,
receivedAt: performance.now(),
receivedAt: this.now(),
});
}
@@ -197,6 +255,10 @@ export class StreamClient {
close() {
this.closed = true;
this.generation += 1;
this.pending.clear();
this.frameTimes.clear();
this.decoding.clear();
if (this.retryTimer !== null) {
window.clearTimeout(this.retryTimer);
+1
View File
@@ -233,6 +233,7 @@ export type WireMsg =
| ({ t: "snapshot" } & Snapshot)
| ({ t: "hit" } & HitEvent)
| { t: "note"; message: string }
| { t: "frameTiming"; camera: number; frameId: number; publishedUnixMs: number }
| { t: "confirmQuit" };
/** 二进制视频帧的头部长度,与 `server::FRAME_HEADER_LEN` 一致。 */
+6 -1
View File
@@ -1,6 +1,8 @@
import { createRoot } from "react-dom/client";
import App from "./App";
import { LatencyPage } from './pages/LatencyPage';
import { invoke } from '@tauri-apps/api/core';
import "./styles/global.css";
const container = document.getElementById("root");
@@ -12,4 +14,7 @@ if (!container) {
// 不用 StrictMode:这个界面的关键路径是命令式的(WebSocket、requestAnimationFrame 绘制
// 循环、canvas 尺寸)。StrictMode 在开发下会把每个 effect 跑两遍,于是连接与绘制循环会
// 短暂地建两份——排查起来的噪音大于它带来的收益。生产构建的行为与这里一致。
createRoot(container).render(<App />);
const root=createRoot(container);
if('__TAURI_INTERNALS__' in window) {
void invoke<boolean>('latency_enabled').then(enabled=>root.render(enabled?<LatencyPage/>:<App/>),()=>root.render(<App/>));
} else root.render(<App/>);
+101
View File
@@ -0,0 +1,101 @@
import { useEffect,useRef,useState } from 'react';
import { invoke } from '@tauri-apps/api/core';
import { api } from '../lib/ipc';
import { startStream,useAppStore } from '../lib/store';
import { decodeMarker,drawMarker } from '../lib/latency-marker';
import { VideoSurface,IDENTITY_VIEW } from '../components/VideoSurface';
import type { FrameInfo } from '../lib/stream';
type Stamp={draw:number;next:number|null;sampled:boolean};
const feedback={flash:false,shake:false,marker:false,intensity:0};
export function LatencyPage() {
const {snapshot,connected}=useAppStore();
const [camera,setCamera]=useState<number|null>(null);
const [status,setStatus]=useState('等待摄像头与屏幕标记');
const [summary,setSummary]=useState({samples:0,p50:0,p95:0,refresh:0});
const target=useRef<HTMLCanvasElement>(null);
const history=useRef(new Map<number,Stamp>());
const scratch=useRef<HTMLCanvasElement|null>(null);
const values=useRef<number[]>([]);
const lastSample=useRef(-Infinity);
const refresh=useRef(16.7);
const active=useRef(true);
useEffect(()=>{
let client:ReturnType<typeof startStream>|null=null,disposed=false;
void api.streamInfo().then(info=>{if(!disposed)client=startStream(info.url);}).catch(e=>setStatus(String(e)));
return()=>{disposed=true;client?.close();};
},[]);
useEffect(()=>{
if(camera===null && snapshot) {
const found=snapshot.cameras.find(c=>c.online);
if(found)setCamera(found.id);
}
},[snapshot,camera]);
useEffect(()=>{
active.current=true;
const canvas=target.current;if(!canvas)return;
const ctx=canvas.getContext('2d');if(!ctx)return;
let raf=0,counter=0,previous:Stamp|null=null,last=0,lastMarker=-Infinity;
const loop=(now:number)=>{
if(previous){previous.next=now;previous=null;}
if(last && now-last<50)refresh.current=refresh.current*.9+(now-last)*.1;
last=now;
// A 180 Hz display can change several IDs within one camera exposure.
// Hold each event for 50 ms, and record only its first decodable echo.
if(now-lastMarker>=50){
lastMarker=now;counter=(counter+1)&4095;
drawMarker(ctx,counter);
const stamp={draw:performance.now(),next:null,sampled:false};
history.current.set(counter,stamp);previous=stamp;
}
// Only the most recent five seconds count, including counter wrap.
for(const [id,value] of history.current) if(now-value.draw>5000)history.current.delete(id);
raf=requestAnimationFrame(loop);
};
raf=requestAnimationFrame(loop);
return()=>{active.current=false;cancelAnimationFrame(raf);};
},[]);
const painted=(canvas:HTMLCanvasElement,frame:FrameInfo,drawMs:number)=>{
if(drawMs-lastSample.current<25)return;
lastSample.current=drawMs;
const started=performance.now();
const small=scratch.current??(scratch.current=document.createElement('canvas'));
small.width=320;small.height=240;
const ctx=small.getContext('2d',{willReadFrequently:true});if(!ctx)return;
// Read the canvas after the production renderer has painted the frame.
ctx.drawImage(canvas,0,0,320,240);
const marker=decodeMarker(ctx.getImageData(0,0,320,240).data,320,240);
const analysisMs=performance.now()-started;
if(!marker){setStatus('未识别到四角标记:请让左侧标记完整入镜并尽量拍大');return;}
const source=history.current.get(marker.counter);
if(!source||source.next===null||source.draw>drawMs){setStatus('等待有效的同一会话标记');return;}
if(source.sampled)return;
source.sampled=true;
requestAnimationFrame(nextRafMs=>{
if(!active.current)return;
const lowerMs=Math.max(0,drawMs-source.next!);
const upperMs=nextRafMs-source.draw;
if(upperMs<lowerMs||upperMs>5000)return;
const value=(lowerMs+upperMs)/2;
values.current.push(value);
const sorted=[...values.current].sort((a,b)=>a-b);
setStatus('标记已锁定,正在记录真实相机 → 客户端画布链路');
setSummary({samples:sorted.length,p50:sorted[Math.floor((sorted.length-1)*.5)],p95:sorted[Math.floor((sorted.length-1)*.95)],refresh:refresh.current});
void invoke('latency_sample',{sample:{frameId:frame.frameId,counter:marker.counter,sourceDrawMs:source.draw,
drawMs,nextRafMs,lowerMs,upperMs,decodeToDrawMs:drawMs-frame.receivedAt,analysisMs}}).catch(e=>setStatus(`报告写入失败:${String(e)}`));
});
};
return <main style={{height:'100vh',display:'flex',flexDirection:'column',padding:20,gap:12,background:'#111820',color:'#f3f6fa'}}>
<h1 style={{fontSize:24}}>屏幕 → 摄像头 → 客户端绘制延迟校准</h1>
<p>将摄像头固定对准左侧标记,完整拍到四个彩色角块。不要拍右侧回显区域;保持窗口位置不动。</p>
<div style={{display:'grid',gridTemplateColumns:'1fr 1fr',gap:16,flex:1,minHeight:0}}>
<canvas ref={target} width={800} height={600} style={{width:'100%',height:'100%',objectFit:'contain'}}/>
<div style={{position:'relative',minHeight:0}}><VideoSurface camera={camera} emptyText="等待真实摄像头" mirrored={false}
crosshair={null} grid={null} view={IDENTITY_VIEW} onView={()=>{}} hitFeedback={feedback} showZoom={false} onFramePainted={painted}/></div>
</div>
<p>{connected?'已连接':'连接中'} · {status}</p>
<p style={{fontSize:22}}>样本 {summary.samples} · P50 {summary.p50.toFixed(1)} ms · P95 {summary.p95.toFixed(1)} ms</p>
<p>每 50 ms 更新标记,记录首次可解码回显,包含相机取样等待、曝光、采集、传输和绘制;混合曝光帧不计入。相邻 rAF 夹定绘制时间,刷新间隔约 {summary.refresh.toFixed(1)} ms;面板像素响应需高速相机复核。</p>
</main>;
}