feat: 链路控制、延迟校准与长测工具,补全文档
- 固件:新增 link_control / stream_control,覆盖运行中 Wi-Fi 断线恢复、取帧节拍、 温控与质量/帧率减载;UDP 发送增加有界重试预算与帧期限,PSRAM JPEG 直发减少拷贝。 - 协议:重组器严格校验分片布局,统计未收齐、超时与完全缺失的帧,处理帧号回绕与设备重启。 - 桌面端:视频改 latest-value 通道并前置帧时间标记,拒绝积压和慢解码的过期画面;新增 延迟标记页与 --latency-report 校准,完善录制、服务端与退出流程。 - 文档与脚本:新增硬件、开发、延迟、功耗、实时性、稳定性文档和改动记录,重写根 README; 新增构建、性能、延迟与串口长测脚本,以及长测客户端脚本。 - 构建:固定 esp-1.97.0.0 工具链,分区表随构建复制,更新 sdkconfig 与忽略规则。
This commit is contained in:
+47
-12
@@ -12,8 +12,8 @@
|
||||
|
||||
```text
|
||||
desktop/
|
||||
core/ cam-core:不含任何界面与 Tauri 的东西(80 个测试)
|
||||
src-tauri/ Tauri 后端:推流、命令、录制(30 个测试)
|
||||
core/ cam-core:不含任何界面与 Tauri 的东西(83 个测试)
|
||||
src-tauri/ Tauri 后端:推流、命令、录制(32 个测试)
|
||||
src/ React 前端:两套页面 + HUD
|
||||
src/lib/ IPC 封装、WebSocket 客户端、状态存储、格式化
|
||||
src/components/ 视频窗格、血条、胜负横幅、瞄准器、装甲板表、队名编辑、录制面板、设置抽屉
|
||||
@@ -27,16 +27,37 @@ cd desktop
|
||||
pnpm install
|
||||
pnpm app # = tauri dev:起 vite + 编 Rust + 开窗口
|
||||
pnpm build # 只构建前端(tsc --noEmit && vite build)
|
||||
pnpm tauri build # 出 .app 与 .dmg
|
||||
pnpm tauri build # 默认打包 macOS 的 .app 与 .dmg;Windows 见下文
|
||||
```
|
||||
|
||||
下面两条是 Rust 侧,用它验证不依赖窗口的部分:
|
||||
|
||||
```sh
|
||||
cd desktop/core && cargo test # 80
|
||||
cd desktop/src-tauri && cargo test # 30,其中几条真的会起 ffmpeg
|
||||
cd desktop/core && cargo test # 83
|
||||
cd desktop/src-tauri && cargo test # 32,其中几条真的会起 ffmpeg
|
||||
```
|
||||
|
||||
### Windows x64 构建
|
||||
|
||||
需要 Node.js 22.12+(或 24)、pnpm 12、Rust stable MSVC,以及 Visual Studio Build Tools 的 C++ 桌面开发组件和 Windows SDK。应用窗口使用 WebView2 Runtime。
|
||||
|
||||
在 `desktop/` 中执行:
|
||||
|
||||
```powershell
|
||||
pnpm install --frozen-lockfile
|
||||
pnpm tauri build --target x86_64-pc-windows-msvc --bundles nsis --ci -- --locked --config 'unstable.build-std=[]'
|
||||
```
|
||||
|
||||
`--target` 覆盖本目录原来的 macOS target,`--bundles nsis` 选择 Windows 安装包;末尾的 Cargo 参数清空从根固件项目继承的 `build-std`。
|
||||
|
||||
- 程序:`src-tauri/target/x86_64-pc-windows-msvc/release/cam-desktop.exe`。
|
||||
- 安装包:同一 release 目录下的 `bundle/nsis/*-setup.exe`。
|
||||
- 核心测试:进入 `core/` 后运行 `cargo +stable test --locked --target x86_64-pc-windows-msvc --config 'unstable.build-std=[]'`。
|
||||
|
||||
视频录制另需可从 `PATH` 找到的 `ffmpeg`,也可以在配置的 `[recording] ffmpeg` 中指定路径;基础图传显示不依赖它。
|
||||
|
||||
设备发现使用板卡的 Hello 广播;客户端除广播订阅外,还会按已发现设备的 IP 定向续订。Windows 开启 VPN / 虚拟网卡时,全局广播可能走虚拟接口;定向续订可避免“能发现并绑定,但始终等待画面”的情况。队伍端的绑定和画面过滤仍然生效。
|
||||
|
||||
### 演示模式:没有板子也能把整条链路跑起来
|
||||
|
||||
```sh
|
||||
@@ -410,11 +431,13 @@ open 'http://127.0.0.1:1420/?preview=referee' # 裁判端(密码 1234)
|
||||
## 测试
|
||||
|
||||
```sh
|
||||
cd desktop/core && cargo test # 80:重组、发现、两道闸门、帧率窗口、槽位、配置、密码、胜负判定
|
||||
cd desktop/src-tauri && cargo test # 30:帧头布局、录制、截图、演示画面、命令行
|
||||
cd protocol && cargo test # 15
|
||||
cd desktop/core && cargo test # 83:重组、发现、两道闸门、帧率窗口、槽位、配置、密码、胜负判定
|
||||
cd desktop/src-tauri && cargo test # 32:帧头布局、录制、截图、演示画面、命令行
|
||||
cd protocol && cargo test # 20
|
||||
```
|
||||
|
||||
前端解码队列测试:在 `desktop/` 运行 `pnpm test:stream`,5 项测试覆盖慢解码时保留最新帧、多路独立处理、关闭清理、坏帧恢复与设备重启。它不替代实际 WebView 的画面验证。
|
||||
|
||||
三条值得单独说的:
|
||||
|
||||
- **录制**那条真的起 ffmpeg、喂 20 帧、停,然后用 `ffprobe` 读回帧数与分辨率。只断言"文件
|
||||
@@ -444,8 +467,11 @@ RUST_LOG=debug ./target/aarch64-apple-darwin/debug/cam-desktop | grep 'stream:'
|
||||
| `demo-*`(仅演示) | 合成协议数据报 | 走同一条真实路径 |
|
||||
| tokio 任务 | WebSocket 服务、5 Hz 快照 | 异步 IO |
|
||||
|
||||
跨线程只有两样东西:`Arc` 与广播通道。帧在接收线程里编码一次(16 字节头 + JPEG),
|
||||
所有界面共用同一份 `Bytes`。
|
||||
帧在接收线程里编码一次(16 字节头 + JPEG),所有界面共用同一份 `Bytes`。
|
||||
视频使用每路只保留最新帧的 watch 通道;状态与控制消息使用有序广播通道。
|
||||
前端每路最多保留一次正在进行的解码和一张最新待解码帧。
|
||||
|
||||
每张本地 JPEG 前增加 `frameTiming` 文本消息,包含同机后端发布时刻;JPEG 二进制头仍为 16 字节。前端在解码前后检查本机队列年龄,超过 100 ms 的待解码帧或解码结果直接释放,恢复后优先显示最新画面。后端不重放超过 100 ms 的缓存帧,本地视频 socket 写入超时后重连。旧版没有时间标记时仍支持 JPEG,但只能按 JS 收到时刻判断本地年龄。这些时间不能用于测量 ESP 到电脑的无线传输延迟。
|
||||
|
||||
## 已知限制与没做的
|
||||
|
||||
@@ -459,11 +485,20 @@ RUST_LOG=debug ./target/aarch64-apple-darwin/debug/cam-desktop | grep 'stream:'
|
||||
`ws://127.0.0.1:*`(端口是启动时随机分配的,写不死)。验证方式是"前端能不能连上后端"
|
||||
——见下面"测试"一节最后一条。
|
||||
- **证书与公证**没做,`.dmg` 只在本机可用。
|
||||
- 前端没有自动化测试。界面这一侧的验证手段是浏览器预览 + 真机运行;逻辑都在 `core/` 里
|
||||
有测试。
|
||||
- 前端已有解码队列测试;布局、实际 WebView 解码和画面显示仍需浏览器预览及真机运行验证。
|
||||
|
||||
## 排查
|
||||
|
||||
需要测量相机到客户端的整条链路时,使用裁判配置启动校准模式:
|
||||
|
||||
```powershell
|
||||
.\cam-desktop.exe --config .\referee.toml --latency-report .\latency-run.ndjson
|
||||
```
|
||||
|
||||
报告路径必须是新文件。让相机完整拍到左侧四个彩色角块,避开右侧回显,固定相机和窗口位置。标记每 50 ms 更新;程序沿正常接收、解码和 `VideoSurface` 绘制路径记录每个标记的首次有效回显,拒绝曝光混合而无法可靠解码的帧。NDJSON 包含相邻 rAF 夹定的时间、解码完成到画布绘制时间和标记分析耗时,不保存图像或配置。面板扫描和像素响应未测量,精确物理光学延迟需用高速相机复核。正常启动不运行标记识别;校准模式不会额外订阅一份 UDP 视频。
|
||||
|
||||
`pnpm test:latency` 验证标记边界、旋转/镜像、混合曝光拒绝及屏幕原色泛白情形。
|
||||
|
||||
| 现象 | 原因 |
|
||||
|---|---|
|
||||
| 起不来,提示绑不上 UDP 5600 | 同一端口上已经跑着另一个客户端(旧的 Slint 版或另一份本程序)。看 `~/.config/cam-desktop/startup-error.txt` |
|
||||
|
||||
Generated
+26
@@ -55,6 +55,7 @@ dependencies = [
|
||||
"rand",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"socket2",
|
||||
"toml",
|
||||
]
|
||||
|
||||
@@ -297,6 +298,16 @@ dependencies = [
|
||||
"serde_core",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "socket2"
|
||||
version = "0.6.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4"
|
||||
dependencies = [
|
||||
"libc",
|
||||
"windows-sys",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "subtle"
|
||||
version = "2.6.1"
|
||||
@@ -391,6 +402,21 @@ dependencies = [
|
||||
"wit-bindgen",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-link"
|
||||
version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
|
||||
|
||||
[[package]]
|
||||
name = "windows-sys"
|
||||
version = "0.61.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
|
||||
dependencies = [
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "winnow"
|
||||
version = "0.7.15"
|
||||
|
||||
@@ -10,6 +10,7 @@ protocol = { path = "../../protocol" }
|
||||
|
||||
anyhow = "1"
|
||||
log = "0.4"
|
||||
socket2 = "0.6"
|
||||
|
||||
# 配置的序列化。这里与旧客户端(client/)的做法相反:那一版的配置只有十来个标量
|
||||
# 字段,手写 TOML 换来零依赖是划算的。现在配置里有 HUD 的嵌套结构和密码哈希,手写
|
||||
|
||||
+114
-22
@@ -11,7 +11,8 @@
|
||||
//! 两个方向各司其职:
|
||||
//!
|
||||
//! - 板子每 2 秒广播一次 [`protocol::MsgType::Hello`],客户端被动收到就知道有谁在线。
|
||||
//! - 客户端广播 [`protocol::MsgType::Subscribe`],板子收到后开始定向推流。
|
||||
//! - 客户端广播 [`protocol::MsgType::Subscribe`],并向已发现的板子单播续订。
|
||||
//! 多网卡 / VPN 可能把全局广播路由到虚拟网卡;单播按板子 IP 选择局域网接口。
|
||||
//!
|
||||
//! 板子靠**源地址**记住客户端,所以客户端只要周期性续订就能维持推流,不存在任何
|
||||
//! 超时淘汰的竞态。
|
||||
@@ -59,7 +60,7 @@ use protocol::{
|
||||
/// 来自网络,而不是来自过小的缓冲。
|
||||
const RECV_BUFFER: usize = 2048;
|
||||
|
||||
/// 重新广播订阅的间隔。
|
||||
/// 广播及定向续订的间隔。
|
||||
///
|
||||
/// 比板子判定"客户端掉线"的超时(6 秒)短得多,所以偶发丢一两个包不会中断推流。
|
||||
const SUBSCRIBE_INTERVAL: Duration = Duration::from_millis(800);
|
||||
@@ -291,8 +292,7 @@ impl Discovery {
|
||||
|
||||
/// 客户端身份。
|
||||
///
|
||||
/// 常量而非 MAC 派生:队伍客户端和裁判终端各自唯一,进程内也不需要区分自己,而一个
|
||||
/// 确定的常量让日志更好读。
|
||||
/// 旧客户端使用的固定标识,保留用于兼容判断。
|
||||
pub const CLIENT_ID: u32 = 0x0C11_0001;
|
||||
|
||||
/// 接收循环的可变状态:只由接收线程在跑循环时访问。
|
||||
@@ -318,6 +318,7 @@ pub struct Receiver {
|
||||
/// 身份可以运行时切换(同一个进程里从裁判端切到用户端),所以订阅里声明的角色
|
||||
/// 也得能改——它决定板子要不要给这个客户端推流。
|
||||
role: Mutex<SubscribeRole>,
|
||||
client_id: u32,
|
||||
camera: Ipv4Addr,
|
||||
stop: Arc<AtomicBool>,
|
||||
}
|
||||
@@ -327,6 +328,10 @@ impl Receiver {
|
||||
///
|
||||
/// `only_sender` 为 `Some` 时只接受那一块板子的帧。
|
||||
pub fn new(camera: Ipv4Addr, role: SubscribeRole, only_sender: Option<u32>) -> Self {
|
||||
let client_id = loop {
|
||||
let id = rand::random::<u32>();
|
||||
if id != 0 && id != CLIENT_ID { break id; }
|
||||
};
|
||||
Self {
|
||||
discovery: Arc::new(Discovery::new()),
|
||||
latest: Arc::new(LatestFrames::default()),
|
||||
@@ -337,6 +342,7 @@ impl Receiver {
|
||||
discovery_all: false,
|
||||
}),
|
||||
role: Mutex::new(role),
|
||||
client_id,
|
||||
camera,
|
||||
stop: Arc::new(AtomicBool::new(false)),
|
||||
}
|
||||
@@ -354,6 +360,15 @@ impl Receiver {
|
||||
) -> std::io::Result<(Self, UdpSocket)> {
|
||||
let socket =
|
||||
UdpSocket::bind(SocketAddrV4::new(Ipv4Addr::UNSPECIFIED, DEFAULT_PORT))?;
|
||||
// Absorb short scheduler/NIC bursts without losing JPEG fragments. The
|
||||
// reader still drains continuously; decoded video uses latest frames.
|
||||
let socket_ref = socket2::SockRef::from(&socket);
|
||||
if let Err(err) = socket_ref.set_recv_buffer_size(1024 * 1024) {
|
||||
log::warn!("could not enlarge the video receive buffer: {err}");
|
||||
}
|
||||
if let Ok(bytes) = socket_ref.recv_buffer_size() {
|
||||
log::info!("video UDP receive buffer: {bytes} bytes");
|
||||
}
|
||||
socket.set_read_timeout(Some(Duration::from_millis(100)))?;
|
||||
|
||||
let broadcast = socket.set_broadcast(true).is_ok();
|
||||
@@ -473,11 +488,17 @@ impl Receiver {
|
||||
.entry(id)
|
||||
.or_insert_with(|| FrameAssembler::new(FRAME_TIMEOUT));
|
||||
|
||||
match assembler.push(frag) {
|
||||
Ok(frame) => {
|
||||
drop(ingest);
|
||||
let dropped_before = assembler.dropped_frames();
|
||||
let result = assembler.push(frag);
|
||||
let dropped = assembler.dropped_frames() - dropped_before;
|
||||
drop(ingest);
|
||||
self.discovery.count(id, |s| {
|
||||
s.datagrams += 1;
|
||||
s.frames_dropped += dropped;
|
||||
});
|
||||
|
||||
self.discovery.count(id, |s| s.datagrams += 1);
|
||||
match result {
|
||||
Ok(frame) => {
|
||||
self.discovery
|
||||
.note_frame(id, frame.width, frame.height, frame.data.len());
|
||||
|
||||
@@ -489,17 +510,7 @@ impl Receiver {
|
||||
sink(frame);
|
||||
}
|
||||
}
|
||||
Err(protocol::DropReason::Superseded) => {
|
||||
// 被更新的帧追上了。这不是丢帧:半截的那张只是被超越,在健康链路上
|
||||
// 这是帧被放弃的**正常**方式。
|
||||
self.discovery.count(id, |s| s.datagrams += 1);
|
||||
}
|
||||
Err(protocol::DropReason::TimedOut) => {
|
||||
self.discovery.count(id, |s| {
|
||||
s.datagrams += 1;
|
||||
s.frames_dropped += 1;
|
||||
});
|
||||
}
|
||||
Err(_) => {}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -521,6 +532,25 @@ impl Receiver {
|
||||
}
|
||||
}
|
||||
|
||||
/// 全局广播用于发现;已知板子还需按 IP 续订,避免广播被 VPN / 虚拟网卡接走。
|
||||
fn subscription_targets(&self) -> Vec<SocketAddrV4> {
|
||||
let mut targets = vec![SocketAddrV4::new(self.camera, DEFAULT_PORT)];
|
||||
let only_sender = self.sender_limit();
|
||||
|
||||
for camera in self.discovery.online_cameras() {
|
||||
if only_sender.is_some_and(|id| camera.id != id) {
|
||||
continue;
|
||||
}
|
||||
// Hello 的来源端口不一定是订阅监听端口;协议固定使用 DEFAULT_PORT。
|
||||
let target = SocketAddrV4::new(*camera.addr.ip(), DEFAULT_PORT);
|
||||
if !targets.contains(&target) {
|
||||
targets.push(target);
|
||||
}
|
||||
}
|
||||
|
||||
targets
|
||||
}
|
||||
|
||||
/// 运行接收循环,直到 [`Receiver::shutdown`]。
|
||||
///
|
||||
/// 阻塞,所以放在单独的线程上。
|
||||
@@ -543,11 +573,14 @@ impl Receiver {
|
||||
if last_subscribe.elapsed() >= SUBSCRIBE_INTERVAL {
|
||||
let subscribe = Subscribe {
|
||||
role: self.role(),
|
||||
client_id: CLIENT_ID,
|
||||
client_id: self.client_id,
|
||||
};
|
||||
|
||||
if let Err(err) = socket.send_to(&subscribe.encode(), target) {
|
||||
log::warn!("could not subscribe at {target}: {err}");
|
||||
let encoded = subscribe.encode();
|
||||
for target in self.subscription_targets() {
|
||||
if let Err(err) = socket.send_to(&encoded, target) {
|
||||
log::warn!("could not subscribe at {target}: {err}");
|
||||
}
|
||||
}
|
||||
last_subscribe = Instant::now();
|
||||
}
|
||||
@@ -639,6 +672,48 @@ mod tests {
|
||||
rx.datagram(&hello.encode(MsgType::Hello), from(from_ip));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn discovered_boards_are_subscribed_directly_when_broadcast_uses_another_interface() {
|
||||
let rx = Receiver::new(Ipv4Addr::BROADCAST, SubscribeRole::Referee, None);
|
||||
let hello = Hello { sender_id: 1, firmware: 7 };
|
||||
// Hello 可从其它端口发来,订阅仍需发往板子的固定监听端口。
|
||||
rx.datagram(&hello.encode(MsgType::Hello), "192.168.1.139:12345".parse().unwrap());
|
||||
feed_hello(&rx, 2, [192, 168, 1, 140]);
|
||||
|
||||
assert_eq!(rx.subscription_targets(), vec![
|
||||
"255.255.255.255:5600".parse().unwrap(),
|
||||
"192.168.1.139:5600".parse().unwrap(),
|
||||
"192.168.1.140:5600".parse().unwrap(),
|
||||
]);
|
||||
|
||||
// DHCP 地址变化后只使用新地址。
|
||||
feed_hello(&rx, 1, [192, 168, 1, 141]);
|
||||
assert_eq!(rx.subscription_targets(), vec![
|
||||
"255.255.255.255:5600".parse().unwrap(),
|
||||
"192.168.1.141:5600".parse().unwrap(),
|
||||
"192.168.1.140:5600".parse().unwrap(),
|
||||
]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn discovery_of_other_teams_does_not_directly_subscribe_to_their_video() {
|
||||
let rx = Receiver::new(Ipv4Addr::BROADCAST, SubscribeRole::Team, Some(1));
|
||||
rx.set_discovery_all(true);
|
||||
feed_hello(&rx, 1, [192, 168, 1, 10]);
|
||||
feed_hello(&rx, 2, [192, 168, 1, 20]);
|
||||
|
||||
assert_eq!(rx.discovery().cameras().len(), 2);
|
||||
assert_eq!(rx.subscription_targets(), vec![
|
||||
"255.255.255.255:5600".parse().unwrap(),
|
||||
"192.168.1.10:5600".parse().unwrap(),
|
||||
]);
|
||||
rx.restrict_to(Some(2));
|
||||
assert_eq!(rx.subscription_targets(), vec![
|
||||
"255.255.255.255:5600".parse().unwrap(),
|
||||
"192.168.1.20:5600".parse().unwrap(),
|
||||
]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn delivers_the_latest_frame_of_the_bound_sender_only() {
|
||||
let rx = Receiver::new(Ipv4Addr::UNSPECIFIED, SubscribeRole::Team, Some(1));
|
||||
@@ -761,4 +836,21 @@ mod tests {
|
||||
|
||||
assert!(rx.discovery().cameras().is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn loss_statistics_include_superseded_frames_and_entirely_missing_ids() {
|
||||
let rx = Receiver::new(Ipv4Addr::UNSPECIFIED, SubscribeRole::Referee, None);
|
||||
feed_hello(&rx, 1, [192, 168, 1, 10]);
|
||||
feed_frame(&rx, 1, 1, [192, 168, 1, 10]);
|
||||
let partial = fragment_frame(1, 2, &jpeg(), 640, 480);
|
||||
rx.datagram(&partial[0].encode(), from([192, 168, 1, 10]));
|
||||
assert_eq!(rx.discovery().stats(1).frames_dropped, 0, "normal pending fragments are not drops");
|
||||
feed_frame(&rx, 1, 4, [192, 168, 1, 10]);
|
||||
let stats = rx.discovery().stats(1);
|
||||
assert_eq!(stats.frames_shown, 2);
|
||||
assert_eq!(stats.frames_dropped, 2, "partial 2 and entirely missing 3");
|
||||
assert_eq!(stats.loss_percent(), 50.0);
|
||||
for frag in partial { rx.datagram(&frag.encode(), from([192, 168, 1, 10])); }
|
||||
assert_eq!(rx.discovery().stats(1).frames_dropped, 2);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -10,7 +10,9 @@
|
||||
"preview": "vite preview",
|
||||
"tauri": "tauri",
|
||||
"app": "tauri dev",
|
||||
"check": "tsc --noEmit"
|
||||
"check": "tsc --noEmit",
|
||||
"test:stream": "node --test scripts/test-stream.mjs",
|
||||
"test:latency": "node --test scripts/test-latency-marker.mjs"
|
||||
},
|
||||
"dependencies": {
|
||||
"@tauri-apps/api": "^2",
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
// Observe the native client's local video channel without adding a UDP peer.
|
||||
// Read the token internally from a deliberately enabled debug log; never print it.
|
||||
import { readFile, writeFile, appendFile, mkdir, access } from 'node:fs/promises';
|
||||
import { dirname } from 'node:path';
|
||||
|
||||
const args = Object.fromEntries(process.argv.slice(2).reduce((pairs, value, i, all) => {
|
||||
if (i % 2 === 0) pairs.push([value.replace(/^--/, ''), all[i+1]]);
|
||||
return pairs;
|
||||
}, []));
|
||||
if (!args.log || !args.out) throw new Error('Use --log <backend-debug-log> --out <output-prefix> [--seconds 1800] [--warmup 30]');
|
||||
const seconds = Number(args.seconds ?? 300);
|
||||
const warmup = Number(args.warmup ?? 10);
|
||||
const sampleEveryMs = 1000*Number(args['sample-every'] ?? 0);
|
||||
const camera = Number(args.camera ?? '0x3D4D8A9A');
|
||||
await mkdir(dirname(args.out), { recursive: true });
|
||||
const log = await readFile(args.log, 'utf8');
|
||||
const url = [...log.matchAll(/ws:\/\/127\.0\.0\.1:\d+\/\?token=[A-Za-z0-9_-]+/g)].at(-1)?.[0];
|
||||
if (!url) throw new Error('No local video-channel URL in the debug log');
|
||||
const ws = new WebSocket(url);
|
||||
ws.binaryType = 'arraybuffer';
|
||||
let latest = null, demo = true, diagnostics = false, clients = 0, connected = false;
|
||||
let binary = 0, invalid = 0, lastArrival = null, gaps = [], bytes = 0;
|
||||
let firstFrame = null;
|
||||
let sampleCount = 0, nextSampleAt = 0;
|
||||
const allGaps = [], windows = [];
|
||||
let measurementStarted = null, previousAt = null, previousStats = null;
|
||||
let windowBinary = 0, windowInvalid = 0, windowBytes = 0;
|
||||
let totalFrames = 0, totalDropped = 0, counterResets = 0, socketCloses = 0;
|
||||
let offlineSnapshots = 0;
|
||||
let diagnosticsUnavailableSnapshots = 0;
|
||||
let finished = false;
|
||||
ws.addEventListener('close', () => { connected = false; if (!finished) socketCloses++; });
|
||||
ws.addEventListener('message', event => {
|
||||
if (typeof event.data === 'string') {
|
||||
const msg = JSON.parse(event.data);
|
||||
if (msg.t === 'snapshot') {
|
||||
latest = msg.stats.find(stat => stat.camera === camera) ?? null;
|
||||
demo = msg.demo;
|
||||
diagnostics = msg.diagnostics;
|
||||
clients = msg.clients;
|
||||
if (measurementStarted !== null && (!latest || !latest.online)) offlineSnapshots++;
|
||||
if (measurementStarted !== null && !diagnostics) diagnosticsUnavailableSnapshots++;
|
||||
}
|
||||
return;
|
||||
}
|
||||
const data = Buffer.from(event.data);
|
||||
if (data.length < 16 || data.readUInt32LE(0) !== camera) return;
|
||||
if (measurementStarted === null) return;
|
||||
const valid = data.length === data.readUInt32LE(12)+16 && data[16] === 255 && data[17] === 216 && data.at(-2) === 255 && data.at(-1) === 217;
|
||||
if (!valid) { invalid++; windowInvalid++; return; }
|
||||
const now = performance.now();
|
||||
if (lastArrival !== null) { gaps.push(now-lastArrival); allGaps.push(now-lastArrival); }
|
||||
lastArrival = now;
|
||||
binary++; windowBinary++;
|
||||
bytes += data.length-16; windowBytes += data.length-16;
|
||||
if (firstFrame === null) {
|
||||
firstFrame = Buffer.from(data.subarray(16));
|
||||
void writeFile(`${args.out}.first.jpg`, firstFrame).catch(error => console.error('Could not save sample JPEG:', error.message));
|
||||
}
|
||||
if (sampleEveryMs > 0 && now >= nextSampleAt) {
|
||||
nextSampleAt = now+sampleEveryMs;
|
||||
const path = `${args.out}.sample-${String(++sampleCount).padStart(4,'0')}.jpg`;
|
||||
void writeFile(path, data.subarray(16)).catch(error => console.error('Could not save diagnostic JPEG:', error.message));
|
||||
}
|
||||
});
|
||||
await Promise.race([
|
||||
new Promise((resolve, reject) => { ws.addEventListener('open', () => { connected = true; resolve(); }, { once:true }); ws.addEventListener('error', reject, { once:true }); }),
|
||||
new Promise((_, reject) => setTimeout(() => reject(new Error('Video channel open timeout')), 15_000)),
|
||||
]);
|
||||
await new Promise(resolve => setTimeout(resolve, warmup*1000));
|
||||
if (demo || !latest?.online || !latest?.hasFrames || clients < 2) throw new Error('Native GUI and online real hardware video are required before starting the soak');
|
||||
if (!diagnostics) throw new Error('Receiver counters require referee diagnostics; team release mode hides them');
|
||||
measurementStarted = previousAt = performance.now();
|
||||
previousStats = latest;
|
||||
await writeFile(`${args.out}.windows.ndjson`, '');
|
||||
|
||||
function percentile(values, p) {
|
||||
if (!values.length) return null;
|
||||
const sorted = [...values].sort((a,b) => a-b);
|
||||
return Number(sorted[Math.floor((sorted.length-1)*p)].toFixed(3));
|
||||
}
|
||||
async function sample() {
|
||||
const now = performance.now();
|
||||
const elapsed = (now-previousAt)/1000;
|
||||
let complete = 0, dropped = 0;
|
||||
if (latest && previousStats) {
|
||||
if (latest.framesShown < previousStats.framesShown || latest.framesDropped < previousStats.framesDropped) counterResets++;
|
||||
else { complete = latest.framesShown-previousStats.framesShown; dropped = latest.framesDropped-previousStats.framesDropped; }
|
||||
}
|
||||
totalFrames += complete; totalDropped += dropped;
|
||||
const row = {
|
||||
utc:new Date().toISOString(), elapsedSeconds:(now-measurementStarted)/1000, windowSeconds:elapsed,
|
||||
completeFrames:complete, framesDropped:dropped, receiverFps:complete/elapsed,
|
||||
frameLossPercent:100*dropped/Math.max(1,complete+dropped),
|
||||
binaryFrames:windowBinary, channelFps:windowBinary/elapsed, invalidJpegs:windowInvalid,
|
||||
meanJpegBytes:windowBinary ? windowBytes/windowBinary : 0,
|
||||
gapP95Ms:percentile(gaps,.95), gapMaxMs:Math.max(0,...gaps, lastArrival===null ? (now-measurementStarted) : now-lastArrival),
|
||||
cameraOnline:latest?.online ?? false, frontendConnected:clients>=2, observerConnected:connected,
|
||||
};
|
||||
windows.push(row);
|
||||
await appendFile(`${args.out}.windows.ndjson`, JSON.stringify(row)+'\n');
|
||||
await writeFile(`${args.out}.latest.json`, JSON.stringify(row,null,2)+'\n');
|
||||
console.log(JSON.stringify(row));
|
||||
previousAt = now;
|
||||
previousStats = latest;
|
||||
windowBinary = windowInvalid = windowBytes = 0;
|
||||
gaps = [];
|
||||
}
|
||||
let stoppedByRequest = false;
|
||||
while ((performance.now()-measurementStarted)/1000 < seconds) {
|
||||
if (args['stop-file'] && await access(args['stop-file']).then(() => true, () => false)) {
|
||||
stoppedByRequest = true;
|
||||
break;
|
||||
}
|
||||
const remaining = seconds-(performance.now()-measurementStarted)/1000;
|
||||
await new Promise(resolve => setTimeout(resolve, Math.min(1,remaining)*1000));
|
||||
if (performance.now()-previousAt >= 60_000 || (performance.now()-measurementStarted)/1000 >= seconds) await sample();
|
||||
}
|
||||
if (stoppedByRequest && performance.now()-previousAt > 1_000) await sample();
|
||||
finished = true;
|
||||
ws.close();
|
||||
const actualSeconds = (performance.now()-measurementStarted)/1000;
|
||||
const receiverMetricsValid = counterResets === 0 && diagnosticsUnavailableSnapshots === 0;
|
||||
const summary = {
|
||||
actualSeconds, requestedSeconds:seconds, stoppedByRequest, warmupSeconds:warmup, camera, demo, windows, binaryFrames:binary,
|
||||
invalidJpegs:invalid, receiverMetricsValid,
|
||||
completeFrames:receiverMetricsValid ? totalFrames : null, framesDropped:receiverMetricsValid ? totalDropped : null,
|
||||
rawWindowCountedCompleteFrames:totalFrames, rawWindowCountedDroppedFrames:totalDropped,
|
||||
receiverFps:receiverMetricsValid ? totalFrames/actualSeconds : null, channelFps:binary/actualSeconds,
|
||||
frameLossPercent:receiverMetricsValid ? 100*totalDropped/Math.max(1,totalFrames+totalDropped) : null,
|
||||
gapP95Ms:percentile(allGaps,.95), gapMaxMs:Math.max(0,...allGaps),
|
||||
meanJpegBytes:binary ? bytes/binary : 0, counterResets, socketCloses,
|
||||
offlineSnapshots,
|
||||
diagnosticsUnavailableSnapshots,
|
||||
diagnosticSamplesSaved:sampleCount,
|
||||
allWindowsOnline:windows.every(window => window.cameraOnline),
|
||||
zeroVideoWindows:windows.filter(window => window.binaryFrames === 0).length,
|
||||
noVideoSecondsAtEnd:lastArrival === null ? actualSeconds : (performance.now()-lastArrival)/1000,
|
||||
maxObservedSilenceMs:Math.max(0,...windows.map(window => window.gapMaxMs)),
|
||||
frontendConnected:clients>=2,
|
||||
measurementScope:'Native receiver and local JPEG WebSocket channel; not optical display latency',
|
||||
};
|
||||
await writeFile(`${args.out}.summary.json`, JSON.stringify(summary,null,2)+'\n');
|
||||
if (firstFrame) await writeFile(`${args.out}.first.jpg`, firstFrame);
|
||||
console.log(JSON.stringify({ ...summary, windows:windows.length }));
|
||||
if (demo || !binary || invalid || socketCloses || counterResets || offlineSnapshots || diagnosticsUnavailableSnapshots) process.exitCode = 1;
|
||||
@@ -0,0 +1,50 @@
|
||||
import {readFileSync} from 'node:fs';
|
||||
import assert from 'node:assert/strict';
|
||||
import {test} from 'node:test';
|
||||
import ts from 'typescript';
|
||||
const source=readFileSync(new URL('../src/lib/latency-marker.ts',import.meta.url),'utf8');
|
||||
const code=ts.transpileModule(source,{compilerOptions:{module:ts.ModuleKind.CommonJS,target:ts.ScriptTarget.ES2022}}).outputText;
|
||||
const exports={};new Function('exports',code)(exports);
|
||||
const {drawMarker,decodeMarker}=exports;
|
||||
function image(counter,w=320,h=240) {
|
||||
const pixels=new Uint8ClampedArray(w*h*4);
|
||||
const ctx={canvas:{width:w,height:h},fillStyle:'',fillRect(x,y,width,height){
|
||||
const c=this.fillStyle.startsWith('rgb(')?this.fillStyle.slice(4,-1).split(',').map(Number):
|
||||
this.fillStyle==='#fff'?[255,255,255]:this.fillStyle==='#000'?[0,0,0]:[60,60,60];
|
||||
for(let yy=Math.max(0,Math.ceil(y));yy<Math.min(h,y+height);yy++)
|
||||
for(let xx=Math.max(0,Math.ceil(x));xx<Math.min(w,x+width);xx++)pixels.set([...c,255],(yy*w+xx)*4);
|
||||
}};
|
||||
drawMarker(ctx,counter);return pixels;
|
||||
}
|
||||
test('screen IDs decode at Gray-code boundaries and across counter wrap',()=>{
|
||||
for(const id of [0,1,2,3,63,127,1023,2047,2048,4094,4095,4096]) {
|
||||
assert.equal(decodeMarker(image(id),320,240)?.counter,id&4095);
|
||||
}
|
||||
});
|
||||
test('camera rotation and reflection preserve the observed ID',()=>{
|
||||
const original=image(1371),rotated=new Uint8ClampedArray(original.length),flipped=new Uint8ClampedArray(original.length);
|
||||
for(let y=0;y<240;y++)for(let x=0;x<320;x++){
|
||||
rotated.set(original.subarray((y*320+x)*4,(y*320+x)*4+4),(x*240+239-y)*4);
|
||||
flipped.set(original.subarray((y*320+x)*4,(y*320+x)*4+4),(y*320+319-x)*4);
|
||||
}
|
||||
assert.equal(decodeMarker(rotated,240,320)?.counter,1371);
|
||||
assert.equal(decodeMarker(flipped,320,240)?.counter,1371);
|
||||
});
|
||||
test('missing fiducials and ambiguous rolling transitions do not invent latency samples',()=>{
|
||||
assert.equal(decodeMarker(new Uint8ClampedArray(320*240*4),320,240),null);
|
||||
const pixels=image(123);
|
||||
for(let y=90;y<152;y++)for(let x=44;x<276;x++)pixels.set([128,128,128,255],(y*320+x)*4);
|
||||
assert.equal(decodeMarker(pixels,320,240),null);
|
||||
});
|
||||
|
||||
test('washed screen primaries survive without mistaking a blue-tinted background',()=>{
|
||||
const pixels=image(1371);
|
||||
for(let i=0;i<pixels.length;i+=4){
|
||||
const r=pixels[i],g=pixels[i+1],b=pixels[i+2];
|
||||
if(r===32&&g===255&&b===32)pixels.set([190,255,208,255],i);
|
||||
else if(r===32&&g===96&&b===255)pixels.set([78,205,255,255],i);
|
||||
else if(r===255&&g===32&&b===255)pixels.set([255,112,255,255],i);
|
||||
else if(r===60&&g===60&&b===60)pixels.set([165,181,225,255],i);
|
||||
}
|
||||
assert.equal(decodeMarker(pixels,320,240)?.counter,1371);
|
||||
});
|
||||
@@ -0,0 +1,41 @@
|
||||
// Exercise the production StreamClient against native hardware video, while
|
||||
// deliberately stalling only this auxiliary local consumer. Adds no UDP peer.
|
||||
// Bitmap decoding is stubbed here; actual JPEG decoding is checked separately.
|
||||
import {readFileSync,writeFileSync} from 'node:fs';
|
||||
import assert from 'node:assert/strict';
|
||||
import ts from 'typescript';
|
||||
|
||||
const [logPath,outPath]=process.argv.slice(2);
|
||||
if(!logPath||!outPath)throw new Error('Use <private native debug log> <result JSON>');
|
||||
const log=readFileSync(logPath,'utf8');
|
||||
const url=[...log.matchAll(/ws:\/\/127\.0\.0\.1:\d+\/\?token=[A-Za-z0-9_-]+/g)].at(-1)?.[0];
|
||||
if(!url)throw new Error('No local video URL in the private debug log');
|
||||
const source=readFileSync(new URL('../src/lib/stream.ts',import.meta.url),'utf8');
|
||||
const compiled=ts.transpileModule(source,{compilerOptions:{module:ts.ModuleKind.CommonJS,target:ts.ScriptTarget.ES2022}}).outputText;
|
||||
const exports={};new Function('require','exports',compiled)(()=>({FRAME_HEADER_LEN:16}),exports);
|
||||
globalThis.window={setTimeout,clearTimeout};
|
||||
globalThis.createImageBitmap=async()=>({close(){}});
|
||||
const rx=new exports.StreamClient({onMessage(){},onConnection(){}});
|
||||
const delay=ms=>new Promise(resolve=>setTimeout(resolve,ms));
|
||||
try {
|
||||
rx.connect(url);
|
||||
const deadline=performance.now()+10_000;
|
||||
while(rx.counters.frames<10&&performance.now()<deadline)await delay(30);
|
||||
assert.ok(rx.counters.frames>=10,'Real native JPEG video did not arrive');
|
||||
const camera=[...rx.frames.keys()][0];
|
||||
const before=rx.latest(camera)?.frameId;
|
||||
const staleBefore=rx.counters.staleFrames;
|
||||
const blockedAt=performance.now();
|
||||
while(performance.now()-blockedAt<350) { /* synthetic consumer stall */ }
|
||||
await delay(700);
|
||||
const latest=rx.latest(camera);
|
||||
assert.ok(rx.counters.staleFrames>staleBefore,'Stale native-timestamped frames were not rejected');
|
||||
assert.ok(latest&&latest.frameId!==before,'The consumer did not recover to new video');
|
||||
assert.ok(performance.now()-latest.receivedAt<150,'Recovered image is still stale locally');
|
||||
const result={passed:true,stall_ms:350,stale_frames_rejected:rx.counters.staleFrames-staleBefore,
|
||||
frame_before:before,frame_after:latest.frameId,frames_received:rx.counters.frames,
|
||||
additional_udp_subscribers:0,jpeg_decoder_stubbed:true,
|
||||
scope:'Production StreamClient over real native WebSocket; tests queued-frame age and recovery, not physical display latency'};
|
||||
writeFileSync(outPath,JSON.stringify(result,null,2)+'\n');
|
||||
console.log(JSON.stringify(result));
|
||||
} finally {rx.close();}
|
||||
@@ -0,0 +1,143 @@
|
||||
import { readFileSync } from 'node:fs';
|
||||
import assert from 'node:assert/strict';
|
||||
import { test } from 'node:test';
|
||||
import ts from 'typescript';
|
||||
|
||||
const source = readFileSync(new URL('../src/lib/stream.ts', import.meta.url), 'utf8');
|
||||
const compiled = ts.transpileModule(source, { compilerOptions: { module: ts.ModuleKind.CommonJS, target: ts.ScriptTarget.ES2022 } }).outputText;
|
||||
const exports = {};
|
||||
new Function('require', 'exports', compiled)(name => {
|
||||
assert.equal(name, './types');
|
||||
return { FRAME_HEADER_LEN: 16 };
|
||||
}, exports);
|
||||
const { StreamClient } = exports;
|
||||
globalThis.window = { clearTimeout, setTimeout };
|
||||
const tick = () => new Promise(resolve => setImmediate(resolve));
|
||||
const bitmap = () => ({ closed: false, close() { this.closed = true; } });
|
||||
function frame(id, camera = 42) {
|
||||
const bytes = new ArrayBuffer(20);
|
||||
const header = new DataView(bytes);
|
||||
header.setUint32(0, camera, true);
|
||||
header.setUint32(4, id, true);
|
||||
header.setUint16(8, 640, true);
|
||||
header.setUint16(10, 480, true);
|
||||
header.setUint32(12, 4, true);
|
||||
return bytes;
|
||||
}
|
||||
function client(now, wallNow) { return new StreamClient({ onMessage() {}, onConnection() {} },now,wallNow); }
|
||||
function deferredDecoder() {
|
||||
const requests = [];
|
||||
globalThis.createImageBitmap = () => new Promise((resolve, reject) => requests.push({ resolve, reject }));
|
||||
return requests;
|
||||
}
|
||||
|
||||
test('a slow decoder keeps only the latest pending frame for each camera', async () => {
|
||||
const requests = deferredDecoder();
|
||||
const rx = client();
|
||||
const finished = rx.onBinary(frame(1));
|
||||
for (let id = 2; id <= 100; id++) await rx.onBinary(frame(id));
|
||||
assert.equal(requests.length, 1);
|
||||
assert.equal(rx.pending.size, 1);
|
||||
const first = bitmap();
|
||||
requests[0].resolve(first);
|
||||
await tick();
|
||||
assert.equal(requests.length, 2);
|
||||
requests[1].resolve(bitmap());
|
||||
await finished;
|
||||
assert.equal(rx.latest(42).frameId, 100);
|
||||
assert.equal(rx.counters.skippedDecodes, 98);
|
||||
assert.equal(first.closed, true);
|
||||
rx.close();
|
||||
});
|
||||
|
||||
test('two cameras decode independently without queues growing behind either', async () => {
|
||||
const requests = deferredDecoder();
|
||||
const rx = client();
|
||||
const a = rx.onBinary(frame(1, 1));
|
||||
const b = rx.onBinary(frame(2, 2));
|
||||
assert.equal(requests.length, 2);
|
||||
requests[1].resolve(bitmap());
|
||||
await b;
|
||||
assert.equal(rx.latest(2).frameId, 2);
|
||||
assert.equal(rx.latest(1), undefined);
|
||||
requests[0].resolve(bitmap());
|
||||
await a;
|
||||
rx.close();
|
||||
});
|
||||
|
||||
test('closing during decode releases the late bitmap and discards pending work', async () => {
|
||||
const requests = deferredDecoder();
|
||||
const rx = client();
|
||||
const work = rx.onBinary(frame(1));
|
||||
await rx.onBinary(frame(2));
|
||||
rx.close();
|
||||
const late = bitmap();
|
||||
requests[0].resolve(late);
|
||||
await work;
|
||||
assert.equal(late.closed, true);
|
||||
assert.equal(requests.length, 1);
|
||||
assert.equal(rx.latest(42), undefined);
|
||||
});
|
||||
|
||||
test('a corrupt JPEG does not stop the next frame from being decoded', async () => {
|
||||
const requests = deferredDecoder();
|
||||
const rx = client();
|
||||
const work = rx.onBinary(frame(1));
|
||||
await rx.onBinary(frame(2));
|
||||
requests[0].reject(new Error('bad JPEG'));
|
||||
await tick();
|
||||
requests[1].resolve(bitmap());
|
||||
await work;
|
||||
assert.equal(rx.counters.decodeFailures, 1);
|
||||
assert.equal(rx.latest(42).frameId, 2);
|
||||
rx.close();
|
||||
});
|
||||
|
||||
test('late frames are rejected but a reboot after silence can restart the counter', () => {
|
||||
const rx = client();
|
||||
rx.injectFrame(42, 100, 640, 480, bitmap());
|
||||
const old = bitmap();
|
||||
rx.injectFrame(42, 99, 640, 480, old);
|
||||
assert.equal(old.closed, true);
|
||||
assert.equal(rx.latest(42).frameId, 100);
|
||||
rx.latest(42).receivedAt -= 2000;
|
||||
rx.injectFrame(42, 1, 640, 480, bitmap());
|
||||
assert.equal(rx.latest(42).frameId, 1);
|
||||
rx.close();
|
||||
});
|
||||
|
||||
test('a WebView backlog is discarded before JPEG decoding using native host timing', async () => {
|
||||
const requests = deferredDecoder();
|
||||
const rx = client(() => 0, () => 2000);
|
||||
rx.onTiming({camera:42,frameId:1,publishedUnixMs:1500});
|
||||
await rx.onBinary(frame(1));
|
||||
assert.equal(requests.length,0);
|
||||
assert.equal(rx.counters.staleFrames,1);
|
||||
rx.onTiming({camera:42,frameId:2,publishedUnixMs:2000});
|
||||
const work = rx.onBinary(frame(2));
|
||||
requests[0].resolve(bitmap());await work;
|
||||
assert.equal(rx.latest(42).frameId,2);
|
||||
rx.close();
|
||||
});
|
||||
|
||||
test('a blocked decoder releases its old result and resumes with the fresh pending frame', async () => {
|
||||
let clock=0;
|
||||
const requests=deferredDecoder(),rx=client(()=>clock);
|
||||
const work=rx.onBinary(frame(1));
|
||||
clock=110;await rx.onBinary(frame(2));
|
||||
const old=bitmap();requests[0].resolve(old);await tick();
|
||||
assert.equal(old.closed,true);
|
||||
assert.equal(rx.latest(42),undefined);
|
||||
requests[1].resolve(bitmap());await work;
|
||||
assert.equal(rx.latest(42).frameId,2);
|
||||
assert.equal(rx.counters.staleFrames,1);rx.close();
|
||||
});
|
||||
|
||||
test('pending frames that expired during a decoder stall never reach the decoder', async () => {
|
||||
let clock=0;
|
||||
const requests=deferredDecoder(),rx=client(()=>clock);
|
||||
const work=rx.onBinary(frame(1));await rx.onBinary(frame(2));
|
||||
clock=200;const old=bitmap();requests[0].resolve(old);await work;
|
||||
assert.equal(old.closed,true);assert.equal(requests.length,1);
|
||||
assert.equal(rx.counters.staleFrames,2);assert.equal(rx.latest(42),undefined);rx.close();
|
||||
});
|
||||
Generated
+1
@@ -476,6 +476,7 @@ dependencies = [
|
||||
"protocol",
|
||||
"rand 0.9.5",
|
||||
"serde",
|
||||
"socket2",
|
||||
"toml 0.9.12+spec-1.1.0",
|
||||
]
|
||||
|
||||
|
||||
@@ -485,11 +485,7 @@ pub fn clear_config(password: Option<String>, state: Shared<'_>) -> Result<Strin
|
||||
|
||||
/// 退出程序。用户端要密码——现场"随手关掉"也是一种需要挡住的误操作。
|
||||
#[tauri::command]
|
||||
pub fn quit_app(
|
||||
password: Option<String>,
|
||||
app: AppHandle,
|
||||
state: Shared<'_>,
|
||||
) -> Result<(), String> {
|
||||
pub fn quit_app(password: Option<String>, app: AppHandle, state: Shared<'_>) -> Result<(), String> {
|
||||
state.require_unlocked(password.as_deref())?;
|
||||
|
||||
// 先把录制收尾,否则 ffmpeg 会留下一个没有索引的 mp4。
|
||||
|
||||
@@ -31,9 +31,7 @@ use std::time::Duration;
|
||||
|
||||
use protocol::{fragment_frame, ArmorHealth, MsgType};
|
||||
|
||||
use crate::state::{
|
||||
AppState, DEMO_ARMOR_A, DEMO_ARMOR_B, DEMO_CAMERA_A, DEMO_CAMERA_B,
|
||||
};
|
||||
use crate::state::{AppState, DEMO_ARMOR_A, DEMO_ARMOR_B, DEMO_CAMERA_A, DEMO_CAMERA_B};
|
||||
|
||||
/// 合成画面的尺寸。与实机一致(VGA),这样界面的缩放/比例在演示下也说得通。
|
||||
const WIDTH: u32 = 640;
|
||||
@@ -83,9 +81,7 @@ fn spawn_camera(state: Arc<AppState>, camera: u32, team: u8) {
|
||||
|
||||
// 板子每 2 秒广播一次在线状态,与固件一致。
|
||||
if last_hello.elapsed() >= Duration::from_secs(2) {
|
||||
state
|
||||
.receiver
|
||||
.datagram(&hello.encode(MsgType::Hello), from);
|
||||
state.receiver.datagram(&hello.encode(MsgType::Hello), from);
|
||||
last_hello = std::time::Instant::now();
|
||||
}
|
||||
|
||||
@@ -140,10 +136,8 @@ fn spawn_armor(state: Arc<AppState>) {
|
||||
hp[slot] = hp[slot].saturating_sub(damage);
|
||||
}
|
||||
|
||||
let from = SocketAddr::V4(SocketAddrV4::new(
|
||||
Ipv4Addr::new(127, 0, 0, 1),
|
||||
*port,
|
||||
));
|
||||
let from =
|
||||
SocketAddr::V4(SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), *port));
|
||||
|
||||
// 主板一次广播四条,整车一条血所以 hp 相同、index 0..3。
|
||||
for index in 0..4u8 {
|
||||
@@ -215,10 +209,38 @@ fn render(camera: u32, team: u8, frame_id: u32) -> Vec<u8> {
|
||||
// 中心的靶心:两个同心圆 + 四臂。瞄准器偏移校准时对着它看。
|
||||
ring(&mut img, cx, cy, 60, image::Rgb([200, 200, 210]));
|
||||
ring(&mut img, cx, cy, 28, image::Rgb([150, 150, 160]));
|
||||
arm(&mut img, cx - 90, cy, cx - 40, cy, image::Rgb([120, 120, 130]));
|
||||
arm(&mut img, cx + 40, cy, cx + 90, cy, image::Rgb([120, 120, 130]));
|
||||
arm(&mut img, cx, cy - 90, cx, cy - 40, image::Rgb([120, 120, 130]));
|
||||
arm(&mut img, cx, cy + 40, cx, cy + 90, image::Rgb([120, 120, 130]));
|
||||
arm(
|
||||
&mut img,
|
||||
cx - 90,
|
||||
cy,
|
||||
cx - 40,
|
||||
cy,
|
||||
image::Rgb([120, 120, 130]),
|
||||
);
|
||||
arm(
|
||||
&mut img,
|
||||
cx + 40,
|
||||
cy,
|
||||
cx + 90,
|
||||
cy,
|
||||
image::Rgb([120, 120, 130]),
|
||||
);
|
||||
arm(
|
||||
&mut img,
|
||||
cx,
|
||||
cy - 90,
|
||||
cx,
|
||||
cy - 40,
|
||||
image::Rgb([120, 120, 130]),
|
||||
);
|
||||
arm(
|
||||
&mut img,
|
||||
cx,
|
||||
cy + 40,
|
||||
cx,
|
||||
cy + 90,
|
||||
image::Rgb([120, 120, 130]),
|
||||
);
|
||||
|
||||
// 二进制帧号:16 个方块,最左边是低位。
|
||||
for bit in 0..16i32 {
|
||||
@@ -241,7 +263,14 @@ fn render(camera: u32, team: u8, frame_id: u32) -> Vec<u8> {
|
||||
|
||||
// 底部随时间右移的指示条:截图里能判断这一帧是不是较新的。
|
||||
let bar = ((t * 60.0) as i32).rem_euclid(WIDTH as i32 - 80);
|
||||
fill_rect(&mut img, bar, HEIGHT as i32 - 24, 60, 8, image::Rgb([90, 200, 255]));
|
||||
fill_rect(
|
||||
&mut img,
|
||||
bar,
|
||||
HEIGHT as i32 - 24,
|
||||
60,
|
||||
8,
|
||||
image::Rgb([90, 200, 255]),
|
||||
);
|
||||
|
||||
// 摄像头 id 的最低位用一个角标表示,进一步区分两路。
|
||||
let corner = if camera == DEMO_CAMERA_A { 0 } else { 1 };
|
||||
@@ -341,8 +370,8 @@ mod tests {
|
||||
assert!(jpeg.starts_with(&[0xFF, 0xD8]), "JPEG SOI");
|
||||
assert!(jpeg.ends_with(&[0xFF, 0xD9]), "JPEG EOI");
|
||||
|
||||
let decoded = image::load_from_memory_with_format(&jpeg, image::ImageFormat::Jpeg)
|
||||
.expect("decodes");
|
||||
let decoded =
|
||||
image::load_from_memory_with_format(&jpeg, image::ImageFormat::Jpeg).expect("decodes");
|
||||
|
||||
assert_eq!((decoded.width(), decoded.height()), (WIDTH, HEIGHT));
|
||||
}
|
||||
@@ -366,7 +395,11 @@ mod tests {
|
||||
fn a_frame_is_big_enough_to_be_a_realistic_load() {
|
||||
// 太小说明画的内容退化了(比如全是纯色),那分片/带宽的验证就失去意义。
|
||||
let jpeg = render(DEMO_CAMERA_A, 0, 1);
|
||||
assert!(jpeg.len() > 3_000, "suspiciously small: {} bytes", jpeg.len());
|
||||
assert!(
|
||||
jpeg.len() > 3_000,
|
||||
"suspiciously small: {} bytes",
|
||||
jpeg.len()
|
||||
);
|
||||
assert!(
|
||||
jpeg.len() < 200_000,
|
||||
"suspiciously large: {} bytes",
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
//! Opt-in, local-only calibration reports. No images, tokens or user config.
|
||||
use serde::Deserialize;
|
||||
use std::{
|
||||
fs::{File, OpenOptions},
|
||||
io::Write,
|
||||
path::PathBuf,
|
||||
sync::Mutex,
|
||||
};
|
||||
|
||||
pub struct Recorder {
|
||||
output: Mutex<Option<File>>,
|
||||
}
|
||||
impl Recorder {
|
||||
pub fn new(path: Option<PathBuf>) -> std::io::Result<Self> {
|
||||
let output = if let Some(path) = path {
|
||||
if let Some(parent) = path.parent() {
|
||||
std::fs::create_dir_all(parent)?;
|
||||
}
|
||||
Some(OpenOptions::new().write(true).create_new(true).open(path)?)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
Ok(Self {
|
||||
output: Mutex::new(output),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Deserialize, serde::Serialize)]
|
||||
#[serde(rename_all = "camelCase", deny_unknown_fields)]
|
||||
pub struct Sample {
|
||||
frame_id: u32,
|
||||
counter: u16,
|
||||
source_draw_ms: f64,
|
||||
draw_ms: f64,
|
||||
next_raf_ms: f64,
|
||||
lower_ms: f64,
|
||||
upper_ms: f64,
|
||||
decode_to_draw_ms: f64,
|
||||
analysis_ms: f64,
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub fn latency_enabled(recorder: tauri::State<'_, Recorder>) -> bool {
|
||||
recorder
|
||||
.output
|
||||
.lock()
|
||||
.expect("latency output mutex")
|
||||
.is_some()
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub fn latency_sample(sample: Sample, recorder: tauri::State<'_, Recorder>) -> Result<(), String> {
|
||||
if ![
|
||||
sample.source_draw_ms,
|
||||
sample.draw_ms,
|
||||
sample.next_raf_ms,
|
||||
sample.lower_ms,
|
||||
sample.upper_ms,
|
||||
sample.decode_to_draw_ms,
|
||||
sample.analysis_ms,
|
||||
]
|
||||
.iter()
|
||||
.all(|v| v.is_finite())
|
||||
|| sample.lower_ms < 0.0
|
||||
|| sample.upper_ms < sample.lower_ms
|
||||
|| sample.upper_ms > 5000.0
|
||||
{
|
||||
return Err("Invalid latency sample".into());
|
||||
}
|
||||
let mut guard = recorder.output.lock().map_err(|_| "latency output mutex")?;
|
||||
let out = guard.as_mut().ok_or("Latency calibration is disabled")?;
|
||||
serde_json::to_writer(&mut *out, &sample).map_err(|e| e.to_string())?;
|
||||
out.write_all(b"\n")
|
||||
.and_then(|_| out.flush())
|
||||
.map_err(|e| e.to_string())
|
||||
}
|
||||
@@ -23,6 +23,7 @@
|
||||
|
||||
mod commands;
|
||||
mod demo;
|
||||
mod latency;
|
||||
mod recording;
|
||||
mod server;
|
||||
mod state;
|
||||
@@ -42,6 +43,7 @@ const USAGE: &str = "\
|
||||
|
||||
--role <team|referee> 身份。默认看配置文件;没有配置文件时是 referee
|
||||
--config <路径> 配置文件位置
|
||||
--latency-report <路径> 打开裁判端屏幕标记校准,写入新的本地 NDJSON 报告
|
||||
--demo 演示模式:画面与血量都是本地合成的
|
||||
--list 打印发现的摄像头与装甲板后退出
|
||||
-h, --help 显示这段帮助
|
||||
@@ -78,6 +80,7 @@ fn main() {
|
||||
struct Args {
|
||||
role: Option<Role>,
|
||||
config: Option<PathBuf>,
|
||||
latency_report: Option<PathBuf>,
|
||||
demo: bool,
|
||||
list: bool,
|
||||
help: bool,
|
||||
@@ -104,6 +107,9 @@ impl Args {
|
||||
"--config" => {
|
||||
args.config = it.next().map(PathBuf::from);
|
||||
}
|
||||
"--latency-report" => {
|
||||
args.latency_report = it.next().map(PathBuf::from);
|
||||
}
|
||||
other => {
|
||||
// 见模块文档:macOS 的 .app 启动参数与 shell 里的笔误都可能出现在这里,
|
||||
// 但报错的代价(窗口起不来)远大于收益。
|
||||
@@ -124,7 +130,10 @@ fn run() -> Result<(), Box<dyn Error>> {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let config_path = args.config.clone().unwrap_or_else(config::default_config_path);
|
||||
let config_path = args
|
||||
.config
|
||||
.clone()
|
||||
.unwrap_or_else(config::default_config_path);
|
||||
|
||||
// 首次启动时导入旧客户端的配置,并把旧文件改名成 .bak。
|
||||
let (mut config, notes) = config::load_with_migration(&config_path)?;
|
||||
@@ -134,6 +143,10 @@ fn run() -> Result<(), Box<dyn Error>> {
|
||||
if let Some(role) = args.role {
|
||||
config.role = role;
|
||||
}
|
||||
if args.latency_report.is_some() && config.role != Role::Referee {
|
||||
return Err("Latency calibration requires a referee test configuration".into());
|
||||
}
|
||||
let latency_recorder = latency::Recorder::new(args.latency_report.clone())?;
|
||||
|
||||
for note in ¬es {
|
||||
log::info!("{note}");
|
||||
@@ -150,8 +163,7 @@ fn run() -> Result<(), Box<dyn Error>> {
|
||||
config_path.display()
|
||||
);
|
||||
|
||||
let (state, video_socket, armor_socket) =
|
||||
AppState::new(config_path, config, notes, args.demo)?;
|
||||
let (state, video_socket, armor_socket) = AppState::new(config_path, config, notes, args.demo)?;
|
||||
|
||||
// 视频端口绑不上时没有 socket,接收线程也就没有意义——界面会显示那条说明。
|
||||
if let Some(socket) = video_socket {
|
||||
@@ -219,7 +231,10 @@ fn run() -> Result<(), Box<dyn Error>> {
|
||||
let app = tauri::Builder::default()
|
||||
.plugin(tauri_plugin_opener::init())
|
||||
.manage(Arc::clone(&state))
|
||||
.manage(latency_recorder)
|
||||
.invoke_handler(tauri::generate_handler![
|
||||
latency::latency_enabled,
|
||||
latency::latency_sample,
|
||||
commands::stream_info,
|
||||
commands::get_snapshot,
|
||||
commands::get_config,
|
||||
|
||||
@@ -99,9 +99,7 @@ impl Recorder {
|
||||
.stderr(Stdio::piped())
|
||||
.spawn()
|
||||
.with_context(|| {
|
||||
format!(
|
||||
"could not start ffmpeg ({ffmpeg}); 用 [recording] ffmpeg 指定它的路径"
|
||||
)
|
||||
format!("could not start ffmpeg ({ffmpeg}); 用 [recording] ffmpeg 指定它的路径")
|
||||
})?;
|
||||
|
||||
let stdin = child
|
||||
@@ -124,8 +122,7 @@ impl Recorder {
|
||||
.spawn(move || {
|
||||
if let Err(err) = pump(&rx, stdin, &mut child, &frames_written) {
|
||||
log::error!("recording {} stopped: {err:#}", path.display());
|
||||
*error.lock().expect("rec error mutex poisoned") =
|
||||
Some(format!("{err:#}"));
|
||||
*error.lock().expect("rec error mutex poisoned") = Some(format!("{err:#}"));
|
||||
}
|
||||
})?
|
||||
};
|
||||
@@ -191,7 +188,10 @@ impl Recorder {
|
||||
dto.active = false;
|
||||
|
||||
if let Some(error) = dto.error.as_deref() {
|
||||
log::warn!("recording 0x{:08X} finished with an error: {error}", self.camera);
|
||||
log::warn!(
|
||||
"recording 0x{:08X} finished with an error: {error}",
|
||||
self.camera
|
||||
);
|
||||
} else {
|
||||
log::info!(
|
||||
"recording 0x{:08X} stopped: {} frames -> {}",
|
||||
@@ -291,7 +291,9 @@ fn pump(
|
||||
let reason = child_stderr(child);
|
||||
return Err(anyhow!(
|
||||
"writing to ffmpeg failed: {err}{}",
|
||||
reason.map(|r| format!(";ffmpeg 说:{r}")).unwrap_or_default()
|
||||
reason
|
||||
.map(|r| format!(";ffmpeg 说:{r}"))
|
||||
.unwrap_or_default()
|
||||
));
|
||||
}
|
||||
|
||||
@@ -356,7 +358,11 @@ impl RecordingSet {
|
||||
// 文件是它启动之后才创建的。先开新的话,新的一段会挑到与上一段相同的名字,然后被
|
||||
// 它的 `-y` 覆盖掉——而上一段刚要收尾。先停就能保证上一个文件已经落地,`exists()`
|
||||
// 于是是可信的。
|
||||
let previous = self.inner.lock().expect("recording mutex poisoned").remove(&camera);
|
||||
let previous = self
|
||||
.inner
|
||||
.lock()
|
||||
.expect("recording mutex poisoned")
|
||||
.remove(&camera);
|
||||
|
||||
if let Some(previous) = previous {
|
||||
log::warn!("0x{camera:08X} was already recording; the previous take is being closed");
|
||||
@@ -399,7 +405,12 @@ impl RecordingSet {
|
||||
/// 把一帧喂给对应那一路的录制器。在 UDP 接收线程里被调用,所以必须快。
|
||||
pub fn feed(&self, frame: &Arc<Frame>) {
|
||||
// 先快速判断有没有在录,避免每帧都抢锁。
|
||||
if self.inner.lock().expect("recording mutex poisoned").is_empty() {
|
||||
if self
|
||||
.inner
|
||||
.lock()
|
||||
.expect("recording mutex poisoned")
|
||||
.is_empty()
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -498,8 +509,7 @@ mod tests {
|
||||
}
|
||||
|
||||
let mut out = Vec::new();
|
||||
let mut encoder =
|
||||
image::codecs::jpeg::JpegEncoder::new_with_quality(&mut out, 80);
|
||||
let mut encoder = image::codecs::jpeg::JpegEncoder::new_with_quality(&mut out, 80);
|
||||
encoder.encode_image(&img).expect("encodes");
|
||||
out
|
||||
}
|
||||
@@ -525,10 +535,8 @@ mod tests {
|
||||
}
|
||||
|
||||
fn temp_dir(tag: &str) -> PathBuf {
|
||||
let dir = std::env::temp_dir().join(format!(
|
||||
"cam-desktop-rec-{tag}-{}",
|
||||
std::process::id()
|
||||
));
|
||||
let dir =
|
||||
std::env::temp_dir().join(format!("cam-desktop-rec-{tag}-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
std::fs::create_dir_all(&dir).expect("creates the temp dir");
|
||||
dir
|
||||
@@ -585,7 +593,10 @@ mod tests {
|
||||
|
||||
let path = dto.path.expect("has a path");
|
||||
let meta = std::fs::metadata(&path).expect("the file exists");
|
||||
assert!(meta.len() > 0, "an empty container is a failure, not a take");
|
||||
assert!(
|
||||
meta.len() > 0,
|
||||
"an empty container is a failure, not a take"
|
||||
);
|
||||
|
||||
if let Some((frames, width, height)) = ffprobe_frames(&path) {
|
||||
assert_eq!(frames, 20, "ffmpeg wrote every frame we fed it");
|
||||
@@ -704,7 +715,10 @@ mod tests {
|
||||
let decoded = image::open(&path).expect("the PNG is readable");
|
||||
assert_eq!((decoded.width(), decoded.height()), (64, 48));
|
||||
assert!(
|
||||
path.file_name().unwrap().to_string_lossy().starts_with("shot-3D4D8A9A-"),
|
||||
path.file_name()
|
||||
.unwrap()
|
||||
.to_string_lossy()
|
||||
.starts_with("shot-3D4D8A9A-"),
|
||||
"文件名要能看出是哪一路截的:{path:?}"
|
||||
);
|
||||
|
||||
@@ -727,7 +741,10 @@ mod tests {
|
||||
let forced = unique_path(&dir, "shot-00000001-fixed", "png");
|
||||
std::fs::write(&forced, b"").unwrap();
|
||||
let forced2 = unique_path(&dir, "shot-00000001-fixed", "png");
|
||||
assert!(forced2.to_string_lossy().ends_with("-2.png"), "got {forced2:?}");
|
||||
assert!(
|
||||
forced2.to_string_lossy().ends_with("-2.png"),
|
||||
"got {forced2:?}"
|
||||
);
|
||||
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
@@ -755,7 +772,10 @@ mod tests {
|
||||
|
||||
assert_eq!(names.len(), 2, "got {names:?}");
|
||||
assert!(names.contains(&"a.mp4"));
|
||||
assert!(names.contains(&"b.PNG"), "后缀大小写不该决定它是不是一张截图");
|
||||
assert!(
|
||||
names.contains(&"b.PNG"),
|
||||
"后缀大小写不该决定它是不是一张截图"
|
||||
);
|
||||
assert_eq!(files[0].kind, "video");
|
||||
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
|
||||
+179
-15
@@ -29,14 +29,15 @@
|
||||
//! 小端。16 字节头,前端切掉它就能拿到一段完整的 JPEG。`sender_id` 让前端知道这一帧是
|
||||
//! 哪块板子的——用户端只画绑定的那一路,裁判端按槽位分配画两路。
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};
|
||||
|
||||
use futures_util::{SinkExt, StreamExt};
|
||||
use serde::Serialize;
|
||||
use tokio::net::{TcpListener, TcpStream};
|
||||
use tokio::sync::broadcast;
|
||||
use tokio::sync::{broadcast, watch};
|
||||
use tokio_tungstenite::tungstenite::handshake::server::{ErrorResponse, Request, Response};
|
||||
use tokio_tungstenite::tungstenite::http::StatusCode;
|
||||
use tokio_tungstenite::tungstenite::{Bytes, Message};
|
||||
@@ -46,24 +47,43 @@ use cam_core::snapshot::{HitDto, Snapshot};
|
||||
/// 二进制帧头长度。
|
||||
pub const FRAME_HEADER_LEN: usize = 16;
|
||||
|
||||
/// 广播通道的容量。
|
||||
///
|
||||
/// 满了之后 `send` 会挤掉最老的(接收慢的连接会拿到 `Lagged`)——对实时画面这正是想要
|
||||
/// 的:落后 200 帧的客户端需要的是最新的那一帧,不是补齐历史。按每帧 16+22 kB 算,
|
||||
/// 256 个槽位约 5.6 MB 上限。
|
||||
/// Ordered control-message capacity. JPEGs use a separate latest-value channel.
|
||||
pub const CHANNEL_CAPACITY: usize = 256;
|
||||
const LOCAL_VIDEO_MAX_AGE: Duration = Duration::from_millis(100);
|
||||
|
||||
/// 向界面推送的消息。
|
||||
pub enum ServerMsg {
|
||||
/// 一帧原始 JPEG(已经拼好 16 字节头)。
|
||||
///
|
||||
/// 载荷用 `Bytes` 而不是 `Arc<Vec<u8>>`:它本身就是引用计数的缓冲,克隆是加一次
|
||||
/// 引用计数,而且正好是 tungstenite 要的类型——每个连接一次多余的拷贝就此消失。
|
||||
Binary(Bytes),
|
||||
/// JSON 文本。
|
||||
Text(String),
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct VideoPacket {
|
||||
frame_id: u32,
|
||||
bytes: Bytes,
|
||||
published_at: Instant,
|
||||
published_unix_ms: u64,
|
||||
}
|
||||
|
||||
pub type VideoFrames = HashMap<u32, VideoPacket>;
|
||||
pub type VideoSender = watch::Sender<VideoFrames>;
|
||||
|
||||
/// Video uses latest-value semantics; control messages keep ordered delivery.
|
||||
pub fn publish_video(tx: &VideoSender, frame: &protocol::Frame) {
|
||||
let packet = VideoPacket {
|
||||
frame_id: frame.frame_id,
|
||||
bytes: encode_frame(frame).into(),
|
||||
published_at: Instant::now(),
|
||||
published_unix_ms: SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_millis() as u64,
|
||||
};
|
||||
tx.send_modify(|latest| {
|
||||
latest.insert(frame.sender_id, packet);
|
||||
});
|
||||
}
|
||||
|
||||
/// JSON 文本消息的信封。
|
||||
///
|
||||
/// 用内部标签 `t` 而不是两个字段:前端一个 `switch (msg.t)` 就能分发,且不需要为每种
|
||||
@@ -82,6 +102,14 @@ pub enum WireMsg {
|
||||
/// 写成带字段的结构体而不是 `Note(String)`:serde 的**内部标签**(`#[serde(tag)]`)
|
||||
/// 只支持载荷是 map 的变体,newtype 里放一个字符串会在运行时报错,而不是编译期。
|
||||
Note { message: String },
|
||||
/// Timestamp precedes its JPEG so a stalled WebView can reject queued data.
|
||||
FrameTiming {
|
||||
camera: u32,
|
||||
#[serde(rename = "frameId")]
|
||||
frame_id: u32,
|
||||
#[serde(rename = "publishedUnixMs")]
|
||||
published_unix_ms: u64,
|
||||
},
|
||||
}
|
||||
|
||||
/// 前端连接这条通道需要的信息。
|
||||
@@ -144,6 +172,7 @@ pub async fn serve(
|
||||
listener: TcpListener,
|
||||
token: String,
|
||||
tx: broadcast::Sender<Arc<ServerMsg>>,
|
||||
video: VideoSender,
|
||||
clients: Arc<AtomicUsize>,
|
||||
) {
|
||||
loop {
|
||||
@@ -159,9 +188,10 @@ pub async fn serve(
|
||||
let tx = tx.clone();
|
||||
let token = token.clone();
|
||||
let clients = Arc::clone(&clients);
|
||||
let video = video.subscribe();
|
||||
|
||||
tauri::async_runtime::spawn(async move {
|
||||
if let Err(err) = handle_client(stream, peer, &token, tx, clients).await {
|
||||
if let Err(err) = handle_client(stream, peer, &token, tx, video, clients).await {
|
||||
log::debug!("stream: client {peer} ended: {err}");
|
||||
}
|
||||
});
|
||||
@@ -173,6 +203,7 @@ async fn handle_client(
|
||||
peer: std::net::SocketAddr,
|
||||
token: &str,
|
||||
tx: broadcast::Sender<Arc<ServerMsg>>,
|
||||
mut video: watch::Receiver<VideoFrames>,
|
||||
clients: Arc<AtomicUsize>,
|
||||
) -> anyhow::Result<()> {
|
||||
stream.set_nodelay(true).ok();
|
||||
@@ -201,6 +232,8 @@ async fn handle_client(
|
||||
let (mut sink, mut incoming) = ws.split();
|
||||
|
||||
let mut rx = tx.subscribe();
|
||||
video.mark_changed();
|
||||
let mut last_sent = HashMap::new();
|
||||
let mut keepalive = tokio::time::interval(Duration::from_secs(20));
|
||||
let count = clients.fetch_add(1, Ordering::Relaxed) + 1;
|
||||
log::info!("stream: interface connected from {peer} ({count} connected)");
|
||||
@@ -212,12 +245,35 @@ async fn handle_client(
|
||||
.unwrap_or_else(|_| "{}".into());
|
||||
let _ = sink.send(Message::text(hello)).await;
|
||||
|
||||
let result = loop {
|
||||
let result = 'messages: loop {
|
||||
tokio::select! {
|
||||
changed = video.changed() => {
|
||||
if changed.is_err() { break Ok(()); }
|
||||
// Release the watch read lock before awaiting socket writes.
|
||||
let latest: Vec<_> = video.borrow_and_update().iter()
|
||||
.map(|(&camera, packet)| (camera, packet.clone())).collect();
|
||||
for (camera, packet) in latest {
|
||||
if last_sent.get(&camera) == Some(&packet.frame_id) { continue; }
|
||||
if packet.published_at.elapsed() >= LOCAL_VIDEO_MAX_AGE { continue; }
|
||||
let timing = serde_json::to_string(&WireMsg::FrameTiming {
|
||||
camera, frame_id: packet.frame_id, published_unix_ms: packet.published_unix_ms,
|
||||
})?;
|
||||
// Keep the 16-byte JPEG header compatible. This small text
|
||||
// message carries host queue age through TCP/WebView queues.
|
||||
let send = async {
|
||||
sink.send(Message::text(timing)).await?;
|
||||
sink.send(Message::Binary(packet.bytes)).await
|
||||
};
|
||||
match tokio::time::timeout(LOCAL_VIDEO_MAX_AGE, send).await {
|
||||
Ok(Ok(())) => { last_sent.insert(camera, packet.frame_id); }
|
||||
Ok(Err(err)) => break 'messages Err(anyhow::anyhow!("video send failed: {err}")),
|
||||
Err(_) => break 'messages Err(anyhow::anyhow!("video client stalled")),
|
||||
}
|
||||
}
|
||||
},
|
||||
message = rx.recv() => match message {
|
||||
Ok(msg) => {
|
||||
let out = match &*msg {
|
||||
ServerMsg::Binary(bytes) => Message::Binary(bytes.clone()),
|
||||
ServerMsg::Text(text) => Message::text(text.clone()),
|
||||
};
|
||||
|
||||
@@ -334,6 +390,114 @@ mod tests {
|
||||
assert_eq!(random_token().len(), 48);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_slow_video_consumer_keeps_only_the_latest_frame_of_each_camera() {
|
||||
let (video, mut reader) = watch::channel(VideoFrames::new());
|
||||
for id in 1..=1000 {
|
||||
publish_video(&video, &frame(1, id, 100));
|
||||
publish_video(&video, &frame(2, id + 10, 100));
|
||||
}
|
||||
let latest = reader.borrow_and_update();
|
||||
assert_eq!(latest.len(), 2);
|
||||
assert_eq!(latest[&1].frame_id, 1000);
|
||||
assert_eq!(latest[&2].frame_id, 1010);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_reconnecting_interface_receives_cached_latest_frames_without_waiting_for_capture() {
|
||||
let (listener, port) = bind().await.unwrap();
|
||||
let (control, _) = broadcast::channel(CHANNEL_CAPACITY);
|
||||
let (video, _) = watch::channel(VideoFrames::new());
|
||||
for id in 1..=100 {
|
||||
publish_video(&video, &frame(1, id, 100));
|
||||
publish_video(&video, &frame(2, id + 10, 100));
|
||||
}
|
||||
let task = tokio::spawn(serve(
|
||||
listener,
|
||||
"test-token".into(),
|
||||
control,
|
||||
video,
|
||||
Arc::new(AtomicUsize::new(0)),
|
||||
));
|
||||
let (mut client, _) =
|
||||
tokio_tungstenite::connect_async(format!("ws://127.0.0.1:{port}/?token=test-token"))
|
||||
.await
|
||||
.unwrap();
|
||||
let mut received = Vec::new();
|
||||
let mut timings = HashMap::new();
|
||||
while received.len() < 2 {
|
||||
let message = tokio::time::timeout(Duration::from_secs(2), client.next())
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
if let Message::Text(text) = &message {
|
||||
let value: serde_json::Value = serde_json::from_str(text).unwrap();
|
||||
if value["t"] == "frameTiming" {
|
||||
assert!(value["publishedUnixMs"].as_u64().unwrap() > 0);
|
||||
timings.insert(
|
||||
value["camera"].as_u64().unwrap() as u32,
|
||||
value["frameId"].as_u64().unwrap() as u32,
|
||||
);
|
||||
}
|
||||
}
|
||||
if let Message::Binary(bytes) = message {
|
||||
let (camera, frame_id, _, _, _) = decode_frame_header(&bytes).unwrap();
|
||||
assert_eq!(timings.get(&camera), Some(&frame_id));
|
||||
received.push((camera, frame_id));
|
||||
}
|
||||
}
|
||||
received.sort_unstable();
|
||||
assert_eq!(received, vec![(1, 100), (2, 110)]);
|
||||
client.close(None).await.unwrap();
|
||||
task.abort();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn reconnect_does_not_replay_expired_cached_video() {
|
||||
let (listener, port) = bind().await.unwrap();
|
||||
let (control, _) = broadcast::channel(CHANNEL_CAPACITY);
|
||||
let (video, _) = watch::channel(VideoFrames::new());
|
||||
publish_video(&video, &frame(1, 1, 100));
|
||||
video.send_modify(|latest| {
|
||||
latest.get_mut(&1).unwrap().published_at = Instant::now() - Duration::from_secs(1)
|
||||
});
|
||||
let task = tokio::spawn(serve(
|
||||
listener,
|
||||
"test-token".into(),
|
||||
control,
|
||||
video.clone(),
|
||||
Arc::new(AtomicUsize::new(0)),
|
||||
));
|
||||
let (mut client, _) =
|
||||
tokio_tungstenite::connect_async(format!("ws://127.0.0.1:{port}/?token=test-token"))
|
||||
.await
|
||||
.unwrap();
|
||||
let stale = tokio::time::timeout(Duration::from_millis(100), async {
|
||||
while let Some(message) = client.next().await {
|
||||
if matches!(message.unwrap(), Message::Binary(_)) {
|
||||
return;
|
||||
}
|
||||
}
|
||||
})
|
||||
.await;
|
||||
assert!(stale.is_err());
|
||||
publish_video(&video, &frame(1, 2, 100));
|
||||
let fresh = tokio::time::timeout(Duration::from_secs(2), async {
|
||||
while let Some(message) = client.next().await {
|
||||
if let Message::Binary(bytes) = message.unwrap() {
|
||||
return decode_frame_header(&bytes).unwrap().1;
|
||||
}
|
||||
}
|
||||
panic!("video connection closed");
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(fresh, 2);
|
||||
client.close(None).await.unwrap();
|
||||
task.abort();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_wire_envelope_carries_the_message_kind() {
|
||||
let json = serde_json::to_string(&WireMsg::Hit(HitDto {
|
||||
|
||||
@@ -24,14 +24,14 @@ use cam_core::armor::{ArmorHealthStore, HealthTracker};
|
||||
use cam_core::config::{Config, Role};
|
||||
use cam_core::password;
|
||||
use cam_core::receiver::{Discovery, LatestFrames, Receiver};
|
||||
use cam_core::ArmorSource;
|
||||
use cam_core::slots::SlotAllocator;
|
||||
use cam_core::snapshot::{
|
||||
resolve_outcomes, ArmorBoardDto, CameraDto, CameraStatsDto, ConfigView, HitDto, Snapshot,
|
||||
SlotsDto, TeamHealthDto,
|
||||
resolve_outcomes, ArmorBoardDto, CameraDto, CameraStatsDto, ConfigView, HitDto, SlotsDto,
|
||||
Snapshot, TeamHealthDto,
|
||||
};
|
||||
use cam_core::stats::{FpsWindow, DEFAULT_WINDOW};
|
||||
use tokio::sync::broadcast;
|
||||
use cam_core::ArmorSource;
|
||||
use tokio::sync::{broadcast, watch};
|
||||
|
||||
use crate::recording::{now_ms, RecordingSet};
|
||||
use crate::server::{self, ServerMsg, WireMsg};
|
||||
@@ -67,6 +67,7 @@ pub struct AppState {
|
||||
pub recordings: Arc<RecordingSet>,
|
||||
|
||||
tx: broadcast::Sender<Arc<ServerMsg>>,
|
||||
video: server::VideoSender,
|
||||
/// 回环上已经绑好的 WebSocket listener,等 `spawn_background` 取走。
|
||||
///
|
||||
/// 先绑再交给后台任务,是为了让"端口绑不上"在启动阶段就报错,而不是变成一个永远
|
||||
@@ -92,38 +93,43 @@ impl AppState {
|
||||
config: Config,
|
||||
mut notes: Vec<String>,
|
||||
demo: bool,
|
||||
) -> Result<(Arc<Self>, Option<std::net::UdpSocket>, Option<std::net::UdpSocket>)> {
|
||||
) -> Result<(
|
||||
Arc<Self>,
|
||||
Option<std::net::UdpSocket>,
|
||||
Option<std::net::UdpSocket>,
|
||||
)> {
|
||||
let role = config.role;
|
||||
let bound = if role == Role::Team { config.camera_id } else { None };
|
||||
let bound = if role == Role::Team {
|
||||
config.camera_id
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
// 视频端口绑不上**不**致命:界面照常起来,并把原因写在最显眼的地方。
|
||||
//
|
||||
// 一开始这里是 `?`(直接退出)。但那样一来,现场看到的是一次闪退加上一句只出现
|
||||
// 在 stderr 的话——从访达双击启动时连 stderr 都没有。让窗口起来并说明原因,比
|
||||
// 静默退出有用得多;而且"另一个客户端已经在跑"恰好是最常见的原因。
|
||||
let (receiver, video_socket) = match Receiver::start(
|
||||
protocol::BROADCAST_ADDR,
|
||||
role.to_protocol(),
|
||||
bound,
|
||||
) {
|
||||
Ok((receiver, socket)) => (Arc::new(receiver), Some(socket)),
|
||||
Err(err) => {
|
||||
log::error!("could not bind UDP {}: {err}", protocol::DEFAULT_PORT);
|
||||
notes.push(format!(
|
||||
"绑不上 UDP {}:{err}。多半是同一个端口上已经跑着另一个客户端 \
|
||||
let (receiver, video_socket) =
|
||||
match Receiver::start(protocol::BROADCAST_ADDR, role.to_protocol(), bound) {
|
||||
Ok((receiver, socket)) => (Arc::new(receiver), Some(socket)),
|
||||
Err(err) => {
|
||||
log::error!("could not bind UDP {}: {err}", protocol::DEFAULT_PORT);
|
||||
notes.push(format!(
|
||||
"绑不上 UDP {}:{err}。多半是同一个端口上已经跑着另一个客户端 \
|
||||
(旧的 Slint 版或另一份本程序)。画面收不到,其它功能照常。",
|
||||
protocol::DEFAULT_PORT
|
||||
));
|
||||
(
|
||||
Arc::new(Receiver::new(
|
||||
protocol::BROADCAST_ADDR,
|
||||
role.to_protocol(),
|
||||
bound,
|
||||
)),
|
||||
None,
|
||||
)
|
||||
}
|
||||
};
|
||||
protocol::DEFAULT_PORT
|
||||
));
|
||||
(
|
||||
Arc::new(Receiver::new(
|
||||
protocol::BROADCAST_ADDR,
|
||||
role.to_protocol(),
|
||||
bound,
|
||||
)),
|
||||
None,
|
||||
)
|
||||
}
|
||||
};
|
||||
|
||||
let discovery = receiver.discovery();
|
||||
let latest = receiver.latest_frames();
|
||||
@@ -141,6 +147,7 @@ impl AppState {
|
||||
};
|
||||
|
||||
let (tx, _) = broadcast::channel(server::CHANNEL_CAPACITY);
|
||||
let (video, _) = watch::channel(server::VideoFrames::new());
|
||||
let (listener, port) = tauri::async_runtime::block_on(server::bind())
|
||||
.context("绑不上本地 WebSocket 端口(回环地址)")?;
|
||||
|
||||
@@ -157,6 +164,7 @@ impl AppState {
|
||||
slots: Mutex::new(SlotAllocator::new()),
|
||||
recordings: RecordingSet::new(),
|
||||
tx,
|
||||
video,
|
||||
listener: Mutex::new(Some(listener)),
|
||||
port,
|
||||
token: server::random_token(),
|
||||
@@ -175,15 +183,14 @@ impl AppState {
|
||||
///
|
||||
/// 必须在 `Arc<AppState>` 建好之后调用(sink 里要持有 state 的通道)。
|
||||
pub fn install_sinks(self: &Arc<Self>) {
|
||||
let tx = self.tx.clone();
|
||||
let video = self.video.clone();
|
||||
let recordings = Arc::clone(&self.recordings);
|
||||
|
||||
// 帧的 sink:在接收线程里跑,只做"编码一次 + 广播"。这一份字节所有界面共用。
|
||||
self.receiver
|
||||
.set_frame_sink(Arc::new(move |frame: Arc<protocol::Frame>| {
|
||||
recordings.feed(&frame);
|
||||
let bytes = server::encode_frame(&frame);
|
||||
let _ = tx.send(Arc::new(ServerMsg::Binary(bytes.into())));
|
||||
server::publish_video(&video, &frame);
|
||||
}));
|
||||
|
||||
let tx = self.tx.clone();
|
||||
@@ -240,9 +247,10 @@ impl AppState {
|
||||
let token = self.token.clone();
|
||||
let tx = self.tx.clone();
|
||||
let clients = Arc::clone(&self.clients);
|
||||
let video = self.video.clone();
|
||||
|
||||
tauri::async_runtime::spawn(async move {
|
||||
server::serve(listener, token, tx, clients).await;
|
||||
server::serve(listener, token, tx, video, clients).await;
|
||||
});
|
||||
|
||||
let state_for_loop = Arc::clone(&state);
|
||||
@@ -306,7 +314,9 @@ impl AppState {
|
||||
// 本来就要看到全场。
|
||||
let open = match role {
|
||||
Role::Referee => true,
|
||||
Role::Team => self.discovery_open.load(Ordering::Relaxed) && self.unlocked.load(Ordering::Relaxed),
|
||||
Role::Team => {
|
||||
self.discovery_open.load(Ordering::Relaxed) && self.unlocked.load(Ordering::Relaxed)
|
||||
}
|
||||
};
|
||||
self.receiver.set_discovery_all(open);
|
||||
|
||||
@@ -457,7 +467,12 @@ impl AppState {
|
||||
|
||||
/// 推进帧率窗口与槽位分配。
|
||||
pub fn tick(&self) {
|
||||
let online: Vec<u32> = self.discovery.online_cameras().iter().map(|c| c.id).collect();
|
||||
let online: Vec<u32> = self
|
||||
.discovery
|
||||
.online_cameras()
|
||||
.iter()
|
||||
.map(|c| c.id)
|
||||
.collect();
|
||||
let online_set: HashSet<u32> = online.iter().copied().collect();
|
||||
|
||||
{
|
||||
@@ -539,8 +554,12 @@ impl AppState {
|
||||
})
|
||||
.collect();
|
||||
|
||||
let armor_boards: Vec<ArmorBoardDto> =
|
||||
self.armor.boards().iter().map(ArmorBoardDto::from).collect();
|
||||
let armor_boards: Vec<ArmorBoardDto> = self
|
||||
.armor
|
||||
.boards()
|
||||
.iter()
|
||||
.map(ArmorBoardDto::from)
|
||||
.collect();
|
||||
|
||||
let slots = self.slots.lock().expect("slots mutex poisoned").slots();
|
||||
|
||||
@@ -627,7 +646,9 @@ impl AppState {
|
||||
|
||||
if let Some(board) = board {
|
||||
return match self.armor.health_of_board(board) {
|
||||
Some(hp) => TeamHealthDto::present(team, Some(board), [hp; 4], low, &config.team_names),
|
||||
Some(hp) => {
|
||||
TeamHealthDto::present(team, Some(board), [hp; 4], low, &config.team_names)
|
||||
}
|
||||
None => TeamHealthDto::absent(team, Some(board), &config.team_names),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -23,6 +23,7 @@ import { useEffect, useRef, type ReactNode } from "react";
|
||||
import { streamClient } from "../lib/store";
|
||||
import { withAlpha } from "../lib/format";
|
||||
import type { CrosshairConfig, GridConfig, HitFeedbackConfig } from "../lib/types";
|
||||
import type { FrameInfo } from "../lib/stream";
|
||||
|
||||
export interface ViewTransform {
|
||||
zoom: number;
|
||||
@@ -61,6 +62,8 @@ interface Props {
|
||||
hitFeedback: HitFeedbackConfig;
|
||||
showZoom: boolean;
|
||||
children?: ReactNode;
|
||||
/** Only used by the opt-in screen/camera calibration page. */
|
||||
onFramePainted?: (canvas:HTMLCanvasElement,frame:FrameInfo,drawMs:number)=>void;
|
||||
}
|
||||
|
||||
export function VideoSurface({
|
||||
@@ -77,12 +80,13 @@ export function VideoSurface({
|
||||
hitFeedback,
|
||||
showZoom,
|
||||
children,
|
||||
onFramePainted,
|
||||
}: Props) {
|
||||
const canvasRef = useRef<HTMLCanvasElement | null>(null);
|
||||
|
||||
// 绘制循环不随 props 重建,所以最新一份 props 放在 ref 里给它读。
|
||||
const props = useRef({ camera, mirrored, crosshair, grid, view, hit, hitFeedback });
|
||||
props.current = { camera, mirrored, crosshair, grid, view, hit, hitFeedback };
|
||||
const props = useRef({ camera, mirrored, crosshair, grid, view, hit, hitFeedback,onFramePainted });
|
||||
props.current = { camera, mirrored, crosshair, grid, view, hit, hitFeedback,onFramePainted };
|
||||
|
||||
const drag = useRef<{ x: number; y: number; originX: number; originY: number } | null>(null);
|
||||
const dirty = useRef(true);
|
||||
@@ -127,6 +131,7 @@ export function VideoSurface({
|
||||
|
||||
const current = props.current;
|
||||
const frame = streamClient().latest(current.camera);
|
||||
const newFrame=frame && frame.frameId!==lastFrameId.current;
|
||||
|
||||
if (frame && frame.frameId !== lastFrameId.current) {
|
||||
lastFrameId.current = frame.frameId;
|
||||
@@ -145,6 +150,7 @@ export function VideoSurface({
|
||||
dirty.current = false;
|
||||
|
||||
draw(canvas, current, frame, timestamp);
|
||||
if(newFrame && current.onFramePainted) current.onFramePainted(canvas,frame,performance.now());
|
||||
};
|
||||
|
||||
raf = requestAnimationFrame(loop);
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
/** A screen marker observed through the real camera. No device clock sync. */
|
||||
export type Point = { x: number; y: number };
|
||||
export const CORNERS = [[255,32,32],[32,255,32],[32,96,255],[255,32,255]] as const;
|
||||
export const MARKER_BITS = 16;
|
||||
|
||||
export function markerBits(counter: number): number[] {
|
||||
const gray=(counter&4095)^((counter&4095)>>>1);
|
||||
return [1,0,1,0,...Array.from({length:12},(_,i)=>(gray>>>(11-i))&1)];
|
||||
}
|
||||
|
||||
export function drawMarker(ctx: CanvasRenderingContext2D, counter:number) {
|
||||
const w=ctx.canvas.width,h=ctx.canvas.height;
|
||||
ctx.fillStyle='#30363e';ctx.fillRect(0,0,w,h);
|
||||
// Static coloured detail keeps the target from being an unrealistically tiny JPEG.
|
||||
for(let y=0;y<8;y++) for(let x=0;x<12;x++) {
|
||||
ctx.fillStyle=`hsl(${(x*31+y*47)%360} 10% ${20+(x+y)%3*7}%)`;
|
||||
ctx.fillRect(x*w/12,y*h/8,w/12,h/8);
|
||||
}
|
||||
[[.06,.08],[.94,.08],[.94,.92],[.06,.92]].forEach(([x,y],i)=>{
|
||||
ctx.fillStyle=`rgb(${CORNERS[i].join(',')})`;
|
||||
ctx.fillRect(x*w-.035*w,y*h-.045*h,.07*w,.09*h);
|
||||
});
|
||||
const bits=markerBits(counter);
|
||||
ctx.fillStyle='#222';ctx.fillRect(.12*w,.34*h,.76*w,.32*h);
|
||||
bits.forEach((bit,i)=>{
|
||||
const x=(.14+.72*i/MARKER_BITS)*w,cw=.72*w/MARKER_BITS;
|
||||
ctx.fillStyle=bit?'#fff':'#000';ctx.fillRect(x,.38*h,cw,.10*h);
|
||||
ctx.fillStyle=bit?'#000':'#fff';ctx.fillRect(x,.52*h,cw,.10*h);
|
||||
});
|
||||
}
|
||||
|
||||
function colour(r:number,g:number,b:number):number {
|
||||
// Screen primaries can clip and bloom through exposure/white balance.
|
||||
// Absolute channel separation admits pastel green/cyan while excluding
|
||||
// the low-saturation background (which can otherwise look more blue).
|
||||
if(r>150 && b>150 && r-g>40 && b-g>40 && Math.abs(r-b)<100) return 4;
|
||||
if(r>150 && r-g>40 && r-b>40) return 1;
|
||||
if(g>150 && g-r>35 && g-b>30) return 2;
|
||||
if(b>150 && b>r*1.7 && b-g>20) return 3;
|
||||
return 0;
|
||||
}
|
||||
|
||||
function project(corners:Point[],u:number,v:number):Point|null {
|
||||
const [p0,p1,p2,p3]=corners;
|
||||
const dx1=p1.x-p2.x, dx2=p3.x-p2.x, dx3=p0.x-p1.x+p2.x-p3.x;
|
||||
const dy1=p1.y-p2.y, dy2=p3.y-p2.y, dy3=p0.y-p1.y+p2.y-p3.y;
|
||||
const det=dx1*dy2-dx2*dy1;
|
||||
if(Math.abs(det)<1e-6) return null;
|
||||
const g=(dx3*dy2-dx2*dy3)/det, h=(dx1*dy3-dx3*dy1)/det;
|
||||
const den=g*u+h*v+1;
|
||||
if(Math.abs(den)<.01) return null;
|
||||
return {x:((p1.x-p0.x+g*p1.x)*u+(p3.x-p0.x+h*p3.x)*v+p0.x)/den,
|
||||
y:((p1.y-p0.y+g*p1.y)*u+(p3.y-p0.y+h*p3.y)*v+p0.y)/den};
|
||||
}
|
||||
|
||||
export function decodeMarker(pixels:Uint8ClampedArray,w:number,h:number):{counter:number;corners:Point[]}|null {
|
||||
const labels=new Uint8Array(w*h);
|
||||
for(let i=0;i<labels.length;i++) labels[i]=colour(pixels[i*4],pixels[i*4+1],pixels[i*4+2]);
|
||||
const best=Array.from({length:4},()=>({count:0,x:0,y:0}));
|
||||
const queue=new Int32Array(w*h);
|
||||
for(let start=0;start<labels.length;start++) {
|
||||
const cls=labels[start]; if(!cls) continue;
|
||||
let head=0,tail=1,count=0,sx=0,sy=0;
|
||||
queue[0]=start;labels[start]=0;
|
||||
while(head<tail) {
|
||||
const i=queue[head++],x=i%w,y=Math.floor(i/w);
|
||||
count++;sx+=x;sy+=y;
|
||||
for(const n of [x>0?i-1:-1,x<w-1?i+1:-1,y>0?i-w:-1,y<h-1?i+w:-1]) {
|
||||
if(n>=0 && labels[n]===cls) { labels[n]=0;queue[tail++]=n; }
|
||||
}
|
||||
}
|
||||
if(count>=6 && count<w*h/6 && count>best[cls-1].count) best[cls-1]={count,x:sx/count,y:sy/count};
|
||||
}
|
||||
if(best.some(b=>!b.count)) return null;
|
||||
const corners=best.map(({x,y})=>({x,y}));
|
||||
const area=Math.abs(corners.reduce((a,p,i)=>a+p.x*corners[(i+1)%4].y-p.y*corners[(i+1)%4].x,0))/2;
|
||||
if(area<1500) return null;
|
||||
const luma=(x:number,y:number)=>{
|
||||
let total=0,count=0;
|
||||
for(let dy=-1;dy<=1;dy++) for(let dx=-1;dx<=1;dx++) {
|
||||
const xx=Math.round(x)+dx,yy=Math.round(y)+dy;
|
||||
if(xx<0||xx>=w||yy<0||yy>=h) continue;
|
||||
const i=(yy*w+xx)*4;total+=(pixels[i]+pixels[i+1]+pixels[i+2])/3;count++;
|
||||
}
|
||||
return count?total/count:NaN;
|
||||
};
|
||||
const bits:number[]=[];
|
||||
for(let i=0;i<MARKER_BITS;i++) {
|
||||
const x=.14+.72*(i+.5)/MARKER_BITS;
|
||||
const top=project(corners,(x-.06)/.88,(.43-.08)/.84);
|
||||
const bottom=project(corners,(x-.06)/.88,(.57-.08)/.84);
|
||||
if(!top||!bottom) return null;
|
||||
const delta=luma(top.x,top.y)-luma(bottom.x,bottom.y);
|
||||
if(!Number.isFinite(delta)||Math.abs(delta)<60) return null;
|
||||
bits.push(delta>0?1:0);
|
||||
}
|
||||
if(bits.slice(0,4).join('')!=='1010') return null;
|
||||
let gray=bits.slice(4).reduce((n,b)=>(n<<1)|b,0),counter=gray;
|
||||
while(gray>>>=1) counter^=gray;
|
||||
return {counter,corners};
|
||||
}
|
||||
@@ -76,6 +76,8 @@ export function currentStore(): AppStore {
|
||||
|
||||
function handleMessage(msg: WireMsg) {
|
||||
switch (msg.t) {
|
||||
case "frameTiming":
|
||||
return; // Consumed by StreamClient before reaching application state.
|
||||
case "snapshot": {
|
||||
const { t: _tag, ...snapshot } = msg;
|
||||
emit({ snapshot: snapshot as Snapshot });
|
||||
|
||||
+82
-20
@@ -28,7 +28,7 @@ export interface FrameInfo {
|
||||
width: number;
|
||||
height: number;
|
||||
bitmap: ImageBitmap;
|
||||
/** 收到的时刻(`performance.now()`),界面用来显示端到端延迟。 */
|
||||
/** 本地完成解码的时刻(`performance.now()`),用于判断帧的新旧。 */
|
||||
receivedAt: number;
|
||||
}
|
||||
|
||||
@@ -39,6 +39,13 @@ interface Handlers {
|
||||
|
||||
/** 断线重连的退避上限。 */
|
||||
const MAX_BACKOFF_MS = 3000;
|
||||
const MAX_LOCAL_VIDEO_AGE_MS = 100;
|
||||
|
||||
interface PendingFrame {
|
||||
bytes: ArrayBuffer;
|
||||
arrivedAt: number;
|
||||
publishedUnixMs?: number;
|
||||
}
|
||||
|
||||
export class StreamClient {
|
||||
private ws: WebSocket | null = null;
|
||||
@@ -48,14 +55,19 @@ export class StreamClient {
|
||||
private retryTimer: number | null = null;
|
||||
private backoff = 250;
|
||||
private closed = false;
|
||||
private generation = 0;
|
||||
private readonly pending = new Map<number, PendingFrame>();
|
||||
private readonly frameTimes = new Map<number, { frameId: number; publishedUnixMs: number }>();
|
||||
private readonly decoding = new Map<number, number>();
|
||||
/** 统计:收到多少帧、解码失败多少张。界面用它区分"链路没数据"和"数据坏了"。 */
|
||||
readonly counters = { frames: 0, decodeFailures: 0, reconnects: 0 };
|
||||
readonly counters = { frames: 0, decodeFailures: 0, reconnects: 0, skippedDecodes: 0, staleFrames: 0 };
|
||||
|
||||
constructor(handlers: Handlers) {
|
||||
constructor(handlers: Handlers, private readonly now = () => performance.now(), private readonly wallNow = () => Date.now()) {
|
||||
this.handlers = handlers;
|
||||
}
|
||||
|
||||
connect(url: string) {
|
||||
this.close();
|
||||
this.url = url;
|
||||
this.closed = false;
|
||||
this.open();
|
||||
@@ -76,11 +88,17 @@ export class StreamClient {
|
||||
this.ws = socket;
|
||||
|
||||
socket.onopen = () => {
|
||||
if (this.ws !== socket) return;
|
||||
this.backoff = 250;
|
||||
this.handlers.onConnection(true);
|
||||
};
|
||||
|
||||
socket.onclose = () => {
|
||||
if (this.ws !== socket) return;
|
||||
this.generation += 1;
|
||||
this.pending.clear();
|
||||
this.frameTimes.clear();
|
||||
this.decoding.clear();
|
||||
this.handlers.onConnection(false);
|
||||
this.scheduleReconnect();
|
||||
};
|
||||
@@ -90,9 +108,12 @@ export class StreamClient {
|
||||
};
|
||||
|
||||
socket.onmessage = (event) => {
|
||||
if (this.ws !== socket) return;
|
||||
if (typeof event.data === "string") {
|
||||
try {
|
||||
this.handlers.onMessage(JSON.parse(event.data) as WireMsg);
|
||||
const message = JSON.parse(event.data) as WireMsg;
|
||||
if (message.t === "frameTiming") this.onTiming(message);
|
||||
else this.handlers.onMessage(message);
|
||||
} catch {
|
||||
// 前端解析不了的消息只可能是版本不匹配,忽略比崩掉好。
|
||||
}
|
||||
@@ -103,31 +124,66 @@ export class StreamClient {
|
||||
};
|
||||
}
|
||||
|
||||
private onTiming(message: { camera: number; frameId: number; publishedUnixMs: number }) {
|
||||
if (Number.isFinite(message.publishedUnixMs) && message.publishedUnixMs > 0) {
|
||||
this.frameTimes.set(message.camera, message);
|
||||
}
|
||||
}
|
||||
|
||||
private expired(frame: PendingFrame): boolean {
|
||||
return this.now() - frame.arrivedAt >= MAX_LOCAL_VIDEO_AGE_MS ||
|
||||
(frame.publishedUnixMs !== undefined && this.wallNow() - frame.publishedUnixMs >= MAX_LOCAL_VIDEO_AGE_MS);
|
||||
}
|
||||
|
||||
private async onBinary(buffer: ArrayBuffer) {
|
||||
if (buffer.byteLength < FRAME_HEADER_LEN) return;
|
||||
if (this.closed || buffer.byteLength < FRAME_HEADER_LEN) return;
|
||||
|
||||
const view = new DataView(buffer);
|
||||
const camera = view.getUint32(0, true);
|
||||
const frameId = view.getUint32(4, true);
|
||||
const width = view.getUint16(8, true);
|
||||
const height = view.getUint16(10, true);
|
||||
const length = view.getUint32(12, true);
|
||||
|
||||
if (buffer.byteLength < FRAME_HEADER_LEN + length) return;
|
||||
if (buffer.byteLength !== FRAME_HEADER_LEN + length) return;
|
||||
|
||||
this.counters.frames += 1;
|
||||
const timing = this.frameTimes.get(camera);
|
||||
const received: PendingFrame = { bytes: buffer, arrivedAt: this.now(),
|
||||
publishedUnixMs: timing?.frameId === view.getUint32(4, true) ? timing.publishedUnixMs : undefined };
|
||||
if (this.expired(received)) { this.counters.staleFrames += 1; return; }
|
||||
if (this.pending.has(camera)) this.counters.skippedDecodes += 1;
|
||||
this.pending.set(camera, received);
|
||||
if (this.decoding.has(camera)) return;
|
||||
|
||||
const jpeg = new Uint8Array(buffer, FRAME_HEADER_LEN, length);
|
||||
let bitmap: ImageBitmap;
|
||||
|
||||
const generation = this.generation;
|
||||
this.decoding.set(camera, generation);
|
||||
try {
|
||||
bitmap = await createImageBitmap(new Blob([jpeg], { type: "image/jpeg" }));
|
||||
} catch {
|
||||
this.counters.decodeFailures += 1;
|
||||
return;
|
||||
while (!this.closed && this.generation === generation) {
|
||||
const next = this.pending.get(camera);
|
||||
if (!next) break;
|
||||
this.pending.delete(camera);
|
||||
if (this.expired(next)) { this.counters.staleFrames += 1; continue; }
|
||||
const header = new DataView(next.bytes);
|
||||
const jpeg = new Uint8Array(next.bytes, FRAME_HEADER_LEN);
|
||||
let bitmap: ImageBitmap;
|
||||
try {
|
||||
bitmap = await createImageBitmap(new Blob([jpeg], { type: "image/jpeg" }));
|
||||
} catch {
|
||||
this.counters.decodeFailures += 1;
|
||||
continue;
|
||||
}
|
||||
if (this.closed || this.generation !== generation) {
|
||||
bitmap.close();
|
||||
break;
|
||||
}
|
||||
if (this.expired(next)) {
|
||||
bitmap.close();
|
||||
this.counters.staleFrames += 1;
|
||||
continue;
|
||||
}
|
||||
this.store(camera, header.getUint32(4, true), header.getUint16(8, true), header.getUint16(10, true), bitmap);
|
||||
}
|
||||
} finally {
|
||||
if (this.decoding.get(camera) === generation) this.decoding.delete(camera);
|
||||
}
|
||||
|
||||
this.store(camera, frameId, width, height, bitmap);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -149,7 +205,9 @@ export class StreamClient {
|
||||
const current = this.frames.get(camera);
|
||||
|
||||
// 带符号差处理 u32 回绕。
|
||||
if (current && ((frameId - current.frameId) | 0) <= 0) {
|
||||
// After a quiet interval the board may have rebooted and restarted its
|
||||
// frame counter. An older in-flight decode cannot hold the display forever.
|
||||
if (current && ((frameId - current.frameId) | 0) <= 0 && this.now() - current.receivedAt < 1000) {
|
||||
bitmap.close();
|
||||
return;
|
||||
}
|
||||
@@ -162,7 +220,7 @@ export class StreamClient {
|
||||
width,
|
||||
height,
|
||||
bitmap,
|
||||
receivedAt: performance.now(),
|
||||
receivedAt: this.now(),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -197,6 +255,10 @@ export class StreamClient {
|
||||
|
||||
close() {
|
||||
this.closed = true;
|
||||
this.generation += 1;
|
||||
this.pending.clear();
|
||||
this.frameTimes.clear();
|
||||
this.decoding.clear();
|
||||
|
||||
if (this.retryTimer !== null) {
|
||||
window.clearTimeout(this.retryTimer);
|
||||
|
||||
@@ -233,6 +233,7 @@ export type WireMsg =
|
||||
| ({ t: "snapshot" } & Snapshot)
|
||||
| ({ t: "hit" } & HitEvent)
|
||||
| { t: "note"; message: string }
|
||||
| { t: "frameTiming"; camera: number; frameId: number; publishedUnixMs: number }
|
||||
| { t: "confirmQuit" };
|
||||
|
||||
/** 二进制视频帧的头部长度,与 `server::FRAME_HEADER_LEN` 一致。 */
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
import { createRoot } from "react-dom/client";
|
||||
|
||||
import App from "./App";
|
||||
import { LatencyPage } from './pages/LatencyPage';
|
||||
import { invoke } from '@tauri-apps/api/core';
|
||||
import "./styles/global.css";
|
||||
|
||||
const container = document.getElementById("root");
|
||||
@@ -12,4 +14,7 @@ if (!container) {
|
||||
// 不用 StrictMode:这个界面的关键路径是命令式的(WebSocket、requestAnimationFrame 绘制
|
||||
// 循环、canvas 尺寸)。StrictMode 在开发下会把每个 effect 跑两遍,于是连接与绘制循环会
|
||||
// 短暂地建两份——排查起来的噪音大于它带来的收益。生产构建的行为与这里一致。
|
||||
createRoot(container).render(<App />);
|
||||
const root=createRoot(container);
|
||||
if('__TAURI_INTERNALS__' in window) {
|
||||
void invoke<boolean>('latency_enabled').then(enabled=>root.render(enabled?<LatencyPage/>:<App/>),()=>root.render(<App/>));
|
||||
} else root.render(<App/>);
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
import { useEffect,useRef,useState } from 'react';
|
||||
import { invoke } from '@tauri-apps/api/core';
|
||||
import { api } from '../lib/ipc';
|
||||
import { startStream,useAppStore } from '../lib/store';
|
||||
import { decodeMarker,drawMarker } from '../lib/latency-marker';
|
||||
import { VideoSurface,IDENTITY_VIEW } from '../components/VideoSurface';
|
||||
import type { FrameInfo } from '../lib/stream';
|
||||
|
||||
type Stamp={draw:number;next:number|null;sampled:boolean};
|
||||
const feedback={flash:false,shake:false,marker:false,intensity:0};
|
||||
|
||||
export function LatencyPage() {
|
||||
const {snapshot,connected}=useAppStore();
|
||||
const [camera,setCamera]=useState<number|null>(null);
|
||||
const [status,setStatus]=useState('等待摄像头与屏幕标记');
|
||||
const [summary,setSummary]=useState({samples:0,p50:0,p95:0,refresh:0});
|
||||
const target=useRef<HTMLCanvasElement>(null);
|
||||
const history=useRef(new Map<number,Stamp>());
|
||||
const scratch=useRef<HTMLCanvasElement|null>(null);
|
||||
const values=useRef<number[]>([]);
|
||||
const lastSample=useRef(-Infinity);
|
||||
const refresh=useRef(16.7);
|
||||
const active=useRef(true);
|
||||
useEffect(()=>{
|
||||
let client:ReturnType<typeof startStream>|null=null,disposed=false;
|
||||
void api.streamInfo().then(info=>{if(!disposed)client=startStream(info.url);}).catch(e=>setStatus(String(e)));
|
||||
return()=>{disposed=true;client?.close();};
|
||||
},[]);
|
||||
useEffect(()=>{
|
||||
if(camera===null && snapshot) {
|
||||
const found=snapshot.cameras.find(c=>c.online);
|
||||
if(found)setCamera(found.id);
|
||||
}
|
||||
},[snapshot,camera]);
|
||||
useEffect(()=>{
|
||||
active.current=true;
|
||||
const canvas=target.current;if(!canvas)return;
|
||||
const ctx=canvas.getContext('2d');if(!ctx)return;
|
||||
let raf=0,counter=0,previous:Stamp|null=null,last=0,lastMarker=-Infinity;
|
||||
const loop=(now:number)=>{
|
||||
if(previous){previous.next=now;previous=null;}
|
||||
if(last && now-last<50)refresh.current=refresh.current*.9+(now-last)*.1;
|
||||
last=now;
|
||||
// A 180 Hz display can change several IDs within one camera exposure.
|
||||
// Hold each event for 50 ms, and record only its first decodable echo.
|
||||
if(now-lastMarker>=50){
|
||||
lastMarker=now;counter=(counter+1)&4095;
|
||||
drawMarker(ctx,counter);
|
||||
const stamp={draw:performance.now(),next:null,sampled:false};
|
||||
history.current.set(counter,stamp);previous=stamp;
|
||||
}
|
||||
// Only the most recent five seconds count, including counter wrap.
|
||||
for(const [id,value] of history.current) if(now-value.draw>5000)history.current.delete(id);
|
||||
raf=requestAnimationFrame(loop);
|
||||
};
|
||||
raf=requestAnimationFrame(loop);
|
||||
return()=>{active.current=false;cancelAnimationFrame(raf);};
|
||||
},[]);
|
||||
const painted=(canvas:HTMLCanvasElement,frame:FrameInfo,drawMs:number)=>{
|
||||
if(drawMs-lastSample.current<25)return;
|
||||
lastSample.current=drawMs;
|
||||
const started=performance.now();
|
||||
const small=scratch.current??(scratch.current=document.createElement('canvas'));
|
||||
small.width=320;small.height=240;
|
||||
const ctx=small.getContext('2d',{willReadFrequently:true});if(!ctx)return;
|
||||
// Read the canvas after the production renderer has painted the frame.
|
||||
ctx.drawImage(canvas,0,0,320,240);
|
||||
const marker=decodeMarker(ctx.getImageData(0,0,320,240).data,320,240);
|
||||
const analysisMs=performance.now()-started;
|
||||
if(!marker){setStatus('未识别到四角标记:请让左侧标记完整入镜并尽量拍大');return;}
|
||||
const source=history.current.get(marker.counter);
|
||||
if(!source||source.next===null||source.draw>drawMs){setStatus('等待有效的同一会话标记');return;}
|
||||
if(source.sampled)return;
|
||||
source.sampled=true;
|
||||
requestAnimationFrame(nextRafMs=>{
|
||||
if(!active.current)return;
|
||||
const lowerMs=Math.max(0,drawMs-source.next!);
|
||||
const upperMs=nextRafMs-source.draw;
|
||||
if(upperMs<lowerMs||upperMs>5000)return;
|
||||
const value=(lowerMs+upperMs)/2;
|
||||
values.current.push(value);
|
||||
const sorted=[...values.current].sort((a,b)=>a-b);
|
||||
setStatus('标记已锁定,正在记录真实相机 → 客户端画布链路');
|
||||
setSummary({samples:sorted.length,p50:sorted[Math.floor((sorted.length-1)*.5)],p95:sorted[Math.floor((sorted.length-1)*.95)],refresh:refresh.current});
|
||||
void invoke('latency_sample',{sample:{frameId:frame.frameId,counter:marker.counter,sourceDrawMs:source.draw,
|
||||
drawMs,nextRafMs,lowerMs,upperMs,decodeToDrawMs:drawMs-frame.receivedAt,analysisMs}}).catch(e=>setStatus(`报告写入失败:${String(e)}`));
|
||||
});
|
||||
};
|
||||
return <main style={{height:'100vh',display:'flex',flexDirection:'column',padding:20,gap:12,background:'#111820',color:'#f3f6fa'}}>
|
||||
<h1 style={{fontSize:24}}>屏幕 → 摄像头 → 客户端绘制延迟校准</h1>
|
||||
<p>将摄像头固定对准左侧标记,完整拍到四个彩色角块。不要拍右侧回显区域;保持窗口位置不动。</p>
|
||||
<div style={{display:'grid',gridTemplateColumns:'1fr 1fr',gap:16,flex:1,minHeight:0}}>
|
||||
<canvas ref={target} width={800} height={600} style={{width:'100%',height:'100%',objectFit:'contain'}}/>
|
||||
<div style={{position:'relative',minHeight:0}}><VideoSurface camera={camera} emptyText="等待真实摄像头" mirrored={false}
|
||||
crosshair={null} grid={null} view={IDENTITY_VIEW} onView={()=>{}} hitFeedback={feedback} showZoom={false} onFramePainted={painted}/></div>
|
||||
</div>
|
||||
<p>{connected?'已连接':'连接中'} · {status}</p>
|
||||
<p style={{fontSize:22}}>样本 {summary.samples} · P50 {summary.p50.toFixed(1)} ms · P95 {summary.p95.toFixed(1)} ms</p>
|
||||
<p>每 50 ms 更新标记,记录首次可解码回显,包含相机取样等待、曝光、采集、传输和绘制;混合曝光帧不计入。相邻 rAF 夹定绘制时间,刷新间隔约 {summary.refresh.toFixed(1)} ms;面板像素响应需高速相机复核。</p>
|
||||
</main>;
|
||||
}
|
||||
Reference in New Issue
Block a user