Horldsence/ESP32S3_vt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ESP32-S3 摄像头图传：固件 + Slint 客户端 + 共享协议

Browse Source 
两块 ESP32-S3-WROOM-1-N16R8 图传板（DVP 接口接 OV2640/OV3660/OV5640），
向队伍客户端与裁判终端推 UDP 图传。固件是跑在 ESP-IDF 上的 Rust，用官方
esp32-camera 组件采集；客户端是 Rust + Slint 桌面程序。

三个 crate 各自独立构建：
  .            firmware，交叉编译到 Xtensa
  protocol/    线路格式与帧重组，两端共用，无 ESP-IDF 依赖，有单元测试
  client/      Slint 桌面客户端

视频走 UDP 而非 TCP：实测传感器产出 22 fps，TCP MJPEG 只送出 1.9 fps，
且只占用链路能力的 1%——链路是空的，帧堵在队头阻塞后面等一个实时画面
等不起的重传计时器。改为独立数据报、不重传、缺片丢整帧后达到 20.8 fps。

发现是双向的：板子每 2 秒广播 Hello，客户端广播 Subscribe，板子据此定向
推流。板子身份由 MAC 派生，重启不变，所以客户端可以把绑定存进配置。

裁判端同时收两块板子（每个发送方一个重组器），队伍端绑定其中一块并只显示
自己那一边的血量。装甲板接口已预留（ArmorSource trait + ARMOR_PORT），
当前无数据源，界面显示占位符。

vendor 了 esp32-camera 2.1.7：组件注册表的版本求解器找不到它，且需要打一处
补丁（见 components/esp32-camera/target/esp32s3/ll_cam.c）。

.gitignore 三条 target/ 都锚定在根目录，因为 protocol/ 与 client/ 各自也是
独立 crate；而 components/esp32-camera/target/ 是上游的按芯片分目录，里面是
源码，必须保留。
This commit is contained in:
Horldsence
2026-10-05 18:22:19 +08:00
commit 4f893ce140
134 changed files with 41128 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/camera.rs
+300
View File
@@ -0,0 +1,300 @@
//! Thin safe wrapper over the `cam_wrapper` C component (see
//! `components/cam_wrapper`).
//!
//! The pin map matches the XIAO ESP32S3 Sense camera connector, which is the
//! common OV2640/OV3660 wiring on an ESP32-S3-N16R8. Adjust `PINS` at the
//! bottom of this file if your board wires the DVP bus differently.
use std::ffi::CStr;
use anyhow::{anyhow, Result};
use esp_idf_svc::sys;
// bindgen names C enum variants `<enum type>_<VARIANT>`, so alias them to
// something readable here instead of scattering those prefixes around.
use esp_idf_svc::sys::camera::{
cam_wrapper_fb_location_t_CAM_WRAPPER_FB_IN_PSRAM as FB_IN_PSRAM,
cam_wrapper_framesize_t_CAM_WRAPPER_FRAMESIZE_CIF as FRAMESIZE_CIF,
cam_wrapper_framesize_t_CAM_WRAPPER_FRAMESIZE_HVGA as FRAMESIZE_HVGA,
cam_wrapper_framesize_t_CAM_WRAPPER_FRAMESIZE_QQVGA as FRAMESIZE_QQVGA,
cam_wrapper_framesize_t_CAM_WRAPPER_FRAMESIZE_QVGA as FRAMESIZE_QVGA,
cam_wrapper_framesize_t_CAM_WRAPPER_FRAMESIZE_SVGA as FRAMESIZE_SVGA,
cam_wrapper_framesize_t_CAM_WRAPPER_FRAMESIZE_SXGA as FRAMESIZE_SXGA,
cam_wrapper_framesize_t_CAM_WRAPPER_FRAMESIZE_UXGA as FRAMESIZE_UXGA,
cam_wrapper_framesize_t_CAM_WRAPPER_FRAMESIZE_VGA as FRAMESIZE_VGA,
cam_wrapper_framesize_t_CAM_WRAPPER_FRAMESIZE_XGA as FRAMESIZE_XGA,
cam_wrapper_grab_mode_t_CAM_WRAPPER_GRAB_LATEST as GRAB_LATEST,
cam_wrapper_pixel_format_t_CAM_WRAPPER_PIXFORMAT_JPEG as PIXFORMAT_JPEG,
cam_wrapper_sensor_t_CAM_WRAPPER_SENSOR_AUTO as SENSOR_AUTO,
cam_wrapper_sensor_t_CAM_WRAPPER_SENSOR_OV2640 as SENSOR_OV2640,
cam_wrapper_sensor_t_CAM_WRAPPER_SENSOR_OV3660 as SENSOR_OV3660,
cam_wrapper_sensor_t_CAM_WRAPPER_SENSOR_OV5640 as SENSOR_OV5640,
};
/// Capture resolution.
///
/// Anything above CIF needs PSRAM, which the N16R8 has; the bandwidth cost of
/// the bigger sizes is real, so start at VGA and only go higher if the link
/// keeps up.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Resolution {
Qqvga,
Qvga,
Cif,
Hvga,
Vga,
Svga,
Xga,
Sxga,
Uxga,
}
impl Resolution {
fn framesize(self) -> sys::camera::cam_wrapper_framesize_t {
match self {
Resolution::Qqvga => FRAMESIZE_QQVGA,
Resolution::Qvga => FRAMESIZE_QVGA,
Resolution::Cif => FRAMESIZE_CIF,
Resolution::Hvga => FRAMESIZE_HVGA,
Resolution::Vga => FRAMESIZE_VGA,
Resolution::Svga => FRAMESIZE_SVGA,
Resolution::Xga => FRAMESIZE_XGA,
Resolution::Sxga => FRAMESIZE_SXGA,
Resolution::Uxga => FRAMESIZE_UXGA,
}
}
}
impl std::str::FromStr for Resolution {
type Err = anyhow::Error;
fn from_str(s: &str) -> Result<Self, Self::Err> {
match s.to_ascii_lowercase().as_str() {
"qqvga" => Ok(Resolution::Qqvga),
"qvga" => Ok(Resolution::Qvga),
"cif" => Ok(Resolution::Cif),
"hvga" => Ok(Resolution::Hvga),
"vga" => Ok(Resolution::Vga),
"svga" => Ok(Resolution::Svga),
"xga" => Ok(Resolution::Xga),
"sxga" => Ok(Resolution::Sxga),
"uxga" => Ok(Resolution::Uxga),
other => Err(anyhow!("unknown resolution {other:?}")),
}
}
}
use log::info;
/// Sensor override. `Auto` lets esp32-camera probe the SCCB bus, which is what
/// you want unless the probe is ambiguous.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum SensorHint {
Auto,
Ov2640,
Ov3660,
Ov5640,
}
impl From<SensorHint> for sys::camera::cam_wrapper_sensor_t {
fn from(hint: SensorHint) -> Self {
match hint {
SensorHint::Auto => SENSOR_AUTO,
SensorHint::Ov2640 => SENSOR_OV2640,
SensorHint::Ov3660 => SENSOR_OV3660,
SensorHint::Ov5640 => SENSOR_OV5640,
}
}
}
/// One captured JPEG frame. Holds the driver frame buffer, so the data is only
/// valid until the `Frame` is dropped.
pub struct Frame {
fb: sys::camera::cam_wrapper_fb_t,
}
impl Frame {
pub fn data(&self) -> &[u8] {
if self.fb.data.is_null() || self.fb.len == 0 {
return &[];
}
// SAFETY: `data` points at `len` bytes owned by esp32-camera, which
// stays valid until this Frame is dropped (cam_wrapper_fb_release).
unsafe { std::slice::from_raw_parts(self.fb.data, self.fb.len) }
}
/// Size of the JPEG in bytes.
///
/// Reported in the sender's statistics, which is how a frame rate problem gets
/// attributed to the capture side or the transport side.
#[allow(dead_code)]
pub fn len(&self) -> usize {
self.fb.len
}
/// Capture width in pixels, as the sensor actually produced it (which can be
/// smaller than the requested frame size).
pub fn width(&self) -> usize {
self.fb.width
}
/// Capture height in pixels.
pub fn height(&self) -> usize {
self.fb.height
}
}
impl Drop for Frame {
fn drop(&mut self) {
unsafe { sys::camera::cam_wrapper_fb_release(&mut self.fb) }
}
}
/// The driver could not hand out a frame.
#[derive(Debug, Clone, Copy)]
pub struct FrameError;
impl std::fmt::Display for FrameError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("camera returned no frame")
}
}
impl std::error::Error for FrameError {}
/// An initialized camera.
///
/// Held for the lifetime of the program: esp32-camera keeps a task and the
/// frame buffers running, and tearing them down would only be needed to power
/// the sensor down, which this program never does.
pub struct Camera {
sensor: SensorHint,
}
impl Camera {
/// Brings up the sensor and blocks until it has produced frames.
///
/// `vsync_invert` should be true for a directly connected DVP VSYNC (the
/// esp32-camera default) and false if the board inverts the line in hardware.
pub fn init(
sensor: SensorHint,
resolution: Resolution,
jpeg_quality: u8,
vsync_invert: bool,
) -> Result<Self> {
if jpeg_quality > 63 {
return Err(anyhow!("jpeg_quality must be 0-63, got {jpeg_quality}"));
}
let config = sys::camera::cam_wrapper_config_t {
pin_pwdn: -1,
pin_reset: pins::RESET,
pin_xclk: pins::XCLK,
pin_sccb_sda: pins::SIOD,
pin_sccb_scl: pins::SIOC,
pin_d: pins::DATA,
pin_vsync: pins::VSYNC,
pin_href: pins::HREF,
pin_pclk: pins::PCLK,
// 20 MHz is what the OV5640 datasheet calls for, and esp32-camera's
// OV5640 driver is only PLL-tuned for it; raising this without
// retuning the sensor's own PLL just corrupts frames.
xclk_freq_hz: 20_000_000,
ledc_timer: 0,
ledc_channel: 0,
pixel_format: PIXFORMAT_JPEG,
frame_size: resolution.framesize(),
jpeg_quality: jpeg_quality as i32,
// Three buffers, not two: esp32-camera needs a free buffer at every
// VSYNC, and with two a client that is still draining the previous
// frame costs a whole capture period. One is spent filling, one is
// being written to the socket, and one is spare for the moment the
// write blocks.
fb_count: 3,
fb_location: FB_IN_PSRAM,
// LATEST keeps the newest frame in the queue instead of handing out
// whatever was captured first, which matters once a client is slow
// enough for frames to pile up.
grab_mode: GRAB_LATEST,
sccb_i2c_port: 0,
// esp32-camera inverts VSYNC by default, which is right for the
// large majority of DVP sensors. A board that inverts the line
// itself needs this cleared: a mismatch here does not fail, it
// makes the driver latch frames on the wrong edge and the frame rate
// collapses while the pictures still look plausible.
vsync_invert: if vsync_invert { 1 } else { 0 },
};
let rc = unsafe { sys::camera::cam_wrapper_init(&config, sensor.into()) };
if rc != 0 {
return Err(anyhow!(
"camera init failed (esp_err 0x{:x}) - check the DVP/SCCB wiring and that \
octal PSRAM is enabled",
rc
));
}
info!("camera ready: {}", Self::sensor_name());
Ok(Self { sensor })
}
pub fn sensor_name() -> String {
unsafe {
let ptr = sys::camera::cam_wrapper_sensor_name();
if ptr.is_null() {
String::from("unknown")
} else {
CStr::from_ptr(ptr).to_string_lossy().into_owned()
}
}
}
/// Grabs the next frame.
///
/// This blocks until the sensor has one, which is what paces the stream.
/// `Err` means the driver could not hand out a frame at all; the camera is
/// still usable afterwards, so callers should keep going rather than treat
/// it as fatal.
pub fn next_frame(&self) -> Result<Frame, FrameError> {
let mut fb: sys::camera::cam_wrapper_fb_t = unsafe { std::mem::zeroed() };
let rc = unsafe { sys::camera::cam_wrapper_fb_get(&mut fb, self.sensor.into()) };
if rc != 0 || fb.data.is_null() || fb.len == 0 {
// Nothing was handed out, but be defensive in case a future
// driver version sets a partial frame.
unsafe { sys::camera::cam_wrapper_fb_release(&mut fb) };
return Err(FrameError);
}
Ok(Frame { fb })
}
}
/// Camera connector of the Freenove ESP32-S3-WROOM CAM board.
///
/// Every pin here is one the ESP32-S3 leaves free: 26..37 are bonded to the
/// module's SPI flash and octal PSRAM, 19/20 are the native USB pair this
/// firmware logs over, and 43/44 are the console UART. Touching IO_MUX on any
/// of the flash/PSRAM pins hangs the CPU mid-transfer, which is what happens if
/// you reuse a pin map written for a different board.
///
/// The data bus is the sensor's Y2..Y9, i.e. the low byte of the 10-bit pixel
/// word, so Y2 is D0 and Y9 is D7.
mod pins {
pub const RESET: i32 = 0;
pub const XCLK: i32 = 15;
pub const PCLK: i32 = 13;
pub const VSYNC: i32 = 6;
pub const HREF: i32 = 7;
pub const SIOD: i32 = 4;
pub const SIOC: i32 = 5;
/// D0..D7 == Y2..Y9, in the order `camera_config_t` expects.
pub const DATA: [i32; 8] = [11, 9, 8, 10, 12, 18, 17, 16];
}
src/main.rs
+340
View File
@@ -0,0 +1,340 @@
//! Camera firmware for an ESP32-S3-N16R8 with an OV2640 / OV3660 / OV5640 sensor,
//! written in Rust on top of ESP-IDF.
//!
//! Captures JPEG frames and streams them as UDP datagrams to any number of
//! clients. The wire format lives in the `protocol` crate, shared with the desktop
//! client in `client/`; see that crate's documentation for why the transport is
//! UDP rather than TCP.
//!
//! WiFi credentials come from the `WIFI_SSID` / `WIFI_PASS` environment variables
//! at build time, so no secrets end up in the source.
use std::time::{Duration, Instant};
use anyhow::{anyhow, Result};
use embedded_svc::wifi::{
AccessPointInfo, AuthMethod, ClientConfiguration, Configuration, PmfConfiguration,
};
use esp_idf_svc::hal::peripherals::Peripherals;
use esp_idf_svc::wifi::{BlockingWifi, EspWifi};
use esp_idf_svc::{eventloop::EspSystemEventLoop, nvs::EspDefaultNvsPartition};
use log::{error, info, warn};
mod camera;
mod udp;
use camera::{Camera, Resolution, SensorHint};
use udp::UdpSender;
const SSID: &str = env!("WIFI_SSID");
const PASSWORD: &str = env!("WIFI_PASS");
/// Authentication mode to assume, overriding whatever the scan reports.
///
/// Usually left unset: the scan tells us the AP's real mode. Pin it with
/// `WIFI_AUTH=wpa2` (or `wpa3`, `wpa2wpa3`, `wpa`, `none`) when the scan cannot
/// see the AP - a hidden SSID, for instance.
fn auth_override() -> Option<AuthMethod> {
let name = option_env!("WIFI_AUTH")?;
Some(match name {
"none" => AuthMethod::None,
"wep" => AuthMethod::WEP,
"wpa" => AuthMethod::WPA,
"wpa2" => AuthMethod::WPA2Personal,
"wpawpa2" => AuthMethod::WPAWPA2Personal,
"wpa3" => AuthMethod::WPA3Personal,
"wpa2wpa3" => AuthMethod::WPA2WPA3Personal,
"wapi" => AuthMethod::WAPIPersonal,
other => panic!(
"WIFI_AUTH={other} is not one of: \
none, wep, wpa, wpa2, wpawpa2, wpa3, wpa2wpa3, wapi"
),
})
}
/// Capture resolution, overridable at build time: `CAM_RES=vga ./scripts/build.sh`
const RESOLUTION: &str = match option_env!("CAM_RES") {
Some(value) => value,
None => "vga",
};
/// JPEG quality 0-63 (lower is better), overridable: `CAM_QUALITY=15 ./scripts/build.sh`
const QUALITY: &str = match option_env!("CAM_QUALITY") {
Some(value) => value,
None => "12",
};
/// Whether the driver inverts VSYNC, overridable so polarity can be A/B tested
/// without editing code: `CAM_VSYNC_INVERT=0 ./scripts/build.sh`.
///
/// A function rather than a `const` because matching on a `str` is not yet
/// allowed in const context on this toolchain.
///
/// `true` is esp32-camera's default and correct for most DVP sensors. `false` is
/// what a board that inverts VSYNC in hardware needs.
fn vsync_invert() -> bool {
!matches!(option_env!("CAM_VSYNC_INVERT"), Some("0"))
}
fn main() -> Result<()> {
// Must run once so the esp-idf-sys runtime patches link correctly.
esp_idf_svc::sys::link_patches();
esp_idf_svc::log::EspLogger::initialize_default();
let resolution: Resolution = RESOLUTION.parse()?;
let quality: u8 = QUALITY.parse()?;
info!("vsync invert: {}", vsync_invert());
let camera = Camera::init(SensorHint::Auto, resolution, quality, vsync_invert())?;
let peripherals = Peripherals::take()?;
let sys_loop = EspSystemEventLoop::take()?;
let nvs = EspDefaultNvsPartition::take()?;
let mut wifi = BlockingWifi::wrap(
EspWifi::new(peripherals.modem, sys_loop.clone(), Some(nvs))?,
sys_loop,
)?;
// `connect` starts the driver, then scans and associates, so that an SSID on
// a 5 GHz network (which this radio cannot see at all) is reported as such
// instead of a bare association timeout.
if let Err(err) = connect(&mut wifi) {
error!("could not join \"{SSID}\": {err}");
error!("check that the SSID is a 2.4 GHz network and the password is right");
return Err(err);
}
let ip = wifi.wifi().sta_netif().get_ip_info()?.ip;
// Modem power save trades latency for current, and for a video stream that
// trade is backwards: with PS on, an idle beacon interval costs hundreds of
// milliseconds, which shows up as stutter and a ping of 200-300 ms on an
// otherwise good link. The board is mains-powered anyway.
//
// This one binds as a bare `esp_err_t` rather than a `Result`.
let ps_rc = unsafe {
esp_idf_svc::sys::esp_wifi_set_ps(esp_idf_svc::sys::wifi_ps_type_t_WIFI_PS_NONE)
};
if ps_rc == 0 {
info!("WiFi power save disabled");
} else {
warn!("could not disable WiFi power save (esp_err 0x{ps_rc:x})");
}
// The board identifies itself by an id derived from its MAC, not its IP: DHCP
// hands out a different address across reboots, and a client that saved its
// binding to one would have to be reconfigured every time.
let sender_id = udp::local_sender_id();
let mut sender = UdpSender::new(sender_id)?;
info!("ESP32 camera is live on {ip}");
info!("clients will find it by broadcast; no configuration needed on this side");
let mut last_report = Instant::now();
let mut last_stats = sender.stats();
loop {
udp::pump(&camera, &mut sender)?;
let now = Instant::now();
if now.duration_since(last_report) >= Duration::from_secs(5) {
// Deltas rather than totals: the totals alone cannot distinguish "the
// sensor is slow" from "the sender is not keeping up", which is the
// distinction that matters when the frame rate is wrong.
let s = sender.stats();
let elapsed = now.duration_since(last_report).as_secs_f32();
info!(
"{:.1} fps, {} datagrams/s, {:.1} kB/s, {} send errors, \
{} clients subscribed, {} frames skipped (no client)",
(s.frames_sent - last_stats.frames_sent) as f32 / elapsed,
(s.fragments_sent - last_stats.fragments_sent) as f32 / elapsed,
(s.bytes_sent - last_stats.bytes_sent) as f32 / elapsed / 1024.0,
s.errors - last_stats.errors,
sender.peer_count(),
s.skipped_no_peer - last_stats.skipped_no_peer,
);
last_stats = s;
last_report = now;
}
}
}
/// Joins `SSID`, scanning first and retrying the plausible authentication modes.
///
/// The scan is not decoration. This radio is 2.4 GHz only, so a 5 GHz SSID never
/// shows up at all and a bare "association timed out" tells you nothing.
///
/// The retries are because the AP's advertised mode is not the same thing as the
/// mode it will accept. Routers commonly run WPA/WPA2 mixed, WPA2/WPA3 mixed, or
/// WPA3 with PMF mandatory, and a client pinned to the wrong one fails the 4-way
/// handshake *after* it has already associated - the failure looks identical to
/// a wrong password, so it is worth walking the list instead of guessing.
fn connect(wifi: &mut BlockingWifi<EspWifi<'static>>) -> Result<()> {
let pinned_auth = auth_override();
// A configuration has to exist before start(), which is what decides
// station vs AP mode; the real one is picked after the scan.
wifi.set_configuration(&client_config(pinned_auth.unwrap_or_else(default_auth_method))?)?;
wifi.start()?;
info!("wifi started");
info!("scanning for 2.4 GHz networks...");
let advertised = match wifi.wifi_mut().scan() {
Ok(aps) => {
report_scan(&aps);
advertised_auth(&aps)
}
Err(err) => {
warn!("scan failed: {err}");
None
}
};
for auth in candidates(advertised, pinned_auth) {
info!("trying {auth:?}...");
wifi.set_configuration(&client_config(auth)?)?;
// Driver-level (non-blocking) connect: `BlockingWifi::connect()` waits
// out the driver's own ~20 s association timeout, which would make
// walking the list of modes take minutes.
wifi.wifi_mut().connect()?;
// Bounded, so one bad mode costs a few seconds instead of the timeout.
let associated = wifi
.wifi_wait_while(
|| wifi.wifi().is_connected().map(|c| !c),
Some(Duration::from_secs(5)),
)
.is_ok();
let got_ip = associated
&& wifi
.ip_wait_while(
|| wifi.wifi().is_up().map(|up| !up),
Some(Duration::from_secs(4)),
)
.is_ok();
if got_ip {
let ip_info = wifi.wifi().sta_netif().get_ip_info()?;
if !ip_info.ip.is_unspecified() {
info!("wifi connected with {auth:?}: {ip_info:?}");
return Ok(());
}
}
warn!(
"{auth:?} did not complete the handshake ({})",
if associated { "associated but no DHCP address" } else { "no association" }
);
wifi.wifi_mut().disconnect().ok();
}
Err(anyhow!(
"no authentication mode joined \"{SSID}\" (tried {:?}); \
association succeeded but the 4-way handshake did not, \
which almost always means the password is wrong",
candidates(advertised, pinned_auth)
))
}
/// Authentication modes to try, most likely first.
///
/// A mode pinned with `WIFI_AUTH` is the only one tried - if it does not work,
/// trying others would just be guessing at a value the builder already stated.
fn candidates(advertised: Option<AuthMethod>, pinned: Option<AuthMethod>) -> Vec<AuthMethod> {
if let Some(auth) = pinned {
return vec![auth];
}
// "WPA" proper is a pre-2004 mode no current router speaks, and "None" is
// only meaningful without a password, so neither is worth a 10 s attempt.
let mut modes = vec![advertised.unwrap_or_else(default_auth_method)];
modes.extend([
AuthMethod::WPA2Personal,
AuthMethod::WPA2WPA3Personal,
AuthMethod::WPA3Personal,
]);
if PASSWORD.is_empty() {
modes.push(AuthMethod::None);
}
let mut seen = Vec::new();
modes.retain(|m| {
let fresh = !seen.contains(m);
seen.push(*m);
fresh
});
modes
}
fn client_config(auth_method: AuthMethod) -> Result<Configuration> {
Ok(Configuration::Client(ClientConfiguration {
ssid: SSID
.try_into()
.map_err(|_| anyhow!("WIFI_SSID too long (max 32 bytes)"))?,
bssid: None,
auth_method,
password: PASSWORD
.try_into()
.map_err(|_| anyhow!("WIFI_PASS too long (max 64 bytes)"))?,
channel: None,
// Advertising PMF keeps routers that offer WPA2+WPA3 mixed mode from
// dropping the handshake, while `required: false` still lets a WPA2-only
// AP through. Leaving this at NotCapable fails on the former.
pmf_cfg: PmfConfiguration::Capable { required: false },
..Default::default()
}))
}
fn default_auth_method() -> AuthMethod {
if PASSWORD.is_empty() {
AuthMethod::None
} else {
AuthMethod::WPA2Personal
}
}
/// What `SSID` advertises, if the scan saw it.
///
/// `None` means the scan did not find this network at all, which on its own is
/// the most useful diagnostic there is: on a 2.4 GHz-only radio it almost always
/// means the SSID belongs to a 5 GHz network, or the AP is on a channel this
/// region's regulatory domain forbids.
fn advertised_auth(aps: &[AccessPointInfo]) -> Option<AuthMethod> {
aps.iter()
.find(|ap| ap.ssid.as_str() == SSID)
.and_then(|ap| ap.auth_method)
}
/// Logs the visible networks, marking the one we are trying to join.
fn report_scan(aps: &[AccessPointInfo]) {
if aps.is_empty() {
warn!("scan found no networks at all");
return;
}
for ap in aps {
info!(
" {:<32} ch {:>2} {:>4} dBm {:?}{}",
ap.ssid,
ap.channel,
ap.signal_strength,
ap.auth_method,
if ap.ssid.as_str() == SSID { " <- target" } else { "" }
);
}
if advertised_auth(aps).is_none() {
error!("\"{SSID}\" was not in the scan results");
error!("this radio is 2.4 GHz only, so the AP is likely 5 GHz or on a forbidden channel");
}
}
src/udp.rs
+481
View File
@@ -0,0 +1,481 @@
//! UDP 视频发送器 + 设备发现。
//!
//! # 为什么不是 HTTP
//!
//! 这里替换掉过一个 MJPEG-over-HTTP 的实现，它在板子上的实测表现很差：传感器产出
//! 22 fps，而 TCP 流只送出 1.9 fps，且只占用了链路能力的 1%——链路是空的，帧却堵在
//! TCP 的队头阻塞后面等一个实时画面等不起的重传计时器。
//!
//! 每一帧改为独立的一串数据报。没有重传也没有确认：缺了分片的帧整个丢弃，下一帧
//! 顶上。这让实测帧率跟上了传感器，也把本来就有的带宽真正用了起来。
//!
//! # 发现与推流
//!
//! 双向，两个方向各司其职：
//!
//! - **板子**每 [`HELLO_INTERVAL`] 广播一次 [`MsgType::Hello`]，让客户端知道有谁
//! 在线、id 是多少。这是单向的，客户端什么都不用做。
//! - **客户端**广播 [`MsgType::Subscribe`]，板子收到后记住这个客户端的地址并开始
//! 定向推流。
//!
//! 两个方向都需要对方先开口，这正是必须双向的原因：只有板子广播的话，板子无从知道
//! "有谁要看我"。
//!
//! 板子用 MAC 派生的 [`sender_id`]（见 [`protocol::id_from_mac`]）而不是 IP 标识
//! 自己，因为客户端要能把绑定存进配置文件并在重启后继续有效——DHCP 换 IP 是常态，
//! MAC 不会变。
use std::net::{Ipv4Addr, SocketAddrV4};
use anyhow::{anyhow, Result};
use esp_idf_svc::sys as ffi;
use log::{debug, info, warn};
use crate::camera::{Camera, Frame};
/// 单个数据报的最大字节数。
///
/// 与 `protocol::MAX_FRAGMENT_PAYLOAD + protocol::HEADER_LEN` 一致。放在这里而不是
/// 从协议 crate 取，是因为这个值决定了发送缓冲的大小，必须在分配时就确定。
const SEND_BUFFER_SIZE: usize = protocol::HEADER_LEN + protocol::MAX_FRAGMENT_PAYLOAD;
/// 板子广播自身存在的间隔。
///
/// 比客户端的订阅间隔宽松：客户端的存在由板子的推流直接体现，而"有谁在线"这条
/// 信息需要快速到达新上线的客户端。
const HELLO_INTERVAL_MS: i64 = 2_000;
/// 同时向多少个客户端推流。
///
/// 每个对端每帧要一次 `sendto`，所以这是帧 CPU 成本的倍数。三台客户端（两队 + 裁判）
/// 够用，且不让一个陈旧的地址条目拖慢每一帧。
const MAX_PEERS: usize = 4;
/// 把 Rust 的 socket 地址转成 lwIP 期望的 C 结构。
///
/// 必须逐字段构造而不是 transmute：Rust 的 `SocketAddrV4` 是紧凑的 6 字节（端口在前、
/// 主机字节序），C 的 `sockaddr_in` 是 16 字节且地址是网络字节序——指针强转会发到
/// 一个乱地址，而且不会报错。
fn to_sockaddr(addr: SocketAddrV4) -> ffi::sockaddr_in {
let octets = addr.ip().octets();
ffi::sockaddr_in {
sin_len: std::mem::size_of::<ffi::sockaddr_in>() as u8,
sin_family: ffi::AF_INET as ffi::sa_family_t,
// 用 lwIP 的 htons，字节序不必在 Rust 侧推理。
sin_port: unsafe { ffi::lwip_htons(addr.port()) },
sin_addr: ffi::in_addr {
s_addr: u32::from_le_bytes(octets),
},
sin_zero: [0; 8],
}
}
/// 一个已知客户端。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
struct Peer {
addr: SocketAddrV4,
/// 最近一次收到它订阅的时刻。
last_seen_us: i64,
}
/// 发送器的统计。
///
/// 与"传感器产出了多少"分开计数：两者的差就是这条链路的全部故事。
#[derive(Debug, Clone, Copy, Default)]
pub struct SendStats {
/// 已交给网络的帧。
pub frames_sent: u32,
/// 已发出的数据报。
pub fragments_sent: u32,
/// 已发出的字节，含头。
pub bytes_sent: u64,
/// `sendto` 失败次数。
pub errors: u32,
/// 因没有客户端而未发送的帧。
pub skipped_no_peer: u32,
/// 收到的订阅数。
pub subscriptions: u32,
/// 已广播的 Hello 数。
pub hellos_sent: u32,
}
/// 向已订阅的客户端推流，并广播自身存在。
pub struct UdpSender {
socket: i32,
/// 本机标识，MAC 派生，重启后不变。
sender_id: u32,
peers: Vec<Peer>,
next_frame_id: u32,
stats: SendStats,
/// 读客户端订阅的暂存。内容无关紧要——只有源地址有用——所以刻意很小。
announce_buf: [u8; 64],
last_hello_us: i64,
}
/// 本机标识：由 MAC 派生，重启后不变。
pub fn local_sender_id() -> u32 {
let mut mac = [0u8; 6];
// SAFETY: `mac` 有 6 字节空间，函数会把 6 字节写满。
let rc = unsafe {
ffi::esp_read_mac(
mac.as_mut_ptr(),
ffi::esp_mac_type_t_ESP_MAC_WIFI_STA,
)
};
if rc != 0 {
warn!("could not read the WiFi MAC (esp_err {rc}); using a fixed id");
return 0xF1F0_0001;
}
let id = protocol::id_from_mac(&mac);
info!("this board's id is 0x{id:08X}, from MAC {mac:02X?}");
debug!("raw MAC bytes: {mac:02X?}");
id
}
impl UdpSender {
/// 打开 socket，绑定到视频端口以便客户端能找到它。
///
/// socket 既收也发：客户端广播订阅进来，板子往它看到的源地址推流。
pub fn new(sender_id: u32) -> Result<Self> {
// SAFETY: 纯 socket 设置，没有指向 Rust 内存的指针。
let socket = unsafe {
let fd = ffi::lwip_socket(ffi::AF_INET as i32, ffi::SOCK_DGRAM as i32, 0);
if fd < 0 {
// lwip_errno 没有单独导出，__errno 是同一个值，新libc 以 int 指针暴露。
return Err(anyhow!(
"could not create UDP socket: errno {}",
*ffi::__errno()
));
}
let enable: i32 = 1;
ffi::lwip_setsockopt(
fd,
ffi::SOL_SOCKET as i32,
ffi::SO_REUSEADDR as i32,
&enable as *const i32 as *const std::ffi::c_void,
std::mem::size_of::<i32>() as u32,
);
// 绑定到视频端口的所有接口。地址留 0，lwIP 视作 INADDR_ANY。
let addr = ffi::sockaddr_in {
sin_len: std::mem::size_of::<ffi::sockaddr_in>() as u8,
sin_family: ffi::AF_INET as ffi::sa_family_t,
sin_port: ffi::lwip_htons(protocol::DEFAULT_PORT),
sin_addr: ffi::in_addr { s_addr: 0 },
sin_zero: [0; 8],
};
let rc = ffi::lwip_bind(
fd,
&addr as *const ffi::sockaddr_in as *const ffi::sockaddr,
std::mem::size_of::<ffi::sockaddr_in>() as ffi::socklen_t,
);
// 非阻塞，这一条不是可选项。同一个 socket 用来在两帧之间查订阅，
// 阻塞的 recvfrom 会把采集循环一直挂到某个客户端发东西为止——那样一来，
// 没有客户端连着时相机会在第一帧后就停住，而不是跑在正常帧率上。
//
// lwIP 的 fcntl 在 C 里是可变参数；bindgen 把第三个参数标成普通 int，
// 对 fcntl 实际要的标志字来说这样正好。
if rc == 0 {
let flags = ffi::lwip_fcntl(fd, ffi::F_GETFL as i32, 0);
if flags < 0
|| ffi::lwip_fcntl(fd, ffi::F_SETFL as i32, flags | ffi::O_NONBLOCK as i32)
< 0
{
let err = *ffi::__errno();
ffi::lwip_close(fd);
return Err(anyhow!(
"could not set the UDP socket non-blocking: errno {err}"
));
}
}
if rc < 0 {
let err = *ffi::__errno();
ffi::lwip_close(fd);
return Err(anyhow!(
"could not bind UDP port {}: errno {err}",
protocol::DEFAULT_PORT
));
}
fd
};
info!(
"board 0x{sender_id:08X} listening on UDP {}, broadcasting presence every {HELLO_INTERVAL_MS} ms",
protocol::DEFAULT_PORT
);
Ok(Self {
socket,
sender_id,
peers: Vec::new(),
next_frame_id: 1,
stats: SendStats::default(),
announce_buf: [0u8; 64],
last_hello_us: 0,
})
}
pub fn stats(&self) -> SendStats {
self.stats
}
/// 广播一次自身存在。
///
/// 客户端靠它发现有谁在线，所以即使当前没有订阅者也照发——这正是"板子主动
/// 开口"的意义。
pub fn broadcast_hello(&mut self) {
let hello = protocol::Hello {
sender_id: self.sender_id,
firmware: 1,
};
let buf = hello.encode(protocol::MsgType::Hello);
// 广播地址 255.255.255.255。
let sa = to_sockaddr(SocketAddrV4::new(protocol::BROADCAST_ADDR, protocol::DEFAULT_PORT));
// SAFETY: 缓冲区与地址在调用期间都有效，sendto 会复制而不是持有指针。
let rc = unsafe {
ffi::lwip_sendto(
self.socket,
buf.as_ptr() as *const std::ffi::c_void,
buf.len(),
ffi::MSG_DONTWAIT as i32,
&sa as *const ffi::sockaddr_in as *const ffi::sockaddr,
std::mem::size_of::<ffi::sockaddr_in>() as ffi::socklen_t,
)
};
if rc >= 0 {
self.stats.hellos_sent += 1;
} else {
warn!("could not broadcast Hello: errno {}", unsafe { *ffi::__errno() });
}
}
/// 处理收到的订阅。
///
/// 新的订阅地址会被加入对端列表；已知地址只刷新时间戳，这样客户端可以靠重复
/// 订阅维持推流，而不需要任何超时淘汰。
pub fn handle_incoming(&mut self) {
loop {
let mut from = ffi::sockaddr_in {
sin_len: 0,
sin_family: 0,
sin_port: 0,
sin_addr: ffi::in_addr { s_addr: 0 },
sin_zero: [0; 8],
};
let mut from_len = std::mem::size_of::<ffi::sockaddr_in>() as ffi::socklen_t;
// SAFETY: `from` 是可写的 sockaddr，`from_len` 是 recvfrom 可以写入真实
// 长度的有效指针，接收缓冲有 `len` 字节空间。
let rc = unsafe {
ffi::lwip_recvfrom(
self.socket,
self.announce_buf.as_mut_ptr() as *mut std::ffi::c_void,
self.announce_buf.len(),
ffi::MSG_DONTWAIT as i32,
&mut from as *mut ffi::sockaddr_in as *mut ffi::sockaddr,
&mut from_len,
)
};
if rc < 0 {
// 没有待处理的数据，这是常态。
let err = unsafe { *ffi::__errno() };
if err != 11 /* EWOULDBLOCK */ {
warn!("recvfrom on the subscribe socket failed: errno {err}");
}
return;
}
if from.sin_family != ffi::AF_INET as ffi::sa_family_t {
continue;
}
let Some(_sub) = protocol::Subscribe::decode(&self.announce_buf[..rc as usize])
else {
// 不是订阅：可能是另一块板子的 Hello，或者将来的其它消息。安静跳过，
// 每包都打日志没有意义。客户端的身份由它的源地址决定，订阅里带的
// id 只是给它自己用的，所以这里不需要读。
continue;
};
let addr = SocketAddrV4::new(
Ipv4Addr::from(from.sin_addr.s_addr.to_le_bytes()),
// lwIP 交回的端口是网络字节序，和它进来时一样。
u16::from_be_bytes(from.sin_port.to_ne_bytes()),
);
self.stats.subscriptions += 1;
self.upsert_peer(addr);
}
}
/// 加入对端，或刷新已知地址的时间戳。
fn upsert_peer(&mut self, addr: SocketAddrV4) {
let now = now_us();
if let Some(peer) = self.peers.iter_mut().find(|p| p.addr == addr) {
peer.last_seen_us = now;
return;
}
if self.peers.len() >= MAX_PEERS {
warn!("peer limit ({MAX_PEERS}) reached, ignoring {addr}");
return;
}
info!("client at {addr} subscribed; streaming to it");
self.peers.push(Peer {
addr,
last_seen_us: now,
});
}
/// 移除超过 [`PEER_TIMEOUT_MS`] 未续订的对端。
///
/// 由客户端周期性续订，所以超时即代表它离开了——不需要任何探测包，也就避免了
/// 往客户端的接收端口发无意义的数据。
fn drop_stale_peers(&mut self) {
let cutoff = now_us() - PEER_TIMEOUT_MS * 1_000;
self.peers.retain(|p| {
let keep = p.last_seen_us >= cutoff;
if !keep {
info!("client {} stopped subscribing; dropping it", p.addr);
} else {
debug!("client {} still subscribed", p.addr);
}
keep
});
}
/// 把一帧发给每个对端。
///
/// 帧缓冲在返回时就交还驱动——多占着只会增加延迟。
pub fn send(&mut self, frame: &Frame) -> Result<()> {
if self.peers.is_empty() {
self.stats.skipped_no_peer += 1;
return Ok(());
}
let frame_id = self.next_frame_id;
self.next_frame_id = self.next_frame_id.wrapping_add(1);
let fragments = protocol::fragment_frame(
self.sender_id,
frame_id,
frame.data(),
frame.width() as u16,
frame.height() as u16,
);
// 每帧一个可复用缓冲：头每个分片重写，但分配每帧只发生一次。在 22fps 下这是
// 每秒 22 次与约 250 次分配的区别。
let mut datagram = Vec::with_capacity(SEND_BUFFER_SIZE);
for frag in &fragments {
datagram.clear();
frag.encode_into(&mut datagram);
for peer in &self.peers {
let sa = to_sockaddr(peer.addr);
// SAFETY: 两个缓冲区在调用期间都有效，sendto 会复制。
let rc = unsafe {
ffi::lwip_sendto(
self.socket,
datagram.as_ptr() as *const std::ffi::c_void,
datagram.len(),
ffi::MSG_DONTWAIT as i32,
&sa as *const ffi::sockaddr_in as *const ffi::sockaddr,
std::mem::size_of::<ffi::sockaddr_in>() as ffi::socklen_t,
)
};
if rc < 0 {
self.stats.errors += 1;
} else {
self.stats.fragments_sent += 1;
self.stats.bytes_sent += rc as u64;
}
}
}
self.stats.frames_sent += 1;
Ok(())
}
/// 周期性的housekeeping：广播 Hello、处理订阅、清理过期对端。
pub fn housekeeping(&mut self) {
let now = now_us();
if now - self.last_hello_us >= HELLO_INTERVAL_MS * 1_000 {
self.broadcast_hello();
self.last_hello_us = now;
}
self.handle_incoming();
self.drop_stale_peers();
}
/// 有多少个客户端在接收。
pub fn peer_count(&self) -> usize {
self.peers.len()
}
}
/// 客户端超过这个时间没有续订就被视为已离开。
///
/// 是订阅间隔的数倍，所以偶发丢一两个包不会把人踢掉。
const PEER_TIMEOUT_MS: i64 = 6_000;
impl Drop for UdpSender {
fn drop(&mut self) {
// SAFETY: socket 由本值拥有，且只关闭一次。
unsafe {
ffi::lwip_close(self.socket);
}
}
}
/// 单调微秒计数。
fn now_us() -> i64 {
// SAFETY: 无参数、无返回缓冲的纯查询。
unsafe { esp_idf_svc::sys::esp_timer_get_time() }
}
/// 采集循环：处理订阅，然后取一帧推出去。
///
/// 顺序是有意的：先处理订阅，一个新客户端就不必多等一整帧才被注意到。
pub fn pump(camera: &Camera, sender: &mut UdpSender) -> Result<()> {
sender.housekeeping();
let frame = match camera.next_frame() {
Ok(frame) => frame,
Err(_) => {
// 丢帧是可恢复的——驱动会重新同步——这里没什么可做的。
return Ok(());
}
};
sender.send(&frame)
}