Netis Cloud Probe Files

This release is a correctness and hardening pass over cpworker, cpdaemon and cpctl: config validation, output statistics, control-socket robustness, packet parsing, and memory-safety fixes, plus the wire-format specifications for the VXLAN and ZMQ outputs. Several fixes change observable behaviour; read Upgrade Notes before upgrading.

⚠️ Upgrade Notes

• cpdaemon memory settings key. Worker memory settings are now read from cpm.worker.memory.*, as documented. The old cpm.worker.memory_policy.* keys (and CPDAEMON_CPM_WORKER_MEMORY_POLICY_* env vars) are no longer read — rename them. Deployments that set cpm.worker.memory.* were previously running with defaults and will now use the configured values. (#266)
• GRE key without a service tag. With no gre.service_tag, the key is now 0x0FFFFFFF / 0x1FFFFFFF / 0x2FFFFFFF (none / request / response) instead of 0xFFFFFFFF for every packet, so the request/response direction is carried again. Receivers that read the tag as key & 0x0FFFFFFF see the same value as before. A tag above 28 bits only keeps its low 28 bits in the key, with a warning. (#296)
• Output byte counters exclude encapsulation. fwd_*, direction_drop_*, ratelimit_drop_* and error_drop_* now count each packet once with its sliced captured length, without GRE/VXLAN/ZMQ headers. gre, vxlan and zmq therefore report lower byte counts (uploaded to CPM as fwdBytes); vxlan with split counts one packet per original packet, not per fragment; rate_limit_mbps no longer charges the 4–8 byte headers. (#308)
• ZMQ heartbeats are no longer counted as forwarded packets. An idle zmq output now reports fwd_packets 0; heartbeats are reported in heartbeat_packets. (#299)
• Stricter config parsing in cpworker. Trailing content after the top-level JSON value is rejected (#238). Out-of-range or fractional numeric values are rejected with an error naming the field and range instead of being silently truncated; a few fields (libpcap.snaplen, libpcap.buffer_size_mb, vxlan.vni1) are normalised with a log line (#234, [#235], [#239], [#250]). cpdaemon now validates CPM strategy values and skips an invalid strategy with a warning instead of failing the whole worker config (#259).
• cpctl. -f jsonl output follows docs/REFACTOR-CPCTL.md: errors on stderr are JSON, stats records use kind (raw/rate), info keys are snake_case (#251). ts now comes from cpworker's new wall_time field, so use cpctl 0.9.5 with cpworker 0.9.5 (#297).
• VXLAN split. IPv4 fragments are no longer split at L4 and are forwarded as captured, so they can exceed max_payload_size (#244).
• ZMQ slice. slice never cuts into the Ethernet header or VLAN tag stack; runt frames are counted as error_drop (#271).
• ZMQ shutdown. A pending batch is flushed on stop; with an unreachable collector, shutdown can take up to 5 s (ZMQ_LINGER) (#253).

🐛 Bug Fixes

cpworker

• Idle capture with timeout_ms > 0: on libpcap ≥ 1.8 an idle link blocked the main loop, so reload, SIGINT/SIGTERM, stats and ZMQ flushes stalled, and one idle task starved the others. The libpcap handle is now non-blocking. (#241, [#253])
• Control socket: concurrent clients no longer fail with EAGAIN; a silent client can't block others; requests split across writes are read correctly. (#260, [#261])
• QinQ direction detection: custom req_pattern now walks 0x88a8 / 0x9100 / 0x9200 tag stacks; previously all QinQ traffic was dropped as a direction drop. (#270)
• gre.pmtudisc had no effect (the code was compiled out); "dont" now clears DF. (#252)
• ZMQ output: VLAN walk out-of-bounds read, uuid now truly optional, pending batch flushed on destroy. (#231, [#249])
• file / rotating_file outputs now apply slice and rate_limit_mbps. (#248)
• rotating_file without max_file_interval rotated on every packet and overwrote files; it now defaults to 60 s. (#250)
• pcap_file read errors are reported instead of a silent "end of file". (#246)
• packet_split: IPv4 fragments, ihl < 5 / TCP data offset < 5 are no longer parsed as L4; a computed UDP checksum of 0 is sent as 0xFFFF. (#233, [#242], [#244], [#245])
• Logging: log lines from different threads no longer interleave; config parse errors report line/column of the config itself; -c <directory> reports not a regular file. (#293, [#294], [#295])
• Leaks and error paths: leak on every reload, crash on netns restore failure, leaks in the stats reply, and a stop/start race on the output and reload threads. (#262, [#268], [#272], [#273])
• Stats carry a wall-clock timestamp (wall_time); cpctl stats no longer prints 1970 dates. (#297)

cpdaemon

• Starting no longer fails on a stale pid file whose pid was reused by another process. (#264)
• CPM client requests honour context cancellation, so shutdown doesn't wait for the HTTP timeout. (#265)
• CPU limits below 1% are raised to the kernel minimum instead of stopping the worker. (#267)
• Logs sent to CPM show error text and group-qualified keys instead of ={e:0x...}. (#290)
• Stop() returns only after the worker's cgroup and pid file are cleaned up. (#292)

cpctl

• cpctl stats shows heartbeat_packets. (#297)

🔒 Security

• Fixed a heap overflow in nic.<if> substitution in capturer BPF filters; tokens now end at ( / ). (#258, [#263])
• Upgraded Go dependencies flagged by the release security gate: golang.org/x/crypto v0.55.0, golang.org/x/text, google.golang.org/grpc v1.83.2, golang.org/x/net v0.60.0. (#313)

📖 Documentation

• New docs/VXLAN-WIRE-FORMAT.md and docs/ZMQ-WIRE-FORMAT.md (with zh-Hans) specifying the output wire formats.
• USAGE-CPWORKER documents every numeric range and default, libpcap.timeout_ms, and the GRE key layout; the zh-Hans version is back in sync.
• REFACTOR-CPCTL.md defines the output counters and drops the unimplemented cpctl config. (#298)

🔧 CI / Tests

• CI now runs the cpworker unit and integration tests and the cripid tests; libzmq is pinned to v4.3.3. (#300, [#301])
• New wire-byte tests for GRE, VXLAN and ZMQ against the specs, and spec-based tests for checksums, packet parsing, ZMQ flushing, the control socket, and the cpdaemon/cpgolib CPM mapping. (#243)

Full Changelog: https://github.com/Netis/cloud-probe/compare/v0.9.4...v0.9.5