Rerun Files

Rerun is an easy-to-use database and visualization toolbox for multimodal and temporal data. Try it live at https://rerun.io/viewer.

• Python: pip install -U rerun-sdk
• Rust: cargo add rerun and cargo install rerun-cli --locked
• Online demo: https://rerun.io/viewer
• C++ FetchContent: https://github.com/rerun-io/rerun/releases/download/0.34.0/rerun_cpp_sdk.zip

---

🧳 Migration guide: https://rerun.io/docs/reference/migration/migration-0-34

✨ Overview & highlights

Viewer MCP

We've added a MCP that allows an llm agent to see and interact with the Viewer! You could ask your agent to

• verify its work looks as expected in the Viewer.
• debug a bug when something doesn't show up right.
• explore a recording or dataset to search for specific patterns.

The agent has full control over the Viewer, meaning it can see and click any widget.

Here's an example where Claude Sonnet was asked to create a fancy particle animation of the Rerun logo and verify its work using the mcp in the open Viewer (sped up by a lot, except when showing the end result):

https://github.com/user-attachments/assets/14ffe7ed-6000-4193-900c-627784682125

Once it wrote the script, it logged the recording to the Viewer, and then iterated until the result looked as requested. It adjusted the camera position, improved the particle rendering by looking at different frames in the animation, and then debugged why the fade out animation was still showing particles on the last frame.

Full prompt

> /goal Create a new rerun python example in this folder that uses reruns 2D shapes to recreate the rerun logo (rerun-wordmark-black.svg). > There should be a nice fade-in animation in the beginning, 10 frames duration. Then pause a bit with the full rerun logo visible and then > the shapes should explosively fade away with a 20 frame animation before the recording ends. > > You may only stop once the recreated logo in the viewer looks close to the provided svg (black text, white background). > Use the mcp to verify in the open viewer, don't ever kill it. Once done, launch an opus agent and ask it to judge how > closely it looks to the original image. Keep going until it's convinced that it looks close.

See our mcp docs to get started.

Learning course

https://rerun.io/learn is a great way to learn how the Rerun data model covers the full physical AI experiment loop. It is a short, hands-on course for robotics ML engineers who want the full robot learning data loop in one place:

raw data -> RRD -> derived layers -> dataset queries -> training -> evaluation

Rerun agent skills

We added new skills to the Rerun repo to make it easier to investigate existing robotics data with Rerun. You can install the skills in your project via:

:::sh
npx skills add rerun-io/rerun

The new learning course also shows how these agent skills can be used to collect, refine and train with robotics data.

VoxelGridMap archetype

Rerun now supports sparse voxel grids through a new VoxelGridMap archetype (thanks to @makeecat for the contribution!). The archetype supports sparse indexing, anisotropic voxel sizes, pose offsets, and optional explicit colors or values & colormap per voxel.

Rerun's MCAP importer now also converts the dense ROS nav2_msgs/VoxelGrid and Foxglove VoxelGrid formats to Rerun VoxelGridMap.

And if you wonder how the smooth 3D navigation through the voxel scene in this video was done, see below!

https://github.com/user-attachments/assets/87fb80da-66dd-4fcd-8b35-ab553696f536

🎮 Gamepad support in 3D views

You can now use a gamepad to navigate 3D views in the native viewer. This makes it easier to do fine-grained, complex maneuvers with varying speed - e.g. for navigating large scenes or for screen videos. Analog sticks control the eye position and look target, shoulder triggers move the eye up and down, and shoulder buttons accelerate/decelerate.

Note: The gamepad feature is currently experimental and can be activated through the settings menu. Switch the 3D view's eye controls to FirstPerson for optimal experience. Under the hood, we use the gilrs crate that supports a wide range of devices.

Drag & drop components

You can now drag & drop a component right from the streams panel to visualize it in a Time series view or Status timeline.

https://github.com/user-attachments/assets/d70587a9-2020-4ae8-9cf3-0fef54dcf896

Transform debugging tool

We added a new debugging UI for visual introspection of the 3D transform cache. This allows to view the tree structure of the transform hierarchy, including potentially disconnected trees, and inspect the latest stored values of each frame node or transform edge. The UI supports horizontal and vertical tree layout and you can filter by transform type (e.g. static or temporal).

Note: this UI is currently a tab in the dev panel (accessible via "Toggle dev panel" in the menu or ctrl/cmd+shift+m). But we are open to making this a dedicated view in the future - let us know if you have any feedback!

https://github.com/user-attachments/assets/b4b1ea6e-bce9-4e88-9ede-262f545e3b47

⚠️ Breaking changes

• SDK: If you relied on the log_tick timeline being automatically created, you'll now have to call set_log_tick_enabled(true).
• Python: rr.send_dataframe is now stricter for more correctness. See the migration guide for more details.
• Python: The deprecated python module rerun.recording has been removed; Use rerun.experimental.RrdReader instead.
• SDK: Several deprecated DatasetEntry methods have been removed.
• SDK: ParquetReader column rules have been removed in favor of lenses.

🧳 Migration guide: https://rerun.io/docs/reference/migration/migration-0-34

🔎 Details

🪵 Log API

• Make log_tick timeline OPT-IN f734978
• Allow to change shading of Points3D cd7fa3e
• Extract out re_mp4_reader from importer 6eeeede
• Add stream-mode support and Python Mp4Reader LazyChunkStream binding e87dd89

🐍 Python API

• Add headless viewer mode 7e31c42
• rerun-sdk[datafusion] and rerun-sdk[dataplatform] extras are now rerun-sdk[catalog] 182cbb0
• Remove deprecated Recording and related APIs 2156155
• Split dataloader queries into windowed and keyframe-anchored 9ccc89e
• Introduce direct ChunkStore querying with .reader() d14018e
• Fix ViewerClient.close not closing subprocesses on Windows 2bc53b6
• feature-removal: drop custom indices 4dd9c78
• Deprecate DatasetEntry.manifest() fd24de9
• Make Chunk.from_record_batch more flexible d1771d4
• Remove column_rules from ParquetReader API d18da8e
• Add casting capability to derive lens f76a7d5
• Introduce pack built-in function to lenses a5ea965
• Introduce DeriveLens helpers for common components a93eb62
• Add support for duration timelines to FixedRateSampler 45bddb7

🦀 Rust API

• Add headless viewer mode 7e31c42

🪳 Bug fixes

• Take grpc server into account when purging viewer memory f83f167
• Move relative time view range when moving time cursor e9b22c5
• Don't show loader for encoded images when playing before their encoding data has been loaded c2eed0a
• GC ever growing fields in ChunkStore f5747cf
• Fix clicking in-view links to entities outside of view (usually via transform tree) b82a1d9
• add missing App::logic callbacks to examples #12810 (thanks @adsick!)
• Do not delete "duplicate" chunks that contain transform data 11b90ed
• Fix sort order of null values in table UI 8a0f7c5
• Navigate back after closing 71ecad0
• Fix custom visualizer example showing a black viewer e068ce8
• Fix face_rendering on arkit_scenes example 3d04f6f
• Fix ROS 2 reflection decoding of byte/char, empty specs, and wstring be0a632
• Handle codec changing for video-likes f7466ef
• Retain entry list in recording panel on refresh e7357ba
• Don't buffer & fetch more of entities based on what's hovered d9b3008
• rerun rrd optimize: continue on error 63e0882
• Look at source_component and selector when assigning colors to plots 71e2cf7
• Fix viewer hang when loading static compressed images b9acd34

🌁 Viewer improvements

• Temporary time pause on scrubbing bfea333
• Experimental gamepad support for 3D spatial views (native only) da470f7
• Rename memory panel -> dev panel c54e7b8
• Show rejection reason on state component drop b6a764d
• Table blueprint registration instead of base64 encoded table blueprints f2a2805
• Hold command/control to see and drag all preview timelines 91286ba
• Improve default & visualizer reporting for GridMap colormap e8386c5
• Table blueprint registration for segment tables 0c60d04
• Add sparse VoxelGridMap archetype fa40ec1 (thanks @makeecat!)
• Expose App::current_query() for external viewer #12811 (thanks @adsick!)
• Don't play time forward when video is buffering c4c6832
• Local catalog server 3c6b02e
• Drag & drop scalars into time series view ae4a157
• Always buffer time 81384a4
• Visualize transform trees in dev panel a534ac6
• Query only visible parts of the state timeline 9118c03
• Add rerun viewer-mcp aa56c88
• Hide screenshot notification on automated screenshots (scripts, mcp) 8924417
• State timeline view is now stable 0a26a8b

🗄️ OSS server

• Fix slow registration calls from OSS Catalog Server 5b74d8f

🚀 Performance improvements

• Cache string interning calls a05a4f8
• Skip empty visualizers 7d6aeef
• Don't load blueprint for previews e861b9e
• Redap client connection pool 5406d3a
• Replace cdr-encoding with re_cdr d2bc3b8
• Make the .ply parser ~10x faster using a custom PropertyAccess 6f8fb76

🧑‍🏫 Examples

• Local Vector Search example 0a336d5

🖼 UI improvements

• Add button to copy server URL 15aeda9
• Implement WatchEvents in re_server 8e07bdb
• Show average bitrate for selected video 9e28016

🧢 MCAP

• MCAP: add support for ROS nav2_msgs/VoxelGrid 651c140
• MCAP: add support for foxglove.VoxelGrid c8580c1
• Move sensor_msgs/msg/MagneticField to lens fe9b74f

🧑‍💻 Dev-experience

• Add Rerun agent skills 6732840
• Show bound and connect URLs when serving web viewer #12753 (thanks @terror!)
• Skills: mandate the idiomatic reader+lens pipeline (steer away from hand-built chunks) c436da5

📚 Docs

• Document catalog entry renaming and directory delimiter 57da6f3

📦 Dependencies

• chore: update lance, datafusion, and arrow 47d29ff

🤷‍ Other

• Push down selected components to fuzzy_descriptors 92f2281
• Add RRD footers section to rerun rrd stats 4ab8388
• Bound server chunk scans with per-segment index-value pushdown 990166c
• sdk: retry+backoff on behalf of the customer 2ef0fde