Skip to content

feat(pai): add radar sensor support to PAI converter and ncore_vis#64

Draft
janickm wants to merge 1 commit intoNVIDIA:mainfrom
janickm:feat/pai-radar-support
Draft

feat(pai): add radar sensor support to PAI converter and ncore_vis#64
janickm wants to merge 1 commit intoNVIDIA:mainfrom
janickm:feat/pai-radar-support

Conversation

@janickm
Copy link
Copy Markdown
Collaborator

@janickm janickm commented Mar 31, 2026

Summary

  • Extend the PAI data converter to discover and convert up to 19 radar sensors per clip into NCore V4 RadarSensorComponent format
  • Add radar visualization to ncore_vis (3D point cloud component with fusing, motion compensation, and radar-specific color styles including radial velocity, RCS, SNR)
  • Add radar projection overlay to the camera component in ncore_vis
  • Generalize ncore_project_pc_to_img.py to auto-detect and support radar sensors via RayBundleSensorProtocol

Data Signal Mapping

Per-detection radar signals from the PAI parquet are mapped as follows:

PAI Column NCore Field Notes
azimuth + elevation direction (float32 [N,3]) Spherical → unit Cartesian
distance distance_m (float32 [1,N]) Single return
radial_velocity generic_data["radial_velocity"] Doppler velocity (m/s)
rcs generic_data["rcs"] Radar Cross Section (dBsm)
snr generic_data["snr"] Signal-to-Noise Ratio (dB)
exist_probb generic_data["exist_probb"] Normalized uint8→float [0,1]
doppler_ambiguity, radar_model generic_meta_data Scan-level scalars

Key Implementation Details

  • Radar extrinsics are only present in the non-offline sensor_extrinsics parquet (the .offline variant only contains cameras + lidar). The converter falls back to the raw variant automatically.
  • Radar scans are grouped by scan_index (DataFrame index level 0), with each scan becoming one store_frame() call.
  • The ncore_vis radar component auto-discovers generic_data fields and offers them as color styles, defaulting to radial_velocity when available.

Test

Verified end-to-end with streaming conversion of clip 0683f871-7d49-4bf9-8201-6227af1221bc:

Loaded metadata: 7 cameras, 1 lidars, 9 radars
Loading non-offline extrinsics for 9 radar sensors missing from offline calibration
Radar radar_front_center_srr_0: 12 frames converted
...
(9 radar sensors total, each producing ~12 frames per second of data)

@janickm janickm self-assigned this Mar 31, 2026
@janickm janickm force-pushed the feat/pai-radar-support branch 5 times, most recently from dfa64b1 to 908caba Compare April 2, 2026 09:13
@janickm janickm force-pushed the feat/pai-radar-support branch from e2eefe8 to 5208697 Compare April 2, 2026 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant