Visualizing Data¶
Don't just take our word for it - check out scenes for yourself! The AVstack API is a flexible and easy-to-use tool for managing AV datasets. Here, we've linked in an example notebook that shows how to perform some visualizations.
import os
import avstack
import avapi
data_base = '../../data'
obj_data_dir_k = os.path.join(data_base, 'KITTI/object')
raw_data_dir_k = os.path.join(data_base, 'KITTI/raw')
data_dir_n = os.path.join(data_base, 'nuScenes')
data_dir_c = os.path.join(data_base, 'CARLA/ego-lidar')
Jupyter environment detected. Enabling Open3D WebVisualizer. [Open3D INFO] WebRTC GUI backend enabled. [Open3D INFO] WebRTCWindowSystem: HTTP handshake server disabled.
Instantiating Scene Managers¶
NOTE: on the first go-around, you will have to convert KITTI Raw To AVstack Longitudinal format. Conversion is fast and is only a one-time thing - once you've converted a sequence once, you'll never have to do it again! Set the convert_raw
flag to True
to perform conversion.
KSM = avapi.kitti.KittiScenesManager(obj_data_dir_k, raw_data_dir_k, convert_raw=False)
NSM = avapi.nuscenes.nuScenesManager(data_dir_n)
CSM = avapi.carla.CarlaScenesManager(data_dir_c)
Deriving SceneDatasets from SceneManagers¶
By selecting a scene index, you can derive a SceneDataset
which is a collection of one longitudinal scene. You can choose to get a scene by its scene_idx
or by its scene_name
.
KDM = KSM.get_scene_dataset_by_name(scene_name="2011_09_26_drive_0005_sync")
NDM = NSM.get_scene_dataset_by_index(scene_idx=3)
CDM = CSM.get_scene_dataset_by_index(scene_idx=0)
Visualizing Ground-Truth and Sensor Data¶
The avapi
comes with a few useful visualization tools.
DMs = [KDM, NDM, CDM] # let's loop over all to show the unified API!
frame_idx = [120, 20, 10] # because I want to show different frames per scene...
for DM, frame_idx in zip(DMs, frame_idx):
print(f'\nVisualization for {DM.name} dataset\n')
frame = DM.get_frames(sensor="main_camera")[frame_idx]
img = DM.get_image(frame, sensor='main_camera')
pc = DM.get_lidar(frame, sensor='main_lidar')
objects = DM.get_objects(frame, sensor='main_lidar')
print('----ground truth objects with projected 3d boxes on main camera')
avapi.visualize.snapshot.show_image_with_boxes(img, objects, inline=True)
print('----point cloud projected onto image')
avapi.visualize.snapshot.show_lidar_on_image(pc, img, inline=True)
Visualization for KITTI dataset ----ground truth objects with projected 3d boxes on main camera
----point cloud projected onto image
Visualization for nuScenes dataset ----ground truth objects with projected 3d boxes on main camera
----point cloud projected onto image
Visualization for CARLA dataset ----ground truth objects with projected 3d boxes on main camera
----point cloud projected onto image
Visualizing Sequences with Movies¶
NOTE: this only works in the notebook, not in the ReadTheDocs...it's also broken right now
# %matplotlib notebook
# object_results = avapi.visualize.replay.replay_ground_truth_from_data_manager(KDM)
# gt_tracks = {frame:KDM.get_objects(frame, sensor="main_camera") for frame in KDM.frames}
# avapi.visualize.create_track_percep_movie(KDM, gt_tracks, inline=True)