[Task Clean-up][Camera] Dexterous Part 6/11: Add renderer presets to the Direct camera task#6416
[Task Clean-up][Camera] Dexterous Part 6/11: Add renderer presets to the Direct camera task#6416hujc7 wants to merge 2 commits into
Conversation
The Shadow Hand camera environment gains an RGB-depth camera preset so it can train with the Newton Warp renderer in addition to RTX, and the feature extractor exposes compute_cube_keypoints (the module-level compute_keypoints shim is deprecated in its favor). Validated by full camera training runs and replay video generation.
Greptile SummaryThis PR adds an
Confidence Score: 4/5The direct-env path (ShadowHandCameraEnv) is clean, and the preset/validation refactor is safe. The new ShadowHandCameraFeatures manager term has an AttributeError waiting to trigger when it is wired in. The new ShadowHandCameraFeatures.call unconditionally calls .torch on physics-state tensors (root_pos_w, root_quat_w), which are plain torch.Tensor objects in Isaac Lab. torch.Tensor has no .torch attribute, so this will raise AttributeError the first time the class is exercised. Camera output is guarded correctly with isinstance, making the inconsistency clear. The class is not yet wired into any env config in this PR, so CI will not catch it now, but it will fail on first real use. feature_extractor.py — specifically the ShadowHandCameraFeatures.call method at the object-state data access lines. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[ShadowHandCameraEnvCfg.validate_config] --> B[validate_shadow_hand_camera_settings]
B --> C{tiled_camera is PresetCfg?}
C -- Yes --> D[resolve to .default]
D --> C
C -- No --> E{renderer_cfg is PresetCfg?}
E -- Yes --> F[resolve to .default]
F --> E
E -- No --> G{renderer_type == newton_warp?}
G -- Yes --> H{unsupported data_types?}
H -- Yes --> I[raise ValueError: unsupported types]
H -- No --> J{depth-only + CNN enabled?}
G -- No --> J
J -- Yes --> K[raise ValueError: depth-only]
J -- No --> L[valid config]
subgraph New Presets
M[ShadowHandTiledCameraCfg]
M --> N[rgb: RGB only 3ch]
M --> O[rgb_depth: RGB+depth 4ch NEW]
M --> P[depth: depth-only benchmark]
M --> Q[albedo/shading variants]
M --> R[default/full: RGB+depth+seg 7ch]
end
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[ShadowHandCameraEnvCfg.validate_config] --> B[validate_shadow_hand_camera_settings]
B --> C{tiled_camera is PresetCfg?}
C -- Yes --> D[resolve to .default]
D --> C
C -- No --> E{renderer_cfg is PresetCfg?}
E -- Yes --> F[resolve to .default]
F --> E
E -- No --> G{renderer_type == newton_warp?}
G -- Yes --> H{unsupported data_types?}
H -- Yes --> I[raise ValueError: unsupported types]
H -- No --> J{depth-only + CNN enabled?}
G -- No --> J
J -- Yes --> K[raise ValueError: depth-only]
J -- No --> L[valid config]
subgraph New Presets
M[ShadowHandTiledCameraCfg]
M --> N[rgb: RGB only 3ch]
M --> O[rgb_depth: RGB+depth 4ch NEW]
M --> P[depth: depth-only benchmark]
M --> Q[albedo/shading variants]
M --> R[default/full: RGB+depth+seg 7ch]
end
Reviews (1): Last reviewed commit: "Add renderer presets to the Direct camer..." | Re-trigger Greptile |
| object_pos = object_asset.data.root_pos_w.torch - env.scene.env_origins | ||
| object_pose = torch.cat((object_pos, object_asset.data.root_quat_w.torch), dim=-1) |
There was a problem hiding this comment.
Unconditional
.torch on physics-state tensors will raise AttributeError
object_asset.data.root_pos_w and root_quat_w return plain torch.Tensor objects — they come from PhysX, not from the Warp renderer. PyTorch tensors do not have a .torch attribute, so these two lines will raise AttributeError: 'Tensor' object has no attribute 'torch' the first time this class is exercised.
The camera output (a few lines below) handles this correctly with an isinstance guard, but the object-state path does not. The same defensive pattern is needed here.
Strict review scope: the camera manager enablement (validation helper, keypoint relocation, manager observation terms) moves to the camera manager part; this part keeps the rgb_depth preset, its tests, and the Benchmark registration deprecation.
Summary
Isaac-Reorient-Cube-Shadow-Camera-Benchmark-Directregistration in favor of theenv.feature_extractor.enabled=falseoverride (no consumer exists; no other task family registers benchmark variants).Dependencies