Integrate graspgen in manipulationmodule as a service

[JalajShuklaSS]

Summary

Adds @rpc generate_grasps(pointcloud, scene_pointcloud) -> PoseArray to ManipulationModule that lazily spins up a DockerModule(GraspGenModule) on first call, managing the full Docker lifecycle

How it works
ManipulationModule lazily creates and starts a GraspGen Docker container on first generate_grasps() call, to the point cloud to GraspGenModule via LCMRPC, and returns ranked grasp poses as PoseArray.

Changes
dimos/manipulation/manipulation_module.py

Closes #1130 DIM-390

[greptile-apps]

Greptile Overview

Greptile Summary

Adds GraspGen grasp generation capability to ManipulationModule via a new @rpc generate_grasps() method. The implementation lazily initializes a DockerModule(GraspGenModule) on first call and properly cleans it up in the stop() method.

Key Changes:

• Added 8 configurable GraspGen parameters to ManipulationModuleConfig (gripper type, number of grasps, collision filtering, etc.)
• Implemented lazy initialization pattern in _get_graspgen() that creates the Docker container only when needed
• Added proper lifecycle management by stopping the Docker container in stop()
• Updated unit tests to initialize _graspgen field

Issues Found:

• Missing exception handling in _get_graspgen() - if Docker initialization or startup fails, the method returns None without logging the error
• The generate_grasps() RPC method silently returns None when GraspGen is unavailable without informing callers why

Confidence Score: 3/5

• Safe to merge with minor risk - missing error handling could cause silent failures
• The implementation follows good patterns (lazy initialization, proper lifecycle management, test updates) but lacks exception handling in the critical _get_graspgen() method. Docker initialization failures will silently return None without logging errors, making debugging difficult.
• Pay close attention to dimos/manipulation/manipulation_module.py - add error handling to _get_graspgen() method

Important Files Changed

Filename	Overview
dimos/manipulation/manipulation_module.py	Adds lazy-initialized GraspGen Docker integration with generate_grasps() RPC method. Missing exception handling in _get_graspgen() could cause silent failures.
dimos/manipulation/test_manipulation_unit.py	Adds _graspgen initialization to test fixture - correctly maintains test isolation.

Sequence Diagram

sequenceDiagram
    participant Client
    participant ManipulationModule
    participant DockerModule
    participant GraspGenModule
    
    Client->>ManipulationModule: generate_grasps(pointcloud, scene_pointcloud)
    ManipulationModule->>ManipulationModule: _get_graspgen()
    
    alt First call (lazy init)
        ManipulationModule->>ManipulationModule: get_data("models_graspgen")
        ManipulationModule->>DockerModule: new DockerModule(GraspGenModule, ...)
        ManipulationModule->>DockerModule: start()
        DockerModule-->>ManipulationModule: started
    else Subsequent calls
        Note over ManipulationModule: Return cached _graspgen
    end
    
    ManipulationModule->>GraspGenModule: generate_grasps(pointcloud, scene_pointcloud)
    GraspGenModule->>GraspGenModule: Run inference
    GraspGenModule-->>ManipulationModule: PoseArray
    ManipulationModule-->>Client: PoseArray
    
    Client->>ManipulationModule: stop()
    ManipulationModule->>DockerModule: stop()
    DockerModule-->>ManipulationModule: stopped

Loading

[greptile-apps]

_{2 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Missing error handling - if get_data(), get_project_root(), DockerModule(), or .start() raise exceptions, _graspgen remains None but no error is logged or raised.

Suggested change

	def _get_graspgen(self) -> DockerModule | None:
	"""Get or create GraspGen Docker module (lazy init)."""
	if self._graspgen is not None:
	return self._graspgen
	from dimos.core.docker_runner import DockerModule
	from dimos.manipulation.grasping.graspgen_module import GraspGenModule
	from dimos.utils.data import get_data
	from dimos.utils.path_utils import get_project_root
	# Ensure GraspGen model checkpoints are pulled from LFS
	get_data("models_graspgen")
	project_root = get_project_root()
	docker_file = (
	project_root / "dimos" / "manipulation" / "grasping" / "docker_context" / "Dockerfile"
	)
	self._graspgen = DockerModule(
	GraspGenModule,
	docker_file=docker_file,
	docker_build_context=project_root,
	docker_image=self.config.graspgen_docker_image,
	docker_env={"CI": "1"}, # skip interactive system config prompt in container
	gripper_type=self.config.graspgen_gripper_type,
	num_grasps=self.config.graspgen_num_grasps,
	topk_num_grasps=self.config.graspgen_topk_num_grasps,
	grasp_threshold=self.config.graspgen_grasp_threshold,
	filter_collisions=self.config.graspgen_filter_collisions,
	save_visualization_data=self.config.graspgen_save_visualization_data,
	visualization_output_path=self.config.graspgen_visualization_output_path,
	)
	self._graspgen.start()
	return self._graspgen
	def _get_graspgen(self) -> DockerModule | None:
	"""Get or create GraspGen Docker module (lazy init)."""
	if self._graspgen is not None:
	return self._graspgen
	try:
	from dimos.core.docker_runner import DockerModule
	from dimos.manipulation.grasping.graspgen_module import GraspGenModule
	from dimos.utils.data import get_data
	from dimos.utils.path_utils import get_project_root
	# Ensure GraspGen model checkpoints are pulled from LFS
	get_data("models_graspgen")
	project_root = get_project_root()
	docker_file = (
	project_root / "dimos" / "manipulation" / "grasping" / "docker_context" / "Dockerfile"
	)
	self._graspgen = DockerModule(
	GraspGenModule,
	docker_file=docker_file,
	docker_build_context=project_root,
	docker_image=self.config.graspgen_docker_image,
	docker_env={"CI": "1"}, # skip interactive system config prompt in container
	gripper_type=self.config.graspgen_gripper_type,
	num_grasps=self.config.graspgen_num_grasps,
	topk_num_grasps=self.config.graspgen_topk_num_grasps,
	grasp_threshold=self.config.graspgen_grasp_threshold,
	filter_collisions=self.config.graspgen_filter_collisions,
	save_visualization_data=self.config.graspgen_save_visualization_data,
	visualization_output_path=self.config.graspgen_visualization_output_path,
	)
	self._graspgen.start()
	return self._graspgen
	except Exception as e:
	logger.error(f"Failed to initialize GraspGen Docker module: {e}")
	return None

[greptile-apps]

No error logging when _get_graspgen() returns None - the RPC call silently returns None without informing the caller why it failed

Suggested change

	@rpc
	def generate_grasps(
	self,
	pointcloud: PointCloud2,
	scene_pointcloud: PointCloud2 | None = None,
	) -> PoseArray | None:
	graspgen = self._get_graspgen()
	if graspgen is None:
	return None
	return graspgen.generate_grasps(pointcloud, scene_pointcloud)
	@rpc
	def generate_grasps(
	self,
	pointcloud: PointCloud2,
	scene_pointcloud: PointCloud2 | None = None,
	) -> PoseArray | None:
	graspgen = self._get_graspgen()
	if graspgen is None:
	logger.error("GraspGen Docker module not available")
	return None
	return graspgen.generate_grasps(pointcloud, scene_pointcloud)