concepts.vision.depth_smoother.depth_anything_smoother.DepthAnythingV2Smoother

class DepthAnythingV2Smoother

Bases: object

The Depth Anything V2 smoother.

The Depth Anything V2 model is a depth estimation model that can be used to smooth the depth map. In particular, we will first use the Depth Anything V2 model to generate a predicted depth map of the input RGB image.

The Depth Anything V2 model is available in three sizes: ‘small’, ‘medium’, ‘large’.

Then, we will fit splines between the observed depth map and the predicted depth map. The spline fitting can be done in multiple ways:

• ‘linear’: fit a single line to the depth map using the predicted depth map.

• ‘multi-linear’: fit multiple lines to the depth map using the predicted depth map. We will take the median of the predicted depth map.

• ‘multi-cubic’: fit multiple cubic splines to the depth map using the predicted depth map. We will take the median of the predicted depth map.

• ‘cluster-cubic’: fit cubic splines to the depth map using the predicted depth map. We will cluster the depth map into several clusters and fit a spline to each cluster.

Finally, we will use the fitted splines to transform the predicted depth map to a smoothed depth map.

Methods

Attributes

MODEL_SIZE

__call__(rgb, depth, mask=None, visualize=False)

Smooth the depth map using the Depth Anything V2 model.

Parameters:

• rgb (ndarray) – the RGB image.

• depth (ndarray) – the input depth map.

• mask (ndarray | None) – the mask of the valid pixels. This mask will used to sample points from the depth/predicted_depth map.

• visualize (bool)

__init__(config=None, device='cpu')

Initialize the Depth Anything V2 smoother.

Parameters:

• config (DepthAnythingV2SmootherConfig | None) – the configuration of the smoother.

• device (str) – the device to use for the model.

__new__(**kwargs)

MODEL_SIZE = {'large': 'depth-anything/Depth-Anything-V2-Large-hf', 'medium': 'depth-anything/Depth-Anything-V2-Medium-hf', 'small': 'depth-anything/Depth-Anything-V2-Small-hf'}