Overview of Spatial Propagation
Spatial propagation is a mechanism used in computer vision tasks to help understand and fill in missing information in an image. One example of where spatial propagation is used is in depth completion tasks. In depth completion, the goal is to fill in missing depth information in an image, so that the image appears more complete and visually appealing. Spatial propagation helps by using non-local displacement and affinity information to guide how the depth information is filled in.
The basic idea behind spatial propagation is that pixels (or depth values) in an image are not always independent of one another. In fact, they can be strongly correlated and dependent on one another due to things like object edges or occlusion. By taking into account these underlying relationships between pixels, spatial propagation can help to more accurately fill in missing information.
Non-Local Disparity Propagation Module
In the context of depth completion, a common technique used to implement spatial propagation is to use a non-local disparity propagation module. This module essentially acts as the "brain" of the propagation process, by generating non-local affinities and offsets that can be used to guide the propagation of depth values.
One of the key benefits of using a non-local disparity propagation module is that it can help to more accurately fill in depth information at object edges. When using a frontal-parallel windowing technique (common in some depth completion algorithms), there is a tendency for disparity values to become blurred or spread out at object edges. By instead using plane affinity relations (as is done in the non-local disparity propagation module), it is possible to more accurately propagate depth values at object edges without this blurring effect.
Optimization of Disparities in Occluded Areas
Another challenge in depth completion is filling in depth values in occluded areas. These areas are typically areas of an image where depth information is not available due to objects blocking the view. In order to still fill in these areas, spatial propagation can be used to propagate depth information from non-occluded areas where the depth values are more confidently predicted.
By propagating depth information from these non-occluded areas to the occluded areas, the disparity values in the occluded areas can be optimized at the same time as the rest of the image. This approach can help to ensure that the final depth image is as complete and visually accurate as possible.
Spatial propagation is a powerful technique used in computer vision tasks like depth completion to help fill in missing information in images. By taking into account the correlations and dependencies between pixels, spatial propagation can help to more accurately infer missing depth values. Non-local disparity propagation modules are often used to implement spatial propagation, and can help to mitigate some of the challenges associated with object edges and occluded regions. Overall, spatial propagation is an effective way to improve the accuracy and completeness of depth images in a variety of applications.
