PROJECT TITLE :

Accurate and Robust Video Saliency Detection via Self-Paced Diffusion

ABSTRACT:

In order to estimate video saliency in the short term, traditional video saliency detection algorithms usually follow the common bottom-up thread. As a result, when the collected low-level clues are repeatedly ill-detected, such approaches cannot prevent the tenacious accumulation of errors. Also, noticing that a fraction of the video frames on the time axis are not close to the current video frame may help with saliency identification in the present video frame.

Thus, we propose to solve the aforementioned problem with our newly-designed key frame strategy (KFS), whose core rationale is to reveal the valuable long-term information using both the spatial-temporal coherency of the salient foregrounds and the objectness prior (i.e., how likely it is for an object proposal to contain an object of any class). We could use all of this newly discovered long-term data to guide our subsequent "self-paced" saliency diffusion, which allows each important frame to establish its own diffusion range and strength in order to remedy those video frames that were missed.

At the algorithmic level, we partition a video sequence into short-term frame batches before obtaining object proposals frame by frame. After that, we use a pre-trained deep saliency model to collect high-dimensional features in order to describe the spatial contrast for each item proposition. Because the contrast computation within several neighbored video frames (i.e., in a non-local way) is highly insensitive to appearance fluctuation, object proposals with high-quality low-level saliency estimation usually show considerable temporal similarity.

The salient foregrounds' long-term common consistency (e.g., appearance models/movement patterns) could then be disclosed directly by similarity analysis. We improve detection accuracy even further by using self-paced long-term information guided saliency diffusion. We ran thorough tests to compare our method to 16 state-of-the-art methods using the four largest publicly accessible benchmarks, and the results show that our method is superior in terms of accuracy and resilience.