Visual Distraction Adversarial Perturbations on SAM2-Based Video Tracking and Segmentation

Adonay Demewez Gebremedhin¹ Mohamad Alansari² Wessam Shehieb¹ Muzammal Naseer^2,3

¹ Ajman University, UAE² Khalifa University, UAE³ University of Western Australia, Australia

✨ We just got accepted to WACV 2027, paper coming soon ✨

Summary

Promptable video trackers such as SAM2 achieve accurate and robust performance making them important foundations for VOT/VOS tasks. We are interested in analysing the adversarial robustness of these trackers. Existing adversarial perturbations largely focus on local mask degradation and short-term feature disruption, leaving longer-term dynamics and competing-object interference, i.e. distractors, insufficiently explored. We introduce a white-box method that uses background distractors to disrupt target tracking persistently. Its Counterfactual Distractor Search propagates trajectories to find plausible competing regions that become increasingly effective under perturbation. We further analyze transferability and defence mechanisms that can be used to mitigate such impact.

Reference

@inproceedings{gebremedhin2027distractors,
  title = {Distractors are the Real Enemy: Visual Distraction Adversarial Perturbations on SAM2-Based Video Tracking and Segmentation},
  author = {Adonay Demewez Gebremedhin and Mohamad Alansari and Wessam Shehieb and Muzammal Naseer},
  year = {2027},
  booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)}
}