Analysis of adversarial attacks on image segmentation systems

Egor Vorobyev

Abstract


The article discusses image segmentation methods and the problems associated with adversarial attacks on these systems. It is necessary to ensure the security of such systems, since segmentation is widely used in various computer vision tasks and can be a weak point in critical applications. An overview of different types of segmentation is presented,
including image segmentation, semantic segmentation, and panoptic segmentation. Popular architectures of segmentation models are considered, such as FCN, U-Net, YOLO, Segment Anything and others. The article analyzes adversarial attacks on image segmentation systems, including both digital and physical attacks. Special attention is paid to methods and algorithms for creating adversarial examples. The aim of the work is to
attract the attention of the research community to the problem of security of segmentation systems, to develop new, state-of-the-art and more robust to adversarial attacks segmentation.
models.


Full Text:

PDF (Russian)

References


«Image segmentation and classification of large scale satellite imagery for land use: A review of the state of the arts», Int. J. Civ. Eng. Technol, vol. 9, no. 11, pp. 1534–1541, 2018.

A. Arnab, O. Miksik, and P. H. Torr, «On the robustness of semantic segmentation models to adversarial attacks», in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 888–897.

V. Badrinarayanan, A. Kendall, and R. Cipolla, «Segnet: A deep convolutional encoder-decoder architecture for image segmentation», IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 12, pp. 2481–2495, 2017.

D. Bolya, C. Zhou, F. Xiao, and Y. J. Lee, «Yolact: Real-time instance segmentation», in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 9157–9166.

H. Caesar, J. Uijlings, and V. Ferrari, «Coco-stuff: Thing and stuff classes in context», in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1209–1218.

M. Cordts et al., «The cityscapes dataset for semantic urban scene understanding», in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 3213–3223.

Z. Keita, An introduction to convolutional neural networks (cnns), Available: https://www.datacamp.com/tutorial/introduction - to - convolutional - neural -networks-cnns, [Accessed July 18, 2024].

R. Didero and G. M. Conti, «Capable: Engineering, textile, and fashion collaboration, for citizens’ awareness and privacy protection», Human Factors for Apparel and Textile Engineering, vol. 32, p. 39, 2022.

A. Dosovitskiy et al., «An image is worth 16x16 words: Transformers for image recognition at scale», arXiv preprint arXiv:2010.11929, 2020.

A. Du et al., «Physical adversarial attacks on an aerial imagery object detector», in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022, pp. 1796–1806.

M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, «The pascal visual object classes (voc) challenge», International journal of computer vision, vol. 88, pp. 303–338, 2010.

I. J. Goodfellow, J. Shlens, and C. Szegedy, «Explaining and harnessing adversarial examples», arXiv preprint arXiv:1412.6572, 2014.

I. Goodfellow, Y. Bengio, and A. Courville, Deep learning. MIT press, 2016.

A. Guesmi, M. A. Hanif, B. Ouni, and M. Shafique, «Physical adversarial attacks for camera-based smart systems: Current trends, categorization, applications, research challenges, and future outlook», IEEE Access, 2023.

K. He, X. Zhang, S. Ren, and J. Sun, «Deep residual learning for image recognition», in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.

K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick, «Masked autoencoders are scalable vision learners», in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 16 000–16 009.

J. Hendrik Metzen, M. Chaithanya Kumar, T. Brox, and V. Fischer, «Universal adversarial perturbations against semantic image segmentation», in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2755–2764.

G. Jocher, A. Chaurasia, and J. Qiu, Ultralytics YOLO, version 8.0.0, Jan. 2023. [Online]. Available: https://github.com/ultralytics/ultralytics.

H. Kumar, Quick intro to semantic segmentation: Fcn, u-net and deeplab, Available: https://kharshit.github.io / blog / 2019 / 08 / 09 / quick - intro - to - semantic - segmentation, [Accessed June 12, 2024].

A. Kirillov et al., «Segment anything», in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 4015–4026.

A. Kurakin, I. Goodfellow, and S. Bengio, «Adversarial machine learning at scale», arXiv preprint arXiv:1611.01236, 2016.

M. Lee and Z. Kolter, «On physical adversarial patches for object detection. arxiv», arXiv preprint arXiv:1906.11897, 2019.

T.-Y. Lin et al., «Microsoft coco: Common objects in context», in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, Springer, 2014, pp. 740–755.

T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, «Feature pyramid networks for object detection», in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2117–2125.

T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, «Focal loss for dense object detection», in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2980–2988.

J. Long, E. Shelhamer, and T. Darrell, «Fully convolutional networks for semantic segmentation», in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431–3440.

S. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, and D. Terzopoulos, «Image segmentation using deep learning: A survey», IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 7, pp. 3523–3542, 2021.

Y. Mirsky, «Ipatch: A remote adversarial patch», Cybersecurity,

vol. 6, no. 1, p. 18, 2023.

R. Mottaghi et al., «The role of context for object detection and semantic segmentation in the wild», in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.

F. Nesti, G. Rossolini, S. Nair, A. Biondi, and G. Buttazzo, «Evaluating the robustness of semantic segmentation for autonomous driving against realworld adversarial patch attacks», in Proceedings of

the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022, pp. 2280–2289.

K. O’Shea and R. Nash, An introduction to convolutional neural networks, 2015. [Online]. Available: https://arxiv.org/abs/1511.08458.

A. Radford et al., «Learning transferable visual models from natural language supervision», in International conference on machine learning, PMLR, 2021, pp. 8748–8763.

J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, «You only look once: Unified, real-time object detection », in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779– 788.

J. Redmon and A. Farhadi, «Yolov3: An incremental improvement», arXiv preprint arXiv:1804.02767, 2018.

O. Ronneberger, P. Fischer, and T. Brox, «U-net: Convolutional

networks for biomedical image segmentation », in Medical image computing and computerassisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5- 9, 2015, proceedings, part III 18, Springer, 2015, pp. 234–241.

N. Shiledarbaxi, Semantic vs instance vs panoptic: Which image segmentation technique to choose, Available: https : / / analyticsindiamag . com / ai - mysteries / semantic - vs - instance - vs - panoptic - which - image - segmentation - technique - to - choose/, [Accessed July 4, 2024].

K. Simonyan and A. Zisserman, «Very deep convolutional networks for large-scale image recognition», arXiv preprint arXiv:1409.1556, 2014.

A. Vaswani et al., «Attention is all you need», Advances in neural information processing systems, vol. 30, 2017.

Satellite imagery is helping governments to combat deforestation, Available: https : / / analyticsindiamag . com/ai-mysteries/semantic-vs-instance-vs-panopticwhich - image - segmentation - technique - to - choose/, [Accessed July 4, 2024].

C. Xie, J. Wang, Z. Zhang, Y. Zhou, L. Xie, and A. Yuille, «Adversarial examples for semantic segmentation and object detection», in Proceedings of the IEEE international conference on computer vision, 2017, pp. 1369–1378.

Z. Zhang, S. Huang, X. Liu, B. Zhang, and D. Dong, «Adversarial attacks on yolact instance segmentation», Computers & Security, vol. 116, p. 102 682, 2022.

B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba, «Scene parsing through ade20k dataset», in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 633–641.

C. Yu, C. Gao, J. Wang, G. Yu, C. Shen, and N. Sang, «Bisenet v2: Bilateral network with guided aggregation for real-time semantic segmentation», International journal of computer vision, vol. 129, pp. 3051– 3068, 2021.

Y. Zhu, C. Miao, F. Hajiaghajani, M. Huai, L. Su, and C. Qiao, «Adversarial attacks against lidar semantic segmentation in autonomous driving», in Proceedings of the 19th ACM conference on embedded networked sensor systems, 2021, pp. 329–342.

A. Nguyen and B. Le, «3d point cloud segmentation: A survey», in 2013 6th IEEE conference on robotics, automation and mechatronics (RAM), IEEE, 2013, pp. 225–230.


Refbacks

  • There are currently no refbacks.


Abava  Кибербезопасность IT Congress 2024

ISSN: 2307-8162