On Cyber Risks of Generative Artificial Intelligence
Abstract
This article is devoted to an overview of the risks of generative models of Artificial Intelligence. The rapid development of large language models has seriously increased attention to the security of Artificial Intelligence models. From a practical point of view, in this case, we are talking about the security of deep learning models. Large language models are susceptible to poisoning attacks, evasion attacks, attacks aimed at extracting training data, etc. But, at the same time, new attacks appear that are related specifically to the generated content. Moreover, the latter constitute an obvious majority. Therefore, recently many works have appeared that try to systematize all the risks of generative models. For example, OWASP and NIST are engaged in this. A complete taxonomy of generative AI risks should serve as a basis for building testing systems for generative models. This paper provides an overview of generative AI risk specifications outlined by OWASP, the NIST profile, and the MIT risk repository. The purpose of such specifications is to create a base for testing generative models and tools intended for AI Red Teams.
Full Text:
PDF (Russian)References
Cyber Risk https://www.theirm.org/what-we-say/thought-leadership/cyber-risk/ Retrieved: Sep, 2024.
Chto takoe kiberriski i kak zastrahovat' svoj biznes https://ir.alfastrah.ru/posts/271 . Retrieved: Sep, 2024.
Namiot, D. E., E. A. Il'jushin, and I. V. Chizhov. "Iskusstvennyj intellekt i kiberbezopasnost'." International Journal of Open Information Technologies 10.9 (2022): 135-147.
Namiot, D. E., and E. A. Il'jushin. "Porozhdajushhie modeli v mashinnom obuchenii." International Journal of Open Information Technologies 10.7 (2022): 101-118.
Chang, Yupeng, et al. "A survey on evaluation of large language models." ACM Transactions on Intelligent Systems and Technology 15.3 (2024): 1-45.
Namiot, Dmitry. "Schemes of attacks on machine learning models." International Journal of Open Information Technologies 11.5 (2023): 68-86. (in Russian)
Mudarova, Ramina, and Dmitry Namiot. "Countering Prompt Injection attacks on large language models." International Journal of Open Information Technologies 12.5 (2024): 39-48. (in Russian)
Namiot, Dmitry, and Elena Zubareva. "About AI Red Team." International Journal of Open Information Technologies 11.10 (2023): 130-139.
Wach, Krzysztof, et al. "The dark side of generative artificial intelligence: A critical analysis of controversies and risks of ChatGPT." Entrepreneurial Business and Economics Review 11.2 (2023): 7-30.
Eiras, Francisco, et al. "Risks and Opportunities of Open-Source Generative AI." arXiv preprint arXiv:2405.08597 (2024).
Duffourc, Mindy, and Sara Gerke. "Generative AI in health care and liability risks for physicians and safety concerns for patients." Jama (2023).
14 Risks and Dangers of Artificial Intelligence (AI) https://builtin.com/artificial-intelligence/risks-of-artificial-intelligence Retrieved: 11.09.2024.
Namiot, Dmitry. "On cyberattacks using Artificial Intelligence systems." International Journal of Open Information Technologies 12.9 (2024): 132-141. (in Russian)
OWASP Top 10 for Large Language Model Applications https://owasp.org/www-project-top-10-for-large-language-model-applications/ Retrieved: 11.09.2024.
AI Risk Repository https://airisk.mit.edu/ Provereno 11.09.2024
NIST Trustworthy and Responsible AI - 600-1 https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf Provereno 11.09.2024
Pathmanathan, Pankayaraj, et al. "Is poisoning a real threat to LLM alignment? Maybe more so than you think." arXiv preprint arXiv:2406.12091 (2024).
Bowen, Dillon, et al. "Scaling Laws for Data Poisoning in LLMs." arXiv preprint arXiv:2408.02946 (2024).
Xu, Zihao, et al. "LLM Jailbreak Attack versus Defense Techniques--A Comprehensive Study." arXiv preprint arXiv:2402.13457 (2024).
Liu, Yi, et al. "Prompt Injection attack against LLM-integrated Applications." arXiv preprint arXiv:2306.05499 (2023).
Galli, Filippo, Luca Melis, and Tommaso Cucinotta. "Noisy Neighbors: Efficient membership inference attacks against LLMs." arXiv preprint arXiv:2406.16565 (2024).
Maini, Pratyush, et al. "LLM Dataset Inference: Did you train on my dataset?." arXiv preprint arXiv:2406.06443 (2024).
LLM AI Cybersecurity & Governance Checklist https://owasp.org/www-project-top-10-for-large-language-model-applications/llm-top-10-governance-doc/LLM_AI_Security_and_Governance_Checklist-v1.1.pdf Provereno 11.09.2024
Namiot, Dmitry, and Manfred Sneps-Sneppe. "On Audit and Certification of Machine Learning Systems." 2023 34th Conference of Open Innovations Association (FRUCT). IEEE, 2023.
Namiot, D., and E. Ilyushin. "On Certification of Artificial Intelligence Systems." Physics of Particles and Nuclei 55.3 (2024): 343-346.
AI RISK MANAGEMENT FRAMEWORK https://www.nist.gov/itl/ai-risk-management-framework Provereno 11.09.2024
Research Initiative: AI Red Teaming & Evaluation https://genai.owasp.org/2024/09/12/research-initiative-ai-red-teaming-evaluation/ Provereno 11.09.2024
OWASP Top 10 for LLM AI Red Teaming Methodologies, Guidelines, and Best Practices https://docs.google.com/document/d/1m06DMhonGuq8hTN30S-fAsuBA-ZK1UHMyzZamsZSTaE/edit Provereno 11.09.2024
MIT Researchers Create an AI Risk Repository https://ide.mit.edu/insights/mit-researchers-create-an-open-ai-risk-repository/ Provereno 11.09.2024
Slattery, Peter, et al. "The AI Risk Repository: A Comprehensive Meta-Review, Database, and Taxonomy of Risks From Artificial Intelligence." arXiv preprint arXiv:2408.12622 (2024).
International Scientific Report on the Safety of Advanced AI https://assets.publishing.service.gov.uk/media/6655982fdc15efdddf1a842f/international_scientific_report_on_the_safety_of_advanced_ai_interim_report.pdf
Verma, Apurv, et al. "Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)." arXiv preprint arXiv:2407.14937 (2024).
Namiot, D. E., E. A. Il'jushin, and I. V. Chizhov. "Tekushhie akademicheskie i industrial'nye proekty, posvjashhennye ustojchivomu mashinnomu obucheniju." International Journal of Open Information Technologies 9.10 (2021): 35-46.
Cifrovaja jekonomika = modeli dannyh + bol'shie dannye + arhitektura + prilozhenija? / V. P. Kuprijanovskij, N. A. Utkin, D. E. Namiot, P. V. Kuprijanovskij // International Journal of Open Information Technologies. – 2016. – T. 4, # 5. – S. 1-13. – EDN VWANDZ.
Razvitie transportno-logisticheskih otraslej Evropejskogo Sojuza: otkrytyj BIM, Internet Veshhej i kiber-fizicheskie sistemy / V. P. Kuprijanovskij, V. V. Alen'kov, A. V. Stepanenko [i dr.] // International Journal of Open Information Technologies. – 2018. – T. 6, # 2. – S. 54-100. – EDN YNIRFG.
Umnaja infrastruktura, fizicheskie i informacionnye aktivy, Smart Cities, BIM, GIS i IoT / V. P. Kuprijanovskij, V. V. Alen'kov, I. A. Sokolov [i dr.] // International Journal of Open Information Technologies. – 2017. – T. 5, # 10. – S. 55-86. – EDN ZISODV.
Refbacks
- There are currently no refbacks.
Abava Кибербезопасность IT Congress 2024
ISSN: 2307-8162