Generative Models in Machine Learning

Dmitry Namiot, Eugene Ilyushin

Abstract


This article, written for the Robust Machine Learning Curriculum, discusses the so-called Generative Models in Machine Learning. Generative models learn the distribution of data from some sample data set and then can generate (create) new data instances. Generative models are popular tools with a wide range of applications. Recent advances in deep learning have led to improvements in the architecture of generative models, and some current models can (in some cases) produce realistic enough results to fool both end-users (humans) and recognition and classification algorithms. Generative models are used in constructing adversarial attacks. Instead of looking for minimal modifications, as in classical evasion attacks, generator models allow, for example, to create adversarial examples completely from scratch. At the same time, generator models are just as vulnerable to adversarial attacks as classifiers. This is our first material on this topic and, obviously, consideration of this important topic will be continued.

Full Text:

PDF (Russian)

References


Generative Adversarial Networks https://developers.google.com/machine-learning/gan Retrieved: May, 2022

Nalisnick, Eric, et al. "Do deep generative models know what they don't know?." arXiv preprint arXiv:1810.09136 (2018).

Oussidi, Achraf, and Azeddine Elhassouny. "Deep generative models: Survey." 2018 International Conference on Intelligent Systems and Computer Vision (ISCV). IEEE, 2018.

Goodfellow, Ian. "Nips 2016 tutorial: Generative adversarial networks." arXiv preprint arXiv:1701.00160 (2016).

Reynolds, Douglas A. "Gaussian mixture models." Encyclopedia of biometrics 741.659-663 (2009).

Mixture models https://en.wikipedia.org/wiki/Mixture_model Retrieved: May, 2022

Gaussian Mixture Model https://www.kaggle.com/code/dfoly1/gaussian-mixture-model/notebook Retrieved: May, 2022

Build Better and Accurate Clusters with Gaussian Mixture Models https://www.analyticsvidhya.com/blog/2019/10/gaussian-mixture-models-clustering/ Retrieved: May, 2022

In Depth: Gaussian Mixture Models https://jakevdp.github.io/PythonDataScienceHandbook/05.12-gaussian-mixtures.html Retrieved: May, 2022

Eddy, Sean R. "What is a hidden Markov model?." Nature biotechnology 22.10 (2004): 1315-1316.

Next Word Prediction using Markov Model https://www.kaggle.com/discussions/getting-started/107497 Retrieved: May, 2022

Skrytaja markovskaja model' https://ru.wikipedia.org/wiki/%D0%A1%D0%BA%D1%80%D1%8B%D1%82%D0%B0%D1%8F_%D0%BC%D0%B0%D1%80%D0%BA%D0%BE%D0%B2%D1%81%D0%BA%D0%B0%D1%8F_%D0%BC%D0%BE%D0%B4%D0%B5%D0%BB%D1%8C Retrieved: May, 2022

Markov and Hidden Markov Model https://towardsdatascience.com/markov-and-hidden-markov-model-3eec42298d75 Retrieved: May, 2022

Jelinek, Frederick, John D. Lafferty, and Robert L. Mercer. "Basic methods of probabilistic context free grammars." Speech Recognition and Understanding. Springer, Berlin, Heidelberg, 1992. 345-360.

Raghavan, Sindhu, Adriana Kovashka, and Raymond Mooney. "Authorship attribution using probabilistic context-free grammars." Proceedings of the ACL 2010 conference short papers. 2010.

Blei, David M., Andrew Y. Ng, and Michael I. Jordan. "Latent dirichlet allocation." Journal of machine Learning research 3.Jan (2003): 993-1022.

Latent Dirichlet Allocation https://devopedia.org/latent-dirichlet-allocation Retrieved: May, 2022

Chen, Serena H., and Carmel A. Pollino. "Good practice in Bayesian network modelling." Environmental Modelling & Software 37 (2012): 134-145.

Nikkarila, Juha-Pekka, Ilmari Kangasniemi, and Janne Valtonen. "Bayesian networks: an example of software and some defence applications." (2015).

Heckerman, David. "A Bayesian approach to learning causal networks." arXiv preprint arXiv:1302.4958 (2013).

Hinton, Geoffrey E. "Boltzmann machine." Scholarpedia 2.5 (2007): 1668.

Boltzmann machine https://en.wikipedia.org/wiki/Boltzmann_machine Retrieved: May, 2022

Book-Recommender-System-RBM https://github.com/adityashrm21/book-recommender-system-rbm Retrieved: May, 2022

Understanding Variational Autoencoders (VAEs) from two perspectives: deep learning and graphical models https://jaan.io/what-is-variational-autoencoder-vae-tutorial/ Retrieved: May, 2022

Generative modelling using Variational AutoEncoders(VAE) and Beta-VAE’s https://medium.com/analytics-vidhya/generative-modelling-using-variational-autoencoders-vae-and-beta-vaes-81a56ef0bc9f Retrieved: May, 2022

Variational autoencoders https://www.jeremyjordan.me/variational-autoencoders/ Retrieved: May, 2022

Ghojogh, Benyamin, et al. "Factor analysis, probabilistic principal component analysis, variational inference, and variational autoencoder: Tutorial and survey." arXiv preprint arXiv:2101.00734 (2021).

Kuzina, Anna, Max Welling, and Jakub M. Tomczak. "Diagnosing vulnerability of variational auto-encoders to adversarial attacks." arXiv preprint arXiv:2103.06701 (2021).

Creswell, Antonia, et al. "Generative adversarial networks: An overview." IEEE Signal Processing Magazine 35.1 (2018): 53-65.

Faster Learning and Better Image Quality with Evolving Generative Adversarial Networks https://neurohive.io/en/state-of-the-art/faster-learning-and-better-image-quality-with-evolving-generative-adversarial-networks/ Retrieved: May, 2022

Generative Adversarial Nets (GAN) https://neerc.ifmo.ru/wiki/index.php?title=Generative_Adversarial_Nets_(GAN) Retrieved: May, 2022

Goodfellow, Ian, et al. "Generative adversarial nets." Advances in neural information processing systems 27 (2014).

Arjovsky, Martin, Soumith Chintala, and Léon Bottou. "Wasserstein generative adversarial networks." International conference on machine learning. PMLR, 2017.

Mirza, Mehdi, and Simon Osindero. "Conditional generative adversarial nets." arXiv preprint arXiv:1411.1784 (2014).

Artificial Intelligence in Cybersecurity. http://master.cmc.msu.ru/?q=ru/node/3496 (in Russian) Retrieved: Dec, 2021

Namiot D.E., Il'jushin E.A., Chizhov I.V. Tekushhie akademicheskie i industrial'nye proekty, posvjashhennye ustojchivomu mashinnomu obucheniju //International Journal of Open Information Technologies. – 2021. – T. 9. – No. 10. – S. 35-46.

Namiot, D. E., E. A. Il'jushin, and I. V. Chizhov. "ATAKI NA SISTEMY MAShINNOGO OBUChENIJa-OBShhIE PROBLEMY I METODY." International Journal of Open Information Technologies 10.3 (2022): 17-22.

Namiot D. E., Il'jushin E. A., Chizhov I. V. Osnovanija dlja rabot po ustojchivomu mashinnomu obucheniju //International Journal of Open Information Technologies. – 2021. – T. 9. – #. 11. – S. 68-74.


Refbacks

  • There are currently no refbacks.


Abava  Кибербезопасность MoNeTec 2024

ISSN: 2307-8162