Synthesis of the Convolution Neural Networks Structures for Images Classification
AUTHORS
Nataliya Boyko,Department of Artificial Intelligence, Lviv Polytechnic National University, Lviv
ABSTRACT
In this work would be analyzed Generative Adversarial Networks (GANs) and how they work on the example of an artist’s style translation to landscape photographs. This architecture consists of a generator and discriminator which oppose each other in the minimax game. The goal of the discriminator is to learn original data distribution. At this time, discriminator evaluates how well a generator performs in reproducing original style. Since in this paper, we will work with photos and pictures, convolutional neural networks will be used as a generator and a discriminator. Also, would be considered a modification to GAN architecture, which can notably improve result: another pair of discriminator and generator will be added, and a new cycle loss would be introduced. A comparative analysis with other methods such as Neural Style Transfer will also be performed. The methods of comparative analysis of Neural Style Transfer and Pix2Pix are presented in the work. Neural Style Transfer approach is more stable and with a well-chosen photo can give better results, but is more limited. Pix2Pix, is another modification to GAN architecture with the only difference that now our data is images and with additional loss that is calculated as some distance between generated and original pictures. The article presents the essence of the architecture of the PatchGAN generator, where the first part of the network takes the original image and encodes it, reducing the dimension, and the other takes this result and tries to reproduce the desired image. The goal of the work is to learn the artist's style, no additional processing was applied except normalization to [-1, 1] before feeding to the neural network (because as activation is used hyperbolic tangent) and resizing to (256, 256, 3). The same actions were applied to the second part of the dataset: photos of nature. Data networks with Ivan Marchuk's paintings and photos of nature were trained with the help of Google Colab. The Adam was the optimizer. The results were compared with the Style Transfer model.
KEYWORDS
Generative adversarial network, PatchGAN, CNN, CycleGAN, Artificial neural network, Convolution neural network Image generation, Style transfer
REFERENCES
[1] C. Li, M. Wand, “Precomputed real-time texture synthesis with markovian generative adversarial networks,” European conference on computer vision ECCV 2016, (2016) DOI: 10.1007 / 978-3-319-46487-9_43(CrossRef)(Google Scholar)
[2] L. A. Gatys, A. S. Ecker, and M. Bethg, “A Neural Algorithm of Artistic Style”, Journal of Vision Sciences Society Annual Meeting Abstract , vol.16, p.326, (2016) DOI: https://doi.org/10.1167/16.12.326(CrossRef)(Google Scholar)
[3] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image to-image translation with conditional adversarial networks” In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017) DOI: 10.1109/CVPR.2017.632(CrossRef)(Google Scholar)
[4] N. Boyko, A. Bronetskyi, N. Shakhovska, “Application of artificial intelligence algorithms for image processing”, Workshop Proceedings of the 8th International Conference on “Mathematics. Information Technologies. Education”, MoMLeT&DS-2019, vol.2386, pp.194-211, (2019)
[5] N. Boyko and B. Mandych, “technologies of object recognition in space for visually impaired people”, The 3rd International Conference on Informatics & Data-Driven Medicine (IDDM 2020), Växjö, Sweden, November 19-21, CEUR, pp.338-347, (2020)
[6] N. Boyko and N. Tkachuk, “processing of medical different types of data using Hadoop and java MapReduce”, The 3rd International Conference on Informatics & Data-Driven Medicine (IDDM 2020), pp.405-414, (2020)
[7] S. Ioffe and Ch. Szegedy, “Batch normalization: accelerating deep network training by reducing internal covariate shift”, Proceedings of the 32 nd International Conference on Machine Learning, Lille, France, 2015. JMLR: W&CP, vol.37, pp.448-456, (2015) URL: arXiv: 1502.03167
[8] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.770-778, (2016) URL: arXiv: 1512.03385
[9] J. Johnson, A. Alahi, and F.F. Li, “Perceptual losses for real-time style transfer and super resolution”, European Conference on Computer Vision ECCV 2016, pp.694-711, (2016) DOI: 10.1007/978-3-319-46475-6_43(CrossRef)(Google Scholar)
[10] D. P. Kingma and J. Ba, “Adam. A method for stochastic optimization”, 3rd International Conference for Learning Representations, San Diego, (2015) URL: https://arxiv.org/abs/1412.6980
[11] Open wikiart Vasil Kandinskiy URL: https://www.wikiart.org/uk/vasil-kandinskiy
[12] Open wikiart Anatoly Kkryvolap URL: https://anatolykryvolap.com
[13] Open wikiart Vasil Lopata URL: https://sverediuk.com.ua/vasil-lopata-hudozhnik-grafik/
[14] Open wikiart Kazimir Malevich URL: https://www.wikiart.org/uk/kazimir-malevich
[15] Open wikiart Mikola PimonenkoURL: https://www.wikiart.org/uk/mikola-pimonenko
[16] Open Landscape pictures dataset: URL:https://www.kaggle.com/arnaud58/landscape-pictures/data#
[17] Dataset Stock dynamics https://www.kaggle.com/econdata/stock-dynamics
[18] I. J.Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets”, NeurIPS Proceedings Advances in Neural Information Processing Systems 27, (2014) URL: arXiv: 1406.2661
[19] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image to-image translation with conditional adversarial networks” In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017) DOI: 10.1109/CVPR.2017.632(CrossRef)(Google Scholar)