Research ArticleNo Access

3D-Convolutional Neural Network with Generative Adversarial Network and Autoencoder for Robust Anomaly Detection in Video Surveillance

Wonsup Shin

Department of Computer Science, Yonsei University, 50 Yonsei-ro, Sudaemoon-gu, Seoul 03722, South Korea

E-mail Address: [email protected]

Search for more papers by this author

Seok-Jun Bu

Department of Computer Science, Yonsei University, 50 Yonsei-ro, Sudaemoon-gu, Seoul 03722, South Korea

E-mail Address: sjbuhan @yonsei.ac.kr

Search for more papers by this author

and

Sung-Bae Cho

Corresponding author.

Department of Computer Science, Yonsei University, 50 Yonsei-ro, Sudaemoon-gu, Seoul 03722, South Korea

E-mail Address: [email protected]

Search for more papers by this author

https://doi.org/10.1142/S0129065720500343Cited by:11

Abstract

As the surveillance devices proliferate, various machine learning approaches for video anomaly detection have been attempted. We propose a hybrid deep learning model composed of a video feature extractor trained by generative adversarial network with deficient anomaly data and an anomaly detector boosted by transferring the extractor. Experiments with UCSD pedestrian dataset show that it achieves 94.4% recall and 86.4% precision, which is the competitive performance in video anomaly detection.

Keywords:

References

1. S. Fleck and W. Straßer, Privacy sensitive surveillance for assisted living — A smart camera approach, in Handbook of Ambient Intelligence and Smart Environments (Springer, 2010), pp. 985–1014. Crossref, Google Scholar
2. Y. Cong, J. Yuan and J. Liu, Sparse reconstruction cost for abnormal event detection, IEEE Conf. on Computer Vision and Pattern Recognition (2011), pp. 3449–3456. Crossref, Google Scholar
3. D. Xu, Y. Yan, E. Ricci and N. Sebe, Detecting anomalous events in videos by learning deep representations of appearance and motion, Computer Vision and Image Understanding 156 (2017) 117–127. Crossref, ISI, Google Scholar
4. A. Tomé and L. Salgado, Anomaly detection in crowded scenarios using local and global Gaussian mixture models, Int. Conf. on Advanced Concepts for Intelligent Vision Systems (2017), pp. 363–374. Crossref, Google Scholar
5. B. R. Kiran, D. M. Thomas and R. Parakkal, An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos, arXiv preprint, arXiv:1801.03149 (2018). Google Scholar
6. R. Chalapathy and S. Chawla, Deep learning for anomaly detection: A survey, arXiv preprint, arXiv:1901.03407 (2019). Google Scholar
7. J. Y. Kim and S. B. Cho, Detecting intrusive malware with a hybrid generative deep learning model, Int. Conf. on Intelligent Data Engineering and Automated Learning (2018), pp. 499–507. Crossref, Google Scholar
8. M. Ahmed and A. N. Mahmood, Network traffic pattern analysis using improved information theoretic co-clustering based collective anomaly detection, Int. Conf. on Security and Privacy in Communication Systems (2014), pp. 204–219. Google Scholar
9. B. Barz, E. Rodner, Y. G. Garcia and J. Denzler, Detecting regions of maximal divergence for spatio-temporal anomaly detection, IEEE Trans. on Pattern Analysis and Machine Intelligence 41 (2018) 1088–1101. Crossref, Medline, ISI, Google Scholar
10. J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko and T. Darrell, Long-term recurrent convolutional networks for visual recognition and description, IEEE Conf. on Computer Vision and Pattern Recognition (2015), pp. 2625–2634. Crossref, Google Scholar
11. N. Ballas, L. Yao, C. Pal and A. Courville, Delving deeper into convolutional networks for learning video representations, arXiv preprint, arXiv:1511.06432 (2015). Google Scholar
12. M. Hasan, J. Choi, J. Neumann, A. K. Roy-Chowdhury and L. S. Davis, Learning temporal regularity in video sequences, IEEE Conf. on Computer Vision and Pattern Recognition (2016), pp. 733–742. Crossref, Google Scholar
13. Y. S. Chong and Y. H. Tay, Abnormal event detection in videos using spatiotemporal autoencoder, Int. Symposium on Neural Networks (2017), pp. 189–196. Crossref, Google Scholar
14. N. Srivastava, E. Mansimov and R. Salakhudinov, Unsupervised learning of video representations using lstms, Int. Conf. on Machine Learning (2015), pp. 843–852. Google Scholar
15. Z. Chen, Y. Tian, W. Zeng and T. Huang, Detecting abnormal behaviors in surveillance videos based on fuzzy clustering and multiple Autoencoders, IEEE Int. Conf. on Multimedia and Expo (2015), pp. 1–6. Google Scholar
16. D. Tran, L. Bourdev, R. Fergus, L. Torresani and M. Paluri, Learning spatiotemporal features with 3d convolutional networks, IEEE Int. Conf. on Computer Vision (2015), pp. 4489–4497. Crossref, Google Scholar
17. S. Smeureanu, R. T. Ionescu, M. Popescu and B. Alexe, Deep appearance features for abnormal behavior detection in video, Int. Conf. on Image Analysis and Processing (2017), pp. 779–789. Crossref, Google Scholar
18. R. Hinami, T. Mei and S. I. Satoh, Joint detection and recounting of abnormal events by learning deep generic knowledge, IEEE Int. Conf. on Computer Vision (2017), pp. 3619–3627. Crossref, Google Scholar
19. J. R. Medel and A. Savakis, Anomaly detection in video using predictive convolutional long short-term memory networks, arXiv preprint, arXiv:1612.00390 (2016). Google Scholar
20. M. Sabokrou, M. Fayyaz, M. Fathy, Z. Moayed and R. Klette, Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes, Computer Vision and Image Understanding 172 (2018) 88–97. Crossref, ISI, Google Scholar
21. W. Chu, H. Xue, C. Yao and D. Cai, Sparse coding guided spatiotemporal feature learning for abnormal event detection in large videos, IEEE Trans. on Multimedia 21(1) (2018) 246–255. Crossref, ISI, Google Scholar
22. J. Sun, J. Shao and C. He, Abnormal event detection for video surveillance using deep one-class learning, Multimedia Tools and Applications 78(3) (2019) 3633–3647. Crossref, ISI, Google Scholar
23. Z. Chen, Y. Tian, W. Zeng and T. Huang, Detecting abnormal behaviors in surveillance videos based on fuzzy clustering and multiple auto-encoders, IEEE Int. Conf. on Multimedia and Expo (2015), pp. 1–6. Google Scholar
24. Y. Zhao, B. Deng, C. Shen, Y. Liu, H. Lu and X. S. Hua, Spatio-temporal autoencoder for video anomaly detection, ACM International Conference on Multimedia (2017), pp. 1933–1941. Crossref, Google Scholar
25. J. Yu, K. C. Yow and M. Jeon, Joint representation learning of appearance and motion for abnormal event detection, Machine Vision and Applications 29(7) (2018) 1157–1170. Crossref, ISI, Google Scholar
26. A. Dimokranitou, Adversarial autoencoders for anomalous event detection in images, Dissertation (2017). Google Scholar
27. M. Ravanbakhsh, E. Sangineto, M. Nabi and N. Sebe, Training adversarial discriminators for cross-channel abnormal event detection in crowds, IEEE Conf. on Applications of Computer Vision (2019), pp. 1896–1904. Crossref, Google Scholar
28. J. Sun, X. Wang, N. Xiong and J. Shao, Learning sparse representation with variational auto-encoder for anomaly detection, IEEE Access 6 (2018) 33353–33361. Crossref, ISI, Google Scholar
29. S. Huang, D. Huang and X. Zhou, Learning multimodal deep representations for crowd anomaly event detection, Mathematical Problems in Engineering 2018 (2018) 1–13. Crossref, ISI, Google Scholar
30. M. Sabokrou, M. Pourreza, M. Fayyaz, R. Entezari, M. Fathy, J. Gall and E. Adeli, AVID: Adversarial visual irregularity detection, arXiv preprint arXiv:1805.09521 (2018). Google Scholar
31. S. Lee, H. G. Kim and Y. M. Ro, Stan: Spatio-temporal adversarial networks for abnormal event detection, Int. Conf. on Acoustics, Speech and Signal Processing (2018), pp. 1323–1327. Crossref, Google Scholar
32. O. Ronneberger, P. Fischer and T. Brox, U-net: Convolutional networks for biomedical image segmentation, Int. Conf. on Medical Image Computing and Computer-Assisted Intervention (2015), pp. 234–241. Crossref, Google Scholar
33. W. Shin and S. B. Cho, CCTV image sequence generation and modeling method for video anomaly detection using generative adversarial network, Int. Conf. on Intelligent Data Engineering and Automated Learning (2018), pp. 457–467. Crossref, Google Scholar
34. J. Y. Kim, S. J. Bu and S. B. Cho, Zero-day malware detection using transferred generative adversarial networks based on deep autoencoders, Information Sciences 460 (2018) 83–102. Crossref, ISI, Google Scholar
35. V. Mahadevan, W.-X. Li, V. Bhalodia and N. Vasconcelos, Anomaly detection in crowded scenes, IEEE Conf. on Computer Vision and Pattern Recognition (2010), pp. 1975–1981. Crossref, Google Scholar
36. B. T. Morris and M. Trivedi, Understanding vehicular traffic behavior from video: A survey of unsupervised approaches, Journal of Electronic Imaging 22(4) (2013) 041113. Crossref, ISI, Google Scholar
37. F. Chen, Z. Liu and M. T. Sun, Anomaly detection by using random projection forest, IEEE Int. Conf. on Image Processing (2015), pp. 1210–1214. Crossref, Google Scholar
38. F. U. M. Ullah, A. Ullah, K. Muhammad, I. U. Haq and S. W. Baik, Violence detection using spatiotemporal features with 3d convolutional neural network, Sensors 19(11) (2019) 2472. Crossref, ISI, Google Scholar
39. T. Gandhi and M. M. Trivedi, Image based estimation of pedestrian orientation for improving path prediction, IEEE Intelligent Vehicles Symposium (2008), pp. 506–511. Crossref, Google Scholar
40. S. Zhou, W. Shen, D. Zeng, M. Fang, Y. Wei and Z. Zhang, Spatial–temporal convolutional neural networks for anomaly detection and localization in crowded scenes, Signal Processing; Image Communcation 47 (2016) 358–368. Crossref, ISI, Google Scholar
41. W. Sultani, C. Chen and M. Shah, Real-world anomaly detection in surveillance videos, in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (2018), pp. 6479–6488. Crossref, Google Scholar
42. A. Adam, E. Rivlin, I. Shimshoni and D. Reinitz, Robust real-time unusual event detection using multiple fixed-location monitors, IEEE Trans. Pattern Anal. Mach. Intell. 30(3) (2008) 555–560. Crossref, Medline, ISI, Google Scholar
43. S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural Computation 9(8) (1997) 1735–1780. Crossref, Medline, ISI, Google Scholar
44. P. Wang and X. Bai, Regional parallel structure based CNN for thermal infrared face identification, Integrated Computer-Aided Engineering 25(3) (2018) 247–260. Crossref, ISI, Google Scholar
45. M. Ahmadlou and H. Adeli, Enhanced probabilistic neural network with local decision circles: A robust classifier, Integrated Computer-Aided Engineering 17(3) (2010) 197–210. Crossref, ISI, Google Scholar
46. M. H. Rafiei, W. H. Khushefati, R. Demirboga and H. Adeli, Supervised deep restricted Boltzmann machine for estimation of concrete, ACI Materials Journals 112(2) (2017) 237–244. Google Scholar
47. M. A. Molina-Cabello, R. M. Luque-Baena, E. López-Rubio and K. Thurnhofer-Hemsi, Vehicle type detection by ensembles of convolutional neural networks operating on super resolved images, Integrated Computer-Aided Engineering 25(4) (2018) 321–333. Crossref, ISI, Google Scholar

Published: 28 May 2020

Remember to check out the Most Cited Articles!
Check out our titles in neural networks today!

Vol. 30, No. 06

Metrics

Downloaded 68 times

History

Accepted 28 March 2020

Keywords

PDF download

System Upgrade on Mon, Jun 21st, 2021 at 1am (EDT)

3D-Convolutional Neural Network with Generative Adversarial Network and Autoencoder for Robust Anomaly Detection in Video Surveillance

Abstract

References

Recommended