Research ArticleNo Access

A Knowledge-Based Deep Learning Architecture for Aspect-Based Sentiment Analysis

Georgios Alexandridis

Cultural Technology Department, University of the Aegean, University Hill, Mytilene 81100, Greece

E-mail Address: [email protected]

Corresponding author.

Search for more papers by this author

John Aliprantis

Cultural Technology Department, University of the Aegean, University Hill, Mytilene 81100, Greece

E-mail Address: [email protected]

Search for more papers by this author

Konstantinos Michalakis

Cultural Technology Department, University of the Aegean, University Hill, Mytilene 81100, Greece

E-mail Address: [email protected]

Search for more papers by this author

Konstantinos Korovesis

Palo Services, 9, Chavriou Street, Athens 10562, Greece

E-mail Address: [email protected]

Search for more papers by this author

Panagiotis Tsantilas

Palo Services, 9, Chavriou Street, Athens 10562, Greece

E-mail Address: [email protected]

Search for more papers by this author

, and

George Caridakis

Cultural Technology Department, University of the Aegean, University Hill, Mytilene 81100, Greece

E-mail Address: [email protected]

Search for more papers by this author

https://doi.org/10.1142/S0129065721500465Cited by:18 (Source: Crossref)

Abstract

The task of sentiment analysis tries to predict the affective state of a document by examining its content and metadata through the application of machine learning techniques. Recent advances in the field consider sentiment to be a multi-dimensional quantity that pertains to different interpretations (or aspects), rather than a single one. Based on earlier research, the current work examines the said task in the framework of a larger architecture that crawls documents from various online sources. Subsequently, the collected data are pre-processed, in order to extract useful features that assist the machine learning algorithms in the sentiment analysis task. More specifically, the words that comprise each text are mapped to a neural embedding space and are provided to a hybrid, bi-directional long short-term memory network, coupled with convolutional layers and an attention mechanism that outputs the final textual features. Additionally, a number of document metadata are extracted, including the number of a document’s repetitions in the collected corpus (i.e. number of reposts/retweets), the frequency and type of emoji ideograms and the presence of keywords, either extracted automatically or assigned manually, in the form of hashtags. The novelty of the proposed approach lies in the semantic annotation of the retrieved keywords, since an ontology-based knowledge management system is queried, with the purpose of retrieving the classes the aforementioned keywords belong to. Finally, all features are provided to a fully connected, multi-layered, feed-forward artificial neural network that performs the analysis task. The overall architecture is compared, on a manually collected corpus of documents, with two other state-of-the-art approaches, achieving optimal results in identifying negative sentiment, which is of particular interest to certain parties (like for example, companies) that are interested in measuring their online reputation.

Keywords:

References

1. K. Dave, S. Lawrence and D. M. Pennock , Mining the peanut gallery: Opinion extraction and semantic classification of product reviews, in Proc. 12th Int. Conf. World Wide Web, WWW’03 (ACM, New York, NY, USA, 2003), pp. 519–528. Crossref, Google Scholar
2. B. Liu , Sentiment Analysis and Opinion Mining (Morgan & Claypool Publishers, 2012). Crossref, Google Scholar
3. M. Hu and B. Liu , Mining and summarizing customer reviews, in Proc. Tenth ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, KDD’04 (Association for Computing Machinery, New York, NY, USA, 2004), pp. 168–177. Crossref, Google Scholar
4. G. Alexandridis, K. Michalakis, J. Aliprantis, P. Polydoras, P. Tsantilas and G. Caridakis , A deep learning approach to aspect-based sentiment prediction, in Artificial Intelligence Applications and Innovations, eds. I. Maglogiannis, L. Iliadis and E. Pimenidis (Springer International Publishing, Cham, 2020), pp. 397–408. Crossref, Google Scholar
5. Y. LeCun, Y. Bengio and G. Hinton , Deep learning, Nature 521(7553) (2015) 436. Crossref, Medline, Web of Science, Google Scholar
6. A. Graves, S. Fernández and J. Schmidhuber , Bidirectional LSTM networks for improved phoneme classification and recognition, in Artificial Neural Networks: Formal Models and Their Applications — ICANN 2005, eds. W. Duch, J. Kacprzyk, E. Oja and S. Zadrożny (Springer Berlin Heidelberg, Berlin, Heidelberg, 2005), pp. 799–804. Crossref, Google Scholar
7. C. Raffel and D. P. W. Ellis, Feed-forward networks with attention can solve some long-term memory problems, CoRR abs/1512.08756 (2015). Google Scholar
8. K. Korovesis, G. Alexandridis, G. Caridakis, P. Polydoras and P. Tsantilas , Leveraging aspect-based sentiment prediction with textual features and document metadata, in 11th Hellenic Conf. Artificial Intelligence, SETN 2020 (Association for Computing Machinery, New York, NY, USA, 2020), pp. 168–174. Crossref, Google Scholar
9. Z. Nasim and S. Haider , Absa toolkit: An open source tool for aspect based sentiment analysis, Int. J. Artif. Intell. Tools 26(6) (2017) 1750023. Link, Web of Science, Google Scholar
10. L. Liu, Y. Yang and J. Hu , Simple but effective: A model for aspect-based sentiment analysis, in Developments of Artificial Intelligence Technologies in Computation and Robotics (World Scientific, 2020), pp. 52–59. Link, Google Scholar
11. K. Ravi and V. Ravi , A survey on opinion mining and sentiment analysis: Tasks, approaches and applications, Knowl.-Based Syst. 89 (2015) 14–46. Crossref, Web of Science, Google Scholar
12. D. Zhang, F. He, Z. Tu, L. Zou and Y. Chen , Pointwise geometric and semantic learning network on 3D point clouds, Integr. Comput. Aided Eng. 27 (2020) 57–75. Crossref, Web of Science, Google Scholar
13. F. Martín, F. Gomez-Donoso, F. Escalona, J. G. Rodríguez and M. Cazorla , Semantic visual recognition in a cognitive architecture for social robots, Integr. Comput. Aided Eng. 27 (2020) 301–316. Crossref, Web of Science, Google Scholar
14. S. Rostami, F. Neri and M. G. Epitropakis , Progressive preference articulation for decision making in multi-objective optimisation problems, Integr. Comput. Aided Eng. 24 (2017) 315–335. Crossref, Web of Science, Google Scholar
15. S. Ruder, P. Ghaffari and J. G. Breslin , A hierarchical model of reviews for aspect-based sentiment analysis, in Proc. 2016 Conf. Empirical Methods in Natural Language Processing (Association for Computational Linguistics, Austin, Texas, 2016), pp. 999–1005. Crossref, Google Scholar
16. C. Sun, L. Huang and X. Qiu , Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence, in Proc. 2019 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (Association for Computational Linguistics, Minneapolis, Minnesota, 2019), pp. 380–385. Google Scholar
17. H. Zhang and J. Wang , An unsupervised semantic sentence ranking scheme for text documents, Integr. Comput.-Aided Eng. 28(1) (2021) 17–33. Crossref, Web of Science, Google Scholar
18. J. Devlin, M.-W. Chang, K. Lee and K. Toutanova , BERT: Pre-training of deep bidirectional transformers for language understanding, in Proc. 2019 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (Association for Computational Linguistics, Minneapolis, Minnesota, 2019), pp. 4171–4186. Google Scholar
19. A. Karimi, L. Rossi and A. Prati , Adversarial training for aspect-based sentiment analysis with BERT, in 2020 25th Int. Conf. Pattern Recognition (ICPR) (Milan, Italy, 2021), pp. 8797–8803. Crossref, Google Scholar
20. D. Meškelė and F. Frasincar , ALDONAr: A hybrid solution for sentence-level aspect-based sentiment analysis using a lexicalized domain ontology and a regularized neural attention model, Inform. Process. Manage. 57(3) (2020) 102211. Crossref, Web of Science, Google Scholar
21. X. Li, L. Bing, W. Zhang and W. Lam , Exploiting BERT for end-to-end aspect-based sentiment analysis, in Proc. 5th Workshop on Noisy User-Generated Text (W-NUT 2019) (Association for Computational Linguistics, Hong Kong, China, 2019), pp. 34–41. Crossref, Google Scholar
22. O. Wallaart and F. Frasincar , A hybrid approach for aspect-based sentiment analysis using a lexicalized domain ontology and attentional neural models, The Semantic Web, eds. P. Hitzler, M. Fernández, K. Janowicz, A. Zaveri, A. J. Gray, V. Lopez, A. Haller and K. Hammar (Springer International Publishing, Cham, 2019), pp. 363–378. Crossref, Google Scholar
23. Y. Ma, H. Peng, T. Khan, E. Cambria and A. Hussain , Sentic LSTM: A hybrid network for targeted aspect-based sentiment analysis, Cognit. Comput. 10 (2018) 639–650. Crossref, Web of Science, Google Scholar
24. N. Majumder, S. Poria, A. Gelbukh, M. S. Akhtar, E. Cambria and A. Ekbal , IARM: Inter-aspect relation modeling with memory networks in aspect-based sentiment analysis, in Proc. 2018 Conf. Empirical Methods in Natural Language Processing (Association for Computational Linguistics, Brussels, Belgium, 2018), pp. 3402–3411. Crossref, Google Scholar
25. D. Zeng, Y. Dai, J. Wang and A. K. Sangaiah , Aspect based sentiment analysis by a linguistically regularized CNN with gated mechanism, J. Intell. Fuzzy Syst. 36 (2019) 3971–3980. Crossref, Web of Science, Google Scholar
26. H. Xu, B. Liu, L. Shu and P. S. Yu , Double embeddings and CNN-based sequence labeling for aspect extraction, Proc. 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (Association for Computational Linguistics, Melbourne, Australia, 2018), pp. 592–598. Crossref, Google Scholar
27. S. S. Hossain, Y. Arafat and M. E. Hossain , Context-based news headlines analysis: A comparative study of machine learning and deep learning algorithms, Vietnam J. Comput. Sci. 9(1) (2022) 1–15. Google Scholar
28. M. Pontiki, D. Galanis, J. Pavlopoulos, H. Papageorgiou, I. Androutsopoulos and S. Manandhar , SemEval-2014 task 4: Aspect based sentiment analysis, in Proc. 8th Int. Workshop on Semantic Evaluation (SemEval 2014) (Association for Computational Linguistics, Dublin, Ireland, 2014), pp. 27–35. Crossref, Google Scholar
29. H. H. Do, P. Prasad, A. Maag and A. Alsadoon , Deep learning for aspect-based sentiment analysis: A comparative review, Expert Syst. Appl. 118 (2019) 272–299. Crossref, Web of Science, Google Scholar
30. D.-H. Pham and A.-C. Le , Exploiting multiple word embeddings and one-hot character vectors for aspect-based sentiment analysis, Int. J. Approx. Reason. 103 (2018) 1–10. Crossref, Web of Science, Google Scholar
31. H. Xu, B. Liu, L. Shu and P. Yu , BERT post-training for review reading comprehension and aspect-based sentiment analysis, Proc. 2019 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (Association for Computational Linguistics, Minneapolis, Minnesota, 2019), pp. 2324–2335. Google Scholar
32. H. Zhang, J. Dong, L. Min and P. Bi , A bert fine-tuning model for targeted sentiment analysis of Chinese online course reviews, Int. J. Artif. Intell. Tools 29(7n8) (2020) 2040018. Link, Web of Science, Google Scholar
33. J. A. García-Díaz, M. Cánovas-García and R. Valencia-García , Ontology-driven aspect-based sentiment analysis classification: An infodemiological case study regarding infectious diseases in latin America, Future Gen. Comput. Syst. 112 (2020) 641–657. Crossref, Medline, Web of Science, Google Scholar
34. S. de Kok and F. Frasincar , Using word embeddings for ontology-driven aspect-based sentiment analysis, in Proc. 35th Annual ACM Symp. Applied Computing, SAC’20 (Association for Computing Machinery, New York, NY, USA, 2020), pp. 834–842. Crossref, Google Scholar
35. K. Schouten, F. Frasincar and F. de Jong , Ontology-enhanced aspect-based sentiment analysis, in Web Engineering, eds. J. Cabot, R. De Virgilio and R. Torlone (Springer International Publishing, Cham, 2017), pp. 302–320. Crossref, Google Scholar
36. L. Zhuang, K. Schouten and F. Frasincar , Soba: Semi-automated ontology builder for aspect-based sentiment analysis, J. Web Semant. 60 (2020) 100544. Crossref, Web of Science, Google Scholar
37. E. Parliament and T. C. of the European Union, Regulation (EU) 2016/679 of the European Parliament and of the council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46/EC (general data protection regulation) (2016). Google Scholar
38. C. Research, Putting the data lake to work: A guide to best practices, Tech. Teport, Teradata (2014). Google Scholar
39. C.-E. Särndal, B. Swensson and J. Wretman , Model Assisted Survey Sampling (Springer Science & Business Media, 2003). Google Scholar
40. P. Bojanowski, E. Grave, A. Joulin and T. Mikolov, Enriching word vectors with subword information, preprint (2016), arXiv:1607.04606. Google Scholar
41. P. K. Novak, J. Smailović, B. Sluban and I. Mozetič , Sentiment of emojis, PLoS One 10(12) (2015) e0144296. Medline, Web of Science, Google Scholar
42. M. Newman , Power laws, Pareto distributions and Zipf’s law, Contemp. Phys. 46(5) (2005) 323–351. Crossref, Web of Science, Google Scholar
43. X. Wang, F. Wei, X. Liu, M. Zhou and M. Zhang , Topic sentiment analysis in Twitter: A graph-based hashtag sentiment classification approach, in Proc. 20th ACM Int. Conf. Information and Knowledge Management, CIKM’11 (Association for Computing Machinery, New York, NY, USA, 2011), pp. 1031–1040. Crossref, Google Scholar
44. C. Bizer, J. Lehmann, G. Kobilarov, S. Auer, C. Becker, R. Cyganiak and S. Hellmann , DBpedia — A crystallization point for the web of data, Web Semant. 7 (2009) 154–165. Crossref, Web of Science, Google Scholar
45. J. Hebeler, M. Fisher, R. Blace and A. Perez-Lopez , Semantic Web Programming (John Wiley & Sons, April 2009). Google Scholar
46. J. Androutsopoulos , Chapter 14. “Greeklish”: Transliteration practice and discourse in the context of computer-mediated digraphia”. Orthography as Social Action, A. Jaffe, J. Androutsopoulos, M. Sebba and S. Johnson (eds.), Berlin, Boston: De Gruyter Mouton, 2012, pp. 359–392, https://doi.org/10.1515/9781614511038.359. Crossref, Google Scholar
47. A. Chalamandaris et al., “ Bypassing Greeklish!.” In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04), European Language Resources Association (ELRA), May 2004, Lisbon, Portugal. Google Scholar
48. P. Zhou, Z. Qi, S. Zheng, J. Xu, H. Bao and B. Xu , Text classification improved by integrating bidirectional LSTM with two-dimensional max pooling, in COLING 2016, 26th Int. Conf. Computational Linguistics, Proc. Conf.: Technical Papers, Osaka, Japan, 2016, pp. 3485–3495. Google Scholar
49. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever and R. Salakhutdinov , Dropout: A simple way to prevent neural networks from overfitting, J. Mach. Learn. Res. 15 (2014) 1929–1958. Web of Science, Google Scholar
50. R. H. Hahnloser, R. Sarpeshkar, M. A. Mahowald, R. J. Douglas and H. S. Seung , Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit, Nature 405(6789) (2000) 947. Crossref, Medline, Web of Science, Google Scholar
51. I. Goodfellow, Y. Bengio and A. Courville , Deep Learning (MIT Press, 2016). http://www.deeplearningbook.org. Google Scholar
52. D. P. Kingma and J. Ba , Adam: A method for stochastic optimization, in 3rd Int. Conf. Learning Representations, ICLR 2015, San Diego, CA, USA, 2015. Google Scholar
53. E. Grave, P. Bojanowski, P. Gupta, A. Joulin and T. Mikolov , Learning word vectors for 157 languages, in Proc. Int. Conf. Language Resources and Evaluation (LREC 2018) (Miyazaki, Japan, 2018), pp. 3483–3487. Google Scholar