Discriminating Multiple Emotional States from EEG Using a Data-Adaptive, Multiscale Information-Theoretic Approach
Abstract
A multivariate sample entropy metric of signal complexity is applied to EEG data recorded when subjects were viewing four prior-labeled emotion-inducing video clips from a publically available, validated database. Besides emotion category labels, the video clips also came with arousal scores. Our subjects were also asked to provide their own emotion labels. In total 30 subjects with age range 19–70 years participated in our study. Rather than relying on predefined frequency bands, we estimate multivariate sample entropy over multiple data-driven scales using the multivariate empirical mode decomposition (MEMD) technique and show that in this way we can discriminate between five self-reported emotions (). These results could not be obtained by analyzing the relation between arousal scores and video clips, signal complexity and arousal scores, and self-reported emotions and traditional power spectral densities and their hemispheric asymmetries in the theta, alpha, beta, and gamma frequency bands. This shows that multivariate, multiscale sample entropy is a promising technique to discriminate multiple emotional states from EEG recordings.
References
- 1. , Neurometrics: Computer-assisted differential diagnosis of brain dysfunctions, Science 239(4836) (1988) 162–169, doi: 10.1126/science.3336779. Crossref, Medline, Web of Science, Google Scholar
- 2. A. Choppin, EEG-based human interface for disabled individuals: Emotion expression with neural networks, Unpubl. Master’s Thesis (2000). Google Scholar
- 3. , Combining brain–computer interfaces and assistive technologies: State-of-the-art and challenges, Front. Neurosci. 4 (2010) 161. Medline, Web of Science, Google Scholar
- 4. , BCI for games: A “state of the art” survey, Entertainment Computing-ICEC 2008 (Springer, 2009), pp. 225–228. Google Scholar
- 5. , Neocortical Dynamics and Human EEG Rhythms (Oxford University Press, USA, 1995). Google Scholar
- 6. , Anterior cerebral asymmetry and the nature of emotion, Brain Cogn. 20(1) (1992) 125–151, doi: 10.1016/0278-2626(92)90065-T. Crossref, Medline, Web of Science, Google Scholar
- 7. , Frequency characteristics of EEG spectra in the emotions, Neurosci. Behav. Physiol. 26(4) (1996) 340–343, doi: 10.1007/bf02359037. Crossref, Medline, Google Scholar
- 8. , Influence of the emotional perception of a signal on the electroencephalographic correlates of the creative activity, Fiziol. Cheloveka 30(2) (2004) 22–29. Medline, Google Scholar
- 9. ,
The International Affective Picture System (IAPS) in the study of emotion and attention , Handb. Emot. Elicitation Assess. (Cambridge University Press, 2007), pp. 29–46. Crossref, Google Scholar - 10. , Functional neuroanatomy of emotions: A meta-analysis, Cogn. Affect. Behav. Neurosci. 3(3) (2003) 207–233, doi: 10.3758/CABN.3.3.207. Crossref, Medline, Web of Science, Google Scholar
- 11. , Determination of hemispheric emotional valence in individual subjects: A new approach with research and therapeutic implications, Behav. Brain Funct. 3(1) (2007) 13. Crossref, Medline, Web of Science, Google Scholar
- 12. , Differences in event-related and induced EEG patterns in the theta and alpha frequency bands related to human emotional intelligence, Neurosci. Lett. 311(2) (2001) 93–96. Crossref, Medline, Web of Science, Google Scholar
- 13. , Relative electroencephalographic desynchronization and synchronization in humans to emotional film content: An analysis of the 4–6, 6–8, 8–10 and 10–12Hz frequency bands, Neurosci. Lett. 286(1) (2000) 9–12. Crossref, Medline, Web of Science, Google Scholar
- 14. , Changes in brain activity during the observation of TV commercials by using EEG, GSR and HR measurements, Brain Topogr. 23(2) (2010) 165–179, doi: 10.1007/s10548-009-0127-0. Crossref, Medline, Web of Science, Google Scholar
- 15. , Human brain EEG indices of emotions: Delineating responses to affective vocalizations by measuring frontal theta event-related synchronization, Neurosci. Biobehav. Rev. 35(9) (2011) 1959–1970. Crossref, Medline, Web of Science, Google Scholar
- 16. , Affective picture processing: Event-related synchronization within individually defined human theta band is modulated by valence dimension, Neurosci. Lett. 303(2) (2001) 115–118. Crossref, Medline, Web of Science, Google Scholar
- 17. , Arousal effect on emotional face comprehension: Frequency band changes in different time intervals, Physiol. Behav. 97(3) (2009) 455–462. Crossref, Medline, Web of Science, Google Scholar
- 18. , Universality in the brain while listening to music, Proc. R. Soc. London B Biol. Sci. 268(1484) (2001) 2423–2433. Crossref, Web of Science, Google Scholar
- 19. , Increase of universality in human brain during mental imagery from visual perception, PLoS One 4(1) (2009) e4121, doi: 10.1371/journal.pone.0004121. Crossref, Medline, Web of Science, Google Scholar
- 20. , An Introduction to the Event-Related Potential Technique (MIT Press, 2014). Google Scholar
- 21. , An ERP investigation on the temporal dynamics of emotional prosody and emotional semantics in pseudo-and lexical-sentence context, Brain Lang. 105(1) (2008) 59–69. Crossref, Medline, Web of Science, Google Scholar
- 22. , Neural processing of vocal emotion and identity, Brain Cogn. 69(1) (2009) 121–126. Crossref, Medline, Web of Science, Google Scholar
- 23. , What is physiologic complexity and how does it change with aging and disease? Neurobiol. Aging 23(1) (2002) 23–26, doi: 10.1016/S0197-4580(01)00266-4. Crossref, Medline, Web of Science, Google Scholar
- 24. , Early seizure detection using neuronal potential similarity: A generalized low-complexity and robust measure, Int. J. Neural Syst. 25(5) (2015) 1550019. Link, Web of Science, Google Scholar
- 25. , Multifractal analysis and relevance vector machine based automatic seizure detection in intracranial EEG, Int. J. Neural Syst. 25(6) (2015) 1550020. Link, Web of Science, Google Scholar
- 26. , Classification of obsessive compulsive disorder by EEG complexity and hemispheric dependency measurements, Int. J. Neural Syst. 25(3) (2015) 1550010. Link, Web of Science, Google Scholar
- 27. , Fractality and a wavelet-chaos-neural network methodology for EEG-based diagnosis of autistic spectrum disorder, J. Clin. Neurophysiol. 27(5) (2010) 328–333. Crossref, Medline, Web of Science, Google Scholar
- 28. , Fractality analysis of frontal brain in major depressive disorder, Int. J. Psychophysiol. 85(2) (2012) 206–211. Crossref, Medline, Web of Science, Google Scholar
- 29. , New diagnostic EEG markers of the Alzheimer’s disease using visibility graph, J. Neural Transm. 117(9) (2010) 1099–1109. Crossref, Medline, Web of Science, Google Scholar
- 30. , Visibility graph similarity: A new measure of generalized synchronization in coupled dynamic systems, Phys. D Nonlinear Phenom. 241(4) (2012) 326–332. Crossref, Web of Science, Google Scholar
- 31. , Complexity of functional connectivity networks in mild cognitive impairment subjects during a working memory task, Clin. Neurophysiol. 125(4) (2014) 694–702. Crossref, Medline, Web of Science, Google Scholar
- 32. , Fuzzy synchronization likelihood with application to attention-deficit/hyperactivity disorder, Clin. EEG Neurosci. 42(1) (2011) 6–13. Crossref, Medline, Web of Science, Google Scholar
- 33. , Fuzzy synchronization likelihood-wavelet methodology for diagnosis of autism spectrum disorder, J. Neurosci. Methods 211(2) (2012) 203–209. Crossref, Medline, Web of Science, Google Scholar
- 34. , Graph theoretical analysis of organization of functional brain networks in ADHD, Clin. EEG Neurosci. 43(1) (2012) 5–13. Crossref, Medline, Web of Science, Google Scholar
- 35. , Non-linear dynamic complexity of the human EEG during evoked emotions, Int. J. Psychophysiol. 28(1) (1998) 63–76, doi: 10.1016/S01678760(97)00067-6. Crossref, Medline, Web of Science, Google Scholar
- 36. , Emotion recognition method using entropy analysis of EEG signals, Int. J. Image, Graph. Signal Process. 3(5) (2011) 30, doi: 10.5815/ijigsp.2011.05.05. Crossref, Google Scholar
- 37. , Emotion recognition based on the sample entropy of EEG, Biomed. Mater. Eng. 24(1) (2013) 1185–1192, doi: 10.3233/BME-130919. Web of Science, Google Scholar
- 38. , Multiscale entropy analysis of complex physiologic time series, Phys. Rev. Lett. 89(6) (2002) 068102, doi: 10.1103/PhysRevLett.92.089803. Crossref, Medline, Web of Science, Google Scholar
- 39. , Physiological time-series analysis using approximate entropy and sample entropy, Am. J. Physiol. Heart Circ. Physiol. 278(6) (2000) H2039–49. Crossref, Medline, Web of Science, Google Scholar
- 40. , Multivariate empirical mode decomposition, Proc. R. Soc. A Math. Phys. Eng. Sci. 466(2117) (2009) 1291–1302, doi: 10.1098/rspa.2009.0502. Crossref, Web of Science, Google Scholar
- 41. , Application of intrinsic time-scale decomposition (ITD) to EEG signals for automated seizure prediction, Int. J. Neural Syst. 23(5) (2013) 1350023. Link, Web of Science, Google Scholar
- 42. , Application of empirical mode decomposition (EMD) for automated detection of epilepsy using EEG signals, Int. J. Neural Syst. 22(6) (2012) 1250027. Link, Web of Science, Google Scholar
- 43. , Application of entropy measures on intrinsic mode functions for the automated identification of focal electroencephalogram signals, Entropy 17(2) (2015) 669–691. Crossref, Web of Science, Google Scholar
- 44. , Multivariate multiscale entropy: A tool for complexity analysis of multichannel data, Phys. Rev. E 84(6) (2011) 061918, doi: 10.1103/PhysRevE.84.061918. Crossref, Web of Science, Google Scholar
- 45. , Predicting subject performance level from EEG signal complexity when engaged in BCI paradigm, 2014 IEEE Int. Work. Mach. Learn. Signal Process. (2014), pp. 1–5, doi: 10.1109/MLSP.2014.6958897. Google Scholar
- 46. , Assessing the effectiveness of a large database of emotion-eliciting films: A new tool for emotion researchers, Cogn. Emot. 24(7) (2010) 1153–1172, doi: 10.1080/02699930903274322. Crossref, Web of Science, Google Scholar
- 47. , What is physiologic complexity and how does it change with aging and disease? Neurobiol. Aging 23(1) (2002) 23–26. Crossref, Medline, Web of Science, Google Scholar
- 48. , EOG correction: A comparison of four methods, Psychophysiology 42(1) (2005) 16–24, doi: 10.1111/j.1468-8986.2005.00264.x. Crossref, Medline, Web of Science, Google Scholar
- 49. , High-quality recording of bioelectric events, Med. Biol. Eng. Comput. 29(4) (1991) 433–440. Crossref, Medline, Web of Science, Google Scholar
- 50. , The cognitive control of emotion, Trends Cogn. Sci. 9(5) (2005) 242–249. Crossref, Medline, Web of Science, Google Scholar
- 51. , Neuropsychological mechanisms of individual differences in emotion, personality, and arousal, Neuropsychology 7(4) (1993) 476. Crossref, Google Scholar
- 52. , Self-generated happy and sad emotions in low and highly hypnotizable persons during waking and hypnosis: Laterality and regional EEG activity differences, Int. J. Psychophysiol. 24(3) (1996) 239–266. Crossref, Medline, Web of Science, Google Scholar
- 53. , The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis, Proc. R. Soc. A Math. Phys. Eng. Sci. 454 (1971) (1998) 903–995, doi: 10.1098/rspa.1998.0193. Crossref, Web of Science, Google Scholar
- 54. , A differential entropy based method for determining the optimal embedding parameters of a signal, in Acoustics, Speech, and Signal Processing, 2003. Proc. (ICASSP’03). 2003 IEEE Int. Conf. Vol. 6 (IEEE, 2003), p. I-29. Google Scholar
- 55. , Random-effects models for longitudinal data, Biometrics 38 (1982) 963–974. Crossref, Medline, Web of Science, Google Scholar
- 56. , Linear Mixed Models for Longitudinal Data (Springer Science & Business Media, 2009). Google Scholar
- 57. , simsum: Analyses of simulation studies including Monte Carlo error, Stata J. 10(3) (2010) 369. Crossref, Web of Science, Google Scholar
- 58. , Flexible Imputation of Missing Data (CRC Press, 2012). Crossref, Google Scholar
- 59. , Approach-withdrawal and cerebral asymmetry: Emotional expression and brain physiology: I, J. Pers. Soc. Psychol. 58(2) (1990) 330. Crossref, Medline, Web of Science, Google Scholar
- 60. , Emotion classification based on gamma-band EEG, Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual Int. Conf. IEEE (IEEE, 2009), pp. 1223–1226. Google Scholar
- 61. , Consciousness and arousal effects on emotional face processing as revealed by brain oscillations. A gamma band analysis, Int. J. Psychophysiol. 67(1) (2008) 41–46. Crossref, Medline, Web of Science, Google Scholar
- 62. , Continuous emotion detection in response to music videos, Face Gesture 2011 (2011), pp. 803–808, doi: 10.1109/FG.2011.5771352. Google Scholar
- 63. M. Soleymani, M. Pantic and T. Pun, Multimodal emotion recognition in response to videos, Available at: http://ibug.doc.ic.ac.uk/media/uploads/documents/tac_multimodal_emotion_recognition.pdf. [Accessed on 18 March, 2015]. Google Scholar
- 64. , The influence of affective factors on time perception, Percept. Psychophys. 59(6) (1997) 972–982, doi: 10.3758/BF03205512. Crossref, Medline, Google Scholar
- 65. , Conceptualizing emotions along the dimensions of valence, arousal, and communicative frequency–implications for social-cognitive tests and training tools, Front. Psychol. 2 (2011), doi: 10.3389/fpsyg.2011.00266. Crossref, Medline, Web of Science, Google Scholar
- 66. , The James-Lange theory of emotions: A critical examination and an alternative theory, Am. J. Psychol. 39(1/4) (1927) 106–124. Crossref, Google Scholar
Remember to check out the Most Cited Articles! |
---|
Check out our titles in neural networks today! |