World Scientific
  • Search
  •   
Skip main navigation

Cookies Notification

We use cookies on this site to enhance your user experience. By continuing to browse the site, you consent to the use of our cookies. Learn More
×

System Upgrade on Tue, May 28th, 2024 at 2am (EDT)

Existing users will be able to log into the site and access content. However, E-commerce and registration of new users may not be available for up to 12 hours.
For online purchase, please visit us again. Contact us at [email protected] for any enquiries.

A Computational Model for the Neural Representation and Estimation of the Binocular Vector Disparity from Convergent Stereo Image Pairs

    https://doi.org/10.1142/S0129065718500296Cited by:6 (Source: Crossref)

    The depth cue is a fundamental piece of information for artificial and living beings who interact with the surrounding environment in order to handle objects and to avoid obstacles: in such situations, the disparity patterns, which arise when agents fixate objects, are vector fields. We propose a biologically-inspired computational model to estimate dense horizontal and vertical disparity maps by exploiting the cortical paradigms of the primate visual system: in particular, we aim to model the disparity sensitivity of the V1–MT visual pathway. The proposed model is based on a first processing stage composed of a bank of spatial band-pass filters and a static nonlinearity, mimicking complex binocular cells. Then, subsequent pooling stages and decoding strategies allow the model to estimate the vector disparity, after having represented it as a population of MT-like units. We assess the proposed model by using standard benchmarking stereo images, the Middlebury dataset, and specific stereo images that have horizontal and vertical disparities, which characterize the stimuli produced by active vision systems. Moreover, we systemically analyze how the different processing stages affect the model performance, and we discuss their implications for the neural modeling.

    References

    • 1. J. E. Cutting, How the eye measures reality and virtual reality, Behav. Res. Methods Instrum. Comput. 29(1) (1997) 27–36. Google Scholar
    • 2. J.-B. Durand, S. Zhu, S. Celebrini and Y. Trotter, Neurons in parafoveal areas V1 and V2 encode vertical and horizontal disparities, J. Neurophysiol. 88(5) (2002) 2874–2879. Medline, Web of ScienceGoogle Scholar
    • 3. Y. Trotter, S. Celebrini and J. B. Durand, Evidence for implication of primate area V1 in neural 3-D spatial localization processing, J. Physiol., Paris 98(1) (2004) 125–134. MedlineGoogle Scholar
    • 4. S. A. Chowdhury, D. L. Christiansen, M. L. Morgan and G. C. DeAngelis, Effect of vertical disparities on depth representation in macaque monkeys: MT physiology and behavior, J. Neurophysiol. 99(2) (2008) 876–887. Medline, Web of ScienceGoogle Scholar
    • 5. N. Kruger, P. Janssen, S. Kalkan, M. Lappe, A. Leonardis, J. Piater, A. J. Rodriguez-Sanchez and L. Wiskott, Deep hierarchies in the primate visual cortex: What can we learn for computer vision? IEEE Trans. Pattern Anal. Mach. Intell. 35(8) (2013) 1847–1871. Medline, Web of ScienceGoogle Scholar
    • 6. N. K. Medathati, H. Neumann, G. S. Masson and P. Kornprobst, Bio-inspired computer vision: Towards a synergistic approach of artificial and biological vision, Comput. Vis. Image Understand. 150 (2016) 1–30. Web of ScienceGoogle Scholar
    • 7. D. Scharstein and R. Szeliski, A taxonomy and evaluation of dense two-frame stereo correspondence algorithms, Int. J. Comput. Vis. 47(1–3) (2002) 7–42. Web of ScienceGoogle Scholar
    • 8. S. Baker, D. Scharstein, J. Lewis, S. Roth, M. J. Black and R. Szeliski, A database and evaluation methodology for optical flow, Int. J. Comput. Vis. 92(1) (2011) 1–31. Web of ScienceGoogle Scholar
    • 9. D. Scharstein, H. Hirschmüller, Y. Kitajima, G. Krathwohl, N. Nešić, X. Wang and P. Westling, High-resolution stereo datasets with subpixel-accurate ground truth, in Proc. German Conf. Pattern Recognition (Springer, Cham, 2014), pp. 31–42. Google Scholar
    • 10. D. Marr and T. Poggio, A computational theory of human stereo vision, Proc. R. Soc. Lond. B, Biol. Sci. 204(1156) (1979) 301–328. Medline, Web of ScienceGoogle Scholar
    • 11. I. Ohzawa, G. C. DeAngelis and R. D. Freeman, Stereoscopic depth discrimination in the visual cortex: Neurons ideally suited as disparity detectors, Science 249(4972) (1990) 1037–1041. Medline, Web of ScienceGoogle Scholar
    • 12. D. J. Fleet, H. Wagner and D. J. Heeger, Neural encoding of binocular disparity: Energy models, position shifts and phase shifts, Vis. Res. 36(12) (1996) 1839–1857. Medline, Web of ScienceGoogle Scholar
    • 13. Y. Chen and N. Qian, A coarse-to-fine disparity energy model with both phase-shift and position-shift receptive field mechanisms, Neural Comput. 16(8) (2004) 1545–1577. Medline, Web of ScienceGoogle Scholar
    • 14. B. Decoux, Self-supervised learning in cooperative stereo vision correspondence, Int. J. Neural Syst. 08(1) (1997) 101–111. Link, Web of ScienceGoogle Scholar
    • 15. F. Wörgötter, An ASIC-chip for stereoscopic depth analysis in video-real-time based on visual cortical cell behavior, Int. J. Neural Syst. 09(5) (1999) 417–422. LinkGoogle Scholar
    • 16. H. Wersing, S. Kirstein, M. Götting, H. Brandl, M. Dunn, I. Mikhailova, C. Goerick, J. Steil, H. Ritter and E. Körner, Online learning of objects in a biologically motivated visual architecture, Int. J. Neural Syst. 17(4) (2007) 219–230. Link, Web of ScienceGoogle Scholar
    • 17. J. J. Tsai and J. D. Victor, Reading a population code: A multi-scale neural model for representing binocular disparity, Vis. Res. 43(4) (2003) 445–466. Medline, Web of ScienceGoogle Scholar
    • 18. N. Chumerin, A. Gibaldi, S. P. Sabatini and M. M. Van Hulle, Learning eye vergence control from a distributed disparity representation, Int. J. Neural Syst. 20(04) (2010) 267–278. Link, Web of ScienceGoogle Scholar
    • 19. A. Gibaldi, M. Chessa, A. Canessa, S. P. Sabatini and F. Solari, A cortical model for binocular vergence control without explicit calculation of disparity, Neurocomputing 73(7–9) (2010) 1065–1073. Web of ScienceGoogle Scholar
    • 20. K. Pauwels and M. M. Van Hulle, Head-centric disparity and epipolar geometry estimation from a population of binocular energy neurons, Int. J. Neural Syst. 22(03) (2012) 1250007. Link, Web of ScienceGoogle Scholar
    • 21. J. Burge and W. S. Geisler, Optimal disparity estimation in natural stereo images, J. Vis. 14(2) (2014) 1. Medline, Web of ScienceGoogle Scholar
    • 22. J. A. Martins, J. M. Rodrigues and H. du Buf, Luminance, color, viewpoint and border enhanced disparity energy model, PLoS ONE 10(6) (2015) e0129908. Medline, Web of ScienceGoogle Scholar
    • 23. K. Pauwels, N. Krüger, M. Lappe, F. Wörgötter and M. M. Van Hulle, A cortical architecture on parallel hardware for motion processing in real time, J. Vis. 10(10) (2010) 18. Medline, Web of ScienceGoogle Scholar
    • 24. M. Chessa, V. Bianchi, M. Zampetti, S. P. Sabatini and F. Solari, Real-time simulation of large-scale neural architectures for visual features computation based on GPU, Network 23(4) (2012) 272–291. Medline, Web of ScienceGoogle Scholar
    • 25. M. Chessa and G. Pasquale, Graphics processing unit-accelerated techniques for bio-inspired computation in the primary visual cortex, Concurrency Comput. Pract. Exp. 26(10) (2013) 1799–1818. Web of ScienceGoogle Scholar
    • 26. R. T. Born and D. C. Bradley, Structure and function of visual area MT, Annu. Rev. Neurosci. 28 (2005) 157–189. Medline, Web of ScienceGoogle Scholar
    • 27. N. Rust, V. Mante, E. Simoncelli and J. Movshon, How MT cells analyze the motion of visual patterns, Nat. Neurosci. 9 (2006) 1421–1431. Medline, Web of ScienceGoogle Scholar
    • 28. F. Solari, M. Chessa, N. K. Medathati and P. Kornprobst, What can we expect from a V1-MT feedforward architecture for optical flow estimation? Signal Process., Image Commun. 39(B) (2015) 342–354. Web of ScienceGoogle Scholar
    • 29. M. A. Goodale and D. A. Westwood, An evolving view of duplex vision: Separate but interacting cortical pathways for perception and action, Curr. Opin. Neurobiol. 14(2) (2004) 203–211. Medline, Web of ScienceGoogle Scholar
    • 30. E. Kaplan, K. Purpura and R. M. Shapley, Contrast affects the transmission of visual information through the mammalian lateral geniculate nucleus, J. Physiol. 391 (1987) 267–288. Medline, Web of ScienceGoogle Scholar
    • 31. V. Bonin, V. Mante and M. Carandini, The statistical computation underlying contrast gain control, J. Neurosci. 26(23) (2006) 6346–6353. Medline, Web of ScienceGoogle Scholar
    • 32. L. Spillmann and J. S. Werner, Long-range interactions in visual perception, Trends Neurosci. 19(10) (1996) 428–434. Medline, Web of ScienceGoogle Scholar
    • 33. J. N. McManus, S. Ullman and C. D. Gilbert, A computational model of perceptual fill-in following retinal degeneration, J. Neurophysiol. 99(5) (2008) 2086–2100. Medline, Web of ScienceGoogle Scholar
    • 34. G. Mather, A. Pavan, R. M. Bellacosa and C. Casco, Psychophysical evidence for interactions between visual motion and form processing at the level of motion integrating receptive fields, Neuropsychologia 50(1) (2012) 153–159. Medline, Web of ScienceGoogle Scholar
    • 35. M. S. Silverman, D. H. Grosof, R. L. De Valois and S. D. Elfar, Spatial-frequency organization in primate striate cortex, Proc. Natl. Acad. Sci. USA 86(2) (1989) 711–715. Medline, Web of ScienceGoogle Scholar
    • 36. N. Wadhwa, M. Rubinstein, F. Durand and W. T. Freeman, Phase-based video motion processing, ACM Trans. Graph. 32 (2013) 80:1–80:10. Web of ScienceGoogle Scholar
    • 37. B. Farell, Two-dimensional matches from one-dimensional stimulus components in human stereopsis, Nature 395(6703) (1998) 689–693. Medline, Web of ScienceGoogle Scholar
    • 38. M. Morgan and E. Castet, The aperture problem in stereopsis, Vis. Res. 37(19) (1997) 2737–2744. Medline, Web of ScienceGoogle Scholar
    • 39. S. Nishimoto and J. L. Gallant, A three-dimensional spatiotemporal receptive field model explains responses of area MT neurons to naturalistic movies, J. Neurosci. 31(41) (2011) 14551–14564. Medline, Web of ScienceGoogle Scholar
    • 40. M. Carandini and D. J. Heeger, Normalization as a canonical neural computation, Nat. Rev. Neurosci. 13(1) (2011) 51–62. Medline, Web of ScienceGoogle Scholar
    • 41. J. G. Daugman, Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters, J. Opt. Soc. Am. A 2(7) (1985) 1160–1169. MedlineGoogle Scholar
    • 42. N. Qian and Y. Zhu, Physiological computation of binocular disparity, Vis. Res. 37(13) (1997) 1811–1827. Medline, Web of ScienceGoogle Scholar
    • 43. D. J. Heeger, Normalization of cell responses in cat striate cortex, Vis. Neurosci. 9(02) (1992) 181–197. Medline, Web of ScienceGoogle Scholar
    • 44. E. Simoncelli and D. Heeger, A model of neuronal responses in visual area MT, Vis. Res. 38 (1998) 743–761. Medline, Web of ScienceGoogle Scholar
    • 45. G. C. DeAngelis and T. Uka, Coding of horizontal disparity and velocity by MT neurons in the alert macaque, J. Neurophysiol. 89(2) (2003) 1094–1111. Medline, Web of ScienceGoogle Scholar
    • 46. L. Paninski, Maximum likelihood estimation of cascade point-process neural encoding models, Network 15(4) (2004) 243–262. Medline, Web of ScienceGoogle Scholar
    • 47. J. H. Maunsell and D. C. Van Essen, Functional properties of neurons in middle temporal visual area of the macaque monkey: I: Selectivity for stimulus direction, speed, and orientation, J. Neurophysiol. 49(5) (1983) 1127–1147. Medline, Web of ScienceGoogle Scholar
    • 48. S. Paris, P. Kornprobst, J. Tumblin and F. Durand, Bilateral Filtering: Theory and Applications (Now Publishers Inc., 2009). Google Scholar
    • 49. M. Chessa, N. K. Medathati, G. S. Masson, F. Solari and P. Kornprobst, Decoding MT motion response for optical flow estimation: An experimental evaluation, in Proc. 2015 23rd European Signal Processing Conf. (EUSIPCO) 2015 23rd European (IEEE, 2015), pp. 2241–2245. Google Scholar
    • 50. A. Pouget, K. Zhang, S. Deneve and P. E. Latham, Statistically efficient estimation using population coding, Neural Comput. 10(2) (1998) 373–401. Medline, Web of ScienceGoogle Scholar
    • 51. K. R. Rad and L. Paninski, Information rates and optimal decoding in large neural populations, in Proc. Conf. Advances in Neural Information Processing Systems 24 eds. J. Shawe-TaylorR. S. ZemelP. L. BartlettF. C. N. PereiraK. Q. Weinberger (NIPS Foundation Inc., 20110), pp. 846–854. Google Scholar
    • 52. D. C. Bradley and M. S. Goyal, Velocity computation in the primate visual system, Nat. Rev. Neurosci. 9(9) (2008) 686–695. Medline, Web of ScienceGoogle Scholar
    • 53. K. Foster, J. P. Gaska, M. Nagler and D. Pollen, Spatial and temporal frequency selectivity of neurones in visual cortical areas V1 and V2 of the macaque monkey, J. Physiol. 365(1) (1985) 331–363. Medline, Web of ScienceGoogle Scholar
    • 54. A. J. Parker, Binocular depth perception and the cerebral cortex, Nat. Rev. Neurosci. 8(5) (2007) 379–391. Medline, Web of ScienceGoogle Scholar
    • 55. C. A. J. R. Bergen, E. H. Adelson, P. Burt and J. Ogden, Pyramid methods in image processing, RCA Eng. 29 (1984) 33–41. Google Scholar
    • 56. E. P. Simoncelli, Course-to-fine estimation of visual motion, in Proc. IEEE Eighth Workshop Image and Multidimensional Signal Processing (1993). Google Scholar
    • 57. M. Chessa, F. Solari and S. P. Sabatini, A virtual reality simulator for active stereo vision systems, in Proc. Fourth Int. Conf. Computer Vision Theory and Applications, Vol. 2 (2009), pp. 444–449. Google Scholar
    • 58. A. Canessa, A. Gibaldi, M. Chessa, M. Fato, F. Solari and S. P. Sabatini, A dataset of stereoscopic images and ground-truth disparity mimicking human fixations in peripersonal space, Sci. Data 4 (2017) 170034. Medline, Web of ScienceGoogle Scholar
    • 59. H. Hirschmuller, Accurate and efficient stereo processing by semi-global matching and mutual information, in Proc. IEEE Computer Society Conf. Computer Vision and Pattern Recognition, Vol. 2 (IEEE, 2005), pp. 807–814. Google Scholar
    • 60. L. Nalpantidis and A. Gasteratos, Stereo vision for robotic applications in the presence of non-ideal lighting conditions, Image Vis. Comput. 28(6) (2010) 940–951. Web of ScienceGoogle Scholar
    • 61. M. Chessa, G. Maiello, P. J. Bex and F. Solari, A space-variant model for motion interpretation across the visual field, J. Vis. 16(2) (2016) 12. MedlineGoogle Scholar
    • 62. M. Chessa, S. P. Sabatini and F. Solari, A systematic analysis of a V1-MT neural model for motion estimation, Neurocomputing 173 (2016) 1811–1823. Web of ScienceGoogle Scholar
    • 63. J. C. A. Read and B. G. Cumming, Does depth perception require vertical-disparity detectors?, J. Vis. 6(12) (2006) 1. Medline, Web of ScienceGoogle Scholar
    • 64. I. Serrano-Pedraza and J. C. A. Read, Stereo vision requires an explicit encoding of vertical disparity, J. Vis. 9(4) (2009 3. Medline, Web of ScienceGoogle Scholar
    • 65. G. F. Poggio, Mechanisms of stereopsis in monkey visual cortex, Cereb. Cortex 5(3) (1995) 193–204. Medline, Web of ScienceGoogle Scholar
    • 66. B. G. Cumming and G. C. DeAngelis, The physiology of stereopsis, Annu. Rev. Neurosci. 24(1) (2001) 203–238. Medline, Web of ScienceGoogle Scholar
    • 67. S. Prince, B. Cumming and A. Parker, Range and mechanism of encoding of horizontal disparity in macaque V1, J. Neurophysiol. 87(1) (2002) 209–221. Medline, Web of ScienceGoogle Scholar
    • 68. R. Blake and H. Wilson, Binocular vision, Vis. Res. 51(7) (2011) 754–770. Medline, Web of ScienceGoogle Scholar
    • 69. M. R. Jenkin, A. D. Jepson and J. K. Tsotsos, Techniques for disparity measurement, CVGIP: Image Understand. 53(1) (1991) 14–30. Web of ScienceGoogle Scholar
    • 70. J.-P. Roy, H. Komatsu and R. H. Wurtz, Disparity sensitivity of neurons in monkey extrastriate area MST, J. Neurosci. 12(7) (1992) 2478–2492. Medline, Web of ScienceGoogle Scholar
    • 71. A. W. Roe, A. J. Parker, R. T. Born and G. C. DeAngelis, Disparity channels in early vision, J. Neurosci. 27(44) (2007) 11820–11831. Medline, Web of ScienceGoogle Scholar
    • 72. X. Xu, J. M. Ichida, J. D. Allison, J. D. Boyd, A. Bonds and V. A. Casagrande, A comparison of koniocellular, magnocellular and parvocellular receptive field properties in the lateral geniculate nucleus of the owl monkey (aotus trivirgatus), J. Physiol. 531(1) (2001) 203–218. Medline, Web of ScienceGoogle Scholar
    • 73. P. Neri, A stereoscopic look at visual cortex, J. Neurophysiol. 93(4) (2005) 1823–1826. Medline, Web of ScienceGoogle Scholar
    Remember to check out the Most Cited Articles!

    Check out our titles in neural networks today!