World Scientific
Skip main navigation
Close Drawer MenuOpen Drawer Menu

Cookies Notification

We use cookies on this site to enhance your user experience. By continuing to browse the site, you consent to the use of our cookies. Learn More
×
Our website is made possible by displaying certain online content using javascript.
In order to view the full content, please disable your ad blocker or whitelist our website www.worldscientific.com.

System Upgrade on Mon, Jun 21st, 2021 at 1am (EDT)

During this period, the E-commerce and registration of new users may not be available for up to 6 hours.
For online purchase, please visit us again. Contact us at [email protected] for any enquiries.
Free Access

Influence of Data Representations and Deep Architectures in Image Time Series Classification

    https://doi.org/10.1142/S0218001421600016Cited by:0
    Previous
    Next
    This article is part of the issue:
    Special Issue on the International Conference on Pattern Recognition and Artificial Intelligence (ICPRAI 2020)
    Guest Editors: Yue Lu (East China Normal University), Nicole Vincent (Université de Paris, France), Ching Y. Suen (Concordia University, Canada) and Patrick S.P.Wang (Northeastern University, USA)

    Abstract

    Image time series, such as Satellite Image Time Series (SITS) or MRI functional sequences in the medical domain, carry both spatial and temporal information. In many pattern recognition applications such as image classification, taking into account such rich information may be crucial and discriminative during the decision making stage. However, the extraction of spatio-temporal features from image time series is difficult to handle due to the complex representation of the data cube. In this paper, we present a strategy based on Random Walk to build a novel segment-based representation of the data, passing from a 2D+t dimension to a 2D one, more easily manipulable and without losing too much spatial information. Such new representation is then used to feed a classical Convolutional Neural Network (CNN) in order to learn spatio-temporal features with only 2D convolutions and to classify image time series data for a particular classification problem. The influence of the way the 2D+t data are represented, as well as the impact of the network architectures on the results, are carefully studied. The interest of this approach is highlighted on a remote sensing application for the classification of complex agricultural crops.

    References

    • 1. L. Andres, W. Salas and D. Skole , Fourier analysis of multi-temporal AVHRR data applied to a land cover classification, Int. J. Remote Sens. 15(5) (1994) 1115–1121. Crossref, ISI, Google Scholar
    • 2. A. Bagnall, J. Lines, A. Bostrom, J. Large and E. Keogh , The great time series classification bake off: A review and experimental evaluation of recent algorithmic advances, Data Min. Knowl. Discov. 31(3) (2017) 606–660. Crossref, ISI, Google Scholar
    • 3. L. Bruzzone and D. Prieto , Automatic analysis of the difference image for unsupervised change detection, IEEE Trans. Geosci. Remote Sens. 38(3) (2000) 1171–1182. Crossref, ISI, Google Scholar
    • 4. M. Chelali, C. Kurtz, A. Puissant and N. Vincent , Urban land cover analysis from satellite image time series based on temporal stability, in JURSE Proc. (2019), pp. 1–4. Crossref, Google Scholar
    • 5. P. Coppin, I. Jonckheere, K. Nackaerts, B. Muys and E. Lambin , Digital change detection methods in ecosystem monitoring: A review, Int. J. Remote Sens. 25(9) (2004) 1565–1596. Crossref, ISI, Google Scholar
    • 6. N. Di Mauro, A. Vergari, T. M. A. Basile, F. G. Ventola and F. Esposito , End-to-end learning of deep spatio-temporal representations for satellite image time series classification, in [email protected]/ECML Proc. (2017), pp. 1–8. Google Scholar
    • 7. L. Grady , Multilabel random walker image segmentation using prior models, in CVPR Proc. (2005), pp. 763–770. Crossref, Google Scholar
    • 8. B. Huang, K. Lu, N. Audebert, A. Khalel, Y. Tarabalka, J. Malof and A. Boulch , Large-scale semantic classification: Outcome of the first year of inria aerial image labeling benchmark, in IGARSS Proc. (2018), pp. 6947–6950. Crossref, Google Scholar
    • 9. F. Iandola, M. Moskewicz, K. Ashraf, S. Han, W. Dally and K. Keutzer, Squeezenet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size, Comput. Res. Reposit., arXiv:abs/1602.07360. Google Scholar
    • 10. D. Ienco, R. Gaetano, C. Dupaquier and P. Maurel , Land cover classification via multitemporal spatial data by deep recurrent neural networks, IEEE Geosci. Remote Sens. Lett. 14(10) (2017) 1685–1689. Crossref, ISI, Google Scholar
    • 11. J. Inglada, A. Vincent, M. Arias, B. Tardy, D. Morin and I. Rodes , Operational high resolution land cover map production at the country scale using satellite image time series, Remote Sens. 9(1) (2017) 95–108. Crossref, ISI, Google Scholar
    • 12. H. Ismail Fawaz, G. Forestier, J. Weber, L. Idoumghar and P. Muller , Deep learning for time series classification: A review, Data Min. Knowl. Discov. 33(4) (2019) 917–963. Crossref, ISI, Google Scholar
    • 13. J. R. Jensen , Urban change detection mapping using Landsat digital data, Cartogr. Geogr. Inf. Sci. 8(21) (1981) 127–147. Crossref, Google Scholar
    • 14. R. Johnson and E. Kasischke , Change vector analysis: A technique for the multispectral monitoring of land cover and condition, Int. J. Remote Sens. 19(16) (1998) 411–426. Crossref, ISI, Google Scholar
    • 15. A. Krizhevsky, I. Sutskever and G. E. Hinton , ImageNet classification with deep convolutional neural networks, in NIPS Proc. (2012), pp. 1106–1114. Google Scholar
    • 16. C. Pelletier, G. Webb and F. Petitjean , Temporal convolutional neural network for the classification of satellite image time series, Remote Sens. 11(5) (2019) 523–534. Crossref, ISI, Google Scholar
    • 17. F. Petitjean, J. Inglada and P. Gançarski , Satellite image time series analysis under time warping, IEEE Trans. Geosci. Remote Sens. 50(8) (2012) 3081–3095. Crossref, ISI, Google Scholar
    • 18. F. Petitjean, C. Kurtz, N. Passat and P. Gançarski , Spatio-temporal reasoning for the classification of satellite image time series, Pattern Recognit. Lett. 33(13) (2012) 1805–1815. Crossref, ISI, Google Scholar
    • 19. P. Ravikumar and V. S. Devi , Weighted feature-based classification of time series data, in CIDM Proc. (2014), pp. 222–228. Crossref, Google Scholar
    • 20. M. Russwurm and M. Korner , Temporal vegetation modelling using long short-term memory networks for crop identification from medium-resolution multi-spectral satellite images, in [email protected] Proc. (2017), pp. 1496–1504. Google Scholar
    • 21. C. Senf, P. Leitao, D. Pflugmacher, S. Van der Linden and P. Hostert , Mapping land cover in complex mediterranean landscapes using landsat: Improved classification accuracies from integrating multi-seasonal and synthetic imagery, Remote Sens. Environ. 156 (2015) 527–536. Crossref, ISI, Google Scholar
    • 22. K. Simonyan and A. Zisserman , Very deep convolutional networks for large-scale image recognition, in ICLR Proc. (2015). Google Scholar
    • 23. D. Tran, L. Bourdev, R. Fergus, L. Torresani and M. Paluri , Learning spatiotemporal features with 3D convolutional networks, in ICCV Proc. (2015), pp. 4489–4497. Crossref, Google Scholar
    • 24. J. Verbesselt, R. Hyndman, G. Newnham and D. Culvenor , Detecting trend and seasonal changes in satellite image time series, Remote Sens. Environ. 114(1) (2010) 106–115. Crossref, ISI, Google Scholar
    Published: 16 September 2021