• Search

×
Our website is made possible by displaying certain online content using javascript.
In order to view the full content, please disable your ad blocker or whitelist our website www.worldscientific.com.

### System Upgrade on Mon, Jun 21st, 2021 at 1am (EDT)

During this period, the E-commerce and registration of new users may not be available for up to 6 hours.
Free Access

# Influence of Data Representations and Deep Architectures in Image Time Series Classification

Image time series, such as Satellite Image Time Series (SITS) or MRI functional sequences in the medical domain, carry both spatial and temporal information. In many pattern recognition applications such as image classification, taking into account such rich information may be crucial and discriminative during the decision making stage. However, the extraction of spatio-temporal features from image time series is difficult to handle due to the complex representation of the data cube. In this paper, we present a strategy based on Random Walk to build a novel segment-based representation of the data, passing from a 2D$+t$ dimension to a 2D one, more easily manipulable and without losing too much spatial information. Such new representation is then used to feed a classical Convolutional Neural Network (CNN) in order to learn spatio-temporal features with only 2D convolutions and to classify image time series data for a particular classification problem. The influence of the way the 2D$+t$ data are represented, as well as the impact of the network architectures on the results, are carefully studied. The interest of this approach is highlighted on a remote sensing application for the classification of complex agricultural crops.

## References

• 1. L. Andres, W. Salas and D. Skole , Fourier analysis of multi-temporal AVHRR data applied to a land cover classification, Int. J. Remote Sens. 15(5) (1994) 1115–1121. Crossref, ISI
• 2. A. Bagnall, J. Lines, A. Bostrom, J. Large and E. Keogh , The great time series classification bake off: A review and experimental evaluation of recent algorithmic advances, Data Min. Knowl. Discov. 31(3) (2017) 606–660. Crossref, ISI
• 3. L. Bruzzone and D. Prieto , Automatic analysis of the difference image for unsupervised change detection, IEEE Trans. Geosci. Remote Sens. 38(3) (2000) 1171–1182. Crossref, ISI
• 4. M. Chelali, C. Kurtz, A. Puissant and N. Vincent , Urban land cover analysis from satellite image time series based on temporal stability, in JURSE Proc. (2019), pp. 1–4. Crossref
• 5. P. Coppin, I. Jonckheere, K. Nackaerts, B. Muys and E. Lambin , Digital change detection methods in ecosystem monitoring: A review, Int. J. Remote Sens. 25(9) (2004) 1565–1596. Crossref, ISI
• 6. N. Di Mauro, A. Vergari, T. M. A. Basile, F. G. Ventola and F. Esposito , End-to-end learning of deep spatio-temporal representations for satellite image time series classification, in [email protected]/ECML Proc. (2017), pp. 1–8. Google Scholar
• 7. L. Grady , Multilabel random walker image segmentation using prior models, in CVPR Proc. (2005), pp. 763–770. Crossref
• 8. B. Huang, K. Lu, N. Audebert, A. Khalel, Y. Tarabalka, J. Malof and A. Boulch , Large-scale semantic classification: Outcome of the first year of inria aerial image labeling benchmark, in IGARSS Proc. (2018), pp. 6947–6950. Crossref
• 9. F. Iandola, M. Moskewicz, K. Ashraf, S. Han, W. Dally and K. Keutzer, Squeezenet: AlexNet-level accuracy with 50x fewer parameters and $<$1MB model size, Comput. Res. Reposit., arXiv:abs/1602.07360. Google Scholar
• 10. D. Ienco, R. Gaetano, C. Dupaquier and P. Maurel , Land cover classification via multitemporal spatial data by deep recurrent neural networks, IEEE Geosci. Remote Sens. Lett. 14(10) (2017) 1685–1689. Crossref, ISI
• 11. J. Inglada, A. Vincent, M. Arias, B. Tardy, D. Morin and I. Rodes , Operational high resolution land cover map production at the country scale using satellite image time series, Remote Sens. 9(1) (2017) 95–108. Crossref, ISI
• 12. H. Ismail Fawaz, G. Forestier, J. Weber, L. Idoumghar and P. Muller , Deep learning for time series classification: A review, Data Min. Knowl. Discov. 33(4) (2019) 917–963. Crossref, ISI
• 13. J. R. Jensen , Urban change detection mapping using Landsat digital data, Cartogr. Geogr. Inf. Sci. 8(21) (1981) 127–147. Crossref
• 14. R. Johnson and E. Kasischke , Change vector analysis: A technique for the multispectral monitoring of land cover and condition, Int. J. Remote Sens. 19(16) (1998) 411–426. Crossref, ISI
• 15. A. Krizhevsky, I. Sutskever and G. E. Hinton , ImageNet classification with deep convolutional neural networks, in NIPS Proc. (2012), pp. 1106–1114. Google Scholar
• 16. C. Pelletier, G. Webb and F. Petitjean , Temporal convolutional neural network for the classification of satellite image time series, Remote Sens. 11(5) (2019) 523–534. Crossref, ISI
• 17. F. Petitjean, J. Inglada and P. Gançarski , Satellite image time series analysis under time warping, IEEE Trans. Geosci. Remote Sens. 50(8) (2012) 3081–3095. Crossref, ISI
• 18. F. Petitjean, C. Kurtz, N. Passat and P. Gançarski , Spatio-temporal reasoning for the classification of satellite image time series, Pattern Recognit. Lett. 33(13) (2012) 1805–1815. Crossref, ISI
• 19. P. Ravikumar and V. S. Devi , Weighted feature-based classification of time series data, in CIDM Proc. (2014), pp. 222–228. Crossref
• 20. M. Russwurm and M. Korner , Temporal vegetation modelling using long short-term memory networks for crop identification from medium-resolution multi-spectral satellite images, in [email protected] Proc. (2017), pp. 1496–1504. Google Scholar
• 21. C. Senf, P. Leitao, D. Pflugmacher, S. Van der Linden and P. Hostert , Mapping land cover in complex mediterranean landscapes using landsat: Improved classification accuracies from integrating multi-seasonal and synthetic imagery, Remote Sens. Environ. 156 (2015) 527–536. Crossref, ISI
• 22. K. Simonyan and A. Zisserman , Very deep convolutional networks for large-scale image recognition, in ICLR Proc. (2015). Google Scholar
• 23. D. Tran, L. Bourdev, R. Fergus, L. Torresani and M. Paluri , Learning spatiotemporal features with 3D convolutional networks, in ICCV Proc. (2015), pp. 4489–4497. Crossref
• 24. J. Verbesselt, R. Hyndman, G. Newnham and D. Culvenor , Detecting trend and seasonal changes in satellite image time series, Remote Sens. Environ. 114(1) (2010) 106–115. Crossref, ISI
Published: 16 September 2021