World Scientific
  • Search
Skip main navigation

Cookies Notification

We use cookies on this site to enhance your user experience. By continuing to browse the site, you consent to the use of our cookies. Learn More
×
Our website is made possible by displaying certain online content using javascript.
In order to view the full content, please disable your ad blocker or whitelist our website www.worldscientific.com.

System Upgrade on Tue, Oct 25th, 2022 at 2am (EDT)

Existing users will be able to log into the site and access content. However, E-commerce and registration of new users may not be available for up to 12 hours.
For online purchase, please visit us again. Contact us at [email protected] for any enquiries.
Special Issue: Selected Papers from the First IEEE International Conference Artificial Intelligence and Virtual Reality (AIVR 2018) and 20th IEEE International Symposium on Multimedia (ISM 2018); Guest Editors: C.-M. Chang and R. S. AygunNo Access

Efficient Live and On-Demand Tiled HEVC 360 VR Video Streaming

    360 panorama video displayed through Virtual Reality (VR) glasses or large screens offers immersive user experiences, but as such technology becomes commonplace, the need for efficient streaming methods of such high-bitrate videos arises. In this respect, the attention that 360 panorama video has received lately is huge. Many methods have already been proposed, and in this paper, we shed more light on the different trade-offs in order to save bandwidth while preserving the video quality in the user’s field-of-view (FoV). Using 360 VR content delivered to a Gear VR head-mounted display with a Samsung Galaxy S7 and to a Huawei Q22 set-top-box, we have tested various tiling schemes analyzing the tile layout, the tiling and encoding overheads, mechanisms for faster quality switching beyond the DASH segment boundaries and quality selection configurations. In this paper, we present an efficient end-to-end design and real-world implementation of such a 360 streaming system. Furthermore, in addition to researching an on-demand system, we also go beyond the existing on-demand solutions and present a live streaming system which strikes a trade-off between bandwidth usage and the video quality in the user’s FoV. We have created an architecture that combines RTP and DASH, and our system multiplexes a single HEVC hardware decoder to provide faster quality switching than at the traditional GOP boundaries. We demonstrate the performance and illustrate the trade-offs through real-world experiments where we can report comparable bandwidth savings to existing on-demand approaches, but with faster quality switches when the FoV changes.

    References

    • 1. H. Ahmadi, O. Eltobgy and M. Hefeeda, Adaptive multicast streaming of virtual reality content to mobile users, in Proc. Thematic Workshops of ACM MM, 2017, pp. 170–178. CrossrefGoogle Scholar
    • 2. T. Aykut, S. Lochbrunner, M. Karimi, B. Cizmeci and E. Steinbach, A stereoscopic vision system with delay compensation for 360 remote reality, in Proc. Thematic Workshops of ACM MM, 2017, pp. 201–209. Google Scholar
    • 3. Camargus. Premium Stadium Video Technology Infrastructure. https://www.youtube.com/watch?v=SO32pEgCeDI. Google Scholar
    • 4. P. Carr and R. Hartley, Portable multi-megapixel camera with real-time recording and playback, in Proc. IEEE DICTA, 2009, pp. 74–80. CrossrefGoogle Scholar
    • 5. P. Carr, M. Mistry and I. Matthews, Hybrid robotic/virtual pan-tilt-zom cameras for autonomous event recording, in Proc. ACM MM, 2013, pp. 193–202. CrossrefGoogle Scholar
    • 6. X. Corbillon, F. De Simone and G. Simon, 360 degreee video head movement dataset, in Proc. MMSYS, 2017, pp. 199–204. Google Scholar
    • 7. X. Corbillon, A. Devlic, G. Simon and J. Chakareski, Optimal set of 360-degree videos for viewport-adaptive streaming, in Proc. ACM MM, 2017, pp. 943–951. CrossrefGoogle Scholar
    • 8. Facebook. Transform360. https://github.com/facebook/transform360, 2008. [Online; accessed 13-March-2018]. Google Scholar
    • 9. C. Fehn, C. Weissig, I. Feldmann, M. Muller, P. Eisert, P. Kauff and H. Bloss, Creation of high-resolution video panoramas of sport events, in Proc. IEEE ISM, 2006, pp. 291–298. CrossrefGoogle Scholar
    • 10. E. Foote, P. Carr, P. Lucey, Y. Sheikh and I. Matthews, One-man-band: A touch screen interface for producing live multi-camera sports broadcasts, in Proc. ACM MM, 2013, pp. 163–172. CrossrefGoogle Scholar
    • 11. V. R. Gaddam, R. Langseth, S. Ljødal, P. Gurdjos, V. Charvillat, C. Griwodz and P. Halvorsen, Interactive zoom and panning from live panoramic video, in Proc. ACM NOSSDAV, 2014, pp. 19–24. CrossrefGoogle Scholar
    • 12. V. R. Gaddam, H. B. Ngo, R. Langseth, C. Griwodz, D. Johansen and P. Halvorsen, Tiling of panorama video for interactive virtual cameras: Overheads and potential bandwidth requirement reduction, in Proc. Packet Video — Picture Coding Symposium, 2015, pp. 204–209. Google Scholar
    • 13. V. R. Gaddam, M. Riegler, R. Eg, C. Griwodz and P. Halvorsen, Tiling in interactive panoramic video: Approaches and evaluation, IEEE Trans. Multimedia 18(9) [2016] 1819–1831. Crossref, ISIGoogle Scholar
    • 14. M. Graf, C. Timmerer and C. Mueller, Towards bandwidth efficient adaptive streaming of omnidirectional video over http: Design, implementation, and evaluation, in Proc. MMSYS, 2017, pp. 261–271. CrossrefGoogle Scholar
    • 15. R. Guntur and W. T. Ooi, On tile assignment for region-of-interest video streaming in a wireless LAN, in Proc. ACM NOSSDAV, 2012. CrossrefGoogle Scholar
    • 16. P. Halvorsen, S. Sægrov, A. Mortensen, D. K. C. Kristensen, A. Eichhorn, M. Stenhaug, S. Dahl, H. K. Stensland, V. R. Gaddam, C. Griwodz and D. Johansen, Bagadus: An integrated system for arena sports analytics - a soccer case study, in Proc. ACM MMSys, 2013, pp. 48–59. CrossrefGoogle Scholar
    • 17. Fraunhofer Heinrich Hertz Institute. HEVC Test Model (HM). https://hevc.hhi.fraunhofer.de/HM-doc/, 2015. [Online; accessed 15-March-2018]. Google Scholar
    • 18. E. Jacobsen, C. Griwodz and P. Halvorsen, Pull-patching: A combination of multicast and adaptive segmented http streaming, in Proc. ACM MM, 2010, pp. 799–802. CrossrefGoogle Scholar
    • 19. M. Jeppsson, H. N. Espeland, T. Kupka, R. Langseth, A. Petlund, P. Qiaoqiao, C. Xue, K. Pogorelov, M. Riegler, D. Johansen, C. Griwodz and P. Halvorsen, Efficient live and on-demand tiled hevc 360 vr video streaming, in Proc. ISM, 2018. CrossrefGoogle Scholar
    • 20. H. Kimata, D. Ochi, A. Kameda, H. Noto, K. Fukazawa and A. Kojima, Mobile and multi-device interactive panorama video distribution system, in Proc. GCCE, 2012, pp. 574–578. CrossrefGoogle Scholar
    • 21. F. Liu and W. T. Ooi, Zoomable video playback on mobile devices by selective decoding, in Proc. PCM, 2012. CrossrefGoogle Scholar
    • 22. A. Mavlankar and B. Girod, Video streaming with interactive pan/tilt/zoom, in High-Quality Visual Experience, Signals and Communication Technology (Springer Verlag, 2010), pp. 431–455. CrossrefGoogle Scholar
    • 23. D. Naik, I. Curcio and H. Toukomaa, Optimized viewport dependent streaming of stereoscopic omnidirectional video, in Proc. Packet Video, 2018. CrossrefGoogle Scholar
    • 24. O. A. Niamut, E. Thomas, L. D’Acunto, C. Concolato, F. Denoual and S. Y. Lim, Mpeg dash srd: Spatial relationship description, in Proc. MMSYS, 2016, pp. 5:1–5:8. CrossrefGoogle Scholar
    • 25. S. Petrangeli, V. Swaminathan, M. Hosseini and F. De Turck, An http/2-based adaptive streaming framework for 360 virtual reality videos, in Proc. ACM MM, 2017, pp. 306–314. CrossrefGoogle Scholar
    • 26. S. Petrangeli, V. Swaminathan, M. Hosseini and F. De Turck, Improving virtual reality streaming using http/2, in Proc. MMSYS, 2017, pp. 225–228. CrossrefGoogle Scholar
    • 27. M. Pettersson, R. Sjöberg and J. Samuelsson, Dependent random access point pictures in hevc, in Proc. IEEE ICIP, 2015, pp. 867–871. CrossrefGoogle Scholar
    • 28. P. R. Alface, M. Aerts, D. Tytgat, S. Lievens, C. Stevens, N. Verzijp and J.-F. Macq. 16k cinematic vr streaming, in Proc. ACM MM, 2017, pp. 1105–1112. Google Scholar
    • 29. D. Salomon, Data Compression: The Complete Reference (Springer-Verlag, New York, 2006). Google Scholar
    • 30. Y. Sanchez, D. Podborski, C. Hellge and T. Schierl, Shifted idr representations for low delay live dash streaming using hevc tiles, in Proc. IEEE ISM, 2016, pp. 87–92. CrossrefGoogle Scholar
    • 31. Y. Sanchez, R. Skupin and T. Schierl, Compressed domain video processing for tile based panoramic streaming using hevc, in Proc. ICIP, 2015, pp. 2244–2248. CrossrefGoogle Scholar
    • 32. H.-Y. Shum, K.-T. Ng and S.-C. Chan, A virtual reality system using the concentric mosaic: Construction, rendering, and data compression, IEEE Trans. Multimedia 7(1) [2005] 85–95. Crossref, ISIGoogle Scholar
    • 33. J. Son, D. Jang and E.-S. Ryu, Implementing motion-constrained tile and viewport extraction for VR streaming, in Proc. NOSSDAV, 2018. CrossrefGoogle Scholar
    • 34. X. Sun, J. Foote, D. Kimber and B. S. Manjunath, Region of interest extraction and virtual camera control based on panoramic video capturing, IEEE Trans. Multimedia 7(5) [2005] 981–990. Crossref, ISIGoogle Scholar
    • 35. R. Szeliski and H.-Y. Shum, Creating full view panoramic image mosaics and environment maps, in Proc. SIGGRAPH, 1997, pp. 251–258. CrossrefGoogle Scholar
    • 36. W.-K. Tang, T.-T. Wong and P.-A. Heng, A system for real-time panorama generation and display in tele-immersive applications, IEEE Trans. Multimedia 7(2) [2005] 280–292. Crossref, ISIGoogle Scholar
    • 37. Teamcoco. Conan 360: The New Angle on Late Night, 2014. http://teamcoco.com/360. Google Scholar
    • 38. S. Tzavidas and A. K. Katsaggelos, A multicamera setup for generating stereo panoramic video, IEEE Trans. Multimedia 7(5) [2005] 880–890. Crossref, ISIGoogle Scholar
    • 39. M. Xiao, C. Zhou, Y. Liu and S. Chen, Optile: Toward optimal tiling in 360-degree video streaming, in Proc. ACM MM, 2017, pp. 708–716. CrossrefGoogle Scholar
    • 40. L. Xie, Z. Xu, Y. Ban, X. Zhang and Z. Guo, 360probdash: Improving qoe of 360 video streaming using tile-based http adaptive streaming, in Proc. ACM MM, 2017, pp. 315–323. CrossrefGoogle Scholar
    • 41. T. Yokoi and H. Fujiyoshi, Virtual camerawork for generating lecture video from high resolution images, in Proc. IEEE ICME, 2005. CrossrefGoogle Scholar
    • 42. M. Yu, H. Lakshman and B. Girod, A framework to evaluate omnidirectional video coding schemes, in Proc. IEEE ISMAR, 2015, pp. 31–36. CrossrefGoogle Scholar
    • 43. A. Zare, A. Aminlou and M. Hannuksela, 6K Effective Resolution with 4K HEVC Decoding Capability for OMAF-compliant 360 Video Streaming, in Proc. Packet Video, 2018. CrossrefGoogle Scholar
    • 44. Q. Zhao, L. Wan, W. Feng, J. Zhang and T.-T. Wong, Cube2video: Navigate between cubic panoramas in real-time, IEEE Trans. Multimedia 15(8) [2013] 1745–1754. Crossref, ISIGoogle Scholar
    • 45. C. Zhou, Z. Li and Y. Liu, A measurement study of oculus 360 degree video streaming, in Proc. MMSYS, 2017, pp. 27–37. CrossrefGoogle Scholar
    • 46. G. Zoric, L. Barkhuus, A. Engström and E. Önnevall, Panoramic video: Design challenges and implications for content interaction, in Proc. EuroITV, 2013. CrossrefGoogle Scholar
    Remember to check out the Most Cited Articles!

    Check out our titles in Semantic Computing!