Adaptive streaming for 3D video

Motivation

The distribution of multimedia content, in particular video streaming, currently dominates the global Internet traffic, and its importance will be even greater in the future. According to forecasts and statistics available, IP video traffic will account for 82 percent of all Internet traffic in 2020, up from 70 percent in 2015 [1], with Youtube, Netflix and Hulu being the videostreaming service providers more popular. Recent improvements in 3D video technology have led to a growing interest in the use of such content as an alternative to expand the user experience. New content demands new representation and coding schemes, which adjust to the conditions of transport and restrictions of bandwidth, and that allow to maximize the quality of user experience.

For this reason, the QoE (Quality of Experience) perceived by users of adaptive 3D video distribution systems has become a research topic with numerous contributions in recent years. The concept of user-centered QoE actually complements the concept of QoS, focusing on technical aspects (loss rate, delay, jitter, throughput, zapping time, etc.) of the networks that support these services. When evaluating the QoE and QoS of a video distribution system, it must be taken into account that both the losses associated with the coding and compression processes and the errors and losses during transmission can affect the quality perceived by the user.

Overview

In the 3D video scenario as described in [2], 3D video distribution systems use frame-compatible or full-resolution frame-compatible formats to be compatible with existing 2D video distribution systems, thus avoiding making massive upgrades of transmission infrastructures by the provider and receiving hardware by the user. While most 3D video content on the market is based on stereo video, 3D video technologies face challenges and opportunities to support more demanding application scenarios, such as systems based on self-stereoscopic displays or immersive telepresence applications In 3D. This, along with the stereoscopic video formats (H.265-HEVC High Efficiency Video Coding [3], H.264-AVC (Advanced Video Coding [4]), has led to the apperance of multivista formats based on the standard encoders HEVC-MVC and H.264-MVC (Multiview Video Coding) [5] and MVD (Multiview Video Plus Depth).

As the first objective of this line, a comparative study of the quality of video obtained using the most popular coding standards associated with 3D video is applied, applying both objective metrics and evaluation methods (PSNR Peak Signal-to-Noise Ratio, SSIM Structural Similarity Index, VQM Video Quality Metric) as subjective (ITU-T P.914 [6], ITU-T P.915 [7], ITU-T P.916 [8], ITU-R BT.2021-1 [9]) and using various implementations and coding parameters of each of the standards (Figure 1).

Figure 1. General scheme about a comparative study of 3D video coding standards

From the point of view of the mode of transport of video traffic on the Internet, the use of HTTP is its progressive download mode (Progressive Download) has evolved to its use by adaptive systems [10] (HAS, HTTP Adaptive Streaming). In the first case, the typical parameters for the evaluation of the QoE correspond to the initial delay (depending on the size of the buffer) and the number and duration of the interruptions. However, when HAS is used, because the client monitors and adapts the bit rate of the contents of the segments that are downloaded, there are practically no interruptions and the effect on the quality change of the content becomes more important. In this sense, it is proposed to evaluate DASH performance against different scenarios of variation of bandwidth and degradation of quality of service (QoS) and impact on the Quality of Experience (QoE) of the user in a distribution scenario 3D video as shown in Figure 2.

Figure 2. 3D video adaptive streaming system through Internet

References

[1] Cisco visual networking index: Forecast and methodology, 2015−2020, Jun. 2016.
[2] C. G. Gurler, B. Gorkemli, G. Saygili, A. M. Tekalp, “Flexible Transport of 3-D Video Over Networks,” Proceedings of the IEEE, vol. 99, no. 4, pp. 696-707, Apr. 2011.
[3] G. J. Sullivan, J.R. Ohm, W.J. Han, y T. Wiegand, “Overview of the High Efficiency Video Coding (HEVC) Standard,” IEEE Trans. Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1649-1668, Dec. 2012.
[4] T. Wiegand, G.J. Sullivan, G. Bjontegaard, and A. Luthra, “Overview of the H.264/AVC video coding standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no.7, 560-576, 2003.
[5] A. Vetro, T. Wiegand, and G. J. Sullivan, “Overview of the stereo and multiview video coding extensions of the H.264/MPEG-4 AVC standard,” Proceedings of the IEEE, vol. 99, no. 4, pp. 626-642, Apr. 2011.
[6] ITU-T Recommendation P.914, “Display requirements for 3D video quality assessment,” International Telecommunication Union, Mar. 2016.
[7] ITU-T Recommendation P.915, “Subjective assessment methods for 3D video quality,” International Telecommunication Union, Mar. 2016.
[8] ITU-T Recommendation P.916, “Information and guidelines for assessing and minimizing visual discomfort and visual fatigue from 3D video,” International Telecommunication Union, Mar. 2016.
[9] ITU-R BT.2021-1, “Subjective methods for the assessment of stereoscopic 3DTV systems,” International Telecommunication Union, Feb. 2015.
[10] A. Vetro, I. Sodagar, “The MPEG-DASH Standard for Multimedia Streaming Over the Internet,” Mitsubishi Electric Research Labs, 2011.

Related publications

[11] P. Guzmán, P. Acelas, T. R. Vargas, P. Arce, J. C. Guerri, E. Macías, and A. Suárez, “QoE evaluation and adaptive transport for 3D mobile services,” presented at the 2nd Workshop on Future Internet: Efficiency in High-Speed Networks (W-FIERRO), Cartagena, Murcia, Spain, Jul. 2012.
[12] P. Arce, I. de Fez, F. Fraile, S. González, P. Guzmán, and J. C. Guerri, “QoE en redes adhoc, descarga adaptativa de contenidos y vídeo 3D,” in Proc. of Jornadas de Ingeniería Telemática (JITEL), Palma de Mallorca (Spain), Oct. 2015, pp. 339-346.
[13] W. Castellanos, P. Guzmán, P. Arce, and J. C. Guerri, “Mechanisms for improving the scalable video streaming in mobile ad hoc networks,” in Proc. of ACM Int. Symposium on Performance Evaluation of Wireless Ad Hoc, Sensor, and Ubiquitous Networks (PE-WASUN), Cancun (Mexico), Nov. 2015, pp. 33-40.
[14] S. González, W. Castellanos, P. Guzmán, P. Arce, and J. C. Guerri, “Simulation and Experimental Testbed for adaptive video streaming in ad hoc networks,” in Ad Hoc networks, vol. 52, pp. 89-105, 2016.