Overview
6 degrees of freedom (DoF) is used in virtual reality (VR) applications to improve the user experience compared to standard 3 DoF solutions. Due to its sparse nature, 6 DoF information is usually represented in point cloud form. Each element describes the position of a point in 3D space and its attributes (such as color and transparency). 6 DoF improves the user experience, but requires a lot of data compared to 3 DoF, making content delivery difficult and limiting applications to high-end purpose-built machines. The aim of our work was to design a novel point cloud compression scheme that would allow 6DoF VR applications to run in real time on high-end consumer devices such as gaming laptops and desktop machines. Our solution is designed specifically for the PresenZ 6 DOF VR movie format, but can be easily applied to other volumetric video formats.
prologue
A typical virtual reality (VR) scenario uses degrees of freedom (DoF) to track the movement of a user wearing a headset in three-dimensional (3D) space and adjust the image the user sees accordingly. adjust. A 3 DoF application only tracks rotational motions (pitch, yaw, roll) around the x, y, z axes, while a 6 DoF application also tracks translational motions (surging, sway, heaving) and adds movement, etc. enable the effect. Back and forth, left and right, up and down. In addition to improving the user experience, 6DoF VR helps reduce motion sickness and disorientation by providing a greater sense of presence.
Due to its sparse nature, 6 DoF information is usually represented in point cloud form. Each element represents a point’s 3D position, color, transparency, orientation, and movement. It may also contain additional data, such as information about the camera used to capture the 3D view. The actual number of points depends on the complexity of the visual scene. A typical frame may consist of over 5 million points.
6 DoF improves the user experience, but requires a lot of data compared to 3 DoF, making content delivery difficult and limiting applications to high-end purpose-built machines. The main challenges that need to be addressed are 1) high data entropy, which typically exceeds the capacity of traditional communication channels such as 500 MB/s solid-state drives (SSDs), and 2) real-time video rendering. Requirements at a relatively high frame rate (30 fps). In this work, we describe an approach to address the above challenges using a new data compression scheme designed specifically for point cloud datasets.
Our data compression format describes each frame individually and consists of a fixed header layer and some optional data layers. The fixed header layer describes basic information such as the number of points and the color space used, as well as the types of coding tools and techniques used for different point cloud subgroups and their attributes. Depending on the information contained in the fixed header, additional header layers may be present in the bitstream to further describe encoding methods, parameters and metadata. Finally, an additional core layer is used to store the encoded value of each attribute.
Also designed and implemented the codec API. This allows you to encode a series of point cloud frames and decode them in real time on high-end laptops and gaming desktop machines. The actual encoder and decoder implementations were developed in C++ using techniques such as multithreading and IntelTM Single-Instruction-Multiple-Data (SIMD) intrinsics.
This white paper provides background research on point cloud compression and VR applications, followed by details of the approach, experimental results, conclusions, and possible future developments.
Comments
Post a Comment