Creating the "3D Panoramic Tour" Secret in H5

[[165029]]

VR virtual reality technology has been booming recently. I took some time to think about the "Planet Project" project completed at the beginning of the year. In this summary article, I would like to share with you the secrets of making a 3D panoramic tour based on Html5.

[[165030]]

QQ IoT cooperates with Shenzhen Astronomical Observatory. In the "Discover New Devices"-"Public Devices" of QQ Mobile, users can connect to QQ IoT cameras to provide live broadcasts of major celestial events in 2016. Through QQ Mobile, users can watch celestial phenomena such as solar eclipses and meteors from the best observation points around the world in real time. As the operation page that carries the entire "Planet Plan" activity, after much discussion, we decided to try the H5 operation page with 3D panoramic roaming mode for promotion. Today, we will not elaborate on the specific content of the activity, but first talk to you about the production method of "3D panoramic roaming" in this H5.

Here is an experience link (please ignore the lag and fonts in GIF recording, and turn on the gyroscope on iOS to experience it***). The universe part of Page 3 - turning the phone to search for planets in the simulated universe is what we are going to talk about today, the 3D panoramic tour based on Html5.

[[165032]]

To make a panoramic tour, you must first have a panoramic image. Panoramic images are usually obtained by shooting with a fisheye panoramic camera, or a combination of a SLR camera, a fisheye lens, a gimbal, and a tripod. You need to take a set of photos by rotating 360 degrees in one direction, and there must be some overlapping parts between the photos to facilitate later stitching and fusion. After taking the photos, you need to seamlessly stitch them together. The generated panoramic images can be divided into spherical panoramas, cubic panoramas, and cylindrical panoramas. (The street view experience of our Tencent Map is the most common panoramic tour technology)

Even without a panoramic camera, we can still get a panoramic image suitable for our project through some material sites. For example: a material site

Of course, the background image of the planetary plan is the universe and the stars, which is relatively disordered, so it is also possible to rely on visual designers to splice and draw it.

What is panoramic roaming? Panoramic roaming technology allows users to switch perspectives in a panoramic space constructed by panoramic images. It is achieved by shooting panoramic images and then using computer graphics technology to construct a panoramic space, allowing users to control the direction of browsing, viewing objects or scenes from left or right, up or down, as if they were there. Compared with traditional 3D modeling, panoramic roaming technology is simple to make, has a small amount of data, low system consumption, and is more realistic. Therefore, in recent years, it has also been a popular implementation method of VR technology. Let's use the previous mapping example for a demo. Panoramic roaming on mobile terminals can even be bound to a gyroscope to make you feel more immersive.

[[165034]]

At the beginning of the project, some panoramic tour production solutions were pre-researched, including the most commonly used panoramic tour production tools Pano2vr & Krpano, as well as Flash, QuickTime, Java, js and other methods to produce panoramic tours. However, according to the preliminary research, the advantages and disadvantages of each solution are compared as shown in the figure below. Combined with the specific situation of the "Planet Project" project, the Threejs implementation solution was finally selected as the priority.

Here I would like to talk to you about the most commonly used panoramic tour production tools: Pano2vr & Krpano.

(1) Pano2vr is easy to operate and has few functions, but it is very practical. You can directly produce flash, html5, Quicktime and other formats by "importing panorama - setting interactive hotspots - fine-tuning - exporting".

Pano2vr is very suitable and fast for use only on PC and iOS, but its support for Android devices is poor.

(2) Krpano, with powerful and comprehensive functions, high compatibility with various platforms, strong scalability, and can support various VR scene special effects. However, it is a self-contained system that needs to follow the Krpano xml programming language. It does not have a gui software interface, and the cost of getting started and subsequent maintenance is high for novices. The generated panoramic roaming is professional, but it has an impact on loading speed and memory usage. However, for those who want to do high-level, personalized, and customized panoramic roaming projects, Krpano is the best choice.

However, both tools require the purchase of authorization codes for commercial use and are not free.

(3) Three.js is an open source project from Github, https://github.com/mrdoob/three.js, official website: http://threejs.org/.

I saw an explanation from a colleague, saying that it can be understood as Three + js. Three means 3D, js means javascript, so three.js means using javascript to write 3D programs, which is very straightforward and clear. Relying on the javascript language to write, naturally brings this solution the advantages of high scalability, high compatibility, low development cost, high performance and free.

(4) Flash, QuickTime, other methods based on Java, js, etc. are not described in detail here. Please refer to the table above for a rough comparison of their advantages and disadvantages (the specific scores are for reference only, and software version updates may have upgrades in various aspects).

[[165035]]

To use Three.js to render an object to a web page, you need to build these three components: scene, camera, and renderer:

(1) Scene

The canvas is the container for all objects. Instantiate the scene at the beginning and add the objects you build later to the scene.

(2) Camera

Users use cameras to view 3D scenes in the scene. Three.js includes two types of cameras: orthographic cameras and perspective cameras. From the perspective of simulating the way human eyes see objects, perspective cameras are more suitable. As shown in the figure below, fov is the angle of the camera's view, aspect is equal to the camera's frame ratio, near and far are the closest and farthest distances from the camera to the view volume, both are positive values, and far should be greater than near.

(3) Renderer

The renderer is used to set the elements on the page where the rendering results will be presented and the rendering rules.

In Three.js, the scene is a container that places the stars of our planet plan in different positions in the constructed 3D scene; the camera shoots the scene, and the shooting results are drawn in real time on our browser through the renderer.

(4) Build the universe and place it in the scene

After defining these three elements, the next step is to construct the universe required for our planetary plan. As mentioned earlier, panoramic images can be divided into spherical panoramas, cubic panoramas, and cylindrical panoramas.

Although spherical panoramas have the closest construction mode to human eyes, they require many facades to build a sphere. The longitude and latitude coordinates of the sphere cannot be unfolded into a flat map. Compared with other solutions, the performance consumption is too high and the stitching method is too complicated. The vertical field of view of cylindrical panoramas is small, and it is not easy to make top and bottom pitch perspectives. We selected the most common cubic panorama to build our 3D scene.

The cube panorama has 6 faces. We need to define the background image, 3D position, and rotation angle of each face texture (the default 6 faces are facing us. We need to define a 90-degree rotation in each direction of the coordinate axis to build a cube).

Then add it to the THREE.Object3D array, so that we can build a 3D universe in the scene.

Here, considering that the starry sky background is mainly for atmosphere setting, we reduce the 6-surface textures to 2, thereby reducing the size of the resource file, as shown in the figure below.

(5) Rendering

Here we use the real-time rendering of Threejs: it needs to render the picture continuously, even if nothing has changed in the picture, it needs to be re-rendered. One of the important functions is requestAnimationFrame, which asks the browser to execute the function in the parameter once. In this way, by calling the requestAnimationFrame() function in the above animate, the requestAnimationFrame() function asks animate() to execute again, forming what we usually call a rendering loop.

Through the above steps, we have built this 3D universe.

(6) Build a planet and place it in the universe

In the first phase of the planet plan, 8 planets need to be added. In order to avoid the screen being too crowded, the planets are scattered on 6 planes. When we defined the six planes of the universe before, we gave each plane a fixed section id. Through simple js, we can add the DOM structure of the planet to the plane.

Therefore, you can also easily use CSS to position these planets and add personalized animation effects. Here, you must pay special attention not to add animation effects such as shadows that consume a lot of memory, as they are the culprits of crashes.

'

(7) Bind the gyroscope

The last step is to bind the panoramic roaming to the gyroscope. This involves the need to make a protection code for the gyroscope event to determine whether the device supports the gyroscope. After completing the above steps, you can realize a panoramic roaming on the mobile terminal.

(8) Others

In the early stage of the project, the memory consumption of some Android devices was still too large. For this reason, we continued to try some optimization work after completing the project, including reducing the size of the universe, merging panoramic maps, disabling gyroscopes, preloading and lazy loading, planet CSS3 animation reduction, deep compression of resource files, etc. However, the risk of crash on Android devices with insufficient memory cannot be avoided. In order to ensure the stable launch of the project, we made a compatible version experience for Android devices as a second choice. It is expected that the performance of the page on Android will be optimized in subsequent project iterations to achieve a unified experience on all platforms.

***, this article only summarizes the trial of building 3D panoramic roaming on mobile terminals. This attempt can basically meet the needs of the project, but it still needs to be polished in terms of performance optimization and detail improvement. I hope it can bring some help to interested partners^^.