Introduction to Cloud Desktop Transmission Protocol

1. Definition

As shown in the figure, the cloud desktop protocol is a communication protocol used to transmit graphics, audio, video, input information devices, peripherals and other data between the server and the terminal in the cloud computer scenario.

2. Introduction to mainstream protocols

There are currently four mainstream cloud desktop protocols, all based on TCP or UDP protocols, namely ICA (HDX), PCoIP (Blast), RDP, and SPICE.

➤ ICA protocol: The cloud desktop protocol can be traced back to 1989, when Citrix was founded. The ICA protocol was born and is still confidential. It can be imagined that when Linux and Java languages ​​were not yet born, Citrix began to develop virtualization software. ICA has developed from the narrowband era, and its stability and reliability are indeed due to its reasons. HDX, which appeared later, is aimed at higher network bandwidth. On the basis of ICA saving traffic, it is committed to delivering rich media and other forms of experience, making the user experience of virtual desktops close to that of physical PCs. Therefore, under the current situation of broadband speed-up, the overall technical solution of HDX has more room to play.

➤ PCoIP (PC over IP) protocol: PCoIP is a remote display protocol developed by Canadian company Teradici. In 2008, VMware announced that it would jointly develop the PCoIP protocol with Teradici to improve the performance of desktop virtualization. In 2009, VMware obtained authorization to release the VMware Horizon View product. Later, in order to get rid of the dilemma of relying on other companies for core technology, VMware tried to develop its own protocol Blast, which was officially released in 2013.

➤ RDP protocol: A proprietary protocol developed by Microsoft that provides users with a graphical interface for connecting to another computer over a network. It can be used in most Windows operating systems, and of course Linux and Mac systems also provide corresponding support.

➤ SPICE protocol: SPICE is an open source protocol developed by Redhat. Currently, most of the desktop protocols developed by major domestic companies are deeply modified and optimized based on this protocol.

3. Protocol Analysis

The main function of the cloud desktop protocol is to transmit the keyboard and mouse operation instructions of the local terminal to the cloud. After receiving the instructions, the cloud computer will perform calculations in response. After the desktop changes accordingly, the entire screen is pushed to the local terminal for display and playback, making the user feel like they are using a local computer. The entire process is to move the computing power to the cloud through the cloud desktop protocol, making the terminal significantly slimmer and more lightweight. As shown in the following figure:

The cloud desktop protocol itself can be subdivided into multiple channels. Different channels carry different data and correspond to different functions. Take open source SPICE as an example, as shown in the following figure:

It is mainly divided into the following transmission channels:

• Main Channel: handles connections and disconnections with clients;
• Inputs Channel: Processes data from input devices such as mouse, keyboard, and touchpad;
• Display Channel: handles desktop image transmission data;
• Cursor Channel: handles the transmission data displayed by the mouse pointer;
• Playback Channel: processes the host sound data of GuestOS;
• Record Channel: processes the recording data on the terminal side;

As shown in the figure, the cloud desktop is connected through the spice client of Windows;

The figure shows the connection status of the spice port. It can be seen that after a client is connected, 6 socket connections will be established on the server, corresponding to 6 channels respectively. The specific packet capture data of each channel is as follows:

1️⃣ Main Channel

The Main Channel is mainly used to process operation instructions such as connection and disconnection with the client. When the client starts, it sends a connection authentication request through this channel to establish communication with the server. Subsequent network quality monitoring, heartbeat keep-alive, client session management, etc. are all processed through this channel.

Protocol Data 👇

The following figure shows the UUID data of the Windows host.

The following figure shows the Windows host server name, which is reflected in the spice main channel as follows:

As shown in the figure, the packet capture data of the main channel is combined with the business, and the entire desktop connection authentication process can be associated and processed through data such as UUID.

2️⃣ Inputs Channel

Inputs Channel is mainly used to process data from input devices such as mouse, keyboard, touchpad, etc.

Protocol Data 👇

The figure shows the mouse position message type defined in the official input channel protocol, which is 112, corresponding to hexadecimal 0x70;

The figure shows the packet capture data of the input channel. The client sends the current xy coordinate information of the mouse to the server through the desktop protocol.

3️⃣ Display Channel

Display Channel is mainly used to process desktop image transmission data;

Protocol Data 👇

The following figure shows the display stream data defined in the official display channel, which is 123, and the hexadecimal value is 0x7b;

The figure shows the packet capture data of the display channel. The stream data format is JPEG, the size is about 193KB, and the bandwidth occupied is about 45Mbps at 30FPS. It can be seen that there is a lot of room for optimization.

4️⃣ Cursor Channel

Cursor Channel is mainly used to process the transmission data displayed by the mouse pointer;

Protocol Data 👇

The figure shows the definition of the mouse display channel. The official description "The relevant remote display area for a cursor channel is the one of the display channel that has the same channel id (ie, SpiceLinkMess.channel_id)" indicates that the mouse display is included in the display channel data; initialized to 111, corresponding to hexadecimal 06x5;

The figure shows the packet capture data, message type = 101, which is equivalent to the definition of SPICE_MSG_DISPLAY_MODE in the display channel;

5️⃣ Playback Channel

Mainly used to process the host sound data of GuestOS and transmit it back to the end side for playback;

Protocol Data 👇

The figure shows the data definition in the audio channel, which is 101, and the hexadecimal is 0x65;

The figure shows the packet capture data, message type=101, data is the specific sound data;

6️⃣ Record Channel

Mainly used to process the recording data on the terminal side and transmit it to the cloud;

Protocol Data 👇

The picture shows the recording channel data;

The specific definition format of all the above channel protocol data can be referred to the official website definition:​ ​https://www.spice-space.org/spice-protocol.html

4. Summary

After reading the original SPICE protocol data and preliminary tests, it can be found that there is still a lot of room for optimization in the current end-to-end transmission performance. When the frame rate is less than 20FPS, it occupies about 45Mbps of transmission bandwidth, and it cannot present a continuous high-quality picture quality experience, which is basically not directly commercially available. During the entire desktop channel data transmission process, the data flow of the display channel basically accounts for more than 90%, which is the focus of optimization; Looking at the four major desktop protocols of ICA, PCoIP, RDP, and SPICE, ICA, which has developed from the narrowband era, provides users with a better cloud desktop experience with its high-quality protocol algorithm and up to 32 channels. The definition, but because its overall closed-source protocol and high licensing fees, for most cloud desktop manufacturers, cooperating with it for research and development is not a long-term solution, so self-developed desktop protocols have become the key to the competition and differentiation of many domestic manufacturers. At present, China Mobile Smart Home Operation Center has a certain amount of technical accumulation in the fields of audio and video encoding and decoding, GPU hardware acceleration, network transmission, etc. related to desktop protocols, and has formed a corresponding scale of implementation practice solutions based on SPICE end-to-end solutions, and has achieved certain results. In the future, China Mobile's Smart Home Operation Center will further focus on the three key goals of improving frame rate, reducing latency, and low-code high definition, and is committed to providing users with a better and smoother product experience.

With the popularization of 5G+gigabit broadband, with the support of high-speed + low-latency basic technologies, combined with the increasingly mature cloud computing base, in the future, the competition for cloud desktops in the 2C/2H field will become increasingly fierce.