The fifth issue of the Aiti Tribe live broadcast class: How to use the Jiegou SDK to instantly build a real-time voice and video APP

Online video live streaming has existed for a long time. With the increase of mobile uplink and downlink bandwidth and the reduction of tariffs, video live streaming has been given more entertainment and social attributes. People enjoy live streaming and watching anytime and anywhere. Live streaming has been widely used in voice social networking, video social networking, interactive live streaming, and claw machine live streaming.

In the current live broadcast field, claw machines have become the new era's favorite. How to push two streams at the same time and low latency are the key points that everyone wants to know. So, how to use the iQiyi SDK to instantly build a claw machine video APP? We specially invited Mr. Lu Aihui, senior front-end technical support supervisor of iQiyi Technology, to share his experience in the live broadcast.

Contents:

1. Introduction to the entire architecture of iQiyi Voice and Video

1) Application scenario introduction

2) Architecture Implementation

3) Architecture diagram description

2. How to access the iQiyi Voice and Video SDK (for claw machine scenarios)

1) Claw machine control terminal interface call

2) Claw machine APP interface call

Hello everyone, the topic of this sharing is the recently popular live broadcast topic, how to use the iQiyi SDK to instantly build a real-time voice and video APP.

Here is a brief introduction to the background of our company, as shown in the screenshot below.

As we all know, the field of live broadcasting is very broad, and the application scenarios are also very wide, such as voice social networking, video social networking, interactive live broadcasting, and the recently popular claw machine scenario. Here I will mainly share with you how to use zegoSDK to make a live broadcast of a claw machine.

1. Introduction to the entire architecture of iQiyi Voice and Video

1) Scenario Introduction

Let's first look at the offline deployment of Play Dolls:

Two cameras are facing a claw machine, and the claw machine is pushed in two directions. In this way, the operator can see the claw machine from two angles.

Example of the effect produced by the user-side APP:

Support: 2 viewing angles, 100 milliseconds ultra-low latency, real-time synchronization of video and overhead crane, so that players can experience a smooth operation, and the two viewing angles will not make the picture too flat.

2) Architecture Implementation

Next, let me tell you about the architecture implementation of this live broadcast scenario. There are two sides to explain:
One end is the claw machine. There is a chip on the claw machine. This chip is an Android system chip, which can be understood as a small Android phone.

From the above claw machine scene, we can see that there are two cameras next to the claw machine, which means that the claw machine needs to push out two video streams.

The other end is the user APP end, which can be a mobile end or a PC end.

User APP: The user needs to pull two streams at the same time to watch. At the beginning, one view is hidden first, and everyone sees one picture. When clicking to switch the camera, it switches to another view to see another perspective.

3) Architecture diagram description

This architecture diagram means that the claw machine pushes two streams, first to zegoserver, and then through the distribution network to ordinary players for viewing. The delay at this time is about 1-2 seconds.
On the user side, when a player wants to go online to catch dolls, he goes online and zegoSDK will switch to zegoServer to pull the stream to achieve low latency, which can reach 100-300ms.

The following is the flow of the entire architecture diagram.

2. How to access the iQiyi Voice and Video SDK (for claw machine scenarios)

Next, we will explain how to use zegoSDK to make an APP, which involves calling the SDK interface level.

1) Claw machine control terminal interface call

First initialize the SDK and log in to the room:

Push two streams at the same time:

As you can see, when pushing the first stream, enableMic is called and set to FALSE. This is because the claw machine does not need to push the audio out, so the microphone is turned off.

The second stream does not push audio by default, so there is no need to continue calling this interface.

2) Claw machine APP interface call

After pushing the two streams, the client can come in to watch. The client also needs to initialize the SDK and log in to the room first.

Once you enter this room, you can start watching

After entering the room, some users want to grab the dolls on the machine, and the interface is called:

After getting on the machine, you need to start controlling the crane of the claw machine, move it left and right, front and back, and then start grabbing. This signaling is implemented through our interface sendCustomCommand.

After the player grabs the doll on the machine, no matter whether we succeed or not, we have to get off the machine. The interface call for getting off the machine is as follows:

After logging off, if you want to continue logging on, you can repeat the above login interface process. If the player does not want to continue watching after logging off, he can end the viewing and exit the room.

3) Claw machine control terminal interface call

The user end sends some custom messages to the claw machine end. After receiving them, the claw machine end converts them into serial port commands that can be recognized by the claw machine on the chip. The interface of these commands received by the claw machine end is onReceiveCustomCommand.

After talking about the interface call on the APP side, we still lack an end to the push stream on the claw machine side. The SDK interface call is as follows. Because the claw machine pushes two channels, both channels must be ended at the end, and finally exit the room.

Let's summarize the main process of calling:

Claw machine side:
1. Initialize SDK
2. Log in to the room
3. Start live streaming and push the first stream
4. Start live streaming and push the second stream
5. Receive instructions from the APP
6. End the live broadcast and exit the room

User side:
1. Initialize SDK
2. Log in to the room
3. Stream and watch
4. Operate on the machine and upgrade to a player
5. Get off the plane and become a spectator
6. End viewing and exit the room

The above is the call of the entire SDK interface, that is, the claw machine solution. We can download Zego's Demo for free to try it out.

iOS Version >>
Android Version >>
At the same time, you can also visit our official website for the specific scheme introduction and flow chart of the claw machine:
https://www.zego.im/html/solution/wawaji.html
https://www.zego.im/html/document/#Application_Scenes/WaWaJiClient/Overview

The following questions are from the 51CTO developer community:

Q: Java-Eason-Fujian: rtmp streaming?

A: Mr. Lu from Zego Technology: We push the stream through zegoserver first, so we push the stream through our udp private protocol first, and then when we forward it to the distribution network, we push the stream through rtmp.

Q: Java-Eason-Fujian: Is what he said right?

A: Mr. Lu from Zego Technology: The specific type of chip depends on the manufacturer. The chip is installed with the Android system, and an APP for controlling the overhead crane is installed on the Android system. This APP is also used for streaming.

Q: Java-Eason-Fujian: The claw machine and the two cameras are connected together, right?

A: Mr. Lu from Zhigou Technology: Yes, they are connected through the serial port on the chip.

Q: Java-Eason-Fujian: It should be based on the same principle as Alibaba Cloud’s live video streaming.

A: Mr. Lu from ZEGO Technology: Alibaba Cloud has live video streaming, but it does not have a microphone connection, and their latency cannot achieve as low a latency as ZEGO.

Q: Data-unicorn-Beijing: The SDK is live broadcast + interactive, right? Does it support video conferencing? Can there only be one anchor? Does it support multiple anchors?
A: Mr. Lu from Zego Technology: It does support video conferencing, and the same SDK is used for video conferencing to achieve low latency. Video conferencing is actually multiple anchors interacting in a room to push and pull streams, using ZegoServer to push and pull streams, so low latency is achieved. Therefore, multiple anchors are OK.

Q: Data-unicorn-Beijing: What is the charging standard? For example, for video conferencing, how many terminals can be online at the same time?

A: Mr. Lu from Jigou Technology: For the charging standards, we can contact our business colleagues on the official website for specific consultation. Video conferencing supports 12 terminals to chat online at the same time.

Q: PHP-Coeus-Anhui: Is there an H5 SDK? I saw that there is an H5 SDK, but I didn't find the documentation or the H5 SDK.

A: Mr. Lu from Jigou Technology: The document is in the link I sent you. I will send it to you:
https://www.zego.im/html/document/#Application_Scenes/WaWaJiClient This link contains demo downloads and SDK downloads.

Regarding the H5 SDK, we already have a plan and the finished product will be available at the end of this month and can be used directly on H5.

Q: java-Eason-Fujian: What should I do if I want to push the stream on the PC? Does it have to run on the APP? The scenarios are different.

A: Mr. Lu from Jigou Technology: We also have SDK for PC. The PC provides a C++ interface. You can develop a client based on the SDK. For PC, it is like connecting multiple cameras to the USB of a computer. It is similar in nature. But for PC, the cost may be slightly higher, because the cost of an Android board is definitely cheaper than that of a computer, right?

I used iOS as an example above. On iOS, we provide OC interface, on Android we provide Java interface, and on PC we provide C++ interface.

Q: java-Eason-Fujian: Is the switch for controlling streaming on the claw machine turned on when the claw machine is started, or does it start streaming when a customer connects?

A: Mr. Lu from Jigou Technology: The traffic should be pushed when the claw machine is turned on, because you don’t know when the customers will come in, so it should be turned on from the beginning.

Q: Network-Xiao Wang-Guangzhou: If I connect to an APP, how much space will the base package take up?

A: Mr. Lu from Jigou Technology: The size of the SDK is about 7M for Android and about 10M for iOS dynamic library. For iOS, the packaged ipa is about 6M. We are still optimizing the size of the SDK.

Q: Data-unicorn-Beijing: Does video conferencing support private cloud deployment?

A: Mr. Lu from Zhegou Technology: Yes. For those with higher security, we can consider private deployment, and you can just provide the corresponding machine cluster.

[51CTO original article, please indicate the original author and source as 51CTO.com when reprinting on partner sites]