HTTP2 summary and simple practice summary

HTTP Development History

Before summarizing http2, let's review the development history of http. The following three pictures are from Jerry Qu

HTTP/0.9 (1991)

HTTP/1.0 (1996)

HTTP/1.1 (1999)

HTTP Communication Process

As we all know, HTTP is an application layer protocol based on TCP, that is, after the TCP connection is established, data is transmitted on the TCP link.

1. First, a TCP connection is established, with three handshakes: C --(SYN{k})--> S, S --(ACK{k+1}&SYN{j})--> C, C --ACK{j+1}--> S
2. After the client sends ACK, it sends an HTTP request
3. The server receives ACK, confirming that the TCP connection is established, and then receives the HTTP request, parses it and returns the result to the client.
4. The client receives the HTTP request result.

In HTTP/0.9 and HTTP/1.0, after step 3, the server will close the connection, which is TCP's four waves. However, after HTTP/1.1, the client can bring Connection: Keep-Alive in the HTTP request header when sending HTTP requests, which tells the server to keep the connection and not close TCP. When Connection: Close is sent, the server will close the connection.

The communication process of HTTP2 is nothing more than this process, but the data transmitted through TCP will be different, and the behavior of the client and server has new rules. The four concepts of Connection, Stream, Message, and Frame are introduced. The relationship between them can be roughly seen from the figure below.

• Connection: Actually it is a TCP connection
• Stream: A bidirectional stream of bytes over an established connection
• Message: request or response, composed of one or more frames
• Frame: A binary frame in a message, the smallest unit of HTTP/2 communication, which will be explained in detail later.

HTTP/2 New Features

• Binary framing layer
• Multiplexing
• One connection per origin
• Stream prioritization
• Header Compression
• Flow control
• Server Push

These new features are mainly to solve previous problems. Let's compare them with the previous HTTP/1.1 to see what problems are solved.

Binary framing layer

Binary framing is to encapsulate http data in a specified format, similar to IP and TCP data packets. A simple Ethernet frame structure that carries HTTP2 data is drawn for easy understanding.

The structure of http2 can be seen by capturing the packet with wireshark

• Length: Unsigned natural number, 24 bits, indicating only the number of bytes occupied by the frame payload, excluding the 9 bytes occupied by the frame header. The default size range is 0~16,384 (2^14). Once the default maximum value of 2^14 (16384) is exceeded, the sender will no longer allow sending unless it receives a notification of the SETTINGS_MAX_FRAME_SIZE value defined by the receiver (usually this value range is 2^14 ~ 2^24).
• Type: 8 bits, defining the specific format of the frame payload and the semantics of the frame. The HTTP/2 specification defines 10 frame types, excluding experimental and extended frame types.
• Flags: 8 bits represent specific frame types, with a default value of 0x0. There is a little trick to note. Generally speaking, 8 bits can hold 8 different flags. For example, the PADDED value is 0x8, which is represented by 00001000 in binary; the END_HEADERS value is 0x4, which is represented by 00000100 in binary; and the END_STREAM value is 0X1, which is 00000001 in binary. Three flags can be conveyed in one byte at the same time, which is represented by 00001101 in binary, or 0x13. Therefore, in the following frame structure, flags are generally represented by 8 bits. If a bit is uncertain, a question mark ? is used instead to indicate that the flag may be set here.
• R: Reserved bit in HTTP/2 context, fixed value is 0X0
• Stream Identifier: Unsigned 31 bits representing an unsigned natural number. A value of 0x0 indicates that the frame only applies to the connection and does not belong to a separate stream.

The types in HTTP2 frames are as follows:

If you want to know the detailed data structure of each type, please refer to my other article http2 frame type detailed explanation

Through a diagram from Google Developers, we can better understand the location of HTTP2 framing in network data and how it differs from HTTP/1.1.

The header in HTTP/1.1 becomes a HEADERS type frame, and the request body/response body becomes a DATA type frame. Through binary framing, the transmitted data is transmitted in binary format, which reduces the amount of data compared to the text format; different types of frames are used to implement flow control, server push and other functions.

Multiplexing & One connection per origin

We know that before HTTP2, if we want to speed up the loading of web resources, we will use the method of establishing multiple connections at the same time, but the efficiency of establishing TCP connections each time is relatively low, and browsers often limit the maximum number of connections (for example, the maximum number of connections for Chrome is 6). In addition, Pipeline was introduced in HTTP/1.1, which allows multiple requests to be sent continuously in one TCP connection without worrying about whether the previous response has arrived, but the server must respond in the order in which the requests are received. In this way, once the previous request is blocked, the subsequent requests will not be able to respond in time.

In HTTP2, the use of new binary frames makes it easy to reuse a single TCP connection. Clients and servers can break HTTP messages into independent frames, send them interleaved, and then reassemble them at the other end.

Still a picture from Google Developers:

You can see that we can send multiple responses and requests in parallel and use the same TCP connection without any order. Each frame carries information about how to assemble them, and the client will wait until all the resources required for a certain task are ready before executing.

Stream prioritization

Since single connection multiplexing is possible, the frames of the server and the client are sent interleaved. For the frames sent to the server, in order to determine which ones should be processed first and which ones should be processed later, the priority of the data stream is introduced, and the server allocates resources according to the priority. For example, higher priority ones get more CPU and bandwidth resources. So how is the priority marked? Remember that there is a Type called PRIORITY in the previous frame type. This type of frame is used to tell the server the priority of this stream. In addition, the HEADERS frame also contains priority information.

HTTP/2 uses parent dependencies and weights to indicate priorities. Each stream is marked with a parent stream id. If it is not marked, it is assumed to be a virtual root stream. In this way, a dependency tree is constructed according to this dependency relationship. The streams in the upper layers of the tree have higher weights. Streams in the same layer will have a weight to distinguish the resource allocation ratio.

The figure above shows some examples of dependency trees, from left to right, a total of four trees.

• The first two streams A and B do not have a parent stream marked. By default, they rely on the virtual root node. A and B are at the same layer and have the same priority. Resources are allocated based on their weights. A gets 12/(12+4)=3/4 of the resources, and B gets 1/4 of the resources.
• Second, D and C have a hierarchical structure, and C's parent is D. Then the server takes the complete resources to process D first, and then processes C.
• Third, the server processes D first, then C, and then A and B. A gets 3/4 of the resources and B gets 1/4 of the resources.
• Fourth, deal with D first, then split the resources in half to deal with E and C, and then deal with A and B according to the weights.

One thing to note is that the stream priority is not a mandatory constraint. When a high-priority stream is blocked, it does not prevent the server from processing low-priority streams.

Header Compression

As the current website content is becoming more and more complex, the number of requests for a single page is basically dozens or even hundreds. Each request must carry the client or user's identification, such as: UA, cookie and other header data. As the number of requests increases, the traffic consumed by transmitting the http header is also very considerable, and most of the header data is the same, which is a naked waste. Therefore, header compression technology was created to save traffic.

• Maintain a static table containing common header names and particularly common header name and value combinations
• Maintain a dynamic table to add content dynamically
• Support Huffman Coding based on static Huffman code table

Static dictionary

The static dictionary maps the commonly used headers to shorter byte index numbers, as shown in the following figure, which intercepts the first few mappings. For all definitions, see Static Table Definition

For example, when there is a field in the header: method: GET, then we can look up the table and see that it can be identified by the serial number 2, so the data of this field is 0000010 (binary representation of 2)

Dynamic dictionary

After all, the header data that a static dictionary can represent is limited, and the compression rate is not high. However, for a site, when interacting with a certain user, a lot of requests will occur, but the headers of each request are not much different, and there will be a lot of repeated data, because the user and browser identifiers are unchanged. Then we can generate a dynamic dictionary that can add mappings for a HTTP2 connection, so that the serial number in the dynamic dictionary can be used in subsequent connections. The process of generating a dynamic dictionary is actually to notify the other party to add a mapping. The client can notify the server to add, and vice versa.

The specific notification method is to transmit data in the format specified by the protocol.

Huffman Coding

The characteristic of Huffman coding is that the higher the frequency of occurrence, the shorter the coding length. The HTTP2 protocol generates a canonical Huffman code based on a large number of request header data samples, which are listed in Huffman Code.

Flow control

The goal of HTTP/2 flow control is to give the receiving end full authority to control the amount of traffic it wants to receive at the moment, subject to the constraints of the initial value of the flow window.

algorithm:

• Both ends (sending and receiving) maintain an initial value of the flow control window.
• Each time the sender sends a DATA frame, it decrements the window by the size of the frame. If the window is smaller than the frame size, the frame must be split. If the window is equal to 0, no frame can be sent.
• The receiving end can send a WINDOW_UPDATE frame to the sending end, and the sending end uses the Window Size Increment specified in the frame as the increment and adds it to the window.

Server Push

process:

• When exchanging SETTINGS frames, the client sets the field SETTINGS_ENABLE_PUSH (0x2) to 1 to explicitly allow server push.
• When the server receives the request, it analyzes the resources to be pushed and first sends a PUSH_PROMISE frame to the browser.
• Then send each response header and response body
• When the browser receives the PUSH_PROMISE frame, it can determine whether it is currently cached based on the URL in the header block fragment field, and thus decide whether to receive it. If not, the browser will send a RST_STREAM to terminate the server push.

question:

• Traffic waste. If the browser has a cache and does not want this push, there will be a waste of traffic, because the whole process is asynchronous. When the server receives the RST_STREAM, the response is likely to be partially or completely sent.

HTTP/2 Simple Practice

Okhttp is a well-known http client in the Java ecosystem. It is easy to use, has good performance, and supports http2. Let's use this tool to practice. Since my blog has already configured http2 on nginx, I will use this blog to experiment.

 public class Http2Example {
    final static OkHttpClient client = new OkHttpClient.Builder().build();
 public   static void main(String[] args) {
        Request request = new Request.Builder()
                .url( "https://blog.fliaping.com" )
 .build();
 try {
            Response response = client.newCall(request) .execute ();
            System. out .println(JSON.toJSONString(response.protocol()));
            System. out .println(response.headers().toString());
            System. out .println(response.body().string());
        } catch (IOException e) {
            e.printStackTrace();
 }
 }
 }

Those who have used Okhttp will find that this is the same as the usual method, there is no difference, yes, there is no difference. Without saying anything else, run it and see, unfortunately you will find that the protocol is still http1.1, not h2, what's the matter? This is because HTTP2 has newly added ALPN (Application Layer Protocol Negotiation), which literally means application layer protocol negotiation, that is, the two parties discuss which protocol to use. Unfortunately, jdk8 was released in 2014, when the HTTP2 protocol was not yet born, but fortunately, ALPN can be supported through a third-party jar package. In addition, jdk9 already supports HTTP2. Although it has not been officially released, we can try JDK 9 Early-Access Builds.

JDK7 and JDK8 add third-party alpn support packages by adding jvm parameters. Note that the versions cannot be wrong. JDK7 uses alpn-boot-7.*.jar and JDK8 uses alpn-boot-8.*.jar. Here is the version correspondence alpn-versions

 # jdk8
 -Xbootclasspath/p:/home/payne/Downloads/alpn-boot-8.1.11.v20170118.jar
 # jdk7
 -Xbootclasspath/p:/home/payne/Downloads/alpn-boot-7.1.3.v20150130.jar
 # jdk9
 # When using the jdk9 platform, please note that the okhttp version is greater than 3.3.0  
 # https://mvnrepository.com/artifact/org.mortbay.jetty.alpn/alpn-boot