HTTPS for the entire site is here

Recently, when you use Baidu, Google or Taobao, have you noticed that a green lock has appeared in the upper left corner of the browser? This lock indicates that the website has been protected by HTTPS. If you observe carefully, you will find that these websites have already used HTTPS for the entire site. At the same time, the iOS 9 system changes all http requests to HTTPS requests by default. With the development of the Internet, the modern Internet is gradually entering the era of full-site HTTPS.

Therefore, a developer asked in an email to Tencent Bugly: What advantages can full-site HTTPS bring? What is the principle of HTTPS? At the same time, what are the difficulties that hinder the popularization of HTTPS? In order to answer everyone's confusion, Tencent Bugly specially invited our senior engineer Liu Qiang to comprehensively refer to various materials and verify them in practice, explore the basic principles of HTTPS, analyze the basic HTTPS communication process, and welcome the arrival of full-site HTTPS.

1. HTTPS Basics

HTTPS (Secure Hypertext Transfer Protocol) is a secure communication channel developed based on HTTP for exchanging information between client computers and servers. It uses Secure Sockets Layer (SSL) for information exchange. In simple terms, it is a secure version of HTTP, which is an HTTP protocol encrypted using TLS/SSL.

The HTTP protocol uses plain text to transmit information, which poses the risk of information eavesdropping, information tampering, and information hijacking. The TLS/SSL protocol has the functions of identity authentication, information encryption, and integrity verification, which can avoid such problems.

TLS/SSL stands for Transport Layer Security, which is a security protocol between TCP and HTTP. It does not affect the original TCP and HTTP protocols, so using HTTPS basically does not require much modification of HTTP pages.

2. TLS/SSL principles

The main functions of the HTTPS protocol basically rely on the TLS/SSL protocol. This section analyzes the implementation principles of the security protocol.

The functional implementation of TLS/SSL mainly relies on three basic algorithms: hash function, symmetric encryption and asymmetric encryption. Asymmetric encryption is used to implement identity authentication and key negotiation. Symmetric encryption algorithm uses the negotiated key to encrypt data and verifies the integrity of information based on hash function.

Hash functions, such as MD5, SHA1, and SHA256, are one-way irreversible, very sensitive to input, and have a fixed output length. Any modification to the data will change the result of the hash function, which is used to prevent information tampering and verify data integrity. Symmetric encryption, such as AES-CBC, DES, 3DES, and AES-GCM, uses the same key to encrypt and decrypt information. Only by mastering the key can information be obtained, which can prevent information eavesdropping. The communication method is one-to-one. Asymmetric encryption, namely the common RSA algorithm, also includes ECC, DH, and other algorithms. The algorithm features that the keys appear in pairs, generally called public keys (public) and private keys (confidential). Information encrypted by public keys can only be decrypted by private keys, and information encrypted by private keys can only be decrypted by public keys. Therefore, different clients that hold public keys cannot decrypt information from each other, and can only communicate encrypted with the server that holds the private key. The server can achieve one-to-many communication, and the client can also use it to verify the identity of the server that holds the private key.

During information transmission, hash functions cannot achieve information tamper-proofing alone, because in plain text transmission, middlemen can modify the information and recalculate the information digest, so the transmitted information and information digest need to be encrypted; the advantage of symmetric encryption is that information transmission is one-to-one, and the same password needs to be shared. The security of the password is the basis for ensuring information security. When the server communicates with N clients, it needs to maintain N password records, and there is no mechanism to change the password; the characteristic of asymmetric encryption is that information transmission is one-to-many, and the server only needs to maintain one private key to communicate encrypted with multiple clients, but the information sent by the server can be decrypted by all clients, and the algorithm is complex to calculate and the encryption speed is slow.

Combining the characteristics of the three types of algorithms, the basic working method of TLS is that the client uses asymmetric encryption to communicate with the server, implement identity authentication and negotiate the key used for symmetric encryption, and then the symmetric encryption algorithm uses the negotiated key to encrypt the information and information summary for communication. Different nodes use different symmetric keys, thereby ensuring that the information can only be obtained by the communicating parties.

3.PKI system

3.1 Risks of RSA Authentication

Authentication and key negotiation are the basic functions of TLS, which require that the legitimate server holds the corresponding private key. However, the RSA algorithm cannot ensure the legitimacy of the server's identity because the public key does not contain the server's information, which poses a security risk:

Client C and server S communicate, and the middle node M intercepts the communication between the two;

Node M calculates and generates a pair of public key pubM and private key priM;

When C requests the public key from S, M sends his public key pub_M to C;

The data encrypted by C using the public key pubM can be decrypted by M because M has the corresponding private key priM, and C cannot determine the identity of the server based on the public key information, thus establishing a "trusted" encrypted connection between C and M;

A legal connection is then established between the intermediate node M and the server S, so the communication between C and S is completely controlled by M, who can eavesdrop on and tamper with the information.

In addition, the server can also deny the information it has sent and not acknowledge that the relevant information was sent by itself.

Therefore, there are at least two types of problems under this scheme: man-in-the-middle attack and information denial.

3.2 Authentication - CA and Certificates

The key to solving the above identity authentication problem is to ensure that the public key is obtained in a legal way and can verify the identity information of the server. For this purpose, an authoritative third-party CA is required. CA is responsible for verifying the information of the owner of the public key and issuing the authentication "certificate". At the same time, it can provide certificate verification services to users, namely the PKI system.

The basic principle is that the CA is responsible for reviewing the information, and then "signing" the key information with the private key, and publishing the corresponding public key. The client can use the public key to verify the signature. The CA can also revoke the issued certificate. The basic methods include two types of CRL files and OCSP. The specific process of CA use is as follows:

a. The service provider S submits the public key, organizational information, personal information (domain name) and other information to the third-party CA and applies for certification;

b. CA verifies the authenticity of the information provided by the applicant through various means, including online and offline means, such as whether the organization exists, whether the enterprise is legal, and whether it owns the domain name;

c. If the information is reviewed and approved, CA will issue a certification document - certificate to the applicant.

The certificate contains the following information: the applicant's public key, the applicant's organizational and personal information, the issuing authority CA's information, the validity period, the certificate serial number and other plain text information, as well as a signature;

Signature generation algorithm: First, use the hash function to calculate the information summary of the public plaintext information, then use the CA's private key to encrypt the information summary, and the ciphertext is the signature;

d. When client C sends a request to server S, S returns the certificate file;

e. Client C reads the relevant plaintext information in the certificate, uses the same hash function to calculate the information digest, then uses the corresponding CA's public key to decrypt the signature data and compares the certificate's information digest. If they are consistent, the legitimacy of the certificate can be confirmed, that is, the public key is legitimate;

f. The client then verifies the domain name information, validity period, and other information related to the certificate;

g. The client will have the certificate information (including the public key) of the trusted CA built in. If the CA is not trusted, the corresponding CA certificate cannot be found and the certificate will be judged illegal.

A few points to note during this process:

a. No private key is required when applying for a certificate, ensuring that the private key can only be controlled by the server forever;

b. The legitimacy of the certificate still relies on the asymmetric encryption algorithm, and the certificate mainly adds server information and signature;

c. The certificate corresponding to the built-in CA is called a root certificate. The issuer and the user are the same, and the certificate is signed by itself, which is a self-signed certificate.

d. Certificate = public key + applicant and issuer information + signature;

3.3 Certificate Chain

If a certificate authority, namely an intermediate certificate, is added between the CA root certificate and the server certificate, the principles of certificate generation and verification remain unchanged, only an additional layer of verification is added, as long as it can be verified as legitimate by any trusted CA root certificate in the end.

a. The issuer of the server certificate server.pem is the intermediate certificate authority inter. Inter verifies that server.pem is indeed a valid certificate issued by itself based on the certificate inter.pem;

b. The issuing CA of the intermediate certificate inter.pem is root. Root verifies that inter.pem is a legitimate certificate issued for itself based on the certificate root.pem;

c. The client has built-in trust in the CA's root.pem certificate, so the server certificate server.pem is trusted.

The server certificate, intermediate certificate and root certificate are combined into a legal certificate chain. The verification of the certificate chain is a bottom-up trust transfer process.

The advantages of the secondary certificate structure are:

a. Reduce the management workload of the root certificate structure, and conduct more efficient certificate review and issuance;

b. The root certificate is usually built into the client, and the private key is usually stored offline. Once the private key is leaked, the revocation process is very difficult and cannot be remedied in time;

c. If the private key of the intermediate certificate structure is leaked, it can be quickly revoked online and a new certificate can be reissued to the user;

d. Certificate chains within four levels generally do not have a significant impact on HTTPS performance.

The certificate chain has the following characteristics:

a. There may be multiple valid certificate chains for the same server certificate.

Because the generation and verification of certificates are based on public and private key pairs, if the same public and private keys are used to generate different intermediate certificates, for the issuer, the issuing authority is a legitimate CA, the difference is that the issuing authority of the intermediate certificates is different;

b. The levels of different certificate chains are not necessarily the same, they may be two-level, three-level or four-level certificate chains.

The issuing authority of the intermediate certificate may be the root certificate authority or another intermediate certificate authority, so the certificate chain hierarchy may not be the same.

3.4 Certificate Revocation

CAs can issue certificates, and there are also mechanisms to declare previously issued certificates invalid. If the certificate user is illegal, the CA needs to revoke the certificate; or if the private key is lost, the user applies to invalidate the certificate. There are two main mechanisms: CRL and OCSP.

(a) CRL

Certificate Revocation List, a separate file. This file contains the serial number (unique) and revocation date of the certificates that the CA has revoked. The file also contains the effective date and the time to update the file next time. Of course, the file must contain the signature of the CA private key to verify the legitimacy of the file.

The certificate generally contains a URL address CRL Distribution Point, which informs the user where to download the corresponding CRL to verify whether the certificate is revoked. The advantage of this revocation method is that it does not require frequent updates, but it cannot revoke the certificate in time, because the CRL update time is generally a few days, during which time great losses may have been caused.

(b) OCSP

Online Certificate Status Protocol, a real-time way to check whether a certificate is revoked. The requester sends the certificate information and requests a query, and the server returns any status of normal, revoked, or unknown. The certificate generally also contains an OCSP URL address, requiring the query server to have good performance. Some CAs or most self-signed CAs (root certificates) do not provide CRL or OCSP addresses, which can be very troublesome for revoking certificates.

4.TLS/SSL handshake process

4.1 Handshake and key negotiation process

The handshake process is explained in detail using the example of client authentication server based on RSA handshake and key exchange.

1.client_hello

The client initiates a request and transmits the request information in plain text, including version information, encryption suite candidate list, compression algorithm candidate list, random number, extension field and other information. The relevant information is as follows:

The highest supported TLS protocol version is SSLv2, SSLv3, TLSv1, TLSv1.1, TLSv1.2. Currently, versions lower than TLSv1 are basically no longer used.

The list of cipher suites supported by the client. Each cipher suite corresponds to a combination of the four functions in the previous TLS principle: authentication algorithm Au (authentication), key exchange algorithm KeyExchange (key negotiation), symmetric encryption algorithm Enc (information encryption) and information digest Mac (integrity check);

A list of supported compression methods, used for subsequent information compression transmission;

Random number random_C, used for subsequent key generation;

The extension field extensions supports relevant parameters of the protocol and algorithm and other auxiliary information. The common SNI belongs to the extension field, and the function of this field will be discussed separately later.

2.serverhello+servercertificate+severhellodone

(a) serverhello, the server returns the negotiation information, including the selected protocol version, the selected cipher suite, the selected compression algorithm, the random number randomS, etc. The random number is used for subsequent key negotiation;

(b) server_certificates, server-side configuration of the corresponding certificate chain for identity authentication and key exchange;

(c) serverhellodone, notifies the client that the sending of server_hello information has been completed;

3. Certificate verification

The client verifies the legitimacy of the certificate. If the verification is successful, subsequent communication will be carried out. Otherwise, prompts and operations will be given according to the error situation. The legitimacy verification includes the following:

Trusted certificate path of the certificate chain, the method is as described above;

Whether the certificate is revoked or not, there are two ways: offline CRL and online OCSP. Different clients will behave differently.

Expiry date: whether the certificate is within the valid time range;

Domain name domain, check whether the certificate domain name matches the current access domain name, and analyze the matching rules later;

4.clientkeyexchange+changecipherspec+encryptedhandshakemessage

(a) clientkeyexchange, after the legitimacy verification is passed, the client calculates a random number Pre-master, encrypts it with the certificate public key, and sends it to the server;

(b) At this point, the client has obtained all the information needed to calculate the negotiated key: two plaintext random numbers randomC and randomS and the Pre-master generated by itself, and calculates the negotiated key;

enckey=Fuc(randomC, random_S, Pre-Master)

(c) changecipherspec, the client notifies the server that all subsequent communications will be encrypted using the negotiated communication key and encryption algorithm;

(d) EncryptedHandshakeMessage, which combines the hash values ​​of all previous communication parameters and other related information to generate a piece of data, which is encrypted using the negotiated key session secret and algorithm, and then sent to the server for data and handshake verification;

5.changecipherspec+encryptedhandshakemessage

(a) The server uses the private key to decrypt the encrypted Pre-master data and calculates the negotiated key based on the two plaintext random numbers randomC and randomS exchanged previously: enc_key=Fuc(randomC, randomS, Pre-Master);

(b) Calculate the hash value of all previously received information, then decrypt the encrypted handshake message sent by the client to verify the correctness of the data and key;

(c) changecipherspec, after verification, the server also sends changecipherspec to inform the client that subsequent communications will use the negotiated key and algorithm for encrypted communications;

(d) EncryptedHandshakeMessage, the server also combines all the current communication parameter information to generate a piece of data and encrypts it using the negotiated key session secret and algorithm and sends it to the client;

6. Handshake ends

The client calculates the hash value of all received information, and uses the negotiated key to decrypt the encrypted handshake message to verify the data and key sent by the server. If the verification is successful, the handshake is completed;

7. Encrypted Communications

Start encrypted communication using the negotiated key and algorithm.

Notice:

(a) The server can also request verification of the client, i.e., two-way authentication. In process 2, the client sends the clientcertificaterequest message. In process 4, the client first sends the clientcertificate and certificateverifymessage messages. The certificate verification method is basically the same. The certificateverify_message is a piece of data encrypted by the client's private key based on the negotiated communication information. The server can use the corresponding public key to decrypt and verify.

(b) Depending on the key exchange algorithm used, such as ECC, the negotiation details are slightly different, but the overall situation is similar;

(c) The role of server key exchange is to send it to the client to calculate the pre-master when the server certificate does not carry enough information. For example, for DH-based certificates, the public key is not included in the certificate and needs to be sent separately.

(d) change cipher spec can actually be used to notify the peer to change the encryption communication method currently used. Currently, there is no in-depth analysis;

(e) alter message is used to indicate a status change or error message during the handshake or communication process. Generally, the alarm message triggering conditions are connection closure, receipt of illegal information, information decryption failure, user cancellation of the operation, etc. After receiving the alarm message, the communication will be disconnected or the receiver will decide whether to disconnect.

4.2 Session Cache Handshake Process

In order to speed up the establishment of handshakes and reduce the performance degradation and resource consumption caused by the protocol (detailed analysis is provided later), the TLS protocol has two types of session cache mechanisms: session ID and session ticket.

Session ID is supported by the server side. It is a standard field in the protocol, so it is supported by almost all servers. The server side saves the session ID and negotiated communication information. In Nginx, 1M memory can save about 4,000 session ID machine-related information, which occupies more server resources.

Session ticket requires support from both the server and the client. It is an extended field with a support range of about 60% (no reliable statistics or source). The negotiated communication information is encrypted and sent to the client for storage. The key is only known to the server, and it occupies very little server resources.

The main difference between the two is the location and method of storing negotiation information, which is similar to session and cookie in http.

When both exist, (nginx implementation) session_ticket is used first.

The handshake process is as follows:

Note: Although the handshake process has 1.5 round trips, the first application data sent by the client to the server does not need to wait for the server to return information, so the handshake delay is 1*RTT.

1. Session ID

(a) If a connection has been established between the client and the server, the server will return the session ID after a successful handshake and save the corresponding communication parameters in the server;

(b) If the client needs to establish a connection with the server again, it sends the recorded information in the session ID in client_hello to the server;

(c) The server retrieves the cache record based on the received session ID. If no record is found or the cache is expired, the normal handshake process is followed.

(d) If the corresponding cache record is retrieved, the changecipherspec and encryptedhandshakemessage information are returned. The two information have similar functions. Encryptedhandshakemessage is the hash value of the current communication parameters and master_secret.

(f) If the client can verify the encrypted data through the server, the client also sends the ChangeCipherSpec and EncryptedHandshakeMessage information;

(g) If the server verifies the data, the handshake is established successfully and normal encrypted data communication begins.

2. Session Ticket

(a) If a connection has been established between the client and the server, the server will carry the encrypted session_ticket information in the newsessionticket data, and the client will save it;

(b) If the client needs to establish a connection with the server again, it will carry the encrypted information in the sessionticket extension field in the clienthello and send it to the server;

(c) The server decrypts the sesssion_ticket data. If the decryption fails, the normal handshake process is followed;

(d) If the decryption is successful, the changecipherspec and encryptedhandshakemessage information are returned. The functions of these two information are similar to those in the session ID.

(f) If the client can verify the encrypted data through the server, the client also sends the ChangeCipherSpec and EncryptedHandshakeMessage information;

(g) If the server verifies the data, the handshake is established successfully and normal encrypted data communication begins.

4.3 Reestablishing the connection

Renegotiation is the process of abandoning the current TLS connection and re-authenticating and negotiating the key. It can re-authenticate, update the key or algorithm without disconnecting the current data transmission, so the information stored and cached on the server can be maintained. Both the client and the server can initiate the process of re-establishing the connection. Currently, Windows 2000 & XP and SSL 2.0 do not support this.

1. Server reestablishes connection

The server-side reconnection usually occurs when the client accesses protected data. The basic process is as follows:

(a) A valid TLS connection is established and communication is established between the client and the server;

(b) Client access to protected information;

(c) The server returns the hello_request message;

(d) After receiving the hellorequest message, the client sends a clienthello message to start reestablishing the connection.

2. The client reestablishes the connection

The client reestablishes the connection generally to update the communication key.

(a) A valid TLS connection is established and communication is established between the client and the server;

(b) The client needs to update the key and actively sends a client_hello message;

(c) After receiving the client_hello message, the server cannot immediately recognize that the message is not application data, so it will submit it to the next step for processing. After processing, it will return a notification that the message is a request to reestablish the connection;

(d) Before confirming the reestablishment of the connection, the server will not immediately stop sending data to the client. There may be cached data that needs to be sent to the client at the same time, but the client will not send any information to the server;

(e) After the server recognizes the request to reestablish the connection, it sends a server_hello message to the client;

(f) The client is also unable to immediately determine that the information is not application data, and also submits it to the next step for processing. After processing, it will return a notification that the information is a request to reestablish the connection;

(g) The client and server begin a new process of reestablishing the connection.

4.4 Key Calculation

The previous section mentioned two random numbers randomC and randomS transmitted in plain text and the Pre-master exchanged between the server and the client through encryption. These three parameters serve as the basis for key negotiation. This section discusses the basic calculation process of key negotiation and the use of keys during communication.

1. Calculate Key

The parameters involved are random client and random server, Pre-master, Master secret, key material. When calculating the key, both the server and the client have this basic information. The exchange method is described in the previous section. The calculation process is as follows:

(a) The client generates a Pre-master using an encryption algorithm such as RSA or Diffie-Hellman;

(b) Pre-master combines the two random numbers from random client and random server through PseudoRandomFunction (PRF) to calculate the Master secret;

(c) The master secret is combined with two random numbers from the random client and random server to obtain the key material through iterative calculation;

The following are some important records that can solve the doubts of some friends who love in-depth research. I copy the materials and share them with you:

(a) The first two bytes of PreMaster secret are the TLS version number, which is an important version number used to verify the handshake data. Because in the Client Hello stage, the client will send a list of encryption suites and the currently supported SSL/TLS version number to the server, and it is transmitted in plain text. If the handshake data packet is cracked, the attacker is likely to tamper with the data packet and select a less secure encryption suite and version to the server, thereby cracking the data. Therefore, the server needs to compare the PreMaster version number decrypted from the ciphertext with the version number of the previous Client Hello stage. If the version number becomes lower, it means that it has been tampered with, and then stop sending any messages immediately. (copy)

(b) Both the client and the server need random numbers so that the generated key will not be the same every time. Since the certificate in the SSL protocol is static, it is necessary to introduce a random factor to ensure the randomness of the negotiated key.

For the RSA key exchange algorithm, the pre-master-key itself is a random number, plus the random number in the hello message, the three random numbers are finally derived into a symmetric key through a key exporter.

The existence of pre master is that the SSL protocol does not trust every host to generate completely random numbers. If the random numbers are not random, then the pre master secret may be guessed. Then it is not appropriate to use only the pre master secret as the key. Therefore, a new random factor must be introduced. Then the key generated by the client, server and pre master secret together will not be easy to guess. One pseudo-random number may not be random at all, but three pseudo-random numbers are very close to random. For every additional degree of freedom, the randomness increases by more than one.

2. Key Usage

After 12 rounds of iterative calculations, Key will obtain 12 hash values, which are grouped into 6 elements. The list is as follows:

(a) mac key, encryption key and IV are a set of encryption elements, used by the client and server respectively, but both sets of elements are obtained by both sides at the same time;

(b) The client uses the client group element to encrypt data, and the server uses the client element to decrypt; the server uses the server element to encrypt, and the client uses the server element to decrypt;

(c) Different directions of two-way communication use different keys, and cracking the communication requires at least two cracks;

(d) encryption key is used to symmetrically encrypt data;

(e) IV is used as the initialization vector for many encryption algorithms, specifically symmetric encryption algorithms;

(f) Mac key is used for data integrity verification;

4.4 Data Encryption Communication Process

(a) Segment the application layer data into appropriate blocks;

(b) Number the fragmented data to prevent replay attacks;

(c) compress the data using the negotiated compression algorithm;

(d) Calculate the MAC value and compress the data to form the transmission data;

(e) Encrypt the data using the client encryption key and send it to the server;

(f) After receiving the data, the server uses the client encrytion key to decrypt, verify the data, decompress the data, and reassemble it.

Note: The calculation of the MAC value includes two Hash values: client Mac key and Hash (number, packet type, length, compressed data).

4.5 Packet capture analysis

I will not analyze the packet capture in detail. According to the previous analysis, the basic situations can be matched. Based on the usual process of locating problems, I would like to point out some points that I think need attention:

1. When capturing HTTP communication packets, you can clearly see the communication header and plain text of the information. However, HTTPS is encrypted communication, so you cannot see the relevant header and plain text information of the HTTP protocol.

2. Packet capture HTTPS communication mainly includes three processes: TCP connection establishment, TLS handshake, and TLS encrypted communication. The main analysis is information such as the handshake establishment and status of HTTPS communication.

3.client_hello

The version information can tell you the highest protocol version number supported by the client. If it is a lower version protocol such as SSL 3.0 or TLS 1.0, be very careful about the possibility of handshake failures due to the lower version.

Determine whether SNI is supported based on the server_name field in the extension field. If it exists, it is supported, otherwise it is not supported. This is very useful for locating handshake failures or certificate return errors.

The session ID is part of the standard protocol. If no connection has been established, the corresponding value is empty. If it is not empty, it means that the corresponding connection has been established before and cached.

The session record session ticket is the extended protocol part. The existence of this field indicates that the protocol supports session tickets. Otherwise, it does not support it. If it exists and the value is empty, it means that no connection has been established and cached before. If it exists and the value is not empty, it means that there is a cached connection.

4.server_hello

The TLS version field can be used to infer the highest version of the protocol supported by the server. Different versions may cause handshake failure.

Determine the encryption protocol that the server supports first based on the cipher_suite information;

5.ceritficate

The server configures and returns the certificate chain. The server compares the certificate information with the server configuration file to determine whether the request is consistent with the expectation. If not, whether to return the default certificate.

6.alert

The alarm information alert will explain the reason for the failure to establish a connection, that is, the alarm type, which is very important for locating the problem.

5.HTTPS performance and optimization

5.1 HTTPS performance loss

The previous article discussed the principles and advantages of HTTPS: identity authentication, information encryption and integrity verification, etc., and did not make any changes to the TCP and HTTP protocols. However, adding new protocols to achieve more secure communications will inevitably come at a cost. The performance loss of the HTTPS protocol is mainly reflected in the following:

1. Increase delay

Analyzing the previous handshake process, a complete handshake requires at least two rounds of communication between the two ends, which increases the delay by at least 2*RTT. Using session cache to reuse connections also increases the delay by at least 1*RTT*.

2. Consume more CPU resources

In addition to data transmission, HTTPS communication mainly includes symmetric encryption and decryption, asymmetric encryption and decryption (the server mainly uses private keys to decrypt data); stress test of the single-core CPU of the TS8 model: symmetric encryption algorithm AES-CBC-256 throughput 600Mbps, asymmetric RSA private key decryption 200 times/s. Without considering other software-level overhead, a 10G network card consumes about 17 CPU cores for symmetric encryption, and a 24-core CPU can connect to a maximum of 4,800 HTTPS connections;

The HTTP access capacity of the TS8 model with a current 10G network card is about 100,000/s. If all HTTP connections are converted to HTTPS connections, RSA decryption will obviously become the bottleneck first. Therefore, RSA decryption capability is the main problem that currently plagues HTTPS access.

5.2 HTTPS access optimization

1. CDN access

The delay added by HTTPS is mainly the transmission delay RTT. The characteristic of RTT is that the closer the node is, the smaller the delay is. CDN is naturally closest to the user, so choosing to use CDN as the entrance to HTTPS access will greatly reduce the access delay. CDN nodes can greatly reduce the delay caused by HTTPS through controllable methods such as maintaining long connections with business servers, session reuse, and link quality optimization.

2. Session Cache

Although it was mentioned above that HTTPS will have a delay of at least 1*RTT even with session caching, the delay has been reduced to half of the original delay, which is a significant delay optimization. At the same time, the HTTPS connection established based on the session cache does not require the server to use the RSA private key to decrypt and obtain the Pre-master information, which can save CPU consumption. If the business access connection is concentrated and the cache hit rate is high, the access capability of HTTPS will be significantly improved. The cache hit rate of the current TRP platform is greater than 30% during the peak period. The access resources of 10k/s can actually carry 13k/s of access, which is very effective.

3. Hardware Acceleration

Installing a dedicated SSL hardware acceleration card for the access server works like a GPU, freeing up the CPU, and providing higher HTTPS access capabilities without affecting business programs. Testing of a hardware acceleration card shows that a single card can provide 35k decryption capabilities, equivalent to a 175-core CPU, or at least seven 24-core servers. Considering the overhead of other programs on the access server, a hardware card can achieve access capabilities close to 10 servers.

4. Remote decryption

Local access consumes too much CPU resources, wasting resources such as network cards and hard disks. Consider transferring the RSA decryption computing tasks that consume the most CPU resources to other servers, so that the access capabilities of the servers can be fully utilized and bandwidth and network card resources can be fully utilized. The remote decryption server can be a machine with a low CPU load to achieve machine resource reuse, or it can be a specially optimized server with high computing performance. Currently, it is also one of the solutions for CDN to use for large-scale HTTPS access.

5. SPDY/HTTP2

The previous methods improve HTTPS access performance by reducing transmission delay and single-machine load, but all of these methods are based on optimization methods that do not change the HTTP protocol. SPDY/HTTP2 uses the advantages brought by TLS/SSL to improve HTTPS performance and download speed by modifying the protocol.

For more information, please follow our WeChat ID: weixinBugly or search for our WeChat ID: weixinBugly.

Tencent Bugly

Bugly is an external version of Tencent's internal product quality monitoring platform, which supports the two major platforms of iOS and Android. Its main function is to monitor and report crashes and freezes that occur on the user side after the App is released, so that developers can understand the quality of the app at the first time and make timely modifications. Currently, all products within Tencent are using it to monitor the crashes of online products. Tencent's internal team has been polishing it for 4 years, and all of Tencent's internal products are currently using it, basically covering the mobile devices and network environments in the Chinese market, and its reliability is guaranteed. By using Bugly, you are using the same