理解TCP之Keepalive

waitingnothing 2018-04-19

展開全文

理解Keepalive(1)

大家都聽過keepalive，但是其實對于keepalive這個詞還是很晦澀的，至少我一直都只知道一個大概，直到之前排查線上一些問題，發(fā)現(xiàn)keepalive還是有很多玄機的。其實keepalive有兩種，一種是TCP層的keepalive，另一種是HTTP層的Keep-Alive。這篇文章先說說tcp層的keepalive

tcp keepalive

設(shè)想有一種場景：A和B兩邊通過三次握手建立好TCP連接，然后突然間B就宕機了，之后時間內(nèi)B再也沒有起來。如果B宕機后A和B一直沒有數(shù)據(jù)通信的需求，A就永遠都發(fā)現(xiàn)不了B已經(jīng)掛了，那么A的內(nèi)核里還維護著一份關(guān)于A&B之間TCP連接的信息，浪費系統(tǒng)資源。于是在TCP層面引入了keepalive的機制，A會定期給B發(fā)空的數(shù)據(jù)包，通俗講就是心跳包，一旦發(fā)現(xiàn)到B的網(wǎng)絡(luò)不通就關(guān)閉連接。這一點在LVS內(nèi)尤為明顯，因為LVS維護著兩邊大量的連接狀態(tài)信息，一旦超時就需要釋放連接。

Linux內(nèi)核對于tcp keepalive的調(diào)整主要有以下三個參數(shù)

1. tcp_keepalive_time

 the interval between the last data packet sent (simple ACKs are not considered data) and the first keepalive probe; after the connection is marked to need keepalive, this counter is not used any further

2. tcp_keepalive_intvl

 the interval between subsequential keepalive probes, regardless of what the connection has exchanged in the meantime

3. tcp_keepalive_probes

 the number of unacknowledged probes to send before considering the connection dead and notifying the application layer

Example

$ cat /proc/sys/net/ipv4/tcp_keepalive_time
  7200
$ cat /proc/sys/net/ipv4/tcp_keepalive_intvl
  75
$ cat /proc/sys/net/ipv4/tcp_keepalive_probes
  9

當(dāng)tcp發(fā)現(xiàn)有tcp_keepalive_time(7200)秒未收到對端數(shù)據(jù)后，開始以間隔tcp_keepalive_intvl(75)秒的頻率發(fā)送的空心跳包，如果連續(xù)tcp_keepalive_probes(9)次以上未響應(yīng)代碼對端已經(jīng)down了，close連接

在socket編程時候，可以調(diào)用setsockopt指定不同的宏來更改上面幾個參數(shù)

TCP_KEEPCNT: tcp_keepalive_probes

TCP_KEEPIDLE: tcp_keepalive_time

TCP_KEEPINTVL: tcp_keepalive_intvl

Nginx配置tcp keepalive

Nginx對于keepalive的配置有一大堆，大伙每次看都迷茫了，其實Nginx涉及到tcp層面的keepalive只有一個：so_keepalive。它屬于listen指令的配置參數(shù)，具體配置

so_keepalive=on|off|[keepidle]:[keepintvl]:[keepcnt]

this parameter (1.1.11) configures the “TCP keepalive” behavior for the listening socket. If this parameter is omitted then the operating system’s settings will be in effect for the socket. If it is set to the value “on”, the SO_KEEPALIVE option is turned on for the socket. If it is set to the value “off”, the SO_KEEPALIVE option is turned off for the socket. Some operating systems support setting of TCP keepalive parameters on a per-socket basis using the TCP_KEEPIDLE, TCP_KEEPINTVL, and TCP_KEEPCNT socket options. On such systems (currently, Linux 2.4 , NetBSD 5 , and FreeBSD 9.0-STABLE), they can be configured using the keepidle, keepintvl, and keepcnt parameters. One or two parameters may be omitted, in which case the system default setting for the corresponding socket option will be in effect.

Example

so_keepalive=30m::10
    will set the idle timeout (TCP_KEEPIDLE) to 30 minutes, leave the probe interval (TCP_KEEPINTVL) at its system default, and set the probes count (TCP_KEEPCNT) to 10 probes.

在Nginx的代碼里可以看到

./src/http/ngx_http_core_module.c

static ngx_command_t  ngx_http_core_commands[] = {
...
    // listen 指令解析 -->> call ngx_http_core_listen()
    { ngx_string('listen'),
      NGX_HTTP_SRV_CONF|NGX_CONF_1MORE,
      ngx_http_core_listen,
      NGX_HTTP_SRV_CONF_OFFSET,
      0,
      NULL },
...
}

static char *
ngx_http_core_listen(ngx_conf_t *cf, ngx_command_t *cmd, void *conf){

...
        // 下面就是 so_keepalive 后面的參數(shù)解析
        if (ngx_strncmp(value[n].data, 'so_keepalive=', 13) == 0) {

            if (ngx_strcmp(&value[n].data[13], 'on') == 0) {
                lsopt.so_keepalive = 1;

            } else if (ngx_strcmp(&value[n].data[13], 'off') == 0) {
                lsopt.so_keepalive = 2;

            } else {
            // 自定義系統(tǒng)keepalive的相關(guān)設(shè)置
            ...

        }

    if (ngx_http_add_listen(cf, cscf, &lsopt) == NGX_OK) {
        return NGX_CONF_OK;
    }

}

./src/core/ngx_connection.c


        if (ls[i].keepidle) {
            value = ls[i].keepidle;
            // 設(shè)置 tcp_keepalive_time 
            if (setsockopt(ls[i].fd, IPPROTO_TCP, TCP_KEEPIDLE,
                           (const void *) &value, sizeof(int))
                == -1)
            {
                ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_socket_errno,
                              'setsockopt(TCP_KEEPIDLE, %d) %V failed, ignored',
                              value, &ls[i].addr_text);
            }
        }

        if (ls[i].keepintvl) {
            value = ls[i].keepintvl;
            // 設(shè)置 tcp_keepalive_intvl
            if (setsockopt(ls[i].fd, IPPROTO_TCP, TCP_KEEPINTVL,
                           (const void *) &value, sizeof(int))
                == -1)
            {
                ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_socket_errno,
                             'setsockopt(TCP_KEEPINTVL, %d) %V failed, ignored',
                             value, &ls[i].addr_text);
            }
        }

        if (ls[i].keepcnt) {
            // 設(shè)置 tcp_keepalive_intvl
            if (setsockopt(ls[i].fd, IPPROTO_TCP, TCP_KEEPCNT,
                           (const void *) &ls[i].keepcnt, sizeof(int))
                == -1)
            {
                ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_socket_errno,
                              'setsockopt(TCP_KEEPCNT, %d) %V failed, ignored',
                              ls[i].keepcnt, &ls[i].addr_text);
            }
        }