High Traffic Server Configuration
We have a REST API server with 25 million calls each day. Our stack consists of Haproxy + Gunicorn + Flask also we have a MongoDB database that's used by our Rest API. We monitor it with Netdata and watch the statistics with Elasticsearch. Server has 64 GB Ram, AMD Ryzen 7 1700x Pro and SSD storage. Sometimes, netdata used to alarm us about "Accept Queue Overflow" and "Listen Queue Overflow", when we look at these alarms over google, we see that there are some stuff to be changed over at the sysctl.conf and we increased the neccesseary values little by little. After we changed the values we stopped getting alarms. But even though, when we look at the sysctl.conf we have a feeling that the values we set are absurt. So if you could take a look at our sysctl.conf and make a comment about it, we would be glad. Thank you.
net.ipv4.tcp_max_syn_backlog = 1000000 net.core.somaxconn = 8192 net.core.netdev_max_backlog = 900000 net.netfilter.nf_conntrack_max = 1024288 net.netfilter.nf_conntrack_tcp_timeout_close_wait = 20 net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 30 net.netfilter.nf_conntrack_tcp_timeout_time_wait = 20 net.core.wmem_default=8388608 net.core.rmem_default=8388608 net.core.rmem_max=16777216 net.core.wmem_max=16777216 net.ipv4.tcp_rmem=4096 8388608 16777216 net.ipv4.tcp_wmem=4096 8388608 16777216 net.ipv4.tcp_mem=4096 8388608 10388608 net.ipv4.route.flush=1 net.ipv4.ip_local_port_range = 10000 61000
netstat -s | grep -i list output;
netstat -s | grep -i list 644 times the listen queue of a socket overflowed 279381 SYNs to LISTEN sockets dropped
We currently see no problem because we moved our Rest API to Websockets, but still; we are curious and we would like to know if what we are doing is wrong. (Our concurrent connection is around 1200-1500 on Websockets).