kernel: Out of socket memory
One machine running CentOS 5.5 x64 becomes non-responsive yesteraday , and I could only find some records in /var/log/messages :
kernel: Out of socket memory
Other than that , nothing significant is found. In this case , I don’t know if the max memory allocated for TCP stack usage is all used up , or due to other reasons like too many orphans socket .
To see the current kernel parameter for maximum memory value on TCP stack :
% sysctl -a | grep tcp | grep mem
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 16384 4194304
net.ipv4.tcp_mem = 786432 1048576 1572864
Note:
net.ipv4.tcp_rmem : setting for recieve buffer size for all types of connection
net.ipv4.tcp_wmem : setting for send buffer size for all types of connection
net.ipv4.tcp_mem : memory size for TCP Stack , including all TCP sockets
To see current kernel parameter for maximum orphans socket :
%sysctl -a | grep tcp | grep orphans
net.ipv4.tcp_max_orphans = 65536
To know the current usage Value of socket memory :
% cat /proc/net/sockstat
sockets: used 997
TCP: inuse 49 orphan 14 tw 267 alloc 589 mem 905
UDP: inuse 18 mem 0
RAW: inuse 0
FRAG: inuse 0 memory 0
The above example shows orphan 14 and TCP memory used 905 which are far lower than the corresponding kernal parameter . I logged down the above value periodically , and will use that to compare with kernal parameter with the last record once the machine got crashed again.
