Sending data to peer through TCP connection slow

When sending lots of data to a peer using a TCP connection, the throughput is low under certain conditions: In order to achieve high unidirectional throughput on TCP connections, the sender must send multiple packets at once in order to prevent the Delayed Ack meachnism of the receiver. For that purpose the Nagle meachnism on the socket must be disabled and at least two consecutive TCP writes (or Send To services) must be executed. During to an internal limit in the lwIP implementation, the IP stack will refuse to hold more than 1472 byte unacknowledged bytes per socket. As a result, if the application requests a consecutive write which exceed this limit, the IP stack will not send another frame and the transfer rate will drop tremendously.

The problem occurs when using the SendTo Service and a new SendTo request is sent while another one is outstanding and the sum of the data length of both services exceeds 1472 byte. E.g. the application splits bigger send data into 800 Byte packets. For older implementations the limit when the problem occurs might be even lower. Local applications (LOM) will observe in the problem case that writing to tcp socket is rejected at all.

Workaround for Send To Service: Use data size smaller than 400 Byte.