Ad

Is There Any Way To Avoid Missing Bytes On A Python Socket Data Transmission?

- 1 answer

I have two different python clients communicating with each other (Client A and Client B) through an external node server on the internet. Client A should send many identified messages in a short period of time to client B, where the node server is responsible to forward all received messages from A to B.

What is happening is that client A is overwriting some message.

I have detected the problem on client B, which is where I analyze the messages. However, the problem occurs in the transmission of messages between client A and Node server, because I am already able to see the broken message on node server logs.

I will show some pseudo-code along with some logs regarding the error (Client A and Server only) :

Client A

Code ... import socket

server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.connect((HOST, PORT))

...

while not END:
    #Prepare message_to_clinet

    #If I uncomment this timer, the error will not appear (or appear later)
    #time.sleep(0.2)

    # I have tried either sendall and send. But it seems that is not effective
    server_socket.sendall(message_to_client)
    print '(Sent):' + message_to_client

Logs when the error occurs Imagine that I want to send 100 times the message: 'Where is the bug?'

...
(Sent): Where is the bug?
(Sent): Where is the bug?
(Sent): Where is the bug?
(Sent): Where is the bug?
(Sent): Where is the bug?
...

Node

Code

const net = require('net');
const tcpServer = net.createServer(connection.requestHandler);
tcpServer.listen(connection.port, (err) => {
    if (err) {
        return console.log('tcp socket error', err);
    }

    console.log(`tcp socket listening on ${connection.port}`);
});

...

const requestHandler = (sock) => {   
    sock.on('data', (data) => {
        const protocol_message = new Buffer(data).toString('ascii');

        //Send this message to remote client
        currentConnection.sendMessagesToClient(protocol_message)
        console.log('Received: ' + protocol_message)
    }
}

Logs when the error occurs. I receive more messages for each socket.send maybe because socket.send it is called several times in a short time on client A (which is not a problem because on the real code I have a 4 byte-length prefix to prevent errors when reading on clinet B)

...
Received: Where is the bug? Where is the bug? Where is the bug?  Where is the bug?Where is the bug?
Received: Where is the bug? Where is the bug? Where is the bug?  Where is the bug?Where is the bug?
Received: Where is thWhere is the bug? Where is the bug?Where is the bug?
...

Here I can see that one of the 100 messages I repeatedly sent is broken. However, on client A, everything is apparently well.

Does anyone know what is a possible cause of this error?

I know if put a time.sleep(0.1) between each message on client A, the error takes more time and transmitted bytes to happen. But when it works, it is just too slow. I also know that if I use the developer local server, I cannot reproduce the error. My guess is that this is related to socket buffer and delay on transmission.

EDIT: I am using Python 2.7

Ad

Answer

I found the answer to my problem. Basically, everything was correct with python clients. However, with TCP sockets, messages can be split in chunks on an upper layer, which leads to a stupid bug on the server side because I was looking for the first character to decide to which socket I'll redirect the message. Of course, this is not a good approach at all, and now it is solved and working.

Although it was a bug on the server side, I only found it because I did some research and conclude that TCP guarantees the same order but it can split in chunks the data.

Thank you anyway!

Ad
source: stackoverflow.com
Ad