Best Practice Of Sending Data Of Different Sizes Over The Network

- 1 answer

I want to send data in varying sizes over UDP. The size of data to be sent is not fixed. I have the following scenario:

unsigned char buffer[BUFFERSIZE];
int bytes = fill_buffer(buffer, sizeof(buffer)): // Returns number of filled bytes.
sendto(socket, buffer, bytes, 0, (struct sockaddr *)&server, sizeof(server))

In the example above, receiving side does not know how many bytes to receive. I also thought of first sending the number of bytes to receive and then sending the data. But in that case, I don't know what would happen if the packets arrive out-of-order.

Sender side would be

sendto(socket, &bytes, sizeof(bytes), 0, (struct sockaddr *)&server, sizeof(server))
sendto(socket, buffer, bytes, 0, (struct sockaddr *)&server, sizeof(server))

Receving side would be

recvfrom(socket, &bytes, sizeof(bytes), 0, NULL, NULL)
recvfrom(socket, buffer, bytes, 0, NULL, NULL)

But could it be that sent data comes out-of-order?



I think you can send both in a single datagram if you add a message header.

The sender only sends the amount of payload data it has.

The receiver always requests the maximum payload size but examines the header and the return from recvfrom to determine the actual length.

Here's some rough code that illustrates what I'm thinking of:

struct header {
    u32 magic_number;
    u32 seq_no;
    u32 msg_type;
    u32 payload_length;
} __attribute__((__packed__));

#define MAXPAYLOAD  1024

struct message {
    struct header info;
    unsigned char payload[MAXPAYLOAD];
} __attribute__((__packed__));

sendone(int sockfd,const void *buf,size_t buflen)
    struct message msg;
    static u32 seqno = 0;

    memcpy(&msg.payload[0],buf,buflen); = 0xDEADADDE; = seqno++; = buflen;

    sendto(sockfd,&msg,sizeof(struct header) + buflen,...);

getone(int sockfd,void *buf,size_t buflen)
    struct message msg;
    ssize_t rawlen;
    ssize_t paylen;
    static u32 seqno = 0;

    rawlen = recvfrom(sockfd,&msg,sizeof(struct header) + MAXPAYLOAD,...);

    paylen =;

    if (rawlen != (sizeof(struct header) + paylen))
        // error ...


    return paylen;

The receiver can check the magic number and sequence number to look for corruption or missing/dropped packets, etc.

In fact, you can probably get more efficiency by using sendmsg and recvmsg since they allow you to send a single message using a scatter/gather list. (i.e.) The data would not have to be copied in/out using memcpy from the message struct [you'd only need struct header], so closer to zero copy buffering.

Another option may be to use the MSG_PEEK flag with the recvfrom/recvmsg. I've never used this myself, but it would be something like:

  1. Do recvmsg with length of sizeof(struct header) with MSG_PEEK flag
  2. Do second recvmsg with length of sizeof(struct header) +

This is just a nicety of not having to always provide a maximum sized buffer. Since it involves two syscalls, it may be a bit slower. But, it might allow allow some tricks with selecting a payload buffer from a pool, based on the type of message and/or length