How Can I Reliably Read Exactly N Bytes From A TCP Socket?
It is common that a binary protocol defines frames of a given size. The
struct module is good at parsing that, provided everything has been received in a single buffer.
TCP sockets are streams. A read from a socket cannot give more bytes than requested but can return less. So this code is not reliable:
def readnbytes(sock, n): return sock.recv(n) # can return less than n bytes
The naive workaround:
def readnbytes(sock, n): buff = b'' while n > 0: b = sock.recv(n) buff += b if len(b) == 0: raise EOFError # peer socket has received a SH_WR shutdown n -= len(b) return buff
may not be efficient, because if we ask a large number of bytes, and the data if very fragmented, we will repeatedly re-allocate a new byte buffer.
How is it possible to reliably receive exactly n bytes from a stream socket with no risk of re-allocation?
Those other questions are related, and do give hints, but none give a simple and clear answer:
You can use socket.makefile() to wrap the socket in a file-like object. Then reads will return exactly the amount requested, unless the socket is closed where it can return the remainder. Here's an example:
from socket import * sock = socket() sock.bind(('',5000)) sock.listen(1) with sock: client,addr = sock.accept() with client, client.makefile() as clientfile: while True: data = clientfile.read(5) if not data: break print(data)
from socket import * import time sock = socket() sock.connect(('localhost',5000)) with sock: sock.sendall(b'123') time.sleep(.5) sock.sendall(b'451234') time.sleep(.5) sock.sendall(b'51234')
12345 12345 1234
- → What are the pluses/minuses of different ways to configure GPIOs on the Beaglebone Black?
- → Django, code inside <script> tag doesn't work in a template
- → React - Django webpack config with dynamic 'output'
- → GAE Python app - Does URL matter for SEO?
- → Put a Rendered Django Template in Json along with some other items
- → session disappears when request is sent from fetch
- → Python Shopify API output formatted datetime string in django template
- → Shopify app: adding a new shipping address via webhook
- → Shopify + Python library: how to create new shipping address
- → shopify python api: how do add new assets to published theme?
- → Access 'HTTP_X_SHOPIFY_SHOP_API_CALL_LIMIT' with Python Shopify Module