Ad

A Lot Of Socket Endpoints In Python?

I need to parse some Crypto exchanges, such as Poloniex and e.t.c.. I can subscribe their socket apis to getting order-books. Which is the best way to connect to as many as possible order-books? (at least 6 pairs on 4 exchanges, which means I need 24-threads to be used only to listening)

Ad

Answer

You do not need to use threads for this. A reasonably modern server or desktop should be able to receive 24 feeds in a single thread. You will be limited in the amount of data you can receive by your internet connection and by the exchanges' own throttles (they are not interested in publishing 100 Mbps of traffic to you).

Instead of threads, you can use asyncio to listen to as many sockets as you like on a single thread: https://docs.python.org/3/library/asyncio.html

If you find that your single thread truly cannot keep up, you might consider using one thread per exchange or per currency pair (depending on which data is more likely to be used together).

Ad
source: stackoverflow.com
Ad