I always knew that UDP is an unreliable transport protocol, but Today I Learned that it is far less reliable than that.
To wit, consider the following code, which, trivial though it is, demonstrates my misunderstanding:
#!/usr/bin/env python
import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(('127.0.0.1', 0))
sock.settimeout(0.5)
addr = sock.getsockname()
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
msg = "hello how are you today"
print "Sending data, len {0}".format(len(msg))
s.sendto(msg, addr)
s.close()
first_chunk = 4
print "Getting first {0} bytes".format(first_chunk)
recvd = sock.recvfrom(first_chunk)[0]
print "Got {0} bytes ({1})".format(len(recvd), recvd)
if len(msg) > len(recvd):
left = len(msg) - len(recvd)
print "Waiting for {0} more bytes".format(left)
recvd += sock.recvfrom(left)[0]
sock.close()
assert msg == recvd
I expected the first recvfrom call to leave the remaining 19 bytes in the buffer, to be returned by the second recvfrom call. No such luck. Looking at the manpage for recvfrom, I can more or less piece together the reason. The sender sent a 23-byte datagram, the receiver asked for 4 bytes, and the kernel said to itself, "Huh, I've got all these bytes hanging around, but you only asked for 4. I GUESS YOU'LL NEVER EVER BE ABLE TO HANDLE ALL THIS DATA" and discarded the remaining 19 bytes rather than leaving them in the buffer. Turns out that this is a decision left up to the implementation. (I tested this on OS X 10.9.4.)
This is more explicit if you're using sendmsg/recvmsg; there's a flag that gets set if this kind of truncation occurs. Again, I get that it's not a stream, but wow, I didn't expect to lose most of the data by calling recvfrom twice for one sent datagram.
I guess this outs me has never having written a proper UDP application. Oops. It's true; I spent my grad school (and before) days writing atop TCP and getting to know its quirks instead. Learn something new every day.