FWIW there is readv() / writev(), splice(), sendfile(), funopen(), and io_buffer() as well.
splice() is great when transferring data between pipes and UNIX sockets with zero-copy, but it is Linux-only.
splice() is the fastest and most efficient way to transfer data through pipes (on Linux), especially for large volumes. It bypasses memory allocations in userspace (as opposed to read(v)/write(v)), there is no extra buffer management logic, there is no memcpy() or iovec traversal.
Sadly on BSDs, for pipes, readv() / writev() is the most performant way to achieve the same if I am not mistaken. Please correct me if I am wrong.
At any rate, this is a great article.
Great article, discussed previously on HN:
https://news.ycombinator.com/item?id=31592934 (200 comments)
https://news.ycombinator.com/item?id=37782493 (105 comments)
This is such a dope article. I love that it comes from time to time.
I feel bad that this doesn't have any comments, the article was really great.
I'd like to use splice more, but the end of the article talked about the security implications and some ABI breaking.
I'm curious to know if long term plans are to keep splice around?
I'd also be curious how hard it would be to patch the default pipe to always use splice for performance improvements.
Does modern Linux have anything close to Doors? I’ve an embedded application where two processes exchange small amounts of data which are latency sensitive, and I’m wondering if there’s anything better than AF_UNIX.
(2022)
Seared into my soul is the experience porting a linux pipe-based application to Windows, thinking it's all posix and given it's all in memory the performance will be more or less the same. The performance was hideous, even after we found that having pipes waiting for a connection more or less ground windows to a halt.
Some years later this got revisited due to needing to use the same thing under C# on Win10 and while it was better it was still a major embarrassment how big the performance gap was.