Maintained by: NLnet Labs

Multi-threaded operation?

W.C.A. Wijngaards
Mon Oct 5 14:53:19 CEST 2015


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi Havard,

On 05/10/15 14:43, Havard Eidnes via Unbound-users wrote:
> Hi,
> 
> it looks like I'll have to answer my own question, which is a 
> little disappointing:
> 
>> I'm running unbound 1.5.4 on NetBSD/amd64 7.0, and I notice that 
>> despite me having configured
>> 
>> server: num-threads: 12 so-reuseport: yes
>> 
>> only one of the threads is handling all the queries, according
>> to the output from "unbound-control stats".  "Not what I
>> wanted."
> 
> It turns out that using the "so-reuseport" setting to distribute 
> the load over the threads is a fairly recent Linuxism, and relying
> on it causing the kernel to distribute the load over the different
> sockets is not portable.

Yes, I did not realise this.  If you disable reuseport on BSD I think
it will start to work.

> 
> The first answer in
> 
> http://stackoverflow.com/questions/14388706/socket-options-so-reuseadd
r-and-so-reuseport-how-do-they-differ-do-they-mean-t
>
>  says it quite clearly:
> 
> Linux 3.9 added the option SO_REUSEPORT to Linux as well. [...]
> Additionally the kernel performs some "special magic" for
> SO_REUSEPORT sockets that isn't found in any other operating system
> so far: For UDP sockets, it tries to distribute datagrams evenly,
> for TCP listening sockets, it tries to distribute incoming connect
> requests (those accepted by calling accept()) evenly across all the
> sockets that share the same address and port combination.  [...]
> 
> I'll try to turn off the so-reuseport option later today, and see 
> if that improves the situation.

The option provides nice benefits on Linux; somehow having one file
descriptor per thread/process that is tied to the same socket allows
for much faster operation in the network stack.

Best regards, Wouter

> 
> Regards,
> 
> - Håvard
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJWEnK/AAoJEJ9vHC1+BF+N1v8P+gKLb2H6TDIQiV9ZFvEfg18Y
JQN6jdL1MaDQFQO07SRt3xTKP+GRVy8M3FrSIFC0GeFIR2IYku9FJz7cHFchU3Dp
TimQd6tP/D6qOoOT3W6zoGAFy+RREoc3ZJK66nLMOY5U6PWlgekJsHwgH+LN1whO
2WWSg7om7Fe0xpYdhKaTlBpliWHDXO/J/8jBd+v1+SCqWpg+EsH8NUQkzNEoHE8q
EGs6TjKv/UgnumzapPlKgA2PxGUQArdwCrY4V7/5AdLUa+9qxLjAvu+X17Sj7iHd
grXbygi/h4kXjZyPx9KbzTCtG1NNH1n/cFcfAoGi4n+UBFAfrSr3dF8r/n9CIXCw
M/OmEZ592sKMihtVk5qWi2uXMJD6OPZoCLC+7SEhSfkPTaI4P7phlczBEmt9i9Ro
fCOezRVRg75GU4t1A5vtVLV+SHlFV6wfsaxbLgOSi8OpTmlOs6HvfFlr7wnQb/Jd
OUu30ltwbV3kLVJQngSAtOAOByt6dviCJwcjk5S/LEqOOfTJ1iT809OMI4y3E0aY
t1N+tBXi3MtvQaeid9Drf9COVWIrmj6qFwZuLH+wdPbXKuRpeSdYOi+0HgYMBUBP
6N9DDF3CykVNBJeJMyEsmuIczRRsUZxIGv0Ls0wCLeeBpxmCF0wpicKAu8Z2n8xa
V4hZ0GAkNMOHNY031vRR
=EoJr
-----END PGP SIGNATURE-----