Maintained by: NLnet Labs

[Unbound-users] Unbound not resolving after ISC.org attack

Dominick Rivard
Thu Mar 14 21:10:12 CET 2013


Hi Wouter, 

Thanks for your answer, I will be honest we haven't checked the firewall at
that moment to look if we were denying ourselves.
Another info I can tell is that the servers resources were in no point
overloaded, we balance the port 53 to many servers and they were all fine.

14:09:07 up 147 days,  3:47,  2 users,  load average: 0.00, 0.01, 0.00 that
almost like this on each server.
That day the ram was used at about 60% and cpu at 20%.

How much space do you think I need to activate the verbosity 2 for let say a
week? I get between 15 and 20 million requests a day.

Best regards.
Dominick

-----Original Message-----
From: unbound-users-bounces at unbound.net
[mailto:unbound-users-bounces at unbound.net] On Behalf Of W.C.A. Wijngaards
Sent: March-14-13 4:10 AM
To: unbound-users at unbound.net
Subject: Re: [Unbound-users] Unbound not resolving after ISC.org attack

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Dominick,

Likely, unbound itself is not the issue here.  Unbound, after a quick lookup
of isc.org, answers from cache.  And thus there is not much traffic from
unbound to upstream.  The isc.org requests should not really interfere with
other unbound lookups (until the CPU is overloaded).

If the firewall has connection tracking, which, for UDP, and with an attack,
could run out of space and deny lookups.  Then, unbound, when it tries to
make connections is denied connectivity to the internet and starts failing?
What I suggest here is that simply your network kit has overloaded and thus
unbound cannot make connections any longer.  This would also fit with a
different IP (which may have different treatment by your own firewall) that
makes things work?

You can increase verbosity on unbound (level 2 may already log enough for
you to know it is trying to contact upstream servers).  But that is likely
to simply tell you that unbound is indeed trying to send out traffic, but
that this traffic fails.

If somehow unbound is the problem here, please, let me know.  We think
performance under high load is important.

Best regards,
   Wouter


On 03/13/2013 07:52 PM, Dominick Rivard wrote:
> Hi,
> 
> 
> 
> I am trying to find the cause of an issue we have been experiencing 
> last Thursday. We are running multiple Unbound servers
> 
> In order to provide internet to our users. I would say we were under 
> attack with a couple of IPs trying to request as many as possible 
> records for “ANY? isc.org.”. The DNS were resolving until a certain 
> period of time where it became just impossible to resolve anything. My 
> first try was to restart the unbound service and it worked for a 
> couple of second, then it failed again. Next step was to block these 
> IPs with iptables, but still it wasn’t resolving, even after a second 
> restart of the unbound service.
> 
> 
> 
> What resolved the issue was to route the trafic to a different NAT ip 
> so the unbound servers were seen as a different public ip when going 
> to internet. At that point I thought I could have been throttled or 
> blacklisted by the roots servers.  I wrote to them and they explain to 
> me that they don’t have such a mean to  limite the rate of queries or 
> throttle any of our request.
> 
> 
> 
> So I am turning to you guys to ask some question what could be slowing 
> me down or blocking me from my local unbound server to resolve any 
> name? Is there any configuration I  need to change?
> 
> 
> 
> How would you prevent these kind of attack in the future?
> 
> 
> 
> Unbound version: 1.4.18
> 
> OS: Debian Squeeze 6.0.6
> 
> 
> 
> Best regards.
> 
> /Dominick Rivard,/ /Solutions Architect/
> 
> image001
> 
> 5275 Queen Mary Montréal, Qc H3W 1Y3 Tel: 514-385-4448 ext 126 Fax:
> 514-385-6660
> 
> */Notice: /*/This message is confidential and privileged. If you are 
> not the addressee, please inform the sender by return e-mail 
> immediately and delete this message and destroy all copies./
> 
> */Avis :/*/Ce message est confidentiel et protégé par le secret 
> professionnel. Si vous n’êtes pas le destinataire, veuillez informer 
> l’expéditeur par courrier électronique immédiatement et effacer ce 
> message et en détruire toute copie./
> 
> 
> 
> 
> 
> _______________________________________________ Unbound-users mailing 
> list Unbound-users at unbound.net 
> http://unbound.nlnetlabs.nl/mailman/listinfo/unbound-users
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJRQYXCAAoJEJ9vHC1+BF+NtJkP/i5IhBM6HGMfQ497IcoGboGB
ZAS+lkk+f7aYIR5nFohypje4ekFzAlk17y88Xvidy9Pag2xou7+VRlALATemCOCG
74TZsd/fsQWRppIBuyVYMDZHk9T52IXe00j3RmtGewewmX8xAjiHM0BhwGwORV/7
QJBxdq0SdOfOpYOcgbt9YLa1o98ACL6fo3OfqXgKYgk49WH9ngB7dqhk/H21p3CY
BQNzXda+iJA0Cu0rTH+F6KgQbMcyHUuDFiEkJtrZPycW8esjnfPtVvBcOwUjh8a4
x9zGWpSE6IUC9z9bVSs5/pHww8enmXfBOsUFfiMmcJbDClCZd4sQPN+QIhXyH4Os
+kgiCwJgPFykg82SIXT154AASwYSafn48tMcSlJJExh4mNuL5K/ytDLHIHUzhYJu
12kCepz0ntuTt+afmWnLX5DGhWg+NVQ1j4/r9RVms1uIIvBtY4xdPL7Vj65nZSTT
j//K0KOJ7hxOJ1+2HXOvgFRTFPBTZ5fx8He+fkz85ZoHZcRAdtVgvAGI8ZI6HAPP
N8/O3ZTshCN3vl37AvNfIzkqsTTd7vrnEz8GhiGBcEYwySvdqugyscRXxapZH5sy
nSneyEJfgLDXxlH//5KRwQhNsNUFiR0DqhMxyv2iods0Wi2xEETp0CgWuB2TNbX+
eYmfu2haYNDUL5lH3Fd7
=q2Lv
-----END PGP SIGNATURE-----
_______________________________________________
Unbound-users mailing list
Unbound-users at unbound.net
http://unbound.nlnetlabs.nl/mailman/listinfo/unbound-users