Hi Wouter,I'm very glad to tell you,I patched the diff file you gave me and now unbound works well on the server,thanks very much! I'm sorry that I didn't tell you clearly about the compile problem.When we found we couldn't compile the code with "--enable-alloc-checks",we added another option "--enable-static-exe" and then finished compiling smoothly,that's all. Thanks again for your warmhearted help. Best Regards, Tao Ma --- 09年9月16日，周三, W.C.A. Wijngaards <wouter at nlnetlabs.nl> 写道： 发件人: W.C.A. Wijngaards <wouter at nlnetlabs.nl> 主题: Re: [Unbound-users] Is there any memory leak in unbound? 收件人: "Tao Ma" <matao_reg at yahoo.com.cn> 日期: 2009年9月16日,周三,下午8:47 Hi Tao Ma, Does this solve your memory leak problem? Index: libunbound/libworker.c =================================================================== --- libunbound/libworker.c (revision 1831) +++ libunbound/libworker.c (working copy) @@ -514,6 +514,7 @@ ldns_buffer_write_u16_at(w->back->udp_buff, 2, qflags); if(local_zones_answer(ctx->local_zones, &qinfo, &edns, w->back->udp_buff, w->env->scratch)) { + regional_free_all(w->env->scratch); libworker_fillup_fg(q, LDNS_RCODE_NOERROR, w->back->udp_buff, sec_status_insecure); libworker_delete(w); @@ -630,6 +631,7 @@ ldns_buffer_write_u16_at(w->back->udp_buff, 2, qflags); if(local_zones_answer(w->ctx->local_zones, &qinfo, &edns, w->back->udp_buff, w->env->scratch)) { + regional_free_all(w->env->scratch); q->msg_security = sec_status_insecure; add_bg_result(w, q, w->back->udp_buff, UB_NOERROR); free(qinfo.qname); Index: daemon/worker.c =================================================================== --- daemon/worker.c (revision 1831) +++ daemon/worker.c (working copy) @@ -806,6 +806,7 @@ } if(local_zones_answer(worker->daemon->local_zones, &qinfo, &edns, c->buffer, worker->scratchpad)) { + regional_free_all(worker->scratchpad); if(ldns_buffer_limit(c->buffer) == 0) { comm_point_drop_reply(repinfo); return 0; Sorry about the compile problem with unbound-host and alloc-checks, I'll look at it later. Best regards, Wouter On 09/16/2009 10:05 AM, Tao Ma wrote: > Hi Wouter,thanks for your reply. > > We use a 32-bit Gentoo and I don't think there's a special memory > allocator in libc. > I can always find the memory consumed by unbound keep growing from ouput > of 'ps' and 'top',the result of sbrk estimate seems to be in accord with > this. > > Following your suggestion,we configure with --enable-alloc-checks and > compile unbound again,now we can find the "Memory conditions" value keep > growing in the log. > > We use memstats to parse the log,but found nothing but the following ouput: > Reading /var/unbound/log_test of size 38767531 > 1% 2% 3% 4% 5% 6% 7% 8% 9% 10% 11% 12% 13% 14% 15% 16% 17% 18% 19% 20% > 21% 22% 23% 24% 25% 26% 27% 28% 29% 30% 31% 32% 33% 34% 35% 36% 37% 38% > 39% 40% 41% 42% 43% 44% 45% 46% 47% 48% 49% 50% 51% 52% 53% 54% 55% 56% > 57% 58% 59% 60% 61% 62% 63% 64% 65% 66% 67% 68% 69% 70% 71% 72% 73% 74% > 75% 76% 77% 78% 79% 80% 81% 82% 83% 84% 85% 86% 87% 88% 89% 90% 91% 92% > 93% 94% 95% 96% 97% 98% 99% 100% done > ------------ > 0 / 0 total in 0 code lines > > We got a complete record in the log from the start of unbound to the > exiting of unbound,and the log file has been attatched to the mail,maybe > it will help you to analyse. > > Let me know if you find any mistakes in my operation. > > By the way,when we configure unbound-1.3.3 with "./configure > --prefix=/usr/local/unbound --enable-alloc-checks" and compile,we got > the following error: > build/smallapp/.libs/unbound-host.o: In function `print_rd': > /home/mt/tools/unbound-1.3.3/smallapp/unbound-host.c:233: undefined > reference to `unbound_stat_malloc_log' > /home/mt/tools/unbound-1.3.3/smallapp/unbound-host.c:255: undefined > reference to `unbound_stat_free_log' > build/smallapp/.libs/unbound-host.o: In function `pretty_rdata': > /home/mt/tools/unbound-1.3.3/smallapp/unbound-host.c:247: undefined > reference to `unbound_stat_free_log' > build/smallapp/.libs/unbound-host.o: In function `lookup': > /home/mt/tools/unbound-1.3.3/smallapp/unbound-host.c:384: undefined > reference to `unbound_stat_free_log' > collect2: ld returned 1 exit status > make: *** [unbound-host] Error 1 > > We did some modification to the code and finished compiling.Is there > anything wrong in the code? > > Thanks & Best regards, > Tao Ma > > > > --- *09年9月15日，周二, W.C.A. Wijngaards /<wouter at NLnetLabs.nl>/* 写道： > > > 发件人: W..C.A. Wijngaards <wouter at NLnetLabs.nl> > 主题: Re: [Unbound-users] Is there any memory leak in unbound? > 收件人: "W.C.A. Wijngaards" <wouter at nlnetlabs.nl> > 抄送: "Tao Ma" <matao_reg at yahoo.com.cn>, unbound-users at unbound.net > 日期: 2009年9月15日,周二,下午5:29 > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi Tao Ma, > > Just realized something else, the sbrk estimate is a dirty hack. > Gentoo may report a different sbrk() value, and the dirty hack > does not work. The sbrk() value keeps growing but maybe it is > not true that unbound's memory keeps growing. Does the 'top' or > 'ps' memory number also grow? > > Best regards, > Wouter > > On 09/15/2009 11:26 AM, W.C.A. Wijngaards wrote: > > Hi Tao Ma, > > > >> debug: cache memory msg=16528 rrset=33040 infra=672 val=24888 > > > > The values from your log file look very reasonable. No problems here. > > The sbrk keeps growing? > > > > Ok, it is possible to enable even more memory debug, configure with > > --enable-alloc-checks and compile again. Unbound prints: > > info: Memory conditions: %u front=%u back=%u mesh=%u msg=%u rrset=%u > > infra=%u iter=%u val=%u alloccache=%u globalalloccache=%u me=%u > > But with values. > > > > It also prints a log line for *every* alloc call. So, you can see > where > > in the code it allocates all that memory. This will make the > > server very slow. If you send me the log file (off-list to > > wouter at nlnetlabs.nl </mc/compose?to=wouter at nlnetlabs.nl> , then I > can parse it with testcode/memstats.c; > > or you can do that yourself: make memstats ; ./memstats < logfile ). > > > > To answer your question, unbound keeps some memory around, up to a > > maximum, because free-ing something that you need later again is > slow. > > This should top out at 40Mb for the standard config. > > > > Gentoo? Is it 64bit? Does it use a 'special' memory allocator in > libc? > > > > Best regards, > > Wouter > > > > On 09/11/2009 10:06 AM, Tao Ma wrote: > >> What's more,I found that the memory occupied by unbound would not be > >> freed when all requets finished,that means,unbound only consume > memory > >> but never free them. > >> What does unbound hold the memory for?It quite like that a > memory leak > >> happens... > >> Anyone else has been in the same trouble? > > > > > >> Best Regards, > >> Tao Ma > > > > > >> --- *09t911氓h聰, Tao Ma /<matao_reg at yahoo.com.cn > </mc/compose?to=matao_reg at yahoo.com.cn>>/* 聶S > > > > > >> 脩枚潞: Tao Ma <matao_reg at yahoo.com.cn > </mc/compose?to=matao_reg at yahoo.com.cn>> > >> ;聵: Re: [Unbound-users] Is there any memory leak in unbound? > >> 6枚潞: "W.C.A. Wijngaards" <wouter at nlnetlabs.nl > </mc/compose?to=wouter at nlnetlabs.nl>> > >> 聞: unbound-users at unbound.net > </mc/compose?to=unbound-users at unbound.net> > >> 氓: 2009t911氓,h聰, > > H11:50 > > > >> Hi Wouter,thanks very much for your reply. > > > >> I tried the method you mentioned,here is the info I got from the > log: > >> debug: cache memory msg=16528 rrset=33040 infra=672 val=24888 > >> This info keeps the same from the start of the unbound(I have only > >> about 5 records in unbound). > > > >> I also found the value of mem..total.sbrk printed by > "unbound-control > >> stats" keeps growing.The item "mem.total.sbrk" is an estimate of the > >> heap size of unbound in bytes,it is close to the memory used by > unbound. > > > >> I have a script keeps recording mem.total.sbrk in a file,and found > >> when mem.total.sbrk grew to 2017873920,unbound was killed by > >> system(My server has a 2G memory). > > > >> Unbound runs in a Gentoo Linux with 2.6.23 kernel,does the problem > >> relate to the OS? > > > >> Thanks & Best regards, > >> Tao Ma > > > > > > > >> --- *09t910氓h脹, W.C.A. Wijngaards /<wouter at NLnetLabs.nl > </mc/compose?to=wouter at NLnetLabs.nl>>/* > >> 聶S > > > > > >> 脩枚潞: W.C.A. Wijngaards <wouter at NLnetLabs.nl > </mc/compose?to=wouter at NLnetLabs.nl>> > >> ;聵: Re: [Unbound-users] 脼 Re: Is there any memory leak in > >> unbound? > >> 6枚潞: "Tao Ma" <matao_reg at yahoo.com.cn > </mc/compose?to=matao_reg at yahoo.com.cn>> > >> 聞: "Paul Wouters" <paul at xelerance.com > </mc/compose?to=paul at xelerance.com>>, unbound-users at unbound.net > </mc/compose?to=unbound-users at unbound.net> > >> 氓: 2009t910氓,h脹,H10:25 > > > >> -----BEGIN PGP SIGNED MESSAGE----- > >> Hash: SHA1 > > > >> Hi Tao Ma, > > > >> The memory statistics that unbound makes can help? > > > >> It prints them into the log file (depending on verbosity > >> level and --enable-debug to configure) like this: > >> debug: cache memory msg=1620127 rrset=2964258 infra=738306 > >> val=128832 > > > >> Also the memory statistics are printed when you enable extended > >> statistics and do unbound-control stats, like this: > >> mem.cache.rrset=2980222 > >> mem.cache.message=1633031 > >> mem.mod.iterator=16460 > >> mem.mod.validator=130126 > > > >> Best regards, > >> Wouter > > > > > >> > ------------------------------------------------------------------------ > >> }漏:aI`脩庐卤:ah掳 > > 驴 > >> > <http://cn.rd.yahoo.com/mail_cn/tagline/card/*http://card.mail.cn.yahoo.com/> > > > >> -----b:D枚聟鹿----- > > > >> _______________________________________________ > >> Unbound-users mailing list > >> Unbound-users at unbound.net > </mc/compose?to=Unbound-users at unbound.net> > </mc/compose?to=Unbound-users at unbound.net > </mc/compose?to=Unbound-users at unbound.net>> > >> http://unbound.nlnetlabs.nl/mailman/listinfo/unbound-users > > > > > >> > ------------------------------------------------------------------------ > >> }漏:aI`脩庐卤:ah掳 > > 驴 > >> > <http://cn.rd.yahoo.com/mail_cn/tagline/card/*http://card.mail.cn.yahoo.com/> > > > > > _______________________________________________ > Unbound-users mailing list > Unbound-users at unbound.net </mc/compose?to=Unbound-users at unbound.net> > http://unbound.nlnetlabs.nl/mailman/listinfo/unbound-users > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (GNU/Linux) > Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ > > iEYEARECAAYFAkqvXmkACgkQkDLqNwOhpPisoACeKatx2Pl4/B7k5MtJ86KTPBaR > 2WkAn0sd6bW9UDjCbdQ4pOLHW2WiFOg+ > =NrkI > -----END PGP SIGNATURE----- > > > ------------------------------------------------------------------------ > 好玩贺卡等你发，邮箱贺卡全新上线！ > <http://cn.rd.yahoo.com/mail_cn/tagline/card/*http://card.mail.cn.yahoo.com/> > ___________________________________________________________ 好玩贺卡等你发，邮箱贺卡全新上线！ http://card.mail.cn.yahoo.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://unbound.nlnetlabs.nl/pipermail/unbound-users/attachments/20090917/d4dd1c52/attachment.htm>