DNS-based language dictionary

This example shows how to create a simple language dictionary based on DNS service within 15 minutes. The translation will be performed using TXT resource records.

Key parts

Initialization

On init() module loads dictionary from a text file containing records in word [tab] translation format.

def init(id, cfg):
   log_info("pythonmod: dict init")
   f = open("examples/dict_data.txt", "r")
   ...

The suitable file can be found at http://slovnik.zcu.cz

DNS query and word lookup

Let’s define the following format od DNS queries: word1[.]word2[.] ... wordN[.]{en,cs}[._dict_.cz.]. Word lookup is done by simple dict lookup from broken DNS request. Query name is divided into a list of labels. This list is accessible as qname_list attribute.

aword = ' '.join(qstate.qinfo.qname_list[0:-4]) #skip last four labels
adict = qstate.qinfo.qname_list[-4] #get 4th label from the end

words = [] #list of words
if (adict == "en") and (aword in en_dict):
   words = en_dict[aword]

if (adict == "cs") and (aword in cz_dict):
   words = cz_dict[aword] # CS -> EN

In the first step, we get a string in the form: word1[space]word2[space]...word[space]. In the second assignment, fourth label from the end is obtained. This label should contains “cs” or “en”. This label determines the direction of translation.

Forming of a DNS reply

DNS reply is formed only on valid match and added as TXT answer.

msg = DNSMessage(qstate.qinfo.qname_str, RR_TYPE_TXT, RR_CLASS_IN, PKT_AA)

for w in words:
    msg.answer.append("%s 300 IN TXT \"%s\"" % (qstate.qinfo.qname_str, w.replace("\"", "\\\"")))

if not msg.set_return_msg(qstate):
    qstate.ext_state[id] = MODULE_ERROR
    return True

qstate.return_rcode = RCODE_NOERROR
qstate.ext_state[id] = MODULE_FINISHED
return True

In the first step, a DNSMessage instance is created for a given query (type TXT). The fourth argument specifies the flags (authoritative answer). In the second step, we append TXT records containing the translation (on the right side of RR). Then, the response is finished and qstate.return_msg contains new response. If no error, the module sets module_qstate.return_rcode and module_qstate.ext_state.

Steps:

  1. create DNSMessage instance
  2. append TXT records containing the translation
  3. set response to qstate.return_msg

Testing

Run the Unbound server:

root@localhost>unbound -dv -c ./test-dict.conf

In case you use own configuration file, don’t forget to enable Python module:

module-config: "validator python iterator"

and use valid script path:

python-script: "./examples/dict.py"

The translation from english word “a bar fly” to Czech can be done by doing:

>>>dig TXT @127.0.0.1 a.bar.fly.en._dict_.cz

; (1 server found)
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48691
;; flags: aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;a.bar.fly.en._dict_.cz.    IN  TXT

;; ANSWER SECTION:
a.bar.fly.en._dict_.cz. 300 IN  TXT "barov\253 povale\232"

;; Query time: 5 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Mon Jan 01 17:44:18 2009
;; MSG SIZE  rcvd: 67

>>>dig TXT @127.0.0.1 nic.cs._dict_.cz

; <<>> DiG 9.5.0-P2 <<>> TXT @127.0.0.1 nic.cs._dict_.cz
; (1 server found)
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58710
;; flags: aa rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;nic.cs._dict_.cz.      IN  TXT

;; ANSWER SECTION:
nic.cs._dict_.cz.   300 IN  TXT "aught"
nic.cs._dict_.cz.   300 IN  TXT "naught"
nic.cs._dict_.cz.   300 IN  TXT "nihil"
nic.cs._dict_.cz.   300 IN  TXT "nix"
nic.cs._dict_.cz.   300 IN  TXT "nothing"
nic.cs._dict_.cz.   300 IN  TXT "zilch"

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Mon Jan 01 17:45:39 2009
;; MSG SIZE  rcvd: 143

Proof that the unbound still works as resolver.

>>>dig A @127.0.0.1 www.nic.cz

; (1 server found)
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19996
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 5

;; QUESTION SECTION:
;www.nic.cz.            IN  A

;; ANSWER SECTION:
www.nic.cz.     1662    IN  A   217.31.205.50

;; AUTHORITY SECTION:
...

Complete source code

# -*- coding: utf-8 -*-
'''
 calc.py: DNS-based czech-english dictionary

 Copyright (c) 2009, Zdenek Vasicek (vasicek AT fit.vutbr.cz)
                     Marek Vavrusa  (xvavru00 AT stud.fit.vutbr.cz)

 This software is open source.
 
 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions
 are met:
 
    * Redistributions of source code must retain the above copyright notice,
      this list of conditions and the following disclaimer.
 
    * Redistributions in binary form must reproduce the above copyright notice,
      this list of conditions and the following disclaimer in the documentation
      and/or other materials provided with the distribution.
 
    * Neither the name of the organization nor the names of its
      contributors may be used to endorse or promote products derived from this
      software without specific prior written permission.

 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
 TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE
 LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
 SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
 INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
 CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
 ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 POSSIBILITY OF SUCH DAMAGE.
'''
import os
cz_dict = {}
en_dict = {}

def init(id, cfg):
   log_info("pythonmod: dict init")
   f = open("examples/dict_data.txt", "r")
   try:
      for line in f:
         if line.startswith('#'):
            continue
         itm = line.split("\t", 3)
         if len(itm) < 2: 
            continue
         en,cs = itm[0:2]

         if not (cs in cz_dict):
            cz_dict[cs] = [en]     # [cs] = en
         else:
            cz_dict[cs].append(en) # [cs] = en

         if not (en in en_dict):
            en_dict[en] = [cs]     # [en] = cs
         else:
            en_dict[en].append(cs) # [en] = cs

   finally:
      f.close()
   return True

def deinit(id):
   log_info("pythonmod: dict deinit")
   return True

def operate(id, event, qstate, qdata):
    if (event == MODULE_EVENT_NEW) or (event == MODULE_EVENT_PASS):

       if qstate.qinfo.qname_str.endswith("._dict_.cz."):
        
         aword = ' '.join(qstate.qinfo.qname_list[0:-4])
         adict = qstate.qinfo.qname_list[-4]

         log_info("pythonmod: dictionary look up; word:%s dict:%s" % (aword,adict))

         words = []
         if (adict == "en") and (aword in en_dict):
            words = en_dict[aword] # EN -> CS
         if (adict == "cs") and (aword in cz_dict):
            words = cz_dict[aword] # CS -> EN

         if len(words) and ((qstate.qinfo.qtype == RR_TYPE_TXT) or (qstate.qinfo.qtype == RR_TYPE_ANY)):

            msg = DNSMessage(qstate.qinfo.qname_str, RR_TYPE_TXT, RR_CLASS_IN, PKT_RD | PKT_RA | PKT_AA)
            for w in words:
                msg.answer.append("%s 300 IN TXT \"%s\"" % (qstate.qinfo.qname_str,w.replace("\"","\\\"")))

            if not msg.set_return_msg(qstate):
               qstate.ext_state[id] = MODULE_ERROR 
               return True

            qstate.return_rcode = RCODE_NOERROR
            qstate.ext_state[id] = MODULE_FINISHED 
            return True

         else:
            qstate.return_rcode = RCODE_SERVFAIL
            qstate.ext_state[id] = MODULE_FINISHED 
            return True

       else: #Pass on the unknown query to the iterator
         qstate.ext_state[id] = MODULE_WAIT_MODULE 
         return True

    elif event == MODULE_EVENT_MODDONE: #the iterator has finished
         #we don't need modify result
         qstate.ext_state[id] = MODULE_FINISHED
         return True

    log_err("pythonmod: Unknown event")
    qstate.ext_state[id] = MODULE_ERROR
    return True

def inform_super(id, qstate, superqstate, qdata):
   return True