Configuring djb’s dnscache on Interworx Servers for Speed

Configuring djb’s dnscache on Interworx Servers for Speed

Mar 26

  • Created: Mar 26, 2012 4:35 PM

Configuring djb's dnscache on Interworx Servers for Speed

The control panel we use, Interworx, comes with djbdns as its DNS server. djbdns is actually a suite made up of several separate servers, the two main ones being tinydns for iterative (non-recursive) lookups and dnscache for recursive lookups. Other DNS servers, like BIND, support combining the iterative and recursive DNS servers in to one but djbdns requires them to be split apart for various reasons. This blog post is about improving DNS performance with dnscache but if you’ve never looked at tinydns, you should check it out, it is a nice lightweight DNS server that is easy to manage. It has its quirks but is still nice.

By default, when dnscache is set up, it acts as a recursive lookup server which is bad for performance. Every single server will walk the DNS tree to get a record for www.example.com. It will ask the root servers for the name servers of .com, then the .com servers for name servers of example.com, then the example.com for the A record of www.example.com. All of this takes less than a second usually and it will cache all of the results it gets but every single server is still incurring a delay to look up common records.

This is what the response distribution chart looked like for the first 200ms when 127.0.0.1 is doing its own recursive lookups, a higher percentage of responses in less time is better (generated using namebench):

Obviously having every server doing its own recursive lookups isn’t a good idea. It would be better to have all the servers send their DNS queries to one or two servers which would function as the cache.

Luckily dnscache supports this type of configuration, it is called a forwarding cache. If dnscache gets a request for something that isn’t in its cache, it will forward the request to a recursive lookup server, then cache the result when it comes in. The server doing the recursive lookup can be one provided by your data center or hosting company, one you run yourself, or some public DNS server. It’s really easy to reconfigure dnscache, just replace ’1.1.1.1′ and ’1.2.3.4′ with DNS server(s) of choice:


# put your recursive DNS server(s) in the file named @
echo '1.1.1.1' > /service/dnscache/root/servers/@
echo '1.2.3.4' >> /service/dnscache/root/servers/@

# configure dnscache so it acts as a forwarding cache
echo 1 > /service/dnscache/env/FORWARDONLY

# restart dnscache
svc -t /service/dnscache/

A while back I reconfigured all of our servers to forward their dns requests to two of our own servers. I then waited a few days to let the cache build up and ran namebench again. The results showed a good improvement from when every server operated as its own recursive DNS server, but it was still too slow.

I looked through namebench’s log to see what lookups our servers were slow on and checked the logs on the recursive servers. When there were slow lookups, it was because the recursive server was waiting on a response from another DNS server. There’s nothing I can do to speed up some one else’s nameserver and there’s nothing the other faster recursive servers are doing either, it’s mostly a matter of who has the best cache.

I started looking at what records namebench actually looks up, it uses a list called alexa-top-2000-domains.txt which confusingly enough has 33,575 domains in it instead of 2000. The alexa-top-2000-domains.txt is geared for what a home internet user will look up and not necessarily what a server will look up. For example, facebook.com is a popular website and is included in the Alexa list with www.facebook.com, apps.facebook.com, new.facebook.com, en-gb.facebook.com, etc. But severs don’t use those, they’ll use things like graph.facebook.com and api.facebook.com. The same thing applies to a lot of domains in Alexa’s list.

To come up list of domains that better matches what our servers actually look up, I generated a list of the top A records looked up by running the following on dnscache’s logs:

awk '$2 ~ /query/ && $5 ~ /^1$/ {freq[$6]++} END {for (x in freq) {print freq[x], x}}' /service/your-dnscache/log/main/@* | sort -rn

I pulled the top 30,000 A records from the previous day’s logs and put them in the format that namebench needs and re-ran namebench. After changing the tests to better match our server’s use we got much better results. The 5-20ms lead the recursive servers get over the other DNS servers, by virtue of being closer, proves very advantageous:

Posted in: Nexcess
  • David

    Hello Mark,

    I would like to know how to reconfigure dnscache through the Interworx interface? to then follow the steps you explain?