Hand writing DNS queries | Web from scratch pt. 1

Click here to skip to the tutorial.

About this series

In this tutorial series, I will show how to build a simple web browser from start to finish, assuming only the existence of an operating system, sockets, and an ability to draw to the screen. The rest we will fill in ourselves. This web browser will be very basic and unable to interpret CSS, JavaScript, or really any kind of styling, but it will illustrate how the internals of a web browser function -- and be able to fetch and render the page you are currently reading.

This will be done in C++, but the reader may follow along in any language. This tutorial is meant for people who have at least some experience in HTML and web development.

An understanding of the following things is assumed before this tutorial proceeds. Links are included to read about the topics if you want to learn them.

• An intuitive understanding of DNS
• Knowledge of sockets, TCP, and UDP

• An understanding of basic HTML
• What is a web server

It may be beneficial to periodically refer to the official document, RFC 1035, to clear up any questions you may have and to learn more about DNS.

Outline of this post:

1. Structure of DNS requests
2. Crafting a request by hand
3. Making a real request
4. Structure of response
5. Interpreting the response
6. Next steps

Messages

Term: The RFC defines on page 25 a message, the structure which all DNS messages, whether requests or responses, must follow.

Each message is divided into 5 sections, all of which (except for the header) can be empty.

1. Header
2. Question: the question for the name server
3. Answer: records answering the question
4. Authority: records pointing towards an authoritative name server
5. Additional: records that may relate to a query but are not strictly answers to a question

In this tutorial, we will only concern ourselves with sections 1 (header), 2 (question), and 3 (answer). When crafting requests, we will be using the first two sections, and when parsing respones, we will use all three sections.

I like to learn by example, so let's start by crafting a request to find the IP of google.com.

Typical requests to get an IP from a domain name include only a header and a question (from the message format shown above.

The header (skip)

DNS is not an ASCII-based format like HTTP, but binary. While this makes it easy for computers to understand, it unfortunately requires some extra effort for us humans. Regardless, we've just gotta suck it up and read the spec.

To construct the header, we will refer to the following table given on page 26 of the RFC 1035:

                                    1  1  1  1  1  1
      0  1  2  3  4  5  6  7  8  9  0  1  2  3  4  5
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                      ID                       |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |QR|   Opcode  |AA|TC|RD|RA|   Z    |   RCODE   |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                    QDCOUNT                    |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                    ANCOUNT                    |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                    NSCOUNT                    |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                    ARCOUNT                    |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+

Each "row" on this table, (e.g. ID, QDCOUNT, NSCOUNT) contains 16 bits, shown as columns on the top row from 0 to 15. There are 6 rows, so our header will be 96 bits, or 12 bytes. Row 2 contains many smaller flags that are varying widths.

The header can be a little confusing because the exact same format is used for requests and responses, so some fields may not have meaning in a request and should simply be set to 0.

Try and skim the sections below, but don't stress too hard. We'll come back to all this later when we actually construct the request.

Also, you are not expected to know what everything means in the below list. Don't worry about it, we'll do an example.

Let's go through the rows one by one:

• ID This is a 16 bit identifier. We can set this to whatever we want. When the server sends back its response, it marks it with the same exact ID we put here. This can be useful to match up requests if we make multiple at once, especially since we're going to be doing our DNS queries over UDP.
• Flags (row 2):
  • QR 0 if request, 1 if response.
  • Opcode What type of query we want to make. 0 is a standard query, and that is all this tutorial will care about. For more query types, visit page 26 of the RFC 1035.
  • AA Has no meaning in requests, should be set to 0. In a response, this bit indicates whether or not the server responding is an authority for the domain requested.
  • TC Specifies if the message was truncated due to being too long. If it has, we can listen for additional messages to get the rest of the data.
  • RD "Recursion desired". In many DNS queries, the first server we ping may not have the IP itself, but may know of another nameserver that does. If we set this bit in a request, it indicates whether or not we want the server to forward along the request to the next nameserver recursively until it finds an answer. If the bit is not set, then it will just tell us the next nameserver to look toward. In the response header, this bit is set exactly to whatever it was in the request header.
  • RA "Recursion available". This bit has no meaning in a request, so just set it to 0. In a response, this indicates whether or not recursive queries (as described above) are supported on the name server. Not all nameservers support recursive queries.
  • Z Reserved for future use. Just set it to 0 always.
  • RCODE No meaning in requests, only valid in responses. Can be any of the following values:
    • 0 - no error
    • 1 - format error (we screwed up formatting our request
    • 2 - server failure (our request is fine, something messed up with the server)
    • 3 - name error (only valid for authoritative name servers, means the domain we requested does not exist)
    • 4 - not implemented (the server does not support our opcode)
    • 5 - refused (the server could answer your request, but does not want to. maybe you need permission or the server is just angry)
    • 6-15 - reserved for future use
• QDCount How many question sections are present (unsigned 16 bit)
• ANCount How many answer sections present (unsigned 16 bit)
• NSCount How many name server records present in the authority records section (unsigned 16 bit)
• ARCount How many resource records present in the additional resources section (unsigned 16 bit)

Let's go from left to right. Our first two bytes, the ID, are arbitrary (why?). I'll pick 0xDEAD. For our flags, we'll let:

Most of these are unimportant, so I've highlighted the ones you should care about in blue.

To put our flags all together, we just concatenate the bits. So 0000 0001 0000 0000 is our flags.

The "Count" sections.

As for QDCount, ANCount, NSCount, and ARCount, all we'll be sending is one question record and nothing else, so QDCount (number of questions) should be 1 and everything else 0. Each of those are 16-bit, so:

Finally done!

In conclusion, our header is:

All put together, 1101 1110 1010 1101 0000 0001 0000 0000 0000 0000 0000 0001 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 for a total of 12 bytes.

In hex, this is 0xDEAD01000001000000000000

The question section

We specified in the header that we're sending 1 question (in QDCount), so let's build it.

Each question corresponds to one name request. We're going to make an A record request for google.com.

Luckily, the question section is much easier than the header section. Here is the table given by the RFC (again, each column is 1 bit):

                                    1  1  1  1  1  1
      0  1  2  3  4  5  6  7  8  9  0  1  2  3  4  5
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                                               |
    /                     QNAME                     /
    /                                               /
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                     QTYPE                     |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
    |                     QCLASS                    |
    +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+

... where QName is the domain name, QType is the type of record we want, and QClass is what "kind" of query we are making.

QType lets you pick between A, CNAME, NS, and the like types of DNS lookups. We'll do an A lookup, so QType = 1 (by the RFC 1035, page 11).

QClass lets you pick whether you are doing an internet, CSNET, Chaosnet, or Hesoid lookup. Obviously, we are going to do an internet lookup, so QClass = 1 (by the RFC 1035, page 12). If you're curious like I am, go have fun looking the rest of those up.

QName

I've given QName its own section because it's a bit more involved - and exciting. This is where we tell 'em which URI we want to look up, aka google.com. However, instead of either specifying a string length or just sending it over null-terminated as most everyone with a brain stem would do, they decided to combine the two. Here's how we have to encode google.com:

    0x06        'google'        0x03       'com'         0x00
------------------------------------------------------------------
 6, length                    3, length               end of QName
 of 'google'                  of 'com'

It seems a bit odd, but it makes a lot of sense when you consider the true structure and purpose behind each component of the hostname.

As you can see, you just encode each 'section' of the URL one at a time, by giving a length of that specific section, then the string, then the next length, and so on. To end the QName, you just toss in a null byte at the end.

If you wanted to do something like www.google.com, it'd be the same as above but with a 0x03 and then a www at the beginning. Clearly, only ascii-characters can be used in the strings, as each letter is exactly 1 byte.

Let's get to encoding.

In binary, the string 'google' is 0110 0111 0110 1111 0110 1111 0110 0111 0110 1100 0110 0101, and the string 'com' is 0110 0011 0110 1111 0110 1101 (note: these strings are case insensitive).

Putting together our whole QName from the table above gives:

[0000 0110] [0110 0111 0110 1111 0110 1111 0110 0111 0110 1100 0110 0101] [0000 0011] [0110 0011 0110 1111 0110 1101] [0000 0000] (brackets added for clarity)

In hex, this is 0x06676F6F676C6503636F6D00.

Finishing up the question section

We have our QName, QType (0000 0000 0000 0001), and QClass (0000 0000 0000 0001). To finish our question section, we just concatenate 'em as shown in the table to get:

0x06676F6F676C6503636F6D0000010001

Putting it all together

To combine our header with our question, all we have to do is concatenate 'em.

If you remember, our header is 0xDEAD01000001000000000000 and our request body/question is 0x06676F6F676C6503636F6D0000010001.

So, our final request is 0xDEAD0100000100000000000006676F6F676C6503636F6D0000010001.

You've gotta hand it to the RFC, they really know how to make their specs human-friendly.

Finally! We can make our request. Here's the code:

#include <iostream>
#include "connection.h"

#define REQUEST_LENGTH 28

int main(int argc, char** argv) {

	/* Load in our request data */
	unsigned char request[REQUEST_LENGTH] = {
		0xDE, 0xAD, 0x01, 0x00,
		0x00, 0x01, 0x00, 0x00,
		0x00, 0x00, 0x00, 0x00,
		0x06, 0x67, 0x6F, 0x6F,
		0x67, 0x6C, 0x65, 0x03,
		0x63, 0x6F, 0x6D, 0x00,
		0x00, 0x01, 0x00, 0x01
	};

	/*
	 * Connect to the DNS server.
	 * 8.8.8.8 is a very common DNS server owned by Google
	 * 53 is the port we are using
	 * and false = UDP (true = TCP)
	 */
	connection conn("8.8.8.8", 53, false);

	conn.send((const char*)request, REQUEST_LENGTH);

	char response[4096];
	int chars_read = conn.recv(response, 4096); //Get response back

	std::cout << "Received " << chars_read << " character(s) from the server." << std::endl;

	/* Print out our response */
	std::cout << "0x";
	for(int i = 0; i < chars_read; i++) {
		std::cout << std::hex << (0xff & response[i]);
	}
	std::cout << std::endl;
}

As you can see, I've constructed a connection class to abstract away all of the hard-to-read c sockets that would otherwise be present.

Don't worry though, I'm not hiding any magic, as it is just standard boilerplate sockets code. If you'd like to see, here's the code on Github.

Hopefully, everything else is self-explanatory in the above code. Go read through it and make sure it makes sense.

It might also be helpful to fetch my connection.cpp and connection.h from this link, and play around with the above code yourself. Try requesting another URL, or using another DNS server.

The response

After running this on my machine, I get:

Received 44 character(s) from the server.
0xdead8180010100006676f6f676c653636f6d00101c0c01010012b04acd9c4e