RDAP, obtaining information on a domain name

What is RDAP?

Before RDAP, whois was the star solution for automated access to information on domain names. The whois protocol, standardised in RFC 3912, is very simple, and for this reason writing a whois client or server is an exercise sometimes set by programming teachers. But this simplicity also has its disadvantages:

• There is no standard mechanism for finding the whois server for a given domain name; each whois client uses its own heuristics², with varying degrees of success.
• There is no standard mechanism for indicating search options to the server.
• There is no confidentiality, everything is in plain text.
• No authentication is possible, apart from the source IP address; it is not easy to differentiate access by user.
• There is no internationalisation, since it is not possible to indicate the set of characters and encoding used³.
• The data are not structured, it’s simply free text, which makes analysing it with a program very difficult⁴.

RDAP resolves all these problems. It relies on HTTPS (Hypertext Transfer Protocol Secure) techniques, with the REST (REpresentational State Transfer) principles and the JSON (JavaScript Object Notation) format:

• There is a standard mechanism for discovering servers, from an IANA registry⁴.
• Options can be indicated in the URL.
• Thanks to the use of HTTPS, everything is encrypted.
• And we can use the various authentication techniques of HTTP, such as the countless REST APIs in existence.
• The set of characters and its encoding are standardised (Unicode and UTF-8, naturally).
• Thanks to the use of JSON, the data are structured and much easier to analyse.

RDAP is standardised in RFC 9082 and RFC 9083.

Since this article is designed mainly for programmers, we’re going to talk about code. The programs will be presented only rather superficially, the complete code being available online. We’re going to use the Python programming language, given how simple it is to read, but of course any programming language that allows you to make HTTPS requests and to analyse JSON, in other words any programming language currently in use, can be used to develop an RDAP client⁶.

The three programs presented depend on the same library, which is in the file ianardap.py. Every RDAP client must start by knowing which RDAP server serves the domain we’re interested in. Unlike with whois, there’s a standardised method for this: downloading the JSON file containing the list of RDAP servers from IANA^7-8. Using the excellent Python requests library to make the HTTP request, this translates to a simple:

response = requests.get(“https://data.iana.org/rdap/dns.json”)

You then analyse this file with the json module of the standard Python library:

database = json.loads(content)

self.version = database["version"]

self.services = {}

for service in database["services"]:

    for tld in service[0]: # Plusieurs TLD peuvent partager le même serveur

         for server in service[1]:

            self.services[tld] = server

The IanaRDAPDatabase class provides a ‘find’ function for locating the server of a given domain name:

% python

>>> import ianardap

>>> base = ianardap.IanaRDAPDatabase()

>>> base.find("toto.fr")

'https://rdap.nic.fr/’

Armed with this module, we can go on to the useful programs. Three examples are given hereunder.

Supervising expiration

Our first example deals with a common problem: how to monitor name expiration. In many registries, domain names expire if they are not regularly renewed. Sometimes the holder forgets to renew the name, which is then deleted, with the obvious consequences. It is therefore crucial to monitor upcoming domain expirations⁹, and this can be automated with tools such as Icinga or Zabbix. To monitor domains in various different TLDs with whois, we need to know the formats of the various registries and to analyse dates in a number of different formats. Things are simpler with RDAP: you make an RDAP request and look for the expiration event in the ‘events’ member of the returned JSON object:

response = requests.get("%s/domain/%s" % (server, domain))

rdap = json.loads(response.content)

for event in rdap["events"]:

    if event["eventAction"] == "expiration":

        expiration = datetime.datetime.strptime(event["eventDate"], RFC3339)

We will now look at whether this date is too close, the calculations being made with the standard Python library’s ‘datetime’ module:

now = datetime.datetime.now(tz=UTCINFO)  

rest = expiration - now

if rest < critical:

     print("CRITICAL: domain %s expires in %s, HURRY UP!!!" % (domain, rest))

     sys.exit(STATE_CRITICAL)

An example of use from the command line would be:

% ./check_expiration.py -H quimper.fr  

quimper.fr OK: expires in 14 days, 9:43:40.111753.

Or, for a more neglected domain:

% ./check_expiration.py -H nic.abogado

nic.abogado CRITICAL: domain nic.abogado is already expired (34 days, 8:22:36.758468 ago)

All this can then be used in an automatic supervision program (the example given uses Nagios’ API, which is common to a number of supervision programs).

Obtaining information on the holder’s country

Now let’s turn to something else – the name holder’s country. The problem is more complicated than it seems because although RDAP standardises the format, it doesn’t standardise the data sent: not all registries distribute this information. In the case of the .fr domain, it is given only for legal persons, information on natural persons not being distributed by default¹⁰. The program must therefore be ‘defensive’ and prepared not to obtain everything:

response = requests.get("%s/domain/%s" % (server, urllib.parse.quote(domain)))

rdap = json.loads(response.content)

if "entities" not in rdap:

    error("No entities in RDAP response")

if rdap["entities"] is not None:

    for entity in rdap["entities"]:

        if "registrant" in entity["roles"]:

            if "vcardArray" in entity:

                for item in entity["vcardArray"][1]:

                    if item[0] == "adr":

                        country = item[3][6]

And then the corporate data concerning an entity (here, the holder) are not easy to analyse. RDAP uses the jCard¹¹ format, which is a version of the very old vCard¹² format used in all address books, adapted to JSON. Several important items of information are in fixed size arrays (such as the address, which must necessarily be in a six-element array), rather than in dictionaries. The content of the elements is not always uniform either. For example, whereas vCard requires the country to be indicated in letters, in natural language, most RDAP servers send a two-letter code taken from ISO 3166¹³. Here are some examples of results from the above program:

% ./country.py ssi.gouv.fr

FR

% ./country.py amazon.fr

LU

% ./country.py durand.fr 

durand.fr: No country found for durand.fr, may be restricted access for privacy reasons

In the first case, we see that the holder of the ANSSI domain is in France, logically enough. In the second case, the holder is in a tax haven. In the third case, the holder is a natural person and no information on his or her country has been provided. Let’s look outside France:

% ./country.py guaranifc.com.br

BR

The holder of this football club’s domain is in Brazil¹⁴.

Checking the consistency of a domain

One of the main security risks linked to domain names is that of the misuse of the name, as in the notorious case of Middle-Eastern governments’ domain names in 2018-2019. There are several ways to limit the risks¹⁵, but in any case it is prudent to monitor one’s critical domains and to make sure they haven’t changed at the level of the registry¹⁶. RDAP can be used to automatically compare the theoretical list of domain name servers with the information in the registry, so as to be alerted in the event of change.

In such a case, we ask the user to provide a list of the expected name servers in the form of a JSON object, and then send a RDAP request. We construct a Python dictionary indexed by the names of the servers:

r = requests.get(“%s/domain/%s” % (server, urllib.parse.quote(domain)))

nameservers = {}

response = json.loads(r.text)

for ns in response[“nameservers”]:

nameservers[ns[“ldhName”]] = []

for family in [“v4”, “v6”]:

if family in ns[“ipAddresses”]:

for addr in ns[“ipAddresses”][family]:

nameservers[ns[“ldhName”]].append(addr)

And we compare the list returned by RDAP (nameservers) with the expected list (delegation), the one provided by the user:

for ns in nameservers:

if ns not in delegation:

error += “Name server %s should not be there\n” % ns

else:

if nameservers[ns] is not None:

for addr in nameservers[ns]:

if delegation[ns] is None or addr not in delegation[ns]:

error += “Glue record %s should not be there for server %s ” % (addr, ns)

for ns in delegation:

if ns not in nameservers:

error += “Name server %s is missing\n” % ns

else:

if delegation[ns] is not None:

for addr in delegation[ns]:

if nameservers[ns] is None or addr not in nameservers[ns]:

error += “Glue record %s is missing for server %s ” % (addr, ns)

An example of use¹⁷ from the command line (but of course in practice it will often be launched from a supervision program):

Some final remarks

Limitation of traffic

An RDAP request may entail a lot of work for the server. To prevent data gluttons from making excessive volumes of RDAP requests, many RDAP servers impose limits on traffic. Your programs must therefore behave reasonably and react appropriately if they receive an HTTP 429¹⁸ response status code indicating too many requests.

Deployment of RDAP in TLDs

As yet, by no means all TLDs have an RDAP server. Apart from those under contract with ICANN, which are obliged to deploy RDAP servers, only a few country TLDs, such as .fr, have an RDAP server.

^{1 – Note that online interfaces for accessing information are sometimes wrongly referred to as whois, whereas in fact they have nothing to do with whois. This leads to frequent confusion. Thus we hear people talking about the “whois database” or about “changing the whois”, neither of which make any sense.}

^{2 – For example, GNU-whois has a predefined list of servers, managed manually by the author of the software application, and FreeBSD’s whois consults a list distributed by the DNS in the whois-servers.net domain.}

^{3 – In practice, many clients and servers use Unicode and UTF-8 encoding, which works… usually.}

^{4 – Note however that there are free software libraries such as Net::DRI for the Perl language, which considerably simplify this task.}

^{5 – It is standardised in RFC 9224.}

^{6 – Users of the Unix shell will be able to do lots of things with curl and jq, for example.}

^{7 – The module remembers this file, taking account of the IANA HTTP indications to avoid overloading its server.}

^{8 – If you wish to short-circuit the registry server, or if the TLD doesn’t have RDAP but the registrar does, as noted at the end of this article, IANA also has a file containing all the registrars accredited with ICANN and indicating their RDAP server.}

^{9 – It is of course recommended to activate automatic renewal. However, belt and suspenders is better. Renew and monitor.}

^{10 – And this is not because of the GDPR; it was already the case long before this regulation was issued.}

^{11 – Standardised in RFC 7095. The IETF (Internet Engineering Task Force) has a plan to replace jCard with JSContact.}

^{12 – Standardised in RFC 6350.}

^{13 – This seems more reasonable, since these codes are international and unambiguous.}

^{14 – Some ccTLDs require the holder to be resident in the country, while others require residence in a more extensive space such as the European Union and yet others impose no requirements.}

^{15 – Such as locking the name, a very effective method recommended by the ANSSI in its guide to good practices for the acquisition and exploitation of domain names.}

^{16 – Note that this only protects against certain misuses, those where the attacker alters the list of name servers. If a successful attack has targeted the DNS hoster for example, the attacker will perhaps change only some IP addresses or MX records, without interfering with the registry’s database.}

^{17 – One important point with a program like this: if you replace the list of name servers, or their IP addresses with those in the domain served, you will have to alter the parameters of the supervision program. So if you use a program like this you’ll have to be rather rigorous in managing the domain.}

^{18 – Standardised in RFC 6585.}