Dataface Dataface

DatafaceResearch, our research bot

Who runs the DatafaceResearch user agent, what it fetches, the rules it keeps and how to ask it to stop.

If you have seen this user agent in your server logs, this page explains it:

DatafaceResearch/1.0 (+https://dataface.uk/bot)

Who runs it

DatafaceResearch is run by Dataface Ltd, a software company in Glasgow, registered in Scotland (company number SC631408). It is not a search engine, and it does not collect material to train AI models.

What it does

It supports editorial research. It checks public news feeds and public web pages for new material on the subjects we research and write about for the businesses we work with. A person then decides what is worth reading and citing. Anything we cite links back to the original page.

It does not:

  • log in, get round a paywall or submit forms;
  • collect email addresses or other personal details for marketing;
  • pretend to be a web browser, or use any other name;
  • train AI models on what it reads.

The rules it keeps

  • robots.txt. It follows your robots.txt for the name DatafaceResearch, or the rules for all robots (*) if there is no section for us, read as RFC 9309 describes. It reads your robots.txt again at least once a day.
  • Refusals. If your server refuses our request for robots.txt (401 or 403), or cannot answer it, we treat the whole site as closed to us until it can.
  • AI opt-outs. If your robots.txt blocks well-known AI crawlers by name, or your pages carry a noai directive, we treat that as a request to keep your text away from AI tools, even though our name is not on your list. We then do not pass your text to any AI tool.
  • Text and data mining reservations. We honour TDMRep (the tdm-reservation header and meta tag, and /.well-known/tdmrep.json), X-Robots-Tag directives and Content-Signal lines.
  • Payment required. If a site answers 402, we stop. We never try another route to the same content.
  • Pace. At least 5 seconds between requests to the same site, longer if your robots.txt sets a Crawl-delay, and only one request at a time. Each feed is checked no more than once every three hours.
  • Back-off. A 429 or 503 response pauses all our requests to your site for as long as Retry-After asks, up to 24 hours.
  • Conditional requests. We send If-Modified-Since and If-None-Match, so an unchanged page costs you a short 304 response.
  • Where requests come from. Our own network in the UK. We never use a VPN, a proxy service or rotating addresses.

What we keep

For open-licence sources (for example, material under the Open Government Licence) and sites that allow it, we may keep a copy. For everything else we keep the link, the headline, the date, our own short summary and no more than three short quotes, each credited to you.

How to ask us to stop

Email info@dataface.uk with your site’s address. We will stop fetching it within two working days and reply to confirm.

Or block us in your robots.txt. We read it again at least once a day, so the block takes effect within 24 hours, and you do not need to contact us:

User-agent: DatafaceResearch
Disallow: /

How we use AI more widely

How we use AI in our own work and in the systems we build is set out in How we use AI responsibly.

Contact

Email info@dataface.uk. Dataface Ltd, 6th Floor, Gordon Chambers, 90 Mitchell Street, Glasgow G1 3NQ.