A simple HTTP client/server (HTTP/1.1 & h2) with Miou

httpcats (HTTP + cats because miou) is an implementation of an HTTP client and server (HTTP/1.1 & h2) in pure OCaml. This implementation is based on the miou scheduler, ocaml-dns (for domain name resolution), happy-eyeballs (to manage connections), ocaml-tls (for TLS protocol) & mirage-crypto (for cryptography), ca-certs to obtain system certificates and h1 and h2 to implement HTTP protocols. In all, httpcats requires 58 packages (including dune & ocamlfind) for a single installation.

U: That's a lot of packages!

That's what's needed to end up with a pure OCaml http client. curl, for example, has 13 dependencies and also contains implementations such as ftp or smtp that are not related to an http client. A comparison would therefore be difficult, you just have to choose your poison (OCaml or C?).

U: However, there are other implementations of HTTP client & server in OCaml. Why implement it yet again?

These implementations don't use miou, however. What's more, since http-lwt-client, we're opposed to the (ultimately complex) feature of being able to choose the TLS implementation (although we understand the constraints some users may have in wanting to use OpenSSL) and prefer to offer an HTTP client that uses strictly ocaml-tls. Finally, we also want to have control over domain resolution, rather than having to use the system's resolver.

---

1: Here, we point the finger at the software components Conduit and Gluten, which perform dynamic and/or static dispatching of TLS layer implementations that we do not find suitable.

U: So how does httpcats work?

You need to initialize the random number generator required by mirage-crypto and ocaml-tls and make your request like this:

let fn _meta _req _resp () = function
  | Some str -> print_string str
  | None -> ()

let () = Miou_unix.run @@ fun () ->
  Mirage_crypto_rng_unix.use_default ();
  ignore (Httpcats.request ~fn ~uri:"https://robur.coop/" ());

It's quite... simple. You can, of course, make POST requests, consume the response body in a more complex way (store it in a buffer, for example), process the received response and lots of other things like:

• forcing the use of a version of the HTTP protocol
• define your own TLS configuration
• accept certain certificates (such as self-signed ones)
• follow or not follow redirects
• resolve domain names via happy-eyeballs & ocaml-dns

U: What about the server?

You can also have an HTTP/1.1 and h2 server (with TLS and a certificate you can handle with x509). As an example, here's a simple HTTP/1.1 server:

let text = "Hello World!"

let[@warning "-8"] handler _ (`V1 reqd : [ `V1 of H1.Reqd.t | `V2 of H2.Reqd.t ]) =
  let open H1 in
  let request = Reqd.request reqd in
  match request.Request.target with
  | "" | "/" | "/index.html" ->
      let headers =
        Headers.of_list
          [
            ("content-type", "text/plain; charset=utf-8")
          ; ("content-length", string_of_int (String.length text))
          ]
      in
      let resp = Response.create ~headers `OK in
      let body = Reqd.request_body reqd in
      Body.Reader.close body;
      Reqd.respond_with_string reqd resp text
  | _ ->
      let headers = Headers.of_list [ ("content-length", "0") ] in
      let resp = Response.create ~headers `Not_found in
      Reqd.respond_with_string reqd resp ""

let server sockaddr = Httpcats.Server.clear ~handler sockaddr

let () =
  let sockaddr = Unix.(ADDR_INET (inet_addr_loopback, 8080)) in
  Miou_unix.run @@ fun () ->
  let domains = Miou.Domain.available () in
  let prm = Miou.async @@ fun () -> server sockaddr in
  if domains > 0
  then Miou.parallel server (List.init domains (Fun.const sockaddr))
       |> List.iter (function Ok () -> () | Error exn -> raise exn);
  Miou.await_exn prm

Again, it's pretty straightforward. This server takes the opportunity to use all your cores thanks to miou. You can also run the program with a specific number of domains:

$ ocamlfind opt -linkpkg -package digestif.c,httpcats server.ml
$ MIOU_DOMAINS=2 ./a.out

Benchmarks

Some contributors to the OCaml community wanted to benchmark different HTTP implementations in OCaml. You can find more details here. As for httpcats, a benchmark was developed and proposed here.

We decided to propose a protocol which describe how we benchmark OCaml HTTP implementations. Everything is described here: PROTOCOL.md. Results are also available here: BENCH.md and a view of these results are available here: Benchmarks.