Designing an API gateway on a laptop

With everything in my local stack behind Caddy already, the laptop was the obvious place to try a gateway in front of the API.

Only the routes listed in one file get through, and everything else answers 404, which means an empty list is a host with nothing on it.

A request that needs a token goes through the following steps, where auth is the backend service that handles sign-in.

flowchart TD
  accTitle: Path of a request that needs a token
  browser["Browser"] -->|"https://portal.localhost/api/v1/orders/42"| caddy["Caddy"]
  caddy -->|"route not listed"| notfound["404"]
  caddy -->|"route listed"| auth["auth checks the token"]
  auth -->|"no token, bad token or wrong type"| refused["401 or 403"]
  auth -->|"valid: strip /api, set the identity and client IP headers"| api["api, a container or a port on your machine"]

So all we need to do to expose a route is add it to the list.

groups:
  webhooks:
    public: true
    routes:
    - POST /webhooks/payments
  signed-in:
    routes:
    - GET /v1/orders/{id}
    - ANY /v1/admin/*
surfaces:
- {name: public-api, hosts: [api.localhost], type: user, groups: [webhooks]}
- {name: portal, hosts: [portal.localhost], mount: /api, type: user, groups: [signed-in]}

A surface puts groups on a host, under a mount if it has one, and the service never sees the mount.

Listed routes need a valid token unless the group says public: true, and stack renders each protected one into the following forward_auth block.

forward_auth auth:8080 {
    uri /auth/verify?type=user
    copy_headers X-Auth-User X-Auth-Type X-Auth-Tenant X-Auth-Session
    header_up -Via
}

A lot of problems turned up while building it here, long before production.

The first version of the stack had an API hostname for each brand, which exposed the whole API with no gateway in front, so the allowlist could just be skipped, and they had to go.
The API saw the proxy’s address as every client’s address, which matters because its rate limits depend on it, and a client could forge the client IP header anyway, so the gateway now sets it and removes any others.
Answers had no Cache-Control, so they’re no-store now, and a public route’s service can override that.
A public and a protected route can match the same request, and the first match wins, so that could open a protected route by accident, and now the render fails instead.
Client-supplied X-Auth-* headers have to be removed on every request, or anyone can just claim to be someone.
Some paths carry tokens and email addresses, so the access log records the decision and the route, and never the real URL.

stack smoke, the end-to-end check, now tries path tricks against the allowlist, and I’d recommend that to anybody with a gateway.

When a request doesn’t do what you expected, stack gateway explain says what the gateway would do with it, without sending it.

Let’s take a look at a protected route, you can see which route matched, who checks the token, and what the service ends up with once the /api mount is stripped.

$ stack gateway explain GET https://portal.localhost/api/v1/orders/42
surface   portal (mount /api)
route     GET /v1/orders/{id} (group signed-in)
access    token of type user, checked by auth
refused   401 no or bad token, 403 wrong type or brand; the service never sees it
result    api gets GET /v1/orders/42 (image)

The (image) at the end says how the service is running.

A public route skips the token, and the output says so.

$ stack gateway explain POST https://api.localhost/webhooks/payments
surface   public-api (mount /)
route     POST /webhooks/payments (group webhooks)
access    public; client X-Auth-* headers are removed
result    api gets POST /webhooks/payments (image)

And if the route isn’t in the list, the request stops at the gateway.

$ stack gateway explain GET https://portal.localhost/api/v1/invoices
surface   portal (mount /api)
route     none listed for GET /v1/invoices
result    404 from the gateway; api never sees it (X-Gateway-Result: not-listed)

The command needed a fix of its own, because it matched paths differently from the proxy, so it could tell you something the proxy wouldn’t do.

Now it cleans the path first, the same way the proxy does, so if you try a path trick you can see where the request really lands, which here is a different route from the one the URL starts with.

$ stack gateway explain GET https://portal.localhost/api/v1/orders/../admin/users
surface   portal (mount /api)
route     ANY /v1/admin/* (group signed-in)
access    token of type user, checked by auth
refused   401 no or bad token, 403 wrong type or brand; the service never sees it
result    api gets GET /v1/admin/users (image)

Same story with the echo service that shows what a request looks like after the gateway, it started at a fixed /_echo path, which only proved that /_echo worked.

So now stack gateway attach <url> sends one real URL to echo, through the real mount, allowlist and token check, and only on your machine.

What isn’t done

Timeouts and body limits per route group aren’t there yet, and in production behind a CDN the gateway will have to trust it and take the client address from the header it sets, which is the opposite of what it does locally.