Network rules and secrets

Network rules

A sandbox may have its own rules for its traffic out: where on the internet it may connect, which HTTP requests it may send to a host, and which of its requests get headers added on the way. egress is taken when a sandbox is made, changed, or run once (POST /v1/run). Without it there are no rules. It is an object like this one, which lets the sandbox install Python packages and read repositories through GitHub’s API with a token it never sees, and nothing else (the secret GITHUB_TOKEN is below):

{"default": "deny",
 "allow": ["pypi.org", "files.pythonhosted.org"],
 "http": [{"host": "api.github.com", "allow": [{"methods": ["GET", "HEAD"], "paths": ["/repos/*"]}]}],
 "inject": [{"host": "api.github.com", "headers": {"Authorization": "Bearer ${secrets.GITHUB_TOKEN}"}}]}

• default: allow (the default) or deny, for where neither list names.
• allow, deny: up to 1,000 entries each: a host name (pypi.org), a wildcard (*.github.com, every name below it but not github.com itself), an address or a range (140.82.112.0/20). deny is looked at first, so it can cut holes in allow.
• http: up to 100 hosts, each with the methods and paths that may be sent to it; the node refuses the others (403 egress_denied).
• inject: up to 100 hosts, each with 1 to 20 headers added to the HTTPS requests the sandbox sends there. ${secrets.NAME} in a value is replaced by the secret’s value on the way out.
• deny_ports: up to 64 ports (1 to 65535) that no connection of the sandbox reaches, over TCP or UDP, wherever it goes and whatever the rest of the rules say. They are refused first, by the host’s packet filter, and never handed to the proxy. A Browser session has the mail and mining-pool ports closed so, for good.

glac sandboxes create --name agent --egress '{"default": "deny", "allow": ["pypi.org", "files.pythonhosted.org"]}'
glac sandboxes update agent --egress '{"allow": null}'      # empties one list, keeps the rest

--egress takes the JSON itself, a file that holds it, or - for standard input. A change replaces the lists it gives, empties those given as null, and keeps the others; "egress": null removes every rule.

Where they hold. The rules are enforced on the host, outside the machine: root in the sandbox cannot change them. Names are matched against what each connection says (the name in its TLS handshake, its HTTP Host), and DNS answers only names the rules allow, so with default: deny a sandbox reaches only what the rules allow. Some places no rule opens: private address ranges, the cloud’s metadata address, the host and other sandboxes.

Over IPv6. A sandbox with a public IPv6 address of its own (networking) is held to the same rules over it, names included: its TCP connections and DNS questions over IPv6 go through the proxy as those over IPv4 do, its machine answers AAAA questions as it answers A ones, and what the proxy connects for it leaves from the sandbox’s own address, counted against the same limits. Where the sandbox connects over IPv6 the proxy does too, and tries IPv4 to the same host beside it after 250 milliseconds, or at once where IPv6 is refused: the first to connect is used, so a program that would have fallen back to IPv4 by itself still gets through. A connection that gets through neither way, to an address its machine knows no name for, has a second to name its host in its first words, as a TLS handshake or an HTTP request does at once, and is then taken to that host; one that says nothing in that second, or names no host, is refused.

Connections through the proxy. Rules that name hosts send every TCP connection of the sandbox, over IPv4 and IPv6 alike, through its machine’s egress proxy, which holds the account’s sandboxes on that machine to a share of connections at once, by the plan, and to a pace of new ones (2,000 a second, which past it wait): see limits. Past the share, a new connection is refused, and may be tried again (503 egress_connections_full to a plain HTTP request).

What they do not stop. A host that serves many sites behind one certificate can be asked for another site after a handshake with an allowed name, and an allowed host that forwards traffic (a proxy, a store anyone can write) is a way out. Each entry of allow is that much trust. default: allow with a deny list stops software that means no harm; it is not a wall.

Secrets

A secret is a value kept by name and never shown again (/v1/secrets). It goes only to the hosts it allows (allowed_hosts): a rule or an image that would send it elsewhere is refused (403 secret_host_not_allowed).

glac secrets put GITHUB_TOKEN --allowed-host api.github.com    # asks for the value, not echoed
printf '%s' "$TOKEN" | glac secrets put GITHUB_TOKEN - --allowed-host api.github.com
glac secrets hosts GITHUB_TOKEN api.github.com '*.githubusercontent.com'

• In network rules: an inject header that names ${secrets.GITHUB_TOKEN} gets the value on the way out, so the program in the sandbox never holds it.
• As a registry login: a secret whose value is username:password pulls a private image (image_pull_secret).
• In a Cube: --secret VAR=NAME gives an app’s instances the value as an environment variable, which they do hold.

A project keeps a number of secrets by its plan, each of at most a size, and a sandbox is given at most so much of them together:

Storing, changing and deleting secrets needs a key with the admin scope. A key without it sees their names, never their values.