diff --git a/network-go/implementation.md b/network-go/implementation.md index 463c516..1b8f248 100644 --- a/network-go/implementation.md +++ b/network-go/implementation.md @@ -2,7 +2,8 @@ A Go-based replacement for the `firewall/firewall-add` shell script. It watches `/etc/user/config/networks.json` for changes and reconciles Docker networks, -container connections, and iptables firewall rules using the Docker SDK. +container connections, and iptables firewall rules — all via the Docker SDK +and `nsenter` (no Docker CLI calls). ## Project Structure @@ -12,72 +13,102 @@ network-go/ ├── config/ │ └── config.go Parses /etc/user/config/networks.json into typed structs ├── docker/ -│ └── docker.go Docker SDK wrapper (network create, container connect, -│ container PID for nsenter, route management) +│ └── docker.go Docker SDK wrapper: network create, container connect, +│ container PID for nsenter, route management ├── firewall/ │ └── firewall.go Orchestrator: translates policies → iptables rules ├── iptables/ │ └── iptables.go Manages iptables CLI: PREROUTING DNAT, POSTROUTING MASQUERADE, │ FORWARD ACCEPT, nsenter for container network namespaces ├── resolver/ -│ └── resolver.go Resolves names → IPs using networks.json config +│ └── resolver.go Resolves names → IPs using networks.json config only ├── watcher/ │ └── watcher.go Periodic file change detection via MD5 hash polling ├── go.mod / go.sum Module definition with Docker SDK dependency └── .gitignore ``` -## Packages +## Docker Run — Required Environment & Volumes -### config -- `NetworksConfig`, `NetworkConfig`, `IPConfig`, `PolicyConfig` structs matching `networks.json` -- `FirewallRule` struct for resolved executable rules -- Helpers: `IsIP`, `ToCIDR`, `NetworkPrefix`, `ParseCIDR`, `Load` +This program is designed to run inside a Docker container with the following +setup: -### docker -- `Client` wrapping `github.com/docker/docker/client` -- `EnsureNetwork` — creates a Docker network if it doesn't exist -- `ConnectContainer` / `DisconnectContainer` — manages container ↔ network membership -- `GetContainerPID` — returns the container PID for nsenter -- `AddRouteInContainer` — adds routes inside a container namespace via nsenter -- `WaitForContainerRunning` — polls until a container is running +```bash +docker run -d \ + --name network-go \ + --network host \ + --cap-add NET_ADMIN \ + --cap-add SYS_ADMIN \ + --pid host \ + -v /var/run/docker.sock:/var/run/docker.sock \ + -v /etc/user/config:/etc/user/config \ + -v /proc/sys/net/ipv4/ip_forward:/proc/sys/net/ipv4/ip_forward \ + network-go +``` -### resolver -- Resolves names → IPs using the `networks.json` config only -- Looks up by `container_name` and `selector` in the `ips` section -- Falls back to prefix matching (e.g. `smarthost-backend-1` → prefix `smarthost`) +### Required Volumes -### iptables -- Auto-detects `iptables-legacy` vs `iptables` (matching shell script logic) -- `EnsureIPForward` — enables `/proc/sys/net/ipv4/ip_forward` -- `EnsureEstablishedRelated` — inserts ESTABLISHED,RELATED ACCEPT at top of chain -- `InsertPreroutingRule` / `InsertPreroutingRuleInContainer` — DNAT rules -- `InsertPostroutingMasquerade` — MASQUERADE rules -- `InsertForwardAccept` — FORWARD/DOCKER-USER ACCEPT rules -- Rule deletion by line-number + grep pattern matching (matching shell script) -- nsenter-based execution inside container network namespaces +| Mount | Purpose | +|---|---| +| `/var/run/docker.sock:/var/run/docker.sock` | Docker SDK communication (create networks, inspect containers, get PIDs for nsenter) | +| `/etc/user/config:/etc/user/config` | Access to `networks.json` configuration file | -### firewall -- `Orchestrator` ties all packages together -- `ReconcileAll` runs the full reconciliation cycle: - 1. Enable IP forwarding - 2. Ensure all Docker networks from config - 3. Connect containers to networks with assigned IPs - 4. Apply firewall policies as iptables rules -- Policy → rule mapping: - - `from` field → FORWARD ACCEPT rule on DOCKER-USER or FORWARD chain - - `nat: dnat` field → PREROUTING DNAT inside container namespace via nsenter - - Interface-based rules (e.g. `wg0`) → host-level PREROUTING DNAT +### Required Flags -### watcher -- `FileWatcher` polls a file at a configurable interval -- Computes MD5 hash on each tick -- Fires `onChange` callback when hash changes -- `Start` / `Stop` for lifecycle management +| Flag | Purpose | +|---|---| +| `--network host` | Run in the host network namespace so iptables rules apply to the host | +| `--pid host` | Access host PIDs for `nsenter -t ` into container network namespaces | +| `--cap-add NET_ADMIN` | Required to manipulate iptables rules | +| `--cap-add SYS_ADMIN` | Required for `nsenter` to enter other containers' namespaces | -## Configuration +### Environment Variables -The file `/etc/user/config/networks.json` defines: +| Variable | Default | Description | +|---|---|---| +| `NETWORKS_CONFIG_PATH` | `/etc/user/config/networks.json` | Path to the configuration file (inside the container) | +| `WATCH_PERIOD_SECONDS` | `30` | Polling interval in seconds for config file changes | +| `DEBUG` | `false` | Enable debug output (`1` or `true`) | +| `DOCKER_HOST` | (empty = `/var/run/docker.sock`) | Docker daemon socket — automatically used by Docker SDK | + +### Why `--network host` and `--pid host`? + +The program uses `nsenter` to enter other containers' network namespaces and +insert iptables rules. This requires: + +1. **Host PID namespace** (`--pid host`) — to know the PID of the target + container (obtained via Docker SDK `ContainerInspect().State.Pid`) +2. **Host network namespace** (`--network host`) — so the program can also + manage host-level iptables chains (DOCKER-USER, FORWARD, POSTROUTING) +3. **NET_ADMIN capability** — iptables manipulation requires this Linux capability +4. **SYS_ADMIN capability** — `nsenter` needs this to switch namespaces + +Without these, the program cannot: +- List/manage host iptables rules +- Insert PREROUTING/POSTROUTING rules inside other containers +- Add routes to container network namespaces + +### Minimal Docker Compose Example + +```yaml +version: "3.8" +services: + network-go: + build: ./network-go + network_mode: host + pid: "host" + cap_add: + - NET_ADMIN + - SYS_ADMIN + volumes: + - /var/run/docker.sock:/var/run/docker.sock + - /etc/user/config:/etc/user/config + environment: + - WATCH_PERIOD_SECONDS=30 + - DEBUG=false +``` + +## Configuration — `/etc/user/config/networks.json` ```json { @@ -86,6 +117,11 @@ The file `/etc/user/config/networks.json` defines: "network_name": "smarthost-loadbalancer", "subnet": "172.18.103.0/24", "gateway": "172.18.103.1" + }, + "smarthost_backend-1": { + "network_name": "smarthost_backend-1", + "subnet": "172.18.104.0/24", + "gateway": "172.18.104.1" } }, "ips": { @@ -94,6 +130,12 @@ The file `/etc/user/config/networks.json` defines: "container_name": "smarthostloadbalancer", "selector": "smarthostloadbalancer", "service_name": "smarthost-proxy" + }, + "172.18.104.2": { + "ip": "172.18.104.2", + "container_name": "smarthostbackend-1", + "selector": "smarthostbackend-1", + "service_name": "smarthost-proxy" } }, "policies": [ @@ -120,37 +162,92 @@ The file `/etc/user/config/networks.json` defines: } ``` -## Environment Variables +## How Reconciliation Works -| Variable | Default | Description | +When the config file changes, `Orchestrator.ReconcileAll()` runs: + +1. **Enable IP forwarding** — writes `1` to `/proc/sys/net/ipv4/ip_forward` +2. **Ensure Docker networks** — creates bridge networks with specified subnet/gateway +3. **Connect containers** — attaches containers to networks with assigned static IPs +4. **Apply firewall policies** — processes each policy entry into iptables rules: + +### Policy Types + +| Policy Pattern | Action | Target | |---|---|---| -| `NETWORKS_CONFIG_PATH` | `/etc/user/config/networks.json` | Path to the configuration file | -| `WATCH_PERIOD_SECONDS` | `30` | Polling interval in seconds for config file changes | -| `DEBUG` | `false` | Enable debug output (`1` or `true`) | +| `from` field present | FORWARD ACCEPT rule | Host chain: DOCKER-USER (iptables-legacy) or FORWARD | +| `nat: "dnat"` with `iface` | PREROUTING DNAT on interface | Host PREROUTING chain | +| `nat: "dnat"` with `selector` | PREROUTING DNAT inside container | Container namespace via nsenter | -## Key Differences from Shell Script +### nsenter Implementation -| Shell Script (`firewall-add`) | Go Implementation | -|---|---| -| `docker ps --format`, `docker inspect` | Docker SDK (`github.com/docker/docker`) | -| `/etc/dns/hosts.local` lookup | networks.json config lookup | -| `$SOURCE` / `$TARGET` env vars | `from` / `to` fields in `networks.json` | -| iptables via bash + grep + awk | Go `os/exec` with structured line matching | -| `nsenter -t PID -n` for container iptables | `nsenter` via `os/exec` in `iptables.Manager` | -| `$ROUTE=true` + `ip route` | `docker.Client.AddRouteInContainer()` | -| Manual per-rule invocation | `Orchestrator.ReconcileAll()` batch reconciliation | +The shell script uses: +```bash +nsenter -t $(docker inspect --format {{.State.Pid}} $NAME) -n -- /sbin/iptables-legacy -t nat -I PREROUTING ... +``` + +The Go implementation does the same: +```go +// 1. Get container PID via Docker SDK +pid, _ := dockerClient.GetContainerPID(ctx, containerName) + +// 2. Execute iptables inside container namespace via nsenter +exec.Command("nsenter", "-t", fmt.Sprintf("%d", pid), "-n", "--", + "/sbin/iptables-legacy", "-t", "nat", "-I", "PREROUTING", ...) +``` + +- `-t ` — target the container's PID +- `-n` — enter the network namespace only +- `--` — separator, then the iptables command + +## Packages + +### config +`NetworksConfig`, `NetworkConfig`, `IPConfig`, `PolicyConfig` structs. +Helpers: `IsIP`, `ToCIDR`, `NetworkPrefix`, `ParseCIDR`, `Load`. + +### docker +Docker SDK wrapper: `EnsureNetwork`, `ConnectContainer`, `DisconnectContainer`, +`GetContainerPID`, `AddRouteInContainer`, `WaitForContainerRunning`, `InspectContainer`. + +### resolver +Name→IP resolution using `networks.json` config only. Looks up by `container_name` +and `selector` fields in the `ips` section, with prefix fallback matching. + +### iptables +Auto-detects `iptables-legacy` vs `iptables`. Manages: +- PREROUTING DNAT (host and container via nsenter) +- POSTROUTING MASQUERADE (host and container) +- FORWARD/DOCKER-USER ACCEPT with ESTABLISHED,RELATED +- Rule deletion by line-number + pattern matching + +### firewall +`Orchestrator` ties all packages together. `ReconcileAll()` runs the full cycle. +Policy→rule mapping: `from` → FORWARD ACCEPT, `nat: dnat` → PREROUTING DNAT. + +### watcher +`FileWatcher` polls a file via MD5 hash comparison. `Start`/`Stop` lifecycle. ## Build & Run ```bash +# Build +cd network-go go build -o network-go . -./network-go -``` -In Docker: -```bash -go build -o network-go . -# Mount Docker socket and config -docker run -v /var/run/docker.sock:/var/run/docker.sock \ - -v /etc/user/config:/etc/user/config \ - network-go \ No newline at end of file +# Run locally (requires Docker socket access) +./network-go + +# Build and run in Docker +docker build -t network-go . +docker run -d \ + --network host \ + --pid host \ + --cap-add NET_ADMIN \ + --cap-add SYS_ADMIN \ + -v /var/run/docker.sock:/var/run/docker.sock \ + -v /etc/user/config:/etc/user/config \ + -e WATCH_PERIOD_SECONDS=30 \ + -e DEBUG=false \ + --name network-go \ + network-go \ No newline at end of file diff --git a/network-go/network-exec b/network-go/network-exec new file mode 100755 index 0000000..83ded2b Binary files /dev/null and b/network-go/network-exec differ