Migrating the Docker stack from the PC to the VPS

TL;DR — Don't transfer the Windows containers. Recreate the stack on the VPS from source (docker compose up --build). Only transfer the data (Postgres + card images) if you want to skip the ingest step.

Why recreate instead of transfer

You have two options:

Option	Description	Verdict
A. Transfer the containers	docker save images + copy volumes over SSH	Avoid
B. Recreate from source (IaC)	git clone + docker compose up --build on the VPS	Recommended

Recreate (option B) — why it's better

1. Everything is already reproducible — compose.yml, the *.Dockerfiles and the source live in tcg-collector-backend/. The VPS already knows how to build the stack.
2. Lighter — you transfer neither the ~5 GB of Docker images nor the buildkit caches. The VPS only pulls the bases (pgvector/pgvector:pg16, nginx:alpine, node:22-alpine) and builds the custom layers from a few-MB repo.
3. Cleaner — no accumulated stale layers (the tcg-collector-ingest-1 container exited 5 days ago, the older embeddings sidecar is likely replaceable, the solidity-dojo_default network is orphaned… it all starts fresh).
4. Avoids OS pitfalls — copying a raw data/pg from Docker Desktop (WSL2) to Linux works in practice but stays fragile (pg_dump is the safe path, see below).
5. Future-proof — every change later goes through git pull && docker compose up -d --build on the VPS, not another manual transfer.

Transfer (option A) — when it's worth it

Only if all of these are true:

• You modified application state by hand (data outside ingest).
• Re-downloading from TCGdex would be too slow or rate-limited from the VPS.
• You need to ship the service quickly.

In that case, do not transfer the containers — only the data: pg_dump + tar of the images (see §3).

---

1. Prepare the VPS

Prerequisites assumed to be in place on the VPS:

• Docker Engine + compose plugin installed.
• User kael in the docker group.
• gh installed and authenticated.
• Repo snag-tcg/tcg-collector-backend clonable.

Sanity-check first:

docker compose version
docker run --rm hello-world

2. Recreate the stack — happy path

# 1. Clone (if not already done)
mkdir -p ~/skn-tcg && cd ~/skn-tcg
gh repo clone snag-tcg/tcg-collector-backend
cd tcg-collector-backend

# 2. Configure the environment
cp .env.example .env
# Edit .env:
#   - PGUSER / PGPASSWORD (a STRONG password, not "tcg")
#   - API_WRITE_TOKEN (random token if /scan or writes are exposed)
#   - CORS_ORIGIN (the front-end domain, not "*" in production)
nano .env

# 3. Bring the stack up
docker compose -f docker/compose.yml up -d --build

# 4. Verify
docker compose -f docker/compose.yml ps
docker compose -f docker/compose.yml logs -f api
curl http://localhost:8080/health   # or via nginx: :8081/api/health

# 5. First ingest (long — depends on INGEST_SETS)
docker compose -f docker/compose.yml run --rm ingest

Expected build time: ~5 to 15 min depending on the VPS CPU (the longest is tcg-vision, which downloads ~85 MB of DINOv2 weights and builds CPU PyTorch).

3. Transfer the data (optional — if you want to skip the ingest)

Do this after the stack is running on the VPS, replacing only the Postgres contents and the data/images folder.

3.1 Postgres — pg_dump (clean, recommended)

On the Windows PC:

# Full dump (compressed, custom format)
docker exec tcg-collector-postgres-1 `
  pg_dump -U tcg -d tcg -Fc -Z 9 -f /tmp/tcg.dump
docker cp tcg-collector-postgres-1:/tmp/tcg.dump .\tcg.dump

# Send to the VPS
scp .\tcg.dump kael@VPS_IP:/tmp/tcg.dump

On the VPS:

# Stack already running, but empty
docker cp /tmp/tcg.dump tcg-collector-postgres-1:/tmp/tcg.dump

# Restore
docker exec -i tcg-collector-postgres-1 \
  pg_restore -U tcg -d tcg --clean --if-exists --no-owner /tmp/tcg.dump

Why not just copy data/pg? It often works (same pg16 version), but it's fragile: differences in endianness, locale, page size, or a pg_control corrupted mid-copy. pg_dump is portable and version-tolerant.

Note on the index corruption detected in the PC's data/pg (see container-inventory.md): pg_dump reads tables logically, so the dump regenerates clean indexes on the VPS. That's a clear win over a raw copy.

3.2 Card images — tar + scp (or rsync)

The ~1.9 GB of visuals are static, regenerable files, but slow to re-download.

# On Windows, from tcg-collector-backend\
tar -czf data-images.tar.gz data\images
scp data-images.tar.gz kael@VPS_IP:/tmp/

On the VPS:

cd ~/snag-tcg/tcg-collector-backend
tar -xzf /tmp/data-images.tar.gz   # recreates data/images/
sudo chown -R $USER:$USER data/images

The compose's nginx service already mounts ../data/images; a simple docker compose restart nginx makes it live.

3.3 pHash & FAISS indexes

• The pHash index (scanIndex.json) is generated in the sibling poc-tcg-collector repo (npm run ingest:cards). No need to migrate it: regenerate on the VPS if you also deploy the RN client there, otherwise leave it on the build machine.
• The tcg-vision sidecar's FAISS index is prebuilt into the Docker image (docker/tcg-vision.Dockerfile clones a pinned commit of KaelSensei/tgc-app). No manual action.

4. Should the older embeddings sidecar stay?

Probably not — tcg-vision replaces it functionally with a more complete approach. Before removing it:

1. Grep the API source: is EMBEDDINGS_URL still read?
2. If yes, does it point at the older embeddings or at tcg-vision?
3. If the older one is no longer consumed, remove the embeddings service from compose.yml from the start on the VPS. Savings: ~1.25 GB of image and some RAM.

5. Production — minimum security checklist

To configure on the VPS before any public exposure:

• .env: strong Postgres password, long random API_WRITE_TOKEN, CORS_ORIGIN restricted to the front-end domain.
• Do not expose port 5432 publicly. In compose.yml, bind it to localhost explicitly:

  ports:
    - "127.0.0.1:5432:5432"

• TLS reverse proxy (Caddy / Traefik / host nginx) in front of :8080 and :8000. No plaintext API.
• UFW / iptables: open only 22/tcp (SSH) + 80/tcp + 443/tcp.
• Rate-limiting in nginx or Fastify (API_RATE_LIMIT_MAX).
• docker compose ... up -d at boot via systemd (the restart: unless-stopped in compose is enough after a first up).

6. Recommended order of operations

1. Repair the index corruption on the PC (optional — to have a clean state to dump).
2. On the VPS: complete the provisioning prerequisites listed in §1, then clone the repo.
3. cp .env.example .env + edit secrets.
4. docker compose up -d --build (lets Postgres + nginx + api + tcg-vision run empty).
5. Choose:
   • Without transfer: docker compose run --rm ingest → let it run ~30 min.
   • With transfer: pg_dump + pg_restore + tar/scp images (§3).
6. Health checks: curl /api/health, curl :8000/health, scan a test card.
7. Move private ports behind a TLS reverse proxy (§5).

7. Once it's live — recurring workflow

No more transfers. The pattern becomes:

# On the VPS
cd ~/snag-tcg/tcg-collector-backend
git pull
docker compose -f docker/compose.yml up -d --build

If a new ingest is needed (new sets): docker compose run --rm ingest.