File sync

How folders travel between your machine and an environment: content-defined chunks, ignore rules, secrets, —include and extra files.

Each share has one writer and one receiver. The writer watches its folder and syncs within about a second of an edit. envctl sync syncs now and returns once the receiver has acknowledged.

What travels

Files are cut into content-defined chunks of 16 to 256 KiB, 64 KiB on average. Each folder is a tree listing its entries and their chunks. The receiver walks the tree from the top and asks only for the folders and chunks it doesn’t have. So:

Change	What’s sent
One line added to a source file	One chunk and the trees above it: about 4 KB
100 bytes inserted at the start of a 7 MB binary	One chunk
A folder of 400 files renamed	One tree, under 1 KB
A binary copied to a new path	Only the new path’s tree
Nothing	Nothing; a resync of 12,000 files takes 40 ms

Each sync is a numbered generation, authenticated with the project’s key. Changed files are written to a staging folder and renamed into place, so a build never sees a half-written tree. Every chunk is sealed under your project’s key before it leaves.

What stays home

Leaving a file out is decided in this order:

1. Never sent

   .git, envlink’s own files, and the template’s excludes: for linux-build, out/**, node_modules/**, *.env and secrets/**.

2. Secret files

   .env and .env.*, private keys and certificates (*.pem, *.key, *.p12, id_rsa, id_ed25519, …), .netrc, .pgpass, .git-credentials, and folders named secrets or .ssh. Examples and templates like .env.example are fine. --include can’t bring secret files back; see Extra files.

3. Your --include patterns

   Anything they name is sent, even when the next rules would leave it out.

4. .gitignore and the defaults

   The .gitignore files from your repository’s root down to each folder, and .git/info/exclude, with git’s rules: the last match wins, deeper files override shallower ones, ! re-includes, and a trailing / matches only folders. Then the defaults: node_modules/, .venv/, __pycache__/, *.pyc, .pytest_cache/, .mypy_cache/, .ruff_cache/, .tox/, .gradle/, .next/, .nuxt/, .svelte-kit/, .parcel-cache/, .turbo/, .terraform/, DerivedData/, .build/, bazel-*, .DS_Store, Thumbs.db, *.swp and .idea/.

The sync log says what was left out:

gen 1: 5 of 5 chunks and 3 trees sent, 6 files written, 0 deleted, 1 extra files; left out 3 by .gitignore and defaults, 1 secret files

Bring something back with —include

envctl env create linux-build --share src=. --include 'dist/app.js' --include 'vendor/**'

Patterns are globs relative to the share. A pattern without a slash, like *.log, matches a name at any depth. An include can reach a single file inside an ignored folder. --no-ignore turns off .gitignore and the defaults altogether; secret files still stay back.

Extra files

Some builds need a file from outside the folder, such as a registry login. Name it with --file source=destination:

envctl env create linux-build --share src=. --file ~/.npmrc=.npmrc --file ./ci/netrc=.netrc

An extra file:

• is sent sealed inside the sync and written readable only by the environment (mode 0600);
• never enters a store, a snapshot or object storage;
• replaces a file of the same name in the share, and is the only way to send a secret file;
• is at most 1 MiB, with up to 64 per environment.

Changing the source re-sends just that file.

Results coming back

Out shares, like linux-build’s out, run the other way: the environment writes, and your sandbox receives into .remote/<environment>/<share>/. envctl exec waits for them before it returns.

Watching it work

envctl logs -n 20       # this machine's syncs and connections
envctl status           # each share's last generation