Deploying an EDD site

Deployment installs the persistent EDD core and supporting services at an observatory. It is distinct from provisioning, which starts an observing mode for a limited period. Treat the site repository as configuration-as-code: pin versions, review changes, and retain deployment logs.

Platform requirements

The exact requirements depend on enabled plugins, but every site needs:

Linux hosts reachable by stable hostnames;
Python 3.11 or newer on the deployment host;
SSH access from the deployment host and from the Master Controller’s Ansible interface where runtime provisioning requires it;
privilege escalation for container runtime, device, network, and host setup;
a supported Docker installation on container hosts;
shared or explicitly routed data-product storage;
synchronized clocks and observatory reference-time distribution;
configured management, data, and storage networks; and
compatible GPU drivers, CUDA version, FPGA runtime, RDMA or DPDK setup for plugins that require them.

EDD containers and KATCP endpoints are intended for trusted observatory networks. Do not expose the control, registry, database, or monitoring services directly to untrusted networks. Store secrets with Ansible Vault and restrict the vault-password and SSH-key files.

Site repository contract

A deployable site repository should contain:

one or more Ansible inventories;
a site playbook, such as site.yml;
pinned EDD core and plugin versions;
provision_descriptions/ containing paired YAML and JSON files;
encrypted site credentials where needed; and
site-specific roles or interfaces that cannot live in a reusable plugin.

Use an existing EDD site repository as a structural example, but do not copy host groups, addresses, storage paths, secrets, or hardware assumptions without review.

Clean, reproducible deployment with edd-tool

Install the deployment utility in a virtual environment as shown in Getting started. The deploy command is the operational path for a clean deployment from a repository. It always:

clones site_repo into a new temporary directory at the requested version;
installs the collections selected by EDD_PLUGINS into that temporary deployment;
optionally pulls the selected container images; and
runs the site playbook before removing the temporary checkout.

There is deliberately no option to deploy an existing local checkout. A deployment is therefore not affected by uncommitted site changes, a collection left over from an earlier run, or edits made during a debugging session.

A production deployment should normally use a named profile from ~/.edd-tool.rc:

production:
  site_repo: https://git.example.org/observatory/edd_site.git
  version: 2026.1
  inventory: inventories/production
  site_config: site.yml
  vault-pass-file: /secure/edd-production.vault

Execute the pinned deployment:

$ edd-tool deploy production

The clean checkout makes the procedure repeatable, but reproducibility still depends on its inputs. Pin the site version and every EDD_PLUGINS version to immutable tags or commits. Use immutable container tags or image digests where supported. A moving branch named main or an image named latest can produce a different system even when the command is unchanged.

Useful deployment options include:

--no-pullremote: Skip the remote-image pull phase. Use only when the required images are already present and verified.
--ansible-args "...": Pass reviewed extra arguments to ansible-playbook, for example a host limit during staged deployment.

Development environments with make-devenv

Use make-devenv when the goal is to inspect, change, and repeatedly run deployment Ansible rather than perform a clean operational deployment. It creates a persistent directory containing:

edd-dev/
├── site/                  managed site checkout, when one is requested
├── collections/
│   └── ansible_collections/
├── activate              shell environment
├── README.md             generated command reference
└── .edd-devenv.json      environment paths and input fingerprints

Create an environment with a tool-managed checkout:

$ edd-tool make-devenv ./edd-dev development

The profile supplies the same repository, version, inventory, site playbook, Vault, and extra-variable inputs used by deploy. A managed checkout is pinned to its original repository and version; create another environment to test a different site revision.

To develop a site repository that is already checked out locally, keep the environment and checkout separate:

$ edd-tool make-devenv ./edd-dev development \
      --site-dir ~/src/edd_site

make-devenv does not pull or modify the external site checkout. It installs the collections below ./edd-dev/collections and prints direct ansible-playbook commands with the required ANSIBLE_COLLECTIONS_PATH. The same syntax-check, image-pull, full-deployment, and activation commands are written to ./edd-dev/README.md.

Source the generated environment before running additional Ansible commands:

$ source ./edd-dev/activate
$ cd "$EDD_SITE_DIR"
$ ansible-playbook -i inventories/development site.yml --syntax-check
$ ansible-playbook -i inventories/development site.yml

This supports two related development cycles:

Site deployment development: Edit the external checkout supplied with --site-dir, then rerun the generated Ansible commands. Collections remain installed between runs.
Core and plugin deployment development: Edit the installed collection below $ANSIBLE_COLLECTIONS_PATH/ansible_collections/EDD/. For example, EDD core deployment roles and playbooks are below EDD/core and a plugin named EDD.gated_spectrometer is below EDD/gated_spectrometer. Direct Ansible runs use those edits immediately. This changes collection deployment behaviour; it does not replace the Python installation that provides the edd-tool executable.

The installed collection tree is a disposable development copy, not the authoritative plugin repository. Transfer successful changes to the relevant core or plugin source repository and test them there. Re-running make-devenv reuses collections while its recorded inventory and extra-variable inputs are unchanged. --refresh-collections, or an automatic refresh after those inputs change, can overwrite edits in the installed collection tree.

The generated state file stores hashes rather than Vault values or inline extra-variable contents. It does record the site repository URL, so use SSH or a credential helper instead of putting credentials in that URL. The generated README.md reproduces the command line, so do not put plaintext secrets directly in --ansible-args; reference an encrypted Vault file instead.

Update plugin pins

The tool can inspect and update plugin versions in a site repository:

$ edd-tool update-plugins production

Use --commit only after reviewing the generated diff and testing the new combination in a development inventory. Plugin updates may require a matching core, suite, container image, or provision-description change.

Manual deployment

For development, direct Ansible execution should normally start from a make-devenv environment so that the selected collections and search path match edd-tool. Fully manual execution is reserved for recovery when edd-tool cannot be used; the administrator must then reproduce the repository checkout, collection installation, image update, Vault, and playbook steps correctly.

        sequenceDiagram
    rect rgb(166, 200, 255)
    user->>site_inventory: git checkout
    user->>site_inventory: parse inventory to get core+plugin versions
        user->>edd_core: install correct version (git, galaxy)
        user->>plugins: install correct version (git, galaxy)
        user->>site_inventory: ansible-playbook site_config.yml
    end
    site_inventory->>edd_core: import basic playbook
    edd_core->>site_inventory: COPY to temp to work around rootsquash
    edd_core->>site_inventory: Commit possible changes if deployed from development (only local)
    edd_core->>site_inventory: Clone tmp to global location on shared file system
    edd_core-->>edd_core: deploy core
    edd_core->>plugins: galaxy install to global location
    edd_core->>plugins: pull container / run installer

A typical manually prepared checkout is applied in stages:

$ ansible-inventory -i inventories/development --graph
$ ansible-playbook -i inventories/development site.yml --tags baremetal
$ ansible-playbook -i inventories/development site.yml --tags pullremote
$ ansible-playbook -i inventories/development site.yml

Available tags are defined by the pinned core and site repository; inspect the playbook before assuming a tag exists. Avoid building production images on a telescope host when a tested binary image is available from the registry.

Post-deployment verification

Record the site revision and verify:

all expected persistent containers are running with the pinned image tags;
the Master Controller is reachable and unprovisioned;
Redis, InfluxDB, Grafana, and Loki are healthy;
the Master Controller sees the intended provision descriptions;
Ansible can reach every runtime provisioning host;
data, storage, and time interfaces match the inventory; and
a representative provision test completes on the development system.

Deployment rollback

Rollback means redeploying a previously tested combination of the site repository, core collection, plugin collections, packages, and images. A git revert alone is insufficient if tags in a mutable registry were reused. Keep immutable version tags or image digests and retain the prior inventory and deployment log.