Development Standards

Conventions and requirements for writing automation in this repository. This page starts with the expectations for contributing, then covers role authoring — the rules every role must follow — and will grow to cover other areas (group_vars/host_vars, inventory, playbooks) as those standards are written.

Many rules here are enforced by ansible-lint or by Ansible itself; where a rule has a deeper reference elsewhere in these docs, it links to it rather than repeating the detail.

Contributing

Expectations for anyone contributing to the Ansible code base:

• Everything on this page is binding. The standards below are requirements, not suggestions — a change is expected to meet all of them before it merges, not as a follow-up.
• Every change goes through a pull request. Never commit directly to master. This applies to every kind of change — playbooks, roles, inventory, Coder templates, workflows, scripts, and documentation alike — and to every repository involved: the parent ansible-epic repo and each individual role repo (Sapphire-Health/ansible-role-<name>).
• Every pull request requires at least one approval before it is merged.
• Commits (and therefore PRs) route to the repository that owns the changed files — see commit routing. A change that spans the parent repo and a role means two PRs in the two repos.
• Documentation ships with the change, in the same PR — usage guides, argument specs, and CLAUDE.md files stay in sync with the code, and mkdocs build --strict must pass for any docs-site change. The mechanics of contributing to the docs site itself (local preview, Cloudflare Pages PR previews) are on the Documentation page.

Code quality & safety

Prefer native modules over shell/command

If a task can be done with a native Ansible module, it must be. Never use a shell/command invocation to do something a module is designed to do.

Dropping to shell throws away everything the module gives you — idempotency, --check (dry-run) support, structured return values, change reporting, and cleaner error handling. A shell command reimplements (usually badly) what the module already handles correctly.

shell/command/win_command/win_shell are for genuine gaps — an operation no module covers. When you do drop to one, the idempotency rule below still applies: you take on by hand what the module would have given you.

Idempotency

Every task must be idempotent: running the role a second time with no change in intent makes no changes and reports none. Preferring native modules gets you this for free, but the requirement is on the task, not the module — it holds even for the genuine gaps where you've had to drop to shell/command.

In that case you are responsible for reproducing idempotency by hand:

• a creates:/removes: guard,
• a when: condition that checks current state first, or
• an accurate changed_when: so the task only reports (and only performs) a change when one is actually needed.

The same applies to --check (dry-run) support.

Portability across execution platforms (CLI, AWX, AAP)

Role code runs unchanged under ansible-core on the CLI, under AWX / ansible-runner, and under Ansible Automation Platform (AAP). These platforms differ in ways your code must not assume away — most importantly the remote temp directory each one uses to stage files. Write to the lowest common denominator; never to how your controller happens to invoke Ansible.

Single-quote any substituted path in a validate: string (and in any command/shell that interpolates a path):

# WRONG — breaks when the remote temp path contains whitespace
validate: "visudo -c -f %s"
# RIGHT
validate: "visudo -c -f '%s'"

Ansible runs a string validate by shlex.split-ing it — there is no shell, but the string is still word-split. If the temp path Ansible substitutes for %s contains a space, an unquoted %s splits into multiple arguments and the validator (visudo, sshd -t, nginx -t, …) receives garbage — it aborts with a usage/parse error before ever reading the file, and the task fails with a misleading "failed to validate". Because the fault lives in the remote temp path, it passes on a developer's clean /home/<user>/.ansible/tmp/… path and only surfaces on a customer's AWX/AAP host whose remote_tmp (or the become user's home) contains whitespace. Quote every %s: sshd -t -f '%s', nginx -t -c '%s', visudo -c -f '%s'. (Observed in production on a customer AWX host, 2026-08 — see the iris role's TESTING.md.)

Pass CLI flags in their portable, unbundled form. Don't rely on GNU-style conveniences the target's tooling may not support. Pass validator flags unbundled — visudo -c -f, not visudo -cf: sudo-rs, the Rust sudo/visudo reimplementation now shipping as the default sudo on newer distros (e.g. Ubuntu 26.04), does not accept GNU-style bundled short flags, while stock sudo accepts either. The unbundled form works everywhere; the bundled form is a portability trap. The same tooling also warns on a sudoers drop-in with no trailing newline, so templates that render config files should end with a newline.

The underlying discipline is the same one the Kerberos considerations impose on Windows roles: code to what the target provides, not to connection- or platform-derived context. Resolve identity, paths, and capabilities from the managed node, never from assumptions about the controller.

Don't depend on a playbook's own location

A playbook or role must not assume anything about where it lives on disk. We want to be free to reorganize playbooks into whatever directory layout makes sense in the future — move a playbook up a level, group it into a subdirectory, rename the folder around it — without breaking it. Any path derived relative to the playbook's own location is a hidden coupling to the current layout that a reorganization silently breaks.

Concretely:

• Don't build paths from playbook_dir (or role_path/inventory_dir) to reach sibling content — vars_files, include_tasks, include_vars, -e @… extra-var files, or template/copy src: — with a relative walk like {{ playbook_dir }}/../group_vars/… or ../../extra_vars/foo.yml. The moment the playbook moves, the .. chain points somewhere else.
• Rely on Ansible's own resolution instead. Roles are found via roles_path; group_vars//host_vars/ are loaded automatically relative to the inventory, not the playbook; a role's tasks:/templates:/files: are resolved within the role regardless of where the calling playbook sits. Let those mechanisms locate things rather than reconstructing a path by hand.
• Reference shared repo files by a stable, layout-independent handle — an absolute path, a value passed in via -e, or a variable — not by counting .. hops from the playbook.

Within a role, role_path-relative references to the role's own bundled content are fine — that content moves with the role. The rule is about depending on the location of the playbook/role relative to other things in the repo.

Verify end state, not just task completion

Where feasible, a role should verify the thing it installed or configured actually works, not just that its tasks reported OK. The kuiper role, for example, requests https://localhost/Kuiper after install to confirm the app responds. This catches the "green run, broken result" case and pairs with fail-fast validation — check preconditions up front, verify the outcome at the end.

Handlers for service restart/reload

Restart or reload a service through a handler (notified by the task that changed its config), not an inline restart task. The handler fires once, at the end of the play, and only when something actually changed — which is both correct and idempotent.

Exception

When a change must be verified within the same run — a later task depends on the restart having already happened, so a deferred handler is impractical — an inline restart gated on when: <thing>.changed is acceptable. Suppress the lint warning with # noqa: no-handler:

- name: Restart nginx to apply new limits  # noqa: no-handler
  ansible.builtin.service:
    name: nginx
    state: restarted
  when: perimeter_auth_limits_drop_in.changed

Pass ansible-lint

New code must pass ansible-lint. Run it from inside the role directory for role changes and resolve findings before considering the work done. The line limit is 160 characters (long repo: strings in apt_repository tasks are the most common violation — split them with a YAML block scalar or a variable).

ansible-lint in the pipeline

Every role repo runs ansible-lint in CI. Like Molecule, this rides a single reusable workflow in the parent repo — .github/workflows/ansible-lint-role.yml (on: workflow_call) — so the logic lives in one place instead of being copied, and drifting, across every role. Each role repo carries only a thin caller at .github/workflows/ansible-lint.yml:

name: Ansible Lint
on:
  # Feedback while iterating on a branch, before a PR exists.
  push:
    branches-ignore:
      - master
  # Every PR, whatever it targets.
  pull_request:
  workflow_dispatch:
jobs:
  ansible-lint:
    uses: Sapphire-Health/ansible-epic/.github/workflows/ansible-lint-role.yml@master

Unlike the Molecule check, lint is not label-gated — it runs automatically on every push to a non-master branch and on every PR, and its cost is trivial, so there's no opt-in. New role repos get this by copying the caller above verbatim.

Two things the caller must get right, both baked into the snippet:

• Never filter pull_request with branches-ignore: [master]. On the pull_request event that filter matches the branch being merged into, so it excludes every PR targeting master — i.e. lint would never run as a PR check at all. A bare pull_request: is correct. (push legitimately ignores master, since direct pushes to master don't happen — everything lands via PR.) This bug shipped, silently disabled in most roles, until the workflows were consolidated.
• Reference the reusable workflow at @master, matching the Molecule caller convention. A caller referencing a not-yet-merged change to the reusable workflow can pin a branch ref temporarily to test, but must return to @master before merging.

What the reusable workflow does that a bare ansible-lint invocation does not, so callers don't have to:

• Restages the repo as roles/<galaxy_name>/ (repo name minus the ansible-role- prefix, hyphens → underscores) and lints from that project root, so role-scoped rules resolve correctly.
• Writes a project-root .ansible-lint that warns (not fails) on role-name and excludes the role's own .github/ from linting.
• Installs role dependencies (meta/main.yml) and collection dependencies (collections/requirements.yml) so rules that need them can resolve. Both steps no-op when the file is absent, so every role calls the workflow unchanged.
• Accepts an optional python_version input (default 3.11).

Role structure & conventions

Fully-qualified collection names (FQCN)

Reference every module and plugin by its fully-qualified collection name — ansible.builtin.copy, ansible.windows.win_command, community.general.* — never the bare short name. ansible-lint enforces this; it's stated here as the explicit convention.

Role scope & reuse

Tasks stay within the role's designed purpose. When work starts to stray from what the role was built to do, that's a signal to pick a better home for it rather than bolting it on:

• If the new capability is significant and coherent on its own → create another role.
• If it's a genuine one-off → an ad-hoc command or a one-off playbook, not a permanent addition to an unrelated role.

This keeps roles cohesive, testable, and documentable — their usage guide and argument spec stay meaningful.

Reuse existing roles; don't reinvent them. If a capability is already implemented by another role, use that role — do not duplicate its features. For example, certificate operations go through the certificate_authority role; any role that needs certificates issued or managed calls it rather than growing its own certificate handling. A prerequisite role like this is also a dependency the consuming role must declare in its usage guide ("other roles that must run beforehand").

Argument spec + fail-fast validation

Every role ships a meta/argument_specs.yml. It serves three purposes:

1. Man page — ansible-doc -t role <name> renders it (variables, defaults, required flags, per-tag prose descriptions).
2. Context for AI — a structured, machine-readable description of the role's inputs.
3. Early validation — Ansible inserts a "Validating arguments" task at role entry, so supplied variables are checked for presence/type/choices before any real work runs. Mark required inputs required: true so a run missing them fails immediately with a clear message instead of mid-phase.

Validate and verify as early as possible. If a variable needs validation beyond what the argument spec expresses (connectivity, permissions, a value that must resolve against the target), do it as early in the role as possible — don't make the user wait through a long sequence of tasks to hit an error that was knowable up front. Fail fast, with a message that says what's wrong and what to supply. The kuiper role's tasks/install/prerequisites.yml runs its SQL connectivity and permission preflight before any install task.

Two argument-spec gotchas

• Every option listed in the spec is templated at role entry, even if no task that would use it runs. So do not list an option whose default references a typically-undefined variable — it will fail validation on every run. (Role quirks like this belong in the role's own CLAUDE.md; see roles/kuiper/CLAUDE.md for a concrete case.)
• No literal Jinja ({{ ... }}) in option descriptions — descriptions are themselves templated when the validation task reports. Reference variables by name in prose instead.

Role metadata (meta/main.yml)

Every role includes a meta/main.yml with its galaxy metadata: supported platforms, min_ansible_version, any collection dependencies the role's modules require, and license/author info. This makes the role well-formed for Galaxy tooling and documents its runtime assumptions in a machine-readable place alongside the argument spec.

Tag-gating & task structure

Multi-operation roles fail fast without a tag. A role with multiple distinct operations (install, configure, upgrade, restart…) must require an explicit tag to indicate intent. Running with no tags, or with --tags all, fails rather than doing everything at once:

- name: Require a specific tag to indicate intent
  ansible.builtin.fail:
    msg: >-
      A tag must be specified to indicate intent (e.g., --tags install).
      Running without tags or with --tags all is not supported.
  when: "'all' in ansible_run_tags"
  tags: always

Exemption

Single-purpose declarative roles (e.g. firewalld) skip this — they always do one thing and are always called intentionally.

Phase-directory structure + manually-maintained tag union. Multi-phase roles organize tasks by feature phase (the kuiper pattern):

• Task files grouped under tasks/<phase>/ (e.g. install/), each phase with its own main.yml dispatcher. Future features (upgrade, configure) get sibling directories.
• The top-level tasks/main.yml holds the tag-gate fail task above, plus one include_tasks: <phase>/main.yml entry per phase.
• The tag list on each top-level phase include is a deliberate, manually maintained union of every tag used inside that phase's main.yml. Ansible only follows the include when a listed tag matches — so if a tag is missing from the outer list, running with it silently does nothing (the outer include is skipped before the inner dispatch is read).

Why the union instead of tagging the include always? always would remove the duplication, but ansible-playbook --list-tags does not expand dynamic includes, so the role's tags would no longer be discoverable from the CLI. The union keeps --list-tags accurate.

Keep the union in sync

When adding or renaming a tag inside tasks/<phase>/main.yml, update the matching include's tag list in tasks/main.yml in the same change.

Grouping a single multi-file operation into its own subdirectory. The phase-directory pattern above organizes a whole role by phase. The same subdirectory-plus-main.yml idea also applies one operation at a time inside a role whose top-level tasks/main.yml is otherwise a flat, one-include_tasks-per- operation dispatcher (the iris pattern): when a single operation's implementation needs more than one file, promote it into its own tasks/<operation>/ subdirectory with a main.yml sub-dispatcher instead of scattering those files flat in tasks/. iris's cloud-CLI install is the example — tasks/main.yml includes cloud_cli/main.yml under one cloud_cli tag, and that sub-main.yml gates the whole group on a variable (iris_install_cloud_cli) and dispatches by cloud and OS family (install_aws_cli.yml, install_azure_cli_{{ ansible_os_family | lower }}.yml), keeping all the operation's files together in one directory.

This nests the dispatcher recursively — main.yml → operation main.yml → leaf files — with the always-block contract (below) at each level. Crucially, unlike the multi-phase kuiper case, there is no tag union to maintain here: the whole operation sits behind a single tag, its sub-main.yml exposes no additional tags (everything inside is in an always block gated by the group variable), so the parent include carries just that one tag and --list-tags stays accurate without a union. Reach for this when the trigger for a subdirectory is "this one operation took several files," and for the full phase-directory pattern when the trigger is "the whole role has distinct phases."

Dynamic includes wrap tasks in an always block

Roles use include_tasks (dynamic), not import_tasks (static). Dynamic includes do not propagate the caller's tags into the included file, so every included task file must wrap all its tasks in a block tagged always — otherwise those tasks are skipped when the play runs with --tags <something>:

- name: Run tasks
  tags:
    - always
  block:
    - name: ...

OS-specific task dispatch

For cross-platform roles, dispatch to OS-specific files from main.yml:

ansible.builtin.include_tasks:
  file: install_nginx_{{ ansible_os_family | lower }}.yml

ansible_os_family returns RedHat (RHEL), Debian (Ubuntu), or Windows. File names use underscores to match the rest of the codebase convention.

Variable namespacing

All variables a role defines (defaults, vars, set_facts, registered results) must be namespaced with the role name — kuiper_*, system_pulse_*, etc. — to avoid collisions across the many roles that share one inventory and run in the same play context.

Sensible defaults, no hardcoded values

Two goals that pull together, not against each other:

• Minimize required input. Give every value a sensible default so the role runs with as few variables as the user must supply as possible. If there's an obvious default, set it — don't force the user to define later what you could have defaulted.
• No hardcoded literals in tasks or playbooks. A configurable value is a variable even when it has an obvious default. Set the default so nobody has to pass it, but keep it a variable so it stays overridable and shows up in the argument spec and usage guide.

The goal is a role that "just runs" out of the box yet exposes every meaningful value for override, with nothing baked into a task or a playbook literal.

defaults/ vs vars/

A value meant to be overridable gets its default in defaults/main.yml; a value that must not be overridable is defined in vars/main.yml (role vars outrank most sources, which is the point). This keeps the role's public surface — what a user may set — distinct from its internals.

Directional — pattern still being settled

Exactly what lives in vars/ is still being finalized (the <role>_effective_* centralization pattern). Treat the split as directional until that's decided.

Molecule testing

Roles can carry a Molecule scenario that tests them against real VMs provisioned through the aws-test-vm or azure-test-vm Coder template — real systemd, SELinux, firewalld, and raw disks carrying the production disk tag for that cloud (ansible_key on AWS, disk_label on Azure).

A scenario is not required for every role. Some roles don't have one (yet), and that's an accepted state — everything in this section, including the CI gate and the reviewer convention below, applies only to roles that have a scenario. When a role does gain one, it comes with the obligations here from then on. The shared create/destroy plumbing, prerequisites, and pilot-proven molecule.yml boilerplate live in the parent repo's molecule/README.md; copy from there rather than from upstream examples (this repo's tag-gated roles, agent-only SSH keys, and proxy-only egress all require specific settings that generic examples get wrong).

• Scenarios live in the role repo (molecule/default/) — that's the norm. A scenario in the parent repo is the documented exception, reserved for genuinely orchestration-heavy playbooks (multi-role sequencing, cross-group coordination); thin role-wrapper playbooks don't get one.
• Either cloud, one scenario each: molecule/default/ provisions on AWS, molecule/azure/ on Azure (provisioner.env.MOLECULE_CLOUD: azure). A role deployed on both clouds should carry both, sharing converge/verify — import the default scenario's playbooks rather than copying them, and keep the role's variables in one file both load, so the clouds cannot drift into testing different configurations. Each scenario runs from a workspace (or runner) in its own cloud.
• What the lifecycle buys without writing tests: converge proves the role on a clean machine; the built-in idempotence step re-runs converge and fails on any changed task — the mechanical enforcement of the idempotency standard above.
• verify.yml stays small and outside-in: assert what a customer would notice broken (service running, endpoint answering, mount present) — don't re-implement the role as assertions.
• CI: each role repo carries a small caller workflow invoking the reusable molecule-role-test.yml from the parent repo (runs on a self-hosted runner — only a runner inside the target cloud can reach its test VMs — created for that job and deleted afterwards, which needs a CODER_SESSION_TOKEN secret and secrets: inherit; see molecule/README.md). Runs are deliberate, not per-push: apply the molecule-aws label to the PR to produce the check for the current head commit, or use workflow_dispatch for branches without a PR. Azure is a second caller workflow passing cloud: azure, on its own molecule-azure label, so either cloud can be exercised independently. Whether that label fires once or keeps re-running on later pushes is the role repo's choice — both wirings are in molecule/README.md. The reviewer convention is the same either way: the green check must be on the head commit.

Reviewer convention: green Molecule check before approval — where a scenario exists

Applies only to roles that have a Molecule scenario. For those, GitHub's plan tier doesn't enforce required checks on these private repos, so this rides the same convention as PR approval itself: don't approve the role's PR without a green Molecule check on the current head commit. If the label was applied before the latest push, the check is stale — ask for a re-run. Roles without a scenario are reviewed the usual way; there is no Molecule obligation to satisfy.

Secrets

Secret handling has its own detailed reference: Secret Management. In short:

• Accept each secret as a lowercase Ansible variable (vault) with a matching uppercase environment variable as a fallback; the Ansible variable wins when set. See the convention.
• Set no_log: true on any task whose arguments or return value contain a secret.
• Never hardcode or commit a secret value — defaults for secret variables are empty/undefined, and the real value arrives only from the vault or the environment.

Authentication — Kerberos

Under Kerberos, role code cannot rely on connection-derived context. Two rules follow; full detail and rationale are on the dedicated Kerberos Considerations page.

• Resolve identity from the target, not ansible_user. Query the session directly (e.g. whoami) rather than assuming ansible_user is defined.
• Supply credentials explicitly for the second hop. A Kerberos ticket can't be delegated to a third machine; assert the credential is present, then pass it explicitly (e.g. become: runas).

Documentation

Per-role CLAUDE.md

Each role repo carries its own CLAUDE.md documenting behavior specific to that role — its tags, variables, task files, and quirks — committed to the role repo. roles/certificate_authority/CLAUDE.md is the reference example. Keep the parent ansible-epic CLAUDE.md limited to what is global across all roles or specific to the parent repo.

Because each role is its own git repo, someone working inside a role sees only that role's CLAUDE.md. Each role's CLAUDE.md should therefore include a short pointer back to the parent ansible-epic CLAUDE.md / these standards as the home of the global rules — framed as "when developing within the ansible-epic monorepo," since the role has its own remote and may be consumed standalone.

Every role has a usage guide

Every role must have, at minimum, a usage-guide page on this docs site, structured per the Role Documentation standard: a brief guide covering Requirements (collections, roles, and required internet access with exact destinations — customers build egress allowlists from it), Prerequisites (other roles that must run first, certificates, anything the role assumes exists), and the canonical commands — with the tag/variable reference and scenario samples split into that standard's companion pages. New roles are added under the Roles section of the nav: in mkdocs.yml, and mkdocs build --strict must pass before the docs change is done.

Keep CLAUDE.md, argument specs, and docs in sync

CLAUDE.md (parent and per-role), meta/argument_specs.yml, and the usage guide must be kept in sync with development. Any change to a role's variables, tags, defaults, prerequisites, or behavior updates all three that apply in the same piece of work — not as a follow-up. Don't let docs drift from what the code actually does.

• New/renamed/removed variables or tags → update meta/argument_specs.yml.
• Behavior/quirks/conventions → update the relevant CLAUDE.md.
• User-facing usage → update the usage guide, and run mkdocs build --strict.

Repository & commits

Role repositories are private

Every role repository (Sapphire-Health/ansible-role-<name>) is private. Roles are customer/infrastructure automation and are not published publicly. This also matters for tooling that clones the role repos — CI mints a GitHub App installation token to reach them, which a public-repo assumption would get wrong. New role repos are created private and stay private.

One repo per role — commit routing

roles/ is git-ignored in the parent repo; every role is its own git repository with its own remote (Sapphire-Health/ansible-role-<name>). Commits route by ownership:

• Changes under roles/<name>/ are committed and pushed from inside that role's repo (git -C roles/<name> …), against its own remote.
• Changes to parent-repo files (playbooks, group_vars/, inventory, docs/, coder/, CLAUDE.md, etc.) are committed and pushed from the parent repo.

Never stage a role's files from the parent repo, and never mix role changes and parent changes in one commit. When a task spans both (e.g. a new role plus the playbook that calls it), make two separate commits in the two separate repos. Note that git status in the parent repo will not show changes made inside a role subrepo — check the role repo directly.

Roles are searchable but stay uncommitted: the root .ignore file

Because roles/ is git-ignored, ripgrep-based search — VS Code workspace search, Quick Open, and rg in a terminal — would normally skip role code entirely, making a role cloned into ./roles invisible to search while you develop it. The .ignore file at the repo root fixes this with a single whitelist entry (!roles/): .ignore uses gitignore syntax but takes precedence over .gitignore for search tools, while git itself never reads it. Roles therefore show up in search but remain untracked in the parent repo. If search ever seems to "miss" role files, check that this file is present before touching search.useIgnoreFiles (turning that off would drag venv, site, and .terraform back into search results).