Building a Custom AWX Execution Environment for Windows Automation

As enterprise Ansible environments grow, default execution environments often become insufficient for real-world Windows infrastructure automation. Many organizations discover that the stock AWX execution environment images lack required PowerShell modules, Python libraries, Windows collections, mail utilities, or enterprise-specific tooling.

Custom execution environments solve this problem by allowing infrastructure teams to create repeatable, version-controlled automation containers tailored specifically for their operational needs.

For Windows administrators managing patching, Active Directory, Exchange, monitoring, or endpoint automation, custom execution environments quickly become essential.

Why Build a Custom Execution Environment?

• Pre-install enterprise Ansible collections.
• Include PowerShell and Windows automation dependencies.
• Add Linux utilities required by playbooks.
• Maintain version consistency across automation jobs.
• Reduce repeated package downloads during execution.
• Support internal repositories and private registries.
• Improve AWX job reliability and portability.

Common Problems with Default AWX Images

Many administrators encounter issues shortly after deploying AWX using the default execution environments. Typical problems include missing collections, unsupported Ansible versions, missing sendmail utilities, or unavailable Windows automation dependencies.

• community.general collection incompatibilities
• Missing Chocolatey support
• No sendmail or s-nail package available
• PowerShell dependencies not installed
• ansible-core version mismatches
• Missing xmlsec or libxml dependencies
• Container registry pull and push failures

Recommended Architecture

A practical enterprise architecture typically consists of AWX running on Kubernetes while execution environments are hosted inside an internal container registry.

AWX Cluster
 |
 +-- Custom Execution Environment
 |
 +-- Internal Registry
 |
 +-- Windows Collections
 +-- PowerShell Modules
 +-- Reporting Tools

Building the Execution Environment

The recommended approach uses ansible-builder to generate a reproducible container image. This allows dependencies to remain version-controlled alongside automation projects.

version: 3
images:
  base_image:
    name: quay.io/ansible/awx-ee:latest
dependencies:
  galaxy: requirements.yml
  python: requirements.txt
  system: bindep.txt

Example Galaxy Requirements

collections:
  - name: ansible.windows
  - name: chocolatey.chocolatey
  - name: community.windows
  - name: community.general
  - name: microsoft.ad

These collections provide the foundation for enterprise Windows management workflows including patching, software deployment, Active Directory management, and reporting.

Adding System Dependencies

git
wget
curl
tar
gzip
krb5-workstation
mailx

Many organizations also add troubleshooting tools such as htop, nslookup, telnet, or SQL client utilities for infrastructure validation.

Building the Image

ansible-builder build -t awx-ee:community-general

The resulting image can then be pushed into an internal registry for AWX consumption.

podman tag awx-ee:community-general 192.168.128.154:5000/awx-ee:community-general
podman push 192.168.128.154:5000/awx-ee:community-general

Handling Registry Problems

Internal container registries frequently introduce operational issues during initial setup. Teams commonly encounter blob upload failures, ImagePullBackOff states, unknown manifests, or certificate trust problems.

• Validate registry firewall rules.
• Confirm registry storage permissions.
• Verify container image tags.
• Check certificate trust chains.
• Validate podman login authentication.
• Inspect registry logs during failed uploads.

Using the Execution Environment in AWX

Once uploaded to the registry, the custom execution environment can be added directly into AWX.

1. Navigate to Administration → Execution Environments.
2. Create a new execution environment.
3. Specify the internal registry image URL.
4. Assign registry credentials if required.
5. Attach the execution environment to job templates.

Real-World Enterprise Enhancements

Mature enterprise environments often extend execution environments significantly beyond basic collections.

• Custom PowerShell modules
• SQL Server client tools
• Internal certificate authorities
• Mail relay utilities
• API integration tooling
• Security scanning dependencies
• Infrastructure reporting scripts

Execution Environment Versioning

One operational best practice is separating production releases from rapid patch builds. Stable version numbering should remain predictable while hotfix revisions can use lightweight patch labels.

awx-ee:community-general-v2
awx-ee:community-general-hotfix-u3

This approach simplifies rollback procedures and helps infrastructure teams identify which execution environment versions are actively deployed.

Operational Lessons Learned

• Keep execution environments as lightweight as possible.
• Avoid unnecessary package installation.
• Pin collection versions when stability matters.
• Use staging execution environments before production rollout.
• Monitor registry storage growth.
• Validate execution environment compatibility after AWX upgrades.

Final Thoughts

Custom AWX execution environments are one of the most important long-term improvements an enterprise automation team can implement. Properly designed execution environments increase automation reliability, reduce troubleshooting time, and provide a consistent foundation for infrastructure-as-code operations.

For Windows infrastructure automation specifically, custom execution environments frequently become the difference between fragile automation and scalable enterprise orchestration.