Intention for this repository is to provide a basic introduction to Ansible while also providing a usable base structure with examples.
The goal is not to cover everything Ansible is capable of but enough to make practical use of it.
| đź““ NOTE |
|---|
The examples are tested on a Debian based System. |
| Be aware of possible differences on other Systems. |
Ansible is an IT automation tool. It can be used to Configure and Deploy Systems and Software.
It can be viewed, in a classical way, as a collection of installer script to ensure a System reaches a desired state of operation, but with the intend of making these scripts reusable on multiple Systems while making it possible to re-run the same scripts on a System multiple times without making changes if the desired state has already been reached (idempotent).
Through SSH it connects to Systems and Devices and manages them based on YAML and Python Code.
Because of this Ansible is essentially agent-less. The target Systems only need to be able to accept SSH connections (e.g. openssh-server).
It also has decent Documentation.
Key term definitions can be found here.
Install Ansible on your System:
sudo apt update
sudo apt install -y openssh-client python3 python3-pip ansible
pip install --user --upgrade pip
pip install --user --upgrade ansibleI always recommend to update ansible via pip to the latest version, this way you will be warned in advance about deprecations when using it.
Also, new features.
Default search paths are:
~/.ansible//usr/share/ansible//etc/ansible/But it is also possible to use a different path, like we will be this repository structure.
There are many ways in how to build the directory structure for Ansible.
You can find additional examples here.
The structure within this repository is just what worked best for me.
- In the base directory is the Ansible Configuration file
ansible.cfgand the so-called playbooks*.yml/*.yaml(more later). - The
inventorydirectory contains one or multiple inventory files (all files are read automatically) that contain hosts that can be combined by groups. host_varsandgroup_varscan contain files named after hots or groups as they are defined in the inventories, they are matched automatically.- In the
rolesdirectory are multiple roles that each contain individual tasks and generic files, variables etc. to fulfill those tasks. - The
filesdirectory is not Ansible related, I just found it useful to have a separate directory for files that I want to copy to multiple target Systems (e.g. public SSH-Keys).
It is possible to have an own directory for the playbooks but you would have to define the path for the roles in the ansible.cfg using roles_path, because Ansible assumes that roles are stored in a sub-directory where the playbooks are.
Ansible is capable of reading the ~/.ssh/config.
Through this you con't have to manage any hosts file.
Just add the Systems that you would like to mange with Ansible into the configuration like:
Host sample1
HostName target-server.com
User user1
IdentityFile ~/.ssh/key_file_name1
Host sample2
HostName 172.1.0.5
User user2
PreferredAuthentications password
Host sample3
HostName 10.1.0.5
IdentityFile ~/.ssh/key_file_name3
ProxyJump jump-host
Host *
User user_default
Port 22
PreferredAuthentications publickey
IdentityFile ~/.ssh/default_key_file
IdentitiesOnly yes
ForwardAgent no
UseRoaming no
SendEnv LANG LC_*
Compression yesIt is recommended to avoid always reusing the same SSH-Key and also having one without passphrase.
If having individual SSH-Keys and passphrases for each System is too tedious consider having one SSH-Key for a group of Systems or one for each customer or whatever fits your infrastructure and requirements.
# Generating SSH Key
ssh-keygen -t ed25519 -a 64 -b 4096 -C "Ansible SSH Key" -f ~/.ssh/ansible_keyHere are some great talks about SSH and how it can be used in a good way:
Talk about SSH by Leyrer Part1 (german)
Talk about SSH by Leyrer Part2 (german)
Talk about SSH by Leyrer (english)
Some are saying that to manage Windows Systems winrm(Windows Remote Management) can/should be used.
I highly discourage you to do that, as you can simply use SSH.
Also the example scripts that you may find online or by Microsoft to enable winrm usually aren't meant for production and disable security features.
To remove the need to always insert the passphrase for the SSH-Keys, the ssh-agent command can be used.
For some reason the SSH-Agent requires to be started for each new terminal session.
This can be circumvented by auto-loading the SSH-Agent env vars on each terminal session.
I am adding it to /etc/bash_completion.d because avoiding adding anything and everything to .bashrc should be preferred, also having individual files makes cleanup easier.
echo 'eval "$(ssh-agent -s)" > /dev/null' | sudo tee /etc/bash_completion.d/ssh-agent.bash_completion > /dev/nullAlternatively creating a .bashrc.d directory in your home is also recommended:
mkdir ~/.bashrc.d
chmod 700 ~/.bashrc.d
sudo tee -a ~/.bashrc > /dev/null <<EOT
for file in ~/.bashrc.d/*.bashrc;
do
source “\$file”
done
EOT
echo 'eval "$(ssh-agent -s)" > /dev/null' | sudo tee ~/.bashrc.d/ssh-agent.bashrc > /dev/null
chmod +x ~/.bashrc.d/*.bashrcNow you can place files inside of that directory as <filename>.bashrc, but you may have to add execution permission to each file.
chmod +x ~/.bashrc.d/*.bashrcNow you can load the public key into your SSH-Agent which requires you to only enter the passphrase ones for as long as the terminal session is open.
ssh-add ~/.ssh/key_file_nameIt is possible to have SSH-keys in the TPM 2.0 Chip, but I haven't used that yet.
It is well explained here:
Talk about SSH by Leyrer Part2 (german)
Linux (Debian):
sudo apt update
sudo apt install -y openssh-server python3Windows Powershell:
Get-WindowsCapability -Online | Where-Object Name -like 'OpenSSH*' -OutVariable cmdOutput
# should return two entries
# OpenSSH.Client~~~~0.0.1.0
# OpenSSH.Server~~~~0.0.1.0
Add-WindowsCapability -Online -Name $cmdOutput[0].name
Add-WindowsCapability -Online -Name $cmdOutput[1].name
Start-Service sshd
# Add SSH to autostart
Set-Service -Name sshd -StartupType 'Automatic'After the OpenSSH Server is installed add the SSH-Keys from the System where Ansible will be run from to the target System.
# From Ansible System
ssh-copy-id -i ~/.ssh/key_file_name <target_system>Also consider disabling the option to allow login via SSH using Password (after you made sure login via SSH-Key works):
#/etc/ssh/sshd_config
PasswordAuthentication nosystemctl restart sshdList of settings
The ansible.cfg allows you to configure Ansible to your needs.
E.g.:
- add log directory
- change directories for playbooks, inventories, etc.
- set defaults
- ...
To generate a fully commented-out example ansible.cfg run:
ansible-config init --disabled > ansible.cfgOr to also include existing plugins:
ansible-config init --disabled -t all > ansible.cfgFor this repository I have preset two settings:
First the path for the directory containing the inventory files. This one is required, otherwise Ansible won't find the files, sadly this also means you have to be in the directory where all the Ansible files and dirs are located ot it won't work (except if you change the config to the full path).
Secondly the option to make the terminal output formatted in YAML, which is optional and can be removed if needed.
Download this repository and switch into its directory.
Run the first example (Connections to localhost don't require SSH):
ansible-playbook example_01.yml
The inventory which contains a structure of your host, groups and if desired variables and connection settings can be built in multiple ways.
The most common ones are either as YAML or ini.
As there is not much of a difference, except for how the files are structured, I will only go over the YAML structure for now.
---
all:
children:
internal:
children:
ansible_vm:
hosts:
local_system:
ansible_connection: local
host_test_var: test
vars:
group_test_var: test
hosts:
ungrouped_host:
...
- The
---and...mark the beginning and end of aYAMLfile and are used in all of them. The...is optional but I add anyway. - Under
allyou place all your hosts and groups, if needed it is also possible to define hosts separately. - With
childern:you can define sub-groups to separatehostsby type, task, service, cluster or however you want hostscontains all Systems for a group or sub-group.- You can define any variable or connection option either directly below a specific System or for a whole group or sub-group.
When running a playbook you can specify with -i <path> which inventory should be used.
With my configuration any YAML or ini will be automatically loaded from the inventory/ directory.
When building your inventory be careful with naming a group, because groups with the same name will be grouped together when called directly. Whis is also known as metagroup
This can be seen when running the following example:
ansible-playbook example_host_grouping.ymlIn the recap you will see two host because both are in the sub-group docker_host even though they are in separate different sub-groups.
Because of this I recommend using unique names for a group like adding a suffix or prefix to the name of the parent group or something like prod_ and staging_.
But you can also use this function to build your inventory like this:
leafs:
hosts:
leaf01:
ansible_host: 192.0.2.100
leaf02:
ansible_host: 192.0.2.110
spines:
hosts:
spine01:
ansible_host: 192.0.2.120
spine02:
ansible_host: 192.0.2.130
network:
children:
leafs:
spines:
webservers:
hosts:
webserver01:
ansible_host: 192.0.2.140
webserver02:
ansible_host: 192.0.2.150
datacenter:
children:
network:
webservers:This example contains a metagroup network that includes all network devices and a metagroup datacenter that includes the network and webservers groups.
ansible-playbook example_inventory.ymlPlaybook guide
Official examples
A playbook is the starting point similar a setup script that is executed.
In a playbook you define a group from the inventory which will be targeted by its content, using the hosts variable.
---
- hosts: linux_server
...
It is also possible to exclude one or multiple sub-groups:
---
- hosts: linux_server !webservers
...
And wildcards:
---
- hosts: linux_*
...
It can also contain connection settings to override those in the inventory.
You can also define settings like become: (default: false) to define if everything should be run with privilege escalation (sudo) or gather_facts: (default: true) to collect facts which contain information about the target System like operating system, IP and more, or serial:to define how many Systems should be configured at the same time and many more settings.
Additionally you can define additional vars that may also contain or overwrite connection variables like ansible_user, ansible_host or ansible_port.
It is also possible to define variables under vars_prompt: to ask for input for specific variables that you don't want to write down in any file.
Important to note that become: true is handled differently when used at the beginning of a playbook compared to when used in a specific task or role.
When used at the beginning of a playbook Ansible will change the user on the target host to root, while when used in e.g. a task it is similar to using sudo.
This is a majorly change that should be taken into account when e.g.: using ansible_user_id.
Using become: true in the heading of a playbook should be avoided if possible as running everything as root also may create undesired security issues.
After the first 'configuration' blog it is possible to define one or all of the following sections:
- pre_tasks: list of tasks executed before roles
- roles: list of roles to be imported
- tasks: list of tasks executed after roles
- post_tasks: list of tasks executed after tasks section
- handlers: list of tasks, but ony run when a task contains the
notifykeyword and has the state changed.
Futhermore you can define multiple of these sets in one playbook:
---
- name: Configure Linux server
hosts: linux_server
tasks: ...
- name: Configure Windows server
hosts: windows_server
tasks: ...
...
A playbook is run with the command:
ansible-playbook <playbook_name>.ymlWith ansible-playbook --help you can also check out its multiple additional options.
ansinle-playbook example_playbook.ymlSee how in this example windows_docker is in the list of targeted host because of the docker_host group.
So again, be careful with how you name your groups.
A task is a singular action that should be run. It can be anything from a simple console command, to disk formatting, network configuration and more. Every possible action is grouped in namespace and a collection reaching from builtin to community managed.
There is also the ansible-galaxy which contains even more collections outside of the official documentation.
| Namespace | Collection | Module |
|---|---|---|
| ansible | builtin | debug |
| community | general | sudoers |
| kubernetes | core | helm |
The example_tasks.yml file contains basic examples in how tasks are structured.
ansible-playbook example_tasks.ymlA task can additional to it's action contain where, when and with what privileges it should run and more.
Each task is at the end like a Python module defined in a collection within a namespace.
This is also known as Fully Qualified Collection Name (FQCN).
While it is possible to only use the module name (like debug) it is highly recommended to use the FQCN to ensure you don't accidentally run a module from a different collection or namespace that has the same name.
Handlers are singular tasks that only run when notified by a task and only when that task has the status changed.
This can be useful for example when a change in configuration request the restart of a service but the service shouldn't be restarted when it is not needed.
- name: Update package cache
ansible.builtin.package:
update_cache: true
notify: handler1
- name: Update package cache
ansible.builtin.package:
update_cache: true
notify:
- handler1
- handler2Handlers are run in the order they are defined in the handlers section and not in the order of the notify section.
handlers can have utilize the keyword listen so that multiple handlers can be grouped together.
tasks:
- name: Restart everything
command: echo "this task will restart the web services"
notify: "restart web services"
handlers:
- name: Restart memcached
service:
name: memcached
state: restarted
listen: "restart web services"
- name: Restart apache
service:
name: apache
state: restarted
listen: "restart web services"By default, handlers run after all the tasks in a particular play have been completed. Notified handlers are executed automatically after each of the following sections, in the following order: pre_tasks, roles/tasks and post_tasks. This approach is efficient, because the handler only runs once, regardless of how many tasks notify it. For example, if multiple tasks update a configuration file and notify a handler to restart Apache, Ansible only bounces Apache once to avoid unnecessary restarts. If you need handlers to run before the end of the play, add a task to flush them using the meta module, which executes Ansible actions:
tasks:
- name: Some tasks go here
ansible.builtin.shell: ...
- name: Flush handlers
ansible.builtin.meta: flush_handlers
- name: Some other tasks
ansible.builtin.shell: ...The meta: flush_handlers task triggers any handlers that have been notified at that point in the play.
Once handlers are executed, either automatically after each mentioned section or manually by the flush_handlers meta task, they can be notified and run again in later sections of the play.
Handlers from roles are not just contained in their roles but rather inserted into the global scope with all other handlers from a play. As such they can be used outside of the role they are defined in. It also means that their name can conflict with handlers from outside the role. To ensure that a handler from a role is notified as opposed to one from outside the role with the same name, notify the handler by using its name in the following form: role_name : handler_name.
Handlers notified within the roles section are automatically flushed at the end of the tasks section but before any tasks handlers.
| :notebooke: Note |
|---|
Handlers can run include_tasks, import_task and meta module |
Handlers can't run include_role, import_role, and meta module flush_handlers |
The example_handlers.yml file contains basic examples for handlers.
ansible-playbook example_handlers.ymlA role can be viewed as a set of tasks and other things grouped together.
It has a defined directory structure with eight main standard directories, but it is only required to include at least one of them. Directories not used can be omitted.
roles/
example_role/ # this hierarchy represents a "role"
defaults/ #
main.yml # <-- default lower priority variables for this role
tasks/ #
main.yml # <-- tasks file can include smaller files if warranted
handlers/ #
main.yml # <-- handlers file
files/ #
bar.txt # <-- files for use with the copy resource
foo.sh # <-- script files for use with the script resource
templates/ # <-- files for use with the template resource
ntp.conf.j2 # <------- templates end in .j2
vars/ #
main.yml # <-- variables associated with this role
meta/ #
main.yml # <-- role dependencies
library/ # roles can also include custom modules
module_utils/ # roles can also include custom module_utils
lookup_plugins/ # or other types of plugins, like lookup in this case
webtier/ # same kind of structure as "example_role" was above, done for the webtier role
monitoring/ # ""
fooapp/ # ""Ansible looks for roles in the following locations:
- in collections (if you are using them)
- in a directory named
roles/, relative to the playbook file - in the configured
roles_path. The default search path is~/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles - in the directory where the playbook file is located
Alternatively the full path can be provided when calling a role:
---
- hosts: webservers
roles:
- role: '/path/to/my/roles/common'
...
You can use roles in three ways:
- At the play level with the
rolesoption: This is the classic way of using roles in a play. - At the tasks level with
include_role: You can reuse roles dynamically anywhere in thetaskssection of a play usinginclude_role. - At the tasks level with
import_role: You can reuse roles statically anywhere in thetaskssection of a play usingimport_role.
By default a role is only run once no matter how often it is defined in a playbook.
There are two ways to circumvent this:
- Passing different variables to the role
- Within
meta/main.ymlof a role setallow_duplicates: true
There is also the ansible-galaxy which contains even more roles.
ansible-playbook example_role.ymlUncommenting allow_duplicates: true in roles/example_role/meta/main.yml will show the difference when passing different variables.
Using Variables Filter plugins
Variables can be bool, integer, string, list, dictionary and any combination of lists and dictionaries
vars:
int_var: 1
string_var: fuu
list_var_1: [ fuu, bar, 1 ]
list_var_2:
- fuu
- bar
- 1
dict_var_1: { name: fuu, number: 2 }
dict_var_2:
name: fuu
number: 1
list_dict:
- name: fuu
number: 3
- name: bar
number: 4
dict_list:
names:
- fuu
- bar
number: 5There are also Special Variables and Facts that can be used as variables.
These contain variables containing connection information, paths to ansible files/dirs, facts about a host like OS, IP Address und much more.
Variables can be defined and overwritten in multiple places, which results in a list of Variable Precedence from least to greatest (the last listed variables override all other variables):
- role defaults (defined in role/defaults/main.yml) 1
- inventory file or script group vars 2
- inventory group_vars/all 3
- playbook group_vars/all 3
- inventory group_vars/* 3
- playbook group_vars/* 3
- inventory file or script host vars 2
- inventory host_vars/* 3
- playbook host_vars/* 3
- host facts / cached set_facts 4
- play vars
- play vars_prompt
- play vars_files
- role vars (defined in role/vars/main.yml)
- block vars (only for tasks in block)
- task vars (only for the task)
- include_vars
- set_facts / registered vars
- role (and include_role) params
- include params
- extra vars (for example, -e "user=my_user")(always win precedence)
To use variable, they can be referenced via double curly braces (Jinja2 syntax) "{{ int_var }}".
To create valid YAML syntax, usually the whole expression containing a variable needs to be quoted.
This is usually not needed when used in a when: statement.
It is also possible to define Custom Facts.
They will be added automatically when running gather_facts, but they can also be loaded manually with optionally adding path using:
tasks:
- name: Filter and return only selected facts
ansible.builtin.setup:
filter:
- 'ansible_local'
fact_path: "{{ playbook_dir }}/facts.d"The default path Ansible is looking for is /etc/ansible/facts.d but that can be changed in ansible.cfg with fact_path=.
Custom facts need to be either in JSON or INI format.
The content of the files under facts.d can be anything, just the return value needs to be correctly formatted.
This allows to also run script files (bash, python, ...) that return e.g. JSON formatted key/value pairs.
For that the scripts have to be executable with shebang (e.g.: #!/usr/bin/env bash) and proper permissions (e.g.: chmod +x facts.d/date_time.fact).
ansible-playbook example_vars.yml
ansible-playbook example_facts.yml
ansible-playbook example_custom_facts.yml
ansible-playbook example_query.yml
ansible-playbook example_lookup.yml
ansible-playbook example_vars_nested.ymlAnsible allows to check for specific conditions before running a task, role, block, etc. using the when keyword.
It can be used to check if some variable contains desired information, like if something is set to true, is a specific package installed, etc..
when can either be a single line string or a multiline list.
When stated with a list each part in that list is connected as if and is used.
Conditions can be linked together with and and or keywords.
ansible-playbook example_conditionals.ymlLoops allows repetitive execution of a single task.
This can be useful to e.g. install multiple packages, create multiple user, etc..
Most commonly loops are used to iterate over a list where each singular part is available with the "{{ item }} keyword.
ansible-playbook example_loops.ymlLoops can be managed with the loop_control keyword.
It allows to set a labelwhich can be used to define the console output during the task run. This can be used e.g. when iterating over list of dicts "{{ item.name }}".
Additionally it is possible to extend the loop information by setting extended: true in the loop_control, which adds the following information:
| Variable | Description |
|---|---|
| ansible_loop.allitems | The list of all items in the loop |
| ansible_loop.index | The current iteration of the loop. (1 indexed) |
| ansible_loop.index0 | The current iteration of the loop. (0 indexed) |
| ansible_loop.revindex | The number of iterations from the end of the loop (1 indexed) |
| ansible_loop.revindex0 | The number of iterations from the end of the loop (0 indexed) |
| ansible_loop.first | True if first iteration |
| ansible_loop.last | True if last iteration |
| ansible_loop.length | The number of items in the loop |
| ansible_loop.previtem | The item from the previous iteration of the loop. Undefined during the first iteration. |
| ansible_loop.nextitem | The item from the following iteration of the loop. Undefined during the last iteration. |
Sometimes it can also be useful to loop over something but only do something when an item meets a specific condition.
ansible-playbook example_loop_conditionals.ymlThrough blocks it is possible to group multiple tasks together and is the only way to create proper exception handling.
They are defined like tasks with all tasks being grouped below it through indent and directives like become defined on the block definition are inherent by all tasks grouped within the block.
Should a task within a block fail sections like rescue and always can be defined.
rescue will run should a task within block fail and always will run no matter what happened in block or rescue.
- name: Sample block
become: true
block:
- name: Some task
ansible.builtin.debug:
msg: "some task"
rescue:
- name: Some task when block fails
ansible.builtin.debug:
msg: "some task"
always:
- name: Some task always run
ansible.builtin.debug:
msg: "some task"The example_block.yml file contains basic examples for block, rescue, always.
ansible-playbook example_block.ymlTemplate testing with lookup
Template testing with python script
Jinja2 is a template engine for Python used by Ansible nearly everywhere (e.g. variables, when...).
It can also be used to define templates for any kind of file that should be placed on the target system.
These templates can containing anything from variables, conditionals to loops and more.
The example_role contains a short example in how it can be used.
A more detailed example in how a Jinja2 template file can look, please check out this repository.
Within a role:
- name: Generate file from Template
ansible.builtin.template:
src: example.j2
dest: /etc/exportsAnsible provides vault capabilities to encrypt files or strings.
This part is kept short to give you a rough idea on what you can do with it, but it is recommended to use a proper Vault like HashiCorp Vault, because it provides permission management and more.
The Ansible vault is essentially just an encrypted file providing limited security improvement.
You can use either fully encrypted files containing multiple variables or a singular encrypted string.
In both cases you have to provide a password through either prompt or a file:
ansible-playbook example_vault.yml --ask-vault-password
# password: testansible-playbook playbook.yml --vault-password-fileShould you need to provide different passwords for your vault files/strings, you have to provide for each of them a Vault ID with --vault-id label@source.
It is also possible to provide the information in the ansible.cfg, e.g.:
ask_vault_pass=False
vault_encrypt_identity=
vault_identity=default
vault_password_file=Create a Vault file:
env EDITOR=nano ansible-vault create <Vault>.ymledit a Vault file:
# Using default editor (vi)
ansible-vault edit <Vault>.yml
# Specifying editor
env EDITOR=nano ansible-vault edit <Vault>.yml- name: Vault
hosts: localhost
vars_files:
- "vault.yml"ansible-playbook <name>_playbook.yml --ask-vault-passTo create a single encrypted string that can be used as a variable run:
ansible-vault encrypt_string secure_password
# Type in your desired passwordThis returns a multiline string that can be added in full directly in a playbook.
vars:
my_secret: !vault |
$ANSIBLE_VAULT;1.1;AES256
35643133326464663036346464306666623738343062636663666536346239396233626331346435
6335623333626634316262663337336533333730353633660a346530316433656533613332363438
34303163353263353263613938653239353266393735626361346233323831653061336338653162
3637626338613261360a323938656564383734303139383135343662313230663839383738646430
3433There are some additional tools provided by Ansible that may be of interest, but since I haven't used them yet, they will only be listed here.
Containerized Execution Environments
Ansible Builder guide
- Test your playbooks and roles on a minimal version of your desired OS Distribution to ensure you have proper tasks added to install dependencies
- Increase verbosity with -v, -vv ...
- To save the output of an Ansible run to a file, set
log_path=in theansible.cfg
ToDo: Finish:
- Grouping and Error handling
- Jinja2 Templates