In this post, I'll be expanding on the previous instructions for installing Hashicorp Vault using Ansible

While setting up HCP Vault and in the role, I created to prove it was possible to populate the new HCP Vault instance from code I made use of the root/admin token.
While this obviously works, the model promoted within HCP Vault is one of "least privilege". Giving the least amount of access needed to get something done.
While there is some setup that using the Admin Token can and should be used for, one of the first things which HCP suggest getting setup are roles and policies.
In much the same way you wouldn't run a Linux box as root, you don't run HCP Vault using the admin token.
This post is split into two distinct areas the bash API code using Curl followed by the Ansible implementation and the choices made while doing this.
Things to know
The more astute will notice I'm using --insecure at the end of each curl line. the Dev HCP Vault server is using self-signed certs and I've set up HTTPS access to the API
you may need to install
jq
I've used a couple of variables as well
$VAULT_TOKEN = Random HCP Vault admin token
$VAULT_ADDR = https://127.0.0.1:8200
Reading
The first part of this post is pulled out of the HCP training

From Curl to Secrets

The flow to go from
- Enable AppRole
- Create a Policy to see the Secret store
- Create a role with limited access
- Pull the Details out to need to use the policy and role
- Show secrets accessible
Enable AppRole
curl --header "X-Vault-Token: $VAULT_TOKEN" --request POST --data '{"type": "approle"}' $VAULT_ADDR/v1/sys/auth/approle --insecure
CREATE a policy
Generate a policy file
This will provide read and update access, but not create
tee payload.json <<"EOF"
{
"policy": "# Read-only permission on 'homeserver/*' path\npath \"homeserver/*\" {\n capabilities = [ \"read\", \"update\" ]\n}"
}
EOF
Create a policy
curl --header "X-Vault-Token: $VAULT_TOKEN" --request PUT --data @payload.json $VAULT_ADDR/v1/sys/policies/acl/homeserver --insecure
CREATE a role
This file will create a name the token will last 1hr by default
There are additional items
tee rolepayload.json <<EOF
{
"token_policies": "homeserver",
"token_ttl": "1h",
"token_max_ttl": "4h"
}
EOF
create role
curl --header "X-Vault-Token: $VAULT_TOKEN" --request POST --data @rolepayload.json $VAULT_ADDR/v1/auth/approle/role/homeserver --insecure
test role
curl --header "X-Vault-Token: $VAULT_TOKEN" --request GET $VAULT_ADDR/v1/auth/approle/role/homeserver --insecure | jq
GET RoleID and Secret ID
Read the RoleID
curl --header "X-Vault-Token: $VAULT_TOKEN" $VAULT_ADDR/v1/auth/approle/role/homeserver/role-id --insecure | jq
Returns
{
"request_id": "f060d018-4298-3760-4005-d39fea3bd571",
"lease_id": "",
"renewable": false,
"lease_duration": 0,
"data": {
"role_id": "3f54ce6f-07a4-5dd8-11f0-b175094ea4ef"
},
"wrap_info": null,
"warnings": null,
"auth": null
}
Generate a Secret ID, remember this will last about an hour
curl --header "X-Vault-Token: $VAULT_TOKEN" --request POST $VAULT_ADDR/v1/auth/approle/role/homeserver/secret-id --insecure | jq
returns
{
"request_id": "96141158-7867-a52e-694c-e8c48ea04f4e",
"lease_id": "",
"renewable": false,
"lease_duration": 0,
"data": {
"secret_id": "7a0ceffb-41c0-a5e4-6880-df51530fb9b8",
"secret_id_accessor": "ea94b559-d0e4-f320-eacd-72c42712dff7",
"secret_id_ttl": 0
},
"wrap_info": null,
"warnings": null,
"auth": null
}
LOGIN using these creds
Pull the creds out into a json file and from here on out this is an app level token being used.
tee userpayload.json <<"EOF"
{
"role_id": "3f54ce6f-07a4-5dd8-11f0-b175094ea4ef",
"secret_id": "7a0ceffb-41c0-a5e4-6880-df51530fb9b8"
}
EOF
Authenticate to the HCP Vault server using these creds
curl --request POST --data @userpayload.json $VAULT_ADDR/v1/auth/approle/login --insecure | jq
This should return
{
"request_id": "78982acf-abd7-9ca6-5877-ad1769ca7b20",
"lease_id": "",
"renewable": false,
"lease_duration": 0,
"data": null,
"wrap_info": null,
"warnings": null,
"auth": {
"client_token": "s.2oNiabrasc5LRdCa1x9DUf2z",
"accessor": "k7VXQeICA5rTsjKGVjQliVrK",
"policies": [
"default",
"homeserver"
],
"token_policies": [
"default",
"homeserver"
],
"metadata": {
"role_name": "homeserver"
},
"lease_duration": 3600,
"renewable": true,
"entity_id": "179f2cc0-fb0d-390a-3100-3fd1cded2d59",
"token_type": "service",
"orphan": true
}
}
READ Secrets
Use the appRole secret token to pull out the passwords
curl --header "X-Vault-Token: s.2oNiabrasc5LRdCa1x9DUf2z" --request GET $VAULT_ADDR/v1/homeserver/dashboard --insecure | jq
returns
{
"request_id": "bdc9f2b8-0c99-00be-8e09-260c09c3bb34",
"lease_id": "",
"renewable": false,
"lease_duration": 2764800,
"data": {
"data": {
"password": "03TzpMHzNHHIg9K"
},
"options": {
"max_versions": "12"
}
},
"wrap_info": null,
"warnings": null,
"auth": null
}
Which contains the password for Dashboard
"password": "03TzpMHzNHHIg9K"
And if I wait an Hour and try the same thing again
curl --header "X-Vault-Token: s.2oNiabrasc5LRdCa1x9DUf2z" --request GET $VAULT_ADDR/v1/homeserver/dashboard --insecure | jq
Returns
{"errors": ["permission denied"]}
As per the policy and role which was setup.
How to turn this into Ansible?
Remember:
This is not production code, it is littered with debug statements to show what is happening, it would be better split out into a playbook and a role that could be used across other playbooks. I've also elected to use where possible the built-in Ansible modules rather than the hashi_vault in the Galaxy store.
The purpose of displaying like this to show what can be done.
---
- hosts: localhost
become_user: root
# https://tg-test100.com/creating-hashicorp-vault-records-with-ansible
vars:
vault_url: "https://{{ ansible_host}}:8200"
roles:
- gettoken
tasks:
- name: Enable the App Role
command: "vault auth enable approle"
##setup files
- name: Setup Policy Template
template:
dest: /root/hclreadpolicy.json
src: hclreadpolicy.json.j2
owner: vault
group: vault
mode: '0600'
- name: Setup Role Temaplate
template:
dest: /root/hclroletoken.json
src: hclroletoken.json.j2
owner: vault
group: vault
mode: '0600'
##import files
- name: Import RO Policy Temaplate
uri:
body_format: json
validate_certs: false
headers:
X-Vault-Token: "{{ vault_token }}"
url: "{{ vault_url }}/v1/sys/policies/acl/homeserver"
method: POST
src: /root/hclreadpolicy.json
remote_src: yes
status_code: "204"
- name: Import Role Temaplate
uri:
body_format: json
validate_certs: false
headers:
X-Vault-Token: "{{ vault_token }}"
url: "{{ vault_url }}/v1/auth/approle/role/homeserver"
method: POST
src: /root/hclroletoken.json
remote_src: yes
status_code: "204"
##Get RoleID and SecretID
- name: Get RoleID
uri:
validate_certs: false
headers:
X-Vault-Token: "{{ vault_token }}"
url: "{{ vault_url }}/v1/auth/approle/role/homeserver/role-id"
register: roleid
- debug: msg="{{ roleid }}"
- name: Get RoleID
uri:
body_format: json
validate_certs: false
method: POST
headers:
X-Vault-Token: "{{ vault_token }}"
url: "{{ vault_url }}/v1/auth/approle/role/homeserver/secret-id"
register: secretid
- debug: msg="{{ secretid }}"
##Pull Out Role ID
- name: Print returned json dictionary
debug:
var: roleid.json
- name: Print certain element
debug:
var: roleid.json.data['role_id']
- name: set_fact some paramater
set_fact:
sessionroleid: "{{ roleid.json.data['role_id'] }}"
- debug: msg="{{ sessionroleid }}"
##Pull Out secret ID
- name: Print returned json dictionary
debug:
var: secretid.json
- name: Print certain element
debug:
var: secretid.json.data['secret_id']
- name: set_fact some paramater
set_fact:
sessionsecretid: "{{ secretid.json.data['secret_id'] }}"
- debug: msg="{{ sessionsecretid }}"
##login to pull out approle_token
- name: Login to Vault to pull access token
uri:
body_format: json
validate_certs: false
headers:
X-Vault-Token: "{{ vault_token }}"
url: "{{ vault_url }}/v1/auth/approle/login"
method: POST
body:
role_id: "{{ sessionroleid }}"
secret_id: "{{ sessionsecretid }}"
status_code: "200"
register: rotoken
- debug: msg="{{ rotoken }}"
##Read secrets
- name: Print returned json dictionary
debug:
var: rotoken.json
- name: Print certain element
debug:
var: rotoken.json.auth['client_token']
- name: set_fact some paramater
set_fact:
rotokenid: "{{ rotoken.json.auth['client_token'] }}"
- debug: msg="{{ rotokenid }}"
What does this all mean?
The above code can be broken down into 5 key areas
Setting up the ingress files for the policy and role.
vars:
vault_url: "https://{{ ansible_host}}:8200"
roles:
- gettoken
tasks:
- name: Enable the App Role
command: "vault auth enable approle"
##setup files
- name: Setup Policy Template
template:
dest: /root/hclreadpolicy.json
src: hclreadpolicy.json.j2
owner: vault
group: vault
mode: '0600'
- name: Setup Role Temaplate
template:
dest: /root/hclroletoken.json
src: hclroletoken.json.j2
owner: vault
group: vault
mode: '0600'
The First command section Enables the AppRole Authentication method and if viewed in the WebUI this will be seen

The Template sections take the two files I'll use to upload the role and policy settings to the Vault server
hclroletoken.json.j2
{
"token_policies": "homeserver",
"token_ttl": "1h",
"token_max_ttl": "4h"
}
hclreadpolicy.jdon.j2
{
"policy": "# Read-only permission on 'homeserver/*' path\npath \"homeserver/*\" {\n capabilities = [ \"read\", \"update\" ]\n}"
}
I've created these as J2 templates because I will have the "homeserver" as a variable moving forward.
The code puts both files in /root and removes the .j2 at the end of the file name.
Importing the Policy and role definitions
##import files
- name: Import RO Policy Temaplate
uri:
body_format: json
validate_certs: false
headers:
X-Vault-Token: "{{ vault_token }}"
url: "{{ vault_url }}/v1/sys/policies/acl/homeserver"
method: POST
src: /root/hclreadpolicy.json
remote_src: yes
status_code: "204"
- name: Import Role Temaplate
uri:
body_format: json
validate_certs: false
headers:
X-Vault-Token: "{{ vault_token }}"
url: "{{ vault_url }}/v1/auth/approle/role/homeserver"
method: POST
src: /root/hclroletoken.json
remote_src: yes
status_code: "204"
This section ingresses the above files I've moved into /root into HCP vault and sets up a read only access and a 1hr before the secret is invalid.
The Policy looks as follows


The token lease expiration can be seen here.

Pulling out the RoleID and SecretID
##Get RoleID and SecretID
- name: Get RoleID
uri:
validate_certs: false
headers:
X-Vault-Token: "{{ vault_token }}"
url: "{{ vault_url }}/v1/auth/approle/role/homeserver/role-id"
register: roleid
- debug: msg="{{ roleid }}"
- name: Get RoleID
uri:
body_format: json
validate_certs: false
method: POST
headers:
X-Vault-Token: "{{ vault_token }}"
url: "{{ vault_url }}/v1/auth/approle/role/homeserver/secret-id"
register: secretid
- debug: msg="{{ secretid }}"
When these URi statements are run, they are done so to pull out the role_id and secret_id from the HCP Vault server however what is brought back is a lot of JSON code
role_id variable example
ok: [localhost] =>
msg:
cache_control: no-store
changed: false
connection: close
content_length: '208'
content_type: application/json
cookies: {}
cookies_string: ''
date: Mon, 24 May 2021 15:28:55 GMT
elapsed: 0
failed: false
json:
auth: null
data:
role_id: 1927d1f1-5114-c72c-5aed-7cdfeb510e02
lease_duration: 0
lease_id: ''
renewable: false
request_id: 65617dfd-7ca5-8569-82f1-3f0679f0a5b8
warnings: null
wrap_info: null
msg: OK (208 bytes)
redirected: false
status: 200
url: https://127.0.0.1:8200/v1/auth/approle/role/homeserver/role-id
The line we want is
role_id: 1927d1f1-5114-c72c-5aed-7cdfeb510e02
and its the same with the secret URI which you'll notice is a POST not a GET
ok: [localhost] =>
msg:
cache_control: no-store
changed: false
connection: close
content_length: '288'
content_type: application/json
cookies: {}
cookies_string: ''
date: Mon, 24 May 2021 15:28:55 GMT
elapsed: 0
failed: false
json:
auth: null
data:
secret_id: 098ccae8-6568-fa11-a948-d06bda1db290
secret_id_accessor: 9d899248-b13c-5b7b-4e9b-0c11cf888b83
secret_id_ttl: 0
lease_duration: 0
lease_id: ''
renewable: false
request_id: aff8d444-08d3-4c4e-2a2a-9c3c83aff182
warnings: null
wrap_info: null
msg: OK (288 bytes)
redirected: false
status: 200
url: https://127.0.0.1:8200/v1/auth/approle/role/homeserver/secret-id
The line we want here is
secret_id: 098ccae8-6568-fa11-a948-d06bda1db290
Turn the JSON into the fields we need
##Pull Out Role ID
- name: Print returned json dictionary
debug:
var: roleid.json
- name: Print certain element
debug:
var: roleid.json.data['role_id']
- name: set_fact some paramater
set_fact:
sessionroleid: "{{ roleid.json.data['role_id'] }}"
- debug: msg="{{ sessionroleid }}"
##Pull Out secret ID
- name: Print returned json dictionary
debug:
var: secretid.json
- name: Print certain element
debug:
var: secretid.json.data['secret_id']
- name: set_fact some paramater
set_fact:
sessionsecretid: "{{ secretid.json.data['secret_id'] }}"
- debug: msg="{{ sessionsecretid }}"
To pull the lines we want as the variables sessionroleid and sessionsecretid a JSON filter is needed
Above is a VERY verbose method of pulling out the data needed from the JSON data the last section provided
The last Debug lines in each section output the following
role_id = sessionroleid
TASK [debug] *******************************************************************
ok: [localhost] =>
msg: 1927d1f1-5114-c72c-5aed-7cdfeb510e02
secret_id = sessionsecretid
TASK [debug] *******************************************************************
ok: [localhost] =>
msg: 098ccae8-6568-fa11-a948-d06bda1db290
Pull out the Secret ID to use to read the passwords in Vault
##login to pull out approle_token
- name: Login to Vault to pull access token
uri:
body_format: json
validate_certs: false
headers:
X-Vault-Token: "{{ vault_token }}"
url: "{{ vault_url }}/v1/auth/approle/login"
method: POST
body:
role_id: "{{ sessionroleid }}"
secret_id: "{{ sessionsecretid }}"
status_code: "200"
register: rotoken
- debug: msg="{{ rotoken }}"
##Read secrets
- name: Print returned json dictionary
debug:
var: rotoken.json
- name: Print certain element
debug:
var: rotoken.json.auth['client_token']
- name: set_fact some paramater
set_fact:
rotokenid: "{{ rotoken.json.auth['client_token'] }}"
- debug: msg="{{ rotokenid }}"
Once the role_id and secret_id are captured they can be used with a URi POST to approle/login to generate again another JSON stream which looks like this.
ok: [localhost] =>
msg:
cache_control: no-store
changed: false
connection: close
content_length: '484'
content_type: application/json
cookies: {}
cookies_string: ''
date: Mon, 24 May 2021 15:28:56 GMT
elapsed: 0
failed: false
json:
auth:
accessor: DN6V1Ny8mzi9KyPhbYCgvFI0
client_token: s.8fpZZ58WiYSsC5ghQ9brmwyn
entity_id: cd029087-02ca-9eb0-8fa2-784153c81638
lease_duration: 3600
metadata:
role_name: homeserver
orphan: true
policies:
- default
- homeserver
renewable: true
token_policies:
- default
- homeserver
token_type: service
data: null
lease_duration: 0
lease_id: ''
renewable: false
request_id: 9a3e9552-dbe5-831e-484d-5c4675c07cc8
warnings: null
wrap_info: null
msg: OK (484 bytes)
redirected: false
status: 200
url: https://127.0.0.1:8200/v1/auth/approle/login
The line we want is
client_token: s.8fpZZ58WiYSsC5ghQ9brmwyn
Which the remainder of the ansible then pulls out as the read-only token variable rotokenid
TASK [debug] *******************************************************************
ok: [localhost] =>
msg: s.8fpZZ58WiYSsC5ghQ9brmwyn
This can then be used with other playbooks to pull the passwords in a secure manner out of Hashicorp Vault.
Thoughts
I can't state enough, this code needs work done to it to make it production-ready, when running in the automation tool, in my case Jenkins all these passwords and tokens are plainly visible in the output code.

That's not acceptable and some remove of debug statements and no_log=True need to be added at the very least.
Having all this in 1 playbook is nice, however ideally the last couple of sections should be a role that can be called as needed.
However work it does, using native Ansible modules and it's quick.
Using this as a template it's not a leap of faith to create RW roles as well.
Reference
