Following on from my previous post which guides you through setting up Rundeck and adding Nodes.
David Field
The next step is to run some simple Jobs on these remote nodes.
Assumptions
Prior to working through these instructions, the assumption is you have a working rundeck with nodes setup which Rundeck is able to run a test job against.
What is covered below?
This post will cover how to run a command line or a remote script on a remote box, it does not go into using Ansible (thats next on the list) or holding the script locally and using SCM (Later post). The idea here is to be able to get a sysadmin up and running with remote commands or scripts run from Rundeck. At this point I’m not looking to do anything Devops wise
Disclaimer
I’m learning this and putting forward what i’ve learnt, it may not be the best way to do things. If you feel there is a discussion about a better method please get in touch. I will ask you to explain your reasoning, its not me being rude, its me learning. If the reply and conversation seems to be workable, i’ll update the blog and cite accordingly.
Add New Job
What does this job do?
In this example I will setup a job which runs on a reverse proxy and updates my LetsEncrypt Certs (or at least confirms if any need updating) once a week. when the job runs it will send me notifications on Start, Success or Fail
Select the Project
On your Rundeck server login and select your project
In the left hand menu select Jobs and under Job actions select New Job
Setup the Job
Setting up a job in rundeck involves completing a set of cards with a set of actions
Details
Details are where the name of the job and a description are set, the description supports markdown. The group selection can be used to restrict the job to a specific group.
Don’t click save, click Workflow
Workflow
In the workflow section the commands and flow of what you want the job to do are setup
I’ve not used any options, within this menu however we ca do things like hide passwords and scripts
Under workflow we can:
- determine what to do if a step fails (this job only has one step)
- If we were running the command on multiple nodes, choose to do this sequentially, all at once or Chose a specific node order (This job will only run on one node)
- A set of filters based around logging can be applied to the job
- We can add steps for a job to follow
Click on Add Step and you are provided with a choice of steps you can use, out of the box there is a good selection and Rundeck has a solid plugin architecture to expand these steps
For this example I want to run a command on a remote node so I’d select the Command Node Step. Node Steps are run on each node
The other Tab Workflow steps run an action once per workflow on the rundeck box.
As we selected Command we are now presented with the command box which is the command we want to run on the remote box.
The step Label is what is displayed on the workflow screen and as part of the resulting run report.
Note: I’m able to use “sudo” because in the previous tutorial I setup rundeck to be aware of the sudo password.
Click on Save to save the command
Now click on the Nodes tab
Nodes
There are a lot of options on the Node screen where as the name would suggest is where we let the job know which node to run the job on. In this example I’m only running a command on a single node
Select Dispatch to Nodes
There is a list of available nodes under Matched Nodes, type in the name of the node you want to run the job on and click on Search. The filter can be a wildcard, inter* as an example.
I’m happy with the defaults for the rest of the screen, and they are self explanatory
Click on the Schedule tab
Schedule
This is where the start of the power of Rundeck starts to come to fruit, as a replacement for locally un-managed cron jobs just doing this simple LetsEncrypt update we can start to have centrally managed scheduled jobs.
- Schedule Run Repeatedly – Yes if you want the job to run as such (I do) You can select which day of the week, untick monthly and the jobs will run weekly.
- TimeZone – This will run on the timezone of the box, however if you want to manage servers in other timezones Rundeck does honour these (I tested it)
- Enable Scheduling – Set to Yes
- Enable Execution – Set to Yes
Click on the Notifications tab
Notifications
Notifications will either send you an email based on the type of notification you want or can use webhooks to sent updates to Slack or other products.
There are 3 main type of Notifications Success, Fail or Start, its also possible if you know a job may run for a set length of time to set a threshold and have a notification if a job over runs the time frame.
Select Send Notifications = yes and choose On Success
We can setup to send an email, include the log output file inline or as an attachment, who the message is sent from and where to.
Select the Other Tab
Other
The aptly named “other” screen has a whole heap of options, the one I set is to have the default view as being the Log View.
Click on Save
Run the Job
Once the job has been saved we can run the job and we will do this by clicking on the jobs screen within the project.
I’ve got several jobs, and from this screen there are a couple of ways of running the jobs the first option is by clicking on the Green start arrow next to the job name
Running Jobs – Option 1
Clicking on the Green Start button launches the Execute Job dialog box which has some options which allow you to change which nodes the job will run on and there is option on how to Follow the Execution of the job, where I usually select Log Output as it gives a good view to see what’s happening with your job.
Can now click on Run Job Now or the drop down next to it if you want to see the debug output of the job.
Running Jobs – Option 2
Clicking on the job name opens the Job Execution screen which gives you stats on the failed or successful job runs. You can also control the job.
In the Action menu you have options which are pretty self explanatory of which Edit the Job and Download Job definition in ____ options which we can look at later.
Under the Definition button a dialog is presented with the workflow of the job
Just as in option 1 there is a choice how to view the job running and as with Job 1 I prefer the Log Output
The last button is to run the job, as with option 1 clicking on the down button provides options including the additional deferring a job till later.
Job Running
Run with Log View the “screen output” is displayed on the log file. There are additional options on the screen which you can run the job again, delete the job log
The nodes button switches from the log view to the Node view (and in node view there is a Log Output button in the same place.)
Shows the node output.
Setup as code?
To view the code as a Yaml select the Action menu in Option 2 and choose Downlod Job Definition as YAML
This will result in a yaml file like the one below being created which can then be backed up or used with the rundeck-cli command
- defaultTab: nodes description: A monthly job run on server external to check if any LetsEncrypt certsneed updating. executionEnabled: true id: abdc8f24-b052-4b2f-be9f-6abd582cab4d loglevel: INFO name: Update_letsencrypt_certs nodeFilterEditable: false nodefilters:dispatch: excludePrecedence: true keepgoing: false rankOrder: ascending successOnEmptyNodeFilter: false threadcount: '1'filter: external nodesSelectedByDefault: true notification:onfailure: email: recipients: david@safewebbox.com subject: 'Rundeck: Failed - Letencrypt Update'onstart: email: recipients: david@safewebbox.com subject: 'Rundeck: Started - Letencrypt Update'onsuccess: email: attachLog: true attachLogInFile: true recipients: david@safewebbox.com subject: 'Rundeck: Success - Letencrypt Update' notifyAvgDurationThreshold: null plugins: ExecutionLifecycle: null schedule:month: '*'time: hour: '04' minute: '40' seconds: '0'weekday: day: '*'year: '*' scheduleEnabled: true sequence:commands:- description: Run Certbot Renew exec: sudo certbot renewkeepgoing: falsestrategy: node-first timeZone: Europe/London uuid: abdc8f24-b052-4b2f-be9f-6abd582cab4dThoughts
This post should provide a sysadmin with a bastion jumpbox enough to use tools like puppet-bolt or ansible at a command line level the ability to have scheduled jobs centralised and better managed.
However the true power of rundeck comes from its integrations and plugins, so the next post will be based around using Rundeck and Ansible, and if I can get it working using Ansible AWX (Ansible Towers feeder project, think Fedora to Redhat) and Rundeck to compliment each other.