Chatops (Operations automation with a bot)
Table of Contents
1 - About
- Automating common operations task with a bot.
- Breaking communication wall between different teams.
The bot is the primary control for the whole infrastructure.
The chat becomes a console
By placing tools directly in the middle of the conversation, everyone is pairing all of the time. Teaching by making things visible. Jesse Newland
2 - Articles Related
3 - Chat
- Campfire is a web-based group chat tool that lets you set up password-protected chat rooms in just seconds. Invite a client, colleague, or vendor to chat, collaborate, and make decisions. Link to a room on your intranet for internal communications.
4 - Bot
4.1 - Hubot
Ryan wrote Hubot at Github.
4.1.1 - rolling a new version
git checkout -b git-gh13 # bump version number git commit -am 'Bump git version' git push origin git-gh13 # create pull request
Change as passed CI (ie Build was succesful)
hubot deploy puppet to production
- noop means simulate what happens on that branch git-gh13 against the file server fs1, noop mode simulates the changes
hubot puppet noop git-gh13 fs1
hubot puppet force git-gh13 fs1
hubot graph me -1h @collectd.load(fs1)
- The pull request is good, we want to push it in production (everywhere)
#merge pull request
- To ensure that the master is green
ci runs # Build 1087 (eragaf66&&) of puppet/master was successful (399s)
- When the CI passes, the code is deploy in production
- Now a new version of git is on master. It has been shipped everywhere.
# Puppet shipped it on a interval but you can push it hubot puppet force production fs
- Graph of everything (of all file servers)
hubot graph me -1h @collectd.load(fs*)
- Logs to make sure that we didn't break anything.
hubot log me hooks github/github
4.1.2 - Others
- CI status of a random branch
hubot ci status github/smoke-perf
- Roll it out to one file server
hubot deploy github/smoke-perf to prod/fe1
- Verifying things are return to normal
hubot log me smoke fe1
4.1.3 - Nagios
hubot nagios check fs3b/syslog hubot nagios status fs3b/syslog hubot nagios downtime fs3b/syslog 90 hubot nagios mute fs3b/syslog hubot nagios unmute fs3b/syslog