Chatops (Operations automation with a bot)

1 - About

Chatops:

  • The important phases of development cycle are bring in one place, in one room, a chat room through a bot.
  • Automating common operations task with a bot.
  • Breaking communication wall between differents teams.

The bot is the primary control for the whole infrastructure.

The chat becomes a console

By placing tools directly in the middle of the conversation, everyone is pairing all of the time. Teaching by making things visible. Jesse Newland

3 - Chat

  • Campfire is a web-based group chat tool that lets you set up password-protected chat rooms in just seconds. Invite a client, colleague, or vendor to chat, collaborate, and make decisions. Link to a room on your intranet for internal communications.

4 - Bot

4.1 - Hubot

Ryan wrote Hubot at Github.

4.1.1 - rolling a new version

git checkout -b git-gh13
# bump version number
git commit -am 'Bump git version'
git push origin git-gh13
# create pull request

Change as passed CI (ie Build was succesful)

  • Deploy
hubot deploy puppet to production
  • noop means simulate what happens on that branch git-gh13 against the file server fs1, noop mode simulates the changes
hubot puppet noop git-gh13 fs1 
hubot puppet force git-gh13 fs1
hubot graph me -1h @collectd.load(fs1)
  • The pull request is good, we want to push it in production (everywhere)
#merge pull request
  • To ensure that the master is green
ci runs
# Build 1087 (eragaf66&&) of puppet/master was successful (399s)
  • When the CI passes, the code is deploy in production
auto deploy
  • Now a new version of git is on master. It has been shipped everywhere.
# Puppet shipped it on a interval but you can push it 
hubot puppet force production fs
  • Graph of everything (of all file servers)
hubot graph me -1h @collectd.load(fs*)
  • Logs to make sure that we didn't break anything.
hubot log me hooks github/github

4.1.2 - Others

  • CI status of a random branch
hubot ci status github/smoke-perf
  • Roll it out to one file server
hubot deploy github/smoke-perf to prod/fe1
  • Verifying things are return to normal
hubot log me smoke fe1

4.1.3 - Nagios

hubot nagios check fs3b/syslog
hubot nagios status fs3b/syslog
hubot nagios downtime fs3b/syslog 90
hubot nagios mute fs3b/syslog
hubot nagios unmute fs3b/syslog

5 - Documentation / Reference

application/chatops.txt ยท Last modified: 2018/06/12 12:14 by gerardnico