Posted on

Lessons Learned in 2020

A lot has already been written about the year 2020 and I’m confident that an awful lot more will be written about the impact on society this particular year has had. As the events of the year unfolded and various aspects of life were forced to adapt to the different challenges that were thrown up, it seemed like more and more material for academic papers, theses, policy documents, training materials, and even half decent fiction was being presented on an almost daily basis. From the technology leadership perspective I think 2020 showed that there’s work to be done around how to adapt Agile processes to remote work, but while interesting this isn’t the only area where there’s something to be learned, so in the best traditions of the internet here’s a list of things I consider among the lessons 2020 taught.

Communications

Easily the single biggest take away from 2020 is in the field of communications. Over time many of us have developed a strong preference for written communication methods like email and texts to get our messages across to the point where phones are no longer deployed on desks by default and tools like Teams or Slack have become established as the comms tool of choice. However, a bias for a particular type of media doesn’t make it the best way to communicate and during difficult times like those experienced this past year when tensions ran a little higher than usual and nerves were a little more frayed the miscommunication issues known to be associated with written messages became more frequent.

Written communications just don’t work for everyone as some people don’t write clearly and some don’t comprehend willingly, which triggered a need to adapt how to communicate in order to accommodate these differences in style. While this became more obvious this year as all those hallway conversations that normally happen in the office just didn’t take place and were instead replaced by emails, the fact is that there is a broader lesson to be learned about the tools we use to collaborate with others as well as how we respond when faced with an email that seems unfair or overly critical. Taking a beat before responding or doing away with the CYA email in favour of a clarifying phone call is something that should be taken from 2020 and re-established as normal practice.

Risk Management

Wired published a great article in June 2020 about pandemic insurance and how no one wanted to buy any in 2019 (https://www.wired.com/story/nathan-wolfe-global-economic-fallout-pandemic-insurance/) and it speaks to the obvious about how risk is really managed in businesses around the world. Organisations need to get better at risk management. Right now “risk” is too often considered like a gamble that doesn’t pay off and therefore isn’t worthy of attention. Conversations about bus factor issues in teams all the way up to business continuity in the face of global pandemics or more localised and very real risks like fire never seem to resonate the way they should until it’s too late.

2020 made a distant risk into a crushing reality which cost people their jobs as businesses were forced to close and markets became inaccessible. In IT, a lot of firms discovered the consequences of under investment when they tried to get their workforces to operate remotely and in the panic of trying to get teams working the conversations about laptops and cloud as risk mitigators were forgotten. Suddenly, the only issue with getting a laptop was finding somebody to sell you one, whereas at the start of the year the justification arguments still raged as did budget fights around moving on-prem applications to the cloud. In hindsight these fights seem silly but it’s a pity that they ever happened at all.

Culture

The value of culture in an organisation is well documented but what interests me about culture when considering 2020 is how culture manifests in the real world in terms of traditions and cultural norms expressed by a team or firm as a whole. Established traditions are a valuable way to build and reinforce culture but where tradition is helpful at a practical level is as an aid in times of crisis.

Traditions remind us of who we are and help to ground us when times get tough. Teams that are encouraged to connect to their culture (like the very strong culture around software development) and create traditions of their own are able to guide themselves through new, high pressure situations with greater ease as they can base their actions on the standards of behaviour and insights of the community of which they are a part, and of which they are reminded whenever they do something “traditional” like a daily stand-up or distribution of a new set of stickers for laptops.

Agility

The concept of agility, as in adaptability in the face of uncertainty, was strongly tested in 2020. Companies struggle with agility, not just adopting the Agile values and principles used to guide development, but in being responsive to the opportunities and challenges that come along regardless of the scale of those events. Companies strategise and lay plans and place value in adhering to those plans without enough concern for the value to be delivered as a result of those plans. This results in long backlogs that teams are tied to delivering without regard for the potential need to pivot away from the backlog at short notice.

A stronger sense of agility suggests that teams should shorten the timelines of their deliverables in accordance with Agile practices, ensuring that stories are atomic and that it is possible to drop everything and pivot to something completely different by the end of the current sprint. Beyond that specific Agile action, there is a need to build capacity into teams to allow for a disruptive event to be capitalised on.

Away from development, 2020 showed that there is a benefit for organisations to think on shorter time frames, to iterate on most activities, to report frequently so that issues can be identified early and actioned quickly.

Routine

In times of disruption, whatever the cause, reassurance can be found in the regular routines of life. Real disruption triggers ad hoc responses and these responses too quickly become normal – how many times this past year did you hear the phrase “the new normal” to describe pretty much anything that was happening? Accepting ad hoc structures as normal isn’t the most productive approach nor does it make for happy teams, as trying to work in uncertain times using uncertain methods only creates stress.

It may seem contradictory but agility thrives when there are structures in place that promote responsiveness, as long as those structures build capacity and recognise the activities that are really going on in the organisation. Regularly scheduled cadences of work that facilitate as much BAU as possible are absolutely essential as are techniques to identify what constitutes “Business As Usual” in a world where change is a constant. A good example of this that was validated by the events of 2020 is that meetings specific to a narrowly defined topic should be minimised where possible and more flexible structures be implemented in their place that can handle when things change.

Conclusion

It’s a truism that we learn more from hard times than from good, but it’s also too often true that we never learn. Anyone who’s worked a project in the real world knows that the “lessons learned” part of the process is frequently skipped due to time constraints, which is such a shame. 2020 was a year where a lot of people had too much time on their hands, stuck at home with nowhere to go, but even after having all that time we should still take a little more for a retrospective on what happened, how we handled the situations presented, and what we can do better as a result of having learned a little from the experience.

Posted on

My Role

As someone who has spent their career working with systems I have come to love a good framework. Taking a systematic approach to how things should operate enables me to conceptualize the tasks at hand and to identify gaps and weak spots so that they can be managed successfully. Identifying the component parts of my new role as Director of Software Innovation with ACETECH and weaving them together into a framework of sorts has helped me in these early stages to approach some difficult topics with confidence as well as quickly establishing where I can add value.

Having spent some time deep diving on where things are going well and where there are definite needs for improvements with my new employer, I have worked out that there are four major focus areas of the role:

  • Product Management
  • Team Leadership
  • IT Management
  • General Management

As I worked out how to group my work into focus areas and these four headings emerged, I realised that only the people focused area was about leadership in the truest sense whereas the other three seem more accurately defined as management areas. This is reflective of how I have assessed the job ahead of me and how I feel that so much of my focus needs to be on delivery and addressing certain key challenges with processes and approaches, as opposed to needing to “manage” the team I am working with. To put it more succinctly, the processes need management capability whereas the people deserve leadership.

Product Management

The main thrust of the role is about delivering software products. ACETECH produces IoT devices that are installed into vehicles to aid in the management of the vehicle, the operation of the vehicle and more, on into areas like real-time tracking of high-value assets. In order to unlock the value of these devices, customers are given access to a suite of software tools delivered in a SaaS model, that enables them to view, manipulate, and derive even more value from the data that is captured from the devices on their vehicles.

All software benefits from strong product management, and there is a definite opportunity here to implement some new approaches to product management that I am sure will help to improve the pace of delivery while also better addressing the needs of our customers.

Team Leadership

While there is significant focus on the Product Management aspects of the job, for me the most important aspect of the job is managing the development teams, meaning directly managing the team based in Tullamore but also working with remote teams around the world, both in-house and outsourced.

Managing a team is an absolute joy, either regardless of the challenges or maybe because of them. Walking into a situation where a team has been without direct technical management for a little while was especially interesting as it had highlighted the value a manager can bring to a team in terms of offering guidance on activities for a given period, providing structure in terms of how things need to be done, delivering feedback and developmental advice, but most essentially acting as a buffer between often conflicting needs of the business who had gotten used to approaching engineers directly and demanding action on a variety of things.

The nature of managing the outsourced teams is different in that their performance management is more clear cut from my perspective but I still want them to operate in a certain way, to utilize the same tools as everyone else, and to align their processes as closely as possible when it comes to areas like Agile. The distance and time zone differences are challenges but not insurmountable especially when taken properly into consideration – simple tricks like including the timezones of the outsourced teams on my Outlook calendar are gentle reminders that other people are experiencing their day at a different point to us when scheduling meetings or calls.

IT Management

Every company is in someway a technology company due to the reliance on Information Technology that all businesses have. In order to be able to unlock the value of that technology there is an ever present need for someone to be thinking about how the internal network is managed, how the hardware and software we need is purchased and supported in the long run, and how the whole operation can be kept secure.

In a practical sense, for smaller organisations there’s little chance of having a dedicated Systems Administrator, despite the need for that type of focus, and ours is no different with Sys Admin falling to members of the Software Engineering team. I’m not opposed to this in principle as that team are the ones best placed to look after the running of a network but also because it’s good practice for those who are tasked with looking after a cloud estate to understand the fundamentals of networks and the types of issues you will encounter when users are unleashed and where better to learn that in a realistic but also relatively safe manner than a company LAN?

General Management

As a member of the leadership team, I have a duty to contribute to the broader running and direction of the company as a whole, beyond my areas of direct responsibility as part of a holistic management structure. That being said, there is a significant opportunity for me to address the impact of software and data products on the strategic development of the company, by looking at how software can drive sales and help push the company into new markets.

More broadly though, this part of my role is for me all about delivering at a higher level, about making a mark on how the company will grow into the future, about setting the tone in terms of culture particularly around diversity and inclusion, and about guiding us to greater success through initiatives like strategic partnerships with other technology organisations but especially with local educational institutions, and not just for recruitment purposes but also to contribute to, and of course benefit from, original research.

Conclusion

There is always more to a role than can ever be accurately captured in a job spec and it is this element of possibility that often brings the most rewards from work; in the spaces between the lines of a job description lies the chance to do something wonderful, though it is something of a skill to be able to spot those opportunities. To that end I have found that, in the early days of starting in my new position at least, it has been beneficial to classify the major areas where I can make an impact and thus put some level of structure around where I need to focus my attentions and where I can develop the technology offerings of ACETECH.

Posted on

Go Big and Go Home

Introduction

A new opportunity has presented itself to me that is so aligned with my desired next move that it’s hard to believe. Next week, I start a new role heading up the software engineering department of a technology company located in the Irish Midlands; in fact, located about 3 minutes from my home. And this isn’t a small opportunity, this is the chance to head up the software delivery aspects of a global player that makes a real impact on people’s lives.

The Commuter’s Tale

For about twelve years or so, with only one or two very brief exceptions, I have commuted from my hometown of Tullamore, Co. Offaly, to work in Dublin. For most technologists in Ireland, Dublin is where the action is. There you will find all the major technology companies as well as the Irish head offices of so many financial institutions and other organisations that depend on hired geeks that if you’re a technology professional of pretty much any type, most of the opportunities for you will be located there. Not all, of course, but most of them, and certainly the better paying ones.

Commuting to work is an accepted part of modern life and something that nearly everyone must do in some way. What marks out “Commuting” (with a capital C) as something different is the scale involved. Getting to and from work is commuting, getting up at an early hour and taking the train halfway across the country every working day is Commuting!

Humble bragging about how far you’re prepared to travel for work aside, commuting long distances has a lot of downside; it obviously takes a lot of time, time that is spent away from family, friends, and being a part of the local community in any meaningful way. There are financial costs to commuting a long way, with train tickets for my type of journey reaching several thousand Euro every year and travelling by train every day brings its own set of challenges. And of course, there are the early starts and late finishes that combine to make every day a “very long day”.

Making a Down into an Up

During my time commuting on the train from Tullamore to Dublin I have tried to make the best of that challenging situation. At first, I did all the things commuters do; I watched movies on laptops, TV shows on iPads, listened to podcasts on my phone, and read many, many books. After a while though these techniques for killing time lost their appeal and I came to the realisation that I had to do something a little more productive with the roughly ten hours every week I spent on the train. I started studying.

As part of a work initiative around project management, I found myself attending a week-long PRINCE2 course where the plan was to sit the first, foundation-level exam at the end of the third day before trying at the practitioner exam on the fifth day. In order to be in with a fighting chance of passing it was necessary to do some study in your own time. Considering what was at stake as well as how best to pass these exams I made use of the time I had on the train to study, and I found that it worked amazingly well. Not only did I pass the exams but I found myself in the strange situation of being almost disappointed when the train arrived at my stop, I found that I could always use a just a little more time with the books and always expected that I had more time than I really did, a marked difference from times where the journey seemed to drag on for ever!

That course set me on the path of studying on the train as a way of passing the time constructively. After PRINCE2 I studied for and passed the ITIL foundation exam, followed by the AWS Solution Architect Associate exam, and then five of the CIMA accountancy exams to earn the Certificate in Business Accounting. With that hanging on the wall, I turned my attention to degree level study, pursuing the University of London BSc in Management and Digital Innovation.

A Base of Operations

Even though I was making something of the time I spent travelling every day, there is no escaping that I want to be able to spend more time nearer to home, to effectively be based out of the Midlands region as opposed to only living here. The heartlands of the country is my idea of a good base of operations for logical reasons like proximity to several major towns and cities, and for the simple, more human fact that this is my home. Our environment can energise us, it can motivate us to excel, it can help to remind us of who we are and where we come from which is an essential part of figuring out where we are going; and it’s vitally important that it’s “we”, as being a part of a community is important for social animals like us.

The region is not overly populated, so participating in life in the Midlands requires to be part of communities that are pulled together from different towns, villages, and parishes across the four counties, which brings surprisingly different insights and points of view. This is where I want to spend the majority of my time, not all my time to be sure, there’s a wide world out there to explore and experience, and working in technology has provided me so many great opportunities to see the world that I could never fully retreat away from what’s out there.

Stick-with-it-ness

The ability to compromise is important in order to successfully navigate relationships and maintain a healthy balance in life, but there are also times when something is important enough that it’s worth pursuing to the fullest extent, to not settle for something nearly as good, which makes the ability of recognising when a goal is important enough not to compromise on a perhaps more valuable skill than understanding where ‘good enough’ lies.

Regardless of their form, career ambitions can be such significant drivers that they can easily lead to frustration when the pace of progress is slower than desired, which it nearly always will be as there will be so many variables beyond anyone’s control. You will need patience as you gain experience, learn the right skills, build a network, and wait for the right opportunity to present itself, so it is vital that you are happy doing what you’re doing during the wait, especially as you will need to excel in the role you are in so that you have a better chance of achieving the role you want.

My experience taught me all this and more. I had my own ideas around what compromise looked like that I took comfort from them as they were nearly as good and felt more achievable. When asked what the future held for me I always pointed to those ideas, over time almost discounting the possibility that my primary goal could ever be achieved, however when the right opportunity arose I was ready, moved faster than the competition, and achieved that goal. I now eagerly face into a new challenge, one that will give me the chance to make the type of positive impacts I know I can deliver.

One odd thought does occur to me though as I gear up to “work from home(town)”. Maybe there is, in some part of me, an actual desire to Commute. Yes, that arrangement was certainly born out of necessity and was a huge pain, but it does have upsides, with an important aspect for me being the realisation that in many ways our commitments make us better people, like how having to catch a train early in the morning makes you regulate your bedtimes and clothing preparation. Working to the train schedule made me disciplined in ways that I never would have imagined before needing to make that daily journey. Perhaps the far future will lead me back to some sort of hybrid or split-time arrangement between home and a city, but for now I am grateful for the chance to make an impact in the town and region that I call home.

Posted on

Prepare to Fail

There’s always a lot going on in work and it can be easy to sacrifice essential, foundational activities in order to progress the “real work”, but doing so eventually takes a heavy toll. Just like spending enough time contemplating what’s being done and why and if that’s the right thing to be doing, there is a need to be prepared and thus avoid going into every situation trying desperately to figure out what’s going on and determine your position on the fly, a position that you may regret taking and finding you later need to defend out of not wanting to loose face.

Some activities demand preparation, it only takes one attempt at winging a presentation to convince most people of the value of being prepared, whereas it is too often the norm, for example, for people to arrive in meetings late and utterly unprepared, which is just a waste of everyone’s time. Again, this type of problem is all about time management. How many back to back meetings or events fill the calendar? How much time is set aside for preparation, if any? And how would you feel if a colleague blew-off a meeting you considered important in order to prepare for some other meeting or event?

Personally, I’ve seen too many organizations where terrible meeting etiquette prevails, from an over reliance on using meetings to actually do work, to the lack of defined purpose or an agenda, and so on, with a particularly nasty vice being the continuous overloading of calendars with recurring events and double, triple, and more bookings, and in those organizations a common underlying issue (no doubt one of many) is the lack of consideration for colleagues. Being busy can wrap us in a sense of urgency that can manifest as a feeling of self importance that in turn erodes the consideration we would normally have for those around us.

One way to help restore that consideration might be to build enough sacred time into our schedules to ensure we are prepared for the events that keep the wheels turning in our respective organizations, and in that small way show some professional consideration to those around us and start to limit the urgency that’s pressing us to be so unprepared in the first place.

Posted on

Notebooks

I flip-flop on the subject of whether to use a paper notebook or to use a laptop for everything including note taking in meetings and capturing stray thoughts. My indecision comes from appreciating both sides of the argument for either tool with laptops claiming the higher ground in terms of efficiency but loosing out to things like noise levels, having the opened screen imposing a barrier between you and other people, the issues around battery life, and the distractions that come with the installed apps all clamoring for your attention, usually when you need to be more focused.

Recently I’ve noticed a previously hidden benefit of using a paper notebook as a tool to monitor commitment levels, particularly when you’re prone to over-commitment in a busy organisation.

Once upon a time it was normal for students to take notes in lectures to later write-up into proper materials that could be used for study. This concept is where my paper notebook come back into play as it is certainly beneficial to spend some time digesting notes, transcribing them even, as an exercise in thoughtfulness around work.

However, when time is limited, do you have enough for this type of seemingly unnecessary activity? Arguably, if not, can you be sure you’re spending enough time thinking about what you’re doing at all, or are you simply running from meeting to meeting, lurching from issue to issue, and hoping that you’re doing the right thing in the heat of the moment?

Taking time out to be mindful of your work is a proactive thing to do, as counter intuitive as that may seem, and thinking about something is getting ahead of the issue. If there aren’t enough hours in the day to dedicate some time to this, then perhaps it’s time to assess if you’re over committed, need to delegate more, or in some other way need to address your workload; this is the action I’m taking away from the current use of a notebook and pen!

Posted on

AWS ChatOps with Hubot & Slack

There can be many reasons for wanting to explore the use of Chatbots. In situations where collaboration tools like Slack are already in place, introducing a bot is merely another form of automation to support the way people work. In companies that are interested in providing a forum for people to develop technical skills, there are definite possibilities offered through Chatbots, especially as extending their functionality is not overly difficult and actually makes them more useful. For the enthusiast, Chatbots provide a glimpse into a future where interactions with technology are more fluid, based on natural language and conversation, but this is where challenges remain, for despite all the powerful technology that has been used to create Hubot, frankly it isn’t that chatty for a chatbot.

I suspect that many end-users would be disappointed with the way commands are needed for Hubot like pretty much any other software tool. I’d go as far as to say that the interface with Slack is the only “chatty” thing about it. I suspect I’m missing something or that chatbot is a misnomer. As yet, I haven’t been able to ask fuzzy questions like “what time is it?” or “what’s the time?” or “tell me the time please” and get the same result, so I need to look into that. I wonder if there is a script that can give Hubot more personality?

Hubot is able to do what’s asked of it, but so what?

There is a lot of powerful technology under the hood that is used to get Hubot to this point, so I’m sure at some sort of scale you will be able to see the wow factor, but a stand alone instance doesn’t really show that off. I think that if there was greater utilisation, to the point where the back-end was seeing heavy use (of components like Redis) then it would be easier to see the value.

Personally, the introduction to the node.js ecosystem was very interesting especially as it provides context for some real-world activities that are going on around me. I was impressed by how tools like NPM appear quite slick to the Node novice. Slack was also impressive, but again desperately needs users in order to be of value. I think that in the real world I am more likely to be encountering Hipchat (due to the value that can be gained from a Jira integration), but regardless of the technology I am excited at the prospect of a chat forum appearing in an enterprise setting.

Hipchat_Atlassian_logo

Maybe the trick to Hubot is that in order for the bot to develop a personality, it needs to be crafted by the organisation that is using the bot. If Hubot provides a basic bot, the organisation that wants to use the bot can craft that personality. The first challenge that could be opened up to the company could be to work on the programming needed to build the firms Hubot something like a personality. Imagine offering that up to your end users in marketing? Hey, could you lash together a few lines of code and give our bot a soul?

I wonder what they would create?

Posted on

Configure Hubot to run in background via systemd

In /etc/systemd/system create a “hubot.service” file and populate it as follows:


$ sudo vi hubot.service

[Unit]
Description=Hubot
Requires=network.target
After=network.target

[Service]
Type=simple
WorkingDirectory=/io

Restart=always
RestartSec=10

ExecStart=/io/bin/hubot --adapter slack

[Install]
WantedBy=multi-user.target

Hubot uses environment variables to store certain parameters. In order for the systemd Hubot service to be make use of environment variables, a hubot.conf file is needed. This file needs to be stored in a specific location so that systemd can read it when starting the service.

In the example below you can see where I’ve configured the Slack Access Token as an environment variable to enable my Hubot to be able to connect to my Slack team.


$ sudo mkdir hubot.service.d

$ cd hubot.service.d

$ sudo vi hubot.conf

[Service]

Environment="HUBOT_SLACK_TOKEN=xxxx-12345678901-XXXXXXXXXXXXXXXXXXXXXXXX"

$ sudo systemctl daemon-reload

With systemd configured to operate Hubot, commands like service hubot start are now available.

Using systemctl enable hubot set systemd to start Hubot automatically whenever the server is restarted.

Here is the output of the service hubot status command after a reboot of the server:


$ sudo service hubot status
Redirecting to /bin/systemctl status hubot.service
● hubot.service - Hubot
Loaded: loaded (/etc/systemd/system/hubot.service; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/hubot.service.d
└─hubot.conf
Active: active (running) since Sun 2016-08-21 18:07:47 IST; 10s ago
Main PID: 819 (node)
CGroup: /system.slice/hubot.service
└─819 node node_modules/.bin/coffee /io/node_modules/.bin/hubot --name io --adapter slack

systemd[1]: Started Hubot.
systemd[1]: Starting Hubot...
hubot[819]: [Sun Aug 21 2016 18:07:52 GMT+0100 (IST)] INFO Logged in as io of Chatty McChatface
hubot[819]: [Sun Aug 21 2016 18:07:53 GMT+0100 (IST)] INFO Slack client now connected
hubot[819]: [Sun Aug 21 2016 18:07:53 GMT+0100 (IST)] INFO hubot-redis-brain: Using default redis on localhost:6379
hubot[819]: [Sun Aug 21 2016 18:07:54 GMT+0100 (IST)] INFO hubot-redis-brain: Data for hubot brain retrieved from Redis

Cleaning up Hubot’s Configuration

With Hubot up and running and configured to do more than just display pictures of pugs, I noticed the status output of systemd was very noisy:


$ sudo service hubot start

$ sudo service hubot status

Redirecting to /bin/systemctl status  hubot.service

hubot.service - Hubot

Loaded: loaded (/etc/systemd/system/hubot.service; disabled; vendor preset: disabled)

Drop-In: /etc/systemd/system/hubot.service.d

└─hubot.conf

Active: active (running) since Mon 2016-08-15 04:14:45 EDT; 26s ago

Main PID: 23105 (node)

CGroup: /system.slice/hubot.service

└─23105 node node_modules/.bin/coffee /io/node_modules/.bin/hubot --name io --adapter slack

Aug 15 04:14:45 ip-192-168-20-246.eu-west-1.compute.internal systemd[1]: Started Hubot.

Aug 15 04:14:45 ip-192-168-20-246.eu-west-1.compute.internal systemd[1]: Starting Hubot...

Aug 15 04:14:48 ip-192-168-20-246.eu-west-1.compute.internal hubot[23105]: [Mon Aug 15 2016 04:14:48 GMT-0400 (EDT)] INFO Logged in as io of Chatty McChatface

Aug 15 04:14:49 ip-192-168-20-246.eu-west-1.compute.internal hubot[23105]: [Mon Aug 15 2016 04:14:49 GMT-0400 (EDT)] INFO Slack client now connected

Aug 15 04:14:49 ip-192-168-20-246.eu-west-1.compute.internal hubot[23105]: [Mon Aug 15 2016 04:14:49 GMT-0400 (EDT)] WARNING Loading scripts from hubot-scripts.json i... script.

Aug 15 04:14:49 ip-192-168-20-246.eu-west-1.compute.internal hubot[23105]: Your hubot-scripts.json is empty, so you just need to remove it.

Aug 15 04:14:49 ip-192-168-20-246.eu-west-1.compute.internal hubot[23105]: [Mon Aug 15 2016 04:14:49 GMT-0400 (EDT)] ERROR hubot-heroku-alive included, but missing HU...d= -f2)`

Aug 15 04:14:50 ip-192-168-20-246.eu-west-1.compute.internal hubot[23105]: [Mon Aug 15 2016 04:14:50 GMT-0400 (EDT)] INFO hubot-redis-brain: Using default redis on lo...ost:6379

Aug 15 04:14:50 ip-192-168-20-246.eu-west-1.compute.internal hubot[23105]: [Mon Aug 15 2016 04:14:50 GMT-0400 (EDT)] WARNING The HUBOT_AUTH_ADMIN environment variable not set

Aug 15 04:14:50 ip-192-168-20-246.eu-west-1.compute.internal hubot[23105]: [Mon Aug 15 2016 04:14:50 GMT-0400 (EDT)] INFO hubot-redis-brain: Data for hubot brain retr...om Redis

Hint: Some lines were ellipsized, use -l to show in full.

In order to tidy up the log so that important items don’t get lost in the noise, I removed a couple of unnecessary items from Hubots configuration.

I deleted the “hubot-scripts.json” file, as Hubot no longer uses it. All that happens is a warning is thrown when Hubot starts – it’s harmless, I just removed it for the sake of a less noisy log.

I removed the reference to Heroku from the external-scripts.json file so that the Heroku connector didn’t get called when Hubot starts.

Posted on

Using Hubot with Slack

Slack is the latest hip tool that technology teams have come to depend on. Based on an idea eerily similar to Internet Relay Chat (IRC) that was extremely popular back in the mid-nineties, Slack allows teams to collaborate in web and app based chat rooms and  is growing in popularity with enterprises as well as the smaller start-up world where it first took off, as larger businesses are realising that Slack provides a very neat audit trail through the log of the chats themselves.

2016-08-27 11_08_43-Slack_ Be less busy

Slack is extremely extensible, and this is where Hubot comes in. Slack is the perfect interface for Hubot as bots can be team members in Slack, appearing in chat rooms and interacting like everyone else.

Slack offers accounts at different price points, from no cost and quite restricted up to enterprise level that costs more but comes with all the bells and whistles. I dove in with a free account to facilitate testing of my bot.

Once the Slack account was up and running along with the associated team (which I couldn’t resist calling “Chatty McChatface”), I was ready to deploy my Hubot (Io).

In Slack, you simply navigate to the Apps directory and search for Hubot.

2016-08-27 11_18_12-Configure Apps _ Chatty McChatface Slack

There follows a handful of simple configuration steps which include generating an access token. This is important as that token is used on the server hosting Hubot to enable the connection with Slack, in much the same way as the access and secret keys enable access with AWS. (The next post will deal with environment variables like this one).

With Hubot configured in Slack (and connected on the server) it’s possible to use a chat room to interact with Hubot. Initially, the bot will have a direct or private room for 1:1 chat, like any team member. Here, simple tests can be carried out in private to ensure all is working as it should.

2016-08-27 11_30_40-io _ Chatty McChatface Slack

There’s far more value in having Hubot out in the open in one of the team rooms and enabling that is as simple as inviting Hubot to join the room, just like you would for a human team member, through the “/invite @[BOTNAME]” command.

2016-08-27 11_34_52-random _ Chatty McChatface Slack

Now that Hubot was in place and working, AWS functionality could be tested. The hubot-aws script features a lot of commands, but a simple listing (ls) command of one of the AWS services is enough to ensure connectivity. Like any good bot, Hubot will list all the commands he supports through the “help” command.

2016-08-27 11_37_52-random _ Chatty McChatface Slack

So there you have it, a working Hubot implementation, connected to and interacting with AWS, presented through Slack.

Up next, how to use Systemd to automate the running of Hubot.

Posted on

How to Deploy Hubot

Finding a good excuse to learn a new technology is something I’ve been interested in for a while particularly as the pace of change and technology adoption only ever seems to be increasing. Knowing that the IT world is in an ongoing state of flux, finding an engaging way to learn about the tech that technologists are expected to know is more than idle curiosity, it’s essential for career success.

Revisiting Chatbots, a topic I encountered through a Wired article from summer 2015, I discovered that getting a chatbot up and running, and more importantly, useful and usable, involved a wide variety of different technologies. As the article was heaped so much praise on Github’s bot, I decided to try it out with a view to having it do some useful tasks around AWS administration, one of the two challenges that had been put up to our 2016 interns.

This led me to having to work with a number of interesting and topical technologies, including:

  1. Hubot – the Github chatbot
  2. Node.js and npm
  3. Git
  4. Redis
  5. AWS CLI
  6. systemd
  7. Slack
  8. Coffeescript

To get started, as I wanted my bot to work with AWS, it was necessary to prepare a couple of things on that side.

I created an AWS IAM account for my bot to use. I didn’t assign a password to the account as I don’t intend for the account to be used interactively (i.e. for logging onto the console), what I was after was an access key and secret key that can be used with the AWS Command Line and therefore could be used by the bot.

With the AWS pieces in place, I moved on to setting up the host in AWS itself, and selected a Red Hat Enterprise Linux 7.2 machine to act as my Hubot host, as this would approximate a production deployment.

Onto my new RHEL server, I installed and configured the AWS CLI.


$ curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 6984k 100 6984k 0 0 4106k 0 0:00:01 0:00:01 --:--:-- 4105k

$ unzip awscli-bundle.zip
Archive: awscli-bundle.zip
inflating: awscli-bundle/install
inflating: awscli-bundle/packages/jmespath-0.9.0.tar.gz
inflating: awscli-bundle/packages/simplejson-3.3.0.tar.gz
inflating: awscli-bundle/packages/botocore-1.4.43.tar.gz
inflating: awscli-bundle/packages/ordereddict-1.1.tar.gz
inflating: awscli-bundle/packages/awscli-1.10.53.tar.gz
inflating: awscli-bundle/packages/rsa-3.4.2.tar.gz
inflating: awscli-bundle/packages/futures-3.0.5.tar.gz
inflating: awscli-bundle/packages/docutils-0.12.tar.gz
inflating: awscli-bundle/packages/s3transfer-0.1.1.tar.gz
inflating: awscli-bundle/packages/colorama-0.3.7.zip
inflating: awscli-bundle/packages/argparse-1.2.1.tar.gz
inflating: awscli-bundle/packages/pyasn1-0.1.9.tar.gz
inflating: awscli-bundle/packages/virtualenv-13.0.3.tar.gz
inflating: awscli-bundle/packages/python-dateutil-2.5.3.tar.gz
inflating: awscli-bundle/packages/six-1.10.0.tar.gz

$ sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
Running cmd: /bin/python virtualenv.py --python /bin/python /usr/local/aws
Running cmd: /usr/local/aws/bin/pip install --no-index --find-links file:///home/ec2-user/Downloads/awscli-bundle/packages awscli-1.10.53.tar.gz
You can now run: /usr/local/bin/aws –version

$ aws configure

Up next was configuring the EPEL yum repository so that I could use Yum to install things like Redis.


$ wget -r --no-parent -A 'epel-release-*.rpm' http://dl.fedoraproject.org/pub/epel/7/x86_64/e/
$ sudo rpm -Uvh dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-*.rpm
warning: dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-8.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY
Preparing...                ################################# [100%]
Updating / installing...
1:epel-release-7-8          ################################# [100%]
$ ll /etc/yum.repos.d
total 32
-rw-r--r--. 1 root root  957 Jul 23 17:37 epel.repo
-rw-r--r--. 1 root root 1056 Jul 23 17:37 epel-testing.repo
-rw-r--r--. 1 root root  358 Nov  9  2015 redhat.repo
-rw-r--r--. 1 root root  607 Aug  9 04:08 redhat-rhui-client-config.repo
-rw-r--r--. 1 root root 8679 Aug  9 04:08 redhat-rhui.repo
-rw-r--r--. 1 root root   80 Aug  9 04:08 rhui-load-balancers.conf

With the appropriate repos in place, I was able to install redis


$ sudo yum install redis –y

$ sudo systemctl start redis.service

$ sudo systemctl status redis.service

redis.service - Redis persistent key-value database

Loaded: loaded (/usr/lib/systemd/system/redis.service; disabled; vendor preset: disabled)

Drop-In: /etc/systemd/system/redis.service.d
└─limit.conf

Active: active (running) since Tue 2016-08-09 04:37:35 EDT; 5s ago

Main PID: 4971 (redis-server)
CGroup: /system.slice/redis.service
└─4971 /usr/bin/redis-server 127.0.0.1:6379

Aug 09 04:37:35 ip-192-168-20-246.eu-west-1.compute.internal systemd[1]: Started Redis persistent key-value database.
Aug 09 04:37:35 ip-192-168-20-246.eu-west-1.compute.internal systemd[1]: Starting Redis persistent key-value database...

Hubot is basically a bunch of node.js, so it was necessary to get it and the Node Package Manager installed too.


wget https://nodejs.org/dist/v6.3.1/node-v6.3.1-linux-x64.tar.xz

sudo tar --strip-components 1 -xf /home/ec2-user/Downloads/node-v6.3.1-linux-x64.tar.xz

I called my Hubot “Io” (as in I/O), and created a directory for Io to live in. Once installed, Hubot can be started in a basic “shell” mode that enables testing and local interactions with the bot on the host server.


$ sudo npm install -g yo generator-hubot

$ cd /io

$ yo hubot

$ bin/hubot

At this point, a working Hobot is in place but it isn’t very useful. In order to get more out of him, Hubot can be extended by adding scripts. In the fine tradition of Open Source, many scripts that others have developed are available to download and install via NPM, and of course you can create your own.

Up next, extending Hubot through scripts.

Posted on

Solving Problems with Chatbots

Employee engagement is a big issue for any company, put simply disengaged employees leads to departing talent, so it’s vital to constantly be looking for ways to develop teams, especially those working with technology, in ways that result in high levels of ongoing engagement.

During a recent conversation about engagement and issues around internal communications, a colleague of mine suggested making better use of the screens we have deployed around the office to help keep everyone in the loop and up to date. This is an admirable idea but like all comms plans depends on the content more so than the delivery mechanism; the medium may be the message, but it’s important to have a message in the first place!

The conversation about content reminded me of an article in Wired from about a year ago about Github and how they had deployed a chatbot. What was interesting for me in that article was the story of how Github employees are extending the chatbot’s capabilities by scripting new features. What makes this doubly interesting is that not only are the engineers at Github doing this but so are non-technical people. One example in the piece was about someone from the marketing department creating a script that their chatbot uses to check on the status of local street vendors.

It occurred to me that a chatbot system like this could help out with two focus areas for developing team engagement. Firstly, it could provide that content that can be so hard to source. By making a conversational interface to a content management and deployment system, the process of gathering and displaying interesting material could be dramatically improved by providing a reason to generate content as it would give everyone an excuse to interact with the cahtbot. Secondly, having a chatbot system in place could provide an outlet for non-developers who want to spend some time learning development as it could provide a purpose beyond developing the standard issue “Hello World” script, without which too many people abandon their learning efforts.

anatomy_of_a_chatbot
Content generation would be part of the Workflow use case for a bot

Every summer we get a bunch of interns, and in addition to their regular assigned work they are tasked with completing a technical challenge. The challenge is meant to be, well challenging, and this year we settled on Chatbots (I suspect this was entirely my fault and an abject lesson in reaping what you sow)! One team of interns was tasked with setting up a bot that can field queries about one part of the business, while the other (my team) looked at connecting a bot to AWS in order to complete Cloud tasks through a chat interface.

It is something of a tradition that while the interns get on with developing a solution in their own way, I go ahead and do the challenge myself in my own way to determine if it can be done and how differently I’d tackle the solution over how the interns go about it.

As the interns had gone down the AIML route, I decided to deploy Hubot, Github’s own chatbot.

hubot

Up next: Getting started with Hubot