Why do we have process anyway?

Leave a comment
Growth
Photo by Med Badr Chemmaoui on Unsplash.

Have you ever had a colleague complain about there being too much process? Or pointed out, powerlessly, that an existing process is pointless? I would posit that you have. In fact, it may have been you doing the complaining. 

I’ve certainly been there. 

I’ve been there in times where my jaw has dropped when opening up the description of the deployment process for an older part of the system, and in times where I’ve torn my hair out as a simple approval for a flight has gone round and round in endless repeating circles.

“Surely it must be easier than this,” I hear you cry and hear myself cry.

Yes, processes can be extremely annoying. But they don’t have to all be annoying. And, in fact, they should do more help than harm.

What is a process anyway?

Process is a very abstract term: in the dictionary, which I have just opened, it is defined as a series of actions or steps taken in order to achieve a particular end. In business in general, this could be the process in order to get some budget signed off or to hire a new person. 

If you’re an engineer reading this post, then the word process may make you think about the steps that your JIRA tickets take from being created to being complete, or the way that deployments of your live application are done, or even the forms you need to fill in for your annual review.

Regardless of whatever the end is that the process is there for, the definition of that process is meant to make that end result more replicable, predictable and consistent regardless of the individual(s) that execute it. 

Process as code

You could think of a process as the code that the company has written about how to do a particular thing. Everyone else following that process is executing it to achieve the same desired, consistent result.

But, as all programmers know, code evolves over time. Old code gets stale and misses out on the latest updates and features. The original authors of that code have also since learned more and become better programmers since they wrote it.

Have you ever gone back to code you wrote years ago and wondered what on earth you were doing at the time? The same is true of processes. Companies can grow, shrink, mature or completely reform: you can’t expect the processes to stay the same as a result.

Processes, like code, shouldn’t be set in stone. They should be revisited, tweaked, refactored, rewritten and deleted. 

Arguing against processes

If processes don’t change despite being inefficient, then you’ll need to work on getting them changed. But why do processes get left alone and not continually updated?

Sometimes a business won’t want to change processes because those in charge are lazy: they’d rather not have to invest any effort into something that works well enough, regardless of how inefficient it is in practice. Tut.

Additionally, a process can often become orphaned because the creation of that process is to ensure that knowledge can be handed over (e.g. the process of deploying the live application) or to ensure the consistency expected by a particular individual (e.g. the annual review process as defined by the head of HR). As time passes, the relevance of those processes fades and need updating. But those who defined them may no longer care as deeply about them, since their attention is now elsewhere.

Still, in the same way that the management of a business would expect their employees to be efficient and diligent with their work, that same management should also expect to continually update the collective ways of working that they have codified in order to demonstrate that they are being efficient and diligent also. 

A process should fundamentally serve those that are continually applying it. If those closest to the ground want to make it better, then they should absolutely be allowed to change that process for themselves. It’s likely they know more than the original authors about the latest and greatest way to get that work done.

Let chaos reign, rein in chaos

Still, even the best processes don’t have their place. There is also a good argument for process to not exist.

When a team is embarking on innovative or unknown work, process seems only get in the way, constraining ways of working and thinking. Chaos often breeds innovation, so why should we stifle it? But, on the other hand, if there’s no process, aren’t we all going to hurtle into the sea in a fiery ball?

There’s a neat quote that suggests a way of dealing with these situations:

Let chaos reign, then rein in chaos.

Andy Grove

Often during innovative or chaotic periods, it is best to let go of wanting to control things and let them unfold in a natural way, even if it seems like bedlam.

Chaos often gives birth to a new way of working; an emergent practice which can be the new best way of tackling a problem. After people converge on this new way naturally, then the emergent practice can be codified into the new process. Chaos is transitory, but necessary. Things will stabilize eventually.

A practical example is running R&D projects. Often truly innovative developments need space and lack of constraints for those that are working on them to explore all aspects of the problem. Too many meetings, checkpoints, reporting upwards and real-world engineering constraints can kill creativity.  

Although it can be deeply uncomfortable, especially for directive-driven managers, to think that an investment into R&D is “not being managed”, letting go of the situation and trusting staff to do things the way that they want to can, paradoxically, get better results.

As the R&D project begins to move into a prototype and requires more input from designers and engineers, then chaos can be reined in. 

But not before.

So, in summary

Process is good when it creates predictable results. However, like code, processes must be continually updated and revised to stay relevant, and the updating should be done by those closest to the process: the staff actually executing it. 

Creative and innovative projects can benefit from having no process to let chaos become an emergent behavior which will eventually become the new way of doing things. Let chaos reign, then rein in chaos.

Now, according to my own process, this is where I hit “publish”.

A fistful of radishes

Leave a comment
Growth

The man pulling radishes

pointed my way

with a radish.

Kobayashi Issa

I first came across this haiku when I was reading Jon Kabat-Zinn’s Wherever You Go, There You Are. Something about it really captivated me. 

What was it about? 

An image comes to mind of walking through fields, lost, trying to navigate to the nearest road. Upon noticing that a man is bent over working the ground, the traveller shouts out the name of the destination that she is trying to find, seeing the farmer pop up with a fist full of radishes. He waves one in the general direction.

I researched this poem further, and I found some exploration of the theme on the Poetry London website. Here, it is interpreted through the lens of teaching poetry: the farmer pulling the radishes is the poet, and the traveller is the student learning the way.

When learning poetry, we can only see a poem through the lens that we have inherited: the context of our own lives. In the case of the farmer, the embodiment of the practitioner and the teacher of poetry, his bias is clear: he grows and harvests radishes, and uses them to instruct; he teaches what he knows. There are many things that he does not know that he does not teach. But is he aware of this?

Photo by Daiga Ellaby on Unsplash.


When thinking about this haiku, it made me consider my own role as a manager, and even though it still feels uncomfortable to say so at times, a leader. When I am asked for my opinion, I am aware that it carries weight. But am I also always aware of my own inherent biases? 

I carry a number of biases, without malice. These are biases that I must work at keeping in check. Here are some that come to my mind.

  • By trade I am a backend engineer, with a PhD in compilers. My view on solving technology problems often steers from this skillset first, and not necessarily from other methodologies and technologies that may be better suited. What am I missing out on when I think of the solutions to problems, or give my advice to others?
  • I am quite confident in front of groups of people, but others may not be. Am I preventing others from talking?
  • I like to be decisive and tend to think fast without necessarily knowing all of the details. Others are slower, more analytical thinkers. Do I drown them out?
  • I am a white male in technology. I am inherently part of the gender and diversity problem in our industry. Do I have any biases that I don’t even know about that prevents people that aren’t like me progressing? How will I even know?

I struggle with these kinds of “checking myself” issues daily. There are ways, however, that we can combat our own biases. The best way is by surrounding yourself purposefully with others that are not like you.

Some of the most productive and effective relationships that I’ve had with my staff are when they are totally unlike me. Even abrasively so.

Slower thinkers challenge my expediency. 

JavaScript engineers, data scientists, delivery managers and QA engineers all force me to think about problems from different angles, in the best interests of different groups of people. 

The opinions of the most reserved staff often reveal the best and most cerebral parts of a discussion. 

Ensuring that we have excellent female managers, technical leads and interviewers has made our culture more inclusive.

The only way I’ve found to tackle your own biases, which are rarely malicious, but always apparent, is to surround yourself with people that are different, able to challenge you, and are equally aware of their own biases. 

The conflict and debate force better outcomes.

If you surround yourself with others who are just like you, then you’re stuck with a fistful of radishes.

Ownership

Leave a comment
Growth
Photo by Agê Barros on Unsplash.

Tick tock

How much time is wasted in the cracks between phases of work?

A feature is code complete, but the pull request is waiting on review. 

Stalled

The feature branch is deployed, but it needs QA. 

Stalled

Everything is finished, but it needs deployed to live. 

Stalled.

As a phase winds down before it becomes the next one, either due to needing internal input from the same team, or due to needing external input from another team, the person doing the work has to stop and wait.

That individual may naturally turn their attention to another task so that they can remain productive, such as starting on the next ticket.

However, an individual’s focus on individual productivity can often cause the total productivity of the group to suffer.


“Oh yeah, that’s just waiting for QA. It’s been like that for a while now.”

“Did you let them know it was ready?”

“Nah, the ticket’s been moved into the right column, so they’ll pick it up.”

Well, maybe they’ll notice that it’s there, and maybe they won’t, because they’re busy with something else.

Even highly productive individuals can demonstrate a lack of ownership over the overall goal of shipping software. Instead the individual is only focussing on their part of the task. The other parts are on the other side of the fence, and that’s somebody else’s problem.


“That one’s waiting on the other team to push their changes upstream.”

“OK, did they know that you’re waiting for them?”

“They looked pretty busy, so I thought I’d let them get on with their stuff for now.”

Sometimes work can take hours, or even days, longer than it needs to because an individual only pushes forward the part that they can control. A particular process doesn’t guarantee that complex work being done by lots of people will be completed. The people themselves make it happen by doing whatever is required to see the whole task through.

When individuals embrace total ownership of their collective output, things get done faster. This is predominantly a change in mindset, not a change in process. 

But before we explore that mindset more, what if people don’t understand their responsibilities in the first place?

RACIs

Often when roles aren’t clear, the natural thing to do is to collectively spend time defining them. A tool that many find useful is the Responsibility Assignment Matrix, commonly referred to as a RACI

Within teams, departments, or even whole companies, you can draw up a RACI to better understand the different roles that individuals take for particular tasks.

These roles are, for a given task:

  • Responsible: Those that do the work to complete the task.
  • Accountable: The person ultimately responsible for delivery of the task. They approve the work that the responsible person does.
  • Consulted: Those that have their opinion sought through two-way communication.
  • Informed: Those that are updated on progress via one-way communication.

Hopefully you can see where the acronym comes from now.

Here’s an example of what part of a RACI might look like. The uppercase letters in each box correspond to the role definitions above.

CTOVP EngineeringTeam leadDevelopersQAScrum Master
Develop softwareIIARCI
Remove blockersIIRRRA
Ensure quality of softwareCRRAI

If a team is finding that tasks keep dropping through the cracks, then taking that team through a RACI exercise can often uncover all sorts of incorrect perceptions about how things get done. Give it a go and see what happens. I guarantee you’ll have some interesting debates.

Yet, remember what we said before: people make things happen, not processes. The RACI will not solve your ownership problems, but it will stimulate interesting discussion.

People solve ownership problems by realizing that they need to take total ownership of their collective output. You may have the most detailed RACI filled in, but without a mindset of ownership, little will change.

Hiding in definitions and process

Processes and definitions create places to hide within them. If you know you’re responsible for X, why should you care about Y? After all, the RACI says so, right?

“Oh, that’s her remit.”

“Nothing to do with me.

“Yeah, the other team should probably look at that.”

“Not my responsibility. I’m busy enough as it is.”

We could imagine a situation where a RACI could be abused to encourage a “not my problem” mindset.

After all, if the QA is ultimately accountable for ensuring the quality of the software being released, should the developers care less about it? 

Of course not.


When technology companies are start-ups, there is no choice for staff but to care about everything, because there simply is no other way. Start-ups embrace total ownership through lack of resources.

But success breeds growth, causing expansion of people and divisions of responsibilities and roles. The more you divide and spread that ownership, the more gaps and barriers that form between those subdivisions. Individual remits become narrower and deeper. 

You work in your box, and they work in theirs.

Instead of the founder selling, building and fixing, you now have developers on specific teams, numerous specialties of salespeople for diverse regions and products, and multiple roles for keeping customers happy and engaged. 

The risk of this scale is the collapse of total ownership.

When the remit focussed on is a tiny part of the whole, the individual optimizes for individual performance. This can easily be at conflict with the overall goal of being a successful company. 

Your lead generation machine could hit target, but those leads could be junk. Your engineers could ship to a deadline, but the quality could be awful. Your marketing event could have record attendance, but none of the attendees convert to sales. 

Overly local focus creates local maxima, and people across the company could be kings and queens of the minor mountains without conquering anything of worth.

And, of course, fingers will point: sales can’t convert those leads because they’re junk. They think it’s the fault of marketing. But marketing thinks that better salespeople could convert them. Engineering get accused of shipping a bad product, but they blame the deadline that was set by others. 

When you only care about your little patch, you’ll dump the rubbish over the fence to keep it looking pristine.

Owning it all

Instead of hiding behind processes, or driving relentlessly towards individual KPIs, truly excellent staff take ownership of the entire thing

It wasn’t your fault, it was mine. 

Leaders take ownership for their teams, and for their departments, even if it hurts and it bruises their ego when things go wrong that were out of their control.

Ownership as a mindset spreads. It builds respect. It causes different questions to be asked.

“Oh yeah, that’s waiting for QA, but I totally forgot to ask them. I’ll do that now and pair with them on it.”

Much better.

“This is waiting for the other team to push their changes upstream. I’ll ask if they can do that now as it’s blocking us.”

Excellent.

“That pull request is still waiting for review. I’ll go ask a couple of people if they can review it now.”

Perfect.