Ownership

Leave a comment
Growth
Photo by Agê Barros on Unsplash.

Tick tock

How much time is wasted in the cracks between phases of work?

A feature is code complete, but the pull request is waiting on review. 

Stalled

The feature branch is deployed, but it needs QA. 

Stalled

Everything is finished, but it needs deployed to live. 

Stalled.

As a phase winds down before it becomes the next one, either due to needing internal input from the same team, or due to needing external input from another team, the person doing the work has to stop and wait.

That individual may naturally turn their attention to another task so that they can remain productive, such as starting on the next ticket.

However, an individual’s focus on individual productivity can often cause the total productivity of the group to suffer.


“Oh yeah, that’s just waiting for QA. It’s been like that for a while now.”

“Did you let them know it was ready?”

“Nah, the ticket’s been moved into the right column, so they’ll pick it up.”

Well, maybe they’ll notice that it’s there, and maybe they won’t, because they’re busy with something else.

Even highly productive individuals can demonstrate a lack of ownership over the overall goal of shipping software. Instead the individual is only focussing on their part of the task. The other parts are on the other side of the fence, and that’s somebody else’s problem.


“That one’s waiting on the other team to push their changes upstream.”

“OK, did they know that you’re waiting for them?”

“They looked pretty busy, so I thought I’d let them get on with their stuff for now.”

Sometimes work can take hours, or even days, longer than it needs to because an individual only pushes forward the part that they can control. A particular process doesn’t guarantee that complex work being done by lots of people will be completed. The people themselves make it happen by doing whatever is required to see the whole task through.

When individuals embrace total ownership of their collective output, things get done faster. This is predominantly a change in mindset, not a change in process. 

But before we explore that mindset more, what if people don’t understand their responsibilities in the first place?

RACIs

Often when roles aren’t clear, the natural thing to do is to collectively spend time defining them. A tool that many find useful is the Responsibility Assignment Matrix, commonly referred to as a RACI

Within teams, departments, or even whole companies, you can draw up a RACI to better understand the different roles that individuals take for particular tasks.

These roles are, for a given task:

  • Responsible: Those that do the work to complete the task.
  • Accountable: The person ultimately responsible for delivery of the task. They approve the work that the responsible person does.
  • Consulted: Those that have their opinion sought through two-way communication.
  • Informed: Those that are updated on progress via one-way communication.

Hopefully you can see where the acronym comes from now.

Here’s an example of what part of a RACI might look like. The uppercase letters in each box correspond to the role definitions above.

CTOVP EngineeringTeam leadDevelopersQAScrum Master
Develop softwareIIARCI
Remove blockersIIRRRA
Ensure quality of softwareCRRAI

If a team is finding that tasks keep dropping through the cracks, then taking that team through a RACI exercise can often uncover all sorts of incorrect perceptions about how things get done. Give it a go and see what happens. I guarantee you’ll have some interesting debates.

Yet, remember what we said before: people make things happen, not processes. The RACI will not solve your ownership problems, but it will stimulate interesting discussion.

People solve ownership problems by realizing that they need to take total ownership of their collective output. You may have the most detailed RACI filled in, but without a mindset of ownership, little will change.

Hiding in definitions and process

Processes and definitions create places to hide within them. If you know you’re responsible for X, why should you care about Y? After all, the RACI says so, right?

“Oh, that’s her remit.”

“Nothing to do with me.

“Yeah, the other team should probably look at that.”

“Not my responsibility. I’m busy enough as it is.”

We could imagine a situation where a RACI could be abused to encourage a “not my problem” mindset.

After all, if the QA is ultimately accountable for ensuring the quality of the software being released, should the developers care less about it? 

Of course not.


When technology companies are start-ups, there is no choice for staff but to care about everything, because there simply is no other way. Start-ups embrace total ownership through lack of resources.

But success breeds growth, causing expansion of people and divisions of responsibilities and roles. The more you divide and spread that ownership, the more gaps and barriers that form between those subdivisions. Individual remits become narrower and deeper. 

You work in your box, and they work in theirs.

Instead of the founder selling, building and fixing, you now have developers on specific teams, numerous specialties of salespeople for diverse regions and products, and multiple roles for keeping customers happy and engaged. 

The risk of this scale is the collapse of total ownership.

When the remit focussed on is a tiny part of the whole, the individual optimizes for individual performance. This can easily be at conflict with the overall goal of being a successful company. 

Your lead generation machine could hit target, but those leads could be junk. Your engineers could ship to a deadline, but the quality could be awful. Your marketing event could have record attendance, but none of the attendees convert to sales. 

Overly local focus creates local maxima, and people across the company could be kings and queens of the minor mountains without conquering anything of worth.

And, of course, fingers will point: sales can’t convert those leads because they’re junk. They think it’s the fault of marketing. But marketing thinks that better salespeople could convert them. Engineering get accused of shipping a bad product, but they blame the deadline that was set by others. 

When you only care about your little patch, you’ll dump the rubbish over the fence to keep it looking pristine.

Owning it all

Instead of hiding behind processes, or driving relentlessly towards individual KPIs, truly excellent staff take ownership of the entire thing

It wasn’t your fault, it was mine. 

Leaders take ownership for their teams, and for their departments, even if it hurts and it bruises their ego when things go wrong that were out of their control.

Ownership as a mindset spreads. It builds respect. It causes different questions to be asked.

“Oh yeah, that’s waiting for QA, but I totally forgot to ask them. I’ll do that now and pair with them on it.”

Much better.

“This is waiting for the other team to push their changes upstream. I’ll ask if they can do that now as it’s blocking us.”

Excellent.

“That pull request is still waiting for review. I’ll go ask a couple of people if they can review it now.”

Perfect.

What does beta really mean?

comments 2
Growth
Beta. Just look at it. Majestic!

What’s the longest period of time that you’ve let software that you’ve shipped stay in beta? A week? A month? A year? 

What does beta even mean in the SaaS world of continuous delivery? Is it a relic of the past?

Let’s explore further.

Showing my age

Sarge.

It’s 1999. 

In the very early hours of a Sunday morning in April I am hunched over a desk, curtains closed, in my bedroom on the first floor of my childhood home. My face is lit dimly by the glow from a CRT monitor. 

I’ve been downloading a file called Q3Test.zip for most of the last twenty four hours, praying that my 56K modem doesn’t disconnect before it finishes. 

The dialog box reads 99%. 

This file was the public beta test of Quake III Arena, which to a die-hard Quake and Quake II fan like me was like having my birthday and Christmas rolled into one. It was the first chance to play the game that I was going to be spending many, many hours mastering (or, at least, trying to).

Beta releases of games were a big deal back then. It was a chance to feel part of the inner circle.

The anticipation of games in development was primarily generated by reading physical copies of PC Zone or PC Gamer, which would be the only reason my parents could convince me to come to the supermarket. 
I would browse the screenshots of current builds, which were hyped to fever pitch by the writers. I would imagine the secretive locations in which they were being created.

After many months of following the development of a game in a magazine, news would break that there was a beta available to download, and I would scramble online to IRC chat rooms to find out where it was hosted, and whether the mirror was reliable.

Reliability was key: in the late 90’s the fastest consumer Internet connection available in my area was dual ISDN, and that was way too expensive for my family. I needed somewhere to download it from that would not kick my little modem off while it was hammered by kids with fibre connections in the Nordics. 

99.6%… come on…

Stages

Following the games industry enabled me to begin to learn about how software was made. The beta stage was usually where people like me began interacting with it, as I was an avid gamer as a teenager, and I saw it as a matter of pride to be the first at my school to play the latest releases.

Jeff Atwood outlines the stages:

  • Pre-alpha: The software is under active development and isn’t ready for consumption by users.
  • Alpha: The software is ready enough for internal testing.
  • Beta: The software is ready for external testing, but has known bugs or limitations.
  • Release Candidate: The software is almost ready for final release and only tightly scoped bug fixes are allowed.
  • Gold: It’s done to the best of our knowledge.

Note: if you’ve ever been naughty and downloaded leaks of games, the “RC” in the filename stands for Release Candidate…

Now, for betas specifically, there are closed and open betas. 

A closed beta typically involves an invite-only period featuring users that have been selected to use the software, either because the number of concurrent users needs to be controlled, or because they are friendly customers who aren’t bothered by some bugs, or both. 

Open betas are where normal people like you and me get involved. As much as I would dream of id Software inviting me to their office, unfortunately it never happened.

What about in SaaS?

Now that I’m older, I don’t work in games; I work in SaaS. 

It’s an industry that moves very fast. You could posit that the software development stages listed above seem fairly archaic compared to the mindset of shipping to production continuously.

However, the formalism of the aforementioned stages is a defense mechanism: very large software projects such as AAA games and operating systems can’t ship broken code as the consequences would be devastating. 

Either a company could be humiliated by shipping millions of copies of physical media containing a broken build, or, even worse, a poorly tested operating system could ruin a person’s computer.

But, fortunately, in SaaS, we deliver via the browser and we can ship updates whenever we want. With the expectations of the expedience of delivery being high, we continually ride a line between acceptable speed of delivery and acceptable quality

This is why beta programs are so popular: allow an increase in the acceptable speed of delivery and a reduction in the acceptable quality. Win-win, right? 

Atwood pointed out two trends in 2008 which are more true than ever ten years later:

  • The definition of beta grows more all-encompassing and elastic every year.
  • We are awfully eager to throw alpha quality code over the wall to external users and testers.

Faster, faster, faster!

A mantra from Facebook’s past. It’s now “Move Fast With Stable Infra”, which isn’t quite as catchy.

Despite some parts of the software industry still following the release stages mentioned above, SaaS and the drive towards continuous delivery has created a trend of perpetual beta. 

There are a lot of large websites that continually have functionality in beta. Sometimes the entire service is beta: Gmail famously had a beta label for many years and so did Flickr.

I could summarize the current climate that I experience in my area of SaaS as:

  1. Expedited delivery of something of value to customers is the number one priority of any development effort.
  2. Done is always better than perfect.
  3. Perceived speed of development teams is the most important trait for Engineering in competitive VC-driven marketplaces. 

The last point may be cynical, but sometimes I feel it.

In 2010, an article suggested that delivering software using agile encouraged perpetual beta. But does this incremental, speed-first completion-second delivery mean that nothing ever gets finished properly, or is it actually useful?

Usage of beta in SaaS

In my own experience, there is often a lot of confusion around what a beta release actually means, and this confusion often compounds when views of engineers are mixed with those of non-engineers. 
Here is a list of situations that I’ve seen a beta flag used:

  • Features that are still slightly buggy but generally work and deliver value.
  • Features that aren’t yet complete (i.e. have a subset of proposed functionality).
  • Features that may or may not remain in the software based on customer feedback. This includes innovative or experimental features.
  • Features that are conceptually complete but haven’t yet reached the desired level of engineering quality (e.g. scaling, monitoring, or infrastructure robustness).

If this is the case, then there can be two forces that can drive the labelling of a feature as beta:

  1. Product: The feature or product isn’t complete or useful enough for all users to find value in it.
  2. Engineering: The feature or product is not scaled or robust enough for all users to have an acceptable experience.

This is very much a trade-off between Product and Engineering, and requires both disciplines to come  together to make a call on the use of the label, and most importantly, where progress on that software needs to get to before the label can be removed. 

And yes, it should get removed.

A healthy dialogue between both sides ensures that new features get into the hands of users through beta programs early enough so that feedback can be collected, but late enough so that they won’t be disappointed with what they see. You only get one chance at a first impression.

The positive side of beta

There is an overwhelmingly positive side to being able to run beta programs, and in users being more open to using beta software, even at work.

In fact, I’ve seen the excitement that I used to experience as a teenager when gaining access to beta releases of new games manifest in some of our own users when we invite them into beta programs that we are running. (OK, I exaggerate, but some do get very excited.)

By running beta programs in SaaS, there are a number of tangible benefits:

  • They allow you to get feedback much earlier in the development process.
  • They encourage more creative and innovative ideas because the impact of failure is much lower.
  • They allow your infrastructure to go under an initial stage of real load, rather than the simulated load you have done during development.
  • They can strengthen your relationship with your most engaged customers by giving them something new and exciting to play with earlier than everyone else.

We learn a lot from our beta programs, and some of our customers give such great feedback we should probably be paying them rather than the other way round.

Abuse of beta

However, for all of the positive talk about beta software, I’ve also seen the beta label be abused. 

Here are a list of things that I feel beta software should not be, yet have witnessed:

  • Untested. I mean come on, that’s just incredibly lazy. Your users are not your QA, and you shouldn’t feel comfortable or proud giving untested software to them.
  • Finished or indefinitely paused. Sticking a beta label on something that will never be completed is an anti-pattern. The point of a beta is to learn something, and should not be used as an excuse to ship substandard or orphaned work.
  • Used to mask diffidence. There is nuanced case where the beta label is used as a proxy for being nervous or embarrassed about the quality of what is being shipped.

Whenever I’ve observed the above happening, it’s often not via the intention of the team.

The first two situations often occur when there is external pressure to ship regardless of quality, such as when a feature has been promised and needs to go out to save face. Alternatively, they can both happen when reprioritization causes a team to drop what they are doing and work on something else immediately, and the “beta” launch is the salvage from the wreck.

The last situation stems from a nervous team, and may not even be indicative of their anxiety of how their users will react, but rather how important internal people will react, such as their managers, peers or the executives. The beta label is used as a buffer for any initial disappointment with the shipped feature.

Abuse of the beta label is often not because you have a poor performing team: you may have a poor performing culture or organization. It’s worth being mindful of this happening.

To conclude

Try not to spend over 5 years in beta. This product seemed to take off, though.

Beta releases, when handled correctly, are excellent tools to ship faster, please users, generate excitement and get valuable feature and infrastructure feedback. 

However, you must always ensure that beta releases are being done for the right reason and are not being used to hide smells of poor organization or a culture that doesn’t care about the quality of the code that is being given to its customers.

Beta is for learning and experimentation. It is not an excuse for unfinished work.

Project X

Leave a comment
Growth
“Ouch”, cries today. Photo by Johnson Wang on Unsplash.

In an open plan office in a galaxy not so far away…


Lisa rotates in her chair and looks upwards at the ceiling. She poses a question:  

“I’m just about to create the repository in Github. What should we call it?”

“Well, we’re not even sure what it’s going to do yet. I don’t even know if what they’re asking for is possible.”

Ben looks back towards his monitor, opens up Google and types in “cool project names” and hits the return key with vigor. Lisa scoots her chair over to Ben’s desk to look at the results. 

“That’s the one.”

“Oh yeah,” she replies.

Switching back to his initial browser tab, Ben types in “Project X” and creates the repository so that the Data Science team can start hacking around with ideas.

Lisa uploads the first file. It’s a Python script that prints “What on earth is Project X? 💁” when executed. Ben laughs. He commits a README.md that states:

The first rule of Project X is that you don’t talk about Project X.

Lisa stretches her arms into the air and cracks her knuckles. She sighs.


A gentle clinking sound repeats as Lisa whirls the milk into her teacup in the kitchen. She looks through the glass partition that separates her from the breakout area. 

The light in the open space is dimmed, and row upon row of chairs are packed tightly with what looks like the entire commercial team, listening intently to the presentation of the company roadmap.

Leaning against the kitchen wall, she watches the projection as screenshots of the latest iteration of the mobile app are shown. Laptops glow as the audience take notes.

Sipping her tea so that it doesn’t spill over, she turns to leave the kitchen, but is distracted by the reflection of a giant “X” on door. It’s coming from the presentation.

The slide says what she had immediately feared: “Project X”. The CEO is animated. The crowd are clapping. Lisa’s heart sinks. She fumbles with her phone to send Ben a Slack message.

Lisa: They’ve just presented Project X during the roadmap deck.

Ben: Are you serious?

Lisa: Yes.

Ben: But we don’t even know what it does yet.

Lisa: What do we do?

Ben: What did the slides say?

Lisa: It’s coming next quarter, and the whole of downstairs was hyped.

Ben bursts through the kitchen door. 

“What the hell is going on?”

“We are absolutely screwed,” replies Lisa, who is now pacing in circles.

“It’s literally a CSV file with two hundred lines of test data for a feature that hasn’t been designed. How are they announcing it already?”

Lisa stares into her mug. “Will they notice if I leave the country?”

“I’m looking up flights to Siberia,” replies Ben.


Lisa! Our rockstar data scientist! So good to see you!”

It’s the head of sales.

“Oh, hey Mia.”

“Let me tell you this, Lisa. I love the website. I love what you’re doing for this company. It’s going to absolutely blow the competition away. My team are going to push this so hard when it’s done.”

“Website? What do you mean?”

Double-you double-you double-you dot project x dot com. It looks incredible. The announcement last night on the webinar? Fantastic. Thousands of our customers watched it, Lisa. Thousands.”

“What?”

“I love it, Lisa. Love it!”

Mia is already out of the kitchen and involved in another discussion. Lisa walks back over to her team and taps Ben on the shoulder. After finishing the current line of code with a semi-colon, he removes his headphones.

“What’s up?”

“Can you go to project x dot com?”

“What?”

“Just type it.”

Hitting the return key, the widescreen display is filled with a humongous “X”. Wagner’s Ride of the Valkyries is audible from the headphones on the desk.

“Oh my God.”

The X fades away to reveal a satellite image of the Earth, spinning gently on its axis. Small X markers start appearing all over the globe. London. Tokyo. Hyderabad. Dubai. Lines then arc from city to city, forming an elaborate mesh across the planet.

There is stunned silence.

As the horn section reaches crescendo and the cymbals crash, the globe fades and a tagline appears:

Everything changes. November.

Ben is livid. “What the hell was that? Where did that November date come from? It’s already October!”

Lisa is staring through her hands. “We are so screwed.”


Errrrr… Photo by rawpixel on Unsplash.


Ben is having difficulty moving his mouse cursor because his trackpad is covered in sweat. Usually the Data Science demo sessions have a small handful of attendees, but today is the first live demo of Project X, and there isn’t anywhere left to stand.

Mia has brought her entire sales team. Most of the engineering team are present. Even the CEO is there. Lisa is leaning against the wall for support. Her legs don’t feel like they are able to stabilize the rest of her body.

Ben coughs. His cheeks are burning.

“OK, hello – wow, there’s lots of you here. A couple of demos today, but let’s kick off by showing you the latest build of Project X. Please bear in mind that we’re not finished yet and this is just a preview.”

Leaning over his laptop, he types his username and password into the login prompt and clicks the button to log in. An error pops up: login failed.

“Damn it,” he whispers to himself.

Lisa leans over so she can whisper to him. “It’s not hooked up to the live environment; try your development password.”

Ben enters his credentials again, but manages to type them in the address bar, revealing his password to the entire room. His cheeks flush for a second time.

Somebody from Engineering is humming the Ride of the Valkyries. Another sniggers.

Lisa rapidly shifts over and takes control of the laptop, successfully logging in to reveal a white screen with a text box on it.

“Er, OK, so this is our latest prototype. It’s probably just easiest if we give it a go. Can someone suggest a word to me?”

Somebody pipes up at the back of the room.

“Eggs!”

“OK, let’s type that in.”

Lisa successfully manages to type “Eggf”, which is quickly corrected to “Eggs”. She presses the return key. A table of data is returned with some fairly arbitrary numbers about eggs: their size, the quantity that are produced each year, and the ratio of egg size to the creature that laid it.

“There you go! Cool, huh?”

There is silence in the room. Faces look blank, confused, and disappointed. Ben and Lisa exchange glances. 

“Any other words to try?”

Mia pipes up. “Is this Project X? Is this our biggest new feature of the quarter? How are we meant to sell this?”


Ben is leaning over and prodding at the Nespresso machine with his left hand, which has managed to swallow his coffee pod and jam the input tray, much to the dismay of his colleagues who are impatiently waiting for the day’s first shot of caffeine. With his other hand, he is scrolling through a forum looking for instructions on how to hard reset the machine to release the drawer.

“Technology, eh?”

It’s Lisa, back from vacation after the rather intense end to Project X. 

“Why does the drawer have to be controlled by software, rather than it just being a manual latch?”

“Because software is eating the world, Ben. And now it’s eaten your coffee pod.”

Back at their desks, Lisa gets the rundown from Ben on the next project.

“So they’ve asked us to work out whether we can link together the browsing patterns of our users with the best times of the week to send them promotional emails.”

Lisa leans back in her chair.

“OK, sounds reasonable. Have you looked at any prior art?”

“Not yet. I’ve still been dealing with the fallout from this ridiculous Project X thing. I’m answering what feels like hundreds of emails a day about what it is, how it works, and why it didn’t change history, resurrect Martin Luther King or put another man on the Moon.”

Ben pauses for a second. “What did they expect?”

Lisa rests her head on her chin.

“Let’s just forget about that. I’m ready to start on this new thing. I think I saw something about behavior tracking at the last CHI conference that could be a good place to start. But before we do, this project needs a name.”

“Oh, how about we name it after a goddess? Oh, oh – or maybe one of the Grecian Fates? I always liked the name Atropos. Not a huge fan of Clotho though…” 

“No, no, no,” interjects Lisa. “We’re calling it ‘Best Time To Email’. We’re not having Project X happen all over again.”

Lisa commits README.md into the new best-time-to-email project on Github. It reads:

A project to find out the best time to email our users.

“Let’s see them make an animated spinning globe out of that.”