Cadence

comment 1
Growth

Keep on pedaling

Cadence is a term used to describe the rate at which a cyclist is pedaling. Different cyclists find their efficiency in pedaling at different rates. Looking at the professionals you can see that Chris Froome has an unusually high cadence whilst Nairo Quintana tends to cycle at a much lower cadence. They’re both effective riders despite this difference.

As each cyclist has a preference for the cadence that suits them most. When the gradient of the terrain on which they are cycling changes, they switch gears up or down so that they can maintain it. Getting the gearing wrong can be catastrophic as it causes lost ground and wasted energy.

If the gear is too low, then the cyclist pedals too fast – what is known as spinning – and a lot of energy is consumed for a negligible amount of overall movement. If the gear is too high then the cyclist can find themselves having to push extremely hard to move the pedals – known as mashing – and again, energy is wasted. When you mash going uphill it’s very hard to change gears because you can’t turn the pedals fast enough.

Cadence in software delivery

Similar to the way that cyclists need to discover which cadence is right for them, you as a leader need to define what your cadence of delivery should be for your team, division or department. What sort of projects are you expected to ship and at what frequency? What defines a good quarter or a bad quarter? A good year or bad year?

The right cadence for your department will depend on a multitude of factors, such as the industry that you’re in, the speed of the competition, the way that your software is delivered (e.g. SaaS versus on-premise), and the number of staff that you have. You may also find that the executives at the company have a predefined expectation of how many big launches are expected per year, for example, and how fast other features should iterate.

If you’re new to the company, it’s good to solicit opinions on what is expected. Is it one headline-maker per month, quarter, or year? Are your users expecting increments every week, month, or quarter? How do you plan with longer-term, risky, innovative initiatives that might fail?

Others expect steady cadence

Regardless of what the expectation is for the cadence of delivery coming from your department, I’ve found that there is one very important thing: those outside the department are very sensitive to your cadence changing. Life is more comfortable for everybody when there is a predictable flow of completed projects, rather than a packed quarter followed by a quiet one, followed by an average one.

Now, people aren’t silly: they understand that some features are small when compared to building a whole new product or revamping the entire back end infrastructure. However, having a steady and predictable cadence is important.

  • Marketing are allowed the time to plan and execute their webinars, advertisements and events. If your output is spaced predictably, they too can predictably plan their time. Too many launches in close vicinity isn’t desirable for market impact: it lessens the individual impression of each launch. A steady cadence makes their pipeline easier to manage and the results of their activities more measurable.
  • Sales rely on the your cadence for their ammunition. Walking into a pitch knowing that the Engineering department is shipping predictably and often is a huge confidence boost. When working on important RFPs, knowing that Engineering will be popping out a steady stream of features during the process can be the deal breaker when you are compared to the competition.
  • Engineering themselves love a predictable cadence. Your teams know that there will be plenty of times to launch and celebrate throughout the year and to get their teeth stuck into different parts of the system.

You’ll need to ensure that your selection of ongoing and upcoming projects maintains a steady cadence. That means having smaller projects delivering while long-burners tick away, iterations of existing functionality while technical debt is fixed, and that your headline launches are timed well for the benefit of the company.

Maintaining cadence

Categorizing your projects

If you have an idea of what is expected for the cadence of your department, then the next step is to work out what an ideal flow of projects would look like at any given time. These categories of project can be balanced in such a way that you are always shipping something at regular intervals, regardless of whether it is big or small.

Have a look at the categories below. Next to each is an indicator of visibility and size.

  • Big ticket products and features: These are big, shiny additions to your overall offering. They could be a whole new product or an advanced differentiating feature. These can involve new and uncertain work, and will have a very visible impact if the deadline is missed (e.g. they are to be unveiled at an event, or there is an in-depth marketing announcement planned). These are of high visibility and of large size.
  • Iterations on existing functionality: These are incremental improvements to what you already have. This could include new visualizations of existing data, a slicker UX workflow for a part of the application, or a revamp of the login screen. Medium visibility, medium size.
  • Bug fixes and small improvements: Fixing bugs that customers have reported and tweaking what you already have. For example, fixing the format of some data being returned in an API call, making the delete button not occasionally throw an error, fixing a situation where a pop-up can get stuck. Low visibility, small size.
  • Architectural work: Migrating or redesigning storage, horizontally scaling a previously monolithic process, rewriting an API. These are the long-running initiatives that ensure your future health, but may deliver nothing more than feature parity. Minimal visibility, large size.

Getting the balance right

The exact composition of your teams and the number of important projects will change over time. However, as a leader in Engineering you will need to make sure that the balance is right. There are dangerous situations that you will want to make sure that you don’t get into.

  1. Neglecting work streams, where overall focus is continually biased on some areas resulting in other important streams of work becoming neglected.
  2. Not tackling technical debt proactively, which causes big problems down the line that are often much harder to explain to other parts of the company.
  3. Going long and thin, wherein a lack of bold decision making – typically being unable to say no – means that you have too many concurrent projects, resulting in them being understaffed and slow.

Let’s look at these in turn.

Neglecting work streams

This is a very easy trap to fall into. Running a SaaS software company requires continual improvement of your application estate. The industry and competition moves fast and, most importantly, your users are used to incremental improvements in other software they use.

Ensure that all areas of your application are revisited for refinement with time. Even better, have enough teams so that all areas have a clear owner, and that they work to KPIs that encourage continual improvement. Ignoring them leads to rot in the form of technical debt.

Technical debt

This must continually be tackled, in all forms: upgrading old infrastructure, redesigning and rearchitecting for scale, and ensuring that code is continually refactored as new code is added. Empower your engineers to be confident enough to estimate tasks so that they can always pragmatically follow the Boy Scout rule.

Also empower your engineers to tell you which parts of the application estate are due an overhaul and why. Make plans to do so while you ship other things. Not only will this work make your application more scalable and prevent future crises, but your staff will be motivated to make a difference if they are the ones suggesting it!

Always, if possible, have technical debt being tackled in some capacity all of the time. Your future depends on it: increasing amounts of technical debt will make new projects even harder to build.

This makes it akin to mashing in our cadence metaphor: the pedals get harder to turn as the hill gets steeper, and it becomes even harder to change gear.

Going long and thin

It’s also very easy to go long and thin. As a leader you will need to practice saying no to incoming demands so that you can keep your projects moving at an acceptable pace. Whilst it can be easy in the short-term to say yes to that extra favour from a client or the CEO, it further dilutes the amount of staff that you have to work on each work stream.

Although this creates the illusion that you are being more effective and working on even more projects, the reality is that you’re just getting them all done much more slowly, becoming more exposed to people being ill or going on holiday, and setting yourself up to apologize for being late down the line.

Too many concurrent projects is akin to spinning in our cadence metaphor: lots of work for not much reward.

A good mix

So consider what a good mix is for you. This will shift depending on the life stage of your company, but regardless, the advice is actually quite simple. At any given time, balance teams producing low visibility work with others producing high visibility work. For teams tackling very large projects, ensure that they are covered by other teams shipping small projects.

When you hit it right, it will feel like your projects are flying out the door all year round. That critical year-long rewrite of your backend is protected by tens of smaller features rolling out. When you hit it wrong, you may appear to ship nothing despite putting a lot of effort in, or you may produce a lot of low impact work whilst ignoring the bigger issues at hand.

In summary

We’ve looked at the idea of cadence of delivery, and how it is important it is to not only ship, but to ship predictably often. You need to balance ongoing priorities and projects, continually tackle technical debt and also be confident in saying no if it makes you oversubscribed.

Where do you stand with your current cadence? Do you feel that you are staying steady, regular and efficient, or are you spinning or mashing? If so, why do you think that is? Was it pressure from others, or was it just down to good or bad planning?

Your levers: scope, resources and time

Leave a comment
Growth

The Iron Triangle

I’m sure that you’ve heard a version of the phrase “fast, good, or cheap. Pick two.” In terms of getting a piece of work done, mostly by an external paid contractor, it holds true. If you want something good and fast, it’ll be expensive. If you want something good and cheap, it’ll be slow. If you want something fast and cheap, it’ll be poor quality. This is applicable from dry cleaning to engineering.

In the world of knowledge work, this is often called the Project Management Triangle, the Iron Triangle and the Triple Constraint. Typically when using this model the “cheap” part refers to the amount of cash being spent; i.e. the choice between a renowned agency to complete the work (expensive, predictable) versus an unknown independent party (cheap, unpredictable). Similarly, it could represent the decision as whether to offshore work to a country with lower labour rates (cheap, unpredictable*) or use specialist contractors from, say, the Bay Area (expensive, presumably predictable). I’m sure you can think of other examples too.

Work on the horizon

How does the Iron Triangle relate to projects that you are running with your own teams?

We’ll be limiting the scope of this article to delivering software that you are developing in-house. You will have a finite number of engineers at any given time, and you will routinely find yourself juggling your ongoing projects with the need to forge ahead with the next great thing that you should be building.

As touched upon in a previous post, you will be wanting to make sure that your current projects are delivering as fast as you can and that you are being transparent about their progress. But for new projects that are in the planning stages, how should you be justifying your decisions to other parts of the business to ensure that the next thing is going to come out as expected with the least number of surprises?

Everything, all of the time

When discussing upcoming projects, you may face conflicting opinions from different departments, including your own. These particular examples are taken to an extreme, and in reality people are more pragmatic, but for the sake of discussion we present the following:

  • Commercial want it now! Your salespeople are finding it difficult to pitch against your competitors because of features that are lacking in your application. This next feature can’t come soon enough, because their confidence in you is paramount to their success.
  • Product want it all! Your product manager has a grand vision for the roadmap for the next year. The designs look beautiful and the offering compelling, but you can see that it represents a huge amount of work. Probably too much work.
  • Engineering want it right! Your engineers see this new roadmap as the key trigger to redesign a large part of the system. Building it around what already exists will introduce a lot of technical debt that will cause serious pain in the future.

Three different opinions that are all equally valid. A healthy tension exists between them: where do you begin the discussion?

Your levers

Now it goes without saying that you would not want to compromise on quality. I would argue that if you are happy with shipping poor quality software, then you are probably in the wrong job.

Instead, you have three levers that you can adjust in order to find the right compromise with new projects. They are scope, resources, and time. Sometimes you have flexibility over all of them, and sometimes you won’t.

  1. Scope: The definition of what the project is going to deliver.
  2. Resources: The number of engineers that you are going to assign.
  3. Time: The duration that you have to work on it.

Imagine that you are beginning to explore a new project. How do you frame it amongst everything else that is going on?

Scope

Let’s dig into scope. Your engagement point is to have your team work with your product owner to break the product or feature into epics and stories that each deliver tangible added value to your application. This is your project backlog. Then, from there, work together to prioritize it.

With your prioritized backlog created you can probe more:

  • Launch methodology: Is there going to be a marketing launch for this feature? If so, where in the backlog would the project need to get to for that to occur? Or is it all going to roll out with no fanfare at all?
  • Shipping frequency: Is it shippable to users as each individual chunk of functionality is finished? If not, can it be shipped to production behind a feature toggle instead? Who makes the call on when users should see the new functionality?
  • Criticality of scope: Do we absolutely need every single piece of functionality to be done in the project to consider it a success? Can any of it be considered suitable for stretch goals and/or incremental releases after launch?

Deciding these up-front can save a lot of future stress and panic by giving you buffers to work with.

Talking to marketing right at the beginning of a project can help them prepare way ahead of time, and show you how they’re getting along as they progress. It’s exciting to see a launch develop! Being able to ship frequently, even if it’s hidden behind feature toggles, can prevent high-risk big-bang changes and allows you to decouple the release date of the software from the date that it ships to production. You are given more flexibility to test on real data and real load. Knowing which parts of the scope can be dropped in case of delays and technical issues gives a safety net for when things go awry.

A little extra thought up front can make projects run much more smoothly.

Resources

What size of team are going to be required to get this work done? It can be tempting to throw the proverbial kitchen sink at an important new project, but there are a number of considerations:

  • Parallelism: How much of the work in the scope is sequential and how much can be done in parallel? Two separate services can be developed independently, but iterations of the same service or component cannot.
  • Code vicinity: The last thing you want is multiple engineers all working on top of each other in the same part of the code, causing nasty merge conflicts with their commits. You don’t need four spanners scrabbling over the same nut.
  • Technical difficulty: Is this looking like a business-as-usual increment of functionality in your existing application, or is this a new innovative piece of architecture? This drives decisions about which of your staff might be required to work on it, or at least offer their time to oversee it.
  • Relative importance to other projects: Is this new feature the number one business priority and therefore it needs to ship yesterday, or can it arrive more calmly with a smaller team, as there are other projects providing steady cadence to your overall delivery?

Everything is always a trade-off.

Time

You cannot bend time. However, how much control do you have over the timing of the delivery?

  • Be clear about priority: Each project cannot be considered to have equally immediate urgency, as the reality is a more subtle balance of many concurrently active initiatives for different people with differing priorities.
  • Identify projects that could slip: Is it the case that you have an immovable date such as an unveiling at an event or a client-imposed deadline where money is on the line? Allowing flexibility with timing of other projects allows you to be more inflexible with the critical ones when necessary. If your inflexible projects become unstuck, then resources for your flexible ones give a buffer to regroup staff to address problems. Try to identify which projects are flexible and which aren’t.

You can’t create more time, but you can use it wisely in decisions. Knowing which projects can slip if necessary and those which can’t can contribute to making sensible and pragmatic judgement calls.

Making your levers transparent

If you have a considered approach to your new and ongoing projects using the levers of scope, resources and time, then you can make clear choices that are easy to explain when questioned about your priorities. Making your decisions and their reasoning visible internally to your department can also help teams see how their projects fit into the bigger picture.

The trade-offs of scope, resources and time guide debate around pragmatic concepts that you can discuss openly and honestly, shifting the conversation towards how further trade-offs can be made when under pressure, rather than inviting ungrounded criticism about how hard you and your teams should be working instead.

Visibility of these trade-offs also empowers your teams to help each other too: one team may offer to lend some resource because they know another team’s project is critical and immovable, and their own more flexible.

How many projects are you running that have been considered fully as part of the whole?

 

*Not always unpredictable depending if you have good contacts with offshoring partners. However, the cost of offshore development is being driven up by demand from areas with the highest average salaries to begin with, such as the Bay Area and NYC. If you are hunting for a good offshoring partner from a company with less extreme salaries, you may be surprised at the comparative cost.

Why is Engineering getting so slow?

Leave a comment
Growth

Expansion: the aftermath

The company did brilliantly two years ago.

Last year you managed to open up nearly 40 headcount in Engineering and you managed to fill those roles with talented people, almost doubling the size of the department. Yet, another year on from there, others in the business are left wondering what effect that had.

Shouldn’t have you been able to dramatically ship more software with all of these extra people? Where are all of the features that you were previously unable to prioritize because you didn’t have the staff to do so? Why is it that a small competitor is seemingly moving faster than you?

If you have managed a department through a period of rapid growth, and especially if you have aspirations of doing so in the future, you will experience this scrutiny. You know that simply adding people doesn’t make everything faster, but what can you actually do about it? How can you get others in the business to understand?

Internal and external scrutiny

Let’s divide the source of the pressure into internal and external scrutiny.

  • Internal scrutiny is probing from inside the business as to what has happened to your output after your department expanded: after all, even adding just tens of people, depending on their seniority, can represent a yearly investment from the business of over £1M. They’d like to know where that money is going and what it is being used for.
  • External scrutiny is the perception of the business in the marketplace after it has seen you grow. Maybe there was a PR piece explaining how your company is continuing to expand after a stellar year, or maybe you raised VC capital. Those that track the marketplace and your competitors will observing your business to see proof of what that growth has delivered. No visible progress may suggest to the outside observer that there are problems. You don’t want to look weak to your market.

The unfortunate truth is that even if you are shipping often, you will always face questions about your output, both internally and externally. As a leader, you have two weapons against the dark forces of criticism for being slow.

The first is making sure that you’re building things as quickly and efficiently as you can, given your current circumstances and resources. That’s a given. The second is making sure that you are communicating transparently and often about what you’re doing. Engineers are often very good at building awesome things the right way, but often not as good at communicating what they’re achieving, hence the scrutiny.

Before we look at some techniques you can use, why is it that you might be naturally getting slower as you get bigger?

It’s not people, it’s productivity

Let’s clear something up first. Arguments around perceived speed can become ad hominem very quickly, and that’s a route that you do not want to go down. While it is true that bigger departments can mean that performance issues can go unnoticed more easily, it isn’t a small handful of individuals that are making your whole department slow. It’s about being a victim of your own success, having to deal with legacy systems, and the increased communication overhead of having ever more people. The more accurate way to phrase your perceived slowness is that the productivity per head is decreasing, and this is the problem that needs addressing.

So what makes this productivity drop? Let’s revisit those three areas.

1. Being a victim of your own success

As brilliant as it is to do well, it also creates more work. You will have more customers expecting a higher level of service. You will have to create or expand your support team. You will have to tighten up the security of your system. You will have to improve your monitoring, build up your out-of-hours rota, and so on. Additionally, you will be continually storing more and more data which requires continual expansion and upgrades of storage systems. You may need to rewrite whole parts of your application estate so that they scale. An ever-expanding array of work requires attention just to keep things running the way that they currently are.

2. Legacy systems

Being a victim of your own success also means that you’ve created a lot of code that is fast becoming legacy. It is impossible to predict future needs, so it is therefore impossible to create infinitely extensible systems. A new product direction, new set of features or complete pivot can highlight that your current codebase is going to need dirty great hacks (bad) or a complete rewrite (also bad) to survive.

Even if you are able to continue extending your current codebase, the ever-increasing size means that it takes longer for your engineers to understand where best to fit new features, longer to write tests that actually cover all end-to-end scenarios (if that’s even possible) and longer for new changes to be peer reviewed. There’s the added pressure that having a well-established paying (with money and/or data) customer base means downtime really isn’t acceptable any more, increasing the collective fear of making sweeping changes quickly.

3. Communication and process overhead

When you’re a small start-up, the entire business can make unilateral decisions at speed as they can sit in the same room. When you’re bigger, global, and have clearly defined departments, roles, and responsibilities, decisions can converge towards a glacial pace. This isn’t limited to business decisions: technical decisions can also suffer. A decision on how to overhaul the main codebase could affect hundreds of engineers. How do you make sure that enough people are consulted? This can mean more processes and meetings are required for consensus to be reached. This can get slow, but it’s unavoidable.

Without clear processes, meetings and steps for how decisions are made, life can get very messy. The Mythical Man Month famously outlines the effect of increasing numbers of staff on potential lines of communication. When n people need to communicate amongst themselves, there are n(n-1)/2 channels*. For example, a 400 person company has 79800 communication channels(!) We need meetings and processes to tame this, but not too few and not too many. Getting the balance right is hard.

So what can you do?

We’ve painted a pretty dire situation. What can you do about it?

There’s no silver bullet, but as a leader you only really have two tools at your disposal for ensuring that you’re going quickly enough: ensuring that you’re all building software in the best way that you currently can, and making sure that your progress and decisions are communicated as transparently as possible.

1. Develop software pragmatically

You’ll need to make sure that your engineers are being as pragmatic as possible when it comes to building and maintaining your application estate. As the company grows, so will the codebase, and the risk of it getting gnarly increases dramatically.

To begin with, coach a culture of refactoring as you go along. I like the boy scout metaphor, where each engineer should try and leave an area of the codebase they are working on cleaner than when they arrived. The key in this approach is the scope: just improve the area that you are working on. You wouldn’t expect the boy scouts to improve the entire forest after camping there. Refactor a clear path and move on through.

Actively encourage technical debt being tackled. Perhaps reserve 10% of the time in your sprints to working on technical debt issues, and as a leader, champion this decision. Encourage your staff to all advocate how important technical debt removal is for the long-term success of the department. Related to this: never be afraid of deleting features. Did it not ship fully? Delete it. Did interactions with that area of the product wane significantly over time? Make the hard call and get rid of it. It will only slow you down in future.

2. Have small purposeful teams

Zooming out from the code, you’ll also want to make sure that your teams are right-sized. They need to be big enough and with the right balance of skills so that each team can make progress on their project or feature with as little outside support as possible, yet small enough so that decisions happen quickly. A team that gets too big can become impossible to plan for, both in the quantity of work and in sticking to a theme that unites them.

Each team should have a clear purpose and reason to exist. Perhaps they build and maintain a particular product or feature (i.e. their success is customer-facing) or they are purely building infrastructure (i.e. their success is serving the department). Either way, teams with a clear purpose will have an intrinsic motivation to go fast because they will be self-motivated. They will require less hands-on management as a result.

3. Communicate transparently and often

As a leader in the department, you should not expect that if you are having a stellar quarter that the rest of the business is going to know about it. Part of your role is ensuring that everyone else is informed on your progress. You owe it to your staff to share the great things that they are doing.

There are a number of techniques that you can employ here:

  • Broadcast. You could publish a regular newsletter, written for a non-technical audience (e.g. the commercial mailing list), describing the progress in your team, division or department. It sounds simple, but in my experience, people really engage with these communications and enjoy reading them. Especially so if they are light-hearted and entertaining.
  • Invite outsiders in. Always give other departments the chance to engage. Invite them to be stakeholders in your projects and demos. They will then better understand the pace of progress and the difficulty of technical challenges.
  • Be social. Hold regular catch-ups with those in your network to discuss everything that you’ve been up to. Ask their advice candidly as a peer and friend.
  • Offer Q&A. Ask to drop in on existing non-Engineering meetings where required as a guest. You could even set up your own regular session with the rest of the business.

The idea here is that as the company grows, the noisier the company gets. You need to ensure that even if you are doing great work that people know that you are. No matter where you work, you will probably face accusations of being slow, so you need to make sure that the rest of the business is making that judgement based on the facts that you present rather than just assumptions about what you’re up to.

In summary

It’s not that people get slower as a company gets bigger; it’s that productivity per head decreases when those staff are part of a larger organization. As a leader, it’s important for you to make sure that your teams are developing software in the most pragmatic way possible and that their progress and achievements are transparent to the rest of the business. If you are open, accessible and just plain amicable as a leader, you’ll find that the internal scrutiny will develop into camaraderie as you are able to prove that you’re doing the best that you can with what you have.

The knock-on effect is that those who question your speed can engage with you in conversations that criticize the right things (i.e. resources, scope, prioritization) rather than the wrong things (i.e. not “working hard enough”, performance of individuals), as the right things can be collaboratively discussed and altered where necessary, without judgement.

*This is the same number of edges in a full mesh topology. Humans and computers have more similarities than you think!