Eliminating Web Development Waste

The concept of waste within web development might seem reasonably clear to most managers and developers, however in my experience it is something that most teams need help to identify. I’ve recently worked with a team who became exceptionally good at identifying waste and, as a result of this alongside a whole swathe of other techniques, were by far the most efficient agile team I’ve ever worked with.

Eliminating waste is the first—and most fundamental—principle of Lean Software Development; something that was introduced to me by an exceptionally talented Project Manager whom it was my pleasure to work with. Lean Software Development is a set of seven fundamental principles that really work hand in hand with agile development to improve software delivery. Those principles are: eliminate waste, amplify learning, decide as late as possible, deliver as fast as possible, empower the team, build integrity in, and see the whole. In this article I’m going to look at how we can improve our ability to identify waste and iteratively remove it from our development chain.

Lean manufacturing

The principles of Lean Software Development are basically reinterpretations of the principles of Lean Manufacturing as designed and tested by Mary and Tom Poppendieck. Those original manufacturing principles—formulated by Sakichi Toyoda, the founder of Toyota Industries, and his son Kiichiro Toyoda, the founder of the Toyota Motor Corporation—were realised as an actual method of production by the Toyota engineer Taiichi Ohno. This later came to be known as both Just In Time Manufacturing and the Toyota Production System.

Lean Manufacturing is a production practice that considers the expenditure of resources for any goal other than the creation of value for the end customer to be wasteful, and thus a target for elimination. Working from the perspective of the customer who consumes a product or service, “value” is defined as any action or process that a customer would be willing to pay for. At its most simple, Lean Manufacturing can be considered a move towards manufacturing to order rather than manufacturing to build stock since stock is essentially waste.

In reading descriptions of American supermarkets, Ohno saw the supermarket as model for what he was trying to accomplish in the factory. A customer in a supermarket takes the desired amount of goods off the shelf and purchases them. The store restocks the shelf with enough new product to fill up the shelf space. Similarly, a Toyota work-center that needed parts would go to the inventory storage point for the particular part and withdraw the quantity it needed. The storage point would then be restocked by the work-center that produced the part, making only enough to replace the inventory that had been withdrawn.

The Toyota Production System

The key objective of the TPS was to design out overburden (muri) and inconsistency (mura), and to eliminate waste (muda). A manufacturing process that removes inconsistency is capable of delivering smoothly. However, that process also needs to be as flexible as possible so as not to introduce overburden which creates waste. Also there is clear value in the elimination of waste in any process.

TPS Reports

Anyone who’s worked in the industry for a while, or has seen the film Office Space, will be familiar with the term “TPS reports” being synonymous to pointless paperwork. TPS reports have nothing to do with the Toyota Production System, the TPS stands for “Testing Procedure Specification”.

That being said, TPS reports as pointless paperwork is entirely relevant as waste that could be removed from your processes. We’ll come back to that later.

Web vs. Software Development

Before we get too involved in investigating the waste element of this post, let’s just examine the difference between general software development and web development:

I consider myself to be a web developer and a web development manager. I am used to working with the internet as part of my development landscape and also as part of my toolkit. This means I am used to dealing with high numbers of users, a vast array of user-agents which interpret my code and therefore need to be tested against, a degree of uncertainty as to the capabilities of the user-agents and the users themselves, and a wide number of technologies. This has been described by Douglas Crockford—the inventor of JSON, and often described as the godfather of JavaScript—as the “most hostile environment imaginable”.

This is not to say that software development is any less hard or varied, just that there are a number of specific complications put on me as a web developer; and there are a number of very specific web-sourced problems that I’m used to solving. It’s simply different because it specifically involves the epic WAN that is the Internet.

Lean Software Development translates very well when dealing specifically with web development since the same problems exist in both development life-cycles. What really varies is how we might implement a solution to those problems. Taking that into consideration, be aware that I’m writing this article as a manager of web developers not as a manager of software developers.

Waste in Web Development

The TPS identifies seven types of manufacturing waste:

1. Overproduction
2. Waiting
3. Transportation
4. Extra processes
5. Overstock
6. Motion
7. Defects

In relation to software development, however, it’s clear that not all these types of waste translate. Lean Software Development identifies the following types of waste which translate perfectly between software and web development:

1. Partially complete work
2. Extra processes
3. Extra features
4. Task switching
5. Waiting
6. Motion
7. Defects

Since some of those are not immediately clear as waste, let’s examine them in more detail…

Partially complete work

It’s fair to say that all web development is partially complete at some point in its lifecycle. The priority here is to minimise the amount of partially complete work that is within a project at any given moment.

Partially done web development can quickly become obsolete as it is replaced with a differing method or solution. The only way to be sure what problems exist in a piece of code are to have it fully integrated with the rest of the project. If you are working behind feature branches or feature toggles, or some other method for cloistering partially done work, you are suffering from your isolation as much as you are benefiting from it.

In fact, the only place where you will capitalise on your investment in a feature is when it is in production. Partially done development ties up resources in investments that have yet to yield results. You need to provide integration test coverage, end-to-end test coverage, integration code, and a whole host of other methods of ensuring the minimum of bugs exist when you finally release it from its isolation. If your feature never actually makes it to production, you have some significant investment to write off. Reducing the amount of partially complete work within your project is both a reduction of waste and risk.

Extra processes

We all love our processes and documentation, right? I’d be willing to bet that just about everyone who has worked in an agile organisation of some form has become disillusioned with the amount of meetings or breaks from focus they have to attend as agile “ceremonies”, and the added work dealing with documenting user stories, their requirements, and how they might be tested and accepted. In fact, two of the tennets of the agile manifesto are specifically related to process and documentation:

We have come to value:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation

At every opportunity you should be assessing whether a particular process or document is entirely necessary for your development. Extra processes and documentation consume resource, slow down response times, and can mask problems elsewhere. What’s more they break focus, which is something I’ll come back to when I explain “motion” as a type of waste.

Many development processes require paperwork of one form or another. Most of the companies I’ve worked for recently have replaced actual hard-copy with wiki documentation these days, but those documents are still there. Documents for release processes, documents for sign-off, documents for traceability, not to mention documentation for security or compliance.

The real test of these documents is whether someone is actually waiting for it as a deliverable, and whether it then adds value to their process. If not, dump it. Otherwise, find ways to simplify that process. Reduce the amount of input required by using templates or pre-designed input forms. Condense the amount of information required. In fact, there should always be an ongoing process of refinement and efficiency evaluation for all documents and processes. Any required process should be as efficient and effective as possible. Rapid yet easy to validate.

In terms of documenting code, do it as you go. Write clean, easily understandable, easy to maintain code and enforce style with static-code analysis and code reviews. Make sure code is commented effectively and you’ll cut down on cost. It’s important to remember that sometimes documentation is required for code when knowledge needs to be shared over a wide base, but if you’ve minimised what other documentation is required this will be the exception to the norm.

Extra features

Adding in extra features to a piece of development is over-engineering; don’t do it. There will be many times that you foresee the need for an extra feature before you actually have the requirement. There are other times that a developer might see the opportunity to add something extra that may save time for others later on. Whilst at first glance this may seem relatively harmless and of benefit in the long run, it is almost always a source of waste.

All code in the system should be tracked, reviewed, tested, and integrated every time it is touched. What’s more it then needs to be maintained for the life of the project. With that in mind, it is important to remember that every bit of code adds complexity and a new possible point of failure. It also introduces something new for all the developers to understand and—worse—misunderstand and misuse.

If there is no requirement for a piece of code now, don’t write it. Putting it in is waste. Resist your urge to over-engineer.

Task switching

The fastest way to complete two projects using the same people is to do it one at a time, sequentially. Trying to run two projects in parallel using the same resources introduces a large amount of waste. Likewise, sharing people between teams is also wasteful.

Much like extra process, task switching breaks focus. Every time a developer, QA, IA, or designer have to switch tasks, there is a fair overhead to pay. These sorts of tasks require enough focus to warrant a significant amount of time to get into “the zone” or “flow”. Thus, switching between tasks incurs waste through time.

Similarly, being a member of two teams introduces interruptions that might be entirely unrelated to the person’s current task. This means even more task switching, and probably at an even more granular level. For this reason alone it is unwise to assign people to multiple teams.

To illustrate how much task switching can affect delivery time, list 6 peoples names. Firstly, time how long it takes to write each name one after the other. Next, time how long it takes to write a letter of each in turn until you’ve written all 6 names completely. You’ll find the latter takes significantly longer than the former. This is only the simplest form of task switching and yet the waste incurred is quite clear.

Back in 2009, Paul Graham wrote a fantastic piece on focus entitled “Maker’s schedule, Manager’s schedule”. In it he examined the difference between time schedules for developers and managers. It’s important that all managers understand that their schedule is significantly different to the developers that report to them.

A developer works in larger units of time than a manager, with much longer between breaks of focus; usually something in the region of a half-day unit. They dive into work as soon as they arrive at the office and will break focus at lunch. Likewise, when they return for lunch they’ll dive back in and only come back to the surface when it’s time to leave the office. Usually they’ll arrange their tasks in chunks that align to this and therefore will complete tasks shortly before lunch or shortly before they’re ready to leave for the evening.

Managers work in shorter units; usually around 1 hour in length. This allows time for meetings and context switches in between. As a manager it’s your job to context switch effectively, although even the greatest manager in the world will struggle to do it for too many subjects. This is why it’s important to make sure you don’t bite off more than you can chew in terms of projects to run. If you have, it’s time to start delegating.

If you’re a manager and you want effective makers, you need to arrange your days to suit them, not you. Allow 30 minutes in the morning for email checking and an added 15 minutes for the daily stand-up meeting. Don’t bother them again until directly before or after the break for lunch or at the end of the day. Trust me; they’ll cope with this much better than continuous interruptions throughout the day.

Likewise, if you’re a developer, be strict about what you do at these key times during the day. Make sure you don’t kick off development before your stand-up or you’ll fail to cope with the interruption. If people can’t agree to arrive at the office at around the same time in the morning, then you’ll be better off as a team scheduling the stand-up around lunchtime.

All this might sound obvious but you’ll be surprised how many teams let this sort of misalignment between team members screw up their daily flow. It’s also worth noting that the larger the team, the harder it is to align this stuff. There’s definitely a sweet spot to be found in team size and I personally believe it’s around 6-8 members.

Waiting

Day to day development is thick with delays. It’s a standard part of our working lives for projects to be delayed at the start, awaiting requirements or documentation; for the actual development to be blocked awaiting testing or deliverables from IA and UX; or to delay delivery through an inefficient release process.

In web development, delays can be a serious form of waste. They prevent the completion of work and they prevent the capitalisation of investment in the development itself. The speed at which we can respond to a customer-driven requirement is directly related to the number of delays in our development life-cycle.

As soon as we can accept that delays are a significant form of waste, we can identify where those delays occur and optimise them out of our processes. One of the seven principles of Lean Development is “decide as late as possible”. Taking that into account, it is better to re-prioritise tasks based on the delays you are likely to experience than suffer significant blockages in your delivery. This might mean putting tasks off until you have more information or free resource, or it might mean rearranging tasks so that developers are working on minimising those delays before delivering their original priority.

The most common delays in web development are when work passes between individuals. There’s nothing worse than finding out you’re prevented from completing a task because the wireframe or design hasn’t been completed or the requirements haven’t been finalised due to poor prioritisation. It is important that, as a team, you don’t agree to work on more than you can complete and similarly, it’s also important that you don’t agree to work on stories or tasks that have outstanding dependencies or requirements. Sometimes taking the time to groom your backlog as a team can mean the difference between wasting time on waiting or a user story or task flowing through your process painlessly.

Further to that, you can often reduce the time waiting for a deliverable by eradicating the deliverable altogether. I’ve had great success in the past by removing the need for wireframes and design visuals from the IAs and designers and getting them interacting with the developers on a regular basis. If the designers and developers maintain a clear style-guide with clear links to the front-end code—and at best a library of corresponding modules of code—they can easily relate new features to this through “pair programming” for an hour or two. Remove the requirement for a deliverable and you’re only left waiting for the developer and designer to have some time together in their calendars. This should be significantly easier to achieve since both parties are no longer required to deliver anything but their expertise during the interaction.

Finally, it’s important to remember that implementation is everything. If you can’t implement quickly following a key decision then you can’t delay that decision either. Empowering your team to deliver quickly and efficiently is your primary concern at all times, whether you are a manager or a team member.

Motion

Agile software development favours colocation for teams. Likewise, the agile manifesto states that we should value customer collaboration; involving them in all discussions and decisions throughout the development process. The point of both these principles is the reduction of “motion” within the team.

It is important to make sure that developers, QAs, IAs, designers, product owners, project managers, and any other involved parties are immediately available to each other. Reducing the amount of time it takes for specific development-related questions to be answered is vital. Having everyone next to each other reduces that overhead.

If you can’t colocate people, the next best thing is to make sure they have easy communications as soon as they need them. Get everyone hanging out in a Jabber or IRC room, and make sure that’s as much a social experience as it is a work experience or they won’t come back. It should really imitate the local working environment so is likely to involve both work-related conversations and more water-cooler-type discussions and jokes. Expect that; nurture it; it’ll bring the team closer together.

It’s also important to remember that it’s good to put a name to a face. If some of your team are remote, make sure you get them over for social occasions once in a while. It’s good to break bread with your colleagues, and it’s easier to foster feelings of camaraderie when you’ve all been out to the pub together.

Similarly to social interactions, artefact interactions can also suffer from waste in motion. Reducing the amount of time it takes to pass a particular artefact of work from team member to team member can significantly improve the efficiency of the whole team. Good examples are the hand-off of design deliverables like style guides, wireframes, and visuals. Placing these in a known shared location using source control means all interested parties can have access to them immediately. Similarly, making sure tests are written and run in the same places regardless of whether they are written by developers or QAs means that there will be a clear vision of coverage for the whole team.

Motion might seem like a fairly inconsequential form of waste with a minimum of impact, but really it combines with task switching to be quite a serious problem. A competent team will work with this at a minimum from the outset, and most teams can address this issue quickly to improve their performance.

Defects

Defects are an obvious form of waste, but really it’s only the undetected defects that cause a project to suffer. That might sound obvious at first, but it’s important to remember that the amount of waste generate by a defect is directly proportional to both the bug’s severity and the amount of time it goes undetected. A critical show-stopping defect that is identified and addressed quickly generates significantly less waste than a small insipid bug that hangs around for months undetected.

Reduce the impact of defects by finding them quickly through thorough testing practices and QA. Make sure the whole team are involved in that; quality is too important to be the responsibility of anyone except the whole team. Practices such as test-driven development and continuous integration can massively improve the time it takes to detect issues. It is important to understand what you are trying to achieve by adopting these practices and implement what you can.

Summary

Understanding waste in your development process can help you improve the efficiency of your delivery. Being able to take the principles of lean development and integrate them into your agile teams are a clear way to improve your life as a manager. In my experience it’s almost impossible to implement agile within an organisation without truly understanding the whys and hows. Further to this, it’s everyone in the team that need to buy-in and understand, not just the managers or the organisation. This rings true for Lean Development as well.

This has really been a study of the types of waste discussed in the Poppendieck’s various books on Lean Software Development. If you’re interested, I recommend you buy at least one of the various volumes. They’re absolutely worth it.

This is the first of a couple of articles I’ve started writing around agile and lean development practices and hopefully it’s at least got you thinking about how you can improve your life as a developer or manager. In the future I want to look at the complications around writing user stories, around implementing good testing practices, and how systems-thinking should be a primary goal of all teams.