Tuesday, May 7, 2013

Trust the people, not the process

In my view, one of the most misunderstood pieces of the Agile Manifesto is "People and Interactions over Processes and Tools."

Too many people believe this point is limited to one or both of the following statements:
  • The process steps in your SDLC in Agile is different from the process steps in waterfall.
  • Changing to Agile means you need to use different tools like Mingle or Rally instead of Clearcase, Traq, or MS Project.
In fact, "People and Interactions over Processes and Tools" is much more fundamental mindset change that's one of the hardest things to accomplish in an Agile transformation.

Quick quiz.  In a "traditional" waterfall environment, who is responsible for ensuring the team produces high-quality, useful software?

Is it the project manager, who's responsible for the overall project (even though they don't build anything)?  The business analyst, who puts together the requirements (but doesn't execute them)?  The developers, who build the software (but don't have a lot of say in what we're building or why)?  The testers, who test the application meets its requirements (but don't have any say in what those requirements are, and in many cases don't really understand them)?

I believe the "right" answer for waterfall projects is that it's not any person's responsibility.  Instead, it's the PROCESS that is responsible.

Here's how this usually works.  Before the project begins, a subset of the team spends a lot of time putting together a highly detailed set of requirements.  They write detailed use cases and produce high-fidelity comps.  They make all the decisions on what the moving pieces need to be, and create class diagrams and database models for everything.  They break all the development work down to highly detailed tasks.  And then those artifacts are are handed off to "the team" to execute the tasks.

How does a developer on the team know what they're doing is right?  Easy!  They read the plan.  If the plan says to build these three classes with these 12 methods, they build those classes.  Why those classes and not others?  Because the plan says to, and I trust that the process put together the "right" plan.  Is the stuff I built useful?  It must be - the plan said so!

How does a tester know we built a good user interface?  Easy!  They compare "what it does" to the list of "what it's supposed to do" in the requirements.  Can a customer actually figure out how to use the screen to accomplish a goal?  They must be - the plan said so!

The "build a plan, follow the plan" mentality actively reduces agency by individual team members.  Team members are supposed to do "their job," and if everyone does, well, I guess we'll get high-quality useful code as a byproduct.  The team needs to have faith that the people who built "the plan" knew what they were doing.   They understood all the customer needs, all the architectural foibles, all the possible edge cases, and put together a plan that covered all the contingencies (other than a few tweaks that will come in through the oh-so-friendly change control process).  Because that was their job.  Doing what they told me is mine. 

If I did my job, I'm no longer accountable.  Hey, don't blame me the system cratered - I built it just the way the architect designed it.  Don't blame me - I tested everything against the documented requirements.  Don't blame me - we built exactly what the customer told us when they signed off on the spec's.  If it doesn't work, it's not MY fault.  We all followed the process!  

In an Agile world, yes, we have different steps in our processes.  Yes, we document our requirements differently.  Yes, we are more iterative in our approach.  Yes, we use different tools.  But more importantly than ANY of those things, we stop believing that "the process" is the thing that produces positive results.

In Agile, if a developer doesn't believe that a given user story is the right way to achieve that story's stated goal, we EXPECT that developer to question it.  They need to have a conversation with the analyst who put the story together, the product owner, the customer - whoever is the right person.  We don't trust the "story generation process" produced the right story.  Instead, we trust are smart and thoughtful developer to ask reasonable questions and expect either good answers or appropriate changes.  If a tester doesn't think that the acceptance criteria documented for a story really capture what's necessary for a user to accomplish the stated goal, we don't expect them to ignore that belief and blindly trust that "the process" captured what's needed.  We expect them (we demand of them) that they express their concerns and hold others accountable for getting it right.  If the user has a problem and suggests a possible solution, we expect the business analyst to work with them to explore other solutions, and validate that their suggestion really is the best approach, rather than blindly trusting "the customer must know exactly how to solve the problem" and write down exactly what the user said.  

To move to Agile effectively, we can't just swap out a team's process with a different process, or their tools with other tools.  We have to attack the mindset that the processes and tools are the things that make us successful.  We have to attack the mindset that "doing your job" means "doing what you're told."  We have to attack the mindset that blindly "doing your job" inherently leads to success.  We have to attack the mindset that understanding what someone meant means reading a document.  We have to attack the mindset that responsibility for the project being successful resides with "someone else."  We have to attack the mindset that the thing that makes us successful in Agile is doing standup meetings and estimating in story points. 

This is a hard problem.  It's one a number of putatively "agile" teams I've worked with have not in fact solved. 

Customers and Product Owners need to expect and get used to being questioned on whether what they asked for is right.  Analysts need to get used to pushback on whether what they wrote up is the right thing.  Developers need to get used to being questioned on why they built it that way.  Testers need to get used to being expected to be expert on the business problem, and pushing back on things that (strictly speaking) aren't in the requirements.

People who "grew up" in traditional software environments are often scared of this.  How can I "do my job" when I'm no longer completely sure what "my job" means?  Won't my manager yell at me if I push back on "what the customer asked for" and "slow down" the process?  Will I be called on the carpet if I do something that's not on the script and it turns out to be wrong?

The key to all of this is establishing trust.  The team needs to feel trusted to be good at their jobs.  Trusted to solve problems creatively.  Trusted that they will make the right decisions.  Some of this trust can come from within the team.  But even more important, the team needs to feel trusted by management - that they won't be constantly second guessed.  That they have freedom to occasionally make mistakes.  That the time they spend talking through issues won't be considered "waste" time to be minimized.  That their estimates will be respected.  But most importantly, that they know what they're doing, and they're trusted with ownership of their own quality. 

A team that's trusted, and that trusts each other, will naturally build the communications links necessary to validate their assumptions.  They'll talk to each other constantly.  They'll develop "just enough" processes to ensure they all know what's going on, and that everyone is aligned.  A group of trusted people with a clear goal is the most powerful force in software. 


  1. Have you seen this? http://beyondlean.wordpress.com/2013/04/29/balance-process-and-results/

  2. Thanks for the comment, Jason.

    I don't disagree with the sentiment expressed in that article. I'm not advocating for the absence of process.

    More succinctly stated, my point is:

    In waterfall, a trusted process, followed correctly, is expected to make the team successful.

    In Agile, a trusted team, working together, is expected to find a successful process.

    In manifesto style, I'm not advocating the process has no value, but that the team, properly empowered to shape the process, is more important.

    The anti-pattern, which I've seen in multiple places, is swapping a "just do your job" approach following waterfall processes with a "just do your job" approach following Agile processes. And management is surprised by "hey, we have standup meetings and story points - why aren't we succeeding?" Failure to instill trust and ownership of the process in the team, and not the other way round, leads to cargo cult agile - all the technique, none of the values, and little of the success.

    "The question is,' said Humpty Dumpty, 'which is to be master — that's all."--Lewis Carroll