Thursday, August 28, 2014

What A Piece of Plywood Can Teach Us About Software Design

When I was in high school and college, I was heavily into technical theatre (scenery, lighting, etc).  Today, I want to talk a little an experience I had studying stage design, and somehow try to relate that back to software design, business analysis and problem solving. 

A little background on on me.  I love to build things.  I love the idea of shaping things by hand, of and building things from scratch (it's also why I enjoy cooking).  I got into theatre through carpentry.  I started off building stuff, and eventually worked my way up to designing the stuff to build (hey, software parallels already!).  I loved the idea of designing and building these artificial worlds for characters to inhabit.  So, I did a bunch of shows, and also in college I took a few electives in scenic design.

In my scenic design course, I ran across a quote that was attributed to famous scenic designer Ming Cho Lee, one of the leading voices in contemporary stage design.  I'm unfortunately unable to find the original reference or citation here, so I can't swear to the source of the quote.  But this is more a story about me, so I'm not sweating it.  What Ming Cho Lee (allegedly) said was this: "Once you know a piece of plywood measures 4 feet by 8 feet, you will never be as good a designer again."

I want to pause here and explain why the size of a piece of plywood would be important to a stage designer.  While scenery can come in all kinds of shapes and sizes, there are two major building blocks that go into most designs.  First, there are upright framed pieces called flats, which generally make up the "walls" of most sets.  Second, there are heavy duty flat pieces called platforms or risers, which are used to make raised floors or levels.  Most sets are mostly made of combinations of these two basic components.  

As you've probably guessed by now, both flats and platforms are usually constructed from plywood - in both cases, a wooden frame of some kind covered by a piece plywood (thin paintable plywood for flats, heavy duty flooring grade plywood for platforms).  And since plywood comes in 4' x 8' sheets, the most convenient size of wall flats and floor platforms to build is 4' x 8'.  In fact, many theaters keep a stock of 4x8 pieces, which can simply be reassembled and repainted into any configuration the show requires.

So, anyways, I was pretty angry about what Ming Cho Lee (allegedly) said.  As I said earlier, I came up as a carpenter before being a designer, so I knew exactly how big a piece of plywood was.  I had always considered my knowledge of how to build scenery an asset to my work as a designer - when I designed something, I knew how it would be built.  I knew I wasn't designing anything with impossible geometry.  I could draft up plans for my carpenters, anticipate their problems, and have good conversations.  I'd seen a number of student designers get into trouble from not realizing that there was no way to get out of that exit door, or that the walls couldn't come apart like they were anticipating for the final scene.  I never had those problems.

Knowing about construction (I felt strongly) made me a better designer, not a worse one.  So I interpreted Ming Cho Lee's alleged quote as something an artist who was too avant garde for their own good would say.  Something like "the true artist doesn't concern themselves with the mundane details of how the art is executed" or some such.  I saw it as impractical nonsense - hey, dude, however "pure" your art might be, we're going to build it out of plywood eventually.  Screw you, person who might be Ming Cho Lee!

I didn't really revisit my opinion of (possibly) Ming Cho Lee for a long time.  But a decade later, and after moving into a job where I help build things out of 1's and 0's instead of plywood and 2 by 4's, I think  I finally get what (possibly) Ming Cho Lee was trying to (allegedly) say.

What I think he wanted to do was to draw a distinction between the problem of design, and the problem of execution.

An empty theatre is a magical place.  It can be anything and anywhere, from a Viking long boat to the palace of a king to a shabby apartment to a merry-go-round.  Whatever fits the needs of the show we've chosen to put on and what we're trying to say by producing that show.  The designer's job is to envision a world for these characters to inhabit that looks right - right size, right feel, right flow, right colors.  This is a highly artistic endeavor.  What's right is what looks right in the designer's head (generally communicated in drawings to the rest of the team).

When it comes time to actually construct the design, we might make certain tradeoffs for practicality - if the walls in the drawings are 8'3" tall, we might ask if we're OK shrinking them down to 8'.  If the stair landing measures 3'10" x 4'2", we'll ask if we really want to construct something custom, or if the 4'x4' platform we have in stock will do.  Often, if it's not a big deal, we'll tweak the original vision slightly to make the execution easier.  Sometimes we won't - "look, I realize 8' walls are easier to build, but we really need 10' walls here to convey the grandeur of a Victorian mansion."  Sometimes this is a construction convenience, sometimes this is budget - "I'd really like 6 over 6 divided light windows, but the windows we have in stock are single pane, and we don't have the budget to buy new ones, so we'll use them."

But regardless of how much freedom (and budget and time...) we have, what we want to do is design the thing that's "right," and once we know what that is, consider making tradeoffs for convenience in execution.

But now consider what happens if our designer isn't thinking about what looks "right," and instead tries to think about the execution first - the designer who's thinking about the plywood and not the design.  Rather than designing a properly proportioned space with the right "feel," they'll start lining up 4x8 flats across the space until it's "close enough" to the size we need.  Rather than thinking about what the right proportions are for that balcony hanging over the garden, they'll start by assuming we'll take a 4x8 platform we have in stock and putting it up on legs.  They'll never consider a wall that's taller (or shorter) than 8 feet, no matter what the show is.  They'll never consider a transition into the sunken floor of the living room that's not a straight line across (and a multiple of 4' long).

And this is where they've stopped designing.

It's no longer the case that we're adapting a vision to fit practical reality.  It's that we've never worried about a vision at all.  We've never considered any elements that weren't easy, or weren't "standard" sized.  We'll wind up with a set that's probably functional, and certainly easy (and cheap) to build.  But we've never really considered whether it was "right" for show.  We've abdicated being an artist to instead be a draftsman.  And the art (by necessity) suffers.

OK, so why am I talking about this in a blog that (allegedly) about software in general, and business analysis in particular?  Because I think the same logical trap of the set designer stacking up 4x8 blocks applies to someone designing a piece of software.


The heart and soul of business analysis (to me, anyways) is to understand our users goals, objectives, and desired outcomes, and understanding how we'd know when those goals were achieved.  It's about the "why" and the "what," not the "how."  Eventually, of course, we'll need to work with the rest of our team to define the "how," and the "how" is what we'll actually implement.  But thinking about the "how" too soon can turn your brain off - you stop listening for the goal, and simply start "stacking up" the tasks we need to do to make whatever they asked for "fit" in our current solution.

For example, let's say we're working for an e-commerce company.  They've done all their business to date on a "sales" model, but now they also want to add "rentals."  As we're working with the customer to understand how rentals work, we think to ourselves "OK, we already have an "Order Type" table - we can just add another Order Type called "rental" and it will flag all the rentals.  OK, customer is talking about returns now - we'll just add another "Return_Type" for rental return.  Easy peasy.

Except that we've really stopped listening halfway through the conversation.  We envisioned an easy solution, and started filtering everything the customer said through that lens.  We stopped thinking about what was different for rentals, and started thinking about how we'd make them fit our existing paradigm.  Which might be disastrous.  Because there might be a whole host of things that might be critical to launching a rental business other than just having another order type.

Is rental a separate kind of inventory?  Do we need to asset tag them separately from "purchase" inventory?  Don't we need a way to track which item is where?  What happens if we get something back "damaged."  When do we allow someone else to rent something that's currently rented to someone else but is expected back soon?  Just having two more order types ignores a whole host of areas we might want (and need) to explore.

We need to understand the problem first, and only when we've got that down should we worry about what our options are for implementation, and which options are easier.  The approach that's simplest in our architecture is OFTEN the best approach.  But if we assume an approach too early, we miss the opportunity to discover other options that may be superior.

Listen to your customers, understand them well, and then be the best designer you can be. 


Thursday, June 26, 2014

I Love Pecha Kucha

Hi, folks,

I want to talk this week about my favorite technical/non-technical presentation format - PechaKucha™ **

This week, I want to talk about what Pecha Kucha is, why I like it so much, how to use the format effectively, and close with a Pecha Kucha I gave on how to do a good Pecha Kucha.

What's a Pecha Kucha, anyways?

Simply put, a Pecha Kucha is a type of lightning talk with a very specific format.  I'm using "lightning talk" in the general sense to refer to "any short presentation" here.  All Pecha Kucha's are lightning talks, but not all lightning talks are Pecha Kucha's.

A Pecha Kucha is sometimes referred to as a 20x20, because the specific restriction of the format is "Exactly 20 slides, with each slide timed to auto-advance to the next slide after exactly 20 second."  This means a Pecha Kucha is exactly 6 minutes and 40 seconds long.  By design, the presenter has no active control during the presentation - the slides neither wait for the presenter to be finished nor advance whenever the presenter is ready.  It's up to the presenter to find the right rhythm.

Where did Pecha Kucha come from?

Pecha Kucha was invented in 2003 by Astrid Klein and Mark Dytham, two architects who lived in Tokyo and also owned a bar/experimental space called SuperDeluxe, as a way to get people excited to come to their venue.  They were inspired by a simple idea - architects (like most people) tend to talk too much.  60 minutes of PowerPoint is a lot to listen to.  So, they devise a format to force presenters to boil their presentations down to their essence.

And the first PechaKucha™night was born. 

There are now regular open Pecha Kucha nights in (as of this writing) 700 cities world wide, with people speaking on all kinds of topics.  You can learn more about them at pechakucha.org.  I highly recommend checking out their "Presentation of the Day" archive for some truly mind-blowing stuff. 

What's so cool about Pecha Kucha's?

The big thing I love about the format is that technologists suffer from the same problem as architects - they talk too much, and if you ask the average technologist to write a presentation, it's likely to be long and dry. 

But we're also a learning culture - the state of the art in technology changes rapidly, and it's hard to keep up.  Learning what our peers are up to, what they're experimenting with, what's worked for them -- all of this is vital to keeping on the cutting edge.

Pecha Kucha, with it's short, focused format that forces presenters to get to the point, is ideal for technologists to give each other a peak at what's new in the world, without taking too long or getting into too much detail.  It's just enough content for me to know if I'm interested in learning more about a topic, and get me excited about new ideas.  

It's also an accessible presentation format for people who haven't necessarily had experience giving public presentations before.  The format does have some challenges, but 6 minutes and 40 seconds of engaging content is considerably easier to generate then 60 minutes of content.  (At ThoughtWorks internal training program for new college grads, we require every participant to write and present a Pecha Kucha).  

I host a semi-regular Pecha Kucha night at ThoughtWorks' New York office, where our consultants (and some special guests) get the opportunity to let the rest of ThoughtWorks know what they're working on, looking into, or doing in their spare time to make the world a better place.

Sounds interesting - how do I get started?

Glad you asked!  I actually put together a Pecha Kucha on why you should do a Pecha Kucha, that also contains how to put together a good Pecha Kucha, that was presented at one of our ThoughtWorks Pecha Kucha nights! 




** PechaKucha™ is trademarked to protect the term and the network of open PechaKucha nights around the world. You can read more about the trademark at www.pechakucha.org/faq I very much support their right to trademark their work. However, to avoid crazy overuse of the ™ symbol all over this post, I will be omitting the trademark symbol when I'm talking about the concept (as opposed to the organization).

Thursday, June 12, 2014

Software Doesn't Have Requirements

I'm back!  Today I'm going to talk about one of the most pernicious myths in software development today.  It's this: Software has Requirements.

This may seem like a surprising thing to take issue with.  People have been talking about "software requirements" since, well, since there's been software.  It's the industry standard term to describe "what we're going to build."  Even leaders in the Agile space talk about software having requirements.  

So why am I picking on the word "requirements"?  Because it's the wrong word.  But more than that, it's a dangerous word.  Thinking about software as having "requirements" affects how we think about our development process in many suboptimal ways.

Software (by and large) doesn't have requirements!

In this article, I want to convince you why "requirements" is the wrong word, how thinking about "requirements" holds back our thinking and collaboration, and propose a better way to think and talk about what we're trying to build. 

Why software doesn't have requirements

I'll admit this isn't a completely new idea.  Among others, Kent Beck wrote about this back in 2004, and Jeff Patton wrote a very influential article on the subject back in 2007.

What do I mean when I say software doesn't have requirements?  I mean that very little of what goes under the heading of "requirements" is really "required" for us to build successful software.

If software has requirements, the first question to ask is "required by what (or whom)?"  And "required for what?"  Something that absolutely must be included for our product to have any hope to succeed in the market?  Something that's the only way to solve the business problem we're trying to tackle?  Something that we're certain we can't omit without risking failure?  How would we even know these things (if they exist)?

What we're really talking about when we talk about requirements are DECISIONS made by potentially fallable humans.  A trusted human, perhaps.  One who has a good understanding of the problem, hopefully.  But it's someone's best understanding.  Not some great truth about the market for software that's been hiding out there waiting for us to discover it.  They're not something that's "required" for success.  They're our BEST UNDERSTANDING of what successful software MIGHT look like.

Why am I belaboring the point here?  Because what we call requirements...aren't.   There might be other things we could do that could succeed.  There might be things we think are required that it turns out we could have done without.  They're not objective fact.  They're subjective opinions.

The closest most projects come to having true "must have" requirements are actually the much maligned, little understood non-functional requirements.  If we're rolling out an app to 10,000 people, it's pretty much required that it can support 10,000 users.  If we're building an app to integrate our HR and accounting data, it's got to be interoperable with our HR and accounting systems.  If we're building software for medical records, it needs to comply (in the US) with HIPAA rules.  This isn't hard-and-fast--many non-functionals requirements are still more of decisions.  But to the extent true "requirements" exist, they're more likely to be on the non-functional list. 

Why believing in requirements is dangerous

Regardless of what we call them, in any software project there's going to be a set of stuff that we build.  So why do I care so much what we call them?

Because labeling the "stuff we're building" as requirements is actively dangerous to our thinking. 

First, belief that software HAS objective requirements implies we can (and should) discover those requirements.  The requirements exist independent of the team.  Belief in "requirements" is belief in a right answer.  There's a correct "set of stuff" out there.  We just need to figure out what it is.  This implies investing (potentially considerable) time trying to determine and build a list of requirements.

And once we do, we should think the list is largely static, since everything on the list is "required."  After all, if we can decide not to do it, it must not have been "required" in the first place!  Yes, in Agile projects we manage our backlog, and re-prioritize frequently.  We can discover new "requirements" over time.  But (in my experience) we rarely REMOVE items from the backlog.

Thinking about "requirements" drives a wedge between people who ought to be collaborating.  It sets up a distinction between "the smart people who understand what's required" and "the people who implement those requirements."  It's not the team's job to think about what we should be building.  We're just building what's "required."  Requirements, by definition, are non-negotiable. 

Belief in requirements inverts our thinking.  The most important piece of building software is determining what we want to build and why.  But if the "what" is a non-negotiable list, and the "why" is "because it's required," we're telling the team to only focus on the "how."  But the "how" is the least important piece - we can do the wrong thing as well as humanly possible, but it's still the wrong thing. 

In practice, we know this is the wrong way to think about software.  There are usually multiple ways to solve a problem.  There are many different approaches that could potentially succeed in the market.  There's not a "right" answer - there are MANY right answers.

As Fred Brooks wrote in The Mythical Man Month, "The hardest single part of building a software system is deciding precisely what to build."  (quote stolen wholesale from Patton's article)  We need to keep our best efforts focused on solving that problem, not assuming the answer and focusing on less important matters. 

Rather than try to figure out in advance what will work, it's usually a better idea to TRY things and see what works.  This principle underlies the Lean Startup movement, A/B testing, and other experiment-driven approaches.  Don't try to figure out what OUGHT to work.  Don't rely on an expert to DECIDE what should work in advance.  Try something, see what works, and adjust.

Hey, isn't experimentation a way to "discover" requirements?  Sort of.  But only retrospectively.  Believing that we can have "requirements" for software is the belief that there's something knowable IN ADVANCE that tells us where to go.  Experimentation turns this on its head - explore many possibilities, and the ones that work were the right ones.  Even here, it's not clear that the working approaches were "required" - there could be other things that would have worked we didn't try.

Problems, Ideas, Decisions, Designs

This would be a pretty boring article if all I was doing was complaining about a word without offering any constructive ideas on how to think about "the stuff we build" differently.  Here's one analyst's thoughts on a better way to think about it.

In my view, there are four major components of the "what do we build?" problem.

PROBLEMS

Problems are our current perception of issues that people have that need to be addressed.  They might be based on things someone told us, things we observed, or just things we think could be better.  No matter where they come from, there's some set of "problems" we think exist for some set of people in the world that we could potentially try to address with our software.


There's no guarantee our list of problems is CORRECT.  We might think something's a problem that isn't actually something that's important.  We might not understand a problem that's really important.  Our list of "problems" is our BEST UNDERSTANDING of what is out there to potentially be solved.

If we're thinking about the online banking space, here are some examples of problems we might perceive:
  • When I want to buy something, I don't always know how much money I have.
  • Dealing with paper checks is a hassle.
  • I can't always find an in-network ATM when I want cash. 
  • It's inconvenient to pay monthly bills one at a time on different sites. 
We don't have to solve all of these problems together.  We might decide not to solve some of them at all.  But the list of problems is a good start for the universe of "what issues we MIGHT choose to address."

Problems, by definition, exist independent of any particular solution.  Some problems might have a single solution.  Some might have many.  Some might be insoluble.  When we're identifying problems, we shouldn't care. 

A key thing to avoid when identifying problems is to avoid the self-justifying solution.  The absence of a particular solution is never ipso facto a problem.  "I don't have a mobile application that guides me to the closest ATM for my bank." isn't a good problem statement, for two reasons.  First, it pre-supposes one (and only one) solution (a mobile app).  Second, it doesn't tell us anything about WHY we want that solution - what's the issue that causes me to want an ATM finder app? 

IDEAS

Smart, creative people can formulate a number of ways of ways that we might completely or partially address some of the problems we've identified.  That set of possible solutions are our ideas of things we can potentially do.

There's no expectation that we have a single good idea on how to solve every problem we've identified.  Sometimes we might have no ideas.  For some problems, we might have multiple ideas.  We might have ideas that completely solve a problem, or only partially address it.  Our ideas might be contradictory.  This is OK.

Here are some ideas we might have on how to deal with "I can't always find an in-network ATM when I want cash":
  • Build a mobile app that uses geolocation and a map of known ATM locations to guide me to the closest ATM.
  • Partner with Google Maps to have an option to show our ATM locations as a native overlay.
  • Waive our fees from using out-of-network ATM's and reimburse fees so our customers can use any ATM without penalty.
  • Build a NFC-based mobile wallet application linked to a bank account so I don't need cash so often.
  • Deploy iBeacons on all our ATM's so they're easier to find.  
  • Partner with a well-known company with a large footprint (e.g. McDonald's) to have an ATM in every one of their stores, to increase our footprint and increase our visibility.
  • Build mini-ATM's and install one in the home of every customer who requests one.  
Not all our ideas necessarily need to involve software.  Some of our ideas might not be great in combination (would we partner with Google Maps and ALSO build our own native app?)  Some of our ideas might be ridiculously infeasible (mini-ATM's in everyone's home) or cost-ineffective (waiving all our ATM fees).

Our ideas represent the universe of things we think might be worth doing.

DECISIONS

Obviously, we can't implement every idea.  If we're thinking creatively, we will almost certainly have more ideas that we're capable of implementing.  Some problems are more important than others.  Some ideas are better than others.  Sometimes we need to make a choice between several plausible ideas to address a problem.  These choices are our decisions.

Decisions represent our choices of which of our existing ideas we think are worth implementing and want to implement first.

We don't have to exhaustively decide on every idea we want to implement before we start.  We don't necessarily have to rank every idea we want to do.  It could be enough to start with a single idea to implement (lean-ists would probably recommend this approach).  We don't have to make all our decisions up front (and we probably shouldn't).

Deciding to implement an idea isn't an irrevocable decision.  If we're following an experimental approach, we might determine that an idea we thought was good didn't work out.  That's OK - our decisions can be reconsidered.  

A prioritized backlog of "As X, I want Y, so that Z" user stories can be thought of as the output of our decisions (the ordering of the stories) about our chosen ideas (the "I want" clause of the stories) to address important problems (the "as a" and "so that" clauses).  That's not to say that a good backlog "just implements" my model - while you can produce a backlog this way, the backlog (being the decision output) hides the universe of problems and ideas we did NOT choose to pursure. 

Our decisions represent our evaluation of  which ideas we think are the best way to address our known problems. 

DESIGNS

Once we've decided a given idea is worth implementing, we have to figure out ho we're going about it.  These decisions include the fine details of exactly what our idea means, how we'll determine we're successful, the technical design we plan to use, what our solution will look like visually, and how it will fit in with everything else we've built or are planning to build.  Those decisions encompass our design.

Our design can (and should) be informed by what we know or have learned about the problems we're solving, what other ideas have and haven't worked, and potentially some quick lo-fi experimentation we might chose to do as part of designing our solution.

If we're building software, our design encompasses all the things we need to do to translate our idea into high-quality, well-tested code that addresses our chosen problem.   The output of our designs is working software.

By point of comparison, the "designs" piece is the ONLY piece the team really owns in "requirements-based" thinking.  All the team does is implement other people's choices.  The determination of the problems, generation of ideas, and decisions around which ideas have merit are all part of the "requirements" that come to the team from the outside.

That's not to say designs aren't important - they're the only piece that directly results in working software.  But a good design is only valuable if it represents a good idea on how to solve an important problem.  Keeping the team isolated from the problem determination, idea generation, and decision making makes it very difficult to feel invested in the business problem.  And it keeps your skilled, experienced team at arms length from contributing to those important pieces of the process.

Why change?  

Why go through all the trouble to replace a common, reasonably well understood term?  And why replace it with four concepts that only replace it when taken as a whole? 

The reason I'm advocating changing the terminology is primarily to change our thinking about the most important process in software.  It doesn't matter how well we build the wrong thing.

The thing I like about my terminology is that every one refers to something we shouldn't expect to be static.  We can always identify a new problem.  We can always come up with new ideas.  We can always revisit decisions.  We can always redesign something.

All of these words imply things we should expect to change.  They're all inherently negotiable concepts.  None of them imply a correct answer, or an expectation that a single correct answer exists.  In short, they describe the software development world as we find it - dynamic, indeterminate, complex, and full of valuable, important problems that deserve to be solved.

Thanks to many of my colleagues at ThoughtWorks who gave feedback on a version of these thoughts presented at our North American AwayDay 2014.