Interview with Hiten Shah Founder of Product Habits On Why Popular Product Management Methodologies Provide Inaccurately Estimate User Stories

What’s probably the MOST popular and frustrating question you’ve come across when building a product: “How long do you think it will take to do task X?” People ask you this because they want you to accurately estimate user stories.

But it’s frustrating on so many levels…

First, if our “accurate” estimate is off, there goes our ship date!

Next, we need to give a response that seems “realistic”, i.e. is going to meet the expectations or deadlines set by someone else.

Third, we need to be a fortune teller and anticipate things that come up in the course of completing task X.

Finally, we have to do it the moment we’re asked because we’re expected to know how long any task will take.

I don’t know about you, but despite building and launching a number of software products over the past 14 years, I still struggle with estimating how long a task will take to complete.

There are a number of approaches and methodologies that have sprung up over the years such as Waterfall, Agile and Lean whose goal is to provide a framework that helps engineers, designers, and product managers to estimate user stories: how long something will take to build and ship. However, as you’ve probably experienced, each one of these misses the mark.

In today’s episode we’ll dive into the aftershocks you may experience when it comes to following one of these approaches and providing product estimates.

Next week we’ll tackle an alternate approach that may seem too good to be true…

To help us out, I’ve invited Hiten Shah, who is the founder of a number of software products such as Crazy Egg, Kissmetrics, and his most recent project is called Product Habits.

As you watch the episode you’ll learn the following:

• Why we suck at estimating even if we’ve been doing it for a while
• Why we’re surprised each time our product estimates miss the mark
• What happens if we decide to “pad” our estimates
• What happens when we get rid of estimating altogether
• Why a task we think a task is 80% complete but really it’s more like 50% complete

Have you tried one of these approaches? Let us know in the comments below what you experienced when you did.

Listen to the episode on iTunes!

You can listen to this episode of Build on iTunes.

Build is produced as a partnership between Femgineer and Pivotal Tracker. San Francisco video production by StartMotionMEDIA.

Check out these additional resources on estimating stories for your product:

• Product Habits
• Why It’s Easier For Product Teams To Cram Features & Bugs Into Each Release Instead of Cutting Back
• Tech Debt: The Cost of Putting “Quick Fixes” Into Our Software Product
• Product Debt: What Is Product Debt And Why You Need To Prioritize Paying It Down
• How Long Does It Take To Get A Startup Company Off The Ground
• How Much Time And Money It’s Going To Take To Productize Your Idea
• My Nightmares With Engineering Estimates
• Story Points, T-shirt Sizing and Time Buckets: How Tech Companies Do Engineering Estimates
• Deadline That Are Doomed From The Beginning
• No One Likes A Creep

Episode Transcript

Poornima Vijayashanker:        Welcome to *Build*, brought to you by Pivotal Tracker. I’m your host, Poornima Vijayashanker. In each episode of *Build*, innovators and I debunk a number of myths and misconceptions related to building products, companies, and your career in tech. One of the most elusive processes has to be estimating how long a project or a task is going to take. And no matter how many times we do it, we somehow just always miss the mark because things come up. Well, in today’s episode, we’re going to share some of the aftershocks you may experience despite what approach you take when it comes to estimating. In a future episode, we’ll dive into an alternate approach in how your team can adopt and adapt it to fit your needs.

To help us out in this episode, I’ve invited Hiten Shah, who is a founder of many products. His most recent project is called Product Habits. Thanks for joining us today, Hiten.

Hiten Shah:           Thanks for having me. Happy to be here.

What Happens When We Don’t Accurately Estimate User Stories Or Tasks

Poornima Vijayashanker:        Yeah. You and I have been building products for a number of years. We’ve built a lot of different ones. And I know myself being an engineer and a product owner, no matter how many years I put in, I just constantly miss the mark no matter what. I either end up underestimating or overestimating. Let’s dive right in and talk about what are some of the problems that happen when we don’t accurately estimate user stories.

Hiten Shah:           Yeah. I think, when we’re building products, we don’t generally think about how long it’s going to take to build them even though we pretend we do. We end up creating a road map with a bunch of timelines and we don’t actually talk amongst the team, because, usually, a product can’t be built by a single person.

If it could have been built by a single person, then you don’t have to worry about it as much because that single person has all the answers. One thing that ends up happening is you end up having surprises that come up that you didn’t think of. You’re thinking about using a certain technology, let’s say such as Twilio or SendGrid or an API and the engineer has never used it before. They get in the weeds of it and they realize it’s going to take longer than they think. And then your estimate is blown up and you’re off. That’s a very common problem.

What Causes Scope Creep In Product Management

There’s a few others, too. One other one that I’ve seen people hit continuously is this idea of scope creep. We’re both product people. You happen to be an engineer. I happen to be more a marketer. But we love product and building them and teaching other people how to do it, too.

One of the aspects of that is we might still be learning and doing research as the product is being built, and we all of a sudden have this great idea we want to add. We go in and kind of blow up the whole process and expect that the timeline is not going to change or don’t even think about the timeline and say, “Hey, we’re going to build this new thing on top of the thing we’re doing.” Or, “Can you add this little tweak?” And not realizing how disruptive that is to the process of building the product itself.

What Happens When Communication On A Modern Software Team Breaks Down

Another thing that’s very common is that if you aren’t communicating very well with your team, especially the engineers when you decide that you’re going to build something, what ends up happening is the best thing that they can do if you haven’t spent enough time communicating early and often is they end up padding. They end up actually adding a whole day—or worse yet—a week or months to a rough estimate. We call it a rough estimate because it’s rough.

Those are some of the more kind of common problems that come up when you’re building something and trying to get estimates and actually think you have the right great estimates, which is the most common thing. And then all of a sudden all these things happen that you are probably are not conscious to about what kind of problems they cause in terms of being able to ship something on time.

Poornima Vijayashanker:        Right. I think two other things I’m curious to hear your take on are large tasks. How do you actually divide them up and then the follow through? You think you’re 80% way there and then you discover actually you just finished 50%.

What Causes Scope Creep? Large Tasks That Aren’t Broken Down.

Hiten Shah:           Yeah. I see companies creating, even in my own companies, so much work that is not actually broken down enough. You might think that it’s easy to add a button somewhere, so you say, “OK. This is a button.” You think it’s a small task. Maybe you’re not the engineer, because often times you’re the product person, or even if you are the engineer. Then the engineer, whether you decided to do it as an engineer or the product person, you get into the task and all of a sudden you’re like, “I have to add the button.”

There’s all these other things that need to change whether it’s the user interface, or you’re missing a certain component, or what that button does is more complicated than you thought. What ends up happening is this seemingly small task is actually a large task. It’s usually because you haven’t thought through all the things that you need to think through when you add something like that. And I’m talking about a button. Imagine a whole feature. Right? And considering that to be a small task when it really turns into a large one. I think the most common thing I see is that these things you consider small are actually large.

What Causes Scope Creep? Thinking That Tasks Are ‘Simple’ Or ‘Small’

Another common thing is not realizing that what you’re asking for is actually a large task. Right. Like adding a messaging feature or things like that. Even a lot of times, I’ve had emails come in to me from people who are customers saying, “You can send me SMS as notifications. It will take one hour using Twilio, another two hours through your database and you’re done.”

Poornima Vijayashanker:        Great. Come on down and write it for us.

Hiten Shah:           You want to do it for me? I’ll hold you to the three hours, because it’s never as long as people say it is.

Poornima Vijayashanker:        Yeah. There’s a lot of different approaches we also like to take when it does come to estimating. Even though we know all these problems, we still continue to take an approach. Right? Let’s dive into the approaches.

Well-Intentioned Product Methodologies: Waterfall, Agile, and Lean Startup

Hiten Shah:           Yeah. I think one of the most common approaches that is very still operated by in a lot companies is what we call waterfall, which is one task happens after another task, after another task. A lot of times people are waiting on these things. This is actually the reason another process was invented called agile, which I know both of us are familiar with. Where you’re essentially—the way I would describe it is you’re trying to do things much more efficiently by basically having more regular meetings and having, I guess, smaller batches of work.

What ends up happening there is you create this sort of system that works almost on a weekly basis at best. It means that you have a cadence of following up on all the tasks, we call it agile because you’re supposed to be more agile with it and it’s more nimble than waterfall and that’s completely true, but you lose a lot in the process. Those are the two common ways that I’m most familiar with.

Then there’s a third way, which I think is more inspired by things like *Lean Startup*, which again I like to say that we both probably grew up with that so to speak, around us. That’s where you are even more hyper—I would call that more hyper agile than anything else, where you’re adding in the component of much more customer feedback in the process, because agile wasn’t necessarily invented at a time when customer feedback was a popular thing.

Poornima Vijayashanker:        Right. What are some of the painful after effects of using agile?

How To Estimate User Stories In Agile

Hiten Shah:           What ends up happening with agile—one of my favorites, and there’s a lot of tools out there that facilitate this—is the idea of adding points to an agile process. What you end up doing is you’re officiating time. You’re saying that something that would take one to three hours is like one point. Or something that would take three to six hours is two points or three points. And they have all these things like Fibonacci sequences. There’s a lot of fanciness around points, when you’re really trying to understand time not points. The reason that points exist is because—and not that I think this is necessarily bad, it’s better than other methods—but some engineers decided that they wanted a metric that wasn’t time.

Poornima Vijayashanker:        Right.

Hiten Shah:           They literally said, “It can’t be time, because we cannot estimate user stories.”

Why Software Engineers Created A Complex Measurement For Time

Poornima Vijayashanker:        Because they basically didn’t want somebody to know that something was going to take only 15 minutes or five hours.

Hiten Shah:           Exactly. Your minimum there is an hour at best. Often times it’s much more. The thing is every team that I’ve seen implement agile with points in different ways. Their whole buckets around the number of points something takes is all different. What ends up happening is that engineers now feel great because they have a velocity score. And they can talk about how many more points they’re doing every week or how many points they’re doing every week as a total. And that’s really hiding what I would call the truth, which is how long did something actually take.

Poornima Vijayashanker:        Yeah. I know one of the alternatives is to just get rid of estimates all together, but you’ve probably experienced what that’s like. Talk about what results when you get rid of estimates.

What Happens When You Get Rid of Providing An Estimate For User Stories

Hiten Shah:           Sure. If you get tired of agile for whatever reason—and usually this happens because somebody that’s a non-engineer is really not into it, because they don’t understand what a point is even though it has timings and stuff. Then estimates are completely removed, which means points are removed and then there’s just tasks. Then you run into some of the problems from earlier.

Is it a small task? Is it a big task ? What it really boils down to…if you have no ability to understand how long something’s going to take or even how many points, let’s say, then you end up not knowing how to prioritize what you work on. If there’s just 10 tasks and all of a sudden the engineer’s working on one and then you ask the engineer, “Oh. How are you doing?” They’re like, “Oh, it’s going to take another week.” Well, if I had known that task you took was going to take another week or two weeks or whatever, I probably would have told you to work on something different because our customers are waiting for things. Right.

And that tends to be one of the bigger problems that happens, which is this communication breakdown because nothing is estimated, whether it’s points or hours or whatever way people want to do it. And you end up having a massive communication issue and then you have these fiefdoms that get created. You have engineering not against, but against product, against sales, against marketing. And everyone’s just waiting for product, everyone’s just waiting for things to ship so you can make customers happy.

What Happens When You Pad Your Estimates

Poornima Vijayashanker:        Right. One of the alternatives to both these is to create a buffer. Let’s pad the system so that as an engineer I don’t look bad and as a product person you feel like, oh, OK, there’s some wiggle room. But we know that that has its shortfalls. Let’s talk about those.

Hiten Shah:           Yeah. Of course. I think padding is probably one of the worst practices, and you might as well have no estimate user stories or no points or anything like that, because you’re essentially saying that whatever estimate I give you, I don’t know. I don’t know. I’m going to pad it.

Then you get in this padding mentality and then you still end up with the same problem that you actually didn’t have a real estimate, and things are such a moving target that you failed to ship, you failed to actually do proper planning. You end up having a business where you’re actually not getting the product you need in your customers’ hands fast enough so you can actually grow the business. I think padding leads to a whole different set of issues, because what ends up happening in the worst way I can say to you—and I’ve already said it in pretty aggressive ways—is that everyone’s lying to everyone else. That doesn’t help with prioritization or getting anything done either.

Poornima Vijayashanker:        Well, thank you for taking the time today to share the shortfalls of these three approaches. I can’t wait til next time when you unveil your approach for estimates.

Hiten Shah:           Yeah. Can’t help but share solutions with problems.

Poornima Vijayashanker:        Thank you.

How Do You Estimate User Stories And Tasks?

Now, Hiten and I want to know, have you tried one of these three approaches? And how have they fell short for you? Let us know in the comments below. And that’s it for this week’s episode. Be sure to subscribe to our YouTube channel to receive the next episode where Hiten is going to dive into his approach for doing estimates. Ciao for now.

This episode of *Build* is brought to you by our sponsor, Pivotal Tracker.