Tag: software development

The Cost of Waiting for Feedback in Software Development

[In the series ‘Explain this to me like I’m a manager’]

Let’s imagine you are paid to write emails.

The company you work for has a policy that all public emails from an executive should be written by someone with a clear head and proper training. That someone is you.

It works like this:

  1. You receive an in-person briefing from the executive of what the email is about.
  2. You go back to your office to work on the email. In case a question arises, you phone the executive and usually get an answer right away.
  3. When finished, you present your email and edit it together until you’re both satisfied with the result.

The whole process takes about half an hour up to several hours.
This seems OK. It might even be kind of fun, right?

Now, imagine a different way of working

The only difference is that you’re not sure when you’ll get feedback or approval. It might be right away, or it might take several weeks.

Since you don’t know how long it’ll take, you start working on the next task. Instead of having 1 email “in progress,” you might have 20, 30 or a whole lot more.

This the exact same job, with the same input, but it seems much more cumbersome and a lot less fun, or not?

Well, unfortunately, this is the way most developers are working

In this example, we’re assuming there is no middleman (PM, analyst…) through whom all communication passes, introducing his or her own ideas, obfuscating the message in the process.

As a developer, I have often found it very frustrating when it takes ages to get feedback or answers.

You start working on something. You get stuck, you need feedback. Because you don’t know when you’ll get answers, you put the task on hold and start working on the next one.

But wait, the client has the exact same frustration!

Yes, I’ve been there: for several years, I worked on the other side, steering contractors/freelancers to develop custom software for our organization.

It is just as annoying to give extensive feedback to a developer, only to get additional questions 2 weeks later. These questions are usually ones that could have easily been answered if asked right away, but 2 weeks later, I can hardly remember what it was all about!

Surely, there are better ways to waste my time!

What is the engineer’s solution?

Let’s use a tool to manage all this “work in progress”, with statuses, flags, due dates, source integration, custom fields, assigned persons, reports, milestones and all that jazz.

This is an often unnecessary over-complication that is regarded as standard practice.

“That’s how it’s done!”

There are indeed situations that demand this way of working.

But, most of the software projects out there are actually quite straightforward and don’t benefit from these “solutions.”

The root cause is still there: it takes too long to receive feedback. As humans, we can’t temporarily store our built-up understanding—our “grasp”—somewhere until we need it again.

So, what’s the alternative?

We are all busy and we can’t just drop what we’re working on whenever someone demands our attention. But what if we could?

What if we could work on 2-3 tasks at the same time?
What if we didn’t need tools (except for a basic to-do list) to manage the progress of all our tasks?
What if we could work intensely with the client on a few tasks from beginning to end? And not start a new task until these were actually completely finished?

Why am I asking these questions? Because it would save time, money and frustration!

This is one of the core elements of several task/project management frameworks that suggest focusing on a limited number of tasks during so-called “sprints,” or that impose a limit on the “work in progress” (a WIP limit).

Often advocated. Seldom practiced.

And really, no one is asking you to drop everything at every question; just don’t let things slide forever.

Do we really waste that much time waiting for feedback?

There was a study, conducted at Palm in California, the makers of the first PDAs. This study measured how much longer it takes a developer to fix a bug detected 3 weeks later instead of on the same day.

Now, how much longer do you think it takes?

About twenty-four times as much, no matter if the task is big or small.

Did you also fall off your chair? My initial guess was 5-10 times, which should be enough to inspire anyone to give this some additional thought.

But waiting endlessly for feedback isn’t the only cause of WIP explosion!

It’s also:

  • The delay between analysis and development
  • The delay between development and testing (if you use testers)
  • The delay between a developer reading the tester’s feedback and acting upon it
  • Waiting for approval or a sign-off on something

Now, how do we fix this?

Well, it’s not that hard… to write about! 🙂

Here are my suggestions, which should work no matter what role you’re in:

  • Try to get everyone on board. Send them the link to this article. Or send them this excerpt from The Scrum Book; the author is just a bit more known than me, so it might work better! 😉
  • Lead by example: give feedback and answer questions on the same day or at least within 24 hours.
  • Explicitly state that you try to give feedback asap. Actively request the same from everyone you work with, including clients and managers. Follow up if necessary. A lot of people let things slide until they notice you’re consistently following up.

And, if your role allows:

  • Try to get everyone (necessary/needed) working on roughly the same task(s) at the same time.
  • Try to limit the number of tasks and the “work in progress.” Impose a WIP limit.
  • Cut the number of in-between-persons. They often serve no real purpose, except to consolidate or translate information. Developers often communicate too technically. I still make this mistake myself. But I also believe you can learn to communicate on a more non-tech human level. Listen to what is really being said, and try to respond at the same level using the same words.

I think it’s always better to have direct communication lines because it improves the possibility of learning from each other.

But I’m a developer. I can’t just demand my client’s attention!

You’re not demanding immediate attention; you’re asking not to wait for ages to get a reply.

I know this is not the standard way of working for a lot of developers: being proactive instead of reactive, reaching out instead of tackling that next user story or bug. I don’t mean to be condescending. I speak from my own practice, from the mistakes I’ve made, from experiences with failed as well as successful projects.

This might be one step (of many) to becoming a “10x engineer” and to show that you’re on top of things and care about the project.

Remember, all those non-tech people you’re working with really don’t know much about software development.

There’s nothing wrong with steering and educating them on how to optimize the process and save time and money. Who could be against that? (Unless, of course, your time is billed by the hour 😉

… And what if you get feedback from multiple sources, contradicting each other, even contradicting your own opinion?

Tackling “Design by Committee” is another story. Sign up here and stay informed.


Everyone uses Java/C# and Oracle, why choose something else?

If the tech guy (or girl) advises going for a less common technology, and something goes wrong, he’ll get blamed. “Why didn’t you use Java/Oracle/IBM like everyone else does?

And you can be sure that something WILL go wrong sooner or later… and this guy knows it.

He knows he has a mortgage, kids that have to go through college, a wife with a spending habit… why take any chances? What’s in it for him if it works out? And what if it doesn’t?

People avoid taking risks. Especially people who work a ‘safe’ job as an employee.

Is this the lizard brain at work?

For the record: There’s nothing wrong with Java/C# (or any other language for that matter). They all have their (dis)advantages on speed, memory usage, developer-friendliness, license/price…

Yet NONE of them fit all tasks and situations you can throw at them, while yes: you can use most of them to implement most tasks.

Do you have any idea how many programming languages exist?

Just look at this list. These languages weren’t developed because there’s hardly any difference between them, or because a ‘one size fits all’ language exists.

Now, if you are a developer/sysadmin, I probably needn’t explain. But as I’m hoping to enlighten the non-tech manager here, I think an analogy is in order.

Let’s say you’re building a house. Would you use nails or screws for all attaching situations?

No. Depending on the kind of materials you need to bond, the needed strength, the climate it’s in, the budget, how long it should last… you either use nails, screws, glue, cement, or god knows what else.

You use the right tool for the job. It’s the same with programming languages.

A project done in Java will cost 5 times as much, take twice as long, and be harder to maintain than a project done in a scripting language such as PHP or Perl. … But the programmers and managers using Java will feel good about themselves because they are using a tool that, in theory, has a lot of power for handling problems of tremendous complexity. Just like the suburbanite who drives his SUV to the 7-11 on a paved road but feels good because in theory he could climb a 45-degree dirt slope.

Philip Greenspun, Java is the SUV of programming tools

(I just used this quote to show that there can be some bias and strong opinions concerning certain languages. I also don’t like Java, but that’s beside the point.)

Yes, you can hammer a screw into wood (I tried this as a kid), but it will take a lot of effort and will provide less bonding than a nail. And you might hit your fingers in the process (I also did this as a kid).

Still not convinced? Do you know that some programming languages can be 100 times faster than others? And there are also other differences just as relevant as execution speed.

And though execution speed can be important, it shouldn’t be your guiding factor.

Often people, especially computer engineers, focus on the machines. They think, “By doing this, the machine will run fast. By doing this, the machine will run more effectively. By doing this, the machine will something something something.” They are focusing on machines. But in fact we need to focus on humans, on how humans care about doing programming or operating the application of the machines. We are the masters. They are the slaves.

Yukihiro Matsumoto, Ruby programming language on Wikipedia

People sometimes dismiss Ruby because Ruby is slow. Well it is, and I don’t care. The biggest performance bottlenecks that I have day to day is the database, the file system, the internet and bugs. These are not problems with Ruby, so speeding it up will have a negligible effect.

Additionally, hardware is cheap and developers are not. If you can use a programming language like Ruby to make your programmers more productive, then you can increase the performance of your code with better hardware. You can buy the better hardware with the money you saved by making your developers more productive.

Graham Jenson, What is Ruby? It is fun and makes you happy!

I once was part of a team that had to make an internal web application. We didn’t have any Java experience, but the company already had some customer-facing Java web apps developed by the ‘webdev’ team. So we agreed to use Java.

After evaluating several Java frameworks, one of us bumped into this new thing called ‘Ruby on Rails’ (V0.11 at that time iirc). ‘Rails’, as it’s often called, was like a breath of fresh air after being bombarded with Java that seemed more like configuration than programming at that time.

There was only 1 problem: Rails uses Ruby instead of Java as its programming language.

Even so, we decided to go for it. We finished the project in considerably less time than the original estimate. This also helped management accept that we had neglected to use Java.

Was this all because Rails somehow made us work faster?

Not really, but it was very suited for this job, it was a really fun language (it still is), it was new and exciting, and because we were supposed to use Java, our asses were on the line in a major way.

It was the right tool for the job AND the right tool for us. By the way, last time I heard, this application was still running (I no longer work there).

And when we showed the hardcore Java aficionados, one comment really struck: “it’s easy that way.” And this wasn’t the last time I heard this (often concerning Ruby). And shouldn’t that exactly be the whole point? Making things easier?

Unfortunately, when technology is concerned, the decisions aren’t always that clear and we rely a lot on other (less?) relevant factors to guide us.

Why your devs might not like a language that makes their lives easier

When I started out as a pro programmer, the company I worked for made client software in a not-so-easy language/framework. Why? Primary reason: they wanted the software to be as fast as possible, which is very valid.

Unfortunately, the database technology we were using was so damn slow that any advantage the faster language offered was totally obliterated. This was a huge bottleneck. The language was like a Ferrari, but it was waiting 98% of the time for the database’s traffic light. And that light won’t switch to green faster just because there’s a Ferrari waiting.

(For the tech guys: we were using C++/MFC and an Access database through ODBC… ok, stop laughing, we eventually switched to faster database tech.)

It also took several minutes to compile the project, meaning that we couldn’t just make changes and test them right away. No, we had to wait several minutes to test things, or get an error (and in the old days, that could have been several hours, but only graybeards remember this).

When I suggested that another language/technology (Delphi) was really more suited for the job, colleagues argued: “there’s no demand for Delphi programmers, this isn’t good for us.

They were right. For me, this was my first job and I wanted to make a good impression. However, they had already proven themselves, knew this wasn’t going to be their last job and didn’t want to lock themselves into a framework/language that wasn’t in demand.

And I have no idea if they would have been able to find Delphi programmers.

Besides that, we could be pretty sure that Microsoft (the manufacturer of the tool we used) would still be around in a couple of years… while Borland (the original developer of Delphi) always seemed to be hovering on the edge of bankruptcy.

Would we have been more productive using Delphi? I’m pretty sure we would have. It was just much more suited for what we were doing and more forgiving for the rather average developers we were.

Would it have been good for the company? Not if the devs ran away and couldn’t be replaced by others.

Would it have been good for me? In hindsight, no. Now I can put 8 years of ‘pro’ C++ development on my CV, which is a lot more impressive compared to Delphi/Pascal. A lot of people used Pascal in high school, so it’s often not taken seriously.

So how do you pick a suitable language/framework?

You should use what startups use!

That’s what I used to tell the CxO’s. And I was wrong, as most of them didn’t work for startups.

Tech startups are the hotbed of technology. They use cutting-edge tools and make amazing software that scales to millions of users. And you will have to look hard to find startups using Java/C#.

Why?

Technology is at the core of their business, and their goal is to grow fast or fail fast – which probably isn’t yours. You are probably looking for stability, and aren’t interested in failing, no matter how slow or fast it goes.

You’re part of a startup? You should have brighter guys on board than me who will tell you what language/framework to use. And pssst: don’t waste your time reading this, you’ve got a startup to grow!

So, should you just stick with classic, proven tech then?

Hell no!

The language and framework you use can make a real difference. I think that except for certain edge cases, the human impact far outweighs the technological impact.

Have your programmers experiment with alternatives. Even if they don’t get all excited and suggest ditching your current tools, they might just find them useful somewhere. Also, it will help to fight boreout and keep things interesting.

Depending on the kind of software you make, it might be worth investigating domain specific languages (DSL).

This is something that could motivate the seniors and benefit everyone.

I have worked in EDI in the past, and DSL’s would be perfect there. Of course, I’d do it in Ruby, but I’m biased.

But isn’t it hard to learn a new language?

The basic logical building blocks of all languages are more or less the same. To compare programmers to writers again: a good programmer can tell a solid, logical story, no matter what language or framework (s)he uses.

Regular learning will also make it easier to learn new things in the future. I like to think of this as keeping your ‘learning muscle’ in shape.

And it will certainly not hinder career opportunities to experiment with other tools.
Au contraire mon frère.

Finally, it will also help you attract new talent. A business that uses Java, but with some Ruby, PHP and Node.js where it makes sense, sounds a lot sexier to the kind of people you want (well, it does to me).

Some have even suggested using technology as a hiring filter.

“The programmers you’ll be able to hire to work on a Java project won’t be as smart as the ones you could get to work on a project written in Python. And the quality of your hackers probably matters more than the language you choose. Though, frankly, the fact that good hackers prefer Python to Java should tell you something about the relative merits of those languages.”

Paul Graham, Great Hackers

But don’t take my word (or Paul’s) for all this. Your mileage will vary. There are too many different situations to provide a ready-made answer. I hope I still gave you some food for thought and provided some insights though.

If there’s any meager advice I could give it would be this:

  • Talk to your developers
  • Are they using the right tools for the job?
  • Encourage experimentation. Without experimentation we’d still be using assembler (or gasp: Cobol!).
  • Have them investigate DSL’s.
  • Remember that this isn’t purely a tech/HR matter.

And know that something will always go wrong, and it often isn’t the technologies’ fault.

Had a good read? Sign up at here to get notified of future (dis)informative articles by email.

Agree? disagree? Let me know your thoughts in the comments.

Header image by Duncan Hull, used under CC BY 2.0.