Agile Engineering is Useless...

...if you're programming without having user tested your ideas

Posted by Jim Morris on October 7th, 2015

Agile Engineering is useless because you're sending ideas to the engineers without having user tested them.

You have the best engineers that recruiter’s fees can find and you’re wasting their time. Untested ideas are the business equivalent of guessing. You probably don’t let your finance team GUESS at the numbers when you’re closing the books. Why would you let your Product Managers GUESS at ideas and occupy precious engineering time.

I know why you are sending untested ideas to engineers. These are the common barriers to user testing.

You’re too busy

MultiTasking "The art of doing twice as much as you should half as well as you could." -

You’re a Product Manager and your boss told you that you’d be the General Manager of your product line. Sounds great until you realize that you’re actually responsible for everything in the software development lifecycle. You run the agile team. You write the documentation and release notes. You make your own wireframes. You try to do your own user testing. You do the marketing and social media outreach. All while keeping up with current trends in an amazingly fast moving environment.

Soon enough, you’ll be sucked into every part of the business as the subject matter expert, sales engineer, product marketer, customer support agent (gasp!) and slowly you’ll stop creating new and interesting ideas. You’ll be overworked so your creativity will plummet. You get too close to deadlines and do the bare minimum to get through meetings with the Executives by making elaborate mockups and presenting confidently. Bad idea. You should push back. Eventually the metrics will bear out that your ideas were just GUESSES. Mediocrity is inevitable.

DON’T have right kind of Designer on staff

Limitations "Until you spread your wings, you'll have no idea how far you can walk." -

Most people educated and employed in the field of Design are Visual Designers and Graphic Artists. These folks ruled the early days of internet development because back then we were all transitioning from traditional media where these designers were in high demand. Marketers, MBAs and Executives are comfortable with these types of designers since they are still in the PUSH mentality of business where branding, beauty and boldness are more important than usability and simplicity. Don Draper wins every time when we mistake Advertising and Branding for User Experience. Television and Radio are the land of Advertising and Branding. Websites and Apps are the land of User Experience. Don’t confuse them.

Every tech business needs a User Experience Designer (or UX Designer for short). Other folks call it a User Interaction Designer (UI Designer) or Director of Usability. If you can have only one designer on staff, choose the UX/UI designer. You can always outsource the Visual Design at the end of the project when you know your User Experience is awesome. One way to justify this choice is that modern designs for websites and apps are likely simpler designs in the age of Mobile First.

Wrong team in place

bad news bears Bad News Bears (1976) -

Simply put. You likely DON’T have a qualified UX Designer on staff. Stop here. Find one. That's your top priority.

If you have a qualified UX Designer, then read on.

Do you have all FOUR required members of the Core Product Team to cover the necessary areas of Product Management? The UX designer owns the user. The Product Manager owns the holistic product especially the product-market fit where the product works for users and hits the desired metrics of the company (such as revenue or contracts signed). The Lead Engineer owns the feasibility of the product (Can it be built?). The Data Analyst owns the Key Performance Indicators (KPIs) that hold the team accountable for results and hold the Executives accountable for not micro-managing.

You’re lucky and smart like Steve Jobs and Elon Musk

Smart Luck

In Silicon Valley, the stories of Steve Jobs (Pixar, Apple) and Elon Musk (PayPal, Tesla, SpaceX, Solar City) are legendary. You are not them so how can you succeed. It's all about raising the probabilities of success and having less reliance on luck and guessing. You can raise the probabilities of success by meeting users and investing in user testing. Get that probability higher by eliminating bad ideas and nurturing ideas with potential. Expand and tweak the ideas that your target users love not necessarily the ones that your CEO/Founder/stakeholder loves. Without testing, you might eventually deliver the right product but that's right up there with the concept that thousands of monkeys banging on typewriters will eventually deliver a Shakespeare quality play.

Unfortunately, many product teams are dominated or even run by the CEO or Founder or stakeholder. In practice, this is how most companies are STARTED. Too often, ideas go straight from the CEO to the engineer without any user validation. Venture Capital funding should not be seen as the validation of an idea. That money should be used to EVOLVE that initial idea so that it works with the market and users to make a great product. This is the wise premise of the The Lean Startup.

You have as many engineers as Google does

Google Wave

Most companies don’t have hundreds of engineers. Google can launch 50 products and be successful if just a handful win. Your hit rate must be higher. You probably need to have 1 winner for every 2 or 3 products that you launch. You can raise that probability of success through user testing. Google does user test their products but they have a much WIDER margin of error than all other companies.

Never tested ideas before and don’t know how (no culture of testing)

Maybe you have an engineering focused culture. Maybe you’ve never tested before. There’s no time like the present to start. Pick up a copy of Inspired by Marty Cagan or go to one of his public workshops. In the meantime, sign up for an account on and start running people through your existing website or app. Ask them to compare and contrast your site with your top competitors. If you’re a self-improver, you’ll find plenty of things to improve via these simple tests.

Before you send those new mockups to your engineers, run some comparison tests with your current website or app using the site You’ll appear more confident in front of others when you tell them that you worked with target users to choose the company's next user interface through repeated testing rather than the usual technique of “Just trust me”.

Users don’t know what they want

Workd is Flat

You believe that feedback from any given user is just anecdotal. Once you witness the confusion experienced by a couple users at your seemingly “ready for engineering” mockup, then you’ll start to believe that a few users can teach you quite a bit. Start with a few user tests and you’ll quickly notice and then fix confusing user interface elements BEFORE you send your design to the engineering team saving a few weeks of wasted development.

Don’t boil the ocean with each round of user testing. Boil a thimble.

You might work in a Deadline-Oriented Perfection Environment

As a Tech Executive, you have taught the team to present concepts with finished pictures and polished mockups. Why not. This is what you’ve been used to for years. When presented with a concept, you tend to ask questions about the visual design or that the right color? is that link or button in the right location? is the name of the feature correct?

Instead, ask what users thought of the concept. How did the concept EVOLVE from the beginning to what you are seeing now. Understanding this evolution is key to understanding how your product team is engaging with your target market and will make the company smarter and more nimble.

Fair enough, but maybe the team is too slow. So you set a deadline for a presentation on a new concept. Combine this deadline atmosphere with a line of questioning that always focused on the look and feel of a concept (rather than its usability) and you’ve got a Deadline Oriented Perfection Environment. This is VERY typical. Unfortunately, this environment relies too much on luck. You’re not encouraging your team to EVOLVE their ideas through working with users.

You’re not giving them the time to test and the intellectual space to fail transparently. Time and space will raise the probability of success and reduce the reliance on luck. Remove educated guesses...insert evidence. The transparency of explaining and publishing the actual tests within the company will increase buy-in. You can still create deadlines but those should be focused on milestones such as completing the first prototype, completing the first round of testing, turning around the learnings from user testing in a fixed amount of time. If you move in this direction, the team will likely let you know what’s getting in the way of testing and then the real conversations in a successful product-oriented business begin.

Keep reading about how "Real users don't know jack..."