Are Beta Programs Worth The Effort?

By Eric Spiegel

(Back to article)

Would you ever drive a car that wasn't quite finished yet? What if you were asked to test drive next years' model at no charge? Is there a catch? Well, it turns out the car isn't painted yet, the wipers don't work and the CD player keeps skipping. But what a thrill going from zero-to-60 in five seconds!

As another bonus your input on sound system specifications and color preferences could end up in the final model. To seal the deal, the sales person offers you a significant discount once the car is released. So who needs windshield wipers? You convince yourself it hardly rains in Seattle and drive off the lot.

Now back to reality, or in your case, the office. Whether dealing with a giant like Microsoft or a small startup, all software companies are very eager to find customers who are willing to undertake a similar adventure with their near-to-market products. This is commonly referred to as a beta program.

There are three types of pre-release programs typically offered. The earliest is an alpha program or "tech preview" which is usually offered to only the most enthusiastic, technically savvy customers. Lots of bugs and system crashes can be expected, but you receive the earliest glimpse of the bleeding edge. This program is usually run by the vendor's product management and engineering teams.

The program offered immediately prior to the final release is usually run by sales and marketing, sometimes referred to as an early adopter program. This is targeted at potential customers who are looking to try out a product, but are not willing to deal with the roller coaster ride of a tech preview or beta program. The goal here is to build a solid general availability sales pipeline so the product can be marketed as a must-have, high demand item.

Sandwiched in the middle is the beta program, which is mostly driven by product management with some input from the sales team. This is a more stable release than the alpha and is offered to a broader range of customers with a formal feedback mechanism. The product can still be buggy, but must be stable enough for release to selected end users, not just techies. A successful beta program results in case studies and can prime the pump for a successful early adopter program.Where's The Beef?

We all know that IT departments are stretched thin these days, so why commit your teams' valuable time to a beta program?

"Beta programs are frequently undertaken to fill a need where no (released) software currently exists," said Ray Hamann, manager of Web Services & Application Development at Johns Hopkins' Bloomberg School of Public Health. "We are making an investment in the product's viability and potential."

From a software vendor's perspective the benefits include feature suggestions and real-world debugging, said Marc Rubin, chief operating officer of xlipstream, a provider of XML Appliances. This is especially true for small startup vendors.

"A beta program lets you get your future product out of the lab and into a real world environment," said Rubin. "You hold your breath and then feel the relief when the product does what you say it will do."

Notice the emphasis on "real-world?" No matter how much the target market is researched or how much internal testing is done, there is absolutely nothing like a warm-blooded customer playing with a product to highlight how stable, useful and most important, "sellable" it is.

Extended Engagement

If you are considering participating in a beta, it is important to carefully consider the vendor's expectations which are usually outlined in the beta agreement. Understand the time commitment required, including completing formal test plans and providing written feedback. You may also be asked to participate in weekly conference calls to review bugs and your feature requests.

Be very careful not to overburden resources involved in mission critical projects or production support. In addition, expect to test every inch of the product, not just your areas of interest, said Hopkins' Hamann.

"Beta testers must be prepared for involvement in the testing process beyond the functionality they may have specific need for," he said. Hamman also stresses testers weigh the impact staging or testing environments will have on their existing operations.

When it comes down to it, though, there is one word that reflects the top dog benefit: discount. Since the vendor benefits the most in these beta relationships, therefore the customer has a right to expect a pricing break if the product is purchased at the end of the program. When negotiating a beta agreement be sure to play the "free testing" trump card and the promise of a glowing case study -- assuming the product delivers glowing results, of course.

Just keep that discount in mind if you happen into an unexpected downpour and you regretfully remember the windshield wipers aren't scheduled to work until the general availability release.