5 steps to running a successful beta community
Lessons learned from running RingTo, a service offered by Bandwidth that gives users the ability to virtualize their phone number
As a product owner in the consumer app space you are often challenged with sorting out truly exciting opportunities from “me too” features that your market alternatives might be winning users with. Sometimes you are faced with more mundane challenges like inconsistent behavior across mobile devices due to the infamous Android Fragmentation challenge.
When we launched the RingTo service, we quickly realized that we need a group of users to help us vette ideas and rapidly isolate issues with our app. We needed the group of largely anonymous people to be vocal in sharing opinions, represent the needs of our broader user base, and be willing to work with partially implemented features in exchange for getting early access.
If your service or app is addressing an issue worth solving, you generally won’t have issues with users telling you how to improve. Let’s assume you are solving for a worthwhile problem and getting users is not an issue. The challenge then is to create successful beta community.
1. Exclusivity
The allure for many users that sign up for beta programs is that they want to be an “insider” – access to the latest and greatest. Access to beta should NOT mean that users are an extended QA team. Your beta users will forgive unstable code on new features, but that doesn’t excuse you from maintaining your own regression testing. In addition to accessing the “latest” features / bug fixes, you should consider giving your beta users an increased level of attention / support over the general user base. In our case, the general users only got access to our first tier support. Beta users had the direct ear of the RingTo product owner (me). Depending on the relationship you want with your users, you may consider a closed vs. open beta. For long-term beta programs, a close beta group is likely better.
2. Frequency
You need to keep your audience engaged! Daily interaction is not necessary. As a rule of thumb, we worked hard to get out a new beta release at least once every two weeks. There were some periods when we would do daily updates. Bottom line, you can’t expect a vibrant audience if you are not giving users something to look forward to. Depending on the size of your group and nature of your service, you will find that individuals engage a different points. Some users are “always on”. Some only engage when you push out something new. Some only engage when something breaks.
3. Value
Users that are part of your beta community need to feel there is sufficient value being delivered in exchange for a potentially unstable product. Conversely, you as the service provider need to get value of the beta program! In our case, we had our beta group do three things for us. First, and least exciting, was to validate bugs on specific OS/devices that were not available in our lab. Second, and the reason most people stayed, was the early introduction of features that were at least 45 days out from full production release. Our beta community really excelled in telling us what was great and what needed improvement with our new features. The third area we used our beta community for was for UX validation. Rather than go to our broader user base or try to run a real-time experiment in the app, we would create a lightweight mock
up of a new UI/UX idea and have people give us opinions before our developers wrote a single line of code.
4. Cultivation
As part of the closed beta, we got to control who came in. The reality was that nearly everyone who asked for access was granted. While getting the right people in is critical, it is equally important to keep the wrong people OUT of your beta. We sought to keep the beta community vibrant through regular updates and encouraging civil discourse. There were a few instances where users had to be removed from the beta program, mainly because the users thought beta access equated to getting priority support that could otherwise been addressed by reading the online documentation.
5. Transparency
Be open and honest. You don’t have to reveal how your business operates or the fact that your engineers are all on vacation this week. What is important is that you communicate at a level that builds a deeper level of trust. Openly tell your users that you don’t know the answer (yet). Maybe your engineers still don’t know how to fix that vexing crash issue yet. Did you roll out a beta that broke a critical feature for half your users? Own up to it with a personal message. Not a massaged message for the masses. This: “Sorry beta peeps. We screwed up on this one. Let’s roll back to the previous release while our team has chance to figure out how we got this so wrong” Not “Dear users, at [app name] we strive for excellence. We didn’t meet your expectations with the latest release, blah blah blah”. The point is that if it sounds like corporate speak, users don’t think you are giving the whole story.
In our case we split our beta community across two social platforms. Android users were directed to use Google Plus, because Google made us do for distributing beta builds via Google Play. IOS users were managed through TestFlight (now owned by Apple) and a dedicated community on the RingTo support site. Between the two, we found the use of a private G+ community for managing the Android Betas extremely effective. IOS users could only send direct feedback to the developers. Android beta users could post comments to the entire group, making it easier for the entire community to jump in on the same activity. In the end, both groups gave us valuable feedback, but the G+ community for our Android beta was far more fun to interact with.
As you grow your beta group, treat them like your trusted advisors. Both you and your users will find the experience incredibly valuable. Good luck out there to all you entrepreneurs! There is a tough road to getting your app noticed. Getting your beta users is one of many steps on your road to success.
