Monitoring customer support calls and chats is a good way to get qualitative data from customers, and it’s dead easy to automate it with AI tools these days.

A dark launch is a release of production-ready software features to a small user segment. The practice is commonly used in modern software development to test new software stability and performance before launching it to all users.

Illumination logoVariations

When you carry out a dark launch, the risk is limited but still significant. To reduce it, there are a few more tests you can conduct to ensure you’re developing the right features and they have a solid technical foundation.

Basically, you enable the feature for a small percentage of users and monitor its performance or impact on user experience. As you get insights from feedback and analytics, you iterate, release it to some more users, and so on.

For example, the next step could be pretending that the feature exists but doing all the work manually behind the scenes. This kind of experiment is called Wizard of Oz.

Let’s imagine you’re an Asana product manager and you want to build the Goals feature It’s a theoretical example because, to our knowledge, Asana never did such a test.

Illumination logoWiki

As a product manager of a popular workspace tool for creating, organizing, and sharing various types of content, you’ve been working on an AI writing assistant.

✅ Real-world testing: thanks to a dark launch, you can validate new features and evaluate their performance with real users.

In dark launches, nobody tells the users that they’re taking part in the test or that they are using a new feature. That’s why they’re called ‘dark’.

However, if low-fidelity testing gives you promising results, you can invest more effort into the idea and make your prototypes progressively more realistic.

Illumination logomeaning

If the feedback is positive, that’s great. However, when users submit negative feedback, follow up with them to know in detail about their issues with the new feature.

❌ Technical debt: unused feature flags can make it more difficult for the development team to maintain the code and release new software features, especially if you’re using homegrown feature flag systems.

✅ Reduced risk and damage control: releasing features only to a small number first means that any negative impact on user experience is limited to the small test group, and it doesn’t cause disruption for the whole user base. Also, there’s limited risk for your product reputation if things go south.

Illumination logoDespicable Me 4

Fake door testing is a kind of prototype testing. In this case, the prototype is very low-fidelity as you have nothing under the hood.

A dark launch is an easy way to reduce the risk involved in feature releases. By enabling the feature to a small number of users, you can still test it under real-world conditions. If things go wrong, though, the damage is limited.

Illumination logopng

❌ Complexity: traditionally, dark launches required multiple production environments or complex routing configurations. It’s less of an issue now thanks to feature flags/feature toggles but it still can result in fairly complex code.

Before you do so, however, you simply add it to your menu and trigger a tooltip to prompt user engagement. And then you sit back and watch.

Illumination logohistory

Fake door testing is the least labor or resource-intensive way of testing your feature ideas. The feature doesn’t need to exist as long as you convince your users it does.

Naturally, nobody likes being taken for a ride so when your users actually click on the feature, you need to explain what you’ve just done and why. Like in the modal below.

Get The Insights! The fastest way to learn about Product Growth, Management & Trends. Email CommentsThis field is for validation purposes and should be left unchanged. You might also be interested in ...What Is Pendo and Is It Worth Using?Aazar Ali Shad October 30, 20248 min readHow to Write Better User Stories With Gherkins (Template Included)Andrea October 30, 20244 min readProduct Team Structure - A Guide For SaaS Product TeamsAazar Ali Shad October 30, 202410 min read

Illuminationlogopedia

Both dark launch and canary testing allow teams to test how new functionality affects system performance or product metrics with real users, or in a production environment as a software engineer might say.

As soon as a user interacts with the feature, trigger an in-app survey to ask them for their feedback. Such contextual surveys are a great source of valid insights because the experience is still fresh in users’ minds.

Illumination logo(2024)

So, before you dark launch your feature, beta test it first with an even smaller group of users. This could be a few of your most dedicated power users who will be happy to take the feature for a spin and give you actionable feedback.

Beta-testing is often seen as an alternative to dark launching, but there’s no reason why you can’t combine both of these approaches.

Apart from tracking user in-app behavior, pay attention to support tickets and bug reports. That’s how you will be able to identify technical issues.

Or you can use other existing solutions and botch them together to achieve the desired functionality before you develop your own code.

To be able to roll out the feature in such a way, you wrap it in a feature flag or feature toggle. They allow you to switch the feature on and off quickly without modifying the code.

Whenever issues arise, you fix them. Gradually, as the product gets better, you start releasing it to new user cohorts and continue testing.