If a tester's first experience is a crash on launch, you spend their time on something you could have found in five minutes. A short smoke test before you invite anyone keeps their feedback for the problems only outsiders see.
The 15 minutes are a rough guide. Do it on a build installed the way testers will install it, not from your development setup.
The 12-point smoke checklist
- Clean install. Uninstall the old version, install the build testers will get. Why: leftover data hides first-run bugs.
- First launch. App opens, no blank or frozen screen. Check: watch the first 10 seconds.
- Onboarding. Finish it once, skip it once. Check: nothing blocks you.
- Permissions. Each request is explained and the app still works if you deny it.
- Sign-up and login. Create an account, log out, log in. Check: also password reset if you offer it.
- Test account. Test credentials you hand out really work.
- Main task. Do the one thing the app is for, start to finish.
- Kill and relaunch. Force close in the middle of a task and reopen. Check: data and state make sense.
- No network. Turn on airplane mode. Check: a clear message, no crash.
- Rotation and text size. Rotate, use a larger system font. Check: nothing is cut off.
- Second device. A small and a large screen, or an older Android/iOS version if you have one.
- Leftovers. No debug banners, placeholder text, test data or broken links visible.
Do not skip
- Version and build number. Testers should be able to tell you which build they have.
- Known issues list. Write them down so you can tell testers to ignore them.
Cannot cover everything
You will not find what you are used to not seeing. That is what outside testers are for. Your smoke test just clears the obvious things.
Next steps
- Write the instructions: test instructions for testers.
- Send a clear invitation message.
- Find testers: Find beta testers for your Android app.