In this guide
Your development PC is the worst place to test your installer. It already has the runtimes, certificates, folders and registry entries your app needs. Users' machines do not. This plan tests the install lifecycle on machines that look like theirs.
Set up the test machines
- A clean Windows install in a virtual machine, with a snapshot taken before anything is installed. Revert to it between runs.
- A second machine in a different state: another Windows version you support, a non-admin user, a different system language, a path with spaces or non-ASCII characters in the user name.
- An older build of your own app to upgrade from. Keep every build you ever shipped.
Whether you ship an MSIX package, an MSI or a classic setup program, the scenarios below stay the same. Installer behaviour differs by technology; check your installer's documentation for the details.
The scenarios
| # | Scenario | Expected result |
|---|---|---|
| 1 | Fresh install on a clean machine | Installs without extra steps; app starts; no missing-runtime errors |
| 2 | Install as a standard (non-admin) user | Either works as designed or asks for elevation with a clear message |
| 3 | Install to a custom folder (spaces, non-ASCII) | App starts and can write its data |
| 4 | Start from the shortcut and from the file association (if any) | Both open the app correctly |
| 5 | Upgrade over the previous release | Settings and user data survive; version number updates |
| 6 | Upgrade over a much older release | Data migration works or fails with a clear message |
| 7 | Upgrade while the app is running | Installer detects it and asks to close it, or handles it cleanly |
| 8 | Interrupted install (cancel, power loss, kill the process) | No half-installed state; running the installer again works |
| 9 | Repair (if offered) | Missing or damaged files are restored; user data stays |
| 10 | Uninstall | App, shortcuts and start entries are gone; no leftover running process |
| 11 | Uninstall, then reinstall | Behaves like case 1; you decide whether user data is kept and say so |
| 12 | Security prompts | Note any warning the system shows when first running the installer |
How to run it
- Revert the snapshot, run one scenario, write down the result with the build number.
- After each install and uninstall, look at the install folder, the user data folder and the list of installed apps. Leftovers are findings.
- Check the log your installer writes, if it has one. Attach it to the report.
- Repeat scenarios 5 to 7 for every release, not only the first one.
What usually goes wrong
- The app needs a runtime that was never on the clean machine.
- Settings from an old version are read by a new version that expects another format.
- The uninstaller removes user data without asking, or leaves services and background tasks behind.
- A path with a space breaks a command line argument.
Write each finding with the desktop bug report template so it is reproducible.
What AppReviewX covers
AppReviewX accepts apps by store link from Google Play, the Apple App Store, the Mac App Store and the Microsoft Store. Windows and Mac apps must already be published in their store, and AppReviewX does not distribute builds or installers. Feedback is private and structured, one accepted feedback = one credit, and store ratings are voluntary, unpaid and never required. This page is general guidance.
Start free and get private feedback on your app.
Related: Windows and Mac app testing guide, How to run a beta test for a Windows app, How to run a beta test for a Mac app.