A pilot program was tried during the Windows 10 Fall Creators Update rollout but this was the first time Microsoft performed a large-scale test. The software giant says it's pleased with the results:
Our AI/Machine Learning approach started with a pilot program during the Windows 10 Fall Creators Update rollout. We studied characteristics of devices that data indicated had a great update experience and trained our model to spot and target those devices. In our limited trial during the Fall Creators Update rollout, we consistently saw a higher rate of positive update experiences for devices identified using the AI model, with fewer rollbacks, uninstalls, reliability issues, and negative user feedback. For the April 2018 Update rollout, we substantially expanded the scale of AI by developing a robust AI machine learning model to teach the system how to identify the best target devices based on our extensive listening systems.