Buggy updates: What is wrong with the Windows 10 development cycle

Hot on the heels of the news of data loss and other issues with Windows 10 October 2018 Update, ARS Technica takes a look at what's wrong with the development cycle of Microsoft's operating system. The site suggests the buggy updates point at a deeper problem, it's not about how often but how the OS is updated. Basically, there's a lack of quality control, which results in software that's tested after and not before it's shipped.

Interestingly, the article points out that a full test cycle for Windows takes many weeks. Microsoft does use this test cycle, but not for the Windows 10 builds that actually ship. For example, the Windows 10 October 2018 Update was built on September 15 and went live on October 2. That's too short for a full test cycle, so we end up with a build that contains updates that were not properly tested.

The inadequate automated testing and/or the disregard for test failures means in turn that the Windows developers can't be confident that modifications and fixes do not have ripple effects. This is what gives rise to the "ask" phase of development: the number of changes that are accepted as the update is finalized has to be very low, because Microsoft doesn't have confidence that the scope and impact of each change is isolated. That confidence only comes with massive, disciplined testing infrastructure: you know that a change is safe because all your tests run successfully. Whatever testing the company has in place for Windows, it isn't enough to earn this confidence.