Download PDF
RCA TRAINING
Root Cause Analysis training by Sologic provides the tools, skills, and knowledge necessary to solve complex problems in any sector, within any discipline, and of any scale. Learn MoreSOFTWARE
Sologic’s Causelink has the right root cause analysis software product for you and your organization. Single users may choose to install the software locally or utilize the cloud. Our flagship Enterprise-scale software is delivered On Premise or as SaaS in the cloud. Learn MoreOn 4/21/2010 at approximately 2:00PM GMT Company x released an update to it's Virus Software Enterprise 8.7 (VSE 8.7). The update added detection for variants of the W32/Wecorl.a family of malware. The update included DAT File 5985, which contained an unidentified coding error. This error caused a healthy system file, svchost.exe, to be flagged by VSE 8.7 as being malicious.
Once the file was tagged as malicious, VSE 8.7 killed the svchost.exe process. Microsoft has a built-in safety mechanism that kicks in when a system executable is killed. This safety mechanism causes the system to reboot. Upon reboot, VSE 8.7 attempted to remove the now flagged svchost.exe file, disrupting the normal operation of the system. This caused users to experience the "blue screen of death" or an endless series of attempted reboots.Tens of thousands of users were impacted causing an estimated $50 million in lost productivity.
CODING ERROR: DAT 5985 works by monitoring the memory activity of system files. The W32/Wecorl.a malware attempts to gain and maintain control of a system through the use of memory of executable system files. DAT 5985 mistakenly identified normal memory activity of svchost.exe during system startup as an attempt by malware to gain control of the system. This was due to a coding error. It is unknown why the coding error occurred, but two possible fault paths need to be examined. 1) Was there a coding execution error? 2) Was there a specification error? Either, or both, are possible.
QUALITY SYSTEM FAILURE: Company x's QA process missed the coding error before going into production. This error only manifests in system failure on Windows XP, Service Pack 3 (XP SP3). XP SP3 was not included in the test configuration for VSE 8.7. Also, there was no peer review of the driver completed before release.
Both of these quality system failures require further examination.