CAREER: Testing Evolving Complex Software Systems

Modern computer systems span a wide range of domains, ranging from consumer electronics (e.g., smart-phones) to safety-critical systems (e.g., avionics). These systems evolve rapidly because the competition for market share pushes developers to come up with new features or improve capabilities over existing ones. These software changes may require hardware replacements or upgrades to capitalize on software upgrade opportunities. As such, developers must ensure that changes do not cause any unintended impact to the existing quality of the systems. Regression testing has been widely used to assess whether changes have adversely affected system behavior. While significant work has been accomplished by the software engineering community in improving effectiveness and efficiency of regression testing, most of the existing techniques focus on traditional software that is environment-independent and non-distributed. Real-world software systems, however, are far more complex: they frequently interact with the environment via hardware devices, and employ various concurrency mechanisms to coordinate interrupts, signals, threads, and processes. These characteristics affect various techniques on which existing regression testing approaches rely. Therefore, applying these approaches may lead to problems during maintenance and thus impair software quality. The overall goal of this proposal is to create a novel regression testing framework that can be applied to real-world complex software systems, focusing on the hardware dependence and concurrent control characteristics, throughout their lifetimes. Specifically, this research will develop, evaluate, and make available a family of techniques and tools that can: 1) create comprehensive models of the whole system to analyze change impact across hardware and software layers and across concurrent events, 2) retest the systems accordingly using existing test cases, and 3) generate new test cases when needed. The analytical underpinnings of this research will be applicable not only to the software engineering community, but to industry and other disciplines in which software dependability plays an important role. The associated education agenda paves the way for teaching that cross traditional boundaries among multicore computing, embedded systems and software engineering, which may ultimately, through the dissemination of new curricular materials, have impacts to the broader scientific community.