Collaborative Research: CSR: Core: Medium: Scaling Unix/Linux Shell Programs

The ‘shell’ is the glue that holds modern computer systems together: It is the programming language that connects components and manages systems; it is the quick-and-dirty language for writing scientific workflows and data analysis; it is the control hatch that offers skilled users direct access to their computer's inner-workings. Shell programming remains ubiquitous —steadily in the top 10 most popular programming languages— partly because it allows non-expert developers to combine components available in any programming language with little-to-no understanding of their internals. Unfortunately, this black-box, language-agnostic composition hinders automated acceleration of shell programs. This project will develop techniques to accelerate shell programs without requiring input from shell programmers and little-to-no input from the authors of individual components. This work's approach (1) builds on the real-world success of prior research by the PIs (that requires more user input), (2) brings together expertise in several subfields of computer science —parallel and distributed systems, programming languages, and formal methods— and (3) offers synergy. The individual techniques each offer significant benefits, but yield a transformative whole. By enabling practical, push-button enhancement of a real-world, pervasively-used programming language, the proposed work promises to significantly accelerate existing and new shell programs without any effort from their developers, without any changes to existing infrastructure, and while maintaining compatibility with prior results. The shell's ubiquity means these speedups will have broad impact, especially in disciplines that use computing as a tool to solve problems in their respective fields, e.g., bioinformatics, data science, and economics. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.