Caleb Stanford Blog

Software Reusability PSA


PSA: vetted installation instructions and examples/ directory considered mandatory

Having been on academic “artifact evaluation committees” a few times, there are many ways the process fails to encourage some of the most important aspects of making software reusable. The biggest glaring problem is installation: in my experience, 90% of the pain of trying to reuse someone else’s software or git repository is just trying to get the tool and its dependencies installed, and the present system almost completely ignores this by having researchers submit pre-built virtual machines, so that the installation process is not checked at all by the reviewers.

Besides installation, all software should have an examples/ directory. I am surprised by how often I come across software that doesn’t, and I spend several minutes poking around trying to understand the software without much luck. A basic examples/ helps a lot more than well-documented code, unit tests, or even a README because it’s the first place I will go to see how I might actually use your project for some intended use case.