Delta is my friend

I think it was a couple years ago when working for the CLR and running into what appeared to be a compiler issue in a newer version of gcc. Zipping up the entire enlistment and shipping it to Apple would have been a rube mistake: (1) Microsoft doesn’t really want its source disseminated unnecessarily, even if the partners have a limited non-disclosure agreement. (2) Even the youngest tester knows the mantra: get a small, simple repro.

Suffice it to say that our source code is rather complex (and probably a small factor more than it need be). Generating a giant .ii file and then culling it manually to keep the reproduction while shedding intellectual property is a soul crushing problem. You can make some educated guesses at what is causing the compiler to behave oddly, but sometimes those guesses are ill-founded, and there’s no easy way to get to minimal repro.

Enter Delta, which Mike Stump of Apple pointed out to me. It takes the tedium out of reducing the reproduction down to a minimal one, making the obfuscation/renaming process usually much easier (unless something about the names themselves are problematic).

You start with your giant reproduction: g++ flags flags flags -c file.ii, where file.ii is the preprocessed version of your source (i.e., replacing the “-c” with “-E” at the gcc line). In our cases, this can be something like 1.8 MB of text file. You then write a simple “interestingness” decider script, e.g., that gcc succeeds in compiling the source, but that the output contains a particular warning. Then you run Delta in several iterations, and it selectively removes parts of the source file, tests the file for (continued) interestingness, and if it is still interesting, it caches the smaller result, and keeps going, looking for more text it can remove and still have the repro intact. Several hours later (of the computer’s time rather than your own), it can knock the size down to a few K of text, and then the whole problem becomes much more tractable.

I’ve probably had to post only six or so bugs that fell into this category, but Delta has saved me tremendous time and effort.