Yeah, I'm no compiler engineer, but testing both release and debug builds is the minimum I'd do. That doesn't even get into classes of optimizations, like loop unrolling, binary size, macros, or function inlining, which I also expect to be in a compiler comparison.