Benchmarks

Note: This page has yet to be updated for release 0.4, which implemented a number of optimizations that are relevant to the n-body benchmark.

This chapter contains results for a few preliminary toy benchmarks, all taken from The Computer Language Benchmarks Game. There's only three of them for now: n-body, spectral-norm and binary-trees (the complete list of benchmarks is available here). More will be added in the future.

Here are the results (average of 10 runs, in seconds) on a single-core processor (the Cell version does not take advantage of multiple processors at the moment) compared to the fastest Java version (on the same single-core processor), which provides a floor for the Cell version since it's hard to beat an optimized Java program when compiling to Java:

By far the benchmark that exhibits the largest slowdown is n-body, which is two and a half times slower in Cell that it is in Java. But that's mainly because the Cell code has to use a different, less efficient algorithm that does not use mutability, which hasn't been implemented properly in Cell yet. A Java implementation that uses the same algorithm (code) runs in 23.49 seconds on average on the same machine, which means that the slowdown caused by the intrinsic inefficiency of the code generated by the Cell compiler is a more acceptable 27%.

These benchmarks should of course taken with a grain of salt: there's still a lot of work to do on optimization, and I expect the performance penalty for using Cell over Java to turn out to be larger for more complex program, even when mutability is not needed. Still, this suggests that Cell, while slower than other statically typed, compiled languages, is already reasonably fast for many tasks, and that its very flexible, and harder to optimize, data model and type system need not be too much of a liability in terms of performance.