Ever since last’s years ruby-on-rails project, I love the simplicity and beauty of Haml.
So a couple days ago, I decided to see if there aren’t any implementations for PHP, and maybe even a WordPress plugin for theming in HAML.
Guess what, there are.
Of course, I did what any sane CS person would do and submitted all the implementations I could find to some rigorous testing.
This post is about the results of that testing.
Two things I was most interested in:
- Completeness: in layman’s terms, how good the parser is at its job. Testing if it parses and also if it compiles with PHP, and
- Speed: how fast they are (both startup and parsing speed).
I found and tested 6 implementations:
array('haml2php', 'mthaml', 'fammel', 'connec_phphaml', 'phamlp', 'phphaml', 'baldrs/phphaml');
For testing I used all the different .haml files that the parsers themselves came with (124 in all), I also added a couple myself to test some specific things.
I had to rewrite some of them a little, because they used ruby code, and for this we needed php code.
I also did some changes to the parsers themselves, mostly cosmetical though, like throwing them into their own namespace (so they’d play well together, since they tend to have the same classnames – think Parser, Haml) or changing some accessor modifiers (so I could call the parse functions from the outside).
There are two main schools of compiler-writing present in this field.
Fammel is the only one going the traditional way of having a grammar. It uses lime to turn that into a parser and lexer. One problem with that approach in connection with SASS is that Sass is not completely context-free. Happily, fammel only compiles Haml. It is still far from complete, though. The fact that fammel sports a decent result of 75% success rate is because half of the test .haml files came with fammel (so it knows how to parse them).
Every other compiler uses RegExes in some fashion or other.
I also found quite a few of the .haml files not to be correct haml files (read syntax errors), so I threw them into a special folder called ‘invalid’ which signals to the testing code that those should not parse. Some of the parsers do, however, parse the erroneous scripts, which I then don’t count towards the overall result.
Haml itself is very good in that it even parses the contained ruby code in the template, and throws an error if its not syntactically correct, which sadly also prevents us from running the same templates trough haml for comparison. All the PHP compilers do not parse PHP, so if we want to test the compiled .php files for syntactical correctness we have to save them somewhere and run them thru ‘php -l’. Trying to include or eval files with parse errors in them, throws a Fatal error which cannot be caught and terminates the script.
Without any further ado, here are the results of all 124 templates:
--------------------Results--------------------- Name #compiles/#parses/#count % --->haml2php: 87 / 89 / 124 70.16% --->fammel: 90 / 94 / 124 72.85% --->mthaml: 92 / 100 / 124 74.19% --->connec_phphaml: 93 / 97 / 124 75% --->phamlp: 108 / 113 / 124 87.1% --->phphaml: 116 / 122 / 124 93.55% --->baldrs_phphaml: 116 / 122 / 124 93.55%
Of course, the fact that it parses and compiles with PHP says nothing about correctness, it might be syntactically correct but still be logically incorrect.
PHPHaml is by far the best parser in the field, being able to correctly translate almost 94% of the templates.
It does have its faults, though: Haml templates can be indented by any amount of spaces as long as they remain consistent, or tabs. PhpHaml only allows 2 spaces, nothing more. Furthermore Code insertion (following an euqals sign) does not need a space in Haml, ie. the code can follow immediately after the =. In PhpHaml that breaks. Filter handling broke due to the subclassing that was done downstream, but that is easily fixable. Speaking about filters: calling an nonexistent filter in Haml throws an error, in phphaml it does not. And lastly, whitespace handling (eg. in :preserve) as well as variable interpolation with filters does not work correctly in phphaml.
Now for the timing tests:
--------------------Results (/μs)--------------------- Name startup(cold/hot) min avg (of#) max --->haml2php: 25 / 15 2421 10784.18 (80) 91853 --->mthaml: 351 / 24 819 6682.53 (92) 87258 --->fammel: 253 / 56 553 6015.93 (86) 65379 --->phamlp: 76 / 33 189 2503.01 (109) 33780 --->connec_phphaml: 26 / 20 260 1617.51 (89) 26822 --->phphaml: 5360 / 62 692 3929.44 (119) 61683 --->baldrs_phphaml: 3761 / 57 724 4514.69 (119) 69345
All times are in Micro- (10^-6) seconds.
We can see why PhpHaml is so good, a cold startup takes one order of magnitude longer than everyone else, hot startup is still slowest but better. Parsing itself places it squarely in the middle of the field, in minimum and average [parsing times] it takes third place, whereas in maximum it even makes it to second place.
As an update, I added Baldrs fork of phphaml as baldrs_phphaml. They included some more patches which makes HTML-style attributes work now, but doesn’t increase the compile count, because even though phphamldid compile it, it wasn’t logically correct.
As a downside, the changes makes it slightly slower, sadly!
For anyone interested, here is the complete source, which includes all tested compilers, templates and my testing code. Enjoy!