Blog Posts

HHVM with Symfony 2 looks amazing

With the recent announcement of Facebook that their HHVM is now more and more compatible with most of the popular framework, I was intrigued to finally try it out. We’re currently building a Symfony2 based application, which has pretty high performance requirements (but we can mostly achieve them with varnish), so I went and did some performance tests on that real-life app.

The application basically gets data from an ElasticSearch server and then transforms them to JSON. Nothing too fancy from the output perspective, but we use many features of Symfony2 and what was clear from the beginning is that the more data it gets from ElasticSearch the slower the requests get. Meaning that we loose a lot of time in the serialization part (we use JMSSerializer for that).

In short, the numbers were amazing. I also compared PHP 5.3 with APC against 5.5 with opcache, that alone gave some pretty decent improvements.

The setup I did the tests on was a QuadCore Intel i7-4770 CPU @ 3.40GHz server over at Hetzner with SSD disks and more than enough RAM. The actual application ran in a VirtualBox container with 4 CPUs and 2 GB of RAM. I used Apache Bench for the tests with the HHVM vagrant VM running Ubuntu 12.04, but I installed the pre-built HHVM from their repo in the end.

I made 3 different requests with different amount of data the script had to get from ElasticSearch. One was approx. 7kb in response, the other 80kb and the last 220kb. I didn't it with different concurrency settings (1-50) and ran Apache Bench with 500 requests per run. The figures below are averages over those 500 requests.

I didn't do performance measurements for PHP 5.3 with more than 6 concurrent requests, because I added 10,20,50 later and didn't repeat for 5.3, but the trend is clear nevertheless.

As you can see below, in general the longer the PHP request ran, the more we gained from HHVM, up to and sometimes more than 300% against PHP 5.3, and about 200% against PHP 5.5. Only switching from PHP 5.3 to PHP 5.5 can save you up to twice the time, as well. So it is very much worth upgrading to 5.5. I find both numbers pretty amazing.

If you really need every millisecond performance, considering HHVM is worth some investigation. These numbers are very promising. And I don't had to change one line of code in our application. As we don't use it yet in production, I can't say anything about stability. Obviously tools like New Relic are, at least for now, also out of the question with HHVM.

Requests per second, small response

The higher the better

Concurrency PHP 5.3 PHP 5.5 HHVM HHVM vs 5.3 HHVM vs 5.5 5.5 vs 5.3
1 52 84 81 157 % 96 % 163 %
2 120 163 207 173 % 127 % 136 %
3 154 219 285 185 % 130 % 142 %
4 172 247 317 185 % 129 % 144 %
5 186 256 342 184 % 134 % 138 %
6 203 259 340 168 % 132 % 128 %
10 253 443 175 %
20 247 441 179 %
50 258 380 147 %

/dynimages/370/files/images/blog/hhvm/small-req.png

Requests per second, middle response

The higher the better

Concurrency PHP 5.3 PHP 5.5 HHVM HHVM vs 5.3 HHVM vs 5.5 5.5 vs 5.3
1 10 14 26 269 % 184 % 146 %
2 19 29 56 295 % 193 % 153 %
3 28 42 79 286 % 186 % 154 %
4 34 51 92 271 % 179 % 151 %
5 35 53 98 279 % 186 % 150 %
6 35 50 101 285 % 203 % 140 %
10 56 113 202 %
20 53 105 197 %
50 56 108 193 %

image

Requests per second, large response

The higher the better

Concurrency PHP 5.3 PHP 5.5 HHVM HHVM vs 5.3 HHVM vs 5.5 5.5 vs 5.3
1 4 6 12 283 % 179 % 158 %
2 8 12 25 333 % 207 % 161 %
3 11 18 37 324 % 208 % 155 %
4 15 22 46 307 % 207 % 148 %
5 15 23 48 318 % 208 % 153 %
6 15 24 51 339 % 215 % 158 %
10 24 51 218 %
20 24 49 208 %
50 24 50 212 %

image

Median response time in ms, short response

The lower the better

Concurrency PHP 5.3 PHP 5.5 HHVM HHVM vs 5.3 HHVM vs 5.5 5.5 vs 5.3
1 15 11 9 60 % 82 % 73 %
2 16 12 10 59 % 79 % 75 %
3 19 13 11 55 % 81 % 68 %
4 23 16 13 54 % 78 % 70 %
5 26 18 14 54 % 78 % 69 %
6 29 22 17 57 % 75 % 76 %
10 38 21 55 %
20 78 43 55 %
50 191 105 55 %

image

Median response time in ms, middle response

The lower the better

Concurrency PHP 5.3 PHP 5.5 HHVM HHVM vs 5.3 HHVM vs 5.5 5.5 vs 5.3
1 98 61 32 33 % 53 % 62 %
2 103 66 34 33 % 52 % 64 %
3 107 66 36 34 % 55 % 62 %
4 112 71 40 36 % 56 % 63 %
5 138 89 49 36 % 55 % 64 %
6 171 112 57 33 % 51 % 65 %
10 174 83 48 %
20 366 185 51 %
50 872 451 52 %

image

Median response time in ms, large response

The lower the better

Concurrency PHP 5.3 PHP 5.5 HHVM HHVM vs 5.3 HHVM vs 5.5 5.5 vs 5.3
1 236 149 71 30 % 48 % 63 %
2 249 158 75 30 % 47 % 63 %
3 253 161 77 30 % 48 % 64 %
4 261 168 81 31 % 48 % 64 %
5 327 211 98 30 % 46 % 65 %
6 394 246 115 29 % 47 % 62 %
10 419 189 45 %
20 843 394 47 %  
50 2069 968 47 %

image

I will have a look at memory consumption in a follow up post.

Related Entries:
- Discussions and Pizza at PHPDay Italy
- RESTing with Symfony2
- How to preload ACL in order to get good performances
- Playtime with OroCRM
- Symfony CMF: what is left todo?

About the author

Comments [29]

gggeek, 29.10.2013 10:25 CET

Troll of the day: why not use node.js if your app is not serving a full fledged website but just spewing out json? :-D

And a noobie question: can't you use a serialization format which consumes less cpu on the php side when talking to elasticsearch?

gggeek, 29.10.2013 10:25 CET

Apart from that, great! On the blog posts from HHVM team it is never clear if they compare to php with or without opcaches...

Chregu, 29.10.2013 10:27 CET

The blog post was about measuring the performance of HHVM vs PHP 5.3/5.5, not about if we chose the right tools ;)

But Node.js was actually in discussion before we started, but we were much more comfortable with PHP and the project had enough other uncertainties, so we stick to PHP.

Lukas, 29.10.2013 11:37 CET

Also we are not just returning the JSON from ElasticSearch directly. There is a fair bit of business logic involved though most of this is handled via mappings managed by JMS serializer.

Ziumin, 29.10.2013 11:50 CET

Please, try PHP5.5 + OpCache.

Sebs, 29.10.2013 11:51 CET

As a heavy Node.js advocate I fully understand the PHP choice and it's about time that Projects like HHVM become avail. ;) I really like that and there is less need to port all your stuff over to js, just to find out the grass aint greener in ECMA Script. It isn't. As always.
Those numbers look very promising. That really looks like PHP in 2013. ;)

Chregu, 29.10.2013 11:51 CET

It is with opcache, as mentioned in the text

Mario, 29.10.2013 13:47 CET

Apache/Whatever Php5 version vs HHVM isn't a fair match, with nginx the story is a bit different. Although in the end maybe HHVM is a bit faster, the difference isn't that big.

chregu, 29.10.2013 13:49 CET

Did I say I used Apache somewhere? (just Apache Bench). The PHP part run with nginx/php-fpm. The difference is still big.

Mario, 29.10.2013 13:51 CET

chregu, 29.10.2013 13:54 CET

He didn't do very complex stuff in the PHP code, that's why the difference wasn't big (like in my tests with the small dataset). He acknowledges that (and made a second test with the fibonacci numbers in that post)

Mario, 29.10.2013 15:15 CET

Yeah, I didn't see the update part with fibonacci.

Jeremy, 29.10.2013 17:32 CET

You don't mention your php or hhvm configuration. How many workers/threads were used? It would be helpful if you could post both in full.

chregu, 29.10.2013 17:35 CET

Jeremy: Right, will do that. It shouldn't change the difference in the lower concurrency numbers, but maybe in the upper ones

But in short: I used the default configs and didn't tweak much there.

jfbus, 04.11.2013 16:25 CET

IMHO, "If you really need every millisecond performance", you shouldn't use JMSSerializer...

Till, 11.11.2013 00:45 CET

Christian (or Lukas) — can you guys share some code for others to re-run the tests? I'm interested too to verify some of these benchmarks.

Till

Lukas, 11.11.2013 15:49 CET

Well the appeal of this benchmark was that we were testing a real world client project. The draw back is that we cannot share the code.

Till, 11.11.2013 17:52 CET

@Lukas: Appeal is of limited use if I don't see what's being done. E.g., I'd like to see where the gain is. Do you guys use Thrift with ElasticSearch or HTTP and JSON. Basics like that.

Further, most of the configuration bits are missing from your blog post. E.g. what OS is this, and what are the default settings, etc..

chregu, 11.11.2013 19:20 CET

I can post the configuration details soon and maybe some more details. We're not using Thrift, just HTTP and JSON (or I may be totally wrong) with ElasticSearch.

We unfortunately can't share the code, but as the HHVM part was way faster without much concurrency and all that, the prove was done for me that it's way faster ;) But I try to come up with more details

Jordan, 18.12.2013 05:20 CET

Have you looked into igbinary as a serializer?

https://github.com/phadej/igbinary

I'm doing something very similar in one of my websites, where data gets returned from Apache Solr and then displayed to the end user. Serialization does take up a lot of the milliseconds in this process, so choosing the proper serialization engine makes all the difference.

Lukas, 27.01.2014 13:13 CET

The JMS serializer library to convert object graphs to XML/JSON currently does not support serializing to PHP arrays (https://github.com/schmittjoh/serializer/pull/20) .. but yeah could be interesting to also support PHP serialize in our API to speed up performance for other PHP clients.

PIT, 27.01.2014 14:59 CET

Will HHVM replace PHP?

Andrew McCombe, 27.01.2014 15:31 CET

Have you looked at using Siege as an alternative to ab? Siege will enable you to test a range of URLS and include php and static content in your tests. I'd be very interested in seeing the results from a test of simulated browsing session for example.

I've written a Siege howto on my blog which should help you get started. The posts are at
http://www.euperia.com/linux/tools-and-utilities/speed-testing-your-website-with-siege-part-one/720 and http://www.euperia.com/linux/tools-and-utilities/speed-testing-your-website-with-siege-part-two/771

Massimiliano Arione, 28.01.2014 13:12 CET

If you go from 100 to 300, you're not *gaining* 300%, you're *at* 300%.
You're gaining just 200%

chregu, 28.01.2014 18:49 CET

I know, I made some errors in the percentages, but it's still 3 times faster ;)

Christopher Svanefalk, 13.02.2014 20:18 CET

Tried out HHVM with an in-house Symfony2 app recently, and got sold right away. The performance was mindblowing (benching with PHP 5.5), and I loved the fact that I could deploy it as a stand-alone server with intuitive and easy configuration. I am now advocating for optimizing all our projects for HHVM compliance.

I have no doubt HHVM will have an increasingly strong impact on both PHP and the community in 2014. The performance is great all by itself, of course, but breaking the monopoly of the existing PHP runtime is a good step as well. In the best of worlds, we may finally see not only a real PHP standard, but perhaps a PHP6 which will (finally) ignore BC and remove the slag that has accumulated in the language over the years.

Rishabh, 07.05.2014 13:27 CET

Can you share your HHVM configuration file. Am trying to setup HHVM with Kohana, but its behaving slower than FPM

AlmogBaku, 13.07.2014 01:27 CET

Interesting article!

@Lukas, have tried to run big application on production with HHVM? is it stable enough?
And are you still using HHVM on your project since the date of the article?

Chregu, 13.07.2014 07:05 CET

Yes, we're using it on a fairly large code base with quite some requests. And we will expand the usage of HHVM on that project in the near future. We're still happy with it.

Add a comment

Your email adress will never be published. Comment spam will be deleted!