Articles, Tutorials

Apache, Nginx, PHP, HHVM? Test it yourself! Ansible

The most typical conversation (aside from which plugins to use) in the WordPress sphere is about the speed comparisons between various web serving software like Apache or Nginx. There are tens of articles with nice charts and big claims on the Internet: Nginx is faster than Apache. HHVM is faster than PHP-FPM and so on. The question is: can we trust them? What if the technology behaves differently on our VPS? The only way to be sure is to carry out the benchmarks ourselves. It is, however, time-consuming to install and configure the whole stack. I think I found a (partial) solution.

Ansible to the rescue

Working on my Bachelor’s Thesis, I had to find a way to quickly switch between Apache with mod_php, Nginx with PHP-FPM, HHVM, etc, because I’m comparing and optimizing their performance. Since then, I’ve learned that Ansible might help me, as it can be used to automate mundane server-related tasks.

After some struggling weeks, I’ve put together a collection of Ansible playbooks that install, set up and reset everything that we need. It is enough to input IP address (or hostname) of your VPS server, run a playbook and benchmark the WordPress site.


I’ve created the playbooks to work with Ubuntu 14.04. It should be possible to use any Debian-based Linux distribution, but you will need to change the apt repository versions in the roles. Have a fresh Ubuntu installation on the server because the playbooks will manage it completely: from updating apt and setting up the locales to downloading WordPress and configuring Nginx to serve it.

First steps

  1. Clone the wordpress-ansible repository.
  2. Edit hosts file and add/replace IP address or hostname of your testing server. Assign the servers into the group “webservers”. Ansible Inventory — read more.
  3. Open the group_vars/webservers file and modify the variables to your needs. This file holds most of the variables used during executing the playbooks and roles. Change the path to the public part of your ssh key. Ansible will ask for root password when run for the first time, then copy the ssh key so it will become passwordless. If you need to generate a ssh key, follow GitHub guide. Also change the wordpress_home_url to match your server’s IP address or hostname.
  4. Optional: check the roles so you roughly know what they do.

We are now ready run Ansible. Within your command line, navigate to the cloned repo’s directory and execute following command:

ansible-playbook -i hosts install-all-software.yml

It could take between 10 to 30 minutes to fully complete all the roles and tasks. Let it work and have some coffee. If you run into some errors, open a new issue or notify me in the comment section below, please.

Let’s do some benchmarking

If the previous command finished successfully, your server is almost prepared. The last step is to “activate” (run a playbook) a specific server configuration, so we can actually test the loading speeds.

However, we still don’t have a proper tool to send the testing requests to the server. In the past, I’ve used Apache Bench command line utility to do this. The downsides of ab are that you need to have a fast Internet connection (the test is run from a local computer), there are no nice charts and no history. Nowadays, I’m using (referral link), which is a load testing service with up to 10,000 concurrent requests for free.

Setting up

When you sign up at, you can add a new server for testing. You need to verify the server by placing a txt file on your server where can access it. I’ve created a simple playbook which puts the token into the WordPress directory so its path will be<your-token.txt&gt;. Here’s the playbook:

- name: host verification >

  hosts: webservers
  remote_user: root

  - name: Install verification token
    shell: echo {{ token }} > {{ token }}.txt
      chdir: "{{ remote_wordpress_dir }}"
      creates: "{{ remote_wordpress_dir}}/{{ token }}.txt"

Save it in the cloned repo’s directory and run it if you reset the WordPress installation.

Nginx with PHP-FPM

Let’s test Nginx with PHP-FPM, without FastCGI and WordPress database caching. Execute the following command:

ansible-playbook -i hosts nginx_php-fpm.yml


ansible-playbook -i hosts wordpress_basic.yml

When you navigate to your server’s IP address in your browser, you will see the WordPress with the default theme and “Hello World” post. Good. Now, go to and create a new test. I selected the “Maintain client load” as test type and put clients from 0 to 200 of duration 30 seconds. Feel free to use different settings. Also don’t forget to run the playbook which adds the token to your WordPress site, as described before.

Using my settings and a VPS with 3 CPUs and 4GB of RAM, here’s the resulting chart, concurrent clients versus average response time:

Nginx with PHP-FPM and basic WordPress

Nginx with PHP-FPM and basic WordPress

From the chart, we observe that Nginx with PHP-FPM handles 200 concurrent connections with no problems. Link to full test. If you want to read the nginx.conf or php-fpm.conf, click here and here, respectively.

I want more! WordPress advanced, Nginx and HHVM

HHVM is a beast, no doubt about it. However, to see it shine, we will load-test it thoroughly. Run these commands:

ansible-playbook -i hosts nginx_hhvm.yml
ansible-playbook -i hosts wordpress_advanced.yml

The first playbook activates and configures HHVM. The second one installs:

  • a few large WordPress plugins (you can change them in the group_vars/webservers file), such as WooCommerce, Jetpack, WordPress SEO by Yoast, etc.
  • a free e-commerce theme Storefront
  • WooCommerce dummy data
  • Awesome WP Test dummy data

This makes the WordPress site really huge, taking a lot of memory and CPU. Let’s see how HHVM handles it:

Nginx with HHVM running advanced WordPress

Nginx with HHVM running WordPress advanced

While the response times are lower than in previous test, HHVM copes with the load quite heroically. Link to full test.

What’s next?

Can you see the beauty in this? Never ever will you have to manually install and configure all the software to be able to load test it. Just run some playbooks and you’re done!

There are a few more playbooks you can try:

  • Apache with mod_php
  • Nginx with HHVM and FastCGI caching (full-page cache)
  • WordPress with Redis for database (object) caching

Feel free to create new playbooks by combining some roles or making new ones. If you do, make a pull request so I can merge it into the repository. Thanks.

Bonus: Use htop to measure server load

Sometimes it’s not enough to see the loading time charts to be able to arrive at a correct conclusion. Observing the server load is an important factor too. You can use the htop command line utility (installed with the install-all-software.yml playbook) to measure it. As an example, take a look at Apache with mod_php versus Nginx with PHP-FPM charts and server loads:

Apache with mod_php

Apache with mod_php and basic WordPress

Apache with mod_php and WordPress basic

Apache with mod_php: 25 seconds into the test

Apache with mod_php: 25 seconds into the test

Nginx with PHP-FPM

Nginx with PHP-FPM and basic WordPress

Nginx with PHP-FPM and basic WordPress

Nginx with PHP-FPM: 22 seconds into the test

Nginx with PHP-FPM: 22 seconds into the test

As we can see, while the average response times are similar, RAM usage is whoppingly 17 times higher with Apache and mod_php than with Nginx and PHP-FPM. I know that Apache can be tweaked. On the other hand, I don’t see how you could reduce its RAM usage to be 17 times lower. No way. Don’t use Apache with mod_php.

4 thoughts on “Apache, Nginx, PHP, HHVM? Test it yourself! Ansible

  1. Mxx

    There’s something very wrong with your last claim regarding memory usage.
    Even if you are talking about only “RES” memory column, it shows that MySQL is using 78MB of ram, yet total ram shows as 46MB? That does not seem correct…


    • Good catch. To be honest, I haven’t noticed it before and the picture is not photoshopped. Htop’s help screen clarifies the RAM usage: the green bar is for currently used memory, while the yellow one is for cached memory (see attached screenshot). When PHP-FPM is not started, used memory (green bar) is about 90MB and cached 15MB. However, when I start it, used memory drops to 11MB (no load on the server), while cached goes to 140MB-150MB.

      Looks like MySQL (I’m running MariaDB) memory gets cached but I don’t know why. You could try it on your server and see if you have the same or similar results.



  2. Just passing by

    FYI, links to fail because the results are private and not shared publicly. Hope your thesis finished well.


    • Many thanks for letting me know! Just fixed that.

      Yes, finished quite well. However, the *real* test will be defending it at my university. They prefer more computer sciency papers (matrices and stuff).

      Have a nice day.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s