making bbPress (and WordPress) work better!

The WordPress/bbPress Performance Limit

I was looking at some of the “most requested features” for WordPress and I noticed that “Make It Faster” was pretty high on the list.

I’ve learned a lot about WordPress code over the past few years but even more so after working with it’s forum spinoff bbPress. A few things about WordPress/bbPress performance became obvious when I starting noticing that another plugin developer’s forum running SMF was running considerably faster than bbPress and WordPress on the same poorly performing host. I took SMF apart to understand why and here’s what I found:

WordPress cannot (in theory) be made much faster, unless entire pages are made static with a caching system and it’s core is not even loaded for the visitor. It’s simply a limit of WordPress’s architecture and would require an entire rewrite to change. If you have mysql caching turned on and an opcode cache (apc, eaccelerator, xcache, etc) that’s the most you can (easily) do before solutions like wp-cache.

The reasons why include:

1. dozens upon dozens of files that have to be included before any page rendering can begin. Other blog/cms software tries to keep this number down but in WordPress it’s almost runaway. Even an opcode cache must check the timestamp on the source files so the filesystem is kept constantly busy – if it’s not in the filecache, there is a performance hit. If it’s being rendered on a different server from where the files are stored (like on Dreamhost and others) there is even more of a performance hit.

2. Each plugin is a separate file that is not a direct mod on the source but an additional include that is required to load and more code to execute. All plugins are loaded regardless of what kind of page is being rendered. See above why this is a problem. Your WordPress setup can easily be requiring near 100 files to load and 500k of code to execute before a single character is output.

3. All output goes through a language translation hook. Even if you are using the default language this will always be slower than direct output.

4. Output is done in bits and pieces with typically no buffering done (ala ob_start) to accelerate and smooth this process. Any programmer should know from experience that outputting a few characters at a time, 1000 times is far slower than storing them all up in a string and dumping the few thousand all together at once at the end.

All these weaknesses can also be found in bbPress (the forum software that is based on WordPress architecture). Maybe oneday a great mind will come up with a magic, backwards compatible speedup solution without rewriting the core but I doubt it will be any time soon.

Despite these weaknesses, on a single dedicated or VPS server with proper resources, proper opcode, mysql and file caching, WordPress (and bbPress) can give just about any other blog/cms software a run for it’s money. The problems begin when these performance solutions are simply not available on a shared hosting server.

ps. I’ll also note that I am not abandoning WordPress or bbPress anytime soon. The benefits of their plugin architecture far outweigh the server performance requirements. Another method for speedup that I did not mention is that replacing apache with lighttpd or litespeed which can also be a big benefit but of course that is much more work than simply turning on the mysql cache and installing an opcode cache.

15 responses

  1. Interesting post.

    August 19, 2007 at 9:34 am

  2. Great post, but a few things I’d like to point out. APC, an opcode cache available through pecl, allows you to turn off file stating so that it doesn’t need to check file timestamps on every load which should overcome many of the issues associated with including many files. This is really only useful on production servers where code changes aren’t made, but I would assume that other opcode caches can do this as well.

    Also, I’d like to point out that if the files that are being rendered are stored on a different server as you say Dreamhost does, that is a limitation of Dreamhost’s architecture, not WordPress although I do agree this exacerbates the file include problem with WordPress since each requests includes some amount of network latency.

    August 20, 2007 at 10:45 am

  3. I appreciate the quality feedback Andrew!

    Turning off file checking for an opcode cache is virtually impossible for a plugin developer 😉 Never have had the pleasure of running a perfectly static production machine… must be nice, LOL! But it would be interesting to test turning off the file-check and see what happens performance-wise. I’ll have to see if eaccelerator has a buried option to do that, never did manage to get APC to run on dreamhost though people say it’s possible.

    The way Dreamhost runs is not so unique – it’s unfortunately somewhat common across low-cost shared hosting that oversells its resources. But it is indeed a problem for wordpress/bbpress only because other software doing similar functions is able to (mostly) overcome the problem (Simple Machines Forum).

    I had this daydream the other day where I reversed the way wordpress/bbpress works and instead of running forwards by loading the entire engine first, then the plugins and then finally the template, it worked BACKWARDS – where it first loaded the template and analyzed what was being requested, then loaded only the parts of the core it needed and then the plugins that were flagged as affecting the required output. Way more complex, sure. But way less code execution and less plugin loading – and far, far, far more easy to cache parts of the pages while allowing other parts to be dynamic.

    August 20, 2007 at 10:58 am

  4. How does SMF overcome the same problem? I imagine it either being one of two solutions, less includes, or some type of pre-compilation.

    Just like you though, I have thought about similar ways of improving “plugin” performance because many blog/cms software it seems follows the same execution flow, i.e. load some core files and then every plugin file. Although, I think Drupal alleviates this problem a little by first determining which plugins are needed per page so only those plugins are loaded (not positive, could be wrong). One solution I always thought would be interesting would be if the platform, in this case WordPress, ran off a single file and that file was recompiled each time a change was made to the system, say adding a new plugin.

    For this reason, and a few others, I have actually started working on a simple performance focused MVC framework.

    August 20, 2007 at 1:56 pm

  5. SMF solves the problem by keeping file count down, template count down, and plugins are actually core hacks (diffs) so there typically are no extra files added. They also do virtually everything in the core. I have a feature rich SMF setup running elsewhere with a whopping 3 “plugins” where bbpress needs three dozen to even come close but still doesn’t have things like forum permissions.

    I’ve written a “fastload” concatinator plugin that glues all of the bbpress includes together and then all of the bbpress plugins together. It helps somewhat on hosts like dreamhost but there is still alot of code to execute before any page rendering starts. You can see the effect here:
    and compare to SMF on the same setup:
    There’s always a massive load on that server which is laughable but a good testing grounds.

    If I didn’t have to work for a living and had time on my hands, I’d try to build a bbpress/wordpress plugin compatible system that worked backwards like my other post 😉

    But something does have to be done about plugins because the core design is meant to be as empty of features as possible so you end up with two dozen, sometimes three dozen plugins to round out a feature rich website. Even worse, the plugins often have to work twice as hard then if they had been integrated into the core design.

    Compare SMF’s file loading to render the front page vs bbPress front page:
    SMF 1.1.3
    files for front page: 8 files, 338.45 Kb

    index.php : 14.31 Kb
    Settings.php : 3.71 Kb
    Sources/QueryString.php : 18.84 Kb
    Sources/Subs.php : 132.37 Kb
    Sources/Errors.php : 15.94 Kb
    Sources/Load.php : 92.47 Kb
    Sources/Security.php : 29.88 Kb
    Sources/MessageIndex.php : 30.93 Kb

    bbPress 0.8.3 alpha
    files for front page: 40 files, 437.15 Kb

    index.php : 0.44 Kb
    bb-load.php : 0.46 Kb
    config.php : 2.28 Kb
    bb-settings.php : 7.14 Kb
    bb-includes/db-mysqli.php : 12.77 Kb
    bb-includes/functions.php : 71.73 Kb
    bb-includes/wp-classes.php : 3.37 Kb
    bb-includes/classes.php : 37.98 Kb
    bb-includes/formatting-functions.php : 8.73 Kb
    bb-includes/template-functions.php : 59.12 Kb
    bb-includes/capabilities.php : 13.15 Kb
    bb-includes/cache.php : 8.52 Kb
    bb-includes/deprecated.php : 8.17 Kb
    bb-includes/wp-functions.php : 36.06 Kb
    bb-includes/kses.php : 17.65 Kb
    bb-includes/l10n.php : 1.76 Kb
    bb-includes/bozo.php : 10.1 Kb
    bb-includes/akismet.php : 8.14 Kb
    bb-includes/default-filters.php : 4.45 Kb
    bb-includes/script-loader.php : 9.2 Kb
    bb-includes/compat.php : 0.57 Kb
    my-plugins/_bb-benchmark.php : 4.79 Kb
    my-plugins/_cookie-year.php : 0.92 Kb
    my-plugins/bb-polls.php : 21.04 Kb
    my-plugins/bb-signatures.php : 11.52 Kb
    my-plugins/report-post.php : 3.92 Kb
    my-plugins/user-topics-to-favorites.php : 0.59 Kb
    my-plugins/bb-topic-views.php : 5.91 Kb
    my-plugins/my-views.php : 10.13 Kb
    my-plugins/notification.php : 2.26 Kb
    my-plugins/my-views-started-participated-topics.php : 1.52 Kb
    my-plugins/my-views-most-least-views.php : 2.7 Kb
    my-plugins/plugin-browser.php : 21.72 Kb
    my-plugins/my-views-installed-available-plugins.php : 6.61 Kb
    bb-includes/pluggable.php : 8.86 Kb
    bb-includes/locale.php : 6.91 Kb
    bb-templates/kakumei/front-page.php : 3.03 Kb
    bb-templates/kakumei/header.php : 1.6 Kb
    bb-templates/kakumei/logged-in.php : 0.21 Kb
    bb-templates/kakumei/footer.php : 1.13 Kb

    August 21, 2007 at 9:05 am

  6. Interesting. How extensible is SMF and how easy is it to extend SMF? My initial reaction to your description that all plugin are core hacks leads me to think that its not as simple to extend as WordPress/BBPress. I have always felt that WordPress’ extensibility is part of the reason why it gets so “bloated.” So I assume they have taken the same approach in BBPress. I ask because even though I care alot about performance, flexibility is sometimes more important, and for the purposes of WordPress at least, flexibility is key.

    Regarding having to work, I totally understand what you mean. I myself have a 9-5 job at a large internet media company, so I know the frustration of not being able to do anything “fun” :-p, although it would be truly interesting to see how much your backwards executing plugin would work. It would definitely be adding quite a bit of complexity to the final output.

    August 21, 2007 at 11:08 am

  7. There is definitely a learning curve to making SMF “packages” aka “plugins”. There are lots of people and guides willing to help though. Not that many extensions are actually needed in the first place. It’s been developed for over 5 years, even more if you consider their YaBB roots.

    But by being able to patch the core directly, they are able to do tricks that bbpress plugin developers can only dream of.

    For example I wanted a smf forum to have a viewable front page where all the forums, descriptions and latest posts are visible (and clickable). But I didn’t want unregistered/non-logged-in users to even be able to enter the forum, just get a login/register prompt for any click. While SMF has some well thought out permissions natively and can do “private” forums easily, it didn’t have that ability to show but not allow entrance.

    But one quick search on the SMF mod section found an easy answer:
    Take a look inside the zip at HideBoard.xml
    The simplicity is breathtaking. Just set each board at a minimum user level and it’s all set. They can look but not enter.
    (the comments and insertion code is twice as long as the actual mod code itself!)

    Note how the author didn’t even test past 1.1.1 but it still works on 1.1.3 So there is still some minor version flexibility in their technique too.

    Their 2.0 is in beta test now. I can’t imagine what more they added.

    ps. my backwards executing idea wouldn’t be a plugin – it would have to be a replacement core – hence the near improbability of me producing anything but a simple proof of concept someday

    August 21, 2007 at 11:32 am

  8. By the way Andrew, I really like your wordpress page but two suggestions to try to help visitors:

    1. Text is hard to read for the contrast challanged (like me). You might want to darken the post text a bit if it’s easy in the CSS.

    2. Those green bars on the side are pretty but screaming for attention, drawing the eyes constantly away from the main text in the posts as one reads. Perhaps you can knock their brightness/neon-ness down a few notches and maybe as a bonus make them light back up when that column is moused-over (just an idea!) ooh maybe default is black & white and column moused-over = green again. Eh I’m over thinking it. Fix #1 is far more important to my poor eyes.

    I just have sensitive eyes so these comments may just be me and not the average visitor…

    August 21, 2007 at 11:45 am

  9. Hey, thank you very much for the feedback. I will change around the font color later today when i get a chance. If you don’t mind could take a look after I do so, and let me know what you think? Also, I will try to tone down the side bars as you suggested.

    I found this theme from one of the WordPress theme sites, but haven’t had too much time to work on the smaller details. I definitely agree that it is a bit on the bright side. Also, since I am new to the blogging scene, I don’t really have too many blogs listed under my blogroll, aka friends. Would you mind if I added yours to it?

    August 21, 2007 at 2:37 pm

  10. It’s a nice theme indeed just needs a slightly darker font.

    You’re welcome to add me to your blogroll, I wasn’t aware people typically asked permission first but it’s very polite of you 😉

    August 22, 2007 at 1:03 am

  11. Hehe, I wasn’t sure if there was any etiquette involved, but thought I should ask just in case 🙂 Anyways, I made the text color darker, could you take a look and let me know what you think? Btw, I commented on your post for mouse suggestions, but it never appeared.

    August 22, 2007 at 11:18 am

  12. Is it darker? I guess so. My eyes seem to adapt to it a little easier. Thanks!

    I have no idea why your mouse comment never appended, out of my control since it’s run on but that’s not good to hear… wonder if they have a bug…

    Did you suggest a mouse?

    August 22, 2007 at 4:49 pm

  13. I hope the font color is ok, I can try a bit darker if you think its still too hard. Unfortunately, I can’t go to complete black on white without redoing the entire them to go with it. But yes I did suggest a mouse. I recommended a Microsoft Intellimouse Explorer. It doesn’t have everything you need, but its by far the most comfortable most I have ever used, and I am pretty picky about mice myself since I am a competitive gamer.

    August 22, 2007 at 7:29 pm

  14. Yeah the font color seems fine now.

    I can’t stand microsoft mice, especially the tilt scrollwheels.

    I need an easy to click center button (mousewheel) and none of the current microsoft mice have them.

    August 22, 2007 at 9:33 pm

  15. ブランドコピー 代引き 国内発送 brandsecret-ブランドシークレット-

    April 18, 2015 at 6:14 am

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s