making bbPress (and WordPress) work better!

Author Archive

CentOS 6.4 update finally supports IW10 for tcp/ip tuning

iw10Google and others have published several whitepapers showing benchmarks where tuning tcp/ip on servers can really help web browsing performance.

One of these important tweaks is known as IW10 which improves the tcp/ip congestion window by reducing ACKs (initally sending 10 packets instead of only 3) and is enabled by increasing initcwnd and initrwnd.

Unfortunately those on CentOS (one of the most popular linux production distributions) were not able to take advantage of this – until this weekend when CentOS 6.4 was shipped.

CentOS 6.3 could change initcwnd but NOT initrwnd (RWIN) which requires a kernel > 2.6.33

CentOS 6.4 is 2.6.32-358.0.1.el6 (Red Hat 4.4.7-3) so the feature has been backported

Here’s how to take advantage of IW10 on CentOS 6.4

(more…)


zopfli gzip compressor – binary for Windows

Google recently opensourced an interesting little project in C called ZOPFLI to try to get the most compression possible out of gzip which is one of the only universal compression available in browsers.

This is useful for things that are compressed once and served many times, for example jquery, stylesheets, etc. But in my testing, Zopfli only does very slightly better than 7zip’s ultra gzip compression set at mx9, and zopfli takes much much longer. It’s an interesting experiment but perhaps pointless.

If you want to try it, you can get it at:
(more…)


PHP 5.5 to be delayed so Optimizer Plus can be built-in

There is still another week of voting left before it becomes official on March 7th but the results are pretty much complete. PHP 5.5 will definitely integrate Zend Optimizer Plus opcache, built right into the distribution.

This is a good thing for the future of PHP but will require some changes for webhosts.
Since even PHP 5.4 is not widely adopted yet, we are talking 2015 for the masses.

But you don’t have to wait for PHP 5.5, you can replace APC, Xcache and eAccelerator immediately with Zend Optimizer Plus via the newly opensourced code and I’ve made a free control panel for it. Works fine on PHP 5.3 and 5.4 and you’ll typically see a minimum 10% performance boost on busy websites over those other opcode caches because of it’s extra optimization passes.


Free Control Panel for Zend Optimizer+ Opcache

In celebration of Zend open-sourcing their Optimizer Plus opcache for PHP, I whipped up a quick’n’dirty control panel just now. There doesn’t seem to be any other one? I wanted to learn and review its behavior a little more closely, to compare it to APC, Xcache and eAccelerator opcode caches.

github: https://gist.github.com/ck-on/4959032

(update: Version 0.0.4 now adds file grouping / sorting )

Code is a bit nasty in this early version because I did it in an hour. There is only limited info available about the cache state and I am still learning some of its caveats like it doesn’t actually store the file cached time but the file’s physical timestamp (probably to just see if the file has changed so it can recache it).

screenshot:
(more…)


Nginx 1.2.7 released

A new version of Nginx stable branch has appeared today with a rather long changelog:

http://nginx.org/en/CHANGES-1.2

download: http://nginx.org/en/download.html

When building manually, remember to update pcre, currently at 8.32

ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/


How to manually install the Zend Optimizer Plus Opcode Cache on PHP 5.4

Feb 13 update! Zend Optimizer+ is now officially open source so these instructions to manually install the binary are no longer needed! Go here instead: https://github.com/zend-dev/ZendOptimizerPlus/

I’ve also written a control panel for it which you can find here:
https://ckon.wordpress.com/2013/02/14/free-control-panel-for-zend-optimizer-plus/


With Zend supposedly open-sourcing their Optimizer+ opcode cache for PHP, likely replacing APC, I want to preview what I might be dealing with and any limitations/advantages.

However I did not want to install a bunch of unknown zend stuff into php slowing it down or causing conflicts, I only wanted the bare minimum and know exactly what I am installing.

Turns out it’s rather easy once you know what to look for.
(more…)


WordPress 10% faster with Zend Optimizer Plus vs APC opcode cache

As PHP internals explores replacing the APC opcode cache with an open-sourced Zend Optimizer Plus an interesting benchmark was published. It shows WordPress serving nearly 10% more pages per second than APC while using Optimizer+ (both under PHP 5.5).

apc-vs-optimizerplus


APC 3.1.14 opcode cache for PHP pulled

Following a series of memory leaks and other problems, the 3.1.14 beta release of the APC opcode cache for PHP has been recalled and removed from the pecl site by Rasmus as it’s not currently suitable for use.

I noticed the memory problem a week or two ago. I’ve carefully tested each revision in the trunk after the 3.1.13 release and have found revision 328264 to be the last good update if you want some of the subtle bug fixes since 3.1.13 without the later problems.

You can fetch a specific revision via svn like so:
svn co -r 328264 http://svn.php.net/repository/pecl/apc/trunk/

This was the changelog for 3.1.14 which has been since removed: (more…)


WordPress.com has 2200 servers

500 of them are just for MySQL with 500 million tables. More here:


APC to be replaced by Zend OptimizerPlus with PHP 5.5 release

There has been a growing discussion about this in PHP internals but now there is a semi-official wiki page explaining what is going to happen with the replacement of APC by Zend open-sourcing their OptimizerPlus opcode cache.

https://wiki.php.net/rfc/optimizerplus

Right now it’s only a “proposal” but if you look at the internals discussion, it’s almost certainly going to happen, it’s just a matter of “when” not “if”.

Given that the few people working on APC would mostly likely stop and focus instead on Optimizer+, APC is now in theory, doomed. Which is a bit crazy considering how much code is out there to take advantage of it’s user cache (which is significantly faster than say memcached on a local single server). OptimizerPlus has no user-cache shared memory support at all, it’s only an opcode cache.

Not sure why the benchmark uses “ancient” WordPress 2.1.1 but in theory this would give even the newest WordPress 3.5 a five to ten percent performance boost.


MySQL and MariaDB will start to diverge after version 5.5

MariaDB is a true open-source, drop-in binary replacement for MySQL – that is until the next version of MySQL 5.6 – MariaDB is going to go its own way for its version 10 jump after 5.5

So if you were hoping for innodb fulltext search in MariaDB 10, it’s not going to happen, but maybe in 10.1

Read more here…


PHP 5.5.0 feature freeze February 7th, 2013 – : : – APC possibly doomed

PHP 5.5.0 will reach a feature freeze is on February 7th, 2013, just two weeks away, for it’s first beta release.

There was some discussion of including the APC opcode cache into 5.5 but this will decidedly NOT happen because it would delay release by several months.

However in an interesting twist, there is now discussion of open-sourcing ZEND OPTIMIZER PLUS and including with PHP 5.5 instead.

Zend Optimizer+ is a commercial opcode cache for PHP and is slightly faster than APC but currently used on only a small percentage of servers compared to APC.

This essentially would doom APC, despite the suggestion it would remain available as there are only a few contributors to APC at this time and then would obviously switch to the zend optimizer+ sooner or later.

Remember, PHP 5.3 will reach end-of-life in March with the release of PHP 5.5.0


Some progress in making Suhosin work with PHP 5.4

News of an exciting discovery about making Suhosin work with PHP 5.4 that I missed from last week!

Apparently Jan Ingvoldstad (aka jani) has found that by disabling the multibyte string protection in Suhosin allows it to run with PHP 5.4 and perhaps keep it’s other protections.

So a somewhat serious sacrifice to at least get the remaining protections for now.
It’s better than nothing and at least a start!

Jani points out the problem lies within Suhosin trying to reference internal functions in php 5.4 that no longer exist from php 5.3 – a rewrite of rfc1867.c will be needed to affect a proper fix, but that is a far more serious undertaking that might be a long way away.

I’ve tried to contact him to get his patched source copy. If I do not hear back I will attempt to do the same modifications to the source that he describes. Then it will require a great deal of testing.

This discovery is just in time as PHP 5.3 reaches end-of-life in March.

Update: He has posted his modifications! – I will compile and test this week…


What’s coming in PHP 5.3.22 and PHP 5.3 end-of-life

PHP 5.3.21 was just released on January 17th but it was relatively minor with only a few fixes.

PHP 5.3.22 will be branched on January 30th and released on February 14th. It contains twice the fixes that look a bit more serious.

It should also be noted that PHP 5.2.23 in March 2013 will most likely be the FINAL release of PHP 5.3 as it goes into end-of-life with the release of PHP 5.5 This means that 5.3 will only receive security related bug fixes for one more year and then it’s over.

The really bad thing about this is that suhosin does not exist for the current PHP 5.4 and may likely never happen. To a lesser extent, magic-quotes no longer exists in PHP 5.4 and while it’s the subject of some mockery because of the mess it creates, it does make life a little harder for attackers.

On the plus side PHP 5.4 is measurably faster (10-20%) than PHP 5.3 and uses half the memory in many cases. The often-used silence operator (@) in PHP 5.3 has much better performance in PHP 5.4

The biggest problem most people will face changing from 5.3 to 5.4 is relatively trivial – you will receive many more warnings and deprecation notices but you can solve that for the short term by changing the error_log setting in php.ini to

error_reporting = E_ALL & ~E_NOTICE & ~E_DEPRECATED & ~E_STRICT

Just don’t ignore them forever because those things will certainly break under 5.5

- Zend Engine:
  . Fixed bug #63899 (Use after scope error in zend_compile). (Laruence)
  . Fixed bug #63762 (Sigsegv when Exception::$trace is changed by user).
    (Johannes)
  . Fixed bug #63462 (Magic methods called twice for unset protected 
    properties). (Stas)

- Core
  . Fixed bug #63943 (Bad warning text from strpos() on empty needle).
    (Laruence)

- cURL extension:
  . Fixed bug (segfault due to libcurl connection caching). (Pierrick)
  . Fixed bug #63795 (CURL >= 7.28.0 no longer support value 1 for
    CURLOPT_SSL_VERIFYHOST). (Pierrick)
  . Fixed bug #63352 (Can't enable hostname validation when using curl stream
    wrappers). (Pierrick)
  . Fixed bug #55438 (Curlwapper is not sending http header randomly).
    (phpnet@lostreality.org, Pierrick)

- Date:
  . Fixed bug #55397 (comparsion of incomplete DateTime causes SIGSEGV).
    (Laruence, Derick)

- FPM:
  . Fixed bug #63999 (php with fpm fails to build on Solaris 10 or 11). (Adam)

MySQL 5.6 gold, any week now?

Oracle has promised a “production-ready product release in early 2013” for MySQL 5.6 – which would have to be some week before the end of April?

MySQL 5.6.9 release candidate is currently available for testing.

5.6 is exciting for one interesting reason to me – it adds fulltext indexing support to innodb which was preventing widespread innodb adoption imho, since you currently are tied down to myisam for that feature. Hopefully it requires little to no code modification to keep fulltext support while just altering the db storage engine. From what I can tell it just requires a couple more my.cnf settings for innodb stopwords (for some reason doesn’t use the old myisam settings).

These tests on 5.6 from late 2012 seem to show innodb fulltext search can be three times faster than myisam under heavy load conditions.

Read more at: https://blogs.oracle.com/MySQL/entry/mysql_5_6_9_release


What’s new in PHP 5.4.9 (changelog)

PHP 5.4.9 final will be released on Thursday November 22nd, 2012

PHP 5.4 runs WordPress roughly 10% faster than PHP 5.3 and uses 30% less memory.

However suhosin unfortunately is not available yet for PHP 5.4

Here are the important bug fixes in PHP 5.4.9 (vs 5.4.8)

(more…)


All I want for Xmas is Suhosin for PHP 5.4

I recently tried PHP 5.4.8 on a production server and the memory-use decrease and performance increase was outstanding over PHP 5.2/5.3, truly impressive.

However despite the thrill, I had to roll back to PHP 5.3 after a couple days. Why? Not because of compatibility, there were only a couple of relatively easy to fix issues. Because it felt like we were running around naked without Suhosin which no longer works after PHP 5.3

Stefan Esser seems to have gone idle on updates for PHP 5.4, there was only an initial dev release several months ago (0.9.34-dev) and nothing since.

So I am calling on the “titans of industry” to make a donation to Stefan with a note encouraging him to continue the development for PHP 5.4 There seems to be a donation link here and I found another one here

If you aren’t running PHP 5.4 yet, you should be soon. Meanwhile those on PHP 5.3 who are not running the Suhosin extension should definitely install it. You can easily find php.ini tuning guides for suhosin around the web (specifically for WordPress too). It might save your server someday from being compromised by a 0-day.


An Easy Way to Speed Up MySQL FullText Natural Search on Large Tables

As your WordPress or bbPress install grows over the years, if you have a large site you will notice the search slows down quite a bit on large tables. Another problem is the quality of search results becomes poor as it has to muck through more and more posts.

It’s difficult to jump to a solution like sphinx right away, and things like the google api are not aware enough about the context of your content, so some people will code their own searches within MySQL in the meanwhile.

For example, a minimal search is done something like the following, taking advantage of MySQL’s natural search mode which takes into account the number of times the words appears and their proximity to return a score.

SELECT post_id,  MATCH (post_text) AGAINST ('some words') AS score
FROM posts WHERE MATCH (post_text) AGAINST ('some words')
ORDER BY score DESC

In my situation, using a much more complex version of that query that takes into account all posts in a topic was taking up to 12 seconds which was completely unacceptable.

So it occurred to me that MySQL also has a binary search mode that is far faster. It’s not what I wanted for the overall results because it’s relevancy can be poor, but it instead could just be used to eliminate posts that would have to be scanned. Anytime you can eliminate rows in mysql you will get a speed boost.

So simply do this instead:

SELECT post_id,  MATCH (post_text) AGAINST ('some words') AS score
FROM posts WHERE MATCH (post_text) AGAINST ('some words' IN BOOLEAN MODE)
ORDER BY score DESC

Basically that IN BOOLEAN MODE will eliminate rows very quickly that do not have either of those words in them, but the SELECT still calculates a valuable natural search score for the remaining results.

When using boolean mode you have to take some extra steps to sanitize the query because some characters have special meaning so keep that in mind but it’s trivial to do.

Eventually I took it a step further and preprocessed the search terms even further. (more…)


Fix large image attachments in WordPress

An annoying problem we have encountered with WordPress is that if an attached image is under the alloted dimensions for the post, WordPress will not attempt to further manipulate it or resave it.

This typically happens when non-tech authors upload photos from photoshop saved at the highest quality settings, but already sized down to the post width.

Only the smaller dimensions will be created, sometime leaving main images with extremely large file sizes in posts which are a big problem, especially for mobile visitors. But you really do not want megabytes of images on each page, for any kind of visitor.

So here is a filter I whipped up to catch that problem and resave the jpegs at 90% quality, progressive format, which will typically cut the size down to one-third (100k) or less with no perceptible loss of quality. It has the added bonus of stripping exif data and other meta (like hidden adobe fingerprints) from the images to reduce the size as much as possible for the most speed.

add_filter('wp_generate_attachment_metadata','fix_large_jpg');
function fix_large_jpg($metadata) {
	if (!empty($metadata['width']) && $metadata['width']<601) {
		$file=wp_upload_dir(); $file=$file['basedir'].'/'.$metadata['file']; 
		if (file_exists($file) && filesize($file)>200000) {
			$imageinfo = getimagesize($file);
			if (!empty($imageinfo['mime']) && $imageinfo['mime']=='image/jpeg') {
				$new=imagecreatefromjpeg($file);
				if ($new) {
					imageinterlace($new,1);
					imagejpeg($new,$file,90);
					imagedestroy($new);
				}
			}
		}
	}
	return $metadata;
}

This could be expanded for png but jpeg was really the main problem and easily solvable by forced resaving. Note that the re-save comes AFTER the lower sizes are created, allowing them to use the high quality source before it’s replaced.

The only hardcoded settings are the 600 pixel width which you can edit to what is used for the posts on your site, and the 200k filesize trigger which you can lower or increase as desired.


Examine the size of your MySQL data & indexes

Do you have an active server with several different programs installed that use MySQL?
Maybe even multiple wordPress and bbPress installations?

Here’s a really handy query you can throw into phpMyAdmin or otherwise to see the largest tables, their number of rows, data vs index size and the ratio of data vs index:

SELECT CONCAT(table_schema, '.', table_name) db_table,
       CONCAT(ROUND(table_rows / 1000000, 2), 'M') rows,
       CONCAT(ROUND(( data_length + index_length ) / ( 1024 * 1024  ), 1), 'M') size,
       CONCAT(ROUND(data_length / ( 1024*1024 ), 1), 'M') data,
       CONCAT(ROUND(index_length / ( 1024*1024 ), 1), 'M') indx,
       ROUND(index_length / data_length, 2) ratio 
FROM   information_schema.TABLES 
ORDER  BY data_length + index_length DESC LIMIT  30;

If you want to see a summary by database and not tables, try this:

SELECT count(*) tables,  table_schema dbase,
 concat(round(sum(table_rows) / 1000000, 2), 'M') rows,
 concat(round(sum(data_length + index_length) / (1024 * 1024 ), 1),'M') size,
 concat(round(sum(data_length) / (1024*1024 ), 1), 'M') data,
 concat(round(sum(index_length) / (1024*1024), 1), 'M') indx,
 round(sum(index_length) / sum(data_length), 2) ratio 
FROM information_schema.TABLES 
GROUP BY table_schema ORDER BY sum(data_length + index_length) DESC;

Thanks to: http://www.mysqlperformanceblog.com/2008/02/04/finding-out-largest-tables-on-mysql-server/ and http://www.mysqlperformanceblog.com/2008/03/17/researching-your-mysql-table-sizes/ (with a few tweaks by me)


Firefox Page Timer Lite (shows page load time)

One of the features I really liked in FasterFox plugin was the simple page timer which sat in the statusbar and just showed you how long the page you were viewing took to be loaded.

However they stopped updating that extension and it made my Firefox unstable. There is a FasterFoxLite but it also causes my Firefox to crash sooner or later for some reason.

So after some mucking about, I was able to make a super light version of the same page timer for Firefox.

Download Page Timer Lite for Firefox (pagetimerlite.xpi)

It works in all versions of Firefox, even 15 beta which I am using now, but you MUST install status-4-ever extension first, because my plugin requires the statusbar which was removed in Firefox 4 and newer.

This is because I do not know how to make addon bar extensions yet, someone will have to show me or I’ll eventually find a simple example for converting statusbar to addon bar.


SPDY indicator for Firefox 11+

Firefox 11 will be supporting the SPDY protocol for faster webpages. (Firefox 11 beta seems quite stable and I have been using it for a month now.)

Unfortunately there is no clue to when SPDY is actually working in Firefox because the only way to know is to examine the response headers for X-Firefox-Spdy.

However Cheng Sun has come to our rescue by whipping up a SPDY indicator extension inspired by a similar plugin for Chrome that shows an icon in the address bar.

SPDY is not on by default in Firefox 11, you have to go into about:config and change network.http.spdy.enabled to true.

Then watch for green lightning bolt the extension adds to the url address bar by visiting most google services or now Twitter.

Now available on mozilla.org! Here is a proper XPI version of the extension

Here are some SPDY enabled sites to test it on
(more…)


WordPress nearly 10% faster on PHP 5.4 than 5.3

The speed improvements in PHP 5.3 over PHP 4 were significant.

Now comes news of some additional improvements to squeeze out nearly 10% more performance:

WordPress desperately needs every ounce of help it can get as it grows more and more sluggish with every release, so this is helpful news.


Where is the Commercial section for WordPress Plugins?

WordPress Theme developers have a sanctioned place to sell themes:

http://wordpress.org/extend/themes/commercial/

so where is http://wordpress.org/extend/plugins/commercial/ ?

I want to sell “Commercially Supported GPL Plugins”

Some of my plugins might be worth $1 to for-profit blogs/forums.

The donation model clearly doesn’t work http://bbshowcase.org/donate/

Maybe it’s time for Matt to make the equal of an “app store” for WordPress/bbPress?