making bbPress (and WordPress) work better!

PHP

LoadAVG patch for more frequent logging and bug fixes

LoadAVG is a free, simple but very useful and lightweight load monitor for servers & VPS.

I encourage everyone to install it if you run a VPS because it will give you an easy snapshot of what is really going on with your server’s performance with virtually no impact otherwise.

It was written in 2006 in PHP by Doug Robbins of Labrador Data (with contributions by David Eshelman) then moved to Silversoft and finally released to the public on it’s own website under GPL – it is no longer updated, the final version came out in June 2010.

It does have a couple of bugs, some less than optimal calculations and inconsistent use of PHP, but nothing that is really a showstopper and it still works.

It’s basic design is to log every 6 minutes (10 times per hour) the server loads for 1/5/15 minute average, the amount of data transmitted and received, and how much memory has been allocated/reserved at that moment. Then it graphs the data using simple html and css on demand and also calculates high/low/averages for those numbers.

After using it for years I decided to finally fix some of the minor bugs (including the graph annoyingly doubling in height in certain conditions) and also increase the monitoring to every 3 minutes for more granular data – today’s servers/browsers can easy handle 20 datapoints per hour vs ten, and it’s important to find frequent load spikes that are even just a couple minutes long which are missed with only 6 minute logging.

So I’ve decided to share my replacement for calculations.php which has the bulk of those important fixes/changes.

view / download (save as/replace calculations.php)

Once you have that replaced, if you also want 3 minute updates instead of six, just change your crontab from */6 to */3

I’ve made the calculations.php backwards compatible with old data, so you won’t lose anything. You can replace it at anytime, but you do have to make one other tiny change after you do that, edit the display.css,
search for div.gr,div.bl,div.rd,div.wh,div.yl {
and change the width: 1px; to width: 2px;

That’s it. Then you will notice your old data is spaced apart by 1px but the new data will be a finer resolution.

Note that my update removes IE 5/6/7 support but IE9 should probably function (and maybe IE8 but why are you using IE ?).

If there is any further interest, I’ll eventually release my other improvements to this program since it’s GPL.

By the way, if you are using plesk instead of cpanel, etc. (more…)


Punycode to Unicode Converter in simple PHP

Today I needed a way to simply convert Punycode internationalized domain names to Unicode for proper display in UTF-8. I was hoping for some easy iconv magic but no such luck, PHP can’t even do part of it directly.

Googling for a bit I was only able to find one existing class that did this in pure PHP but it was well over 100k in size which was disturbing for my simple needs.

So I whittled it down to 50 lines or so and made some tweaks:

http://pastebin.com/raw.php?i=M2GzkvFf&punycode_to_unicode.php (download)

It can now handle in one function “multi-part” domains that have punycode in the sub-domain, domain and/or TLD.

ie. all the examples here work:
http://idn.icann.org/#The_example.test_names

so xn--r8jz45g.xn--zckzah is properly converted to 例え.テスト

it also works with mixed domains, ie. xn--54b7fta0cc.idn.icann.org

(you can only pass it the host part of the url, do not pass it the full URL with http or slashes or it will fail – use PHP’s parse_url to get just the host)

Note this does not do any sanitizing or other thorough checks or fixes – if you need that functionality (ie. raw user input from unknown sources) you’ll probably need the original full class over here:

http://phlymail.com/en/downloads/idna/download/


how to use eAccelerator with PHP 5.3.2 and open_basedir

If you are getting an error like this after upgrading your PHP to 5.3.2 while using eAccelerator 0.9.6

open_basedir restriction. in Unknown on line 0

Then you have to re-compile eAccelerator with this option disabled:

--without-eaccelerator-use-inode

UPDATE: ignore my suggestion below for the disabled features – sadly the user cache is completely removed in eAccelerator 0.9.6 for reasons I don’t understand, except maybe their complexity against PHP 5.3

While you are at it you might as well turn on some of the other nifty features that are turned off by default:

--with-eaccelerator-shared-memory
--with-eaccelerator-sessions
--with-eaccelerator-content-caching

And don’t forget to lock down the control panel for eAccelerator if you are in a shared environment where others can upload PHP. This should be in your php.ini settings:

eaccelerator.allowed_admin_path="/home/example/public_html/path-to-control-panel/"

(obviously change the path to your control.php file)

also recommended once you recompile with “eaccelerator-sessions” in your php.ini

session.save_handler=eaccelerator

Which will speed up session data for PHP by using common memory instead of disk.


Here comes PHP 5.3.1

As excited as I was to see the performance benchmarks for PHP 5.3.0
I am glad I resisted temptation and waited a little bit longer.

PHP 5.3.1 is right around the corner, here’s the forthcoming RC1 announcement for August 13th (next Thursday) http://news.php.net/php.internals/45230
(and note “a final release by the end of August”)

and here is an evergrowing list of fixes that will be included:
http://svn.php.net/viewvc/php/php-src/branches/PHP_5_3/NEWS

Now I just need to make sure eaccelerator is up to speed on 5.3


Reduced CPU load with PHP 5 vs PHP 4

Flickr admin have an interesting WebOps Visualizations Pool and they posted something fascinating. After switching to PHP 5 last week (I am assuming 5.2.9 from 4.4.9) they immediately noticed a reduction in CPU load. They claim the number of Apache connections per second remained persistant after the switch so it was not simply because of a reduction in traffic.
flickr-php5-vs-php4

If PHP 5.3 doesn’t go final/gold anytime soon I may just have to try the RC.


PHP 5.3 goes RC1 !

Somehow this happened so quietly that I missed it a couple days ago!
PHP 5.3 has hit release candidate 1. (download)

This is exciting because it’s going to be a “free” 15%-20% performance boost over 5.2 for many programs like WordPress and bbPress. It’s the version I am waiting for to finally replace PHP 4.4

php53-performance

What’s new in 5.3 [PDF] http://ilia.ws/files/phpquebec_php53.pdf


How to fix Internet Explorer 8 messing up WordPress themes

Here’s a mini-plugin (single line of code actually) that will prevent the new IE8 (aka Internet Explorer 8 ) from messing up your carefully crafted WordPress or bbPress design.

<?php if (strpos($_SERVER['HTTP_USER_AGENT'],"MSIE 8")) {
header("X-UA-Compatible: IE=7");} ?>

Make a mini-plugin out of it and activate.
(you might be able to put it into functions.php if that executes before other data is sent but I am uncertain – for the extremely lazy you could in theory put it as the very very first line of header.php in your template)

I also have it listed as a more formal plugin over here

For those that don’t want to install IE8 permanently to just test their webpages, this version will run in it’s own virtual box so it doesn’t affect the rest of your system:
http://start-o.xenocode.com/layers/ie8/8-0-0-0__2/ie8.exe
There’s no install, it just runs instantly from that one file.


Firefox plugin to detect when website sessions in use?

update: presenting my very first firefox plugin!
http://bbshowcase.org/firefox/website-sessions-detector.xpi
Detects if any of several session types are in use on a website. It has some room for improvement but the fact that it works at all is good start…

—–
I dislike the use of sessions to track users between web pages because it slows things down on active websites and if cookies are disabled, they can create a mess when they get appended to the URL. They are also useless across multiple servers if they aren’t stored in a common memory pool (memcached, etc).

Here are some common session names:
ASP uses “ASPSESSIONID”
PHP uses “PHPSESSID”
.NET uses “ASP.NET_SessionId”
JSP uses “JSESSIONID”
ColdFusion uses “CFID”
(let me know if you are aware of others)

I try to avoid using sessions in my bbPress plugins but there are two (Human Test & OpenID) where I’ve been too tired (aka too lazy) to come up with a complex way around them that would involve mysql tables, etc. However I try to make sure that sessions are not activated when not needed (in most cases, only during registration. But there is a popular third-party plugin that I’ve taken over ownership for (bb-topic-views) that uses sessions all the time when reading topics to prevent re-counting on multiple pages of the same topic.

(more…)


simple WordPress and bbPress security plugin to block long requests

I opened a WordPress security ticket today with something that’s been bothering me for awhile – Apache will take long URL queries up to 8k (8192) characters in length but it’s completely unnecessary and allows XSS exploits to get into WordPress and bbPress. Why not block them entirely with this scrap of a plugin (save as “_block_long_queries.php” without the quotes but with the leading underscore so it loads as early as possible and doesn’t need activation – also can’t be easily deactivated by hackers)
(more…)


in search of better tag cloud algorithms

Tag clouds (aka text clouds or word clouds) first appeared on the web around 2004-2005, most notably on Flickr. It took a little while for me to warm up to them, but once I accepted the idea, I found them useful to spot patterns that didn’t strike me previously in linear form.

(Digg used to have a great tag cloud, if anyone knows how to get it back let me know!)

WordPress and bbPress have halfway decent tag clouds, but they are missing some additional features I’ve seen on other clouds. (more…)


filemtime – the performance killer ?!

(This is NOT an issue if you just use filemtime once or twice on a page on the web. It’s just something I discovered after trying for an hour to debug a very large PHP script I wrote to process a great deal of data.)

filemtime is a “simple” function in PHP which just returns the date that a file was last modified. Sounds straightforward enough eh? Well if you are checking 1000 files, filemtime will actually DOUBLE the amount of time used if just reading the file with file_get_contents! Crazy right? Well the logic makes sense when you think about it. file_get_contents can be cached by the OS. filemtime cannot, because it assumed you need the newest, latest, uncached date fresh off the file to see if it was recently modified, even if you run the script twice, immediate after itself.
(more…)


What’s new in PHP 5.3

Besides the 15% speedup for most code (20% in some cases!) and reduced memory use, PHP 5.3 has some very interesting and handy features. Check out this really nice “slideshow” of what’s new in PHP 5.3 by Ilia Alshanetsky:

[PDF] http://ilia.ws/files/phpquebec_php53.pdf

or in plain text (somewhat garbled as formatting is lost: (more…)


PHP 4.4.9 goes RC1

I missed this from a few days ago – PHP 4.4.9 has gone RC1
The update seem trivial except for maybe that open_basedir bug with curl

22 Jul 2008, Version 4.4.9 RC1
– Updated PCRE to version 7.7. (Nuno)
– Fixed crash in imageloadfont when an invalid font is given.
(discovered by CzechSec, fixed by Pierre)
– Fixed open_basedir handling issue in the curl extension. (Stas)
– Fixed bug #37421 (mbstring.func_overload set in .htaccess becomes global). (Rui)

also see: PHP 5.3 goes RC1


PHP 5.3 feature freeze tomorrow? alpha1 July 31st?

I am uncertain if they are still on schedule but if so, this is very exciting because PHP 5.3 will realize significant speed improvements for all PHP programs on the web (15% for WordPress). They are suggesting a “stable release between mid September and mid October”.

http://sebastian-bergmann.de/archives/745-Benchmark-of-PHP-Branches-3.0-through-5.3-CVS.html

PHP 5.3 brings new features (namespaces, late statics binding, and the garbage collector, for instance) but the refactoring of the argument stack, for example, and other optimization patches have lead to a measurable performance improvement (PHP 5.3 is 1.2 times faster than PHP 5.2 for raw execution speed).

Here’s a nice “slideshow” in PDF format of what’s new in PHP 5.3 by Ilia Alshanetsky


Tricks for long or large PHP scripts

I have written some extensive scripts for PHP to do things it probably was not meant to do.
Unfortunately some get so large and take so long that I’ve researched a few tricks that may be helpful to others:

1. put error_reporting(E_ALL); at the start to make sure you find out any little errors like undeclared strings or unexpected output – very helpful for script that may take long periods of time to execute and you want to get it right the first time

2. put ini_set("max_execution_time", "300"); at the start to extend the timeout (typically 30-60 seconds in a default PHP setup). If you are on a shared server this ability may be locked out to you. 300 is an example for 5 minutes.

3. put ini_set('memory_limit','64M'); at the start to boost your memory limit for very complex arrays, etc. The default is typically 16M. Again, this may be unavailable to you on a shared server.

4. best trick of all – unbuffered output to browsers in HTML, so you can see results in realtime, even if the script takes 5+ minutes, etc.
put at the start:
ini_set('output_buffering', 0);
ini_set('implicit_flush', 1);
ob_end_flush();
ob_start();

then after each write (echo, print_r, etc) put ob_flush(); flush();
You’ll know it’s working when you see each line appear as it happens in your browser window.
Note if you have an older Apache 1.x server that uses mod_gzip this trick may not work. It should however on Apache 2.x, lighttpd, litespeed (maybe even IIS, but I dunno)


I want a php elePHPant!

Where do I get one of these adorable PHP elePHPants?
Because I just have to have one!

Here’s a family of them! http://www.dailymotion.com/video/x3lsj6_la-peluche-php_blog

ps. found some interesting links about the PHP elePHPant and it’s history:
http://www.elroubio.net/?p=elephpant
http://www.nexen.net/elephpant.php

There’s even a Flickr pool with some really amazing wilderness scenes!
http://www.flickr.com/groups/elephpants/pool/


WordPress will be 15% faster under PHP 5.3

I may finally upgrade my servers from PHP 4.4.8 to PHP 5.3 when it’s released. I’ve been sticking to PHP 4 because I don’t use anything that needs 5.x and 5.0 is slower than 4.x is some situations.

But apparently the performance has been put back into 5.3 according to this obscure posting on php.net which quotes WordPress (among others) as an example of performance improvements:

http://news.php.net/php.internals/36484


XAMPP, install PHP+Apache on Windows in 1 minute flat!

I feel like I was probably the last PHP coder on the planet to discover this but there is a wonderful open-source, free project called XAMPP that will let you install php+apache+mysql+optional bunch of other nifty things on your windows machine in under a minute flat. It almost took me longer to download it than to install!

I had made a mental note years ago of the idea of running PHP at home but it always seemed easier to just upload and execute on the remote server. I had remembered “PHPtriad” but it’s not developed anymore and the author has made their new version commercial ($$$).

XAMPP is so neat, it took another 10 seconds to turn on Curl and even has eaccelerator available if you want. It keeps everything in one folder (with sub-folders) and no registry settings so you can even keep it portable on a usb stick if so desired.

Oh and XAMPP is not just for Windows, it comes in Mac, Linux and Solaris flavours.


Is it bad when you start daydreaming about code?

I had this daydream the other day where I reversed the way WordPress/bbPress works and instead of running forwards by loading the entire engine first, then ALL the plugins and then finally the template, it worked BACKWARDS – where it first loaded the template and analyzed what was being requested, then loaded only the parts of the core it needed and then the plugins that were flagged as affecting the required output.

Way more complex, sure. But way less code execution and less plugin loading – and far, far, far more easy to cache parts of the pages while allowing other parts to be dynamic, without having to load nearly 500k of code among nearly 100 files.


better PHP programmers use “error_reporting(E_ALL);”

…at least that’s what I’ve been learning this week.

By turning that on at the start of your PHP code, you can see all the previously hidden warnings/notices that your code was still able to run with, but probably should be looked at.

Having no formal school or book training in PHP it’s been learn-as-you-go but this definitely seems to be a good step to learn. I found a few potential problems in virtually all my plugins with all errors reported.

Once you spot the problems, the two biggest helpers are going to be “isset()” and the handy if/then shorthand, ie. (variable==1 ? ‘yes’ : ‘no’ )

For example if ($_GET[‘blah’]) is a very bad method. Should be if (isset($_GET[‘blah’]))

Well I’m off to update the bigger plugins on the svn with the fixes!

ps. surprisingly the bbpress and wordpress cores have a handful of warnings when the full error reporting is turned on… could be cleaned up fairly easily…