making bbPress (and WordPress) work better!

Latest

Even faster APC PHP cache by turning off file stat

I’ve recently had to abandon using eaccelerator because they are just not keeping up anymore with PHP releases and they even removed variable data cache support just to get PHP 5.3 compatibility. It’s sad because they were the fastest and most stable for years.

So I have been testing xcache and APC instead.

One trick I use on large/active servers for an additional performance boost is to turn off the file stat feature on the PHP opcode cache. This makes it stop checking the timestamp on each and every PHP file executed. WordPress for example has over 100 php files totaling nearly 3 megabytes on a typical install.

You’d use ONE of these directives in php.ini depending which opcode cache you use:
Read the rest of this page »

(whatever happened to) MySQLTuner 2.0

MySQL DBAs are amazingly smart, they really impress me.

It’s really hard to figure out the right way to make MySQL perform better. Webservers and PHP have been improved to the point where mysql is often becoming the bottleneck.

I just finished porting over MySQL-Report into bbPress as a plugin. It was originally written by Daniel Nichter in Perl and then Munroe Richard converted it to PHP for Drupal, so I “borrowed” a great deal of his code to save some time. I am not that thrilled with the original code quality but I am feeling too lazy right now to rewrite it, and it works well enough.

Other analysis tools I use are tuner-primer and MySQLTuner (v1.2) The mysql summary from aspersa (now in the Percona Toolkit) is also somewhat helpful but not as useful.

For realtime tracking the updated version of MyTop from Mark Grennan (originally by Jeremy Zawodny) is somewhat useful on busy servers.

Now, does anyone know whatever happened to MySQLTuner 2.0 ?
Seems like they stopped working on it in 2010

Nice slideshow on it from Sheeri Cabral
http://technocation.org/files/doc/2010_10_mysqltuner.pdf

Video presentation with that slideshow
Read the rest of this page »

How to get an element’s TRUE position in javascript, including borders in Firefox

I’ve seen a bunch of scripts around the web to get an element’s (aka object) actual/absolute position on a page in javascript. Virtually all of them are wrong, for one very good reason – if you have borders on an element in ANY version of firefox, the position will be off by the border width.

I haven’t seen any script attempt to deal with borders.

This is because Firefox uses a broken “inner” model, regardless if you are in standards vs quirks mode, that was based on trying to mimic IE6 – the only other game in town when Firefox was evolved. But ironically, even IE has moved on to including borders since version 7/8 while Firefox remains the same from version 1 through 5. Good for backwards compatibility I guess but a headache for web developers.

So in plain English, in Firefox, margins count towards offsetLeft/offsetTop but NOT borders! It’s the only “html5″ browser in existence right now that doesn’t include borders.

See these bug reports from SEVEN years ago, still unfixed:
https://bugzilla.mozilla.org/show_bug.cgi?id=255754
https://bugzilla.mozilla.org/show_bug.cgi?id=387922

Ignore where people claim their solution works – if it doesn’t include border calculations, it doesn’t work, they are misguided.

I used to have a very short and sweet function to grab an object’s true position on the page. (do not use this one)

function XY(o) {var z=o,x=z.offsetLeft||0,y=z.offsetTop||0; while(z=z.offsetParent) {x+=z.offsetLeft||0; y+=z.offsetTop||0;} return {x:o.X=x,y:o.Y=y};}

(it returns an object with .X .Y added (left and top) but also appends them to the original object for convenience)

However do not use the above script, it’s incomplete. Not only does it not properly deal with scrolled objects (div, iframe, etc.) it does not handle borders in firefox.

Sadly there is no fast & clean way to get border width in firefox, you have to use computedstyles from the best I can figure out, which must be relatively slow because of how it traverses stylesheets (if you know a faster way, please let me know).

So this (unfortunately larger) function I’ve come up with does finally produce identical results across Chrome, Firefox and IE8

function XY(o) {
var z=o, x=0,y=0, c; 
while(z && !isNaN(z.offsetLeft) && !isNaN(z.offsetTop)) {        
c = isNaN(window.globalStorage)?0:window.getComputedStyle(z,null); 
x += z.offsetLeft-z.scrollLeft+(c?parseInt(c.getPropertyValue('border-left-width'),10):0);
y += z.offsetTop-z.scrollTop+(c?parseInt(c.getPropertyValue('border-top-width'),10):0);
z = z.offsetParent;
} 
return {x:o.X=x,y:o.Y=y};
}

But even though it’s larger, it’s still far better than turning to an 80k library like jquery. Note my use of “window.globalStorage” which is a trustworthy way of detecting anything based on the real gecko engine (since FF 2+) while ignoring all other browsers. No user-agent sniffing needed. Can’t use the navigator property – it’s useless because Chrome will also return “Gecko”.

LoadAVG patch for more frequent logging and bug fixes

LoadAVG is a free, simple but very useful and lightweight load monitor for servers & VPS.

I encourage everyone to install it if you run a VPS because it will give you an easy snapshot of what is really going on with your server’s performance with virtually no impact otherwise.

It was written in 2006 in PHP by Doug Robbins of Labrador Data (with contributions by David Eshelman) then moved to Silversoft and finally released to the public on it’s own website under GPL – it is no longer updated, the final version came out in June 2010.

It does have a couple of bugs, some less than optimal calculations and inconsistent use of PHP, but nothing that is really a showstopper and it still works.

It’s basic design is to log every 6 minutes (10 times per hour) the server loads for 1/5/15 minute average, the amount of data transmitted and received, and how much memory has been allocated/reserved at that moment. Then it graphs the data using simple html and css on demand and also calculates high/low/averages for those numbers.

After using it for years I decided to finally fix some of the minor bugs (including the graph annoyingly doubling in height in certain conditions) and also increase the monitoring to every 3 minutes for more granular data – today’s servers/browsers can easy handle 20 datapoints per hour vs ten, and it’s important to find frequent load spikes that are even just a couple minutes long which are missed with only 6 minute logging.

So I’ve decided to share my replacement for calculations.php which has the bulk of those important fixes/changes.

view / download (save as/replace calculations.php)

Once you have that replaced, if you also want 3 minute updates instead of six, just change your crontab from */6 to */3

I’ve made the calculations.php backwards compatible with old data, so you won’t lose anything. You can replace it at anytime, but you do have to make one other tiny change after you do that, edit the display.css,
search for div.gr,div.bl,div.rd,div.wh,div.yl {
and change the width: 1px; to width: 2px;

That’s it. Then you will notice your old data is spaced apart by 1px but the new data will be a finer resolution.

Note that my update removes IE 5/6/7 support but IE9 should probably function (and maybe IE8 but why are you using IE ?).

If there is any further interest, I’ll eventually release my other improvements to this program since it’s GPL.

By the way, if you are using plesk instead of cpanel, etc. Read the rest of this page »

Windows can’t find your USB devices? Make sure USBSTOR.INF is still there!

I spent a few days trying to figure this out, and apparently all modern versions of Windows suffer from it, yet it’s barely/poorly documented anywhere:

If your usb devices are no longer recognized by Windows, no matter how many times you try to install or update the drivers, go look in c:\windows\inf\ (make sure you can see hidden and system files) and see if you cannot find:

usbstor.inf
usbstor.pnf

All you have to do is restore/replace them and bingo, everything will work again, I didn’t even have to reboot.

Windows2k (2000), Windows XP, Vista and even Windows 7 suffer from this problem.

Basically my external mass storage (bulk storage) devices suddenly were not being seen when I plugged them in. Reinstalling the drivers was useless and just ended up says that windows could not find the drivers needed.

Some program uninstalls those two when you uninstall it, I am sure what, why or how but it’s yet another Windows mystery solved.

Firefox 4 Mobile – first to support WYSIWYG Editors (contentEditable/designMode)

Firefox 4 Mobile was just released today and I have some incredible news that no-one else seems to have noticed (yet).

It’s the first mobile browser to support contentEditable!

What does this mean? Well TinyMCE, CKEditor (FCKeditor) and other rich, visual editors will now be able to work on your android cellphone (and a very few Nokia devices like the N900)

Check it out here: http://firefox.com/m

If you have Windows, Mac, or Linux on your desktop, you can see what the fuss is about through the developer versions:

http://ftp.mozilla.org/pub/mozilla.org/mobile/releases/4.0/

No other mobile browser to date does this trick. Not mobile Safari for the iPhone, not mobile Chrome for Android, not Opera Mini/Mobile, not IE Mobile. See my post from last week about this lack-of-support problem..

List of mobile browsers that support contentEditable (designMode) WYSIWYG

As of March 2011, this is a trick question – NO mobile browser even attempts to support contentEditable/designMode (aka WYSIWYG, rich HTML textareas) on portable devices like cellphones, smartphones, iPad, iTouch, etc.

Desktop browsers have been supporting it since IE 5.5 in July 2000 over a decade ago. The Mozilla 1.3 (Gecko) engine implemented it in late 2002, so Firefox 1.5 had it in May 2003.

Also see: http://caniuse.com/contenteditable

So which one of these will be the first mobile browsers to have it?

Microsoft IE for Mobile
Google Android (Chrome/webkit)
Apple Safari Mobile (webkit)
Opera Mobile
Opera Mini
Firefox Mobile (Fennec)

However I have no information on the brand new Internet Explorer Mobile 9 (aka IE9 Mobile) which was only demonstrated February 2011 by Microsoft.

Since Microsoft invented content-editable, there is a chance they ported it into the mobile IE9 from their newest Trident engine. If anyone can confirm or deny this, please let me know? Thanks!

How web designers can test websites in IE9 on Windows XP

I’m a diehard Windows XP user and have no desire to upgrade to Vista or Windows 7. But this presents a problem now that Internet Explorer 9 has been officially released and in their (endless lack of) wisdom, Microsoft has decided not to allow it to run on Windows XP, to try to get people like me to upgrade.

So here’s how you can test your web designs on IE9 and make sure that everything is okay.

(the #1 problem I have found is that IE9 does NOT like or deal well with special IE6 fixes that are left over that were either ignored or did no harm in IE7 or IE8 – I will give you a solution for that following this)

First downloaded the free and legal Microsoft Windows 7 enterprise trial ISO (2.2GB)
microsoft.com x86fre_enterprise_en-us_EVAL_Eval_Enterprise-GRMCENEVAL_EN_DVD.iso
(you only need the 32bit version, it will run faster, if it won’t download it’s because of the refer, copy the link and paste it directly in your browser – download is speed limited so it will take at least 20 minutes)

Next install the free VMWare Player into Windows XP
http://download3.vmware.com/software/vmplayer/VMware-player-2.5.5-328052.exe
( or http://www.vmware.com/download/player/download.html you might actually need version 3+ for windows 7)

Last, use one of the free vmx generator tools to make a configuration for VMplayer ready for Windows 7 and set the Windows7 ISO as the CD drive

http://www.easyvmx.com/supersimple.shtml
(choose Vista, not Vista 64 – also, Windows7 needs a TON of space so choose 12GB at a minimum, I did 8GB and it’s already maxed out with nothing else installed – the file starts off small but will grow as needed)

Edit the vmx file and add these lines (removing any existing ide1:0 lines)

ide1:0.present = "TRUE"
ide1:0.deviceType = "cdrom-image"
ide1:0.fileName = "7600.16385.090713-1255_x86fre_enterprise_en-us_EVAL_Eval_Enterprise-GRMCENEVAL_EN_DVD.iso"

It should take 30 minutes to do all this, except maybe the ISO download which is huge and takes quite awhile.

The great thing about the enterprise trial is it lasts for 60 days, then you can “rearm” (google it) for a couple more times, but even when it’s expired, you can use it for an hour before auto-shutdown which is plenty of time to test pages in IE9

Okay now I promised a trick to help deal with IE9 – here’s my solution:

Most javascript that detects IE doesn’t detect the version, just if it’s IE or not. For example: isIE=/blah/; What we really need is something to replace that which is short and sweet and accurate.

So here’s what I came up with:

isIE=window.opera?0:parseInt((navigator.userAgent.match(/MSIE (\d+)/) || [0,0])[1]);

Then just replace any instance of

if (isIE) {blah}

With this:

if (isIE && isIE<9) {blah}

Hope that helps!

Punycode to Unicode Converter in simple PHP

Today I needed a way to simply convert Punycode internationalized domain names to Unicode for proper display in UTF-8. I was hoping for some easy iconv magic but no such luck, PHP can’t even do part of it directly.

Googling for a bit I was only able to find one existing class that did this in pure PHP but it was well over 100k in size which was disturbing for my simple needs.

So I whittled it down to 50 lines or so and made some tweaks:

http://pastebin.com/raw.php?i=M2GzkvFf&punycode_to_unicode.php (download)

It can now handle in one function “multi-part” domains that have punycode in the sub-domain, domain and/or TLD.

ie. all the examples here work:
http://idn.icann.org/#The_example.test_names

so xn--r8jz45g.xn--zckzah is properly converted to 例え.テスト

it also works with mixed domains, ie. xn--54b7fta0cc.idn.icann.org

(you can only pass it the host part of the url, do not pass it the full URL with http or slashes or it will fail – use PHP’s parse_url to get just the host)

Note this does not do any sanitizing or other thorough checks or fixes – if you need that functionality (ie. raw user input from unknown sources) you’ll probably need the original full class over here:

http://phlymail.com/en/downloads/idna/download/

a quarter million people obliterated

65 years ago this week the United States obliterated over a quarter million people from the earth, mostly civilians, in not one but two decisions of insanity. Scientists begged the government to instead do an demonstration offshore in Japan to warn what the atomic bomb would do to them. May we be treated with more humanity than we have treated others.

Another Performance Regression in WordPress 3 Future

MySQL 5 is proven to be slower than MySQL 4, and WordPress doesn’t require any MySQL 5 specific features to operate.

But they announced today that WordPress 3.2 will require MySQL 5.0.15

MySQL 5 has performance improvments mostly for InnoDB which WordPress does not use. MySQL 4 is faster at MyISAM which the is the more common db format (and used by WP).

Also, they are insisting that servers run PHP 5.2 minimum for WP 3.2

While PHP 5.2 is faster than PHP 4.4 (and PHP 5.3 is measurably faster than 5.2) it’s not very hard at all to support PHP 4.4 In fact I’m sure they are going to have to go out of their way to force PHP 5 to be required by actually REMOVING simple code that helps PHP 4.4 already in WP.

So why not keep software flexible when it’s easy? The reality is this is probably to help keep reports of security problems with WordPress down by forcing people to keep their servers up to date.

Personally I don’t think that is WordPress’s business or responsibility but I guess people have a choice which software they want to use, especially when it’s free.

ps. You know what else just dawned on me – Matt has data on a million servers that run WordPress when they “phone home”. This doesn’t bother anyone?

undocumented WordPress.org Plugin API for plugin authors

Did you know you can get info about plugins in WordPress.org’s extend section in json or xml format? Virtually all the info about the plugin is available in machine readable format.

It’s as simple as adding the plugin’s stub name to the end of this url like so:

xml: http://api.wordpress.org/plugins/info/1.0/hello-dolly.xml
json: http://api.wordpress.org/plugins/info/1.0/hello-dolly.json
php serialize: http://api.wordpress.org/plugins/info/1.0/hello-dolly.php

You can request partial info or info on multiple plugins via the full API, which is not documented anywhere except this message from last year:
http://comox.textdrive.com/pipermail/wp-hackers/2009-January/023505.html

PHP code example:
http://wordpress.pastebin.com/raw.php?i=7Ji8rD2P

It was written by DD32 (aka Dion Hulse)

Missing from version 1.0 API are a few finer details like “downloads yesterday” & “last week” but that can always be derived from the graph data like so:

http://wordpress.org/extend/stats/plugin-xml.php?slug=hello-dolly

Graphing was just added to the bbPress.org side but the full API does not appear to be available (yet).

Follow

Get every new post delivered to your Inbox.