making bbPress (and WordPress) work better!

bbPress

An Easy Way to Speed Up MySQL FullText Natural Search on Large Tables

As your WordPress or bbPress install grows over the years, if you have a large site you will notice the search slows down quite a bit on large tables. Another problem is the quality of search results becomes poor as it has to muck through more and more posts.

It’s difficult to jump to a solution like sphinx right away, and things like the google api are not aware enough about the context of your content, so some people will code their own searches within MySQL in the meanwhile.

For example, a minimal search is done something like the following, taking advantage of MySQL’s natural search mode which takes into account the number of times the words appears and their proximity to return a score.

SELECT post_id,  MATCH (post_text) AGAINST ('some words') AS score
FROM posts WHERE MATCH (post_text) AGAINST ('some words')
ORDER BY score DESC

In my situation, using a much more complex version of that query that takes into account all posts in a topic was taking up to 12 seconds which was completely unacceptable.

So it occurred to me that MySQL also has a binary search mode that is far faster. It’s not what I wanted for the overall results because it’s relevancy can be poor, but it instead could just be used to eliminate posts that would have to be scanned. Anytime you can eliminate rows in mysql you will get a speed boost.

So simply do this instead:

SELECT post_id,  MATCH (post_text) AGAINST ('some words') AS score
FROM posts WHERE MATCH (post_text) AGAINST ('some words' IN BOOLEAN MODE)
ORDER BY score DESC

Basically that IN BOOLEAN MODE will eliminate rows very quickly that do not have either of those words in them, but the SELECT still calculates a valuable natural search score for the remaining results.

When using boolean mode you have to take some extra steps to sanitize the query because some characters have special meaning so keep that in mind but it’s trivial to do.

Eventually I took it a step further and preprocessed the search terms even further. (more…)


bbPress still deleted from Wikipedia

5/26 update: a noble group of folks have taken up the effort to restore and rebuild the page on wikipedia – I’ll leave this post for posterity…

bbPress is approaching 10,000 active installs and millions of users by my (heavily researched) estimation and yet it’s still deleted from Wikipedia. However BuddyPress which definitely has less than a thousand active installs, features it’s own page. Go figure.

Wikipedia has pages on ultra obscure fonts that aren’t even in use anymore from ancient Windows 3.1 applications, but a dedicated bbPress page is excessive to the editors?


First ever TalkPress powered website?

Unless I missed an announcement somewhere,
I think I’ve found the first ever TalkPress powered website:

http://ask-the-expert.health.com/

Looks really nice. I believe it quietly went live in the middle of March.

TalkPress is essentially Automattic hosted bbPress forums,
just like what WordPress.com is to WordPress blogs.
(it uses WordPress.com global logins/cookies)

Matt first mentioned TalkPress a year ago.

update: Yup, I did indeed miss an announcement, here it is:
http://watershedstudio.com/blog/2009/03/healthcom-on-talkpress-vip/


Some Tech Tips to find a better WordPress or bbPress Host

I’m going to reprint some advice here that I shared on bbPress.org as there is a great deal of confusion and misguided ideas over how to find a good host for WordPress or bbPress.

A quick google shows there are dozens of “Top 10” lists of hosts. Some are dubious with commission links and others that as well intentioned as they are, focus on non-technical aspects that don’t matter much in the end.

There is a MASSIVE amount of competition in the hosting world and virtually everyone offers things like “uptime guarantees” and seductive low prices to try to grab customers. In the end it doesn’t matter. You can go with a provider that’s been highly recommended and have the worst experience, or try one out that others have complained about and have a good experience.

(more…)


Firefox plugin to detect when website sessions in use?

update: presenting my very first firefox plugin!
http://bbshowcase.org/firefox/website-sessions-detector.xpi
Detects if any of several session types are in use on a website. It has some room for improvement but the fact that it works at all is good start…

—–
I dislike the use of sessions to track users between web pages because it slows things down on active websites and if cookies are disabled, they can create a mess when they get appended to the URL. They are also useless across multiple servers if they aren’t stored in a common memory pool (memcached, etc).

Here are some common session names:
ASP uses “ASPSESSIONID”
PHP uses “PHPSESSID”
.NET uses “ASP.NET_SessionId”
JSP uses “JSESSIONID”
ColdFusion uses “CFID”
(let me know if you are aware of others)

I try to avoid using sessions in my bbPress plugins but there are two (Human Test & OpenID) where I’ve been too tired (aka too lazy) to come up with a complex way around them that would involve mysql tables, etc. However I try to make sure that sessions are not activated when not needed (in most cases, only during registration. But there is a popular third-party plugin that I’ve taken over ownership for (bb-topic-views) that uses sessions all the time when reading topics to prevent re-counting on multiple pages of the same topic.

(more…)


simple WordPress and bbPress security plugin to block long requests

I opened a WordPress security ticket today with something that’s been bothering me for awhile – Apache will take long URL queries up to 8k (8192) characters in length but it’s completely unnecessary and allows XSS exploits to get into WordPress and bbPress. Why not block them entirely with this scrap of a plugin (save as “_block_long_queries.php” without the quotes but with the leading underscore so it loads as early as possible and doesn’t need activation – also can’t be easily deactivated by hackers)
(more…)


bbPress turns 2 today! Happy Birthday bbPress!

bbPress turns 2 years old today!!!

It’s been a big year with new features being added at a rapid pace thanks to hard work by Sam and Mike. It’s an easy guess that we should see a stable 1.0 beta by the end of 2008.

But don’t let the modest sounding “1.0” fool you, it’s practically a “2.0” inside with all the changes since 0.9 including BackPress integration (merges core functions from WordPress) and an XML-RPC api like WordPress as well.
(more…)