Elliott C. Back: Internet & Technology

One of the reasons that Microsoft’s Vista operating system isn’t as successful as Windows XP (see Was Windows XP Microsoft’s last good OS?) is that it suffers from a version explosion. Where Windows XP offered a Home and Professional version which differed from each other in a simple list of features, Windows Vista comes in five versions:

Note that visual studio actually has 8 versions…

This is in comparison to OSX, which comes in one version, and linux which comes in hundreds of versions but is at least free. Perhaps their next os, vienna, will come in a single edition at a single global price, but that’s probably too much to hope for.

It wasn’t my intention, when I wrote about how Google wouldn’t index one of my sites to make it to the #1 keyword for “Google Sucks” but it happened anyway. That’s a story for another day–right now my gripe is with infamous search engine Yahoo, which doesn’t index any of my sites at all!

Try searching for name, Elliott Back on Yahoo and you get absolutely nothing:

I tried adding my feed to Yahoo Site Explorer but it’s been 50 minutes and it hasn’t been indexed yet. I’m going to contact Webmaster Support and see what comes. My suspicion is that this is a side-effect from the rapid adoption by others of WP Autoblog, which links back to me, one of the reasons I stopped distributing it.

Update: I submitted my feed at 05/17/2007 14:18:01 and it was processed on 05/17/2007 14:26:18. Apparently it took a long time to get reported as processed, but the processing itself happened within 10 minutes.

Update 2: Well so far nothing. If any Yahoo people stop by, drop me a note or give me a call and we can get this sorted out.

Update 3: I emailed a friendly Yahoo employee who might be able to help, and also signed up for their paid search inclusion, “search submit basic,” which hopefully can nudge the site back into the index. Although, I don’t feel right paying for being indexed, it feels too black hat.

Update 4: I got the following email back from Yahoo:

Hello,

Thank you for writing to Yahoo! Search.

It has been determined that your site may not comply with Yahoo!’s Content Quality Guidelines located at: help.yahoo.com/help/us/ysearch/deletions/deletions-05.html

Below are some answers to common questions regarding this issue:

Q: What are some of the common reasons that a site may violate Yahoo!’s Content Policy Guidelines?

A: Yahoo!’s Content Quality Guidelines (link above) outline what we are and are not looking for in pages that we index. Listed below are some of the more common reasons that a site may violate these guidelines:

- Cloaking (showing crawlers deceptive content about a site)
- Massive domain interlinking- Use of affiliate programs without the addition of substantial unique content
- Use of reciprocal link programs (aka “link farms”)
- Hidden text
- Excessive keyword repetition

Q: If my site has a judgment against it, can I use the SiteMatch inclusion program?

A: All pages submitted to SiteMatch are editorially reviewed. If a site has a judgment against it, it is likely to be rejected by the SiteMatch program.

Q: If my site has a judgment against it, can I use the Yahoo! Express directory inclusion program?

A: Yes, the Yahoo! Directory and Yahoo! Search Index are different systems. Inclusion or exclusion from one does not affect the other.

Q: How can I have my site re-reviewed?

A: Please review our content quality guidelines to make sure that your site meets all of them. When you feel the site is ready, please complete the form located at: add.yahoo.com/fast/help/us/ysearch/cgi_rereview requesting a re-review of your site. You may wish to include an explanation of unique features on your site, or details of changes to your site’s content that may assist our editors in their evaluation.

Please allow several weeks for the review process, YST indexing, and a complete refresh of the database before checking search.yahoo.com to see if your site is listed in the Yahoo!Search Index. We do not offer specifics detailing how an individual site is not in compliance with our guidelines, but we will review your site individually. You will not be
receiving further notification regarding your request for a second review, and we are not able to offer the option of another review.

Thank you for taking the time to make the Yahoo! Search Index better.
Thank you again for contacting Yahoo! Customer Care.

Regards,
Evan

Yahoo! Customer Care
www.yahoo.com/

Uh, this email contains no information that could help me understand why I’m banned from Yahoo, or how to correct that ban. Thanks for the … wasted bits in cyberspace!

People everywhere complain that Wordpress is slow, or that they can’t survive a digg. They die if they get more than 10,000 visitors a day, their hosting providers ban them for using too many resources, and they cry because they have to purchase expensive hosting plans. If this describes your plight, before you run over to Survive Digg hosting and plunk down even more money, take a look at your Wordpress setup and LAMP stack. You can make them better.

My Hardware

I’m not running on a top-of-the-line multi-processor machine with 16 GB of RAM per node. I have a Pentium 4 with 1M L1 cache, 1 GB of RAM, a 160GB SATA2 drive, and 100MB/s ethernet. I get 1.3 TB of bandwidth of month, as well. The hardware is from Cari.net and I pay $80 a month for it.

WP Built-in Object Cache

Did you know Wordpress will try and cache all kinds of database queries as files on disk? It’s so simple. Just add the following to your wp-config.php:

// Enable the WordPress Object Cache:
define(ENABLE_CACHE, true);

This can give a noticeable and immediate performance benefit. Less queries = less overhead and more CPU to go do other things. In case you don’t believe me, I just had a guy whose load average was between 20 and 50. After making this tiny change, it dropped to 2.

WP-Cache

You’ve seen this before, but if you’re not using the WP-Cache WP Super Cache plugin, we shouldn’t talk. It serializes your posts to a file on disk and later spits them back. It’s the classic caching solution. It also knows how to update itself when comments are received, etc, so your site is always the most up to date. Currently 304 posts are cached on this site in the last hour.

Wordpress Plugins

Here is the complete list of plugins that run on this blog:

Content Filters: Adbright BritePic Enabler, Admin Info, Adsense Injection,Auto-hyperlink URLs, Feedburner Feed Replacement, Terms2tags, Wordpress Duplicate Content Cure, WPvideo, WP Fixed Size, WP Adsense, WP-Stats, wp-cache

New Functions: delicious – Bookmark this!, Elliott’s Asides, Elliott’s Feed Tagger, Google Sitemaps, PJW Mime Config, Plugins Used, Relative Dates

Remote Services: Akismet, Extract Terms, Get RSS, Text Link Ads, WordPress From/Where

As you can see, there are just five that have any kind of processing impact. Akismet has to make calls to their webservice to verify spam. Extract Terms uses the Yahoo API to build keywords for automatic tagging. Get RSS performs similar operations per post. Text Link Ads keeps track of an xml file it updates every now and then. WP From/Where archives the keywords used to find every post. The first four cache their results. WP From/Where is the only “expensive” plugin I have, since it adds a query to every page load.

You should go through your plugins, and see if there’s a way to make them faster. To make them stop creating queries and use flat files, or better yet, cache things. If your plugins are slowing down your Wordpress installation, fix them and submit a patch their author. He’ll probably even give you a free link.

You can install something like xDebug to profile your PHP code for bottlenecks, as well.

MySQL Query Cache

The MySQL Query Cache saves results of queries in case the query comes by again. However, it only knows how to save the byte-text of queries, not their compiled versions, so small changes to the query will create different cache entries. Turn this on if you don’t have unique ids in every query. You can enable it by adding the following to /etc/my.cnf:

query_cache_type = 1
query_cache_size = 26214400

This will turn on the query cache and instruct it to use 26M of RAM. After you’ve changed the config, restart the MySQL process by /etc/init.d/mysqld restart and then run these SQL queries to verify that it’s working properly:

SHOW STATUS LIKE 'Qcache%';
SHOW STATUS LIKE 'Questions%';
Qcache_free_blocks  	2960
Qcache_free_memory 	11693192
Qcache_hits 	10433610
Qcache_inserts 	5221850
Qcache_lowmem_prunes 	2113131
Qcache_not_cached 	1335038
Qcache_queries_in_cache 	8765
Qcache_total_blocks 	20976
Questions 	21338215

When I check, I find the Query Cache efficiency is at worst 10433610/21338215, or 49%. That’s not bad.

PHP Opcode Cache

Why recompile our scripts every time, when we can save the bytecode? I use eAccelerator v0.9.5 to accelerate my PHP classes, currently of which 455 are saved. To install it follow these simple steps:

cd eaccelerator-0.9.5
phpize
./configure
gmake (or make)
gmake install (or make install)
add extension="eaccelerator.so" to your php.ini file
add eaccelerator.cache_dir="/tmp/eaccelerator" to your php.ini file
mkdir /tmp/eaccelerator
chmod 0777 /tmp/eaccelerator

Their wiki has very clear instructions for how to install it. This extension is a win-win. It uses a bit of disk space, but it also optimizes the opcodes and saves you from having to recompile a script every time. It can shave up to 5000ms off your loading time, depending how bad the php code you’re actually running is. For me, it saves about 650ms per page.

Apache httpd.conf

First, and perhaps most importantly, comment out any modules you’re not using. Loading them takes time and memory. Your apache config file is usually located in /etc/httpd/conf/httpd.conf. Some settings to keep in mind:

# Timeout and Keepalive
Timeout 30
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 6

#Maximum Client Connections
<IfModule prefork.c>
StartServers       8
MinSpareServers    5
MaxSpareServers   20
ServerLimit      512
MaxClients       512
MaxRequestsPerChild  4000
</IfModule>

There are tons of how to guides out there for configuring apache to maximize performance, but mostly it’s great software that doesn’t need much tuning.

PHP php.ini

PHP is a beast with all kinds of horrible modules it thinks it needs to load. You find the php configuration usually at /etc/php.ini; here are some settings to observe:

;*Hide our info
expose_php = Off
;*Turn off for performance
register_globals = Off
register_long_arrays = Off
register_argc_argv = Off
magic_quotes_gpc = Off
magic_quotes_runtime = Off
magic_quotes_sybase = Off
;*Allow PHP to accept large data
post_max_size = 6M
file_uploads = On
upload_max_filesize = 6M

The real trick is to disable as many extensions as you aren’t using to save memory. You can check out this guide for more information about optimizing a php configuration.

Linux Kernel

Let’s put it this way; you want to be running the latest and greatest versions of all your software. If you’re using a 2.4 Kernel, don’t. The 2.6 kernel is appreciably faster. For me, getting the latest software is as easy as running the yum update command, as I use FC5. If you’re on a different distro, check their documentation for instructions on how to update to the latest version:

[root@fc435152 ~]# yum update
Loading "installonlyn" plugin
Loading "skip-broken" plugin
Setting up Update Process
Setting up repositories
livna                     100% |=========================|  951 B    00:00
atomic                    100% |=========================|  951 B    00:00
psa-8.1                   100% |=========================|  951 B    00:00
updates                   100% |=========================| 1.2 kB    00:00
core                      100% |=========================| 1.1 kB    00:00
mono                      100% |=========================|  951 B    00:00
extras                    100% |=========================| 1.1 kB    00:00
Dependencies Resolved

==================================
 Package                 Arch       Version          Repository        Size
==================================
Installing:
 kernel                  i686       2.6.20-1.2312.fc5  updates            15 M
 kernel-smp              i686       2.6.20-1.2312.fc5  updates            16 M
 kernel-smp-devel        i686       2.6.20-1.2312.fc5  updates           4.6 M
Updating:
 alsa-lib                i386       1.0.14-0.1.rc1.fc5  updates           404 k
 cups                    i386       1:1.2.8-1.fc5    updates           2.9 M
 cups-libs               i386       1:1.2.8-1.fc5    updates           180 k
 dhcdbd                  i386       1.15-2.FC5       updates            68 k
 gnupg                   i386       1.4.7-4.1        updates           1.9 M
 gpm                     i386       1.20.1-82.fc5    updates           185 k
 krb5-devel              i386       1.4.3-5.4        updates           954 k
 krb5-libs               i386       1.4.3-5.4        updates           558 k
 krb5-workstation        i386       1.4.3-5.4        updates           858 k
 libX11                  i386       1.0.0-4.fc5      updates           761 k
 libXfont                i386       1.2.8-1.fc5      updates           227 k
 libpcap                 i386       14:0.9.4-4.fc5   updates           207 k
 openssh                 i386       4.3p2-4.12.fc5   updates           279 k
 openssh-clients         i386       4.3p2-4.12.fc5   updates           435 k
 openssh-server          i386       4.3p2-4.12.fc5   updates           251 k
 samba                   i386       3.0.24-4.fc5     updates            16 M
 samba-client            i386       3.0.24-4.fc5     updates           4.2 M
 samba-common            i386       3.0.24-4.fc5     updates           8.5 M
 tcpdump                 i386       14:3.9.4-4.fc5   updates           443 k
 tzdata                  noarch     2007d-1.fc5      updates           451 k
Removing:
 kernel                  i686       2.6.17-1.2142_FC4  installed          38 M
 kernel-smp              i686       2.6.17-1.2142_FC4  installed          44 M
 kernel-smp-devel        i686       2.6.17-1.2142_FC4  installed          13 M

Transaction Summary
=================================
Install      3 Package(s)
Update      20 Package(s)
Remove       3 Package(s)

Total download size: 75 M

Architecture

If you’re serving more than 5 million hits a day, you’ll want to take a different approach than if you’re serving 500,000 hits a day. For a small server, like mine, which can handle up to perhaps 1,000,000 hits a day, you’ll want everything running on the same box:

For a more complicated server, you’ll split off the MYSQL component onto a hefty box, with Apache/PHP instances on a cluster of loadbalanced other boxes:

Conclusion

There’s no good reason for Wordpress or your site to be slow, except your own negligence. Cache everything. Monitor performance. Use the latest versions of your software. Configure it intelligently. If you take an active part in every bit of software that powers your site, soon you’ll find things fit together more smoothly than before, and the secrets of a fast server will naturally fall into place.

Other Resources

Call me Lorelle; I’m going to link every other optimization guide for Wordpress I can find:

• Optimize WP Loading Times
• Optimize your database
• How To: Optimize Your CSS Even More
• Optimizing Wordpress Performance
• Optimizing Wordpress for High Volume Traffic
• High Performance Wordpress
• Optimizing Wordpress and LAMP to survive the Digg effect
• Diggproof & Speed up Your Wordpress Blog
• DiggProof your Wordpress
• WP 2.1 Is Faster Than 2.0
• Summer of Code 2007 Performance Testing Project