October 9, 2006 § Leave a comment
Customizing WordPress Series
By popular demand, we will now feature an ongoing series based on our adventures adding and removing WordPress plug-ins for VibeTalk.
Our installments so far include
Why Do This? Why WordPress?
As CEO of Vibe Technology (read: Chief Enlightenment Officer), I’ve been watching our site traffic with great interest as we experiment with the Whatever Works™ business philosophy. As it turns out – most of our visitors are interested in “How To” information.
Having spent a year personally building the kind of company elements you tend to think of as overhead, I am keenly aware of the need.
This series consists of quick overviews and will be a bit more concise than the usual tutorial, but the plan is to roll these up into a “Top WordPress Customizations” feature at some point soon.
Now, if only I could find more people willing to save *me* time and money so I can focus on business.
Actually, I rather enjoy it – I hope you do too!
October 6, 2006 § 2 Comments
Advertizing On YouTube
I recently read an article where YouTube founders were reportedly reluctant to capitalize on their user reach with commercials. They wanted to do it right.
Looks like they may have found the right price, at least – Google is in early talks to buy YouTube for a cool $1.6
million (EDIT: Did I say million? I meant billion. Holy cow), according to the Wall Street Journal.
It will be interesting to see what the Labs concocts next. Combining AdSense with the huge inventory of tagged video is top on the list of potential uses…
User Poll – Is YouTube Worth It?
October 5, 2006 § 33 Comments
Improve PHP Performance
This post continues the series where we embarked on the seemingly simple task of enabling If-Modified-Since headers for PHP scripts. What I discovered is that it’s not as easy at it seems!
- Part I: Understanding IMS
- Part II: Watching IMS In Action (you are here)
- Part III: Using IMS For Optimized RSS Feeds
- Part IV: Implementing IMS On WordPress
Part II: Watching IMS In Action
Up to now I’ve called this process “IMS”. What we’re really doing is a conditional GET. In the next section we’ll go over several useful tools for dissecting a conditional GET and watch real data to help explain the challenges and possible solutions involved.
To perform and watch a conditional GET, you need the following (the tools I use in parenthesis).
- An HTTP Monitoring Tool (Fiddler 184.108.40.206)
Fiddler is an HTTP debugging proxy written by Eric Lawrence of Microsoft that allows you to examine request and response header values as they happen. This the tool I used to check which sites use gzip in a previous tutorial. You can also craft custom requests, attach scripts to particular behaviors and filter various response types. I mostly use it as a quick and dirty way to see what’s getting loaded and which requests cause errors.
- An HTTP 1.1-Compliant Browser (FireFox 220.127.116.11)
FireFox is of course, the tool for compatible browsing, but any modern browser should work – the IMS caching standard has been around for years.
- Any scripting/CGI language that allows raw HTML generation (PHP 5.0)
PHP is perhaps overkill for demonstration purposes (you could use a simple batch file), but it is very common so I suspect this should work for most readers. PHP is the scripting language upon which WordPress is based.
Let’s take a peek at the data exchanged between a browser and server…
Example 1: Watch HTML Caching With Fiddler
First, let’s look at what a normal HTML file looks like with Fiddler. The HTML is stripped to the bare minimum for simplicity:
(screenshot: First pass of static.html)
Because static.html hasn’t been loaded before, the browser does NOT send an IMS date, but the server returns a Last-Modified and ETag for next time.
(screenshot: Second pass of static.html)
Press F5 to reload and this time the server responds with a 304 Not Modified, as expected.
All of this has been automatic so far. Now let’s look a more interesting page…
Example 2: Watch a Simple PHP Page
We’ll execute a basic PHP command (normally this would be a data query or series of pre-processor commands for the server, of course).
Press F5 and notice that even though the file has not changed, the server still generates the content each time. No big deal for a small file, but in a WordPress environment, you’re looking at upwards of 100k of related data requested from a database for each user, rss aggregator and indexing robot that visits the site!
Speaking of unnecessary data, I’ll forego the fat screenshot images – you get the idea.
Example 3: Watch a PHP File Using Conditional GET
This is where it gets fun – a conditional GET.
For simplicity, the next example does not actually store or retrieve a LM date for the file – we’re going to hard code one in the header and embed the doConditionalGet function in the same file. The base code comes courtesy of Simon Willison.
As predicted, the file only gets loaded the first time and as long as the cached copy still exists, should NEVER load again for the same browser.
So, are we done?
No, because now we have to figure out how we’re going to track the LM date for this content without creating more of the server load we set out to prevent! We’ll cover this in Part IV: Implementing IMS On WordPress.
But first, we’ll spend time in Part III understanding how all of this specifically impacts content aggregators and RSS feeds.
To conclude this section, I want to introduce one other very useful tool for webmasters: the Cacheability Test by Mark Nottingham.
This tool analyzes a URL for the presence of cache-ready values like Last-Modified and Entity Tag, as well as a few I haven’t mentioned yet
- Cache Control
- Content Length
Your homework is to perform tests on the following URLs.
To speed up cacheability tests, be sure to uncheck “include referenced frames, images and objects”.
- Test 1: Run the cacheability test on http://www.vibetechnology.com/vt/index.php
- Test 2: Run the cacheability test on http://www.vibetechnology.com/vt/ims/with-ims.php
- Test 3: With Fiddler running, click this link then reload it by pressing F5: http://www.vibetechnology.com/vt/ims/with-ims.php
Question: Why will http://www.vibetechnology.com/vt/index.php NEVER be cacheable?
October 1, 2006 § 2 Comments
Improve PHP Performance
Compressing script output is great, but what if you could frequently circumvent the need to run scripts altogether?
So, this week we’ll talk about another misunderstood and underused web technology – the IMS header.
Part I: Understanding IMS
Implementing IMS for WordPress-generated pages on VibeTalk proved to be more than a small task, so I’m going to break this tutorial into several parts:
- Part I: Understanding IMS
- Part II: Watching IMS In Action
- Part III: Using IMS For Optimized RSS Feeds
- Part IV: Implementing IMS On WordPress
As of this moment, I have a pretty good outline of these parts, but since I haven’t completed the other steps, this information may evolve as I learn more. Our frustrations about this topic will evolve together!
First, let me take a stab at a simple explanation of IMS…
How Does It Work?
IMS is part of the HTTP specification designed to improve performance for scripts like PHP and ASP. IMS is part of the overall concept of caching and is especially important for scripts that generate web pages “on-the-fly”.
Static HTML files have a built-in “last modified” timestamp, but because dynamic content is generated at the time of request, it is “last modified” for each request. How can you determine if a page will be different than a previously-requested version?
Requests that include an IMS date allow the web server to determine if the content has changed since last requested.
Request / Response Overview
By requesting a web page with an IMS date, a browser is saying to the server, “I would like a web page that I previously received from you on this date. Has it changed since then?”
If there has been no change, rather than perform expensive database queries, the server replies with a 304 header, or “Not Modified”.
If the content has changed since last requested, then the server generates the content (executes the script or PHP file) and returns the data with a standard 200 header (“OK”).
(illustration: Server Interaction on 304 “Not Modified”)
By not re-running the script code, servers should be able to respond within a fraction of the time it would otherwise take.
IMS versus The Last-Modified Header
You may ask as I did, “What’s the difference between IMS and the similar header value called Last-Modified?” As it turns out, they are essentially two sides of the same coin.
A browser sends an IMS date to the server; a server returns a LM date to the browser. The two work together along with another bit of information called an Entity Tag (ETag) to accurately identify a unique bit of content.
An ETag is a unique content identifier generated by the server. It can be “weak” or “strong” and is what’s called an “opaque tag”. This just means that the value is entirely up to the server (and you), but it must be unique or your caching method may fail.
When a browser gets an ETag, it is required to save it and resend it with the next request as an If-None-Match header (more on the If-None-Match later).
Caching Affects More Than Your Browser
The big lesson so far is that caching is a large concept affecting more than one browser’s performance. Mechanisms for caching were built to allow proxy servers, content aggregators and indexing robots (a.k.a. “bots”) to share the burden of distributing information and intelligently collect data. This wide-ranging goal is part of why a simple IMS “switch” is more complicated than it seems.
If-Modified-Since is one of several caching headers designed to allow servers to circumvent the need to re-run scripts if content hasn’t changed since last requested.
Automated indexing tools and content aggregators can severely burden ALL servers by uneccesarily hogging internet bandwith. Because of it’s general-purpose design, not only can IMS improve your server performance, it can potentially reduce overall Internet usage for RSS/ATOM feeds and indexing robots by orders of magnitude.
The browser/server responsibility for IMS are as follows:
- If this page is in the local cache, the browser sends an IMS date when requesting the same page again
- If the server provided one, the browser sends an ETag (unique identifier previously provided by the server for this document/web page)
- If requested content has not been modified since IMS date, the server skips page creation and sends a 304 Not Modified response
- If requested content is updated, the server sends a page with new Last-Modified date
- The server generates and sends an ETag
I, for one, want to do my part to make the Internet more efficient for all.
Stay tuned for more on how we can help make this happen…
September 22, 2006 § 5 Comments
Has Broadband Spoiled Us?
Do you know some of the most well-designed website home pages are approaching 400k???
More Automation Means More Code
Since discovering the power of WordPress themes and plug-ins, I’ve battled with growing page size and slowdowns from script complexity. With some of our pages approaching 150k, I felt guilty that I was getting sloppy.
After a day’s work, we now have the holy grail of performance boosts: GZIP compression. Scripts, CSS and HTML files are compressed around 3:1.
As of right now VibeTalk has more features than ever and the homepage is down to 70k!
My question is: why aren’t more people using this?
MyBlogLog Proves Themselves Again
Having compressed everything on our site with GZIP, I began to notice that external scripts (particularly usage analysis code) were now the largest files. Happily, Google is on the bandwagon and GZIP’s their code, but others were not, so I went a’calling on one…
MyBlogLog Communities has got to be the most responsive startup I’ve seen. I sent an enhancement request to earlier this week asking if they’d mind enabling GZIP for their tracking scripts. While I expected promptness, I was amazed that they enabled it in a matter of hours!
The script went from 7k to 2.5k. Sweet.
However, aside from Google, they seem to be the only ones doing it. WebTrends, Site Meter and Performancing do not compress their tracking script files.
Uh Oh. What Happened To Web Design 101?
I got curious who else wasn’t using GZIP and guess what else I found? Compressed scripts were the least problem on most of these sites. It is SHOCKING how many bloated sites there are out there.
The suprising part is that some of these sites are leaders in the blogging space.
Whereas MyBlogLog’s site is an excellent example of a clean, no frills interface, others seem to have forgotten the fundamentals of good web design. As I looked for GZIP usage I found some sites had huge graphics and CSS files.
How Big Is Too Big?
Large pages (particularly your homepage) are *bad*. According to Web Site Optimization, one of Andy King’s many developer tool incarnations, webmasters should target a 30k overall page size. Some header graphics are larger than that!
At 30k, users on a 56k modem should be able to get the full page in about 8 seconds.
Historically, I’ve targeted a 50k home page, but in these days of fancy websites, even getting under 100k seems difficult. That doesn’t take into account the server impact of dynamically-generated content (which taxes servers and can delay the start of content delivery).
The Best And Worst Sites
I picked these sites somewhat randomly but mostly because I frequent them. They also tend to represent key “constituencies” in the web world.
Google, Yahoo and Microsoft are defacto leaders in web design, while the blogging sites are similar in purpose to VibeTalk.
I added a few design- and media-oriented sites because I look to them for ideas (the current and last theme here on VibeTalk are largely based on work from Solostream, one of many excellent WordPress-based sites you’ll find).
Tubetorial is interesting because they just came on the scene as a site that teaches you how to make websites and configure a WordPress blog. Ouch. Very pretty, but an ironically bad example, IMHO.
|Photo Matt||15791||5313||6556||32426 bytes|
|The Undersigned||23226||83094||11267||134567 bytes|
|Blogging Pro||59274||16230||10115||154496 bytes|
GZIP To The Rescue
This did allow me a careful review of several uneccesary plug-ins. For example, I wouldn’t recommend using Edit N Place – handy as it is, it puts way too much unneeded admin code into your general UI.
Then I wanted to compress our CSS files and I found it’s way too difficult to maintain several color versions plus a compressed and uncompressed version for each.
After more searching, I stumbled across a few articles on enabling GZIP through .htaccess rules or through a two-step trick php. Too difficult.
Finally, I found a great article that said IIS already has GZIP ability. Built in!
How To Enable GZIP
Once you’ve got the basics covered, GZIP can make your site lean and mean again.
Enabling GZIP on your server varies. We use IIS6, and I found an excellent step-by-step guide by Scott Forsython on wwwcoder: IIS Compression in IIS6.0. (BTW, I usually put these kind of links on the VibeTalk account at del.icio.us. If you subscribe by email or RSS, you’ll be notified when I post a new one).
Because IIS does not provide an interface for adding GZIP, you will have to edit the metabase.xml file directly. It is very VERY important to back up your metabase.xml file before starting. I spent an hour fixing the site after I mucked it up. Bad Joe.
The gist of what Scott shows you how to do is
- Enable compression at the site level
- Load the GZIP web service extension
- Edit the metabase.xml file to indicate that you want to compress .php files on-demand at level 9 (0-10 scale)
For VibeTalk I have also added .css and .js files to the static file list.
What Not To GZIP
The one thing you probably don’t want to compress are graphics files – they’re already compressed. The same holds true for zip and other archive formats, obviously.
OK, so what’s the lesson here?
- Be sure to regularly review your site for overly-large graphics and unecessary code
- Turn on GZIP compression for everything except graphic and archive files on your web server
- Make sure external script providers have GZIP turned on too
I hope each and every one of you gets your site home page below 100k. Good luck!
September 13, 2006 § 2 Comments
Turn Your Site Into a Video Blog
One of the many features in the works at Vibe Technology involves streaming video. A good part of last year was spent building a media operations center and discovering that delivering online video is not a trivial task.
How Much Server Horsepower Does It Take?
In addition to racks of powerful servers to transcode video for various connections and formats, you also need an enormous amount of storage capacity. An unconverted, full-length movie typically requires 4 Gigabytes per DVD disc. Converting that to a full-quality streaming format can take at least that much space.
Usually, you’ll want to pre-convert each movie into multiple formats for streaming to other devices over lower-bandwidth connections, so that means several versions of the same file to support each format.
The tiny movie used in the example below (decidedly low quality) requires approximately 3.75 megabytes per minute. A 90 minute movie at that size and quality would be 383 Megabytes!
Surprisingly, the storage capacity is the easy part. Learning about those various formats and delivery mechanisms… that’s the hard part.
Example: Hosting Video on Your Website
Let’s take a simple example that doesn’t require special server software. The following video clip is a Flash Video file (FLV) placed in a directory on our storage server. The file is relatively small and the FLV format allows a client to stream the video (simply put: streaming allows playing the file before it has entirely downloaded).
The file was originally encoded as a 30 fps, 320×260, 7 Meg Windows Media Video file then converted using Flash Professional. The result is a little larger, but FLV files stream a bit better:
Length: 2min 40sec
Format: 320×260 Medium Quality Flash Video (FLV)
File Size: Approx. 10 Megabytes
To display the file as an embedded video, we use an excellent free WordPress plug-in called Anarchy Media Player. This customizable plugin adds about 15k of script code to the site page and automatically plays FLV, WMV and MP3 files without launching a seperate program.
(VibeTalk video clip: Sarah Palooza 2006)
This works well and is nearly automatic, once you have everything setup. However, even without considering disk space, processing power and setup, it still takes time and costly software to perform conversions. And the problem gets trickier if you want something more robust, such as an actual streaming video server like Windows Media or Helix Server.
YouTube Servers Are Better Than Yours
What if you could avoid the whole mess and let someone else do the work? As it turns out, the folks at YouTube have put quite a bit of money into virtually unlimited transcoding and storage servers. The best part is that it’s free!
Every video on the YouTube site has a small, almost unnoticable edit box that contains the script code to embed that video in a web page. To show an embedded video on your site, all you have to do is paste this code from any video page into a post or page on your blog:
(screenshot: Video page at YouTube includes code for your site)
Example: Embedding an object from YouTube
First, you need create an account and upload your video file. YouTube doesn’t like our FLV file, so I uploaded the original, high-quality WMV file and allowed the site to convert it for me.
The dimensions were modified to stretch the video to 450×350, so the result is a somewhat grainy version of the video above. You can change the values to the original size of 320×264 or resize the video to YouTube default – your call.
Then paste your script code directly into a blog entry and you will get the following video display (if you want to try this without creating an account, feel free to copy/past from our code below)
(YouTube video clip: Sarah Palooza 2006)
One neat feature is the thumbnail that got created from a frame in the middle of the clip.
As you can see, the embedded player also includes a “share” button to allow viewers to easily share your video.
So what’s the catch?
There’s always a catch to free, right? As it turns out, there’s one here too:
- the resulting video is not (easily) downloadable
- the conversion process overlays a “YouTube” logo in the bottom right corner
(screenshot: End of video page promotes related videos)
Having your video on their server also means that If you wanted to do your own advertizing, you’re out of luck there too.
How To Get All YouTube Options?
So you like the YouTube interface and are OK with the advertizing issue – how can you get the rest of YouTube’s functionality? As it turns out, there are a couple of handy plug-ins for that too!
MyTube by Vaam Yob
For an idea of what this does, check out the screenshot (I don’t have enough videos on YouTube to make testing interesting at the moment).
(screenshot: YouTube gallery using MyBlog plug-in)
Video Blogger by Chad Lancour
Chad Lancour created an even more flexible plug-in called Video Blogger that not only allows easy YouTube integration, but Google Video, vSocial, and Dailymotion. The best part of this plug-in is that each video can be cached.
Caching means that you won’t need to depend on someone else’s servers for reliable delivery (but let’s face it, it’s their reliability that makes this whole project worthwhile, right? :-)).
(screenshot: YouTube video using WordPress plug-in)
Video Blogger doesn’t do anything automatic for displaying whole galleries, and the big the downside of this plug-in is that you’ll need to configure the CURL library (something I couldn’t do after 30 minutes of tinkering).
While professional video services will want to keep their whole shop in-house, the casual user can easily set up video blogging using WordPress, YouTube and one or more handy plug-ins like MyTube and Video Blogger. Surprisingly, the more you do this, the more you may want to rely on external services – their servers are faster and space is currently unlimited.
Who knows? Maybe your video will make the front page!
September 9, 2006 § 2 Comments
New Default Theme Selected
Uh oh, the site layout has changed again! I think you’ll like this new look, based on another Solostream Global Media WordPress theme called Torn.
We’ve modified the layout a bit and renamed it slightly to reflect the pattern of our color-based themes. This one allows you to switch basic color schemes, on-the-fly!
More Than Just a Pretty Face
Besides being a cleaner layout, you will notice a few more features for the site:
- de.lico.us Links
For those of you not in “the know”, this is an excellent service that allows sharing of bookmarks.
- Polling Functionality
A few sample polls will randomly show in the rightmost sidebar. Look for another post on how we plan to use polling.
- Control Panel in Header
When moving between major areas of the site, you lose the ability to login/logout. This will be more of a problem as we add more functionality (hint: we have something major coming soon!). I think this new login layout is a bit less obtrusive too.
(screenshot: Torn Rainbow Theme)
Let’s Call It “BETA”
Hey, it works for Google! Seriously, the theme will still be undergoing much work, so please bear with little oddities. Call it progress.
NOTE: If you don’t see this new look, you need to delete your “cookies”. Firefox Users: press CTRL+SHIFT+Del
Internet Explorer: go to Tools/Options/Delete Cookies….