One of the sites that we develop and maintain is Biofind, an vertical online portal for the biotech industry. The site received the first phase of a series of improvements late last year.
One problem that we knew about at the time was the fact that in many parts of the site we use AJAX paging to display search results and lists of content items. This makes it very difficult for search engine spiders to index all of the available content on the site.
On Monday this week I implemented dynamic sitemaps for the dynamic content sections of the site. A sitemap is a file that Google, Yahoo and MSN (soon) can use to get a full list of the available urls for indexing.
Rather than building a static file containing the page addresses for the site, I built a script that generates the sitemap on demand – when Google or Yahoo download the file, they recieve a full list of every page that is currently available. This has increased the number of urls discoverable by the spider from 190 to 1500+.
Another technique that we’ve implemented is harnessing the user generated content for in-page optimisation. We now generate the page description meta-tag in the Biofind Rumor Mill from the content of the posts on each page. This gives us over 1000 indexable pages with unique page descriptions, pages that will now be displayed in search results rather than hidden as "similar pages".
The results have proven to be fairly conclusive so far. I’ve compared the analytics from the past four days with the equivalent four days in December (3rd week). Not exactly scientific but the stats around the new year are not representative averages due to the holiday season.
| 18th-21st Dec 2006 | 15th-18th Jan 2007 |
|---|---|
| Daily Visits | Daily Visits |
|
|
|
| Visitors by source | Visitors by source |
|
Google: 28.42% Yahoo: 4.10% MSN: 1.60% |
Google: 32.94% Yahoo: 4.90% MSN: 2.02% |
In case the graphs are a little small, the Pre-SEO graph shows the daily number of visitor at between 3500 and 3800. The Post-SEO graph shows the number of daily visitors rising from 4300 to 6399.
The increased traffic to Biofind has raised the requirement for an upgrade to the site’s hosting. Hopefully the new environment will be able to handle the next series of optimisation ideas that I’m working on.
