By Prof. W. Tim G. Richardson, E-Commerce Times, 12/15/03 6:59 AM PT

Be suspicious of sites that claim to mass-submit your URL to many dozens or hundreds of search engines. There are less than 10 major search engines that you should care about being listed in.

The ways in which people search for information on the Web have changed.

Web content (text, images and audio) is being created and uploaded at a rate faster than the ability of search engines to index this data. As a consequence, it is getting more and more difficult to be "found" in an ever-larger mass of unindexed and unconnected pages.

Moreover, people are becoming discouraged when they receive thousands of returns on the keywords they type into a search engine. As a result, it is increasingly vital that a site's listing appear in the first few pages of search results -- because most people will not take the time to scroll further than that.

This two-part article outlines 10 strategies that can help Web sites not only get indexed, but also position themselves for success. None of these points is more valuable than the others, but in combination they can produce results. Focusing on improving your site by addressing just one or two points might not lead to measurable gains, but trying to address four or five points in concert should help you improve your search engine ranking.

1. Proactively Submit the Site URL to a Search Engine

The business model of many search engines is similar to the advertising-based model that periodicals use: Assemble great content to attract the viewer, then sell that demographic to advertisers. Search engines obviously should allow content producers to submit URLs, because doing so strengthens the value of the search engine's collection of indexed pages, which in turn attracts users, giving the search engine a stronger demographic to sell to advertisers.

For a small company, proactively submitting the URL of several of its Web pages is a relatively easy process that begins with simply clicking on a "submit URL" link. Such links usually can be found in one form or another on the sites of the more popular search engines. However, submitting individual URLs to multiple search engines can be time consuming, so, in a Darwinian way, third-party sites have evolved that let users enter information about a particular Web page and click a button to submit that URL to several search engines simultaneously. Some of these sites provide multiple submission services for a fee, while others do it for free and rely on advertising to reap revenue.

On a cautionary note, be suspicious of sites that claim to mass-submit your URL to many dozens or hundreds of search engines. There are less than 10 major search engines that you should care about being listed in. The rest are highly specific to particular industry sectors and probably are not relevant to your business.

2. Consider Paid Submission

While cost-conscious firms may shy away from paid submission services, there are valid reasons to make such an investment. Dennis Buchheim, director of paid inclusion at Inktomi's head office in Sunnyvale, California, explains that paid inclusion not only helps provide consumers with more relevant results, but also helps businesses ensure their presence in algorithmic search results. For example, the service allows a client to receive reports that contain information about specific keywords entered by searchers who clicked on the client's site in a list of search results. This information is valuable because it allows site owners to make adjustments and refinements to improve their ranking. Buchheim says that by analyzing key reporting metrics and measuring the quality of their metadata, businesses can optimize ROI and increase conversion rates

Having your site indexed and catalogued by a spider shortly after any changes have been uploaded to the Web also can enhance your competitive advantage over other companies. Paid inclusion allows a company to have its page indexed and catalogued more quickly, rather than waiting days or weeks for an updated page to be found by a Web crawler. Some variations of paid inclusion services guarantee "recrawling" of particular pages so that changes will be noted and subsequently available in the search engine's database.

3. Tailor Content for Spiders

Search engine indexing software programs, nicknamed "spiders" to reflect how they crawl through the Web's pages, record several aspects of a page, including its text. In the record that is created and indexed, spiders identify the frequency of particular words on a page, and this becomes part of a complicated algorithm that calculates the page's value and, ultimately, its rank. A spider might work like this: If one page contained the word "cancer" 4 times and another page contained the word "cancer" 12 times, plus they both had the word "cancer" in their meta tags and page titles, the second page would rank higher. Spidering algorithms also calculate and rank how words are connected to other words, such as "car parts," "car finance" or "car warranty" on the Web page of an auto dealership.

Because word count traditionally has been part of the indexing algorithm, webmasters often have tried to deceive spiders by adding additional words to their pages to artificially inflate the page rank calculation. Some conniving Web authors even have been known to add dozens or hundreds of keywords at the bottom of a page in white font on a white background. A surfer looking at such a page would see only blank space, while spiders are color blind and would record all of these words as part of the word count. Although some search engines have tried to create algorithms that cannot be tricked in this way, the method still can be somewhat effective and is far simpler than spending management time developing reciprocal links. (A warning, though: Such behavior can get a site banned entirely from a search engine if its underhanded tactics are discovered.)

Of course, content that spiders evaluate for ranking is not just limited to text, but also includes HTML code referencing image files and audio files. This means that naming your image files woodbridge.jpg and stonebridge.jpg is much more useful than naming them image1.jpg and image2.jpg. It is also helpful to make sure ALT text tags for images include some words that can assist in supporting the overall theme of the page.

4. Remember: Page Title Is Vital

A page's title is often confused with its name. To clear things up, the page name is equivalent to the file name -- i.e., abcdefg.htm -- whereas the page title is the word or words that show up in a browser's title bar. The page title should be crafted carefully. AltaVista suggests it is what search engine users see first when they scan a list of query results, and Inktomi's Buchheim notes that it is not enough to rank high in a search engine. "You also have to ... be enticing enough to click." Indeed, enticement to click is based on both an attractively worded title and the accompanying description, which comes from the META description tags that we will discuss in point 5.

Also, consider that many medium- and large-size corporate pages are flush with images, Flash, frames and other features that are not easily recognized by spiders. AltaVista suggests page title is even more important when a particular page (such as one with frames) has little text content.

5. Mind Your Metatags

There are several kinds of metatags, but from a managerial perspective, only two are critical: the Meta Description tags and the Meta KeyWord tags.

Meta Description tags are the carefully crafted phrases and short sentences that can appear under the page title in a listing of search results. Because the attractiveness of these words can determine whether or not a searcher decides to click on a company's link, it is important to craft this text to be compelling. Some webmasters creating pages for highly competitive consumer product companies hire consultants to write Meta Descriptions in hopes that viewers will be enticed to visit the company Web site.

Meta Keyword tags contain the key words and phrases that webmasters place in the background code at the top of the web page. In the late 1990s, spiders often used these tags, found in the HTML header at the top of each page, to pick up "clues" as to the page content -- perhaps akin to reading song titles on an album cover. However, because so many Web page authors misrepresented their site content by including misleading keywords, use of Meta KeyWord tags now plays a far smaller role in determining page value and ranking. Inktomi's Buchheim says his company's search engine pays little attention to Meta Keyword tags, which are considered supplementary to other factors, such as title and number of links. Likewise, www.webrankinfo.com advises that Google no longer relies on Meta Keyword tags either.
In addition to Google, other search engines such as AOL Search also do not use software that responds to Meta Keyword tags; rather, employees visit submitted URLs and determine whether or not they should be included. While the use of people in the screening process is more expensive than relying exclusively on spidering algorithms, the subsequent indexed compilation has greater value and consequently attracts a discriminating demographic that can be sold back to deep-pocketed advertisers.

However, since it seems simple to "pack in" a lot of words in the Meta Keyword section of a page, many Web authors still rely on this technique even though its Golden Age was in 1999, 2000 and 2001. "Many people incorrectly believe that good Meta Keyword tags are all that is needed to achieve good listings in the search engines," cautions www.submit-it.com. The tags still contribute somewhat to site ranking, but by themselves they are not of significant value, considering how the search engines of 2003 operate.

AltaVista director of Internet search John Glick said links that come from link farms are considered by his company to be spam, and if any site uses such links excessively, it might be dropped altogether from AltaVista's listings.

Part 1 of this two-part series detailed five ways companies can improve their position in search engine rankings, including URL submission, paid inclusion, content tailored for Web "spiders," a well thought-out page title and the nuances of META tags.

Note that none of these methods is more valuable than the others, but in combination they can produce results. Focusing on improving your site by addressing just one or two strategies might not lead to measurable gains, but trying to address four or five in concert should help you improve your search engine ranking. Here are five more tactics to add to your arsenal.

6. Cultivate Links In

"The quality and number of Web sites that link to yours can influence its standing with the search engines," says Detlev Johnson, an internationally recognized expert in search engine optimization (SEO). He notes that the process of garnering links is not as simple as convincing a large number of sites to link to you. Those sites must be of high quality so that the referring page is considered "important" by search engines.

Convincing other high-quality sites to link to yours also means people will be able to find your site without going through a search engine. This can be advantageous because the key objective is to get people to see your page -- and high rank in a search engine is just one way of achieving this goal.

The process of obtaining links from other sites may be difficult for a new domain, but if you give other sites a valid, compelling reason to link to your URL, you increase the chances that they will do so. Other than simple reciprocity, you could offer to identify other sites with a brief paragraph. This willingness to go the extra mile might endear you to a site that otherwise would turn you down.

For example: "See bridges www.bridge.com/eastbridges.htm," does not read as well as: "See bridges www.bridge.com/eastbridges.htm for a thumbnail gallery of old and new bridges across the rivers of Eastern Ontario. Some of the images are large enough to use as decorative desktop wallpaper."

Obtaining inbound links also can occur as a consequence of good customer relations: Companies can request that customers with Web sites link to them. Also, if you have membership in an industry association or chamber of commerce, you can request that the organization's site provide a link to your firm's domain. Links in from customers and industry associations are considered strongly in the site-ranking calculation.

To find out if you are already linked to by other sites, type into a search engine "link: www.yourdomain.com." On the results page, the search engine will list all of the sites that are linked to the URL you entered. Many of these pages may be internal links (from your own site), but you may be surprised to find other sites showing up as well. This process of checking who links to you is something you may want to do on a regular basis. If the sites listed are of high quality, you may want to contact them and offer to reciprocate, thereby setting up two-way links and increasing your ranking the next time a spider crawls through your pages.

If links in from other sites are so valuable, can you increase your ranking by adding internal links among your own pages? As John Glick, director of Internet search at AltaVista/Overture, put it, "links in are a vote of confidence." AltaVista doesn't count internal links because "it doesn't make sense to allow people to vote for themselves." However, Glick said, internal links do have value for other purposes that indirectly facilitate ranking. For example, links within your own pages enable better navigation, so people who do find your site will end up visiting more pages there. Internal links also allow spiders that find one of your pages to subsequently find, catalog and index all of your other pages.

7. Place Less Weight on Links Out

Sites that make their living by racking up hits and click-throughs, such as domains with adult or gambling content, may link out to many other sites, hoping this practice will increase their "hub" value. In the late 1990s, it might have been useful to take this hub approach, but by 2001 and 2002, the value of links out had declined significantly.

8. Focus on Reciprocity

Two-way links are more valued by search engines than one-way links that are not reciprocal. One thing managers need to consider, however, is the time that will be required to make contact with various organizations to set up such reciprocity.

For example, it might take days or weeks to forge one reciprocal relationship, as a series of phones calls and e-mails likely will need to be concluded. Because it is widely understood that setting up reciprocal links is time consuming and involves some degree of a relationship between two entities, search engines value such links much more highly than simple one-way links in or out.

If you do not know many online enterprises with which you could dialogue, you might consider paying for the services of link exchanges and link farms. Some people consider link exchanges useful -- but a number of others are critical of the value of these services.

Michael Wong of www.mikes-marketing-tools.com warns, "Never ever use links from link exchange sites and link farms." In Wong's opinion, "link farms are networks of heavily cross-linked pages on one or more Web sites, with the sole intention of improving the link popularity of those pages and sites. Many of the top search engines consider such links as spam, so stay clear of these types of links."

AltaVista's Glick confirms that links that come from link farms are considered by his company to be spam, and if any site uses such links excessively, it might be dropped altogether from AltaVista's listings.

9. Value Time's Passage

If you want a particular Web page to rank high in a search, it helps if the page has been posted on the Web for long enough to be indexed by the major search engines. Because the volume and complexity of Web content is increasing faster than the rate at which search-engine spiders can catalog and index content, it helps to have your page online for several months. Pages that have been online for a year or two are invariably "found" by all of the top 10 search engines and have an advantage over pages that have been uploaded in recent weeks.

10. Update Frequently

For search results that involve human interpretation of site content, frequent updates likely will help because the people evaluating pages will rank a regularly updated site higher than a site on which content is static. However, search results driven completely by an algorithm do not usually include any information about a "last updated" date in the ranking score. Therefore, updating your site frequently without attending to any of the other nine points listed in this article would be a waste of time, as the results likely would not be measured in any algorithmic search ranking. On the other hand, updating a page as a consequence of creating links in or adding additional keywords will help the page accrue value in site rankings.

Checking Progress

After you have attended to several of the 10 points covered in Parts 1 and 2 of this article, how can you determine if your efforts have been fruitful?

One way to verify your success in attracting search engine spiders is by viewing your access logs. Most of the major regional and national ISPs that offer site hosting services provide various monitoring packages that allow customers to review traffic-related information, such as access logs, number of daily hits and bytes downloaded. A spider that has crawled through your site will leave a record in the access logs.

"If you know what to look for, you can tell when a spider has come to call," says Danny Sullivan, editor of SearchEngineWatch.com. "That can save you worrying that you haven't been visited. You can tell exactly what a robot has recorded or failed to record."

Prof. W. Tim G. Richardson is a full-time Professor at Seneca College, and concurrently teaches part-time at Centennial College. He is also a Lecturer in the Division of Management at the University of Toronto in Canada. He can be reached through his Web site, www.witiger.com.