If you have a page about “basketball drills”, shouldn’t it include these words?

What if it didn’t?  How would you find that out quickly?

SEOmoz recently announced (and then corrected the stats for) a new tool in their labs called the “LDA Tool“.  The tool basically takes a search phrase and a page and tells you how much they relate topically.

Unfortunately, it’s not very clear from that report what topics it identifies or what you can do to improve your page’s score. In this post I’ll show you how to just that, and and give you a new LDA Optimization Tool I wrote to make it easy.

Step 1: Identify your Keyword

First up, figure out what keyword you want to improve your page’s rank for. For this test, I’m using the phrase “basketball drills”. I just chose this at random and hoped I’d be able to find a good example site in it. See how that worked out below!

Step 2: Identify the Competition

A quick way (thanks, Rand!) to do this is with the SEOmoz Keyword Difficulty tool, but you can just do a search and make a spreadsheet too.  To the right you can see what I got for the test phrase:

Exporting this to a CSV and using it as your starting point for LDA analysis speeds things up. You can get the raw CSV here.

I uploaded the CSV to Google Apps as a spreadsheet, and you can see a read-only version of it here (note that this spreadsheet contains my final data as well).

Add your page to the spreadsheet if you’re not (yet) in the top rankings.

Step 3: Gather LDA Scores and Competitor Words

Add columns to your spreadsheet for LDA %, and for site keywords (a really wide column!).  Head to the LDA Tool and type in your test phrase and first site URL, then gather the % and keywords for each of the sites.

Repeat this process for each of the sites. Here’s how my spreadsheet ended up.

Step 4: Find Your Missing Keywords

At this point you should have a good feel for how your page stacks up against the competition.  Now let’s see what words they have that you don’t have, and what words they have in greater frequency than you have.

For the example keyword basketball drills, let’s say that our site is www.basketballteacher.com, which has LDA of roughly 61% and ranks #17 on Google for the phrase. Here’s where we’ll use this new LDA Optimizer tool.

This tool takes your words and up to 9 competitors’ words. It then finds words they use that you don’t, and words that they use more on average than you do, and provides a word cloud showing your opportunities.

To exclude words specific to just a handful of sites, you can specify “Show Keywords Seen on at Least” at the bottom of the form. I used a setting of “4 sites” for my test but got similar results with as many as 8 sites. I recommend playing with this until you’re happy with the results, seeing lots of good words to add but not a lot of extraneous noise.

Step 5: Reviewing Missing/Insufficient Keyword Use

For the sample site, we get this word cloud (on the Wordle link, their clouds are prettier than mine!) for missing words:

So this page related to basketball training is missing words like “rebounding”, “dribble”, “jump”, etc. Lots of opportunities to improve LDA score.  (Note that singular vs. plural isn’t handled well here so some of these words (drill) might exist on the page in other forms that Google would find identical.)

Here are the words the page doesn’t have “enough” of relative to competing sites:

A bit more emphasis on these words might help the site show that it is more topically related to the search phrase.

Step 6 Optimize your LDA Score

The LDA Optimizer tool then provides a suggested set of words and densities to improve the page’s LDA score. Just taking these and pasting them in to your page is unlikely to help, but you should now have some guidance in terms of how many instances of the various words to shoot for as you revise copy.

Adding the suggested words to the site’s existing set moves its LDA from 61% to 86%. So we’ve at least helped our LDA score with a change along these lines. Whether that improves search rank is up to you to test!

Step 7 Check for Higher Google Ranking

Hopefully making changes along these lines will improve your page’s Google ranking, but that’s relatively uncharted territory with regard to LDA today. I’d love to hear your results!

1. These guys track their top search phrases, and they haven’t dropped much in rank for any of those.
2. The total number of searches per month Google reports for those phrases have increased moderately.
3. But traffic from search engines is down for this site.

There are a bunch of things that can imply, but to me, this says that they’re probably getting fewer “long tail” searchers in to the site.  Looking at the number of visits you’re getting from long tail searchers is a little tricky, but Analytics gives us a way.  Click to ‘Traffic Sources’, then ‘Search Engines’, then the one you’re interested in.  Select the right date range, and click the leftmost column to show ‘Landing Page’.  Scroll to the bottom and see how many Landing Pages are listed in the “1-10 of 12,345” to the right.

For the site I’m working on, # of unique Landing Pages is down about 30% for December 2009 vs. December 2008.  We also see it in that there are about half as many unique search phrases in Dec. 2009 as in Dec. 2008.  (Select ‘Keyword’ in the leftmost column of Analytics and scroll down to get this info.)

So we have a pretty clear indexation problem.  Many fewer landing pages, many fewer unique phrases.  There are a lot of potential causes, but one interesting idea came up in a quick site review.

Google spiders tend to “follow a link IN to the site, click around a bit, and leave.”  So to increase indexation, we want lateral linking to a diverse but relevant set of pages from deep landing pages.

Right now on the site, we’re linking to “related articles”, and it works by looking at category overlap and article freshness.  That’s a great start, but has some unintended side-effects.  This causes us to weight newer articles more heavily and link to them from many of our top landing pages.  For example, an old article from 2008 links to 5 articles from today and yesterday.  So whatever pages were linked to from that page back in 2008 aren’t getting any link respect now, and that’s lowering their odds of being in the index.

What if instead, we looked at category overlap and weighted articles with similar dates?  So old articles might have lateral links to other older articles.  That might be too extreme, and might starve our newer content of link respect.  So a middle ground where we factor both in, or add a new “Other Archive Articles” box for similarly-old content might help.

This will be an ongoing process of making sure lateral links are done right, but this isn’t the first site I’ve worked on with this kind of category+date lateral linkage, so I thought the above might be useful for others fighting with indexation problems.

Getting the data smoothed was one of the hardest things, and I’m still very much not happy with the results.

To use this, upload the following PHP to a web server, put a linkscape.csv file in the same folder, and run the PHP.  You should see two charts like these:

Links per DmT, rounded to whole DmT values

Links per DmT, rounded to one decimal

Here’s the PHP code.  If you add to it, email me a copy at glenn@netmud.com and I’ll post an update.  I know the code is awful, it’s just a quick hack to play with CSVs and Google Charts.

There was a stray sidebar link that didn’t have a rel=”nofollow” on it, so I made the change (using the WordPress in-browser theme editor):

A second link to the login page was at the bottom of each post.  The code change was in comments.php for the theme I was using:

Then I re-scanned using A1 Site Analyzer and PageRank is no longer flowing to that login page.

A simpler way to do this (but wasn’t an option on this particular site) is the Meta Robots plugin from Yoast.  Good stuff.

]]> http://www.glenncrocker.com/2009/03/pagerank-sculpting-dont-send-pagerank-to-wp-loginphp/feed/ 0 http://www.glenncrocker.com/2009/03/pagerank-sculpting-stop-leaking-pagerank/?utm_source=rss&utm_medium=rss&utm_campaign=pagerank-sculpting-stop-leaking-pagerank http://www.glenncrocker.com/2009/03/pagerank-sculpting-stop-leaking-pagerank/#comments Thu, 19 Mar 2009 18:20:51 +0000 Glenn Crocker http://www.glenncrocker.com/?p=114 Working today on a client site that had a couple of PageRank issues, I once again ran A1 Website Analyzer, and found a couple of interesting problems. Just let it scan the site, then click ‘Analyze website’ and ‘External’ to see how much on-site link love is headed off-site:

On this particular site, each page links to a customer support portal hosted by SalesForce.com. So by not using rel=”nofollow”, we’re sending all that tasty link juice to SalesForce. No good!

The way this link was set up, we were leaking as much PageRank to SalesForce as we send to our own 2nd-level pages.  Fixing this should help significantly.

The other problem is that the home page has lower on-site PageRank than other pages, but the solution for that will have to wait for another day.

I have a great new tool that spiders sites and helps understand what’s up with on-site PageRank flow. It’s A1 Website Analyzer (free full-featured 30-day trial download), from Microsys Tools. I’ve used Xenu and other spider tools in the past, but what sets this tool apart is their on-site PageRank simulator.  This is a GREAT asset for understanding where PageRank is going and how to make it flow better.

So I clicked through to the Twitter user’s profile, then to his web site, and ran A1 Website Analyzer on it.  Took about 6 minutes, just long enough to fetch a tasty beverage.

If we click over to the results, we can see an “Importance” column and an “I. Scaled” column, which is like on-site PageRank (max=10). Here’s what we see for this site:

Sure enough, a sub-page (the blog) is higher than the home page in “PageRank”.  No good.  What’s causing this?  Normally, the home page of a site is extensively linked, and gets more PageRank than other pages on the site.

The site analyzer provides a “linked by” tab, showing that the home page is only linked by 4 pages on the site. This is clearly a problem! I checked other pages for rel=”nofollow”, and that wasn’t the problem, then checked robots.txt to see if it was set up wrong, but it wasn’t there at all.  So, what was up?

It turned out to be the classic “www.sitename.com” vs. “sitename.com”.  Most links to this homepage were to “sitename.com”, but the blog template had it as “www.sitename.com”.

This usually has the effect of splitting PageRank between two pages, which is almost never what site owners want.  There are a few solutions:

1. (best) Only use one version of the home page link, either with or without “www.”
2. (good) Use .htaccess to redirect to the correct URL
3. (not really ideal) Tell Google (and other search engines, ideally) that you have a preferred domain. If you set up Webmaster Tools, you can do this

In this case, looking at www.sitename.com actually redirected to sitename.com, but PageRank wasn’t flowing right according to the site analyzer. What was causing that? A 302 redirect. Looking at the output of this command:

get -edS http://www.sitename.com/

I got these headers back:

GET http://www.sitename.com/ --> 302 Moved Temporarily
GET http://sitename.com/ --> 200 OK

This kind of redirect is “weak” and doesn’t cause PageRank to accumulate on the destination URL, causing problems.  This site needs to do both solutions 1 and 2 above to make sure things work out right.

I hope this helps you with your own site and making sure your most important pages get the highest on-site PageRank!

-glenn