Glenn on the Web » Analytics

Site Indexation and Cross-Linking Best Practices

Glenn Crocker — Wed, 27 Jan 2010 21:28:15 +0000

Most of the sites I work on these days are on the small end of the scale, but a few are large news portals. One of them went through a site redesign a few months ago (before I got involved), and they’ve seen a large dropoff in traffic. I’m starting to look into why that happened, and wanted to share some ideas about how to fix the problem.

These guys track their top search phrases, and they haven’t dropped much in rank for any of those.
The total number of searches per month Google reports for those phrases have increased moderately.
But traffic from search engines is down for this site.

There are a bunch of things that can imply, but to me, this says that they’re probably getting fewer “long tail” searchers in to the site. Looking at the number of visits you’re getting from long tail searchers is a little tricky, but Analytics gives us a way. Click to ‘Traffic Sources’, then ‘Search Engines’, then the one you’re interested in. Select the right date range, and click the leftmost column to show ‘Landing Page’. Scroll to the bottom and see how many Landing Pages are listed in the “1-10 of 12,345” to the right.

For the site I’m working on, # of unique Landing Pages is down about 30% for December 2009 vs. December 2008. We also see it in that there are about half as many unique search phrases in Dec. 2009 as in Dec. 2008. (Select ‘Keyword’ in the leftmost column of Analytics and scroll down to get this info.)

So we have a pretty clear indexation problem. Many fewer landing pages, many fewer unique phrases. There are a lot of potential causes, but one interesting idea came up in a quick site review.

Google spiders tend to “follow a link IN to the site, click around a bit, and leave.” So to increase indexation, we want lateral linking to a diverse but relevant set of pages from deep landing pages.

Right now on the site, we’re linking to “related articles”, and it works by looking at category overlap and article freshness. That’s a great start, but has some unintended side-effects. This causes us to weight newer articles more heavily and link to them from many of our top landing pages. For example, an old article from 2008 links to 5 articles from today and yesterday. So whatever pages were linked to from that page back in 2008 aren’t getting any link respect now, and that’s lowering their odds of being in the index.

What if instead, we looked at category overlap and weighted articles with similar dates? So old articles might have lateral links to other older articles. That might be too extreme, and might starve our newer content of link respect. So a middle ground where we factor both in, or add a new “Other Archive Articles” box for similarly-old content might help.

This will be an ongoing process of making sure lateral links are done right, but this isn’t the first site I’ve worked on with this kind of category+date lateral linkage, so I thought the above might be useful for others fighting with indexation problems.

Seeing Inside the “Bounce” with userfly

Glenn Crocker — Mon, 16 Mar 2009 11:31:10 +0000

I’m a big fan of Google Analytics, and check it often for clients’ sites, to see how things are going and get ideas about how to make improvements. One of the most frustrating stats for me is the “Bounce”. Today, I’ll show you a new tool for getting inside what happens during a bounce.

A “bounce” is a visit where the user clicked to only one page on the site, and never clicked to a second page. This can happen for a lot of reasons, of course!

Page wasn’t what they expected
Didn’t like the content on the page
Found what they needed, then went away
Didn’t find what they needed, didn’t see a link to a page where they thought they could

In most cases for my clients, a bounce is a waste. The site was hoping to make a sale, get a contact, or otherwise engage with the visitor. For whatever reason, it didn’t happen. It’s like a “first date” that didn’t lead to a second one.

The way Google Analytics tracks traffic, each time a user goes to one of your pages, Analytics gets informed. So if a user clicks one page, then another, Analytics knows they were on your site for at least as long as the time beween those clicks. What Analytics doesn’t know is how long the user looked at the last page in their visit. (That’s because Analytics never got told about the “next” page the user clicked, on some other site.)

So to Analytics, a bounce looks like a zero-second-long visit. The user may have looked at the page for 2 minutes, but Analytics can’t tell you that.

Last week, I started testing a new system called userfly, that helps penetrate the veil and look at what goes on during a page view, including during bounces. Like Analytics, userfly works by having you embed a line of JavaScript in your web pages, like this:

Once installed, userfly records a ton of information about the user’s session, far beyond what Analytics tracks. Important details include:

Where the user scrolled while reading
Where the user’s mouse hovered while reading
Areas the user clicks (including “failed” clicks where your navigation confused the user)
The user’s browser window size

(There are huge privacy concerns that I’m sure will be talked about at length regarding userfly. I believe we’ll end up using an image on the page to warn savvy users that we’re recording their session, and let them disable that behavior if they want. But I’ll leave that discussion for another day!)

Userfly charges $0.05 per recorded session, in bundles of 100. For $5, you get 100 “videos” of real user sessions with your site. This is an incredible value relative to other solutions (focus groups, usability studies, etc.), and brings this solution down to a level any web site can afford.

Once you have sessions recorded, userfly lets you play them back and see what each user did on your site. This is great information for understanding usability, and hugely useful for finding navigation problems on sites. But for my money, the most exciting thing is being able to see what really happens during a bounce!

Now instead of thinking users are clicking to a page and going away instantly (remember, Analytics shows a bounce visit duration as 0 seconds), we might see that in fact the user read the above-the-fold content, scrolled down, ran their mouse over an important phrase, then disappeared. Maybe this was a satisfied visitor after all, and we just need to do a better job of drawing them into creating a relationship with the site!

On some sites where I’ve installed userfly, I’ve seen “bounces” as long as two minutes. That’s a slow bounce, full of information about user interest and intent!

A few bounce-specific insights you might gain with userfly:

What parts of your pages are “hot”. Where do users scroll, where do they move their mouse?
What parts of your pages really “bounce”. Do users bounce instantly (maybe headline problems), or do they seem to consistently bounce at the end?
Navigation problems. In many sites, bottom-of-page navigation is absent or terrible. Your “privacy policy” link shouldn’t be the bottom-right link on your pages! That’s prime real-estate for someone who’s read the whole page to be drawn into a call to action!
Page-break opportunities. On blogs and other content-heavy sites, it may make sense to provide intro content, then have a “read more…” link for folks who want more detail. This lets you cross-promote and understand visitors better, and gets them clicking on the site.

More on userfly another day. I’ve got to go review more user sessions now!

-glenn

Getting Referring URLs with Analytics

Glenn Crocker — Thu, 12 Mar 2009 17:53:40 +0000

One of the great mysteries of Google Analytics is just why they won’t let us see referring URLs of our users. Probably some privacy concern of Google’s, but it’s easy as pie to get from web server logs, so I don’t understand the issue.

I’m so dependent on Analytics for day-to-day work that I really would prefer to have all the info in one place (Analytics) instead of having to integrate log-based reporting (and deal with clients asking why the two give different numbers!). Here’s how to do it:

Reuben Yau has a great hack for grabbing users’ referring URLs over here:

http://www.reubenyau.com/google-analytics-hack-obtaining-full-referring-url/

The idea is that you create a custom filter like this:

Then you’ve got a nice complete referring URL in your user-defined field. You can see that over in Analytics here:

Works like a champ. This even works with the new-ish “Custom Reporting” feature, as the referring URL will show up in the Dimensions: Visitors: User Defined Value field, like so:

I hope this saves you some time and lets you find new info about your visitors with Analytics!

-glenn

Handling 404s with Google Analytics

Glenn Crocker — Wed, 11 Mar 2009 13:50:51 +0000

I normally set up sites to just log 404s and check the log files for problems, which is fine for me, but hard for clients. Here’s how I recently set up a site so the client’s marketing folks could spot 404s on their own:

Google Analytics works through JavaScript in the browser. As the browser loads each page on a site, it calls home to Google and gives Analytics a bunch of information about what the user is doing. Here’s the usual block of code:

Works like a champ for normal pages, but your 404 page is often special. To test, head to a made-up URL on your site, and see what it looks like:

http://www.yoursite.com/20092734nv3A.php

Chances are, this will give you a terrible web server or browser error message, and that needs to be corrected so you can take control of the 404 page. If you’re using Apache with the default config, it’ll look like:

To fix this for Apache, use this directive in httpd.conf:

ErrorDocument 404 /404.html

For ColdFusion, use the ColdFusion admin to set the missing template handler for .cfm files.

For IIS, use the MMC management tool, click to ‘Custom Errors’, and change 404 to: URL and /404.cfm or whatever your 404 page is, like this:

Once you have this set, your 404s will go to a page you can clean up, like this:

It’s best to have a mini sitemap on this page, so folks can find what they were actually looking for, of course!

Now, back to Analytics. On this 404 page, we’ll change the JavaScript to send ourselves some clues about the 404 in Analytics. Change this line:

pageTracker._trackPageview(trackPageview();

to this:

pageTracker._trackPageview(trackPageview(”/404.html?page=” + document.location.pathname + document.location.search + “&from=” + document.referrer);

Now your 404s should show up in Analytics as if they were on a “virtual” page called 404.html, with parameters including the missing page AND the referer (so you can ask for the link to be fixed). This will show up in your ‘Top Content’ report.

Very handy!