Sudden drop in 404 errors

Alison Hinchman

When we migrated our web properties to Convio in 2008, we also switched our DNS. Understandably, we had a great number of broken links, both internal and external. Eventually we reduced our 404 pages to about 6k-9k per month. And while we thought this was high, we were unable to reduce it further. On October 26, 2009, our 404 pages dropped to double digits--that is, from 9,000 per month to about 25 per month. We use Google Analytics and a custom 404 Oops page. Does anyone know anything either within Convio or external search engines that changed during that time that could account for this?

Adrian Cotter

Seems a little odd. did you check a sampling of pages that were causing the 404s before to see if they still are? An easy explanation would be that something in the 404 process changed (the page actually getting the 404 without analytics anymore?)

Alison Hinchman

Adrian Cotter:

Seems a little odd. did you check a sampling of pages that were causing the 404s before to see if they still are? An easy explanation would be that something in the 404 process changed (the page actually getting the 404 without analytics anymore?)

Hmm, yes we did find a specific page with a dead internal link.  In October, the link appears but now it doesn't.  No idea what has changed to cause this to no longer record as a 404.

Bruce Keilin

In October 2009, we made a change to the way CMS deals with URLs that point to nothing. For most URLs, we continue to serve the custom 404 page (which in your case includes the GA code) but for URLs that are for images (e.g. PNG, JPG, GIF extensions) we simply return a lightweight, static error page (which does not include any GA code.) The reason for this change was twofold:

1. It seems highly unlikely that the end user would be directly navigating to the missing image (and therefore see the simpler error page) but rather that in almost all cases, the missing images are referenced from HTML pages and therefore the error pages for them don't get displayed.

2. We enhanced the error page handling to check the Convio Online Marketing host to see if the request was for a page that lives there and not in CMS. This is very useful for sites that were originally hosted in PageBuilder but which mostly migrated to CMS. Since CMS would own the old PB hostname in that case, the old PB links would direct people to CMS. So CMS did it's best to get you back to the right place. This checking is too expensive to perform on requests for images, so those were just nipped in the bud.

Does that clear up the mystery?

Alison Hinchman

Bruce Keilin:

In October 2009, we made a change to the way CMS deals with URLs that point to nothing. For most URLs, we continue to serve the custom 404 page (which in your case includes the GA code) but for URLs that are for images (e.g. PNG, JPG, GIF extensions) we simply return a lightweight, static error page (which does not include any GA code.) The reason for this change was twofold:

1. It seems highly unlikely that the end user would be directly navigating to the missing image (and therefore see the simpler error page) but rather that in almost all cases, the missing images are referenced from HTML pages and therefore the error pages for them don't get displayed.

2. We enhanced the error page handling to check the Convio Online Marketing host to see if the request was for a page that lives there and not in CMS. This is very useful for sites that were originally hosted in PageBuilder but which mostly migrated to CMS. Since CMS would own the old PB hostname in that case, the old PB links would direct people to CMS. So CMS did it's best to get you back to the right place. This checking is too expensive to perform on requests for images, so those were just nipped in the bud.

Does that clear up the mystery?

Hmm.  The timing is right but we were never hosted in PageBuilder.  Preservationnation.org has always been CMS.  And since February the majority of our images have been stored in an external IMS.  Our analytics guy is back in the office this week.  I'll report here anything he can figure out.

Bruce Keilin

Alison Hinchman:

Hmm.  The timing is right but we were never hosted in PageBuilder.  Preservationnation.org has always been CMS.  And since February the majority of our images have been stored in an external IMS.  Our analytics guy is back in the office this week.  I'll report here anything he can figure out.

Doesn't matter that you weren't in PB. The fix was for PB migrations, but the side effect applies to everyone. It would only take one or two popular broken image references to cause a dramatic change in results. Let us know what you learn when your analytics guy takes a deeper look.

Alison Hinchman

Bruce Keilin:

Doesn't matter that you weren't in PB. The fix was for PB migrations, but the side effect applies to everyone. It would only take one or two popular broken image references to cause a dramatic change in results. Let us know what you learn when your analytics guy takes a deeper look.

Interesting.  Thanks for the info and clarification.