Google killed the radio star

The man in the picture at right is Mika Makelainen, a well-known Finnish television correspondent (currently living in the US) and a leading DXer: DXing is the hobby of listening and identifying far-away radio stations by tuning in to their signals.

Mika is also the editor of DXing.info, a very popular website in that community, where he tells about recently-identified or newly launched stations and publishes among other things profiles of radio stations (including clandestine ones, such as those currently crowding the waves in Iraq) and up-to-date technical information about geomagnetic storms and other similar phenomena that can have an impact on wave propagation; hosts community forums; and shares "DXpeditions reports" (about going off to Northern Finland or Australia or some parts of the US, setting up special gear and spend days trying to identify the distant signal). In the picture, Mika is setting up to listen during his recent DXpedition to Lemmenjoki, at the far northern end of Finland. (Full disclosure: Mika is a good friend).

Even a superficial look shows that DXing.info is a great site: full of information relevant to its target group; well-written; simple to navigate; fast to download; not overloaded with unnecessary gadgets; coded in basic HTML. It hosts a "shop" which is a simple page linking to DXing books and gear on Amazon. But the rest - and there are hundreds of pages - is good, sound information and journalism, including exclusive content, written by Mika and many other contributors.

And among DXers around the world it's a popular site. It is the first external link in the Wikipedia "DXing" page and the second on the "Shortwave" page. It shows up in the top 4 search results for "DXing" on Yahoo, Ask.com, A9 and other search engines. It is mentioned by serious universities, quoted by the BBC and by the New Scientist, linked to from public libraries.

But not by Google. If you search "DXing" on Google, Mika's site won't show up. Not in the first page of results, nor in any other page. If you search "DXing.info", you get a list of pages that link to the site, or mention it, but the site itself is nowhere to be found. If you do a reverse link search ("link:dxing.info") to see who is linking to it, Google tells you that "your search did not match any document". If you go to the Google Webmaster Central and check the status of the site in the Google index, you're told that "Google does not know about all the pages of your site" - which is a slightly less drastic way to say that the site is not listed in the G index.

This is all news to Mika. For years, DXing.info (started in 1997 under another name, got the current domain name in 2002) has been number two when googling "DXing" and has ranked relatively high also with a few other common DXing-related search terms. Its Google Pagerank has been 6, the highest in the world for DXing websites. Then all of a sudden last November, or maybe a few days before, it literally disappeared from Google's directory and search database. Removed. Gone.

"I have no idea why. I discovered it accidentally when googling. I have contacted G by email, but not surprisingly, no response", says Mika.

But the figures are telling: in October, DXing.info was getting about 1000 visitors a day; after the direct visits (people typing the URL in their browser), the top referral page (page "sending" visitors) was Google.com, with 4806 visits; 3400 visits came from Google image search; a few thousands more from the various Google national sites (.co.uk, .com.mx, .ca, etc). Now, DXing.info is getting about 600 visitors a day, trend downward, and no Google site is sending it any significant traffic.

DXing.info is apparently not the only site that has had problems with the Google indexing recently. The other day Mark Glaser at MediaShift had an interesting story on how just before Christmas a bug made a group of popular sex blogs disappear from the Google search results (I understand that they have all been restored in the meantime). In the past, most notably in 2003 when Google performed what was called the "Florida update" to its algorithms, other sites of small businesses such as FindGreatLawyers.com or Unforgettable Honeymoons also lost positions in the rankings, as Danny Sullivan documented in this very detailed article.

Placement on Google Search is becoming a question of life and death for many small websites. This is a testimony to Google's excellence, but that a single company can have such a decisive role on the fate of others, is quite a new phenomenon, and a worrisome one. While the sites mentioned in Danny's article lost their top placement in Google's results, and the same happened to the sex blogs, which were victims of an effort by Google to tweak its indexes so that it can keep out blog spammers and other unsavory types, they weren't gone: they were just demoted to lower rankings.

DXing.info, which doesn't have anything do to with sex, nor any practice that may relate it to spam, and doesn't even use sophisticated "search engine optimization" techniques that may look suspect to Google's systems, has instead simply vanished from the Google index, after years of being considered among the top two DXing sites in the world. Why does Google not like it anymore? No idea. (I've sent a mail to Google a few days ago, but it has also remained unanswered).

Mika wonders whether this may have something to do with the fact that the site carries previously-unpublished information about the radio propaganda efforts of the US military in Iraq. I am skeptical (although in the recent Google Earth controversy a G spokesman told the Telegraph that Google is "always ready to listen to governments' requests") because that article is about three years old, and it can still be reached through the many links that point to it from other sites which remain indexed by Google. I prefer to believe that the whole thing around DXing.info "disappearing" from Google is a snafu.

Google seems to have been very responsive in the case of the sex blogs - with their head of Web spam, Matt Cutts, getting directly involved as soon as the story started circulating and the noise level going up. It's commendable. But those were sex blogs, including that of a very visible San Francisco Chronicle sex columnist, getting help from the most-visited US blog: Google could certainly not stand still while being accused of censorship, intentional or unintentional. I'm wondering whether this responsiveness will extend to the non-sex, non-boingboinged, non-West-Coast, DXing.info. Stay tuned.

UPDATE 22 Jan 07 - DXing.info is back on Google

Comments

[Johan Meezen]

This is really a shame by Google.
It's my no.1 best DX info page on the internet.
So it should return to there where it belongs!

Johan Meezen
Netherlands
Member: BDXC JMn-2648

[Vanessa Fox]

Hi, I'm the product manager for Google Webmaster Central. We're always on the lookout for issues like this, and I know it can be frustrating when things like this happen. I'm not sure what e-mail address that you and Mika used to contact Google, but two good ways of contacting us are by posting in our discussion forum (linked from webmaster central) -- we monitor that forum for issues like this and post replies when we can be of help,and by filing a reinclusion request in webmaster tools. In this case, since the site seems to be missing entirely, I would suggest that Mika file a reinclusion request so that we can review the site and see what might be happening.

I will take a look as well to see where the issue might be here.

[Mika Makelainen]

Thank you for reacting to the issue. Filing a reinclusion request is not that simple. The reinclusion form in the webmaster tools requires you to:
1) confess having violated against Google guidelines,
2) promise not to do it again, and
3) explain what you have done to cause your own expulsion and what steps you have taken to remedy the situation.

What if you don't know why the site was banned in the first place? I haven't used any SEO services, or tried to deliberately improve the ranking of the site. It is difficult to fix something that you don't know is broken. Also it is difficult to avoid committing the same sin again, if you don't know what the sin is.

The encoding for the site has remained practically the same since 2002, but perhaps the rules of good behavior have changed, because the website is now suddenly being penalized.

I have received a couple of suggestions on how to make the website more likeable in the eyes of Google, and I have implemented them, but I still have no idea what the reason for expulsion was. The website is extensive, containing thousands of files of unique content, and it would be helpful to get Google's comment about what exactly is the problem.

[screu]

Let's imagine the worse. Google want to be closer (participations) to xing network and kill a name close to xing website www.xing.com/

[Jozef Schildermans]

The site is now back in Google when searching for dxing. However, only 58 pages or so are indexed, of which only the first 6 or so are considered unique. Still, one would like to know how such a thing is possoble, especially as this not the only example of such an occurence.

[Mika Makelainen]

Indeed, DXing.info is once again listed, but only a small part of its content can be found in the Google index. And what sounds weird, search results vary wildly from day to day. Yesterday 40+ pages were indexed, right now less than 30 (out of hundreds), and these were at least for the most part different pages compared to yesterday. For example the front page (http://www.dxing.info), a report about the U.S. radio propaganda operation in Iraq (http://www.dxing.info/profiles/clandestine_information_iraq.dx) and an introduction to myself (http://www.dxing.info/about/dxers/mtm.dx) were found yesterday, but no longer today. I checked this both through the webmaster tools and using the regular search. Does anyone know what is going on?

As for the changes made, most recently, on Jan. 17th, I uploaded a blank robots.txt file, and edited the .htaccess page by removing reference to use a custom 404 page as well as redirecting all queries without www to pages with www. I don't know if this had any effect, but in any case, these files had remained unchanged since 2002. I did not file a reinclusion request for reasons explained in my previous comment.

DXing.info doesn't rank as high as earlier in search results, which makes me wonder if the site is still under some penalty, but at least I haven't been informed about such, and it doesn't show in the indexing summary of the Google webmaster tools.

And pagerank 6 is of course gone, wonder if it will ever return.

As far as the Google Directory http://directory.google.com/Top/Recreation/Radio/Shortwave_and_DX_Listening/DX_Listening/ is concerned, DXing.info has also risen from the dead, but under a wrong (pre-2002) name. I emailed about this several times around 2002-2004, but have since lost hope of getting mistakes in the Directory corrected.

There are some interesting points in the Google Webmaster Guidelines that in my opinion could be used as a cause or pretext to delist or downgrade just about any website, such as avoiding links to "bad neighborhoods" on the web. When you have user-generated content like a discussion forum, should you delete most or all of the links that ever get posted, just to be sure that you're not inadvertently voting for the bad guys with your links? I already weed about a dozen fake users per day. Should I delete some valuable link collections because I can't police the hundreds of other websites that would be relevant to my users?

While there are millions of websites, and I realize that Google can't respond personally to all the requests it gets, it still bothers me that I'm not getting a personal response to explain exactly what, in the eyes of Google, was (or is?) wrong with the site.

With added power comes added responsibility, and it should include communicating with websites that can be virtually killed by Google. At least Google has all the money it needs to multiply webmaster help resources.

[BrunoG]

Thank you Vanessa for looking into this; thank you Mika for the details. Vanessa: So many people are relying on Google to find information (or to be found), which is a testimony of Google's excellence, but I believe that it would be good to hear an explanation of how this happened - how a website can just disappear from the Google index.

[DXingInfo.Com]

DXing.Info is a great & helpfull site, It should not ever be blocked.