Blog - Page 33 of 92 - Gravity Search Marketing

Ask the Experts: How can I keep private materials…

April 18, 2012
by Gradiva Couzin

Q: We have some private materials on our site, but we’re not able to use robots.txt or a robots meta tag to disallow the pages from indexing. As long as we don’t link to the pages, and only send them out as links in emails, will Google have any way of indexing these pages?

A: As we’ve said many times, the only way to keep materials truly private online is to password-protect them. Even if you don’t link to your pages, here are a few ways that your private URLs might find their way into Google’s index:

People have reported seeing links from within Gmail messages spidered by Google, although this isn’t something that we have experimented with first-hand. We do know that Google spiders Gmail content emails and uses “content extraction” in order to match advertisements, but there is no documentation of other uses of the spidered content.
Private URLs can be seen by Google in other odd ways. For example, if somebody clicks from your private page to another website, then your private URL would show up as a referrer in the server logs. Some server logs are public, or find their way into the public realm, and that could expose the private URL.
If someone visits the private URL while having the Google toolbar activated, then the URL could get collected and find its way into Google’s index.
If the link is included in a listserv email (seemingly private), then that could be scraped and republished on the web.
One of your authorized visitors could post a link within a forum post, on Facebook or Twitter, or somewhere else that he/she believes is private or semi-private, and that link could eventually be followed and indexed.

As you can see there are a lot of possible sources for leaks!

The best safeguard would be to password protect the individual pages, and your second-best approach is to deindex using the robots meta tag. Without password protection, robots.txt or meta robots to prevent indexing, your next best line of defense is to watch for indexing and then do one of two things:

remove any files that have been indexed and put them in a different URL; or
place all of the private content in the same folder on your server, and then use Google Webmaster Tools and Bing Webmaster Tools to remove the files or folder from the indexes if/when it gets indexed.

These should give minimize your exposure in Google, but if your materials are truly confidential, you need password protection. And without a doubt, ixnay on the social security umbers-nay!

Uncategorized

Google Semantic Sidebar [“Knowledge Graph”] Spied in Australian Results

April 12, 2012
by Jennifer Grappone

We recently noticed that Google Australia is showing much more information-rich search results for certain searches than we’ve ever seen. We’re not seeing this type of result in US searches. Could this be a peek at future, more widespread Google results?

Update: 5/16/12 Google has given the sidebar a name — Knowledge Graph — and has officially announced it in this YouTube video. Universal roll-out is likely imminent at this time.

We don’t know how long this feature has been in testing mode, and not everyone on our staff was able to trigger these results. Here are some examples:

A search for “the walking dead” results in a detail-rich sidebar, with cast, episodes, and more.

A search for “apple fruit” results in a sidebar containing scientific details.

(Admittedly, we’re giving the search engine a lot of context here. A search for just “apple” results in both Apple stores as well as some details about the fruit.)

A search for “Iron and Wine,” the name of a singer/songwriter, results in a list of songs.

Why Is This Important?

We think it’s worth watching as a possible indicator of future semantic/information-rich results on Google.com and worldwide. It’s been reported in The Wall Street Journal that, over the next several months, Google will be adding more facts to prominent positions in search results in addition to standard links. Mashable discussed this evolution, too, describing a the development of a resource called Google’s “knowledge graph.”

This Isn’t New, Just Bigger

Google behaving as an “answer engine” is not new. Currently, for example, if you search Google.com for “what is mitosis” you will see a definition, provided by Google, above the regular search results.

Similarly, if you search for “men in black 3 release date” you will see Google’s best guess for the movie release date, based on several sources.

But there’s a big difference between these definitions and the big semantic sidebar we’re seeing in Australian results. This is the first we’ve seen something so extensive.

What’s Driving These Semantic Sidebars?

The results are coming from a mix of sites, and the links within the sidebars go to various websites as well as Google search results. For example, if you click on a movie poster, you may go to a site such as freemovieposters.net, but if you click on the cast image, you may be taken to a Google search result for the actor name.

What Google does to gather this information is crawl websites and look for data that it can understand. This doesn’t always involve semantic tagging (such as schema.org microformatting) but semantic tagging would certainly make it easier for Google.

We tried to find a clear connection among the sites chosen for inclusion in the sidebar, but so far we haven’t identified one. The image results in the sidebar are not the top ranking Google image results, and we see no evidence that the featured sites are collectively using semantic markup such as Schema microformatting.

What Does This Mean for Site Owners?

If what we’re seeing on Google.com.au is a harbinger of a more widespread Google search result, site owners may feel a pinch in their click-through-rates. We believe Google’s goal in adding more semantic answers in its results is primarily to keep searchers on Google; you might say that Google is eliminating the need for searchers to click away from Google.

Some site owners have expressed concern about this as a growing trend for Google to steer visitors to its own properties, and we share this concern. There’s not much we can do but roll with these changes. But now we may have a new featured spot to vie for: how to get sites featured in these semantic sidebars is likely to be a topic of much research and speculation.

UPDATE: Matt McGee of Small Business Search Marketing pointed out that Google was previously testing this change in 11/2011. You can read the story on Search Engine Land. Thanks, Matt!