« November 2007 | Main | January 2008 »

December 28, 2007

How long before the walls around content come crashing down?

Scott Karp of Publishing 2.0 has posted an interesting article today: What Is The ROI Of Requiring User Registration To Access Online Content? , in which he takes a close look at the registration wall used by the New York Times online and wonders whether it is worthwhile.

The theory goes that personal data collected from registered users enables sites to better target ads and charge premium rates. But I wonder whether the lost traffic from users who choose not to jump through the registration hoop — which I bet is particularly true of NYTimes’ large volume of visitors from search engines — outweighs the gain of higher ads rates (assuming NYTimes.com is consistently able to charge higher rates).

As Karp notes, the registration requirement presents a barrier to access for users who come in through a search engine, at a time when NYTimes.com is  focused on growing their readership beyond the current regular readers; and these casual users are just the type of users who are likely to have a lower tolerance for jumping through registration hoops, notwithstanding the NYTimes.com claim that registration takes "only a minute".

In one of the comments to the article, Howard Owens responds by questioning a critical assumption; Owens asserts that the registration requirement does not, in fact, cause traffic to drop.

I’ve run two registration sites, and have spoken with other newspaper.com site managers who have run their own registration-required sites, and two things I found to be true based on empirical evidence:

1) There is no drop off in traffic past the first 60 days of registration (after 60 days, traffic exceeds pre-registration numbers and continues to grow).
...


Personally I believe that the ROI of requiring user registration is questionable at best. Intuitively, it makes sense that at least some users will get discouraged and drop off when confronted with a "registration required" notice; so there's bound to be some negative impact, with all due respect to Howard Owens [perhaps the numbers he saw can be explained by other changes that happened at the same time, such as SEO enhancements that bumped up traffic, compensating for the impact of the registration?].

At the same time, there is another major trend currently under way that will increase the importance of this debate, and in my opinion, accelerate the crumbling of these registration and payment walls.

This big change is in user behavior. Individual consumers are increasingly flocking first to the major search engines when looking for information and data, rather than to individual web sites, even when they already know high-quality sites that can provide the information. It makes sense from the user's point of view: the user wants to find high-quality content in general, regardless of source, and using a favorite search engine is a quick, easy and comfortable way to do that. This overall trend is inevitable and irreversible. As Don Dodge noted in a recent article: Search engines are the Start page for the Internet.

Empirical evidence indicates that, even for major web sites with strong brands, the number of users coming in from search engines is increasing as a percentage of total traffic (although I do not have hard numbers to back up this claim). This forces content publishers to open up more information in order to satisfy those users, which further solidifies the position of search engines as the starting point - which in turn, forces publishers to open up yet more information - and so on, in a self-reinforcing virtuous cycle.

As publishers see this changing user mix - a higher percentage of traffic consisting of new users coming from search engines - engaging those users will increase in importance, and putting barriers in their path will be less acceptable. Instead, publishers will be forced to find new and innovative models for monetization; similarly, user tracking methods will need to be improved to collect data implicitly rather than requiring explicit action from the user.

As the user starts interacting with the site - if she wants to comment, post or otherwise participate, for example - then progressive upsells into registration and payment are perfectly valid and acceptable.  By that point, the site is dealing with thoroughly-engaged users, not casual visitors.

I see it as a question of time before 99% of content from major publishers (NYTimes.com included) becomes free and openly accessible on the Web.

To paraphrase Cory Doctorow (he of the free books !) and Tim O'Reilly, on content: the real danger isn't loss of revenue through sharing, it's obscurity and irrelevance.



December 23, 2007

Web Poll Results: What is the Most Important Component of a Search Engine?

Our last web poll had asked readers to vote on what they considered to be the most important component of a Search Engine - an indication of the areas a small search startup should focus on to help capture market share away from the major search engines.

27 readers voted (thank you!). The poll results are shown below.



These results are interesting because I had expected a higher percentage of votes for the Results Visualization choice, followed by the Algorithm choice, but the votes did not match my expectations. Part of the reason could be that readers of this blog are predisposed to have a stronger interest in the algorithms and strategies used by various search engines than in their UI paradigms.

As for the size of the Content Index, that's a metric that is slowly declining in importance. There was a time when the major search engines would fall all over themselves in trying to top each other in terms of the amount of data indexed; but as the content on the web explodes and grows progressively richer, it simply does not matter as much, and that is reflected by the votes.

As expected, the Query Spec choice got completely ignored. The search engine input spec could be so much richer than a minimal number of words or a single phrase. However, I fear that we're condemned to using keyword-ese for specifying our needs to the Search Engine for the long-term future, which is a pity; like the QWERTY keyboard, it may stick with us well after its usefulness has waned.



December 19, 2007

Search Improvements 2008 - THAT'S IT?

A few days ago, Gord Hotchkiss, President and CEO of Enquiro , moderated a Webinar with the Search 2010 Panel; the panel is a who's who list of stellar participants in the Search space, including representatives from all the major search engines. You can find the actual Webinar and read Gord's post about it here: Search 2010 - A Review.

Gord writes:

I won’t steal the panelists thunder, but the first question I posed to them was what they see as the biggest change to search in the coming year. Most pointed to the continued emergence of blended search results on the page, as well as more advances in disambiguating intent. A few panelists looked at the promise of mobile, driven by advances in mobile technology such as multi touch displays, embodied in the iPhone.

He adds:

[One area]  ... is how search functionality will start showing up in more and more places. Already, we’re seeing search being a key component in many mash ups. The ability to put this functionality under the hood and have it power more and more functional interfaces, combined with other 2.0 and 3.0 capabilities, will drive the web forward.


Charles Knight of AltSearchEngines, in his reaction to the Webinar [ Thomas Jefferson Dines Alone ], writes:

So what did they see as the biggest change coming to Search in 2008?
...

Let’s break it down: 1) the continued emergence of blended search results 2) more advances in disambiguating intent, and 3) the promise of mobile…such as…the iPhone.

That’s it?  That’s what the key major search engine insiders and industry analysts predict for the roller coaster year ahead?  More of the same - and the iPhone?


Now, (disclosure) I'm an occasional contributor to ASE and Charles is a personal friend of mine, so I grant that I'm biased; but I'm with Charles on this one. That's it? Those are the key changes to search predicted by the major search engines for the next year? Is it just me, or do all of these changes seem - evolutionary, not revolutionary?

In a recent article on Future Directions in Search, I highlighted the major areas for potential advances in search: Query specification, Base Index, Relevance Algorithm, Results Visualization and Ongoing Interest (Notification). In that article, I was looking at a much longer time horizon, but I expect that some discontinuous changes will occur in one or more of those areas within the next year.

Search is a highly dynamic field that presently generates a tremendous amount of interest among scientists, engineers and entrepreneurs. (Google's stratospheric market cap has ensured that!). There are so many search startups coming up, many of which are introducing new concepts and technological innovation, such as - Vertical Search: indeed, Spock, and many, many others; semantic search: hakia, powerset; dynamic results visualization: quintura; ways to add value: trulia, zillow; ways to speed up search: vortexDNA; and so on. At least through acquisition, if nothing else, the mainstream search engines should be able to move ahead quickly.

As a specific example, let's look at the Video search space. I recently discovered Mark Robertson's web site, ReelSEO, which is dedicated to SEO/SEM of video content. On his site, Mark hosts the Comprehensive list of video search engines and video sharing sites, which lists over 100+ sites dedicated to video sharing and search. With so many players, surely there's someone who will introduce a new concept or significant change in video search?

Finally, let us acknowledge the elephant in the room. What about - discontinuous improvements to the heart of the Search Engine, the PageRank algorithm? After all, reduced to essentials, PageRank is only an approximation of the authority of a web page or site, based on the value and authority of incoming links. It was certainly an amazing insight on the part of Google's founders, and worthy of the success it attained; but just because all the major search engines use it today does not make it the right way or the only way to identify relevant results.

Perhaps there are other approximations which may work as well or better? Examples of alternative algorithms include: swarm intelligence (like Ant Colony Optimization), human algorithms (e.g. people-powered engines for popular searches and breaking news), brand authority (hey, we use it for everything else in life!), social graph, and many others.

Regardless of what Gord's A-list panel says, there's one thing I'm sure of: 2008 will be an exciting year for Search!



December 08, 2007

Web Poll: What is the *Killer App* for Semantic Web technology?

I recently wrote an article describing working product demos for Semantic Web applications:  The Semantic Web is becoming real - slowly   . For our latest Web Poll, I would like to ask readers their opinion on this topic.


Which of the following do you see as the most likely Killer App for Semantic Web technology - the one that truly puts it on the map?

  • Internet Search (a la Powerset )
  • Enterprise applications - Supply Chain, Sales Force Automation, et al
  • Social Networking (a la twine )
  • Verticals - Travel, Finance, and the like
  • None; this technology is not real anyway!
  • Other: something else?
     

The web poll appears in the left side bar. Please vote and let us know what you think!

--------

Many thanks to everyone who voted in our previous poll about the most important component of a Search Engine. Check back next week to see the final scores.



December 05, 2007

Quintura: Alternative Search Engine of the Year

Over on the AltSearchEngines blog, Quintura has emerged as the winner of the The Alternative Search Engine of the Year - 2007 contest.

All the finalists are excellent search engines, so it must have been a tough call - as Charles readily admits. Regardless, I think it's a great choice, and a well-deserved honor for Quintura.

I've mentioned Quintura before on this blog ; the company offers some interesting and innovative technology. In addition to regular search results, one of their special features is a dynamic tag cloud, that allows the user to navigate visually through the search results space - as the user selects specific tags, the cloud automatically re-orients itself to highlight related concepts.

A mini tag cloud for this blog, powered by Quintura, is displayed below - you can play with it yourself to see how it works. Is it an intuitive way to surf through search results? Let us know in the comments.  A full-size Quintura cloud for this blog is also available on a separate page.





  • Search This Blog


    Web This Blog