The Software Abstractions Blog

Zvents makes Local Search pop!

There is a class of web search engines that can prove even more useful than Google within a certain context. I'm talking, of course, about Vertical Search engines - the writer and tech strategist Sramana Mitra considers them Google's Achilles heel and Profy.com's Cyndy Aleo-Carreira seems to agree. This blog also has long held the position that vertical search represents a powerful mechanism to find information on the web, and is a key category to watch in the search wars of the future. [see: The rise of Vertical Search Engines from Aug 2006].

Another way of achieving a similar focus, in order to improve the relevance of search results, is by segmenting by location rather than by industry vertical - i.e. create a hyperlocal search engine that limits its search results to a given geographical area.

One such alternate search engine is Zvents, which is relentlessly focused on local information, of any sort. This company, which has been around since early 2005, has just introduced an advanced feature called Federated Local search - basically, its own version of Universal Search (recall that Google introduced its Universal Search feature with much fanfare last May).

Federated Local Search: Multi-Dimensional Results for Local Information

What does Universal Search mean, for a local search engine? Initially this was not very clear to me; an email discussion with Paul O'Brien, Director of Marketing at Zvents, inspired me to draw the following diagram:

The basic idea is to enable the user to implement a general-purpose search within a local context. This allows the user to find local information about a given topic, across many different dimensions. For example, a sports fan living in San Jose, CA who tries a local search for the term "hockey", would get the following different types of results:

• Upcoming games for the San Jose Sharks, the local hockey team
• The location of Roosevelt Park Roller Hockey Rink
• The description and link for a local "Hockey Night" event
• Results about relevant personalities (what Zvents calls "performers")
• And other related links ...

Zvents has already partially implemented this vision, although some of the lower-ranked results could provide a better match. Hopefully these will improve in the future as the search index grows and the algorithm improves. A screen shot of this local Hockey search in Zvents is given below.

Similarly, here's a search for the term "Web 2.0" for Cupertino, CA:

Outcome: Relevance

The big advantage of this type of search, over a general-purpose Google or Yahoo! search, is that the user can obtain the benefits of a broad cross-section of results, while still constraining the search to a limited geographical area.

This is not a significant issue in highly developed, urban, technologically advanced areas like Silicon Valley, Boston or New York; but it could one day make a big difference for someone living in David Letterman's "home office" of Wahoo, NE , or even more important, someone trying to find the Boston Public School located in Boston, Ontario - as we've seen before, highly popular keywords tend to swamp nearby long-tail keywords in the search results for major search engines.

From a business model perspective, hyperlocal searches tend to provide highly qualified prospects for local merchants, so I would guess that this type of search is very easily monetizable in the long run.

From a user interface point-of-view, the NLP-like implementation of time period for the search engine ("when: tonight, this weekend, ...") is a nice touch; I tried different possibilities ("next month"), and it seemed to work just fine.

On a more technical note, Zvents has been making waves with the release of its open-source Bigtable clone called Hypertable, which adds a C++ option for this project.

Going forward, it will be interesting to see how Zvents scales to additional locations, and to additional dimensions within each locality. Will it make inroads into the market share for any of the major search engines, or into that of other locally-focused web sites like topix.com and craigslist?

---

Search Improvements 2008 - THAT'S IT?

A few days ago, Gord Hotchkiss, President and CEO of Enquiro , moderated a Webinar with the Search 2010 Panel; the panel is a who's who list of stellar participants in the Search space, including representatives from all the major search engines. You can find the actual Webinar and read Gord's post about it here: Search 2010 - A Review.

Gord writes:

I won’t steal the panelists thunder, but the first question I posed to them was what they see as the biggest change to search in the coming year. Most pointed to the continued emergence of blended search results on the page, as well as more advances in disambiguating intent. A few panelists looked at the promise of mobile, driven by advances in mobile technology such as multi touch displays, embodied in the iPhone.

He adds:

[One area]  ... is how search functionality will start showing up in more and more places. Already, we’re seeing search being a key component in many mash ups. The ability to put this functionality under the hood and have it power more and more functional interfaces, combined with other 2.0 and 3.0 capabilities, will drive the web forward.

Charles Knight of AltSearchEngines, in his reaction to the Webinar [ Thomas Jefferson Dines Alone ], writes:

So what did they see as the biggest change coming to Search in 2008?
...

Let’s break it down: 1) the continued emergence of blended search results 2) more advances in disambiguating intent, and 3) the promise of mobile…such as…the iPhone.

That’s it?  That’s what the key major search engine insiders and industry analysts predict for the roller coaster year ahead?  More of the same - and the iPhone?

Now, (disclosure) I'm an occasional contributor to ASE and Charles is a personal friend of mine, so I grant that I'm biased; but I'm with Charles on this one. That's it? Those are the key changes to search predicted by the major search engines for the next year? Is it just me, or do all of these changes seem - evolutionary, not revolutionary?

In a recent article on Future Directions in Search, I highlighted the major areas for potential advances in search: Query specification, Base Index, Relevance Algorithm, Results Visualization and Ongoing Interest (Notification). In that article, I was looking at a much longer time horizon, but I expect that some discontinuous changes will occur in one or more of those areas within the next year.

Search is a highly dynamic field that presently generates a tremendous amount of interest among scientists, engineers and entrepreneurs. (Google's stratospheric market cap has ensured that!). There are so many search startups coming up, many of which are introducing new concepts and technological innovation, such as - Vertical Search: indeed, Spock, and many, many others; semantic search: hakia, powerset; dynamic results visualization: quintura; ways to add value: trulia, zillow; ways to speed up search: vortexDNA; and so on. At least through acquisition, if nothing else, the mainstream search engines should be able to move ahead quickly.

As a specific example, let's look at the Video search space. I recently discovered Mark Robertson's web site, ReelSEO, which is dedicated to SEO/SEM of video content. On his site, Mark hosts the Comprehensive list of video search engines and video sharing sites, which lists over 100+ sites dedicated to video sharing and search. With so many players, surely there's someone who will introduce a new concept or significant change in video search?

Finally, let us acknowledge the elephant in the room. What about - discontinuous improvements to the heart of the Search Engine, the PageRank algorithm? After all, reduced to essentials, PageRank is only an approximation of the authority of a web page or site, based on the value and authority of incoming links. It was certainly an amazing insight on the part of Google's founders, and worthy of the success it attained; but just because all the major search engines use it today does not make it the right way or the only way to identify relevant results.

Perhaps there are other approximations which may work as well or better? Examples of alternative algorithms include: swarm intelligence (like Ant Colony Optimization), human algorithms (e.g. people-powered engines for popular searches and breaking news), brand authority (hey, we use it for everything else in life!), social graph, and many others.

Regardless of what Gord's A-list panel says, there's one thing I'm sure of: 2008 will be an exciting year for Search!

---

Reactions: The Future of Alternative Search Engines

I recently wrote an article on this blog about the exit strategies for alternative search engines, that highlighted the recent and growing trend of publishers acquiring search engines; I also speculated about Charles Knight's quest to get these Alts to band together in order to grow overall traffic. I've gotten interesting reactions to the piece from some prominent bloggers.

 

I've been a fan of Ashkan Karbasfrooshan, of the HipMojo blog, for a long time. When I asked him about the Alts, his response was as follows:

No one has a crystal ball to predict with enough accuracy what will happen in the next few years with regards to search, but clearly, history suggests that there is always a period of consolidation after innovation and growth, so that should happen in search too.  And, we're seeing the big companies starting to have difficulty adding market share, it's not like adding one of these fringe search engines (including MetaMojo when I refer to alt engines as fringe) will add market share to a major one, but sometimes by buying a small player, a major company adds technology, know-how, but most importantly, brainpower.  That is the single most important variable for who will win or lose in search over the next decade: you can best deploy technology and market it, and not who can best develop technology.

One thing I was unsure of was whether traditional media would be doing the buying, I think ultimately traditional media will partner with search companies because it is not in the DNA to understand which search companies to acquire... which is a shame, because it's to old media that young search companies can most add value to...

 

David Berkowitz, of Inside The Marketers Studio stuck to his usual line where the Alts are concerned - he doesn't think most of them are good enough to get acquired or survive on their own:

Honestly, I don't think many of the alternative engines are good enough to be acquired, though a few will find that as an exit strategy, and a few others will survive as niche alternatives.

 

Microsoft's Don Dodge, who has a terrific blog The Next Big Thing, got so interested that he wrote a separate blog post to address this question. Here's a snippet from his post:

My thoughts? Some of these will be acquired by the big search engines or big content publishing networks. Most of them will fade away. I don't see any of them breaking out and creating a significant stand alone business with the possible exceptions of Powerset, Hakia, and Mahalo.

Read Don's post to find out where he thinks the big, untapped opportunities lie.

 

Bob Warfield, of the SmoothSpan blog, responded with an interesting blog post, proposing that:

Alt Search providers can get together and create an Open Sourced Collaborative Search-Oriented Social Network ...

In essence, he argues that the Alts could get together to share the costs and burdens of web crawling and the underlying infrastructure. This would help them to reduce the gap with the big players, which have a huge advantage in terms of resources. You can find his post here.

 

Conclusion

The overall consensus is that, unless something changes, most of the alternative search engines will not survive; some will get acquired and a few players may do well on their own, especially if they focus on a specific niche. A convergence in markets (e.g. local and mobile), or joint efforts on building the infrastructure, would give them a better chance.

Personally, I see the lack of web traffic as being the single biggest weakness of the alternative search engines; regardless of their cool technologies, innovative architectures or stunning visualizations, they cannot survive without getting the word out and capturing market share in search. There are simply too many web site destinations for the average user to remember. If the Alts could somehow cooperate to provide a single entry point that then branches off to different specializations, it would be a huge step forward!

---

Survey: The Future of Web Search

In this latest poll on the Software Abstractions blog, we would like to ask readers about the Future of Search .

Specifically, which features do you see as the most important ones for Web Search in the future? Vote now and let us know!

-------------------------------

Update: This survey is now closed. You can view the results of the survey here.

---

Shrinking the World Wide Web

The folks at The Web's Too Big feel that - you guessed it - the web has gotten way too big, making it very difficult to quickly find quality content. So they're doing their best to shrink it, for a very specific domain, to a manageable level to make finding things easier; in other words, they're building their own proprietary Vertical Search Engine.

Searching for PR firms in the UK

The first vertical tackled by this search engine is a search for UK PR agencies and PR information. The user enters a search term which is used to search for information on the sites of these agencies. For example, I typed in "fast food" and got an impressive list of results from a variety of search firms: Levick Strategic Communications, Black and White Communications, Edelman, Layzell PR and so on. (Caveat: Not being involved with the PR world, I personally have no idea if this set of results is relevant and useful - any PR folks out there care to comment?)

Mike Bygrave, one of the co-founders of thewebstoobig, explains it as follows:

The Web's Too Big:PR is a two-part B2B service, comprising a vertical search engine and a lead enquiry system, which provides interested parties with a simple way of searching for and engaging with UK PR agencies.
...
The search engine is primarily intended as a place where users looking to hire a PR agency can do some research and find a suitable agency. Google indexes something like 8 billion pages; some of the new vertical searches index 'only' several million pages. At The Web's Too Big we're trying to take a completely different approach, by selecting a narrow vertical and indexing only sites that fall firmly within that vertical. It's impossible to know exactly how many PR agencies (with websites) there are in the UK at any given time, however we're absolutely confident that we currently index well over 90% of them. Our site users know that everything they see on our site will be very closely related to PR - they're not going to be bombarded by random advertisements (particularly from keyword scammers).
...
The lead generation part of the site was really just a natural progression from the search engine; it allows users to bypass the research element if they want to - they can simply enter their PR requirements into our enquiry form, and then send it off to up to nine pre-selected PR agencies. This saves the user a lot of time and hassle, and it provides the PR agencies with 'hot' leads.

Unique service: PR enquiry

One unique offering from thewebstoobig is their "send enquiry" service that allows you to contact a list of PR agencies directly. You supply some key information about your needs (such as industry, location, timeframe, budget), and your request is matched up with a list of PR agencies; from the agencies' point of view, it works as a lead-generation service, thus providing value to both users and agencies.

 

 

It was the "send enquiry" service that got me interested in looking at this site in detail; this type of service is a perfect illustration of the power of vertical focus.

As I've said before (here and here ), creating specialized, unique services for a particular domain is where VSEs can excel. By focusing relentlessly and with depth on the needs of specific users segments within that domain, VSEs can create unique, defensible value propositions and differentiate themselves, in the minds of users, from the larger mainstream search engines like Yahoo and Google. At the same time, by attracting very specific sets of web site visitors with well-defined characteristics, they can offer, in effect, highly-qualified prospects for marketers.

This seems like a winning situation for all constituencies - users, because they can quickly and easily cut through the clutter; the business, because it can focus on the given domain; and marketers or advertisers, since they can target specific demographics and user types. Only time will tell if specialized search engines like thewebstoobig can divert a significant portion of their target audience away from Yahoo! and Google, or whether the mainstream engines will somehow co-opt this type of functionality into their own offerings. More on that in a future post ...

---

---

Looking for web hosting can be a slightly complex process if you need specifics like Unix web hosting, so before you sink time and money into a web hosting company you should probably research various budget web hosting providers online.

---

---

Elsewhere: Wikiseek, SimplyHired, Wazap!, Findory

Wikiseek is a new search engine (not to be confused with the recently announced Wikiasari), whose searches are restricted to Wikipedia pages and only those sites which are referenced within Wikipedia. It is based on technology from SearchMe , a Palo Alto startup. The results UI for Wikiseek is very similar to the familiar Google interface, with search results on the left and sponsored results on the right, with the addition of a tag cloud of related articles on the top. The majority of the revenue generated by Wikiseek advertising will go to the Wikimedia Foundation.

SimplyHired , one of the leading vertical search engines for job search, continues to raise the bar in new and interesting ways. An article from the LifeTwo web site, describes sH's search offering for "mature worker friendly" companies - i.e. a search engine for job seekers aged 50 and above . This type of parametric search, that leverages the focus on a specific vertical, is an ideal complement to the standard capabilities of a vertical search engine - as we've said here before  .

Wazap!  , a popular gaming vertical search engine in Japan and Germany, has raised $7.9 Million for its US Launch according to the Search Engine Journal and other sources.

Findory  , a personalized news search engine with a devoted fan base, shall be fading away  , as described in a post on Greg Linden's blog  . Greg, the developer behind Findory, says that the site shall continue to live on, "on autopilot", throughout 2007.

Here are a couple of interesting older posts that I came across recently, about Textbookland  , a VSE for (what else?) text books, and BigWords  , which "searches across many stores for the best prices on textbooks, DVDs, music, supplies, and games".

Elsewhere, of interest: Google Patent Search, Kosmix, Zoomf, Nestoria, Buykee

The Read/WriteWeb takes a comprehensive look at search in: The Race to Beat Google . Regular contributor Alex Iskold categorizes the contenders into three groups: better technology, better UI and vertical/semantic search.

In other news:

• Google recently unveiled a patent search engine  ; here's a great article about it   by entrepreneur Paul Allen (the lesser) that also takes a look at the bigger picture.
• Mamma and Looksmart  : Ashkan Karbasfrooshan looks at Search engine stocks on Seeking Alpha.
• From the Official Kosmix blog: Kosmix gets a new UI
• Real estate vertical search in the UK  : Zoomf  and Nestoria  , and their similarities to Trulia
• Here's a review   of Buykee, a Chinese social shopping site consumer electronics feedback community.
• I found this interesting article (it's not new), by Ramana Rao: Beyond Search is REAP

Mid-Week Update: confab.yahoo, Oodle, GenieKnows.com, Trulia

Yesterday, I was over at the confab.yahoo   event on Prediction Markets . You can read my report  about it over on the Read/WriteWeb  . (Thanks, Richard MacManus, for the opportunity!)

The big news in vertical search this week was a white paper   released by Slack Barshinger   and SearchChannel, which projects that revenue from b-to-b vertical search engines will reach $1 billion   by 2009.

In other news:

• Oodle rolled out it's new Personals search service
• GenieKnows.com   launched its Vertical Search Engine
• Roger Ehrenberg talks about Domain Expertise being the key to next-generation search
• Here is an interview with Trulia co-founder, Pete Flint

---

Mid-Week Update: Retrevo, SimplyHired, TechCrunch, Market10 and Search Insider

Notes from around the web
=========================

Alex Iskold just added a great article about Retrevo, the consumer electronics search engine, on Read/WriteWeb, discussing some of the specialized functionality offered by Retrevo, as an example of the rich capabilities a vertical search engine can provide. For VSE-watchers, this article is definitely worth checking out!

I'm a big fan of Alex's writing, but in this case, I disagree with his thesis that current VSEs embody what vertical search will become - I would take it much further, and say that the rich features that Retrevo is currently offering (which are already compelling by themselves, like the new Daily Deals feature) are barely scratching the surface of industry-specific search services. It seems likely that specialized vertical search engines, such as Retrevo, will continue to evolve progressively more domain-specific and richer "findability" features, so that eventually these sites may look nothing like our general-purpose search engines of today.

[Imagine a navigation/parametric "find" interface for electronics products, that allows you to find all the DVD-players within a given budget, that support specific audio-visual features AND are compatible with your existing big-screen HDTV; and then looking for a cable that will provide the best impedance-match between the two. Try doing that with Google!]

******

SimplyHired, one of the leading vertical search engines for jobs, has just announced a partnership with The New York Post and MySpace: it will provide its job search capabilities to nypost.com and has launched Resume Post, its resume syndication service, on MySpace Jobs. Interestingly, this "cheeky" press release is written more in the style of a hollywood tabloid story than that of a staid tech-oriented business article. Methinks SimplyHired must be a really fun place to work!

******

From Hollywood tabloids to dating sites ...

Over at TechCrunch, Steve Poland complains that online job search still sucks. What he would like to see, he says, is a progressive startup that does for jobs what eHarmony has done for online dating - specifically, have candidates go through a 30-min application and testing process, and then match those candidates with their ideal job openings.

I think the basic idea is sound, although a 30-min application process may be carrying it too far - a more subtle approach that starts with a short initial questionnaire that is continually updated and refined as the search engine learns more about you, might work better. Even a candidate who is not actively in the job market might happily subscribe to a "job feed" of matching openings by industry, technology, geographical location. This could work equally well for other markets - a "house feed" of open houses in a given location and price range, or a "deals feed" for special deals on a particular type of stereo equipment.

******

Speaking of job search sites, I recently came across Market10. Started by Rob McGovern, founder of CareerBuilder.com, the company bills itself as "the Top Performer's job market". In many ways, Market10's approach comes close to "e-Harmonizing" the job market, exactly as Steve was suggesting in the TechCrunch article. The company has raised financing from Menlo Ventures and New Enterprise Associates (NEA).

******

A knewworld article has Q&A with David Berkowitz, director of strategic planning at 360i and a regular columnist for MediaPost’s Search Insider publication, in which he discusses search engine issues and barriers for adoption for Google CSEs.

---

A conversation with Tom Eng, Founder of Healia

I recently had a chance to chat with Tom Eng, Founder and Chairman of Healia. Healia is a consumer health search engine. Here are some excerpts from our conversation:

Q - Tell us about Healia.
[Note: For basic background information, see Healia's About page and this excellent post by David Berkowitz.]

TE - Healia is a consumer search engine for medical information. We use technology and rules in our search engine to enable users to find information more easily. For example, consumers may not know the relationships between commonly used terms and related results - there is a gap in the medical informatics world between popular consumer terms and medical information; our semantic matching algorithms can match the related medication or drug. We have our own customized taxonomy. We also have our own web crawlers and parsers and indexers, and use quality scoring algorithms to provide high quality search results.

Q - Some medical professionals frown upon patients using the web to look up medical information; they question its reliability, and feel that "Health information should only come from medical professionals!". How do you address that sentiment?

TE - That is an old way of thinking. Consumers today, in any interaction, are no longer passive - they want to drive it themselves. Studies have shown that people turn to the web first for medical information. Even after consulting their primary physician, they feel better after validating the information from other sources, of which the web is a primary example. There has been a major change in attitudes in the last five years, and Healthcare is catching up rapidly in this context. So we don't see this as continuing to be an issue in the future.

Q - Google OneBox is an attempt to provide vertical search functionality from within a Google web search. How do you think this feature will affect you, and Vertical Search Engines in general?

TE - OneBox and Google Co-op is an improvement, certainly. But using Google for a medical information search - to find out about diseases, symptoms, drugs or medical terms - has its limitations.
First, [Google] results are classified based on voluntary self-tagging by the publisher, which has inherent problems: people gaming the system, built-in bias, how many people are able to do this. This approach is not scalable for less-common diseases and conditions. In contrast, Healia uses algorithmic tagging - we use our technology to automatically add tags and relationships to any collection of content. We also use quality scoring algorithms for spam prevention.
Second, we provide a series of filters that allow a user to focus on results that apply to their unique situation, by enabling post-filtering of search results: by user demographics - such as age, sex or heritage, by the type of article - basic or advanced, and so on; this is especially important in a medical context, since symptoms and drugs for a given disease can vary based on patient attributes.
Third, since we apply a semantic network to the content and understand the relationships between commonly used search terms and related results, we can guide the consumer to other related searches or appropriate sources.

Q - Now that Google has released technology to build custom search engines based on their web crawlers, do you see a big threat from third-party content publishers building their own trusted-source search engines or providing canned searches?

TE - Well, Rollyo and Yahoo! Search have provided this type of technology for a while (although Google got more press), and we haven't seen a lot of that yet. A Google-based custom search engine would suffer from many of the same limitations described above. Another issue is the requirement to include AdSense advertising in the search results. Also, this gets the publisher into a field [search] in which they're not experts - they might choose to partner with someone like us instead, with semantic technology, post-search filtering, etc., to guide users and make sure that they can easily find the information they're looking for.

Q - Plus, you may be able to provide related services based on the results or other features specific to the medical domain, something not everyone can do.

TE - That's something we're looking into. Watch this space ...

Q - One word: Powerset. What do you think about the sudden interest in natural-language search?

TE - We already have natural language associations embedded into the semantic technology we use. Natural language for queries - it's been tried before a number of times, so that in itself is not new. I haven't seen the Powerset search engine yet. Let's see what it is and what happens, when it becomes available.

Q - To drop into cliches for a moment: Does Healia serve the Big Head or the Long Tail of Search queries?

TE - I want Healia to be the place for consumers to find the medical information they need. Having said that, a major part of our current business model is to work with partners to provide services. It depends on which part you tackle - we're really only in the beginning stages of our industry. Healthcare is a really important domain for search, possibly the most important kind of search in an individual's life. With high quality and personalization of search results, we're focused on using our technology to help people, so it's more than just a business.

Q - Do you provide an API?
[Note: Healia also provides a widget for web site publishers, available here .]

TE - We do have a web services platform, and an API. At this point, it's not generally available; it's licensed to partners. We have several big customers already using this API, for example, VA (Veterans Administration) uses our WS platform.

Q - What do you see for the future of Vertical Search Engines? Any specific trends?

TE - We're really at the 1.0 stage for Internet Search - what we've seen so far is just the beginning. When Alta Vista was the leading search engine, people thought that was the endgame for search, then Google came from nowhere and changed everything. I think we will see a shift, especially now with Vista coming out, where users will simply search from within the context of wherever they are, rather than going to a specific site to perform a search. People will default to the easiest thing to do; for example, if you are on a financial news site, and if that site offered a vertical search engine that meets your needs, then you might just search right there. If you had an embedded search within your application, you could search from within the app. So the future looks bright for context-specific, personalized search - we will just enable you to search from wherever you are. It's difficult to imagine one provider who does this for everyone. In a sense, Internet search is almost too important a capability to be left to any one company. Providing the underlying search technology as a service, with a high quality of results, will become increasingly important in the future.
Healia is working with partners to provide high quality, personalized search within clients and applications - watch for some exciting announcements and new features in the next couple of quarters.