The Software Abstractions Blog

Web 2.0: The Real Opportunity Lies Ahead of Us

JP Rangaswami wrote an amazing post on his blog a little while ago: Interesting, but of no commercial value , in which he cites a series of examples of new technologies - like email and spreadsheets - that were initially considered simply interesting, rather than useful; now we cannot imagine living or working without those very same technologies. It seems likely that this will happen with today's emerging technologies, like RSS feeds, popular voting, social networking, micro-blogging, crowdsourcing and so on.

History Repeats Itself

We have already seen this happen with Web 1.0. A series of tiny, well-capitalized startups (remember Webvan? ) gained early traction online in a variety of market segments, from books to furniture to pet food to groceries. The large, established brick-and-mortar players were slow to respond.

As a result, a few Web 1.0 companies - but only a handful - did grow into large, successful companies within their domain: Amazon, EBay, Blue Nile, Expedia, Google, and so on, along with a host of second-tier players. Over the long run, though, pre-existing market leaders like Barnes & Noble, WalMart and Petsmart, were able to rebound, leveraging their strong brands and deep pockets to exert influence online and re-assert their dominance of the various segments.

There is no reason to believe that the outcome will be any different with Web 2.0 . The big established companies must enter the field for competitive reasons, either by acquiring, partnering with or replicating the smaller cutting-edge thought leaders. The game is still in the early stages; once the big brands get moving, they will be hard to stop.

Dave McClure addressed this topic in his recent incomparable blog post Web 3.0 isn't the Semantic Web, it's Hailstorm 2.0 . Of course, Dave's thought process is far ahead of mine, but he makes a similar point: the most likely winners of the Web 3.0 revolution - which he defines as an environment in which the user is always logged in even while navigating around the web -  will be vendor groups composed primarily of today's large companies like Microsoft, Google, Yahoo!, AOL, Facebook, Amazon et al.

Users - Now and In the Future

Who are the heaviest users of Web 2.0 "connectivity" tools like blogs, wikis, twitter, social networks, and so on? There are three major demographics:

• Young people in their 20s and 30s (and possibly even younger)
• Users who work in technology, or play in it (such as gamers)
• Users who live in high-tech hubs, such as Silicon Valley, Boston, New York, etc.

Taken together, these demographics represent only a fraction of the online population at large. The migration of the mainstream users is yet to come. And therein lies a major opportunity ...

As Web 2.0 technologies start to trickle outside of the core Early Adopter demographics - as they "cross the chasm" - we should see a massive increase in adoption for these technologies in both the Consumer and Enterprise space. This will lead to an increase in their value due to network effects, which are particularly meaningful for these social connectivity tools.

Companies that provide products and services to help the transition of mainstream users and facilitate their adoption in these two areas can profit handsomely from these technologies.

---

Enterprise 2.0: The Engineering of Marketing Online

When I was talking with my friend Shreesha Ramdas (from OuterJoin ) last week, he shared a perspective that really resonated with me. In a nutshell, he believes that the Marketing of online products and sites is rapidly becoming an Engineering function, both in terms of operational activities and measurement.

The more I think about it, the more I'm convinced that he's on to something. Marketing of online products and sites is inherently different from classical marketing. Unlike regular marketing channels, online campaigns allow marketers to proceed systematically step-by-step along a predetermined course. The results of each distinct campaign can be measured precisely, even when multiple campaigns are going on simultaneously. Most important, the market can be broken up into thousands of micro-segments, with targeted campaigns aimed at each one.

Online Marketing Activities

This technical approach can be applied to all three types of online marketing activities:

I. Site Effectiveness

A key goal is to increase the popularity of a given web destination. The popularity measure can be expressed in a variety of ways: page views, distinct users, time-on-site, user engagement, and so on; all of which must eventually translate into a conversion rate - the effectiveness with which you can get the user to take the desired action on the site, whatever that might be.

These popularization activities can eventually be boiled down to two main approaches, which are complementary: Search Marketing (PPC) and SEO (Organic results).

A large number of technical tools are available to help with both of these approaches, such as keyword tracking and keyword optimization for the long tail. Although good old-fashioned marketing techniques always help (who can resist good linkbait ?), many of the activities that lead to PPC and SEO success can be automated and enhanced using these tools.

II. Joining the Online Conversation

As the revolutionary cluetrain manifesto proclaimed, markets are conversations, and media-savvy companies today want to be a part of that conversation. Online, many of these conversations take place on blogs, social networks and content aggregators. These types of dynamic online properties also provide great SEO benefits and direct user traffic.

Here again, technical skills that enable automated submissions and social media tracking are extremely helpful in gaining traction. The upcoming OpenSocial API (coverage ) will take this to a higher level, allowing a single submission to provide content to multiple social networks.

III. Measurement and Analysis

The two leading tools for online measurement are Omniture and Google Analytics, although many others are available in this fragmented market. The advanced features and sophisticated tracking enabled by these tools can get quite complex and technical - as anyone who follows Avinash Kaushik's blog (Occam's Razor ) knows.

Impact on Traditional Marketing Departments

The level of technical sophistication needed to achieve these goals tends to overwhelm typical Marketing departments. This is likely to have a big impact on traditional marketing organizations in mid-sized and large companies. These organizations have three main options:

• Hire techies directly into Marketing
• Create an Engineering support team answerable directly to Marketing
• Create small Business Units that incorporate both Development+Marketing

Which approach will prevail in a majority of the cases? That remains to be seen ...

---

Tim O'Reilly and Sir Tim Berners-Lee concur: Semantic Web Likely to be Top-Down

In a previous post, I asked the question: Where are the Meaning-Enabled Authoring Tools?, arguing that publishers who regularly post similar content (especially content that conforms to common formats) would get a big advantage from using Semantic Authoring tools for creating new content. By using semantic tools, not only can you get SEO benefits and improve findability , the content can more easily be re-purposed for other uses such as web applications and services.

This is essentially a bottom-up approach to the semantic web: adding semantic notation to the content itself. However, as the post went on to say, the prevailing view is definitely a top-down one, viz. that semantic meaning will have to be extracted by applications from perfectly ordinary web pages, and that the adding of semantic knowledge to the content itself is unlikely (aside from very limited contexts, such as Microformats).

Two recent podcasts with two of the leading voices in this space further confirm this view.

Rumors of the Death of Indian Outsourcing - Are Greatly Exaggerated

I've long been a fan of Sramana Mitra - she has a terrific blog and has her own deep definition and framework for Web 3.0 . Forbes.com carries a fascinating article by her this morning: The Coming Death Of Indian Outsourcing , in which she talks about the new challenges facing Indian Outsourcing Companies.

As usual, Mitra's basic analysis is spot-on, although one can certainly take issue with her conclusions. There is no question that Indian OCs (Outsourcing Companies)  now face unprecedented challenges, and cannot carry on with a "Business As Usual" approach for much longer.

This should not come as a surprise; competing on the basis of price alone is never a sustainable business strategy. In a price war between brands (or countries!), no supplier wins. And this price advantage is rapidly eroding, with the steep rise in software development labor costs and the deteriorating strength of the dollar.

Enterprise 2.0: Top 5 Corporate Challenges for 2008 and beyond

The Few, the Proud

A few days ago, in its commentary section, the Wall Street Journal reported on an interview with General James T. Conway, Commandant of the U.S. Marine Corps.

In the interview, Gen. Conway muses on the way the tactics and equipment of the Marines are changing, in response to the unique nature of the responsibilities they have in Iraq and the evolving nature of their mission.

One way the Marines are clearly changing is in the vehicles troops use to patrol in Iraq. "If you look at the table of equipment that a Marine battalion is operating with right now in Iraq," Gen. Conway explains, "it is dramatically different than the table of equipment the battalion used when it went over the berm in Kuwait in '03, and it is remarkably heavier. Heavier, particularly in terms of vehicles.  ....these type of things, make us look more like a land army than it does a fast, hard-hitting expeditionary force."

...
In short, wars have a tendency to change the culture of the militaries that fight them. For the Marines, the cultural change they fear most is losing their connection to the sea while fighting in the desert.

In the midst of all this change, Gen. Conway is worried about preserving the essential character of the Marine Corps, even as the rest of the world changes around it. As an organization, the Corps faces one of the most daunting management challenges in the world: keeping individual Marines highly motivated and getting them to excel at a difficult, dirty and dangerous job, in the face of low pay and extreme working conditions. It is critical to preserve this esprit de corps, even while gearing up for new missions for the modern battlefield.

Corporate Trends

What does this have to do with modern corporate organizations? While the specific conditions are very different - no bullets or humvees are involved - companies have also been facing a set of discontinuous shocks in the last few years, and their pace is only increasing. In many ways, corporate leaders are facing major changes with challenges similar to the ones facing Gen. Conway, to which they must respond quickly and effectively, without losing their own organizational culture and common knowledge.

What do these discontinuities represent? Given below are five significant challenges facing corporate organizations in 2008 and beyond. None of these are new but their trends are rapidly accelerating. For a company to survive and compete effectively, it is imperative that its leaders have a strategy to handle each one.

1. Outsourcing Partnerships:
Corporate outsourcing has been growing rapidly over the last ten years. Initially it started simply as a way to find talented technical workers quickly and at low cost.

Recently, though, this trend has been evolving; outsourcing vendors are now seen as strategic partners who participate in the corporate vision of the Enterprise and share in its successes and failures. Bringing these outsourcing partners into the fold (especially if they are off-shore) is neither quick nor easy.

2. Hyper-Informed Consumers:
The average person in the United States and other developed countries already has unprecedented access to information, more so than at any other time in history; this access is now spreading across the rest of the world following the proliferation of mobile phones.

Coupled with a corresponding increase in the willingness and enthusiasm of users to consume that information - e.g. watching the quarterly interest rate changes by the Federal Reserve is now a national pastime in the U.S. - this means that consumers are now exceptionally well-informed.

Companies must act accordingly; they must either join the online conversation as equal partners (as Cluetrain suggests ), or be left out.

3. True Globalization:
Increasingly, corporations find their talent, their suppliers, and most importantly, their customers, at an international level and compete for them with other multi-national conglomerates, both domestically and abroad. As Thomas L. Friedman's popular book says, the World has become Flat again.

This change brings a whole new set of challenges with it; only organizations with a truly international mindset can survive and thrive.

4. Communication and Collaboration across Distributed Teams: 
With this new diversity of cultures, geographies and perspectives within the Enterprise, it is even more critical to get teams to communicate openly and to embrace a shared vision.

The recent uptrend in the use of Enterprise 2.0 strategies and tools is a positive development in addressing this need. These tools help to bridge the gaps; they promote collaborative design and development, and enable rapid dissemination of information among international teams spread across geographical boundaries and many different time zones.

The speed and flexibility with which these distributed organizations can respond to changes in market conditions, is now a major competitive differentiator.

5. The Dominance of Search:
As commerce shifts increasingly online, the use of the Internet for research, analysis and selection of vendors, and for making purchases, is increasing rapidly. In the future, it is expected to become the primary way users find information.

This means that if a company does not show up near the top of search results for the major search engines, then essentially it doesn't exist for new prospects and even for previous clients. Addressing this challenge requires a significant change in Marketing philosophy. Companies can no longer coast on the strength of their brands; they must continually invest time and energy in refining their Search Engine Optimization (SEO) strategies.

Conclusion

These five trends go together and strengthen one another; so do the challenges associated with them. This positive feedback means that the changes are only going to accelerate in the future.

How is your company addressing these challenges? Add a comment below and  let us know!

(Note: This post previously appeared on Profy.com )

---

Enterprise 2.0: Battle of the IT Supply Chains

My dad pointed me to this fascinating article on IT Outsourcing from Information Week: The Second Decade Of Offshore Outsourcing: Where We're Headed by Mary Hayes Weier. It's a long article but well worth reading. The author captures a growing trend in the outsourcing of IT software: as offshore service providers move up the value chain, from tech support and quality assurance, to software design and development, business process re-engineering and eventually to providing direct business value, the relationship between these vendors and their Enterprise clients is changing. Instead of outsourcing specific, contained projects, companies are looking at IT development vendors as strategic partners - essentially, suppliers of IT business value. An example interaction, from the article:

A BT Wholesale executive team recently gave a presentation in Bangalore called "Joining The Dots," intended to help contract workers from Infosys, TCS, and Tech Mahindra understand BT's strategy and the role IT and process engineers play in it. The team explained BT Wholesale's market segments, key competitors, new products, and what it plans to launch over the next few years. "It's about spending time with people and explaining our objectives rather than just describing a piece of software they need to deliver by Friday," Selley says. "We want those employees to bring ideas back to us, too. It's been far too unidirectional between the U.S. and U.K. and suppliers in India."

The focus is no longer primarily on the cost savings obtained through offshoring, but rather on acquiring talent quickly, getting closer to customers - who are increasingly overseas as well - and creating global IT value chains that span people, processes & technology. Outsourcing companies (and in turn, their outsourcing vendors) are now valued partners in the software supply chain, and have a direct impact on the competitiveness of the enterprise.

This reminds me of the changes the automotive industry went through years ago.

Automotive Supply Chains

For example, authors Bill Jackson and Michael Pfitzmann compared different approaches to automotive supplier partnerships in their article Win-Win Sourcing in Strategy+Business magazine (via Evolving Excellence ):

This is one of many ways in which far-reaching manufacturers like Honda and Toyota rewrite the conventional rules of procurement.
...
With this approach, manufacturers and suppliers share a long-term commitment to improving each other’s capabilities, starting by working together to eliminate wasted effort and inefficiencies. The two sides, instead of being at odds, collaborate openly on lowering costs and raising overall performance, with the expectation that this mutuality will continue over many years, benefiting both companies.
...
Contrast this with the alternative ingrained in many companies’ purchasing departments: price-based sourcing. Essentially, this approach pits the interests of the supplier against those of the manufacturer. Each side reveals as little information as possible, for fear of giving the opposing side an edge.

Many others have made this point as well, such as the Economist's View blog: Why Toyota is Better than GM and Ford ; and MRO today: The supply chain is the difference maker .

This enlightened approach is already becoming an accepted view. Toyota sees its suppliers as partners; they, in turn, see their suppliers as partners; and so on. So the competition is not between the Toyota and General Motors; it is really a competition between Toyota's entire supply chain (Toyota, Tier 1 suppliers, Tier 2 suppliers, Tier 3 suppliers ...) and GM's supply chain (GM, GM Tier 1 suppliers, Tier 2 suppliers, Tier 3 suppliers, ...).

IT Supply Chains

Are there close parallels between these two approaches: BT Wholesale's interaction and shared strategy with its IT offshoring vendors - Infosys, TCS, and Tech Mahindra; and Honda and Toyota's closely cooperative relationship with their own vendors?

What are the implications of these changes for the IT outsourcing vendors, and for their Enterprises customers? If the parallel with the automotive industry holds, will we soon see the 1st tier IT software suppliers start to increasingly outsource pieces of their own projects, and so on, until there is a multi-tier supply chain in IT for software projects, business process optimization and quality enhancement? Projecting forward, how much further will offshore IT vendors go in providing value as a part of the business network of the larger enterprise?

---

Enterprise 2.0 and the Culture of Collaboration - Part II

This is Part II of a synopsis of the book The Culture of Collaboration by Evan Rosen. This post is written by guest author Sham Karandikar. Here's the link to Part I.

Effects

True global companies go well beyond marketing, selling and supporting their wares worldwide. They design, assemble and produce both products and services concurrently around the world. Today's Global Collaborative Enterprise (GCE) is a collection of inter-dependent companies that engage in the shared creation of value, often in real-time. The GCE exploits time zones and global work-sharing to design, assemble and create products and services 24 hours a day.

For example, Boeing has re-invented itself as a global collaborative enterprise which includes the supply chain, customers and even the customer's customers (i.e. airline passengers). Boeing's largest multi-media room in Everett - where employees can participate in video and web conferences and share 3-dimensional CAD/CAM software - is named the Global Collaboration Center. Moscow, for instance, is almost exactly on the other side of the world from Everett, with a time difference of 11 hours; so, when Everett is sleeping, Moscow works; and vice versa.

Types of Collaboration

Collaboration can be classified into three types, based on the level of interaction among the participants:

1. Low-level:  Involves exchanging information through threaded text discussions, video conferencing, web conferencing and document-sharing.
2. Mid-level: Involves a more elaborate collaboration tool set, that allows companies to share information with external organizations via the web. These organizations may be partners on some projects and competitors on others.
3. High-level:  Involves designing parts, plans, tools and processes concurrently among global partners.

High-level collaboration is truly the highest form of sharing. At Toyota, product designers, manufacturing specialists and business partners connect globally through its visual and virtual communications system, which reduces cycle time, enhances product quality and boosts production efficiency. Toyota now manufactures vehicles at over 50 sites in 26 countries and must bridge many regional cultures as engineers collaborate.

Boeing and Toyota are setting the standard in the aerospace and automobile industries respectively, utilizing the best talent regardless of location.

Conclusion

As the culture of collaboration permeates work and life styles, our habits are changing faster than we realize. While asynchronous interaction has its place, collaboration is increasingly rich, real-time and spontaneous. Increasingly, businesses are rejecting bureaucracy as wasteful and costly. In contrast to red tape and a serial approach, we create greater value when we engage one another directly, design products concurrently and collaborate across functional, business unit, corporate and regional boundaries. The current economic trend is to exploit the best talent at the best price, regardless of geography. The clear lesson from Toyota, Boeing, Dow Chemicals, Mayo Clinic, Proctor & Gamble and other organizations is that good decisions include the perspective of people throughout the organization at every level. The necessity of maximizing time, talent and tools in the global economy gives the culture of collaboration an edge. The quest to innovate and create greater value drives the desire to collaborate as organizations embrace the global collaborative Enterprise.

This point - that the global collaborative enterprise has to design, assemble and create products and services concurrently around the world - is explained very well in Evan Rosen's book. Following the lessons of the leading corporations described in the many examples in this book is essential in order to survive and thrive in the economy of the future; I highly recommend this book. Good Luck.

                  

---

Enterprise 2.0 and the Culture of Collaboration - Part I

This two-part article is a synopsis of the book The Culture of Collaboration by Evan Rosen. This post is written by guest author Sham Karandikar (my dad) - who has been active in the field of Information Technology longer than I've been living.

Introduction

The world is changing at an ever-increasing rate. New technology has changed our lifestyle completely.  It was not very long ago that the concept of Teleconferencing was beyond our imagination. But now you can chat, discuss and seek opinions from others all over the world as easily as you can from the person in the next office. At the same time, the world is getting so compressed that many corporations have become multinational - some of them operate in over 30 countries. In this situation, adopting a collaborative approach is imperative in order to get total involvement from employees located all over the world

This change is explained very well in the book, The Culture of Collaboration: Maximizing Time, Talent and Tools to Create Value in the Global Economy, by Evan Rosen. The book not only focuses on the theory of collaboration, but also provides lots of live examples to back it up.

The book explains how globalization has created unprecedented opportunities for maximizing resources. Realizing these opportunities however, requires a cultural shift. For example, the changing economics of the automobile industry required faster concept-to-delivery cycles; in response, BMW determined that tele-cooperation was the best way to achieve that goal. Thus, globalization is driving changes to business models, in order to maximize benefits by introducing the culture of collaboration. The name of the game in this changed world is real-time: shorten product development cycles, reduce response times, enhance interconnectivity and enable the free flow of ideas among global partners.

Examples

• The Mayo Clinic was founded on the principle of collaboration. The book spends some time exploring this use case. For example, the Mayo Clinic has already implemented a satellite-based video conferencing network; for each new case, the coordinating doctor immediately assembles a geographically distributed team to evaluate the problem. Because of its track record, and a culture that embraces collaboration, Mayo is experiencing soaring demand.
• Toyota is another company that has exported a collaborative culture: make decisions slowly by consensus. Toyota's culture provides a stark contrast to many organizational cultures that value internal competition; such as Enron, which hired and rewarded only the very best and brightest, and is now in bankruptcy. Internal competition delivers results in the short term but collaboration builds long term value. One outcome is that Toyota has a stable workforce, which is a significant competitive advantage.
• The Dow Chemical Company, which creates value through collaboration, was perhaps the first company of its size to conduct all of its video conferencing over a Converged IP network. Called Dow-Net, it introduced i-Rooms and digital white boards that let users in 43 countries connect via Polycom video conferencing equipment to any Dow site while collaborating on applications and data.
• Proctor & Gamble has embraced spontaneous interaction since the company first made instant messaging available to employees in 2000. P&G employees increasingly connect with each other in real-time so that ideas fly and team members concurrently make better, quicker decisions.
• The World Bank is comprised of 10,000 employees from 184 countries, who provide loans and share knowledge globally. To help developing nations deal with globalization, the World Bank  encourages collaboration among globally dispersed staff. Building trust is a critical success factor in effective collaboration. Drawing on their global knowledge, cross-cultural collaborators can spark synergies and create greater value.

Dissolving Barriers

Within organizations, barriers develop among departments and functions. Marketing develops its own culture but has difficulty in interacting with sales. Product Management and Engineering disagree over product direction and avoid interaction. There was a time when many organizations refused to give employees Internet access because of fears that people would surf the web instead of work - yet another manifestation of an archaic Command & Control approach. Most organizations now realize that blocking such access interferes with information flow and idea generation. Encouraging interaction among people with common interests reinforces cross-functional collaboration.

Taking the example of the Mayo Clinic, perhaps the most important role the SPARC unit plays is to break down the barriers among functions. Because of its methodology for innovation and a physical environment that encourages brainstorming, SPARC consistently succeeds in cross-functional collaboration. The convergence of voice, data and video over IP has forced people, who previously had little or no contact, to collaborate.

Another trend enabling collaboration and breaking down barriers is the implementation of common processes and systems throughout the organization. For example, the use of common processes and systems allows Boeing to easily move people from one aircraft program to another and to share tasks.

Thus the culture of collaboration dissolves the barriers of time and distance, delivers awesome results and creates value. Deriving these benefits, however, requires an understanding of both the potential and the limitations of the tools involved.

Continued in Part II.

                  

---

A Unifying Framework for Web 2.0

What, exactly, is Web 2.0? How does it apply within the Enterprise and to the web at large? What technologies are involved and what are the benefits?

Ross Dawson, of Future Exploration Network tackles these questions in his recently launched Web 2.0 Framework, an excellent work which describes the many aspects of Web 2.0, both within the Enterprise and on the web at large. This unifying framework is remarkable because it brings together the definitions, characteristics, mechanisms, major players and outcomes of Web 2.0, in a simple and intuitive way.

Ross describes the objective as follows:

The intention of the Web 2.0 Framework is to provide a clear, concise view of the nature of Web 2.0, particularly for senior executives or other non-technical people who are trying to grasp the scope of Web 2.0, and the implications and opportunities for their organizations.

I see this as a critical area: in order to effectively sell the ideas of Enterprise 2.0 to the CxO, we have to be able to explain the mechanisms (the "how") in simple terms and focus on the expected outcomes in terms of helping the organization meet its goals ("the why"). Web 2.0 is made up of many different parts. Like the story of the Blind Men and the Elephant, when we focus on a variety of specific perspectives, we experience it in different ways; the framework does a great job of taking these different aspects and putting it all together.

 

 

One area ripe for discussion, where my thinking diverges somewhat from Ross's framework, is in the Emergent Outcomes section. I see three core outcomes from an implementation of Web 2.0 technologies:

• Personalization
• Collective Intelligence
• Semantic appreciation of content

In many ways, I see the other items - finding relevant content, enhanced usability, visibility of interesting content and even community - as inherent in, or following from, the three core outcomes described above.

In any case, this framework certainly moves things forward in the collective wisdom about Web 2.0; I look forward to continuing this discussion further.

---

Beyond PageRank: Using the Wisdom of Crowds in Enterprise Search

The classic problem in Internet Search is: how to display the most relevant results in response to a search query. One would think this problem would be relatively simple within the constraints of an Enterprise Search, since these searches are based on a much smaller overall scope of content and there is little incentive to game the system.

But this is not true, for two reasons. First, as the social principles of Enterprise 2.0 take off, the sheer amount of content is exploding, even within an organization. Second, the concept that relevance is relative is not limited only to internet search engines; this problem carries over to Enterprise Search - what's relevant to one user may be completely useless to another user.

This issue of relative relevance is one that static solutions, like the PageRank algorithm used by the Google Search Appliance, fail to take into account. Just because a given piece of content is wildly popular does not make it relevant for all users. [For a more general example, try searching for the term "apple" in the Google internet search engine - all of the hits on the first page are related to Apple, the computer company; how about apple the fruit?]. Arguably, personalization could help bring more relevant results, but that presents its own privacy-related problems.

How then does one improve the relevance of search results? One method is to use the Wisdom of Crowds: what do other users like, especially the ones whose tastes are similar to mine? Where do they spend their time? What content links do they click on?

I recently had the chance to review a solution in this space: Baynote, a 2-year-old startup based in Cupertino, CA, that addresses this exact problem.

Baynote's approach

How does the Baynote solution work? Essentially, the idea is to find a tag-based "user fingerprint" to associate the user with a like-minded peer group; by creating micro-segments of the community, you can then use WoC to make recommendations to the user. Of course, every user is different, but over a large population, the noise tends to cancel out and the key signal - the core set of interests of the group - is preserved.

Baynote's Affinity Engine tracks about 20 different heuristics of user behavior, such as mouse movement, time spent on the page, number of repeat visits, and so on. By observing this behavior, it promotes the emergence of collaborative knowledge, surfacing information about key content, user likes and dislikes, micro-segments of the user population, and finally, recommendations. Used for Enterprise search, the engine is designed to boost the relevance of search results over time, based on user behavior. Baynote also provides solutions in the E-commerce space (where good recommendations can increase the conversion rate) and for Help systems (where navigation to relevant popular results can improve customer satisfaction).

I saw a demo of Baynote-powered search results. Compared to the "raw results" obtained without using Baynote, the increase in relevance was indeed impressive. You can try out a live example for yourself, by doing a search within the Interwoven web site where Baynote is implemented. A sample screen shot is shown below.

Some of the highlights and concerns with this solution, from my perspective, are noted below.

Highlights:

• Tagging:  Automatically builds a virtual folksonomy that grows with time
• Classification:  Assigns users to peer groups; this is an implicit classification based upon user behavior
• Seasonality:  The algorithm is based on the concept of UseRank rather than PageRank; since the UseRank rises and falls depending upon user interaction, relevance automatically adjusts to the fading of importance for a given piece of content
• Independence:  Users have no direct influence on each other
• Hard to game:  Since interactions are measured implicitly, it's difficult for a single user to game the results; at the same time, no specific action is required on the part of the user
• Rich content: Since ranking is implemented through the use of tags, it works equally well with binary content, such as images, audio and video
• Easy implementation:  According to Baynote, no server-side integration is required; implementation involves the addition of an "observer" script added to the template of the web site, similar to a web analytics tracking bug.

Concerns

• Positive feedback:  Once a piece of content becomes popular, it gets increasingly displayed to users, which is likely to further increase its popularity. Since display priority is based on a combination of relevance and popularity, there is a danger that less-relevant results will be ranked higher, simply because they are more popular. [Although, with time, their popularity should automatically adjust lower for that particular micro-community.]
• Edge cases:  How does one look for obscure information that has low popularity? It's possible that obscure content could be completely overshadowed by highly popular content with similar tags.
• New content:  This is directly related to the positive feedback problem mentioned above. How does new content get traction initially? Jack Jia, CEO of Baynote, pointed out that the engine supports a feature to address exactly this problem: a "merchandising" feature that hard-codes a piece of content to specific query results, such that it cannot be overridden by organic results. This allows the content to gain initial popularity.

Conclusion

Baynote is one of the more interesting Search engine implementations I have come across. The addition of WoC knowledge, gained by observing implicit user behavior, is certain to improve the relevance of search results and help web site managers discover (and act upon) non-intuitive connections between content areas. The case is even more compelling for E-commerce sites, where the display of related and popular products could have a direct impact on the bottom line.

---