The Software Abstractions Blog

Crowdsourcing Charity: Sustainable Innovations

Paul Buchheit (founder of Gmail and Friendfeed) wants to donate money to charity. He's trying an interesting experiment in collaborative charity, where he's planning to crowdsource the list of ideas he should consider funding. The ideas he gets are only recommendations, he says, and he will ultimately choose the best idea that appeals to him and that makes sense.

In that spirit, let me make a suggestion for Paul and others like him in Silicon Valley, one that takes a different perspective. If you're a successful tech founder or VC who likes philanthropy, consider this a free "business idea for giving back" that you can adopt as your own if it interests you. [Now I'm no VC, nor am I a rich, successful tech founder, so I have no idea if the approach below has any appeal to these folks; nevertheless it is an interesting thought exercise.]

Silicon Valley skills

In general, anyone with a bit of time or money and the inclination to help, can donate to charity, and many people do (bless 'em!). The most effective donations, however, leverage the unique skills and perspectives that one can bring as an individual, using one's own talents and gifts in addition to money, to help others.

So what could Silicon Valley entrepreneurs do? How about this: instead of donating money directly, why not use it as seed funding to build an organization that can keep on giving? After all that is what tech founders (and VCs) do best: they fund startups that take some initial amount of money and then leverage it massively to create a disproportionate impact on a target market - or die trying. Why not take the same approach to charity? Build an organization that leverages other people's time and money, and creates massive impact, much bigger than the pure monetary donation would create.

But what type of organization? There are already many, many charities, of many different types. How do you make a difference?

Third level of Abstraction

Give a man a fish, it is said, and you feed him for a day; teach him to catch fish, and you feed him for a lifetime. Extending the metaphor, what if you could create a construct that continuously spawns ideas, technologies and approaches for teaching everyone to catch fish?

What the world needs more than direct charity, is locally-sustainable ideas that allow populations in underdeveloped areas to take care of their own core needs - clean water, sustainable food production, cheap education and economic development - by themselves. As a philanthropist, what if you were to establish an organization to crowdsource innovations and ideas to bring about these very changes? You could create a mechanism to encourage such attempts and to reward successes.

The idea: So the idea I'm proposing, in a nutshell, is this: instead of making a direct monetary donation to charity, set up a recurring contest or start an organization that encourages and rewards innovations in technology, design, reuse, financial models, human factors or ecology conservation, for the social good. Most important, these innovations must lead to local, self-sustainable solutions that do not depend on ongoing external help to keep going.

Real-Life Examples: Business Models and Innovations

There are many examples of business models for encouraging innovation through crowd-sourcing:

• Competitions, such as the Netflix prize for improving their recommendation algorithm (which seems to be beaten) or NYC's design competition for post-disaster provisional housing
• Paul Graham's Y-Combinator micro-funding model for seeding startups
• Social incubators like socialedge

I'm sure there are many others that are not listed here, feel free to list them in the comments if you know of any.

What kind of innovations should such an organization look for? Ideally, these would be locally-managed, self-sustainable or reuse-based product design ideas, original approaches or creative financial models.

Here are some examples of innovations of this type:

• Clean water powered by playgrounds: Playpumps International
• SHRIMP Refugee Housing, which folds up so small that four of these can fit into a standard shipping container, and each one provides shelter and fresh water for a family of 4. [A slide show from Wired Magazine has a whole list of innovative shelter designs for providing emergency housing to disaster areas and refugees.]
• Buy a pair of chickens for AUS $10 and help a family get started in sustainable chicken farming. [via Darren Rowse of Problogger]
• Lighting up remote villages in India, using technology and innovative financial models: SELCO [via Sramana Mitra]

The world needs many more innovations of this type. Why not explicitly encourage them?

---

Marketing 2.0: The Zen of Email Marketing

A few days ago, a leading software company sent me one of their regular marketing emails. This missive is full of information about them: news about their latest triumphs, a message from the company, new product features, support information and so on, interspersed with a few special discounts and items for sale. After a quick glance to look for deals that interested me, I deleted the email and moved on. As I surveyed my overflowing email Inbox, I briefly wondered whether to unsubscribe from their mailing list; perhaps next time.

This got me thinking. Like most other Americans, I now suffer from "E-newsletter fatigue". Email marketing has come a long way since Marketing guru Seth Godin first popularized it circa 1999 in a landmark book, Permission Marketing , and is far from dead, even in the face of competing new technologies like blogs and RSS feeds. Today, the MarCom department of practically every reasonable-sized business spends a significant amount of effort churning out these electronic messages, most of which are virtually indistinguishable from those of their competitors.

Perhaps the time has come to take Email Marketing to the next level!

Engaging the Reader

Ideally, a multi-email campaign is like a good blog; each communication must engage the reader. There is no reason the reader has to read your email, any more than there's a reason that the reader must read a blog post; you must attract readers with interesting and valuable content.

A good example of engaging users, albeit in the offline world, is the Trader Joe's newsletter that shows up regularly in mailboxes around the country. Although hardly cutting-edge technology, the printed version is quite entertaining - it's full of fun facts, bits of history, useful recipes and nutrition information, and of course, references to Trader Joe's offerings. The online version is authored in the same vein: the example below highlights a single product in a fun and interesting way, and provides some additional tidbits that change the whole tone far away from "hard sell".

...

Another example is the Hitwise Intelligence blog, which comments on interesting meta-trends observed in Hitwise data - such as whether the McCain web site appeals to Independents, or if Heely's are making a comeback. The media, industry observers and customers watch these reports closely and then immediately start discussing them, in public forums, conferences, blogs and around the water cooler.

Of course, trend data is Hitwise's business. But anyone can do the same thing. You can highlight information and trends of customer interest, from data that you already have. What are the latest trends in Spring fashion? Is the rising price of gas going to affect the cost of groceries? If you're a grocery chain, how do you plan to react and help keep prices down?

Or you can provide useful information that, even if it doesn't directly sell your product, helps your customers and users. But it has to be specialized information, that only you can provide; for example, informing readers that daylight-savings time starts this weekend is useful, but not particularly compelling.

If your Marketing emails are full of the cool new features you've recently implemented in your product or awards won by your company, then that's only mildly interesting to readers. For most products, only a small fraction of customers care about all the bells and whistles anyway. Let's take Microsoft Office - assuming that you use it, do you know all its features? Do you even care? What about the "advanced" features of your TV?

Most progressive companies have recognized the significance of blog marketing, which sets a new, higher standard for engaging customers in a two-way conversation. This new standard applies equally well to Email Marketing - only the transport mechanism is different. Similar to a blog, most readers will tolerate some level of self-promotion (yes, "special discounts" count as self-promotion) and even some off-topic content; but overall, the signal-to-noise ratio must be very high from the perspective of the reader, not the email sender.

The reality is that most readers are perusing the email thinking - How does this help me? What can I get out of this?

Compelling Content

So imagine the mother of three, logging in for ten minutes early in the morning, before the kids wake up; the hotshot lawyer checking her emails between appointments; the harried Executive at his son's baseball game; the teenager checking his mobile phone on-the-go; or whatever your target demographic is. In the 5 seconds that it takes the reader to scan your email: is your content compelling enough to make them want to remember it, to go back and learn more when they do have time, when they get back to the office or at home?

To take it one step further, is it an email that they're eagerly waiting for? Why not?

---

Founders' Quotes from Launch:Silicon Valley 2008

Launch:Silicon Valley 2008 was an amazing event today at the Microsoft campus in Mountain View. Co-presented by SVASE, Garage Technology Ventures and Microsoft, it brought together a large number of exciting startups and top-tier VCs; each startup presenter was given 10-minutes to make a pitch.

One dimension each founder or CEO had to address during the pitch: what would make their particular idea marketable or defensible? Although the domains and business plans of the various startups varied widely, there was a great deal of similarity at a general level among the responses to this question.

Here are some pearls of wisdom from the founders' responses; this list seems to be representative of this time and place in Internet history:

• Make it very simple and easy to use
• Viral propagation! Usage of the product automatically encourages its propagation to new users
• Consistent with the current mega-trends: Mobile, Voice and Communication.  (Although surprisingly, almost no one mentioned the iPhone!)
• Target a specific audience and feed them technology; rejuvenate an old and tired industry
• Implement an initial private beta for a small set of users, and learn from them
• Create a site or company with personality!
• It's great to have a two-sided business model and multiple potential streams of income
• Find the user pain point to address
• Create a horizontal platform and then build vertical applications on top
• Go where the users are (web site widgets, customizable home pages, social networks, cell phones); the last thing users want is another place on the web to go to, another web site to check

---

Responding to Disruptive Innovation: Threat or Opportunity?

Scott Karp of Publishing 2.0 recently wrote an article about newspapers and the web in general, and about the Washington Post in particular: What Newspapers Still Don't Understand About the Web. Karp feels that the Post doesn't really get the Web - the web site is essentially a reflection of the Print newspaper, rather than leveraging the unique possibilities offered online.

The New York Times, on the other hand, definitely gets it, and is only moving further ahead in  leveraging the web. For example, in technology, the Bits blog regularly shows up on Techmeme, and the NYT is already on twitter.

How does this happen? How can two direct competitors in the same field react so differently to the same change in their industry?

According to Clayton Christensen and Michael Raynor, a large part of the answer lies in the way companies react to a disruptive innovation in their field. In their latest book,     The Innovator's Solution, a sequel to the highly popular book detailing the Innovator's Dilemma, they handle exactly the issue described by Karp's post, in a more general way.

What Christensen and Raynor found is that the way a successful incumbent company responds to a disruptive change in their domain depends on the part of the organization that is tasked with responding to that change.

There are two primary ways in which the company can respond: the disruptive change can be viewed as a threat, so that the company pumps resources into trying to harness the new process, technology or idea as an evolutionary improvement to its existing business model; or the disruptive change can be viewed as an opportunity, in which case the company tries to find the appropriate market or business model that naturally fits the new paradigm.

Obviously, if the existing operational part of the organization - one that is already very successful - tries to respond to the change, the main interest is in preserving the status quo, and the big change is seen as a threat to be neutralized; if instead, the company forms a small, semi-autonomous entity tasked with responding to the change, unhampered by the existing business model, then the big change is likely to be seen as an opportunity to be exploited.

There are many other fields where this type of change is taking place. One example is the iPhone, which is clearly proving to be a major disruptive innovation for handheld computers, causing existing market leaders such as Palm, Microsoft and RIM to scramble.

Are there others? How are they responding to the disruption? Leave a comment below and let us know!

---

Does Web Advertising Work?

Do Web Ads work? Does the vaunted Freemium model actually produce meaningful revenues for anyone but the largest services and content sites? How should Web 2.0 be monetized?

Many Perspectives

There has been a lot of discussion about these and related questions in the past few days among the Techmeme crowd. Here is a cross-section of perspectives from some of the thought leaders in this discussion:

• Unsurprisingly, this discussion was kicked off by Tim O'Reilly, the same ultra-smart observer, blogger and publisher, who originally coined (or at least, evangelized) the term "Web 2.0".
• The Financial Times ran an article provocatively titled: Web 2.0 fails to produce cash , in which the authors question whether popular Web 2.0 companies have actually produced any revenue, and predict a big shake-out in the next year or two.
• The always-interesting Robert Scoble led a widespread discussion, referring to posts by Om Malik and Dare Obsanjo, on whether Twitter (that highly popular micro-blogging service, and train-crash-currently-under-way) should be charging super-users like him, while continuing to keep the core service free for most casual users
• Alexander van Elsas has a wonderful post, referring to an earlier post by Scott Karp of Publishing 2.0, explaining how advertising does not usually provide the user with any value; van Elsas feels that the real value should be created outside the network.
• Coming up on May 30th is a discussion on this topic at TiECON East: "Web 2.0: Viable Business Model or Bursting Bubble?". The discussion includes two prolific and highly popular bloggers among its participants, both of whom have (inevitably) blogged about it - Fred Wilson (Discussing Web 2.0 ) and Don Dodge (Web 2.0 = Bubble 2.0? ).

When Ads Attack!

Clearly, web ads do work under certain circumstances. The poster child, of course, is Google - in addition to ads on its own search results page, Google commands revenues from millions of long-tail publishers via AdSense. Although most of the small publishers in this group actually make very little money, the total revenue in the aggregate for Google is very high.

Web Ads work better in certain situations than in others. At a VLAB event last week, Bill Reichert, Managing Director of Garage Technology Ventures, summed up perfectly the virtuous cycle of content and advertising:
    "If the paid content enhances the product, as in the case of Google, then that attracts more users to the site, which in turn enables more income, which allows the site to further enhance its content. Thus, the ad content should fit the users' actions to be effective; otherwise, as is the case with Facebook, the ads just get in the way."

Monetization Options

Once a web site or service gains traction and gets a sizeable following, there are potentially many creative models for monetization. In a much older post, Fred Wilson wrote about the The Long Tail Of Business Models , working off an initial post on Media Business Models by Chris Anderson (and I've added my take ).

Web 2.0 monetization works especially well in specific situations, and it's likely that totally new models will emerge in the future. One such model is advertisers who create meaningful content for web users, a by-product of which is to drive sales; this type of content can include informative articles, useful services or immersive experiences. Examples of this type of model are:

• Pharmaceutical companies that educate doctors on how to utilize new types of medical devices
• Lawyers who explain the law or provide status updates on government activities
• Home improvement stores that sponsor how-to seminars and then provide discounts on materials

Clearly, the web is a different medium, and simply transposing print ads to the web will not work; the big winners of the future will be those companies that can take some of these radical new models for monetization and make those work for their own use cases.

---

Comparing Search Relevance: The Relona Project

Recently I worked with Charles Knight of the Alt Search Engines blog on a project to compare the search relevance of two major search engines - Google and Yahoo! - and to measure the improvement in relevance by adding a search engine technology called Relona into the mix.

Background

Relona Research Labs is a unique technology targeted at enhancing the relevance of search results by suggesting changes to the search query spec - such as adding keywords, enclosing phrases within double quotes, or even reducing the number of search terms. In future, this type of technology could be plugged in into the search engine back-end to automatically refine the user query and produce better search results.

Procedure

We started with a large number of actual user queries from the AOL open data set, provided to us by Relona. Out of these, we picked a substantial set of queries at random as our test search strings.

For each selected search string, we ran two tests:
    (a) comparing the search results from a raw Google search and a raw Yahoo! search to score the more relevant of the two, and
    (b) comparing the search results from a raw Google search with the results of a Yahoo! search for Relona-optimized queries, again to find the more relevant of the two
     [Deciding which results were the "more relevant" was done manually by eye-balling the results]

The idea was to evaluate whether the addition of Relona optimization for user queries made the Yahoo! results more relevant, relative to Google results.

Observations

Personally, the biggest surprise I found was in the comparison of raw Google and Yahoo! search results. I had always imagined these two leading search engines to be comparable in terms of search relevance. But that wasn't the case; while the results were comparable in about a third of the cases, Google's results were significantly better in about half the cases - the Yahoo! results were better in the remaining cases. Often, 8 or 9 out of the top 10 results from Google would be highly relevant, as compared to 4 or 5 from Yahoo!.

Using Relona's beta solution to improve the search results - often by constraining the query or adding keywords - did appear to enhance the relevance of Yahoo!'s results and help bridge the gap. (For access to the raw data, please send me an email and we can share it with you.)

One limitation of the existing Relona search is that the engine comes up with several suggested improvements to the query, only some of which enhance relevance of results. But in order to be usable in the real world, the technology needs to come up with one "best" query suggestion. Kumar Ramanathan, co-founder of Relona, assures me that they are currently working on improving their solution along these lines.

Conclusions

Based on this limited test (more comprehensive testing is needed), Relona clearly helps in improving the relevance of search results!

Also, based on this constrained data set, Google's dominance as the premier search destination seems well-justified; matching or beating their formidable relevance in Web search is going to be increasingly difficult in the future (especially since web sites may themselves be changing to suit Google's algorithms as well ).

At the same time, this project convinced me that a more systematic approach to measuring search relevance is sorely needed; all the search engines I've talked to so far seem to measure their own relevance using large data sets, manual measurement ("eye-balling") and statistical results. This is certainly one way to measure relevance, but it ignores other dimensions of search relevance, such as breadth, depth and so on.

---

Yahoo! SearchMonkey - Released to Developers

The good folks from Yahoo! unveiled their new open search platform Yahoo! SearchMonkey, at a developer launch party today at their Sunnyvale headquarters. In some ways, the SearchMonkey platform is revolutionary and a major step forward in search, allowing publishers to participate directly in improving the quality of their own information presented on the Yahoo! search results page (this is also implicitly a push for the bottom-up approach to the Semantic Web, which most industry observers have given up on in favor of a top-down approach). The platform also lets publishers and third-party developers build applications aimed at improving the search experience. Finally, and most important, if enough publishers and app developers participate in the program, it promises to improve the quality of search results for end users.

Features

At the simplest level, you can think of SearchMonkey as a community-powered set of rich information boxes (similar to the Google OneBox) that appear on the Yahoo! search results page. Publishers can provide this rich data to the Yahoo! search index in a variety of ways: through structured data feeds (RSS), through RDF or Microformat markup on web pages, or through simple page extraction. The "Information Bar" shows up underneath the main search results. The Yahoo! search team has also provided tools to enable developers to build search-based applications very simply and easily.

Powerset Launches Wikipedia Search

Semantic search engine Powerset, which we've written about here before, has just launched its initial release. The current release is limited to indexing Wikipedia content, but it provides a great showcase for their technology and user experience.

For example, my search for "Alexander the Great" provided the following results page:

Cooperation of Alternate Search Engines: A Manifesto

( This post is inspired by my discussions with my friend, Charles Knight of AltSearchEngines )

Background

I'll be at the Alternative Search Engines Day tomorrow, a unique event in San Francisco put together by Charles and the AltSearchEngines team. The event is sponsored by SeeqPod, UpTake, Matchpoint, HealthPricer, GoPubMed and Blogdimension. (Unfortunately, it's not open to the general public.) If you're part of an Alternative Search Engine, I hope to see you there!

As I was getting ready for the event, it got me thinking about ASEs and how they can work together.

The Case for the Alts

I love the ASEs - Alts rock! Without them, there would be little innovation in Search, no new frontiers to be explored.

The Alts are the ones that keep pushing the envelope with new directions in search technology, whether it's algorithms, user interface, social search or something else.  Although Google has some fine technology and is synonymous with search, I firmly believe that we're still at Search 1.0, and have a long way to go. Because of all this competition from the Alts, and the resulting innovation, web search continues to improve.

WebGuild Web 2.0 Conference: Issues and Challenges for Crowdsourcing

I attended a really interesting session at the WebGuild Web 2.0 Conference and Expo last week: The Power of Crowdsourcing - moderated by Jeremiah Owyang of Forrester Research.

Participants on the panel were:

• Aaron Strout, Vice President, New Media, Mzinga
• Michael Sikorsky, CEO, Cambrian House
• Anil Rathi, CEO, Idea Crossing

This was one of the best panel sessions I attended at the conference, part of the reason being the bang-up job Owyang did as moderator. He took a very active role, bringing up provocative questions, directing those at specific members of the panel and not being shy about treading into the concerns and difficulties of using crowdsourcing and social media - this prevented the session from degenerating into a "Rah, Rah, Crowdsourcing is all good!" type of discussion.

The other reason was that the panel members were all knowledgeable, articulate and open in their remarks; the conversation never flagged, as it did with some of the other sessions I attended.

To kick off the session, Owyang put up a few slides entitled Social Technographics that were intriguing, but more on that in a future post.

One of the panelists, Michael Sikorsky of Cambrian House, listed the three legs of crowdsourcing as follows:

• Wisdom, which can be explicit (e.g. voting in American Idol) or implicit (e.g. links used for calculating PageRank)
• Participation, such as item submissions or code check-ins
• Funding, such as a prize or project funding

The Challenges

Based on the discussion at this session, I've compiled the following list of challenges in implementing crowdsourcing solutions and ways of addressing them.

• Wisdom of Crowds: How do you keep the input quality high?

For any crowdsourcing activity, the first step is to pick the right crowd! Equally important, you must ask the right question.

The next step is to use statistical methods to prioritize high-quality input. Finally, a self-policing community (possibly, with some moderation) can help weed out low-quality input and spam.

• Is there an inverse relationship between those who have the time to contribute, and the quality of the ideas presented?

One caveat to keep in mind is that the vocal minority may not be representative of the majority of users. But this type of forum may act as a funnel for identifying talented people who have not yet been discovered.

By providing rewards or incentives consistent with the value of the ideas being submitted, you can get greater participation from qualified users and a higher level of confidence in the quality of the ideas being submitted. Another alternative is to use some type of game mechanism; games have built-in rewards that encourage participation.

• What if some people don't want to be outsourced?

Tara Hunt, of Citizen Agency, recently wrote a blog post titled: Please Stop Crowdsourcing Me , questioning whether crowdsourcing is a good idea. She has a point - some users may not want to contribute or be involved in a crowdsourcing exercise, especially to benefit a large corporation.

The panelists agreed, and pointed out that you should carefully consider which tasks should be outsourced in this way - for example, product users love to help each other out with solving problems and difficulties, but if participants get the feeling that the company is simply using them to reduce customer service costs, then they will stop being helpful.

Any crowdsourcing program has to be thought through and managed carefully; you don't want to risk users having a bad experience.

• How do you manage and lead a crowd, to create a positive experience?

For the community to be truly engaged, it is extremely important for the company to be very transparent.

One key point to think about, especially for large companies, is that you have to be careful about what you share with the crowd. On the one hand, the more you share, the better the ideas you will get; on the other hand, you risk letting out corporate proprietary secrets.

Finally, some activities simply may not be amenable to crowdsourcing.

• How much control do you want to retain? Do you need a Product Manager as an expert?

A community of users can generate a lot of great ideas, but those don't all necessarily fit together; having an expert in place as a product manager can provide guard rails to keep things on track. The product manager can bring a single, unified vision and - this is critical - can communicate back to the community why a particular idea is not being used.

It's important to find a balance: the community generates the ideas, but the company or organization picks the ones to be used, refines them and implements them. Even the nuggets of ideas can be leveraged to create lots of value.

 

Successful Examples

The panelists also offered examples of actual crowdsourcing implementations:

• The Longitude Prize - one of the earliest examples, was a reward offered by the British government through an Act of Parliament in 1714 for a simple and practical method for the precise determination of a ship's longitude.
• Procter & Gamble has raised the level of outside design and significantly increased the success of product-related improvements.
• Intelpedia from Intel, is an example of crowdsourcing in the Enterprise space. The idea is to look internally for ideas, share best practices and preserve common knowledge. According to reports, Intelpedia has up to 20K pages already.
• Ace Hardware created a community for 300 dealers, whose ROI was measured at 500%; as a result, the community was rolled out to all 5000 of its dealers.
• The Hopelab Foundation created a global competition for kids of all ages and received submissions from 429 teams.
• American Idol has produced highly successful artists, some of whom have sold over 10M CDs; even the worst idol winner has sold 500K CDs.
• Innocentive is a well-known example where companies post complex problems and offer rewards for their solutions.

[Update :  An alert reader pointed out that the P&G name should be spelled with an "&" - this is now fixed. Thanks, John!]

---