Publishing Matters
What's on your mind?
 Tuesday, June 23, 2009
Elisabeth Sifton, in a long article that merits reading (“The Long Goodbye?" May 20, 2009), writing in the Nation with many years of experience in book publishing behind her, does a superb job of synthesizing and providing a chronology for the history of book publishing as a literary enterprise in the past century. She has shown how the manuscript selection and vetting process worked competitively and creatively through numerous independent publishers and booksellers, and she traces its decline along with the consolidation of publishing in the hands of major conglomerates. She writes with the feeling and caring of one who loves books and has enjoyed a long career giving them life.

She then brings us to the present when she writes:
“as we know, all retail businesses collapsed in September, failed to recover during the Christmas season and have been weak ever since. Book sales continued to drop in the spring, but then, they've been stagnant for years. It was in 2001, when the dot-com bubble was beginning to burst but before the shock of 9/11, that I first heard a morose sales director use the catch-phrase 'flat is the new up.' Book publishers and sellers were overextended and had grown careless, like everyone else, in the go-go years, while the digital reading revolution continued and business worsened. In the past six months, layoffs and shutterings have become commonplace.”

She concludes that “It is a confused, confusing and very fluid situation, and no one can predict how books and readers will survive. Changed reading habits have already transformed and diminished them both. I, for one, don't trust the book trade to see us through this. Wariness is in order.”

I have no quarrel with the facts cited in Sifton’s industry review. I do find myself regretting the bleakness of her conclusions.

It is true that the gradual acquisition and absorption of the independently founded publishing houses and booksellers is a sad story of literary loss, but it also echoes the growth of mass market economies in the 20th century. These economies brought other benefits to many people as a tradeoff, providing mobility, abundance and access to diversity, lifting them out of poverty and disease. Was it worth it?

The article implicitly discounts the capacity of humankind—and in particular our own society—to become the master of its technologies and to bring the qualities of thought, art and expression that had been the historic function of book publishing to new media and new modes of communication.

Our experience with ForeWord provides a window on the shrinking of economically viable mechanisms to support the functions we perform. At the same time we are also a window on all of the creative energy relentlessly seeking and finding outlets for its expression. And the empowerment vast numbers of people now enjoy.

Sifton also writes:
“the entire world of American retail business is veering toward obsolescence. Must books now find their way in cyberspace?  This prospect is even more alarming than the crisis threatening brick-and-mortar stores, for the World Wide Web is an ocean with few buoys to mark navigable channels of meaning. The channels we navigate on it are mercantile channels, designed to be lucrative—but not for us”

This won’t last forever, assuming that is the case to begin with. Amazon, Google, Apple, et al, will hold the lock on the distribution of books and content they built until alternatives come about as surely as they also had built on the advances that preceded them.

And as for entrepreneurs cashing in on intellectual and creative property, that has always been the trade off for authors and artists. Short of command and other forms of directed economies, or the unpredictable outcome of viral word of mouth—gatekeepers providing guidance (reviews) and access (distribution) will always be thepartners which any but the most enterprising author or artist will need in order to reach their audience—and without which their audience—all the rest of us—will not  be able to efficiently find the books worth spending time with.

At the same time the most confounding and exciting alternative to these gateways is the "wisdom of crowds" phenomenon—social networking—useful not only in spreading the word but in taking down imposters and empowering revolutions. How far can this go before becoming a danger in itself? How will the much needed gatekeeping functions emerge again? How will they be paid for?

People of taste and discernment will find a way—but it will have to emerge from and be congenial to the media of the times. It will be less likely to happen in the face of pessimism or despair.

Meantime, where is it written that enterprising risk-takers won’t take advantage of opportunities and run with them?

All is not lost.


Posted by: Eugene G. Schwartz, Editor at Large


posted on Tuesday, June 23, 2009 11:28:14 AM (Eastern Daylight Time, UTC-04:00)  #    Comments [0]
 Friday, September 12, 2008
“Generation We,” is not your typical independent publishing story. Yet it is emblematic of how independent publishing can provide a platform that will reach an instant audience in the narrowest or widest range.

It was just a week ago that industry futurist and consultant Mike Shatzkin broadcast a posting to his friends and colleagues announcing this remarkable forthcoming publishing achievement. It was only a month earlier that he was engaged by Greenberg to bring together the ingredients requisite to publication: a production supervisor (Brian O’Leary of Magellan Media) a printer (Quebecor), a publicist (Max Pulsinelli of Maximum Impact) and, with the help of Rich Freese, former CEO of PGW, .a distributor (BookMasters/Atlas in Ashland, Ohio).

Nor is it that Greenberg is your naive innocent operating on a shoestring. He already knew that going with a conventional publisher would mean giving up the production and marketing controls that were so important to his reasons for doing the book in the first place. Having poured over a million dollars into the research that led to the book, what he didn’t anticipate were the logistical intricacies that could impede a quick and effective launch—especially one that wants to be in the market before election day. Hence, Shatzkin.

Bringing the book out now was important because Greenberg chose independent publishing  as a platform from which he expects to accomplish nothing less than the mobilization of a new generation of 95 million Millennials in the cause of the social and political transformation of American politics. Catching the crest of the wave of election campaign interest will help his message become part of the national conversation.

The Personal Odyssey

How did this come about? After all, according to his Barnes and Noble.com bio, “Greenberg has founded and established many businesses in his entrepreneurial career including wind farms in partnership with Native American tribes in the Great Plains; Acumen Sciences and the Acumen Journal of Life Sciences; Scient, a consulting firm focused on eBusiness and emerging technology; and Viant, an internet systems integrator. An award-winning philanthropist, he was named by Worth Magazine as one of the 10 Most Generous Americans Under 45”

Greenberg writes in the book’s introduction, “Through hard work, applied intelligence, and good timing, I was able to prosper. By the turn of the century, I was a paper billionaire at 35 years of age...Although I was lucky enough to have  a stable relationship with my wife, everything else around me devolved into a pit of misfortune, conflict, and poor health. 

“I was miserable. By 2004, I weighed 275 pounds, was dependent on prescription medication, depressed, and sometimes selfish and thoughtless. The world was giving me a valuable lesson: Life is not about things and what you do for yourself.  I was imploding from my ambition-driven ego. My life was unsustainable...I closed my business and stopped working.”  

So it came to pass that an Emeryvlle, CA based business man, had his personal epiphany while in the Amazon jungle in 2006 during a two year process of self-renewal and rediscovery.

He returned determined to do something about the state of crisis he felt was undermining the foundations of American society. This story of personal transformation is a back story interesting and inspiring in its own right.

In addition to launching his Millennial generation project, Greenberg has applied his talents to a socially responsible enterprise as President and Chief Executive Officer of Beautifull, Inc. (Beautifull.com), “a prepared, fresh food company focused on providing tasty, healthy, and real food for retail and home delivery.”

As I listened to Greenberg in our interview and as I later read more of the book, I realized that Greenberg was looking for the “big idea”—a line of attack that would break out of the mold of the conventional with the prospect of leading to serious change. He wanted to find the fertile soil for new ideas, seed it, and enable a new crop of actors to grow and take over.

Once a Reagan Republican (for which he doesn’t apologize in the book), but now in a new place for social and political transformation, Greenberg remains an idealist, but is not a utopian. He is not advocating the overthrow of the system. “If we ever are going to fix the problems we have today we have to do it with political action,” he says, and it needs to be done within the system and within the two parties if possible.

He writes, “This book is for our future.  The most powerful force that can make our future better than our past is the youth binding together on the outcome, resolve, and political will to achieve it, no matter how they may differ on details of implementation. I’m not a member of Generation We, and I don’t aspire to lead it.  My great hope in writing this book is that it will inspire a handful of  great leaders like Dr. Martin Luther King, Jr., or Mahatma Gandhi to emerge and lead their peers.”

At its heart, the purpose of this effort, he told me is “to ignite a passion for the greater good.”

The name of his publishing house, Pachatusan, by the way, is taken from a holy mountain in Peru, which can mean, “he who sustains the world.”

Posted by: Eugene Schwartz, Editor-at-Large

posted on Friday, September 12, 2008 8:49:29 AM (Eastern Daylight Time, UTC-04:00)  #    Comments [8]
 Thursday, May 08, 2008
To know is to have power. Or, in a more muscular way, knowledge is power.

This old maxim, that I first learned in my high school Castillian Spanish class many moons ago, comes to mind as the world of words and pictures and intellectual property is being roiled about by the explosion of digitally based media and instant communication through wireless and the internet.

Do we really understand what it is "to know"? Does it make any difference what is true or not true? And if it does, is there some standard by which we can reach that conclusion? And, even if we are never certain that we know the truth, do we believe that it is knowable?

This is heavy duty philosophy on the one hand – but is also a very simple and practical question whose answer governs how we approach everything we do. It is certainly at the heart of how we see our mission as publishers – of what each of us chooses to publish and why.

Generations of human beings today are being introduced to information and ideas with the expectation that they will be instantly available and instantly validated by virtue of the number of unique visitors, eyeballs, hits, user reviews or comments that accumulate around a citation or a work on Google, Yahoo, You-Tube, Slate, Drudge Report, Huffington Post, Move On, Wikipedia, or you name it.

If enough people line up behind an idea or a fact, that becomes the metaphysical truth of the moment. If enough people desire to have unrestricted access to words or music or images, the wisdom of crowds will view barriers to access as elitism or Berlin walls to be scaled.

If Wikipedia says it is so, even if with a warning that more editing is needed, the information becomes the fact for the day – suitable for a term paper or inclusion in an essay or opinion piece.

These facts and truths of the day carry the enormous power of knowledge into the market place of human behavior and social action. Instant communities coalesce around  what later may turn out to be a misquote, a mis-attribution, a misplaced decimal point, an incomplete or out of context citation, a plagiarism or a made-up observation.

Gone are the days when one had to reason their way through an argument, support it with documented attributions, relate it to universal truths already known or hypotheses previously investigated. Buzz words, slogans, talking points and conventional wisdom pass comfortably among us as the stuff of conversation and dialogue. Ad hominem attacks against the messenger serve to invalidate or quash discussion of the message.

On the other side of the coin on the knowledge issue, Chris Anderson (of The Long Tail) has figured out that to offer content free can be a gateway to drawing eyeballs like flies to pop-up ads pre-targeted to qualified prospects. Cheering him on, the wisdom of crowds says that content should be free in the first place.

Global search disseminates knowledge to the widest audience – now a metaphor for the library of the universe. Because it has brought together at virtually no cost markets as small as one, not to mention markets of millions, and has empowered us with instant access to new knowledge, the wisdom of crowds says it is fair use to copy anything for search and from search regardless of the creator's wishes.

Rachel Donaido writing in the Sunday Times Book review on April 27 observes that "everyone has a story – and everyone wants to tell it." Hence, according Bowker, she reports, "a whopping 400,000 books were published or distributed in the United States in 2007, up from 300,000 in 2006."

This huge addition to the global archive of information and ideas is staggering in its dimensions. Book reviewers, librarians, booksellers and researchers despair at the challenge to seek out those titles worthy of referencing and spending time with.

So what does all this mean? We still have a collective memory (first-hand or passed along) of the analog era of the printed word when a publisher's imprint meant that some entrepreneur had put their assets at risk to bring out books they though were worthwhile.  

Relying on a marketplace to validate their judgements, this was a form of self-screening quality control. Editors, reviewers, librarians and researchers vetted manuscripts and fact-checked each other in a process that could take many months to  many years. There is the feeling that this process yielded up more literary merit, screened out the unfounded and properly labeled the opinionated.

Well, those of us who have been paying attention, know that it was also a process that excluded the unnoticed, unseen and unappreciated, and edited out the unpleasant or undesirable – with little transparency that needed to withstand the wisdom of crowds. In the realm of education and opinion it nurtured as much mythology as it did what we consider the truth of the matter.

What I come to in this brief musing on knowledge and power is to observe that because the power of knowledge drives all human judgement, understanding what we call knowledge and how we arrive at it is paramount to our survival as a species that needs to master its circumstance if it is not to be overcome by it.

Because we are all of us subject to error and mis-judgement, it is better to have many ways in which our findings and opinions can be challenged, as well as many ways by means of which people can bring their versions of things into the arena for examination.

Placing the dissemination of knowledge in the hands of  professionally trained and credentialed gatekeepers who earned their position by education, training and marketplace forces has the benefit of enabling us to rely on easily identifiable authority to validate the information and ideas we depend on.  But it also true that while these gatekeepers could more easily control what found its way into the market, they were also highly visible and it was possible to check out their sources and validate them.

The wisdom of crowds replaces these well-identified knowledge sources with a vast uncertainty as to the source of that wisdom – but because it is so much more diverse and readily challenged, untruths are more readily exposed and quickly become subject to challenge by those same crowds – and more new truths find their way into the marketplace.

The problem and the challenge as I see it is not 400,000 new books a year of uncertain credibility or the subjective wisdom of crowds passing judgement. It is that we are in a marketplace of ideas and information in which the challenges of absolute dogmas,  fundamentalism, and made up realities are not being met by an equally persuasive reliance on reason and philosophy as a means of arriving at truth and judgment and by a willingness to advocate for the importance of reasoned judgment arrived at independently,

The value of crowds is not to determine the truth, but to challenge its advocates to make their case without needing a crowd to validate it. Without advocates there is nothing for the crowd to challenge – so it creates its own truth by its sheer numbers and the lack of, or suspension of independent thought by individuals among its numbers.

Knowledge, to my way of thinking, comes about because of independent thought and reasoning. It is a cognitive function which, when applied is enormously powerful because it activates our creative and purposive faculties. And those faculties brought to bear in the coalescing of crowds now possible in cyberspace es muy poderoso - is very powerful indeed, if I have that right.

Posted by: Eugene Schwartz, Editor-at-Large

posted on Thursday, May 08, 2008 9:15:52 AM (Eastern Daylight Time, UTC-04:00)  #    Comments [0]
 Thursday, April 24, 2008
For the past three weeks or so a lot of digital ink has been spilled weighing the pros and cons of Amazon's announced decision to require publishers to store POD titles with BookSurge if they want Amazon customers to rely on 24-hour shipment for fulfillment on demand rather than from inventory.

Let me say at the outset that there is a lot more here than meets the eye – and in my opinion under-utilized market forces that can restore confidence among publishers are still alive and well, and need to be dusted off and activated. Amazon's move may represent opportunity rather than threat.

What is clear so far is that conglomerate and mid-range publishers are not seriously affected by the Amazon move since very few rely on POD for book-at-a-time fulfillment; rather they follow demand printing and short run strategies. Nonetheless it is known that Amazon had been pressuring and/or negotiating with selected publishers to commit their pod lists to Book Surge.

It also seems to be the general consensus that along with this requirement publishers are having to negotiate unfavorable arrangements and uncertain quality without the option of a competitive choice or else place titles, at addtional expense, with more than one POD printer to gain the distribution advantages that might accrue.

Those who are – or will be – affected are smaller publishers, self-publishers and author-publishing services who have relied both on not having to maintain inventory and also on the shorter discounts they could offer, making it possible to keep their retail prices competitive or to increase their margins while absorbing the increased unit cost of print on demand. Of course, at this writing I have no idea who is in or out. Many are complaining, few are talking.

For the demand publishing community the wake-up call is that there are other online channels and retailers to which they can direct their customers – Barnes and Noble is probably the best positioned because of their full service B&N.com web site as well as their network of stores.

B&N provides the same transparent 24-hour shipping service to its online customers for any titles placed with the Lightning Source. POD publishers who no longer have the "order now" status with Amazon can be referring customers to B&N instead of to Amazon. Other retailers have an opportunity here to step into the breech.

There are at least a half dozen viable online retailers in addition to Amazon who can also create "in stock" arrangements with Lightning Source. There are at least a half dozen other significant digital demand printers who also serve as repositories for the title libraries of demand publishers or for long tail lists of conventional publishers. Online retailers should also offer sales fulfillment arrangements with these other POD printers.

In my opinion the publishing industry trade associations  – PMA, SPAN, ASJA, Authors Guild, AAP, AAUP,ABA, BISG – should create a working committee to develop a best practices code for keeping the internet marketplace open to POD fulfillment and to facilitate for other online retailers and digital demand printers who want to open up sales opportunities similar to the one that Amazon had been offering.

I think it would be a more efficient expenditure of funds for legal advice to activate such a group than going to court to sue Amazon.

There is no doubt in my mind that Amazon is improperly bundling its wholly owned printing service by limiting non-inventoried 24-hour POD shipment to publishers who place titles with Book Surge – especially as it is commonly known that they are doing this selectively and that they continue to drop ship direct from digital printers anyway when they need to.

I also think we don't have a fair trade situation in the retail distribution of POD titles generally. I am told that different kinds of deals are being negotiated with different publishers by Amazon that are not justified by variations in cost or other logistic considerations. Whether that is in fact so is hard to verify since there is no communication here. Nonetheless it is the demand printers like Lightning Source who determine the efficiency of service, not the small one-title or mid-range many-title publisher.

While Amazon remains opaque and elusive after a brief effort to explain itself and respond to the explosion of negative reactions from the publishing community –  I do not believe they are immune to market forces nor to the benefit of maintaining good relations with the people who produce the books they sell. (Of course I don’t have any evidence of this at the moment, since from what I have been able to discover they have been cherry-picking their publishing targets for BookSurge and make no plans public.)

For the moment they have the upper hand, earned by their success in building a fabulous marketing and sales channel for books, although they have been wielding it crudely and unilaterally. With the major exception of bundling their own printer selectively, their moves are within their rights to determine who they will do business with, what discounts they are willing to pay, and now they will ship.

The most efficient antidote is to activate immediate market alternatives such as I have suggested above. There are also longer term strategies. For example, Michael Cairns writes on his blog (April 1 http://personanondata.blogspot.com):

"Perhaps it is time for publishers to be more aggressive in becoming retailers as well as content producers. If so, it’s not as simple as setting up a store front that looks like a mini-version of the Amazon bookstore (obviously) since no one would switch. However, publishers do have the direct relationship with the author and can use this exclusivity to build a more robust presentation of the content. On Amazon you get the Buick version but on the Publisher site you get the Cadillac. None of the added or supplemental content would be made available elsewhere. What that extra content would be I don’t know. Maybe every author is twinned with an additional writer and site designer that builds/creates websites focused on the authors work but with far more expansive material about the works, process, background details, audio, video etc., any of which could be purchased by a consumer. This becomes the new marketing and promotions approach or the way to spend money that is traditionally allocated to print advertising, book tours and launch parties."

As Tim O'Reilly said recently, "Amazon has, so far, created huge value for the publishing ecosystem. Now, as they become more powerful, they need to be especially watchful that they don’t irreparably damage an industry on which they, too, depend."

In the past three weeks or so I have spent a fair amount of time networking with trade association people, industry analysts, distributors, wholesalers, publishers, digital demand printers and, yes, a few go-rounds with Amazon's Director of Corporate Communications, Patty Smith (psmith@amazon.com), until she fell silent.

In fact, falling silent seems to be the default position for the main actors in the demand printing and distribution chain. Until now, Amazon's powerful presence and occasional arbitrary moves had by and large been viewed in the industry with tolerant appreciation for the value it has added to the reach of booksellers and the search of readers.

The new silence is a self-imposed defense by major players in the distribution chain who have just seen how Amazon can wield its power at will and threaten the many business models built around its marketing gateway. Amazon sees it differently - as a reasonable move to improve efficiency and customer service.

Our trade organizations should rise to the occasion and collectively seize the initiative to activate marketplace options for an industry that seems for the moment frozen in fear, anger or frustration.

Posted by: Eugene Schwartz, Editor-at-Large

posted on Thursday, April 24, 2008 11:12:18 AM (Eastern Daylight Time, UTC-04:00)  #    Comments [3]
 Monday, April 14, 2008
A guest blog by publishing attorney Lloyd Jassin


NOTE FROM EUGENE: I have been gathering background on the recent Amazon change in POD order fulfillment policy and will be doing my own report on it next week. Meantime, I asked Lloyd Jassin, a publishing attorney and Chairman of the Executive Committee for the NY Center for Independent Publishing, for comments on the current debate concerning Amazon's new policy. He has provided the following as a private citizen-professional, and not in his capacity with the NYCIP. He can be reached at Jassin@copylaw.com.


As the market changes and we move from traditional distribution options to digital distribution options, I find Amazon's move both troubling and exciting. They want to be active all the way along the supply chain from production, to marketing to distribution. As Amazon gets more involved in digital production and distribution, it's not long before they figure out that there should be an Amazon-based publishing company. Well, on the audio side, they've already figured that out. That's the troubling part.

It's a brilliant move. You have to admit. By force of will, Amazon has become the digital asset warehouse and distributor of choice. And, how many digital asset warehouses / distributors do we need? This gives Amazon the ability to manage digital files for POD, ebooks, mobile phone devices, etc. The exciting part is that when Amazon takes this next step, it will create new revenue streams for smaller presses.

While it doesn't look like the cost of gaining access to the number one online bookstore has gone up, I'm concerned about their monopolistic tendencies. Their claim that they are not seeking exclusively (i.e., requiring POD titles be printed exclusively through Book Surge), seems to be a subtle bit of specious reasoning. Amazon's gain is the ability to monopolize the POD market. They are offering a single printer option. Your email makes that clear.

If I were a publisher, I'd look hard at the current industry model. You have the potential to get squeezed on both ends. For example, you've got the Barnes & Noble - Sterling combo with an increasing number of book sales being titles self-published by B&N. Same deal with Amazon and Audible, both of which are actively going after new product to self-publish. See Amazon's Createspace. To the extent publishers covet virtual shelf space at Amazon (with one-click ordering), Amazon's move makes them the leading POD publisher. Of course, there will also be a plethora of other digital opportunities, including e-reader, iPhone and other selling opportunities, that they should exploit for those whose files have been entrusted to them.

Their virtual warehouse of digital files can now be accessed for all manner of digital derivatives. If Amazon remains committed to the indie press segment, which has been allowed to grow to its present size due, in large part, to Amazon, that's great. Their favoritism to Book Surge, is a slippery slope that can easily decrease diversity. They are steering consumers to books that are produced by their owned and operated press.

So, as a general proposition, I think vertical integration is a bad thing. Perhaps, the market will correct itself, as publishers move over to B&N, and other digital asset distributors pop up. Likely, that won't happen. Book distribution is not sexy enough.

If I had to prognosticate, I'd say in the next 24-months Google buys Ingram (Googlegram?) and out-Amazon's Amazon, by creating the ultimate digital warehouse - distributor in the sky.

If Google were to exhibit digital favoritism, it would steer book buyers to its wholly owned Lightning Source. Amazon owns the store. Google owns the web. Amazon merchandises books. Google sells them contextually. Balance is restored to the planet.
 
-Lloyd Jasssin

Posted by: Eugene Schwartz, Editor-at-Large

posted on Monday, April 14, 2008 10:42:15 AM (Eastern Daylight Time, UTC-04:00)  #    Comments [1]
 Thursday, April 03, 2008
Note: The impact of Google on the way we do business is really a by-product of much more significant culture change in the evolution of human society. Michael Cook, a Managing Director at AG Asset Management, a money management firm in New York City, who is also an essayist, gave me permission to share his thoughts with you. He can be contacted at mcook@ag-am.com.
—Eugene G. Schwartz, Editor at Large


Life as we know it depends on DNA to transmit information from one generation to the next. Until the appearance of the human race, this was the only way favorable adaptations were retained. Thus, only those adaptations that were genetic in nature drove the progress of evolution. With the invention of language, however, a new type of evolution could occur—what Julian Huxley termed “psycho-social” evolution. The DNA of this evolution is language, and with language came the ability for humans to transmit information from one generation to the next linguistically, as well as genetically. This meant that adaptations innovated by individuals not only could be continued and built upon, but also that individual learning could accumulate from generation to generation. This sped up the pace of evolution immeasurably.

The accumulation of social knowledge brought with it new dilemmas. After a period of time, the traditions and knowledge of the human species became so vast that storing it efficiently became difficult. Oral tradition depends upon memory, which is limited. The art of memory systems was developed by the Greeks to extend the range of human memory, and the poetry of Homer used rhythms, rhymes, and other patterns to aid the memory so that it could retain vast amounts of cultural information. But these techniques were limited: ultimately the problem of storing what we could loosely refer to as the psycho-social “genome” became serious. This problem was solved by the invention of writing systems.

However, to be useful, information must not only be stored, it must be retrieved. Fairly recently in human history it was possible to have every book ever written on your bookshelf. The invention of the printing press was a watershed event in the technology of writing, which ensured that this could not remain true for long! Nevertheless, the retrieval of information from the general store was still something that could be done in a fairly straightforward manner. Of course, centers of learning—monasteries, universities, libraries – developed to manage the growing base of human knowledge. But at some point, it started to become clear that the problem of information retrieval was becoming a roadblock to the continuing development of knowledge. It also became clear that computer technology was well suited to addressing the retrieval issue.

In 1965, J.C.R. Licklider wrote Libraries of the Future, which summarized a project he had undertaken at Bolt Beranek and Newman. In his book, Licklider predicted that all human knowledge would be available on a “fast, random access computer” by the year 2000. His vision seems to be coming true. In December 2004, Google announced a project in which the libraries of five of the world’s leading academic institutions are to be digitized and made available for search and reading online.

But still, even if everything is “available” online, how can relevant information retrieval be effectuated? This is the key problem that Google addressed, and its successful solution to it, although just a beginning, essentially created the “search” industry. Google’s initial solution is called the PageRank algorithm. It was the breakthrough that started delivering search results that are relevant to the user’s search. Before Google, this had really not been the case. Their insight was to use the link structure of the web—the fact that web documents “point” to other web documents - to measure how popular sites were, and to then trust the “wisdom of crowds” by using a site’s popularity as a measure of its relevance. This, in conjunction with the appearance of search terms on the site, proved to be a surprisingly effective ranking mechanism, and the first algorithm that consistently gave users results they found useful.

At present the search industry is evolving very fast—everybody seems to have incorporated Google’s insight into their algorithm, and the race is on to understand what users mean, and what they are intending with their searches. Google’s PageRank algorithm does not address semantic content: indeed, this is part of the genius of the solution—the way it neatly sidesteps this very difficult problem. The next generation of Web Search is yet to come! But the major breakthrough that made search results relevant was invented and engineered by Google.

So here’s the progression as I see it—the thumbnail sketch of the evolution of life on earth: DNA, language, writing, printing, computers, the Internet, Google’s search algorithm.

This is why I say that the future of search is the future of life on earth, and that Google’s algorithm represents a watershed event, analogous to the invention of writing, or the invention of the printing press.

Am I overstating my case? Perhaps. But I don’t think so.

—Michael Cook

Posted by: Eugene Schwartz, Editor-at-Large

posted on Thursday, April 03, 2008 11:59:19 AM (Eastern Daylight Time, UTC-04:00)  #    Comments [1]