There’s a reason I love emerging technology so much: over the course of one hour, the entire landscape can be turned on its head. The rumor out of VentureBeat this afternoon, that Microsoft will acquire Powerset for $100 million next month, has produced the predictable memes: Microsoft is desperate after the Yahoo debacle; Powerset overhyped itself to bankruptcy and needs a bailout; Powerset only searches Wikipedia and we like Google just fine, thanks. While neither party will confirm the rumors, it now seems likely that something significant will happen in the semantic sector over the next couple of months. Having analyzed Powerset and semantic search extensively, I think we should keep a couple of key points in mind beyond the arguments over valuation and hype machines.

1) After years toiling in the uber-geek trenches, semantics just got uber-interesting. As I wrote late last week, the days of GoogleKiller.com – an ultimate destination search site to rule them all – are behind us. The next wave of semantics lies in plugging into already existing sites and products that desperately need a smarter framework behind them.

It’s now time to focus on the niggling details: how to identify and amass relevant information, present it to users in an easily consumable yet rich visual format, and, perhaps most importantly, establish a user base that will consistently turn to your product as a valued information resource.

Powerset just landed (speculatively of course) the largest user base in the world. Which brings me to point number two.

2) That massive user base won’t amount to much if users are faced with a complex, hard-to-parse interface. Though some disagreed, I viewed Powerset’s Wikipedia search as a changing of the game in search.

Once you’ve dug into the meat of a Wikipedia article with just a couple of clicks, zeroing in on precise actions and entities and going directly to their citations in the article, paging through flat hyperlinks just ain’t going to cut it.

Many viewed the Wikipedia search product as a disappointing launch from Powerset, after so many months of hype. They were focused on the wrong angle though, fixating on the language used to search rather than the results produced. (And I do blame Powerset for some of this, as previous messaging from the company fixated on search phrasing.) Asking Wikipedia where you should go for dinner tonight is barking up the wrong tree, no matter what engine you use. But what if you applied those same Powerset algorithms to your email? Or to Live Search listings? Or to the documents on your hard drive? Microsoft – or any other company wanting to throw its hat in at this point – sees the potential in Powerset’s algorithms, yes. But more importantly, that intuitive little UI the company threw on top of Wikipedia opens up a wide realm of consumer possibilities.

3) The talk of whether this is an over-valued deal is valid but, in one sense, it would be more than a deal between two companies. It would be an acknowledgement by the business sector at large that semantics’ time has come. That it’s time to take it out of the university labs and hands of the rocket scientists and put it in front of consumers for true vetting. Is semantics ready for that stage? I think it is. Will it look and sound the same after going through the billion-dollar-behemoth ringer? Probably not. But that’s the most exciting part of all.

Back in the old days – or the ’90s as some call them – we utilized the Internet as an information resource. What’s that phone number, where is that address, where can I buy that product – you had concrete questions and were no longer required to speak to a human to get answers. Sure, there were bulletin boards and Usenet forums for discussion but they primarily involved coding arguments and game walkthroughs. The Internet wasn’t truly upended into a community, and all that that entails, until just a couple of years ago. It was then that the inundation of bloggers collided with social networking and lifestreaming to produce a perfect storm of content. (And when I say lifestreaming, I mean the trend of putting as many pieces of our life online as possible – books we’re reading, music we like, etc.) We’ve now backed ourselves into a corner online, raging against the indundation of content even as we scroll through our fifth page of FriendFeed updates. We recommend well-written articles about navigating through the noise, right after sharing 25 items in Google Reader.

The logical next step in this technological journey is to therefore prune, to make our time online more meaningful and relevent, no matter how small the nugget of information. Whether I’m setting out to qualify findings in a drug discovery experiment or wondering when Amy Winehouse was last arrested, I want the most reliable, relevant answer in the shortest amount of time. The problem is no longer whether the information is out there but rather how we can get to it quickly and accurately.

It’s against this background that I’m seeing a gradual evolution of the semantic search market. What was once viewed as a sector attempting to “kill” Google has become something different in recent months. It turns out that what Google does is 1) pretty hard to do well and 2) pretty sufficient for most people. Creating an all-purpose search engine that answers any query in any form and delivers a more targeted result will take time to get right. I don’t think we’re going to wake up one day to find GoogleKiller.com suddenly filling all our search needs.

Several companies I’ve met with lately have innovative takes on making the Web more meaningful for users. But I’ve noticed that my reaction has gone from, “Wow, I’ve never seen that before” to “Oh, that’s an interesting take on what so-and-so is doing.” In other words, I believe smarter search and discovery have reached a new stage in their evolution. The problem has been well defined and the angles from which it can be approached have been established (at least for the foreseeable future.) It’s now time to focus on the niggling details: how to identify and amass relevant information, present it to users in an easily consumable yet rich visual format, and, perhaps most importantly, establish a user base that will consistently turn to your product as a valued information resource.

I’ve written previously about companies that I keep a close eye on in the smarter search space. These products, along with a couple of new entrants, bear repeating, as they’re molding a new shape for the Internet.

Powerset raised the bar for UI with its Wikipedia search engine, and, perhaps more importantly, sent a tacit message to the tech world that a smarter search engine isn’t going to develop overnight. Regular readers of The Guidewire know I won’t shutup about Silobreaker, a current affairs search engine that really shines in visualization and content extraction. Its entity maps are among the best I’ve seen, providing instant insight into relevant connections between people, places and things. Evri is a newer entrant in the field, with a product that combines the visual maps of Silobreaker with the subject and verb extraction of Powerset to give users a deeper dive into news articles. Semantinet brings to mind Yoono‘s discovery feature, but on steroids, recognizing entities on the page you’re viewing and bringing in related info from other sites. It makes discovery more intriguing by adding a personal layer, i.e., while viewing a page about Italy, receiving notification that a Facebook friend just added photos from Italy. True Knowledge is Freebase meets Cyc with natural language search thrown in for good measure.

As I was writing this post, a couple of good articles popped up that are worth reading. ReadWriteWeb took a broad look at trends it believes will shake up search in the near future. And Epicenter’s Betsy Schiffman had an interesting take on Google’s dominance of the market. I have to respectfully disagree with Chris Sherman of Search Engine Land, who said in Schiffman’s piece:

“I certainly think the game’s over in terms of building a world-class search engine. It’s too expensive — Google and others are literally spending hundreds of millions of dollars, in terms of research, development and infrastructure. Start-ups don’t have access to those sorts of resources.”

I see his point about cost but think we’re far from declaring game over. Though GoogleKiller.com may not appear any time soon, the smaller engines are working other avenues to build adoption and user base. The game has changed from pouring millions into a destination site to instead channeling resources into multiple iterations of a product to gain traction. Powerset, for example, is testing a new Facebook app that finally brings capable search into the social network, in addition to its just-announced iPhone app, which generated 46% of its total traffic yesterday.

Sherman is right in one aspect: with such a complex problem and so many angles from which to attack, going it alone as a destination engine makes for a tougher, longer and more expensive fight. Smart companies are now shifting focus, integrating their algorithms into existing communities that are sorely in need of sophisticated search. Look for this integration and coalescing to continue on multiple levels in the next year. It will no longer be about killing Google but about making search, discovery and recommenation a more seamless and intuitive part of our existing online lives.

For all my stewing about presenting an effective panel here at SemTech, I think we did it in spades this morning. I’m biased of course but if the amount of active, engaged audience members and lively conversation following the panel was any indication, Taking Semantic Technology to the Masses was a success. Thomas Tague, Josh Dilworth, Mark Johnson and I had an excellent discussion about the mess the semantics space is currently in, marketing-wise, and how to dig it out and shine it up for mass consumers. We spent the first 25 minutes parsing the problem – an indication of just how deeply semantics geeks can gaze at their navels – and about 20 more minutes discussing possible solutions.

Thomas coined a term I’m stealing that sums up the semantics space perfectly: geekery fiefdom. It’s a great description of a sector that is striving to achieve traction in the consumer space, but continues to pepper its messaging with semantic buzzwords and discussions of the plumbing behind it all. As Thomas quoted one of his customers in the financial sector, “If you have to explain it, I don’t want it.”

We came to a couple of good conclusions worth mentioning:

1) Companies in the semantic space need to take a portion of their impressive brainpower and turn it toward marketing. With literal rocket scientists on the benches, finding innovative, well-packaged messages around a product and company philosophy should be a piece of cake.

2)UI, UI, UI. Mark mentioned this several times and he should know; Powerset has one of the best out there right now. Once you’ve parsed out the complex algorithms of your semantics company, spend some time on a great design. An easy-to-use, intuitive interface can vault a product to the head of the pack.

3) Play nice and share. (I’m reminded of that annoying book/poster from the early 90s – Everything I Need to Know I Learned in Kindergarten.) It’s simple but true. If semantics companies were more open to partnering with each other, the resulting applications would without a doubt take this industry to the next level. The close-to-the-vest attitude is understandable in semantics, as some very sophisticated and complex platforms and algorithms are at stake, but I think we’ve reached the point where it’s time to open up a little.

Everyone seemed to agree, including members of the audience, that semantics is poised to graduate; that it’s time to dust off this fiefdom and take it out into the countryside among real users. When and how that will happen is still undecided but I’d bet on later this year or early next.

That’s it for the moment from SemTech. I’m huddling with Hakia in a bit and can’t wait to hear their news, then it’s time to concentrate on the French Tech Tour for the next 12 hours. More tomorrow…

Anyone with even a remote interest in the semantic space has likely experienced the same roller coaster I have regarding Powerset. When I first spoke with Barney Pell over a year ago, the semantic tech sector was an entirely different landscape. I was intrigued by my conversation with Barney and the short demo I saw of Powerset-enabled search. How nifty that the engine knows what I mean by “who did IBM acquire”! But as months went by, we didn’t hear much from Powerset, save a seemingly incongruous Labs announcement. And we heard much from other players in the space. The focus of the semantics community moved away from search to organization – making users’ Internet activity easier to manage – and answering the question of how to take semantics to the masses. Frankly, I had dismissed Powerset as an early mover in the space that had run out of steam. Boy was I wrong.

Powerset’s introduction today of its new Wikipedia search, which also integrates data from Freebase, could have a significant impact on the tech market overall, in that it changes the rules of the search game. Users who experience the incredibly deep, interactive, and intuitive nature of the Powerset search will be even more frustrated with the standard string of result pages delivered by traditional keyword search. Once you’ve dug into the meat of a Wikipedia article with just a couple of clicks, zeroing in on precise actions and entities and going directly to their citations in the article, paging through flat hyperlinks just ain’t going to cut it.

Powerset’s changing of the rules is evidenced by one key statement made by the company: a page of search results, no matter how targeted, is just the beginning of the effort required by the user. Once you’ve found relevant links, you still have to click through to new pages and scour the text for usable information. Powerset’s new way of searching attempts to do some of that work for you; with the scouring and drilling down already complete, you arrive at what you need much quicker.

The Outline feature of the Powerset search is a real gem and I expect will set a new standard for UI in search technology. Having a constant window beside the text as you browse provides an incredibly simple way to jump back and forth between concepts and facts. It could make the browser’s back button obsolete.

What I don’t love about the new search is that it’s currently only on Wikipedia. There are many searches I typed in that can’t take advantage of all this whiz-bang semantic technology. More nebulous concepts aren’t Wikipedia’s strong suit, so Powerset only returns standard results. Example: “Can Hillary win the democratic nomination” returned relevant results but no Wikipedia entry to plumb. So my big “if” with this announcement is whether Powerset can pursue a successful content partnership strategy. If the right publishers, and enough of them, integrate Powerset search into their sites, the long-anticipated threat to Google could finally take shape. No matter the long-term outcome, though, Powerset has raised the bar for search interaction and usability.