I’ll be at the Enterprise Search Summit in New York next week, participating on a panel Tuesday morning to discuss “Emergent Social Search Experience”. Our game plan as a panel is to discuss what social search is, why it matters, and how to implement it.
Obviously these are broad questions, but here are my rough notes:
WHAT: Social search means many things, but they have one common thread: improving information seeking through the knowledge and efforts other people. Back in the mid 90s, researchers distinguished between semantic and social navigation as the ability to explore information based on its objective, semantic structure, versus choosing a perspective based on the activity of another person or group of people. Perhaps the earliest instance of social search was collaborative filtering, still popular today as driver for product recommendations on sites like Amazon. But social search is much more than collaborative filtering. Building on the 90s vision of social navigation, we can give users full control over a social lens through which to view information, e.g., show me the local restaurants where women in my mom’s demographic like to eat brunch. Social search also includes explicit and implicit collaborative approaches, such as finding an expert to help you with a search, or building shared knowledge management artifacts that increase the collective efficiency of information seeking.
WHY: The “why” of social search depends on the specific aspect of social search that we’re discussing. But the common theme is this: we all know that, for a large swath of information needs, we prefer to turn to a person than to ask a machine. Sometimes that’s appropriate, and it’s a question of finding the right person to ask. But often we have no need to bother any one; we just want to borrow someone else’s perspective—or to assemble a composite perspective. There’s an efficiency gain of not reinventing the wheel, as well as an upside of discovering people (or information by way of those people) that may be valuable to you in ways you didn’t anticipate.
HOW: Again, it depends on the aspect of social search. We need rich knowledge representations that treat both information and people as first-class objects, and interfaces that let people seamlessly use both. Endeca does this by supporting record relationship navigation for multiple entity types (e.g., documents, people), as do interfaces like David Huynh’s Freebase Parallax. To facilitate collective knowledge management, we need to make contribution both easy and rewarding: the reason people don’t contribute to such systems today is that they are onerous and don’t work. Some of the work Endeca has done with folksonomies is encouraging: we found that we can productively recycle folksonomies (or even search logs) in combination with automatic text mining techniques. Finally, we need to rethink our attitudes toward privacy, anonymity, and reputation. Consumer social networks like Facebook and Twitter have shown us that users are willing to forgo privacy in order to gain social benefits. Wikipedia has shown us that a group of strangers can assemble a valuable collective knowledge store. But Wikipedia, product reviews, blog comments, etc. have shown us that the default of anonymity can undermine the trust we have in these socially constructed artifacts. As we evolve these tools—and as we work to apply them within the enterprise, we need to simultaneously work to evolve our social norms.
Those are my thoughts. But, in the spirt of social search, I’d love to reach out to experts here for ideas. If you were attending a panel about social search, specifically in the context of an event target to enterprise search practitioners, what would you want to hear about? For that matter, if you were participating on such a panel, what would you talk about? Bear in mind that the audience will consist of practitioners, not researchers, and I’ll only have one third of a 45-minute session–some of that reserved for Q&A.
The Noisy Channel 
26 replies on “What Should I Say About Social Search?”
You say that social search is important for a “large swath of information needs” — while that’s true, it’s good to be more specific. I’ve seen lots of failures around exploratory searches (open-ended or ill-framed). How much do I harden silver so it won’t bend easily? Where can I buy that hat that Julia Roberts was wearing in that photograph? Is it legal to sell Grade B maple syrup in New York State? Who sits on the New Mexico parole board and how are they selected? [examples from real data.]
An example from my own life: what is the damn species of hummingbird that is nesting on my porch? This has been much harder than it sounds! How do I know when I’ve found my answer?
I think the take away here is that the “information needs” where social search is relevant extends beyond just recommendations, tips for places to eat, or technical help.
Another easy thing to say is that there are 2 models of social search emerging (see ref below): one that involves social recommendations (passive interaction with social data) and one that involves social answering (finding experts, Q-A activity, etc.).
Not sure if these comments are relevant to enterprise practitioners! Good luck!
Ed Chi. Information Seeking Can Be Social. Computer 42, 3 (Mar. 2009), 42-46.
http://asc-parc.blogspot.com/2009/03/information-seeking-can-be-social.html
LikeLike
I’ve seen lots of failures around exploratory searches (open-ended or ill-framed). How much do I harden silver so it won’t bend easily? Where can I buy that hat that Julia Roberts was wearing in that photograph? Is it legal to sell Grade B maple syrup in New York State? Who sits on the New Mexico parole board and how are they selected? [examples from real data.]
Brynn, are you saying that these examples are exploratory search examples? They sound more like known-item, Q-A, fact finding (aka “lookup”) searches, than exploratory searches.
LikeLike
@jeremy They may sound like “lookup” searches but when you get down into the search, you realize that they are not. These subjects spend many minutes (to hours) searching online and asking people for help before they could find what they were looking for.
Perhaps one of the reasons they sound like fact-finding events is that those types of information should be readily available somewhere. But of course, not everything is recorded and indexed in search engines (Grade B syrup example). It also happens that once you get involved in the search, you come to realize that there are more aspects to it than you thought. This is what happened with the New Mexico parole board search: after learning more information, this person realized that there was still more to learn. How do you harden silver just enough so it won’t bend easily? These how-to questions are often quite difficult to solve online. So I would classify these as exploratory searches.
If you disagree, I’d be curious what you think a classic exploratory search looks like.
LikeLike
Daniel: I kinda disagree with yer characterization of social search, about it encompassing both implicit and explicit search activities. See Gene’s post, which reflects a few years of our thinking on the matter: http://palblog.fxpal.com/?p=350
And also my comments, about re-covery vs. dis-covery.
Social search, in my understanding, is about one person or group of people knowing something, another person or group of people NOT knowing something, and then using algorithms and social cues to connect people with people, or people with facts, THROUGH other people.
Explicitly collaborative search, on the other hand, is when two or more people or groups of people get together, and NO one in either group knows the fact. All explicitly collaborating team members do not have any knowledge of the answer, and you want to work together to find it.
I see that as a big, fundamental difference to social search. Social search is built on the foundation of someone else in the system knowing the answer, and social search systems are designed to leverage that person’s existence. Explicitly collaborative search, on the other hand, does not start with that same assumption.
Collaborative search is “social” in the sense that humans are working together, and human activity is a social activity — but that’s about it.
LikeLike
Brynn: good point: I’ll make sure to use concrete examples, like my recent social search experiments.
Jeremy: I grant the point you’re making about the different between social and collaborative search. I suspect that for this audience, however, “social search” encompasses both, and probably more. This is the blurb for the panel (which I didn’t write):
The social web is all the rage, but does “social” work for the enterprise? Like water cooler wisdom, social search can help surface information workers might not know exists. Search inside and outside the enterprise is becoming social. This session will look at the evolution of social search, how crowd wisdom is evolving search interfaces, and what the enterprise can learn from these developments.
LikeLike
Ok, maybe the hardening of silver is exploratory, as one is trying to synthesize and aggregate information 🙂
But finding out who is on a parole board? I mean, that just a fact/answer/item search. It may or may not be difficult to find that answer, depending on how readily available the information is, and how well the search engine does at surfacing that information. But at the end of the day, you are not really analyzing, synthesizing, comparing, or contrasting anything — all aspects of an exploratory information need.
The difficulty of finding the answer is not what makes something exploratory. You can have difficult lookup/known-item information needs. An exploratory search is more about learning and investigation. It is about trying to gain enough information about an area that you are then able to synthesize a new, richer model or coherent explanation of a domain. It’s about forming a new understanding via the overlap of all relevant sources of information.
A great example of an exploratory search is something like: “Which stocks in my portfolio should I sell right now, and which stock not in my portfolio should I buy?” There doesn’t exist a web page anywhere telling me exactly what I should do (and if there is such a page, I wouldn’t trust it anyway 😉 ) Rather, the process itself of investigating (exploratorily searching for) all relevant/related information to the companies in my portfolio, as well as the companies not in my portfolio, plus all relevant related information about what is happening in various sectors of the economy, are what makes the information need exploratory. I have to explore until I have enough information to make my decision.
But whether or not it is legal to sell Grade B maple syrup, or who is on a parole board.. those might be difficult answers to find, difficult facts to look up. But don’t confuse difficulty, or absence of a fact from the index with exploratory search. At the end of the day, they’re still basically facts-lookups, no matter how difficult.
That’s just my opinion/feeling, though. I am totally open to discussion, especially if I’ve erroneously misportrayed anything. In particular, you say: “This is what happened with the New Mexico parole board search: after learning more information, this person realized that there was still more to learn.”
If this search was part of a larger objective — such as finding out how to reduce someone’s prison sentence — and finding the answer to this one fact was but a small part of the overall information need, then yes, it could still be exploratory. But just finding this one answer, by itself, if that is all you’re after, doesn’t seem exploratory. Figuring out how to reduce someone’s prison sentence does.
LikeLike
I hear some of your comments. But I’m not so easily convinced that exploratory searches are just like the example you gave. The stock example seems more like a decision making process to me than just search. While I grant that searches to take place over periods of time and place, I start to wonder what our lowest common denominator of a “search” is then. Every single search query on Google (stocks example) could then be said to be a fact-lookup. The synthesis part comes in between, before or after. So where do we draw the line?
Is it not possible that a question that is perceived to be ill-framed or open-ended from the searcher’s perspective, that results in multiple queries, several passes at trying to understand how to get to that information, etc., cannot also be exploratory? A searcher’s knowledge of a domain must surely affect how straight forward his search seems to him, even if a system designer knows that the information is present in database X.
I guess what I’m pushing on is: exploratory search is such a general term — should we perhaps think about what causes a search to be exploratory or not? I contend that such explorations could occur if the searcher doesn’t have a good search strategy, the information is obscured or hard-to-find, or the problem is inherently ill-formed (among others). From the perspective of the searcher, all these factors could lead to needing to learn and investigate, synthesize and assimilate new information in a domain.
This is an interesting discussion. Daniel, I hope it’s somehow relevant to your panel coming up!
LikeLike
I suspect that for this audience, however, “social search” encompasses both, and probably more. This is the blurb for the panel (which I didn’t write):
Fair enough. Unless you’re totally into this, most people probably don’t care about making such fine distinctions anyway. 🙂
But there is a takeaway point that I would hope attendees could be clear on, and it relates to the blurb for the panel:
The social web is all the rage, but does “social” work for the enterprise? Like water cooler wisdom, social search can help surface information workers might not know exists. Search inside and outside the enterprise is becoming social. This session will look at the evolution of social search, how crowd wisdom is evolving search interfaces, and what the enterprise can learn from these developments.
This is what I am reacting against.. the idea that all social search is “wisdom of the crowd” search. Folks need to understand that there is also a type of search (I say collabo, but it doesn’t really matter what you actually call it) that does *not* rely on the wisdom of the *crowd*. It relies on the wisdom of the very small team. Call it “extreme programming for search” or whatever you’d like. Just don’t call it “crowd-based”.
See also http://irgupf.com/2009/04/21/dagstuhl-seminar-on-content-based-retrieval/
LikeLike
Sorry to be pedantic, but I believe LikeMinds technology as well as Firefly predated Amazon. They were recommender systems but they were used on search and Web sites. John Riedle would probably know who deserves primacy.
LikeLike
That’s a great point, Jeremy. Ed Chi calls that “social answering” (Q-A) — it’s like using an even smaller team for search, but still involves people talking to people, not just wisdom of crowds. I can imagine this distinction would be relevant for the enterprise community.
LikeLike
Every single search query on Google (stocks example) could then be said to be a fact-lookup. The synthesis part comes in between, before or after. So where do we draw the line?
My stocks example wasn’t meant to convey the idea that I wanted to look up the price of the stock. It was meant to convey the idea that I want to find out as much possible information about the company whose stock I own, the industry that company is a part of, and the outlook for that sector of the economy, so that I understand whether or not I should sell that stock, no matter what its current price is. That is definitely not a fact-lookup process. That is a boundary-discovering explorational process.
But I think you’re correct in saying that Google is explicitly designed to be a fact lookup engine. It is not geared at all toward exploratory search at all. To the extent that you and I try to use it in an exploratory manner, anyway, that’s fine. Just because we can do it, doesn’t mean it was designed for that purpose. You can hammer a nail into a board using the butt end of a screwdriver. That doesn’t make the screwdriver a hammer, though. That, to me, is where you draw the line. If you start having to use a screwdriver to hammer in a nail, then the line has been crossed.
Is it not possible that a question that is perceived to be ill-framed or open-ended from the searcher’s perspective, that results in multiple queries, several passes at trying to understand how to get to that information, etc., cannot also be exploratory?
You’re saying that if the searcher is trying to look up a fact, but doesn’t know the right/correct terminology to use, to find that fact.. that is an example of exploratory search?
Arr.. mm.. ahh.. (if you could see me right now, you’d see my face making hilarious contortions and I ponder and deliberate 😉 )
Maybe.
But at the end of the day, once you’ve found the “right” query terms to use, to find your fact, you’re really just interested in the fact, right? You’re not doing any analysis, synthesis, comparison, aggregation, forecasting, etc. And to me, those verbs are what exploratory search is all about.
You do have a good point, and I’m not completely against it. At the same time, it also feels like it’s an instance of the whole “how do I look up the spelling of a word in the dictionary if I don’t know how to spell it” dilemma. It makes the search hard, but you’re really just after the word, the single fact.
Perhaps we should both reread this article, and continue the discussion after the weekend? 🙂
Click to access Marchionini-2006-Exploratory_Search.pdf
LikeLike
That’s a great point, Jeremy. Ed Chi calls that “social answering” (Q-A) — it’s like using an even smaller team for search, but still involves people talking to people, not just wisdom of crowds. I can imagine this distinction would be relevant for the enterprise community.
No, what I am talking about is still not social answering (at least as Ed describes it in this blogpost: http://asc-parc.blogspot.com/2009/03/information-seeking-can-be-social.html)
In social answering, it’s still the case that one person does not know the answer, but that the other person does! And the goal of the system is to come up with ways of connecting these two people to each other. In my mind, that is still a “wisdom of crowds” scenario, because even though you are getting your answer directly from one person, rather than from the crowd as a whole, you are relying on the existence of the crowd to either (1) connect you to the right person, or (2) have enough critical mass so that the right person — the one holding the correct answer — exists in the first place.
No, what I am talking about with collabo search is when you explicitly get together with 1-2 other people, and both of you start looking for relevant information.
For example, you and your partner want to find an apartment to rent. You don’t know the answer, and your partner doesn’t know the answer. So you start both looking, and coming up with candidates. That is not social in any way that most people talk about it, and it certainly isn’t “social answering”. But it is collaborative.
LikeLike
@Whit Andrews: If you’re pedantic, I hate to think of what I probably am. 🙂 I’ll forgive your indulgences, if you forgive mine 🙂
But yeah, speaking of early recommender systems and collaborative filtering, I always though that PARC had come up with the first, back in the early 90’s. See:
http://www.coedu.usf.edu/agents/dlewis/LIS5937/LIS5937paper.htm
LikeLike
Brynn.. you write:
The stock example seems more like a decision making process to me than just search.
That’s not contradictory. People search because they have information needs (they lack information, and require it). People have information needs because they need to take actions or make decisions. Like selling the stock or not. But not matter what the information is used for, it’s still an information need. And where there are information needs, there is information retrieval, or search.
I wrote: My stocks example wasn’t meant to convey the idea that I wanted to look up the price of the stock. It was meant to convey the idea that I want to find out as much possible information about the company whose stock I own, the industry that company is a part of, and the outlook for that sector of the economy, so that I understand whether or not I should sell that stock, no matter what its current price is. That is definitely not a fact-lookup process. That is a boundary-discovering explorational process.
For example, suppose it is 1901 and I own stock in a horse-and-buggy company. if I am trying to figure out whether or not to sell that stock, I want to be able to explore, and find information not only about how well my company is doing in the market, what their costs are, and how well their competition is doing, but about other factors that I might not even know to ask.
For example, the U.S. acquired Panama in 1901, and announced plans to start building a canal in 1903. That could change the cost of shipping, and mean that my horse-and-buggy company can get materials at a cheaper rate, and thus make more profit. And the same time, 1901 was also the same year that the mass production of the automobile was first “invented”. And so even though buggy-cart materials are getting cheaper, the auto might obsolete the whole industry. Sell sell sell!
And exploratory search is one in which you want to be able to find all those factors, so that you can synthesize, aggregate, forecast, etc. and have enough information to finally act. It’s exploratory search because you lack this information, and need to find it.
So to me, exploratory search goes well beyond just learning what terms you need to use to find the answer to some question (e.g. who is on the parole board). And exploratory search is one in which you’re trying to find what it is you don’t even yet know you need to find. Like the Panama Canal or the Oldsmobile when you own horse-and-buggy stock.
This is what you’re saying, when you’re talking about the problem being “inherently ill-formed”, am I correct?
I contend that such explorations could occur if the searcher doesn’t have a good search strategy, the information is obscured or hard-to-find, or the problem is inherently ill-formed (among others). From the perspective of the searcher, all these factors could lead to needing to learn and investigate, synthesize and assimilate new information in a domain.
If so, then I think we’re in agreement, there.
But I’m still not fully on board with the notion that just because something is hard to find, it’s an exploratory search. It’s a hard search, but while exploration often implies difficultly, difficulty does not imply exploration.
But either way, I agree — great discussion! More on Monday? 🙂
LikeLike
Wow, I’m delighted to have catalyzed such a discussion, and I hope some of it will trickle down to the panel discussion!
Brynn, Jeremy: I’m inclined to agree with Brynn that exploratory information seeking processes count as exploratory search, even if the end-goal is fact-finding. More here. But it’s clear from this discussion that it would be useful to have terminology that makes a distinction. In any case, the panel is about social search, not exploratory search–not that I’m uninterested in the latter!
Whit: I didn’t mean to imply that Amazon invented collaborative filtering–in fact, the post I cite on social navigation cites the Tapestry research at PARC from 1992 (also cited un the Wikipedia entry). I believe Firefly was a little bit later.
LikeLike
Yup, I always enjoy a good discussion as well. And whichever way it falls out, that’s ok.. after all we’re researchers and working together to get this stuff straight is an important part of what we do.
I guess, though, that I have a problem with conflating exploratory processes with exploratory goals. By so doing, one can claim that Google is an exploratory search engine. Why? Look at what they say about how we should be using Google, in their “guidelines for better search”:
http://www.google.com/support/websearch/bin/answer.py?hl=en&answer=134479
Describe what you need with as few terms as possible. The goal of each word in a query is to focus it further. Since all words are used, each additional word limits the results. If you limit too much, you will miss a lot of useful information. The main advantage to starting with fewer keywords is that, if you don’t get what you need, the results will likely give you a good indication of what additional words are needed to refine your results on the next search.
Is that not exactly a recommendation to follow an exploratory process? Start small, see what the engine gives you, learn from it, iterate, grow the expression of your info need. That sounds exactly like an exploratory search process. So Google is an exploratory search engine, from the ground up? I don’t quite accept that.
LikeLike
I like the distinction between exploratory processes with exploratory goals–and I think exploratory search is the parent concept.
In any case, I don’t think that Google supports exploratory processes–indeed, the excerpt you cite pretty much says that the user’s only support for progressive information seeking comes from cogitating on the results themselves. That sort of support is a freebie, and hardly a credit to Google.
LikeLike
I like the distinction between exploratory processes with exploratory goals–and I think exploratory search is the parent concept.
Agreed.
But here’s my concern, the precipice that I’m trying to skirt as we zoom at breakneck speed around the curvy cliffs of information retrieval metadiscussion: If you define “exploratory search” (or at least the “exploratory process” subcategory) as any kind of search in which you have to try a couple of times until you find exactly the right words to use to find whatever fact you’re after, then most if not all user information seeking behavior is exploratory search behavior. Most if not all processes, whether fact finding, lookup, or whatever, become exploratory processes, under that definition.
And I want to make sure that we retain a (imho) critical distinction between processes that are simply a little more difficult to express and take a little bit of extra time and care to get right (to verify, navigate, answer), versus processes that necessitate analysis, integration, synthesis, discovery, transformation, etc. in order to properly be carried out. The latter processes are exploratory. The former processes, though more difficult, are still lookup/question-answering/fact retrieval/navigation-based.
Am I.. am I at least not completely out in left field, here?
LikeLike
Jeremy, I see your point that there’s a slippery slope, and I suppose I’m biased towards a broader rather than narrower definition for exploratory search. The question that interests me more is the extent to which a search tool supports exploration. Is the tool simply answering the user’s questions, or is it helping the user formulate them?
I agree that there’s a difference between reformulating questions for a fixed information need and evolving the information need itself. But I consider both to be modes of exploratory search. Though I’ll cede to different definitions if I’m the one out in left field!
LikeLike
Jeremy, I see your point that there’s a slippery slope, and I suppose I’m biased towards a broader rather than narrower definition for exploratory search.
Oh, I hear ya, am on your side, and hope that I’m not giving any impression to the contrary.
The question that interests me more is the extent to which a search tool supports exploration. Is the tool simply answering the user’s questions, or is it helping the user formulate them?
Agreed as well; I also espouse that goal, and believe that IR systems need to become much more interactive and, well, exploratory 🙂
But being too broad can sometimes narrow what it is you’re (we’re) trying to say. You’ve heard the saying, about someone who is “so well rounded, that they have no point?” It’s a clever pun, but with a ring of truth — we don’t want exploratory search to be so broadly defined that it really doesn’t say anything at all.
Let’s again go back to the Google example. Say, Google Suggest. As I start to type my query, Google “autocompletes” a dozen possible query suggestions, thereby going beyond simply letting me type my query in, and does actually help me, the user, formulate my queries.
So Google + Suggest is an exploratory search system? If we’re saying that it’s still exploratory search when someone has a lookup/fact finding/QA need, but then has a system that enables an interactive, multi-option seeking process, then Google Suggest is an exploratory search system.
Somehow, that still doesn’t sit right to me. That doesn’t feel like what I mean when I think about exploratory search.
LikeLike
I still have to take the other side on this one: in my view, Google Suggest does offer token support for exploration. In fact, I’ve used it that way on occasion to get ideas from the suggestions.
LikeLike
Ok, fair enough 🙂 Thanks for the discussion.
LikeLike
Thanks to all of you! Now let’s see what I can use from this in my minutes of stage time tomorrow morning. 🙂
LikeLike
BTW, I went and re-read this article over the weekend, as I suggested on Friday:
Click to access Marchionini-2006-Exploratory_Search.pdf
And I found support for both of our positions. It’s a good read; I’d recommend it to anyone following this thread.
LikeLike
[…] about socially enhanced decision making. I consider Hunch, like Aardvark, to be an example of social search, but I recognize that I use the term in a broad sense. Perhaps, as Jeremy suggests, it’s […]
LikeLike
[…] ‘Tis the season of search conferences, at least for me, and I spent the last couple of days attending the Enterprise Search Summit in New York. I enjoyed it immensely–it’s one of the better networking events in the industry. It was great to catch up with analysts, consultants, information architects, and even competitors–or, as Nate put it, “respected foes”. I’m grateful to Michelle Manafy for putting it all together, and to Will Evans for getting me included on a social search panel. […]
LikeLike