Miriam, I imagine most businesses would prefer negative feedback in private and positive feedback in public. But I could be wrong. Indeed, responding to negative feedback in public may be a great way to win new customers. I’m curious how many businesses encourage such an open conversation.

Gregor, I agree that there’s a lot more information in a review than a star rating. You might want to take a look at http://www.buzzillions.com/.

Bob, that makes sense intuitively–and only reinforces concerns about bias. As for finding negative reviews on the open web, that seems to be one of the canonical applications of text analytics.

Lecia, you raise an important point about private investigators–and you might appreciate my blog post about public expression, liability, and anonymity. As for the opinion marketplace, I don’t see money changing hands, but rather people using their reputations as social currency. People already do that offline–why should the online world be so different?

A big (and growing) field for private investigators is getting the identities of people who post strong, negative opinions of businesses online — hired by the businesses of course.

This lack of “reviewer” transparency clearly has a dollar value to the businesses who are willing to pay to uncover the origin of the review.

Will there ever be a dollar value to uncover positive reviewers? Would consumers be willing to pay to figure it out through some form of an opinion marketplace?

I doubt it…but you pose an interesting question!

The “equation” in the title refers to Nass and Reeves’ and others’ findings that people interact with computers socially with many of the same patterns as they react with people.

As a search problem, many companies are interested in finding negative reviews to see if there are issues they can address.

To bring this all back to Endeca-style search, what Gregor’s saying is that we need to be able to auto-facet the reviews (by what is called “aspect” if you’re looking for papers on the topic).

Like you, I don’t see anything wrong with a business owner asking customers to consider reviewing them. Our company acquires testimonials to be published on our website in this manner, and when a client is really happy, they are glad to oblige.

I am especially in favor of soliciting reviews because of the fact that many people are unaware that user reviews even exist. The business owner has the chance to educate his customer about this, acting, in a sense, as an entree into this world of user generated content.

But you’ve raised a good point: does lack of spontaneity diminish the value of reviews? Inasmuch as any smart SMB is only going to encourage reviews from his happy customers, then, yes…there might be something slightly artificial about a company’s profile with 100% positive reviews. It also might have the effect of distorting an SMB’s understanding of how people are actually talking about him if the majority of his reviews are ones he’s attained rather than one’s that have occurred without any action on his part. So, there is a bit of shaky ground here.

Overall, though, I think that as time goes by and more people understand that they can review businesses, there will be a healthy mix of owner-sought and voluntary reviews.

A good and thoughtful piece.

Agreed. Caveat, however: transparency alone wouldn’t be enough. Because not all queries (e.g. navigation) need transparency. So the independent search engine that we’d need to build to run this experiment has to be equal in quality to Google for navigational queries, with the ability to switch to transparent interaction for informational/exploratory queries. Given that requirement, Wikia Search does not count as such an experiment.

“But the success of Hadoop that I alluded to earlier is evidence that this transparency isn’t completely obscured.”

Yup, I agreed to the Hadoop/Mapreduce benefits in my comment #8, above. Still, that’s not a part of the system (other than query response time) that the user sees. I’m interested in HCIR.

“I’m glad we agree that we’re debating shades of gray.”

Of course. We’re not republicans, now, are we?

“I agree with colleagues that there’s a real risk to users in over-disclosing the details of ranking. I hope to see the day where it’s possible to have relevance without obscurity.”

One day when you can finally, legally talk about it, I’d like to know what it is you’ve learned since joining the Big G that seems to have moved you more toward this risk-averse state. Maybe that’ll have to be in 20 years. But I have sheer academic curiosity about the thinking process that is occurring.

As for the research papers, I think Google has made a point of promoting what it believes to be the most important ones–and they are infrastructure papers, not search papers per se. But the success of Hadoop that I alluded to earlier is evidence that this transparency isn’t completely obscured.

I’m glad we agree that we’re debating shades of gray. I’d like to strive toward a lighter shade than the current one, but I agree with colleagues that there’s a real risk to users in over-disclosing the details of ranking. I hope to see the day where it’s possible to have relevance without obscurity.

Matt’s general argument isn’t that there are things about Google that are open/known. His argument is that Google the company actively has an agenda to make more things transparent, carries through with that agenda, and that Google should get more credit for making things transparent (the active followthrough) than its critics give it.

Because PageRank was disclosed by NSF-funded grad students doing their job of publishing, and not by Google, it doesn’t support Matt’s argument at all.

I’m sorry that I’m being such a stickler about this point, but I see it quite strongly. What we’re talking about is what Google as a company is actively doing.

For that matter, Cutts writes: “Google has continued to publish literally hundreds of research papers over the years. Those papers reveal many of the “secret formulas” for how Google works”

I’ll ask the question again: “And sure, there are hundreds of papers by Googlers. But how many of those papers were done (primarily) by summer interns, who already came to Google with a solid sense of a research idea? And of the ones that are Google-internal only, how many of those are on production-code systems, vs raw research? Microsoft publishes hundreds of papers as well, the vast majority of which are never incorporated into shipped products. Does that also make Microsoft transparent?”

In other words, how much of this is “transparency through obscurity”. I.e. if there are hundreds of papers with secret formulas in them, and only 7 of those formulas actually make it into production code, and it is never make known which 7 those are, then how can this reasonably be called transparency? If there is so much flak thrown up, then any real disclosure is essentially so obfuscated so as to not really be transparency at all.

I do agree with you, Daniel, that it’s not a black and white issue. Google doesn’t get an A+ on transparency, nor do they get an F-. But the picture is also not quite as clean as Cutts paints it, either.

How much of this is chicken/egg? I remember Steve Jobs claiming that there was no evidence that users wanted to watch video on an extremely small screen. Six months later Apple released the video iPod, and now lots of people do.

If users don’t know or don’t understand what is possible, they’ll never ask for it. There has to be some education, first. How much education is Google doing?

But you raise an interesting point. I’m with you on the HCIR perspective of focusing on end-user penetrability. And Gregor is a refreshing example of a user who cares about the details of how the search engine works.

But I see far greater concern about the ranking details by site owners than by users, e.g. a lot more literature about site owners can improve their site ranking than on how users can search more effectively.

According to the Christensen’s theory of disruptive technology (at least Wikipedia’s explanation), ” a disruptive technology may enter the market and provide a product which has lower performance than the incumbent but which exceeds the requirements of certain segments, thereby gaining a foothold in the market.” If users value transparency, then there should be room for someone to offer a product that lags the incumbent on most measures but offers greater transparency. I believe this is already happening in markets like enterprise search, but I don’t see the evidence for it in web search.

And, to be clear, I’ve advocated for it! But that’s doesn’t make it so.

With all respect, I still strongly disagree about giving even a little credit — not to L&S — but to Google. You have to look at the motivation of Larry and Sergey, and what was going on at the time. At the time that they actually submitted the paper for publication, they were researchers at a university and funded by NSF money. Their job, their primary product, their deliverable, was publications. So when they published PageRank they did so before they ever started Google the company, and did so because it was the mandate of their academic position and government funding to do so. Not because of any attempt to be more or less transparent. Had they not published the paper, they would have not been properly doing their jobs at that time. Transparency had nothing to do with it.

I hear what you’re saying; they are indeed the founders of Google. But I still do not see how you can give Google credit for something that was a requirement of a pre-Google job. Doesn’t make any sense.

But I think Gregor gets to the heart of this issue, above. What Gregor is saying is that it doesn’t really matter what the exact mathematics of the Google ranking algorithm is. What matters is the end-user penetrability of that algorithm. Is Google transparent enough so that a searcher using the Google system can give Google the right signals, in the right way, at the right time, so as to find the information that is needed? And I think the answer to that is still no. It’s changing, and I do see small HCIR signs here and there. But at the end of the day, that’s the question that really matters.

Of course, making Google more HCIR-transparent will most likely involve exposing, in some fashion or other, more of the Google ranking algorithm. It might not be a raw exposure, but there will still have to be an exposure of some sort in order to give the users more control over their experience.

So the question: How transparent is Google in that regard?

From that perspective, all those papers on MapReduce (which I do give transparency credit for) and PageRank (which I don’t give transparency credit for — it was their job at the time!) don’t matter one way or the other. Because the technical details contained in those papers are still not exposed to the end user in a transparent way. There is no way for the end user to say “Stop using the PageRank popularity signal for my query, because I know that what I am looking for is in the tail!”

Transparency is therefore still at a minimum for what really matters: End user experience.

In any case, I agree that it would be great for researchers to run different algorithms and approaches on a Google-size index and with real-life queries. I think researchers are in a position to build a sufficiently large index–in part because of the publications I describe above. But what researchers really want–at least from what I have heard–is access to query logs and actual user traffic. That raises major privacy concerns for users, as well as concern about abuse by spammers. Don’t forget what happened a few years ago.

Anyway, there are good reasons for obscurity for the commercial and search quality reasons you mentioned. However, in order to advance scientific understanding of search quality, there is also a need for transparency. I mean that it should be possible for IR and HCIR researchers to run different algorithms and approaches on a Gooogle-size index and with real-life queries, and publish the algorithms and results. It would be nice to see Google contribute to such an endeavour, in order to prevent a situation where the knowledge about how to deliver a great search experience is confined to only a handful of big companies.

And yes, the measure doesn’t conform precisely to the paper–and in any case it’s only one factor. But, to put it in the context of Matt’s post: “One of the most widely-discussed parts of Google’s scoring has always been PageRank. That ’secret ingredient’ is hardly a secret.”

As for the papers, MapReduce, the Google File System, Bigtable, and Protocol Buffers are essential production tools. I agree with Matt that Google deserves credit for enabling Hadoop, thus supporting its largest web search competitor.

In any case, no one at Google denies keeping the precise details of ranking secret. But I’d say that critics who extrapolate that the entire ranking algorithm is a black box overstate their case, given how much has been disclosed. Unless you feel that being transparent is like being pregnant–an all or nothing deal. I don’t believe that.

Of course, the other issue is Google’s motive for what secrecy it does keep. As per an official blog post by Google’s counsel, Google stands accused of demoting the positions of competitive sites. I don’t personally believe this to be the case, nor have I even seen anyone present evidence for it. I do understand how some people–particularly site owners unhappy with their ranking–may feel Google is guilty unless it discloses everything in order to prove itself innocent. Given the stakes, I don’t think it’s reasonable to expect Google to make such a sacrifice–at the expense not only of its own competitive position but also of its users.

Jeremy, I know you have many points of disagreement with Google’s approach, particularly with regard to transparency. Perhaps the extent of Google’s disclosure isn’t enough to earn a passing score in your book. I’m hardly giving it an A+. But I hope we can agree that it isn’t black and white.

So it is disingenuous, at the very least, to give Google credit for this. This information was released (at least submitted for publication with the intention of being released) before Google the company existed. So whatever L&S’s attitudes about transparency were as grad students, I think it changed once they became incorporated as Google.

And I cannot count the number of times I’ve seen Google talks in which the Google speaker has intentionally gone out of his or her way to mention that the form of PageRank that they use in today’s engine is nothing like the one published in the paper. I don’t know if that’s the truth, or a deliberate attempt at obfuscation. But the upshot is that this means that Google is even less transparent than claimed, because it has the feel of trying to throw people off track — now one doesn’t know what to believe anymore.. the published paper or the research talks.

So to go back now and try and claim “transparency credit” for this paper… makes me uneasy.

And sure, there are hundreds of papers by Googlers. But how many of those papers were done (primarily) by summer interns, who already came to Google with a solid sense of a research idea? And of the ones that are Google-internal only, how many of those are on production-code systems, vs raw research? Microsoft publishes hundreds of papers as well, the vast majority of which are never incorporated into shipped products. Does that also make Microsoft transparent?

We had great resonance within Microsoft with the research on attention as scarce resource and with attention-aware machinery/designs at the company especially during the span of years around 1999-2002, with strong support of Bill Gates and other senior executives and group leads. My sense is that the research and prototyping helped with company-wide reflection about ideas and directions, and, perhaps more important, helped to catalyze a persistent shift of thinking about the importance of attentional considerations in computing systems. The shift in perspective is having some long-term influences in both machinery and design (and combinations)–we are only at the beginning! There’s much to be done! Progress in going from exciting research prototypes has been slower than I had expected. I had demo’d a fairly comprehensive (per components), end-to-end attention management system named Notification Platform to Bill Gates and others in 1999 or so-and that had generated a bunch of excitement.

Specific product influences to date include MSN Alerts, several pieces of infrastructure, such as rich schemas/APIs for communicating about urgency and attention, components in products for computing time until user available, next usage, etc.. and several products and product features, such as the Outlook Mobile Manager product–based on the Priorities system (Priorities was available internally at Microsoft since 1998 or so) or so, and on Microsoft Communicator –which evolved out of the Bestcom (best-means communication) prototype that we had developed in a collaboration with a product team. The Bestcom prototype for managing telephony/real-time communications and communication scheduling spread like wildfire within Microsoft around 2003-2005 and was used by about 10,000 Microsoft folks (per our logging–part of the deal for using it) at its peak within Microsoft before the code was forked to start the path toward the Communicator product (now shipping, but lacking a bunch of the deeper attention management features in the research prototype ancestor). There have also been some influences on design. Priorities used an urgency- and context-sensitive fading alert that was picked up by Outlook, but that alert, while resembling the alpha-blended alert in the Priorities system isn’t currently attention sensitive.

So, we’re on the path on this challenging and interesting terrain, and people at all levels seem to be aware and interested, but there’s a great deal to be done to make our systems more aware of human cognition (attention, memory, judgment, etc.)

-Eric

Very interesting (and sorry just finding out now about the conference!).

I’m going to use the Furnas quote in the paper in my next blog – “people use a surprisingly great variety of words to refer to the same thing.” Hm.

I haven’t ever known someone to wait on launching a product until their patents are granted. Folks who do pursue patents will file first, of course, but will then launch with pending applications. Given that inventors can pursue back-date royalties based on published applications, the delay doesn’t strike me as such a big deal. The bigger problem in my view, is that the current system incents the wrong behavior, such as in the story I describe here (which has an unhappy ending) .

http://research.microsoft.com/en-us/um/people/horvitz/attend.htm

It’s a great paper. From the abstract: “We present models and inference procedures that balance the context-sensitive costs of deferring alerts with the cost of interruption.”

Along the lines of a semantic web, what if senders could “tag” their emails, which would allow each receiver to rank what “tags” get higher priority?

A similar system was described at our recent CSCW Collaborative Information Seeking workshop:

http://workshops.fxpal.com/cscw2010cis/

See the paper by Manas Tungare:

http://workshops.fxpal.com/cscw2010cis/AcceptedPapers.aspx

For example, emails tagged assignment, speaking engagement, basketball tickets, deadline changed, or even “new baby” (a la Ian’s example) will come through at real time if you choose and others would be saved for the end-of-day email review.

Of course the above system is easily exploitable by advertisers and spammers, but there are already ways to deal with them.

Enjoying your blog!

Lecia

Your perception may change if you consider an ad as a vehicle that carries the viewer to the web page or as a vehicle that carries the ad to the viewer.

Consider a teeter-totter in a playground with a searcher on one side and a web page on the other. The ad is just the pivot.

Once the ad takes the searcher to the web page or the web page to the searcher then it is useless.

Cheers,
Nick
http://www.neuropersona.com

Max: I hadn’t thought about task-based desktop organization of activities, but I could see how a smarter desktop might be more prudent about interrupting user flow. Still, that feels second-order to the issue of being interrupted by communication.

Ian: fair points. In response to one of them, perhaps we could all be expose to a small amount of random content, thus distributing the work of scouting.

Craig: I agree that the sender should be part of the equation. I was going to go off on a tangent about attention bond mechanisms, but I figured I’d save that for another post.

Jeremy: nice segue! And I think the challenge is understanding what the user will perceive as better. I actually think users would accept an interface that sometimes said “hold on, this will take me a moment” if it then delivered on expectations. But there is a catch: latency in giving users initial feedback may mean that they wait too long to find out that their going on the wrong track. Anyway, this is a another great subject for another post.

David: good point about the cost of soliciting feedback. I think I’m advocating a similar “comparative advantage” approach by having high-urgency users scout for low-urgency ones.

Yes, I would like this to extend to web search, too. Instead of assuming that fast is always better than slow, as certain companies do, I would like the option of being able to say: I don’t care if it take 10 seconds or even 2 minutes for this particular query to return, as long as the results, or the organization of the results, is better than what I’m getting with the 0.34 second result.

Why does this not exist?

http://knowledgeforward.wordpress.com/2006/12/22/my-attention-management-system-conceptual-architecture/

Since it is not one, purpose-built, tightly integrated set of pieces, it takes a walk-through to apply it to any particular problem. The problem you mention in this posting is e-mail, and you’ve provided 3 good suggestions on how to take advantage of urgency. I applied the EAM model to e-mail as an example and yielded 15 examples where technology could help, many of which are indeed available in some e-mail systems (although often buried or cludgy). You can see my list and how the EAM architecture helped derive it here:

http://knowledgeforward.wordpress.com/2009/06/10/e-mail-overload-no-cure-but-enterprise-attention-management-can-shed-some-light/

I really like your thought that urgency should be taken into account in the e-mail process. You have some good ideas for the receiving end of e-mail. I still wouldn’t give up on the sender’s side too. When sending letters and packages, people don’t mind picking between a number of options (ground, express, signature required, etc.) that indicate urgency. If we can do a bit of behavior change (or possible force people via a token system), it’s interesting to think about how much e-mail could be improved. Easier said than done though.

I agree with what you are saying – it’s also not just about the amount of information, but the wasted attention time is spent filtering rather than consuming.

You make a nice point about interrupting too, which is compounded by the wasted attention spent as we get distracted with consuming something that is not relevant.

I love the idea of an e-mail voting system, where the priority of a mail gets voted down – but who chooses this gateway of folks?

Also, if we start consuming stuff that only creates ‘buzz’ – the propagation of content from scouts to followers – don’t we then end up consuming a lot of bland popular content, enjoyed by the noisy few – rather than relevant to us?

Obviously we choose our scouts ourselves and I admit I have scouts that I trust when clicking on a link on Twitter – but don’t we miss the opportunity to find something interesting, maybe surprising and perhaps uniquely relevant to us? Aren’t the majority of us scouts for own small communities?

Also, I think the timeliness of consumption could be a uniquely personal thing. If an ‘all staff’ email tells me someone is joining the organisation that day and I stumble into them at the coffee machine. The timeliness of that email maybe its relevance.

Not sure where I am going with this – whilst I agree I’d love an ‘Ian Truscott’ shaped internet and e-mail filter – but I wonder if that’s more about the delivering and finding of well understood content. It’s actually about the content?

I also wonder if we should actually just make time for the irrelevant? Make time to enjoy a blog post that’s not laser focused on our interests, or enjoy the fact that Mary in some far flung office (who you’ve never met, will probably never meet) is having a baby.

Anyway, nice post…

Ian

one thing, that i saw as a side benefit, as it wasnt mentioned in the talk at CHI, was a potential benefit of a task-based desktop system – on-going work from stephen voida.

http://doi.acm.org/10.1145/1518701.1518744

its not hitting all of your goals for an email prioritization system, but it highlighted new email per task-desktop (based on labelled collaborators).

i know task-based desktop organisation is still a little idealistic, but i liked it and it seemed like an opportunity to interrupt work if new email relating to the current task arrived.

Most other graph visualization approaches try to replace standard list-following interaction with graph-based traversal, and that’s why (imo) they don’t work.

Maybe similar-set-common-membership is the best way to do things. But w/r/t ranked lists, I am proposing the following ordering of systems:

(worst = 3) Graph traversal + graph visualization interfaces
(2) Ranked list traversal + ranked list visualization interfaces
(best = 1) Ranked list traversal + graph visualization interfaces

I’m just trying to make the case that 3 != 1, that you cannot just conflate anything with a graph into the same thing. And that by understanding the difference in interaction (the “I” in HCIR) between (graph traversal + graph visualization) versus (list traversal + graph visualization), we don’t risk throwing the baby out with the bathwater.

I’m still interested in hearing back from Ed Chi, too. Ed, are you following this thread? Is the work that you’ve done in any way similar to Anton’s? Because again, most of everything else that I’ve seen over the past 10 years is not like that. And so I can see why folks dismiss it. They haven’t actually played around with ideas like Anton’s.

I’ve always found that completely problematic, because even if the document similarity algorithm is 100% reliable, there is still a difference between similar documents and relevant documents. A ranked list orders documents by their relevance to the query, whereas a graph layout orders documents by their similarity to each other.

So what you really want is to traverse the documents by their relevance to the query. (Well, I know you’re not a huge fan of ranking, but bear with me for this moment; generally sorting by probability of relevance is a good thing.) So what Anton’s system lets you do is traverse by relevance to the query. But if there then does happen to be a set of documents that do cluster, you can easily visualize that in a way that you can’t from the ranked list. You can quickly jump to those few documents, then jump back to the ranked list traversal.

So that’s the interaction, that’s the process. Walk the list, but see the graph.

Most of everything else I’ve seen in this space requires searchers to both walk and see the graph, with no list in sight. And that’s (imo) why they don’t work.

But that difference in interaction — walk the list vs. walk the graph — is key.

Here’s the reference that Don mentioned: http://ciir-publications.cs.umass.edu/getpdf.php?id=176

I’m not convinced that automation would decrease quality on a per-submission basis, but it certainly introduces thorny dependency issues. But I think that’s an acceptable price to pay in order to potentially expand the marketplace by several orders of magnitude.

Davidc, perhaps I’m naive to take Mechanical Turk at face value. Though even spam creation is ripe for automation. Anyway, I do like Hanson’s attitude (this link worked better for me). But my goal is here is the advancement of commerce rather than science. That the latter would benefit is merely a benign side effect.

In any case, I’d love to play with Anton’s demo. Hard for me to conclude much from a screenshot.

And it’s the interaction, rather than the visualization, that I think is the key here.

I think you have a good idea though. How about a online prize site. You would need measurable tests on each task and give the reward to the best algorithm. Robin Hanson has a good paper on the value of prizes for science here
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.67.9839&rep=rep1&type=pdf

I imagine Amazon’s reasoning is to prevent fraud (e.g., someone builds a script that automatically submits thousands of bad results, hoping the HITs will be approved). Certainly the requestors could build ways to not approve such results, however, I imagine Amazon is requestor-focused, especially in ameliorating their concerns for quality. Some requestors I know approve all HITs as policy and just avoid any fraudulent workers in future tasks.

Humans also occasionally submit bad results, but I think the overall level of bad results may increase if automated submissions were facilitated.

There should definitely be MTurk like platforms that are optimized for developers (ideally also on the Amazon system), for those interested in finding out ways to automate tasks. That way requestors could pick and choose.

Correction: “mark the nodes that you’d like to see more of”.

Too much music IR in my background

It was the interactivity, the back-and-forth play between list and visualization, that I like, that make this work interesting.

Everything else that I’ve seen in graph visualization over the past decade gives you the graph, and only the graph, with maybe a summary snippet of whatever current node you are mousing over. And I will definitely agree with you that having only the graph is not good.

But you do have more experience than me in this, so could you point me to any sort of evaluation of something like Anton’s work, where the user essentially has both visualizations simultaneously, i.e. the list AND the graph, and can see the effect that manipulating one has on the other, simultaneously?

Because again, it is wrong to call Anton’s work a “graph visualization”. Because it’s not. It’s a “list plus graph visualization”. It works because it does both, simultaneously.

Eric’s tool very much reminds me of our own work at http://MrTaggy.com

The issue he speaks of about “It’s too much work to scan to see what’s new, what went away, and what moved”, we spent a lot of time dealing with in our design.

Anton’s work, on the other hand, lets you simultaneously see both the graph and the ranked list (top 50 or 100 results). So when you make a change to the list view, it appears in the graph view, and vice versa.

This is the power of his work, imho. It’s the multiplicity of the views, which share state and are mutually informative.

Jeremy: I’m intrigued by Anton’s work, though I have to confess a certain skepticism about information retrieval using graph-layout visualizations. Perhaps I’m too burned by my own past attempts in this area.

Otis: thanks for the link. I do think global occurrence counts are helpful. I also assume that Yahoo is using its query logs to improve the quality of keyword extraction beyond what would be possible if all you could look at was the document text.

If I had to do that again, I’d use this Key Phrase Extractor (I’m biased):
http://www.sematext.com/products/key-phrase-extractor/index.html

I am particularly leaning towards only having one sidebar per search. Right now I have one sidebar per page of search results. It’s too much work to scan to see what’s new, what went away, and what moved.

Also, I am looking into using Google AdWords keyword cost per day and total searches per day to help order the sidebar results.

I first encountered Anton’s work back in ‘98 or so. I agree; it’s a shame that this sort of interactivity never took off on the open web. I’ve always been fascinated by it.

http://people.ict.usc.edu/~leuski/projects/lighthouse/index.html

One minor thing, in the paragraph 3, “Robert” -> “Richard”

Cheers

Craig