How To

From SWUIWiki

Introduction

This document describes how to perform research and curricula in Semantic Web User Interaction (SWUI). It is in an early stage, for which is serves mainly as a hypothetical document to crystalize discussion on forming a science around SWUI. It should eventually evolve to an instructional document.

This document starts by defining the tasks that it targets, as part of ensuring its usability. It then presents the three main components. First comes the Research Issues section, which presents the domain of research topics related to SWUI. Then the Best Practice section describes the current general understanding of that is wise and unwise in carrying our research and development int his area. Finally, the section Methodology discuss what technique SWUI researchers should apply to ensure the value of their contributions.

Target Tasks

The target audience of this document consists of people engaged in the following tasks: planning and performing research, writing papers, reviewing papers, organizing venues, writing and reviewing proposals and developing curricula.

Research

Good SWUI research should be good research in general, good Semantic Web research and good User Interface research.

Writing

After reading the abstract and introduction section of a SWUI paper, readers should know the key scientific issues mention above, that is, the abstract and introduction should explicitly mention:

what problem motivated the research described in the paper,
what the exact research question is that is being discussed,
what the authors claim to contribute in terms of answers to this research question, and
what methodology is used to evaluate the quality of that contribution.

Reviewing

Organizing

Proposals

Curricula

Research Issues

This section lays out that the research issues are that are relevant to SWUI. It is not meant to be exclusive. Instead, it provides one place where issues that are emerging can be named and described so that others can recognize and refer to them. This helps researchers identify the contribution and context of their work in publications. It also helps communication between researchers on what topics they pursue and how they relate to each other.

General Research Issues

Like all researchers, SWUI researchs should be aware

what problem orignally motivated their research
what the exact research question is that they are researching
what they claim to contribute in terms of answers to this research question, and
what methodology is used to evaluate the quality of that contribution.

Semantic Web Research Issues

Semantic Web User Interfaces have some specific features that make them stand apart form other interfaces. Just the fact that the underlying application happens to encode its information in RDF or OWL does not make the user interface on top of that a SWUI. From the Semantic Web perspective, core research questions include:

Open World Scenarios. Part of the power of the Web is that it allows people to do things the original designers of the Web specifications never thought about. To make that possible, Web applications can often deal with a wide variety of heterogenous data sources. How to design interfaces for highly heterogenous data remains, however, a big challenge and a typical SWUI research question. Even research papers about interfaces designed for very specific vocabularies (for example, about a FOAF network browser) should at least indicate how the interface deals with, for example, data that is based on extentions or specializations of the original vocabulary.

Convey Relationships of Linked Data. An essential aspect of the Semantic Web is that the meaning of an individual unit of information is determined by its relationships with other units. The naive approach, showing data that is structured as a big fat graph by drawing a big fat graph in the interface, has proven not to work [The Pathetic Fallacy of RDF]. How to design more effective alternatives to convey the information that is based on the relationships in the graph is also typical SWUI research question.

Data Distribution, Quality & Trust. Semantic Web User Interfaces often show data that is aggregated from different sources. This can lead to conflicting forces in the UI design. On the one hand, the objective of the interface is typically to shield the user from the complexity of the distribution. On the other hand, because the data is coming from several sources, the user needs to be able to track down the origin of each fact that is shown in the interface. Note that different sources may have data from different quality, and users may stop trusting the entire application based on one bad source. To prevent this, the interface needs to provide them with the means to make informative decisions on what information to trust and what not.

Scalability. To be done - moving from toy examples to real world applications

User Interfaces Research Issues

Prove Added Value. Most SW research originates in computer science, and most researchers active in the field intuitively feel that it is obvious that the ideas on which the Semantic Web is based are useful. In practice, however, it is often quite hard to identify exactly the added value of a given Semantic Web application. SWUI welcomes user evaluation studies and experiments that prove or disprove the added value that SW applications claim to have over non-SW applications.

Methodology Issues. Research proving or disproving the added value of SW applications is notoriously hard. Typically, SW applications solve different problems than the tools users currently use, so there is no baseline for comparison. In addition, SW applications often solve non-trivial problems in non-trivial domains, where any kind of experiment requires extensive user training. SWUI welcomes contributions discussing how user interfaces of Semantic Web applications should be evaluated. For example, do we, as a community, need to set up a evaluation track similar to the Information Retrieval community's TREC? I.e., should we set up public test data collections, uniform scoring procedures and organize regular comparison experiments? If yes, how? If not, why not?

Best Practice

This section discusses established and proposed best practice for research and development of environments proving interaction with information processed using Semantic Web technologies.

Defining URIs

One issue in best practice in URI assignment is the URI's resolvability. Some have proposed that a URI used in RDF should resolve in a browser to a directly presentable resource, such as a web page. The argument here is that the user will always have at least that to see during semantic browsing involving that URI. On the other hand, this risks breaking the distinction between a web page and the content it presents. For example, if you assign the RDF URI of a person as his home page URI, when you use that URI in a triple, does it define a property of that person or the web page itself?

Another issue is the use of natural language in URI's. Is it helpful to users to be able to identify concepts from reading the URI's assigned to them? Does this bias URI's toward one language over the others?

See also the Semantic Web Best Practices and Deployment Working Group's document Best Practice Recipes for Publishing RDF Vocabularies.

In addition to considerations about what to make a URI are considerations of when to make make a new URI. One issue is that URIs and triples that use them are permanent: once introduced to the global Semantic Web, they are never forgetten and cannot be deleted. Therefore, crafters of URIs should be careful to make sure new URIs do not cause problems and that they follow best practice.

Forms of Presentation and Interaction

A variety of tools exist for generating interfaces to knowledge repositories maintained with Semantic Web technologies. Given such tools, interaction designers choose which forms of presentation are suited for the different types and forms of underlying information. These designer should be informed of established practices and techniques for presenting different forms of information in different circumstances.

A core principle is that the encoding and representation that Semantic Web formats give sorted knowledge is most often not the best form of presenting it. At the surface level, the syntax used and the triple form should not be directly conveyed to users in general. Instead, they should be transformed to a form of presentation and interaction that the users understand and find familiar, or at least usable. schraefel and Karger discuss a recent tendency to present Semantic Web repositories as large graphs, with the misguided motivation being that RDF defines a graph structure so that is what users should see. The Semantic Web does not necessarily derive its own form of presentation or interaction. Often, old and familiar forms of presentation apply best to interacting with semantics. However, Semantic Web technologies can increase presentation independence for the knowledge they store, enabling a wider variety of interaction with it.

External Links

Best practice Wikipedia entry

Semantic Web Best Practices and Deployment Working Group

W3C Multimedia Semantics Incubator Group

Methodology

This section identifies describes the methologies that apply to SWUI research. Several fields contribute knowledge and methodology that is often applied in user interface and user interaction design and research. Examples include: usability, human factors, human computer interaction (HCI), industrial design, psychology, information architecture and cognitive science. All of these fields involve research-based principles and guidelines, some type of focus/involvemnt on/of humans, and evaluation. For our purposes here we are focusing on HCI and user centered design methodology because of the nature of the semantic web technology (computer or technology based) and the goal of providing effective, efficient and satisfying user interaction. Both methodologies are described here in the context of SWUI research. This section is helpful for both researchers and reviewers in determining what methodologies are appropriate and sufficient for carrying out a given research goal.

HCI Methodology

In general, HCI methodology has three phases:

analyzing your users' needs, goals, and the context of use
designing an interactive system that meets those needs
evaluating the system you build using this design, iteratively throughout the process and following implementation

There are often two additional phases, particularly in professional practice. These are:

collaborating with developers and other stakeholders during the construction of the design (when many low-level interpretations and adaptations are required)
implementing what is produced, including using what you know about users to ease adoption issues, fit with other tools in the environment, education, etc.

While this approach typically applies to building specific systems, its role in research raises certain issues. One is that research tends, rightly or wrongly (or perhaps due to the nature of research), to be more technology driven, with researchers wishing to test hypotheses about interactive possibilities that specific new technologies enable. Here, the challenge is to make sure any solutions explored have a legitimate user population, a clear articulation of the problem to solve (in context), evaluating that the approach does indeed solve it, and that it solves it better than other available solutions for the same problem. Thus, even with a particular solution technology in mind, the Semantic Web researcher still needs to apply the main HCI phases. In each phase, the technology that the research is applying should be compared with other available technologies that can address the same problem.

An additional challenge for research is gaining access and involvement of representative users within the constraints that may sometimes exist in the academic research environment. It would be useful to articulate that challenge more clearly, to identify options that may be available in order to engage real users more fully in the process.

Analysis

Analysis is the first step. It is where the problem you are trying to solve is articulated and studied. This includes setting goals and objectives. Some evaluation activities to support analysis are: creating user profiles, conducting task analysis, and developing and documenting user scenarios. This is where you decide how you want to measure and evaluate your product or research prototype or idea.

Design

Design is the creative process of instantiating your idea for a user interface design or user interaction. It involves brainstorming, creating screen flows and tasks flows and navigation models. It also involves creating low-fidelity prototypes of your design that you can test with users, further refining the design and getting it ready for the implementation phase. This may be as far as a research project or question goes in the typical HCI methodology process. But even here, cognitive walkthroughs and expert reviews can be conducted based on known research based design principels as well as various feedback activities with users.

Evaluation

Evaluation is a critical component in research and product development. There are many types of evaluation than can be done throughout a research study or development project. Some of these techniques have already been mentioned in the analysis and design phases. Other kinds of evaluation include usability testing, comparative analysis with other designs trying to address the same issue, feature analysis, surveys and structured feedback activities such as a task based focus group. The section below, Evaluation in UCD, provides a nice cateogorization of various evaluation activities.

User Data

Researchers can process data collected from users of systems to obtain measurements from which to derive additional insight. Access logs are one example, whose processing can derive patterns of use. Once a set of user data is collected, it can be used for multiple experiments. Researchers can also record user performance and behavioral data (e.g., task success rates, time on task, error rates) and collect perceptual and satisfcation data about user's experiences with particular interaction designs. Both quantitative and qualitative analyses can be conducted on user data.

User-Centered Design (UCD) Methodology

The methodology for developing usable and accessible applications is called User-Centered Design (UCD). It is based on an iterative development process that includes detailed study of the users' needs, the tasks they carry out in order to meet them and the context in which they are performed. There are many UCD development processes and proposals, but all of them provide a mix of software engineering plus usability and accessibility engineering tasks.

As other software development processes, UCD processes start with the requirements gathering phase. However, the emphasis is placed on users. First of all, it is important to know who the users are. Then, the following step is to identify the tasks they are going to perform.

The development process continues with the common phases, i.e. design, implementation and deployment. Despite these similarities, the focus continues to be placed on the user.

In addition to the user focus, the key elements in UCD are iteration and evaluation in all four phases of analysis, design, implementation and deployment. Evaluation can include such things as user analysis, task analysis, ethnographic studies, site visits, cognitive walkthoughs and usability testing. Prototypes of ideas and designs are used to interact with users from the very early stages and are an important feedback and evaluation mechanism.

Prototypes

Prototypes are created from the beginning, for instance paper prototypes, which do not require any implementation, or simple applications with limited functionality. All of them are used to evaluate the system with users so their requirements are taken into account and contrasted with the developed system just from the beginning and through all the development process iterations.

Evaluation in UCD

Once developed, prototypes are tested and evaluated with users and experts. There are three kinds of evaluation methods:

Inspection: these evaluation methodologies are performed by experts, the evaluators, that inspect the usability and accessibility aspects of the system based on a set of guidelines, e.g. heuristic usability evaluations and walkthroughs.

Inquiry: the objective is to draw usability conclusions from observing and talking with users. There are surveys, interviews, field observations, focus groups, log analysis, etc.

Test: can be performed in a controlled environment, e.g., a usability laboratory, where specialised software applications are used to record and analyse the whole interaction, i.e. screen capture, key strokes, mouse clicks, user video record and voice,... while representative users interact with the system or a prototype. But testing can also be done without specialized testing software with a user, a prototype or application, and an analyst that records information and issues and interacts with the user using various protocols such as the think-aloud method or the active intervention method. Usability testing is very useful because with as few as 5-7 users, 80% of the major usability issues can be uncovered.

One of the valuable contributions of the various evaluation methods in user centered design is they can be done with a small set of users, without a lot of fancy technology or equipment, and still provide useful data. This is especially pertinent to researchers who often have limited time and access to representative users but want to include user feedback and involvement in their research.

Know Thy User

A critical element in researching appropriate user interfaces and user interaction is knowing and understanding your target user group. Both HCI and UCD methodologies stress this aspect. Different groups have different characteristics, needs and knowledge. Designs for knowledge engineers may expose much more semantic web technology as a necessity than a design for general end users that just want the power the semantic web can provide without having to understand or deal with any of the underlying technical aspects. Identifying and understanding your target user group will positively inform your research studies in the arena of semantic web user interaction.

Example "How To" Documents from Related Fields

Contributions

This page contains contributions from

Lloyd Rutledge
Jacco van Ossenbruggen
Duane Degler
Laura Downey
Roberto García

Feel free to add you name to this list if you'd like to be acknowledged for your contribution.

Retrieved from "http://swuiwiki.webscience.org/index.php/How_To"