IS ENOF ENOUGH? DESIGN AND EVALUATION OF AN ELECTRONIC NEWSPAPER OF THE FUTURE Lorrie Ackerman May 1993 CHAPTER 1. INTRODUCTION Electronic newspaper development once focussed on delivering the contents of a printed newspaper electronically. Now, electronic newspaper researchers seek to develop news products with capabilities and features beyond those available from printed newspapers. These new electronic newspapers allow subscribers to perform keyword searches, specify criteria by which news stories are automatically selected for reading, and view photographs and video clips. Further technological advances may allow the development of an electronic newspaper far superior to printed newspapers in the near future. 1.1 RESEARCH OVERVIEW The purpose of this thesis is to determine the preferences of potential electronic newspaper users through the design and evaluation of a working prototype Electronic Newspaper Of the Future (ENOF). This knowledge of user preferences is intended to serve as a guide for future research. This research was conducted in three phases: literature review and user survey, ENOF development, and ENOF evaluation. In the first phase, literature pertaining to electronic newspapers and news reading habits was reviewed. The review led to the formulation of three main assumptions: Interests are dynamic, difficult to specify, and best judged by each individual. People can quickly filter information to eliminate that which they find least interesting. The traditional printed newspaper format is not the ideal format for an electronic newspaper. Also in phase one, potential ENOF users were surveyed about their news media habits and preferences. In the second phase, ENOF was designed and implemented based on the assumptions and survey results. A qualitative evaluation of ENOF was conducted in the third phase. 1.2 POTENTIAL IMPACTS OF ELECTRONIC NEWSPAPERS In order to explain the importance of electronic newspaper development in the context of technology and human affairs, this section presents an overview of potential impacts of electronic newspapers. Most new technologies have some impact on society, however, new communications technologies may have more far reaching impacts than other technologies bib:Katzman . By altering the way in which communications take place, new communications technologies can change the quantity, quality, and distribution of information in society. Newspapers have traditionally played important roles in society as both public educators and ``watchdogs.'' As educators they provide people with the information they need to make informed decisions. As watchdogs, they provide a check on government, holding public officials responsible for their actions. Traditionally newspapers have been able to play these roles because they deliver the same news to all members of a community; by browsing through the paper, people who would not normally seek certain types of news tend to be exposed to it anyway. Arguably, customized electronic newspapers are likely to change the role of newspapers in society. Zerbinos bib:Zerbinos states that newspapers bring people together and make citizens aware of important issues. However, because electronic newspapers can allow readers, rather than information providers, to decide which information they need, readers of electronic newspapers might become isolated from information that they, as citizens, should know bib:Blumler . Automatic filtering enables readers of electronic newspapers to ignore information which is unrelated to the information they are seeking. As a result, readers of finely-filtered electronic newspapers may be less likely to browse than readers of printed newspapers, even if electronic newspapers become well-suited for browsing. Therefore, electronic newspaper readers may not be exposed to information on ``topics that might be important to the smooth working of a democratic society but which, at the time, may not seem to be personally relevant'' bib:Zerbinos . The ``play value'' of newspaper reading and the human need to interact with other people, however, might be incentive enough for electronic newspaper readers to subscribe to coarsely-filtered electronic newspapers and continue to read articles that are not personally relevant. People read newspapers not only for personally relevant information, but for enjoyment and to increase their conversational knowledge bib:Atkin,bib:Stephenson . Katzman bib:Katzman states that a potential social impact of new communications technology is the widening of the gap between the ``information-rich'' and ``information-poor.'' He explains that individuals tend to realize information gains as a result of new communications technologies in proportion to their initial information levels. As a result, the gap between information levels tends to widen because of differences in education, financial resources, and ability to use new technologies. In addition, ``information more naturally boosts the wealth of those who already have material goods'' bib:Dertouzos . Electronic newspapers are thus likely to widen the knowledge gap, possibly leading to an increasing polarization of socioeconomic classes. Electronic newspapers are also likely to have environmental impacts. In 1987, an estimated 14,400,000 tons of newsprint were consumed in the United States bib:Pulp . Electronic newspapers, which would eliminate the need for physical mass production of newspapers, would drastically reduce the amount of newsprint consumed. Paper consumption would probably be reduced even if electronic newspaper subscribers choose to print their newspapers on their own printers. People are likely to print much less news than is contained in most printed newspapers because people tend to read less than half the contents of a daily newspaper bib:Graber . Furthermore, while advertising may be as essential for electronic newspapers as it currently is for printed newspapers bib:Lie , it may take a new form. Electronic newspapers are well-suited for yellow-page-style advertising, in which ads are grouped by product type. Electronic newspaper subscribers may have the option of viewing ads only when they request them. As a result, an electronic ad is likely to be viewed by fewer people than a printed ad. However, people who request to view an ad are more likely to purchase the advertised product than people who view ads involuntarily. In addition, the advent of electronic newspapers could have a major impact on printed newspaper publishers. Some printed newspaper publishers are likely to view electronic newspapers as competition, while others may view them as another outlet for the articles their reporters write. Several major newspapers, including the St. Louis Post-Dispatch , have already started publishing electronic versions of their printed newspapers. In 1992, Knight Ridder, a Miami-based newspaper chain, announced that it would create the Knight Ridder Information Design Laboratory in Boulder, Colorado bib:Battelle . This laboratory is the first newspaper industry sponsored center for electronic news product research. Economics and decisions of publishers will continue to shape the future of the newspaper industry. The ability of electronic newspapers to satisfy media needs is also likely to play a critical role. CHAPTER 2. BACKGROUND 2.1 DEFINITION OF TERMS Electronic publishing is a term used to convey a variety of ideas. Most broadly, it refers to the use of computers in the composing, editing, typesetting, printing, or publication-delivery processes. According to this definition, most printed newspapers can be considered electronic publications because they are composed, edited, typeset, and delivered to printing presses using computers. A more narrow definition of the term electronic publishing refers to the use of computers to deliver the final publication to the reader. The terms electronic publishing and electronic newspaper in this thesis refer to the narrow definition. Hypertext refers to a non-linear electronic document that contains electronic links which allow readers to quickly jump between sections or related documents. Often such links are embedded in the text of the document and can be accessed with a mouse by clicking on selected words. Indexed archive refers to a database with a companion index. The index facilitates the quick location of records in the database. Indexer refers to a program that automatically generates an index for archived text. Inverted file index refers to an index which contains an entry for every word of every indexed text (often excluding certain frequently-occurring words). This is similar to a book or magazine index. Information filtering is the process of retrieving specified information from a stream of incoming data bib:Foltz . Information retrieval is the process of retrieving specified information from a database. Unlike information filtering, which involves continuously monitoring a data stream, information retrieval usually involves a one-time query bib:Foltz . Media is a term with a variety of meanings. In this thesis it refers to the publication and broadcast of news and news-related entertainment. Media include printed and electronic newspapers, radio and television news programs, magazines, and newsletters. The discussion of media in this thesis excludes movies, non-news-related television and radio programs, technical journals, games, and books. Media habits refers to a person's typical media usage. This thesis examines frequency and type of media usage. Media preferences refers to the media types and qualities that a person prefers. Metaphor refers to a physical object or group of objects that a computerized system emulates. Some parts of the ENOF system, such as section and headline, are named using a newspaper metaphor. However, the system as a whole does not emulate a printed newspaper. Unix Unix is a trademark of AT Bell Labs. is a multi-user operating system. The ENOF system was designed to run on a Unix computer system. USENET is a cooperative information exchange service accessible through international computer networks bib:Spafford . USENET is divided into several thousand ``newsgroups,'' each of which contains ``articles'' on a specific topic. While some newsgroups are moderated or compiled from outside sources, most are a from of interactive or two-way media much like electronic bulletin boards. Created in 1979 by two Duke University graduate students, USENET is now accessible to hundreds of thousands of computer users around the world. Window refers to a modular region of a computer screen. A window system allows users to simultaneously display the input and output of multiple programs on a single computer screen. Each program can run in its own window. Windows can usually be moved, resized, and overlapped. XView XView is a trademark of Sun Microsystems, Inc. is one of many X Window System program development tools. It was used to write the ENOF program. X Window System is a window system developed at the Massachusetts Institute of Technology. ENOF is an X Window System program. 2.2 LITERATURE REVIEW Some knowledge of the history of printed and electronic newspapers is important for understanding how newspapers came to exist in their present state as well as the role they play in society. This knowledge, combined with information about reading behavior is also important for understanding the motivations for the ENOF design. PRINTED NEWSPAPERS The first American newspaper, the Boston News-Letter , began weekly publication in 1704. Four-page daily newspapers with circulations in the hundreds first appeared in the 1780s. The invention of the steam press; the automatic typesetting machine; fast-drying inks; and machines to cut, fold, and bundle papers increased the size and circulation of newspapers during the nineteenth century. The nineteenth century also saw the transformation of the newspaper into a form of mass media bib:Davison . Most newspapers today are divided into several sections of twelve or more pages each. As of February 1, 1991, there were 1,611 daily newspapers published in the United States. The top 100 of these newspapers had average circulations ranging from 117,881 to 1,857,131 over the 6-month period ending September 30, 1990. Total daily newspaper circulation in the United States during this period was 62,327,962; about two-thirds of American households bought newspapers daily bib:World . DESIGN Today, newspaper design is usually based on a combination of aesthetics, tradition, design studies, advertising space requirements, and the speed with which a human designer can place the available stories on a page. In the past, available typesetting technology had a major impact on newspaper design, however, the widespread use of computers in the typesetting process has almost completely eliminated this factor. The current newspaper design trend, inspired by USA Today bib:Newspaper features small rectangular blocks of text and extensive use of icons, informational graphics, and colored boxes. USA Today has used this style successfully to appeal to readers who want quick access to news. Most newspapers known for indepth reporting and news analysis have more traditional designs with fewer graphical elements. READING HABITS Most people spend about 24 minutes each day reading a newspaper bib:Newspaper . People usually read newspapers to obtain information; however, their motivation for obtaining information varies. People seek information for both tangible and intangible reasons bib:Dertouzos . Some seek information to help make decisions, to have something to discuss in social settings, or for the sheer pleasure of increasing their knowledge bib:Atkin . Stephenson bib:Stephenson suggests that newspaper reading is usually voluntary and, like a game, satisfying in and of itself. The gathering of information can be thought of as a combination of three methods: grazing, browsing, and hunting bib:Zerbinos . Grazing is a passive activity, in which a person absorbs information with little discrimination. In browsing, readers scan large amounts of information without a particular purpose. In hunting, people search for specific information. Television and novels lend themselves well to grazing, while newspapers lend themselves to browsing. Hunting is most easily achieved using information sources with keyword indexes or search facilities. Unlike books, newspapers are seldom read in order from cover to cover. People tend to browse when reading newspapers, reading articles that catch their eyes and sometimes ignoring whole sections that they know from experience are usually not interesting. Sometimes, however, people hunt through newspapers for specific information bib:Graber,bib:Zerbinos . There are several theories about how people decide what to read and what to ignore bib:Davison . The consistency theory suggests that people select information that is consistent with their existing beliefs about the world and ignore information that seems dissonant. The utility theory states that people try to expose themselves to information that is pleasurable or that will help to satisfy their needs or interests. Other theories do not assume that people always actively select what to read. Some theories suggest, for example, that people pay attention to the communications that are easiest to access and understand. Davison et al. state bib:Davison : ``Information from these communications is learned simply because it is accessible and because there is no particular reason not to learn it.'' In a study conducted by Graber bib:Graber participants ignored on average 67 percent of all stories in a daily newspaper. Of the stories that were noticed, 18 percent were read completely. Graber attributes this partial reading to the inverted pyramid style of news stories, in which the most important information is presented in the first few paragraphs. ``Readers know they can glean the essence of a story without going to the trouble of reading all of it,'' Graber explains. Graber also notes that participants were more likely to notice stories that were cued by large headlines, a prominent position, or frequent mention; contained keywords of interest to them; or were on topics that ``had become the focus of attention for conversation among friends or associates.'' ELECTRONIC NEWSPAPERS The first paperless electronic newspapers were developed in the late seventies. Early commercial electronic newspapers, including Times Mirror's Gateway and Knight-Ridder's Viewtron, were marketed towards home consumers bib:Zerbinos . These electronic newspapers, which simply scrolled selected news stories across subscribers' computer monitors or television screens, introduced a new method for text delivery but did not expand on printed newspapers' functions. They were not commercially successful. The participants in a 1988 American Press Institute seminar imagined electronic newspapers published in parallel with printed newspapers bib:Newspaper . Some envisioned newspapers delivered to home fax machines, while others imagined news outlets for plugging in portable newspaper computers (similar to today's portable pen-based computers). Most predicted electronic newspaper formats with the look of printed newspapers. As with the early electronic newspapers, these hypothetical newspapers focussed on changing the delivery method rather than changing the contents or function of the daily newspaper. Currently, there are several commercial and experimental electronic newspapers in existence. Most of these electronic newspapers are available to computer users over phone lines or through computer networks and consist mostly of news stories gathered from other media. Some allow users to preselect the type of news they want delivered. Compuserve, Prodigy, ClariNet, and Individual, are among the companies currently selling electronic newspaper products. Three advantages of electronic newspapers over printed newspapers are: they can be delivered continuously as news is reported, they can be customized, and they can incorporate a wide variety of media formats. These advantages are discussed below. CONTINUOUS DELIVERY Printed newspapers are economical only if mass produced; it is not financially feasible to print ``extra'' editions with late-breaking news. By eliminating the necessity of physically reproducing a printed document, electronic newspapers, like broadcast media, are able to provide news as it is reported. However, unlike broadcast media which must deliver news according to a set schedule, electronic newspapers can deliver specific news stories whenever consumers request them. CUSTOMIZATION Perhaps the most unique aspect of electronic newspapers is their ability to be customized. Because they need not be mass produced, electronic newspapers can be personalized to include news of particular interest to each reader. This customization can be done automatically by the newspaper service or left to the reader. In the former case, a set of news stories, selected using a predefined list of each subscriber's interests, is delivered to subscribers each time they access the newspaper. In the latter case, subscribers request stories by selecting from a menu of headlines or by entering keywords or phrases. Researchers are exploring a variety of information filtering methods that can be used to automatically generate personalized newspapers and journals that contain only news of interest to each subscriber bib:Foltz . Commercial electronic newspapers currently use imperfect filtering methods. These methods involve searching for keywords, keyphrases, or combinations of weighted keywords bib:Wiegner ; and searching for articles similar to those the user has requested in the past bib:Fischer . John bib:John of Southwestern Bell Technology Resources is developing an electronic newspaper using adaptive neural networks. Articles are grouped by category based on the scripts (groups of adjacent words) they contain. Categories are selected for inclusion in a newspaper according to user preferences. The program learns individuals' preferences through a simple supervised learning procedure. Through informal user tests, John has observed that people's reading habits are inconsistent. However, he has found that the filter becomes stable in the long run. In addition, a test which compared the neural net's filtering with human editors' filtering, demonstrated that the neural nets had about 80 percent accuracy. FORMAT One of the biggest problems with most commercial electronic newspapers is their user interfaces. Most require readers to access news in a manner that is complex, inconvenient, or unfamiliar. As a result, reading news online can often be too difficult or frustrating to be enjoyable. Most electronic newspapers are accessed through a series of menus. However, research shows that menu-driven systems tend not to facilitate browsing, which is one of the three main ways people gather information bib:Zerbinos . Browsing is difficult when small quantities of text are displayed at a time and when the text displayed is a uniform size and typeface. An experiment conducted by Richardson et al. bib:McKnight found that readers clearly preferred large displays over small displays when searching for information in a text, although the display size did not affect their success in finding particular information. Some hypertext information retrieval systems attempt to facilitate browsing by providing a well-defined and consistent hierarchical structure and short machine response time bib:McKnight . Experiments have shown this to be somewhat effective. Current technology limits the use of rapid hypertext systems over computer networks and ``paying on-line time charges...lessens the likelihood of browsing through information which may seem superfluous to one's life,'' bib:Zerbinos . Some experimental electronic newspapers are designed to look like traditional newspapers. A European consortium is developing an Individualized Electronic Newspaper (IEN) bib:Huser , which is graphically composed for each subscriber based on his or her profile. The IEN is delivered in two forms, a hypermedia form and a printable form, both of which are laid out like a traditional printed newspaper. It includes timely articles, images, video, advertisements, access to background material, and access to databases of classified advertisements. The IEN is based on a publishing model that ``combine[s] structured document manipulation tools and knowledge-based systems'' bib:WeberNeuhold . It is designed to be delivered over a broadband network to users with personal computers running hypertext systems. However, the composition and delivery speed for IEN are unknown because the system has yet to be completely implemented bib:Weber . A format similar to a traditional printed newspaper was chosen for IEN because it is familiar to users bib:Weber . IEN researcher Weber explained, ``We think the layout chosen can serve to remedy disorientation and lost in hyperspace problems of hypertext systems. Others have also proposed to learn from paper documents in order to improve the presentation and navigation in hyperdocuments.'' The MIT Media Laboratory's Newspace project displays both traditional and non-traditional news sources using a traditional-looking broadsheet-sized front page metaphor bib:Bender . The Media Lab's Electronic Publishing Group chose this format because of its familiarity and standardization. In addition, citing the 1988 American Press Institute seminar, they note, ``Without exception, industry predictions of the future of the newspaper include a front page,'' However, it is uncertain whether all seminar participants would have predicted a front page if they had not been specifically asked to bring a front page prediction to the seminar. Newspace is among the experimental electronic newspapers that combine elements of traditional newspapers with short video clips typical of television news. The creators of these electronic newspapers think of an electronic newspaper as a medium for conveying news in the manner best suited for each story. However, the bandwidth of digital communications channels restricts the widespread use of video in electronic newspapers. As changes in network technology and the telecommunications infrastructure occur, the technological restrictions on electronic newspaper formats will disappear bib:Hoffert . Newspace is being developed with the assumption that it will be used with paper quality displays and computational access to news data streams. However, few 20 by 20 inch (2048 X 2048 pixels) color CRT displays like the one used by the Media Lab currently exist. In addition, Newspace users are assumed to live and work in an environment in which they can be continuously tracked by a computer. Newspace builds personalized newspapers from a variety of sources including text news services, personal electronic mail files, graphics data bases, and television news bib:Bender,bib:Lie . Another experimental electronic newspaper was developed by Apple Computer for the EDUCOM 1990 conference in Atlanta, Georgia bib:Hoffert . The system combined news wire text, postscript graphics, and Cable News Network video footage in a Macintosh hypertext system. Each day's news was compiled at a production studio, assembled over night, and delivered between 5 and 7 a.m. via a T-1 line (a cable which transmits data at a rate of 1.544 Megabytes per second A standard 3.5 inch floppy disk can hold 1.44 Megabytes of data. ) to a local area network (LAN) at the main conference hotel. About 35 Megabytes of news was compiled each day. It took about 10 minutes to deliver the news to each group of Macintosh computers on the LAN. Following the conference, the system's designers concluded, ``hypertext features were useful for giving more information than what appears on the surface; however, the ability to add a large number of hypertext links and cross-references on a tight production schedule should be perfected'' bib:Hoffert . They listed as possible future extensions to the system news updates throughout the day and a file server from which users could access the previous day's or week's news. However, without hardware changes these extensions could be problematic and cause network overload. 2.3 SUMMARY This chapter has defined terms used in this thesis and surveyed the development of printed and electronic newspapers. The most important points discussed in this chapter follow. Printed newspapers have evolved over the past three centuries. The format of printed newspapers has changed as new technologies have developed. People usually read newspapers to obtain information, however, their motivation for obtaining information varies. People seek information for both tangible and intangible reasons. Most newspaper readers read less than half the articles in each newspaper. Early electronic newspapers, which simply scrolled selected news stories across subscribers' computer monitors or television screens, were not commercially successful. Three advantages of electronic newspapers over printed newspapers are: they can be delivered continuously as news is reported, they can be customized, and they can incorporate a wide variety of media formats. There are a variety of commercial and experimental electronic newspapers in existence. CHAPTER 3. ENOF DESIGN The ENOF design is based, in part, on three assumptions formulated after a literature review. The preferences of a target audience and other considerations were also taken into account during the design process. This chapter describes the three assumptions, details the habits and preferences of the target audience, and explains the rationale behind the design decisions that were made in the course of the project. 3.1 ASSUMPTIONS The following assumptions provide a partial basis for the ENOF design decisions. Furthermore, they allow many of the problems currently under investigation by other researchers to be ignored. These include issues such as developing the capabilities to conduct extremely precise searches, learning subscribers' changing interests, and automatically designing newspaper-like pages. Assumption: Interests are dynamic, difficult to specify, and best judged by each individual. Each person is the best judge of whether something is personally interesting. Most people cannot precisely specify their interests in a way that makes sense to others. Even if they can precisely specify their interests, such a specification is likely to require frequent updating to remain accurate. This assumption is particularly applicable to the assessment of general news interests. While interests in technical literature or business news are often static and easily specifiable, general news interests tend to vary greatly and depend on a variety of interacting and changing factors bib:Allen,bib:Foltz . Such factors include the amount of time a person has to read, listen to, or view news on a given day; what news stories he or she has heard friends and colleagues discuss; and the amount of available news that appears to be immediately relevant. In addition, some people may not want a computer to precisely select articles for them. They may desire the perceived freedom of choosing what to read, or they may feel self-conscious about which articles the computer selects for them bib:Allen . Furthermore, people may want to browse through articles that do not perfectly match their interests. This assumption indicates that a precise filtering technique may not be achievable or desirable for news selection. Therefore, the ENOF design includes only a coarse filter that uses a user-defined profile consisting of news category (specified by the news provider) and keywords. More precise filtering becomes the job of the user. Assumption: People can quickly filter information to eliminate that which they find least interesting. People do not have time to read everything that they find interesting. Therefore they routinely prioritize reading material and select a subset of the information presented bib:Davison . Most people quickly filter information by looking at titles, authors, formats, and other cues bib:Graber . This assumption indicates that the user will be able do precise filtering. ENOF is designed to assist the user with this task by presenting users with headline menus for each section of their newspapers. The headline menus are designed to allow users to quickly select articles for reading. Assumption: The traditional printed newspaper format is not the ideal format for an electronic newspaper. The printed newspaper has been designed based on a variety of factors, many of which are not related to making it easy to read. Printing technology, advertising, and tradition have dictated many elements of newspaper design. Furthermore, traditional newspapers have been designed to fit on a broadsheet or tabloid page, not a computer screen. This assumption indicates that it may be better to design an electronic newspaper that takes advantage of the unique aspects of the electronic format rather that trying to copy the traditional print format. Therefore, ENOF is designed for readability and ease of use on a computer screen. While the newspaper metaphor is used for naming parts of the ENOF system, ENOF has not been designed to look like a printed newspaper. 3.2 HABITS AND PREFERENCES OF TARGET AUDIENCE In order to narrow the scope of this project, a target audience for ENOF was selected. The target audience is composed of people who work full time in a university or research setting and have some familiarity with software applications that run in an X Window System environment. Members of the target audience are assumed to have sufficient computer skills to learn how to use an application program with a well-designed user interface quickly. In addition, they are assumed to be busy people who do not have time to experiment with software that does not appear to meet their needs. This target audience was selected for several reasons. First, it was easy to find volunteers from this group who have access to the necessary computer equipment to run the ENOF software. Second, members of this group have enough computer knowledge that they should be able to learn to use an applications program with little training. Third, members of this group tend to have busy, but fairly regular schedules, and thus have somewhat consistent news media habits and limited time for media exposure. A survey (see Appendix A) was conducted to determine the media habits and preferences of the target audience and to identify participants for the evaluation phase of this study. Surveys were delivered to 30 full time Washington University faculty and and staff members who have computers on their desk at work. Responses were received from three females and 15 males. Respondents included nine computer science or medical school professors, five research associates, one secretary, and three people who did not disclose their job titles. All respondents who supplied work-related information said they usually spend 35 or more hours per week at work and five or more hours per week doing job-related work at home. Many respondents said they work a total of 60 or more hours per week. NEWS MEDIA HABITS OF TARGET AUDIENCE Survey participants were asked to list their favorite news sources and explain what they liked about each source. The most popular news source was the newspaper, with 15 out of 18 respondents listing it among their two favorite news sources. Radio and news magazines were favored by seven and five respondents respectively. No respondents said they preferred television or electronic news services. Reasons for preferring newspapers and news magazines included browsability, depth and breadth of news, portability, ability to be read any time, and lack of commercials. Those who said they preferred radio explained that they liked to listen to it while doing other things such as driving, and that it did not take up as much of their time as a newspaper. In addition, all respondents listed newspaper, news magazine, or radio as the most convenient form of news media for them. These responses are summarized in Table 1. [Table 1 -- not shown] The primary reason most respondents cited for using news media was to stay informed about events in the world, either to ``feel up to date'' or to satisfy their curiosity. Other popular reasons included gaining information needed to make decisions and entertainment. Most respondents cited more than one primary reason for using news media. Only half the respondents said that advertising was among their secondary reasons for using news media, and most of those said they only look for advertisements when they are planning on making a major purchase. The majority of those who responded said they use two or more forms of news media every day. While five respondents reported rarely or never watching television and three reported rarely or never listening to the radio, all said they read a newspaper at least once a week. However 15 out of 18 respondents said they read less than half the text of each newspaper. About half the respondents said they usually do their daily newspaper reading in one sitting. Most of the respondent who said they listen to radio news reported listening mostly while driving to and from work. About half the respondents said they currently read on-line work-related news, and several others said they used to read news online. PREFERENCES OF TARGET AUDIENCE In order to develop an electronic newspaper that improves upon existing media, survey participants were asked what they would most like to change about currently available news media. Respondents said they would prefer less advertising, better indexing of print media, news more related to their interests and delivered when they want it, more indepth analysis and references to background information, newspaper stories that stay on one page, and multimedia news. The survey asked several general questions about how participants might use personalized electronic newspapers. Some questions were left purposely vague so as not to bias the answers and to generate a wide variety of responses. Most respondents said they would be likely to use a personalized electronic newspaper if it were available to them. However, they said it would probably not be a substitute for other news media. Respondents said they would be much more likely to use an electronic newspaper at home than at work. Furthermore, several said they would be more likely to use an electronic newspaper it it was portable and could read the news aloud. One respondent said he would like his electronic newspaper to be water proof so he could read it in the bathtub. A list of desirable electronic newspaper features was compiled from the survey responses. Eight of these features, all of which appeared feasible to implement using available resources and within the time allotted for the project, are listed below. The list includes: ability to present all available articles to subscribers in a format that subscribers can quickly scan and and make selections from. ability to save and print articles. articles grouped by topic. browsability. filter to select news articles according to subscriber specifications. news specific to professional interests. searching capabilities. subscriber-specified limits on quantity of information on each topic. Many suggested features were not deemed feasible to implement at this time. Developing a multimedia electronic newspaper that was portable, waterproof, able to recognize voice commands, or able to speak to the user, was not possible using available resources. Furthermore, current news photographs and video footage are not yet available over the Internet. Also, implementing hypertext links to background material and news summaries with easy access to longer versions of articles, and development of a natural language interface to answer specific questions about the news and rank articles in order of importance is beyond the scope of this project. 3.3 DESIGN OBJECTIVES Information gained by examining the main assumptions and user preferences was synthesized to generate nine design objectives. These objectives were kept in mind throughout the development and evaluation phases of this project. ENOF has been designed to: allow subscribers to access news of professional interest in addition to general news. allow subscribers to browse comfortably. allow subscribers to search the archive for articles on a particular topic. allow subscribers to select the quantity of news to read. be easy to use. filter news precisely enough that subscribers find the service useful, but not so precisely that subscribers miss important stories or are not able to browse. present headlines and articles in a format that allows users to quickly select articles for reading. present headlines and articles in a format that is easy to read on most available X Window Systems displays. CHAPTER 4. IMPLEMENTATION The ENOF system was written in C++ and designed to run in a Unix environment. This project was implemented on a Sun SPARCsystem 400. ENOF was built on standard USENET software and includes an indexed archive, a filter, and an X Window System based user interface. Figure 4.1 shows how ENOF processes new articles. The filter and indexer are programs that automatically run at scheduled times to query the news server for new articles. The filter reads the users' .binx files, which contain specifications for the content of each user's newspaper. The filter then examines each story to determine which users' newspapers to include it in. The title of each story is written into the appropriate users' .bin files. The indexer creates index entries for the new stories so that they can be easily retrieved from the archive. [Figure 4.1 -- not shown] This system was designed as a prototype electronic newspaper for 15 subscribers. Efficiency considerations, essential to the development of a large system, were ignored. The prototype system scales linearly in proportion to the number of subscribers and the number of articles received each day. As the quantity of news and the number of subscribers increase, the amount of time required to filter the articles received in a given time interval will approach (and eventually exceed) the time contained in that interval. For the ENOF system to run properly with a large number of subscribers, the system must be made more efficient, run on an extremely fast computer, or run on a distributed system in which the filtering is done by local workstations. 4.1 ARCHIVE Stories are archived using Thinking Machines Corporation's Wide Area Information Server (WAIS) serial indexer program bib:Kahle . The program creates an inverted file index that can be used to perform searches quickly. The index can be continuously expanded, however, entries cannot be removed without rebuilding the entire index. The WAIS search mechanism uses a weighted search technique which assigns numeric values (weights) to the articles in the database that contain one or more of the search terms. The weight of an article is determined by the number of times the search terms appear in that article. All articles containing search terms proceeded by the word ``not'' have their weight reduced to zero. The articles assigned the highest weights are most likely to match the search request. Four months of wire service news articles were archived in the course of this project. The articles were received and stored in USENET format (the format used by a large electronic bulletin board system accessible through international computer networks) from ClariNet Communications Corporation bib:Templeton . 4.2 FILTER ClariNet ClariNet is a registered trademark of ClariNet Communications Corporation. classifies each of its articles into one or more ``ClariNet groups.'' A set of ``standard groups,'' each containing one or more ClariNet groups, has been defined for ENOF. ENOF subscribers can define ``personal groups'' which contain one or more standard groups. In addition, a personal group may be modified by key words which articles must contain in order to be a member of that group. Figure 4.2 shows how a user's newspaper may be structured. ENOF subscribers define newspaper sections, each containing one or more standard or personal groups. Section and personal group definitions for each user are saved into .bins files. These .bins files are then automatically converted into the .binx files read by the filter. The filter is composed of two parts: a group filter and a keyword filter. The group filter checks each new article against the .binx files and determines which users have requested articles from one or more of the groups which the new article belongs to. The keyword filter then determines whether the new article contains the key words necessary to belong to users' personal groups. Pointers to articles which pass through both filters successfully are written into the appropriate users' .bin files. [Figure 4.2 -- not shown] 4.3 USER INTERFACE The user interface was written using Devguide bib:Sun , a tool for developing XView applications. It consists of a main window, an article window, a subscribe window, an all groups window, and several auxiliary windows. The windows can all be independently repositioned and resized. The ENOF main window is shown in Figure 4.3. This window contains buttons, a scrollable list of sections, and a scrollable list of articles. The left-most button is the Subscribe button. Clicking this button with a mouse opens a menu that allows subscribers to add, remove, and define sections. The Options button opens a menu which allows subscribers to change the article font, specify article expiration conditions, and expire groups of articles. The Search button, allows subscribers to search the archive for articles of interest. Clicking the Help button opens a window containing helpful information about using ENOF. The Exit button, asks the user for confirmation and then exits the program. [Figure 4.3 -- not shown] The top scrollable list contains the subscriber's section names. Next to each name is a number that indicates the number of articles currently in that section. Users can display the headline list for a particular section by clicking on that section name. The headline list appears in the bottom scrollable list. In addition to displaying each article's headline, the headline list also displays a status symbol, the time and date that the article was sent by the wire service (converted to local time), and the article's weight. The status symbol ``r'' indicates that an article has been read. The symbol ``d'' indicates that an article will be deleted. The symbol ``N'' indicates that an article is new and unread. A space in the status symbol column indicates that the article is not new, but it still has not been read. By clicking one of the Sort By buttons in the middle of the main window, users can select whether articles are sorted by date or weight. Clicking on an article's headline selects that article for reading and opens an Article window, as shown in Figure 4.4. The article window contains three buttons, two check boxes, a headline, and the text of the article. The left-most button, the Done button, dismisses the article window. The middle button, the File button, opens a menu that allows subscribers to save, print, or mail articles. The right-most button was designed to search for words within the article being displayed, however it was never implemented. The check boxes indicate the status the article will have once it is dismissed. [Figure 4.4 -- not shown] Users subscribe to ENOF by first adding sections and then defining them. Selecting Subscribe... from the Subscribe menu opens the Subscribe window and the All Groups window. The Subscribe window, shown in Figure 4.5, is used to define both sections and groups. In Figure 4.5, it is shown defining the electronic personal group. The All Groups window, shown in Figure 4.6, allows subscribers to select standard and personal groups from scrollable lists. [Figure 4.5 -- not shown] [Figure 4.6 -- not shown] The user interface also includes several auxiliary windows. These windows report user errors, request confirmation, or request information. CHAPTER 5. EVALUATION 5.1 METHODOLOGY Few studies have been performed to gauge the effectiveness of and user response to information retrieval systems. Furthermore, there is little theoretical support for the methods used to evaluate user response to information retrieval systems bib:Allen . This is due to the inherent difficulty of defining and measuring all relevant variables, establishing the appropriate relationships between causes and effects, and studying experimental controls. The most common methods for collecting data for the evaluation of an information retrieval system are questioning and observing the system's users and examining the system's records. Quantitative studies conducted using these methods tend to have low validity bib:Pao . While only a few guidelines exist for measuring the effectiveness of information retrieval systems, even less has been written specifically about how to evaluate information filtering systems. The evaluation methodology in this thesis is based loosely on evaluation methods used in other information filtering studies as well as guidelines for evaluating information retrieval systems bib:Pao,bib:Stadnyk . Qualitative evaluation methods modeled after social science measurement were also used. Unlike most quantitative studies, qualitative studies do not require the researcher to formulate hypotheses prior to data collection. Evaluator Michael Quinn Patton bib:Patton explains, ``The strategy in qualitative designs is to allow the important dimensions to emerge from analysis of the cases under study without presupposing in advance what those important dimensions will be.'' MEDIA DIARIES Six volunteers participated in the evaluation phase of this project. These volunteers were selected from among the respondents to the media survey described in Chapter 3. The volunteers included three research associates, two professors, and a secretary. Prior to evaluating ENOF, the volunteers were given blank diary forms (shown in Appendix B) and asked to record their media usage for two weeks. Each volunteer's forms were analyzed for media usage patterns. Volunteers were then given short tutorials on how to use the ENOF system. They were asked to continue their diaries for an additional two weeks, while periodically using ENOF. The diaries were analyzed for changes in the volunteers' media usage patterns. INTERVIEWS The six volunteers were interviewed at the conclusion of the evaluation period. Each volunteer was asked 15 standard questions (shown in Appendix C), in addition to questions specific to his or her media usage patterns. The questions were not always asked in the same order and wording. Additional follow-up questions were asked where appropriate. Informal interviews were also conducted with several other Washington University students and staff members who tested ENOF. USER LOGS In order to monitor the way subscribers used the ENOF interface, the program recorded subscriber actions in a log file for each subscriber. The log file contained the date and time that each action took place. It also included the number of articles and sections present at the start and end of each ENOF session. A log for one ENOF session is shown in Figure 5.1. [Figure 5.1 -- not shown] 5.2 RESULTS The media diaries, interviews, and user logs were analyzed and are summarized below. The first subsection contains results from the six volunteer evaluators. The next subsection summarizes the evaluation data and informal interviews with other testers. RESULTS FROM EACH VOLUNTEER The information collected from each volunteer is summarized below. Volunteer 1 had very regular media habits. Every weekday he watched television traffic and weather reports from 5:50 to 6:00 a.m. He then listened to news, music, and weather on the radio while driving to work. Upon arriving at work, he spent about 15 minutes each day reading USENET news relevant to his work. When he returned home in the evening, he read the St. Louis Post-Dispatch from cover to cover. He watched television news while preparing and eating dinner. After dinner he sometimes read a magazine from cover to cover. On weekends he watched CNN for about four hours each day while doing chores. He also read the newspaper. He watched a television news program Sunday evenings. Volunteer 1 did not spend very much time reading ENOF. He was most interested in using ENOF to find important work-related articles. He said he would consider paying for a service similar to ENOF if it provided access to USENET news groups. However, he said he would not use such a service as a substitute for a daily newspaper, radio, or television. Volunteer 2 spent about half an hour reading the New York Times almost every morning. She said she reads the New York Times rather than the local newspaper because the quality of writing is higher. Prior to subscribing to ENOF, the newspaper was her only regular source of news. Volunteer 2 read ENOF for five to 30 minutes almost every work day. Overall, she said she liked the format and functionality of the ENOF system. Here biggest complaint was the low quality of writing in ENOF's wire service articles. She said she would consider paying for a service similar to ENOF rather than buying a daily newspaper if the service could be accessed from her home and contained news articles from the New York Times . Other improvements she suggested were the inclusion of photographs and maps, a bird's eye view of the entire newspaper, large headlines, and some minor changes to the user interface. Volunteer 2 said her normal news reading method consists mostly of hunting, with some browsing of headlines. She said it was easier to browse ENOF headlines than printed newspaper headlines. She liked the ability to define custom sections that let ENOF do the hunting for her. In addition, because ENOF displays only one article at a time, she found focusing on her reading easier with ENOF than with a printed newspaper. Volunteer 3 listened to National Public Radio news each day while getting ready to go to work, driving to work, and driving home from work. She often read the St.~Louis Post-Dispatch , but at different times each day. On some days she also watched the news on television. She was busier than usual during the ENOF evaluation period, and was unable to read a newspaper regularly. However, she said she was able to stay up to date by reading ENOF for about 30 minutes every other work day. Volunteer 3 said she liked the format and timeliness of ENOF news. She said she would consider paying for a service similar to ENOF rather than buying a daily newspaper if the service contained local columns. Other improvements she suggested were the inclusion of photographs and more weather information. She liked the fact that there was no paper to throw away when she finished reading ENOF. In addition, she said she found focusing on her reading easier with ENOF than with a printed newspaper. Volunteer 4 watched a television news and weather report every weekday morning. He then listened to National Public Radio while getting ready to go to work and driving to work. He watched news and sports on television in the evening. Some evenings he also read USENET electronic news. After subscribing to ENOF, he read ENOF news for five to 10 minutes every other work day. While he liked the ENOF user interface, Volunteer 4 said he was not interested in the type of news articles ENOF contains. He said he would consider paying for a service similar to ENOF rather than buying a daily newspaper if the service contained the news found in his local newspaper. In addition, he said he did not spend enough time exploring ENOF to learn how to use it completely. However, if he could use ENOF to read the articles he was interested in (especially college sports articles), he said it would be worth taking the time to learn how to use the interface. Other improvements suggested by volunteer 4 were the inclusion of images and video, a pen based interface, and a format that resembles a printed newspaper. He said he likes the fact that printed newspapers use icons, type size, and format to indicate the importance of stories. He said that electronic newspapers should try to emulate printed newspapers until people get used to the electronic format. Then, he said, electronic newspapers might develop new formats that give the same visual cues but do not necessarily look like printed newspapers. He also suggested adding the ability for subscribers to create archival editions containing their favorite stories. Volunteer 5 had irregular media habits. He often listened to books on tape while driving to and from work, but sometimes he listened to National Public Radio instead. He sometimes watched television news programs in the evening. He also read a variety of newspapers, magazines, and books in the evening, often in the bath tub. After subscribing to ENOF, he read it briefly every other work day. Volunteer 5 said he found ENOF most useful for finding specific news such as game scores, stock prices, or a particular news article. He especially liked the idea of receiving news on demand. He said he would consider paying for a service similar to ENOF rather than buying a daily newspaper if ENOF could be run on a portable computer. Volunteer 5 described himself as a browser. However, he felt the ENOF system lends itself best to hunting for information. He said that font cues, photographs, and magazine-style layouts were necessary to facilitate browsing. Other improvements he suggested were the use of boolean modifiers in personal group definitions, hypertext layout, and the ability to select articles based on human editors' ratings. Volunteer 6 usually listened to National Public Radio while driving to work. He often read work-related USENET news groups or computer magazines for about half an hour during the day. He watched television news programs and read the St. Louis Post-Dispatch during the evening. After subscribing to ENOF, he read ENOF news every other work day. Volunteer 6 said, overall, that he liked the ENOF system. He said he would consider paying for a service similar to ENOF rather than buying a daily newspaper if it was portable and contained more local news. He especially liked the fact that he could save and file articles of interest without having to physically cut them out of a printed newspaper. Although he said he was able to quickly scan ENOF headlines to decide what to read, volunteer 6 said ENOF headlines often were not informative enough. He said he would prefer a format more similar to a printed newspaper. In addition, he suggested that the system include several predefined sections. 5.3 SUMMARY The small sample size made any qualitative or quantitative assessment of trends impossible. However, after many months of reading, discussion with experimental subjects and members of the general population, and analysis of the limited data, the following hypotheses are suggested. These hypotheses should be tested in a larger study. The evaluators liked the fact that ENOF could be customized. Most evaluators felt they could fine tune their ENOF subscriptions to deliver the quantity and type of news they wanted. Most liked the fact that ENOF delivered some articles that they had not specifically requested. Some suggested that ENOF include the ability to request articles selected by a human editor. When asked whether ENOF best facilitates grazing, browsing, or hunting, most evaluators responded that it worked well for their favorite reading methods. A few evaluators said that the lack of visual cues and the fact that only one article was displayed at a time inhibited browsing somewhat. There was a definite difference in opinion among evaluators about whether an electronic newspaper should look like a printed newspaper. Some said the single article presentation format allowed them to focus on their reading, while others said it inhibited browsing. Some felt that visual cues could best be provided by copying the types of cues used in printed newspapers; others felt new types of visual cues should be developed. Most of the evaluators from the target audience said portability was an important feature for an electronic newspaper. However, the graduate students who tested the program (who tend to spend a lot of time at their computer terminals) said portability was less important. Most of the evaluators said the ability to display photographs and graphics is an important feature for an electronic newspaper to have. The type of graphics desired varied among evaluators. Some were interested in news or sports photographs, while others were interested in weather maps or comics. Most evaluators said the ENOF system lacked important types of news. Indepth New York Times style news articles, local news, local columns, certain nationally syndicated columns, local entertainment information and television guides, editorials, and detailed local weather reports were among the news evaluators said they wanted to read. Most evaluators reported some confusion when they first defined their ENOF sections. Some suggested a better help facility or improved subscribe interface would be useful. All felt the current system was not unreasonably difficult to learn how to use. Most evaluators said they would most likely use a service similar to ENOF as a substitute for a daily newspaper. Some said that if the service included audio and video they might use it as on demand radio or television. However, one evaluator said he considered television more of a group activity and not something he would use his computer for. Some evaluators said a service that could run on a portable computer with an audio interface would be useful for listening to news while driving. One evaluator, who said he does not like talking to answering machines, said he probably would not like communicating orally with a computer. The evaluators said they did not mind reading news on a computer screen. They liked the fact that there was no paper to throw away or recycle when they finished reading. CHAPTER 6. CONCLUSIONS AND FUTURE RESEARCH 6.1 CONCLUSIONS A working prototype Electronic Newspaper Of the Future was designed and implemented. Listed below are conclusions drawn from a qualitative user evaluation. ENOF's coarse filter seems appropriate for most members of the target audience. Evaluators liked having the ability to customize their news, but they also liked receiving some articles that they had not specifically requested. Most said they could customize ENOF to deliver the quantity and type of news they wanted. In addition, they said they were able to quickly scan article headlines and decide what to read. This conclusion is consistent with the first two assumptions formulated in the first phase of this project. It is unlikely that a single user interface will suit all electronic newspaper subscribers. Even the six volunteer ENOF evaluators had conflicting news reading preferences. Therefore, user interface development efforts should be aimed at developing a variety of compatible interfaces. While it may not be necessary for an electronic newspaper to closely resemble a printed newspaper, an electronic newspaper should contain some visual cues similar to those found in printed newspapers. Some evaluators said the use of differing fonts, icons, and layout would make browsing easier. In addition, some evaluators said they would prefer multiple articles to be displayed at a time. However, others said they liked the fact that ENOF displays only one article at a time. Some said it would not be necessary for electronic newspapers to resemble printed newspapers once people get used to reading news electronically. More research is necessary to determine which aspects of a printed newspaper are most important to emulate in an electronic newspaper. The more portable an electronic newspaper system is, the more subscribers it is likely to have. Most evaluators said they would be more likely to use an electronic newspaper at home than at work. Furthermore, some said they would be more likely to use an electronic newspaper if they could read it at the breakfast table, in a comfortable chair, while waiting in line, or wherever is most convenient. The addition of still and moving images to electronic newspapers is likely to increase their popularity. Most evaluators said they like to see photographs and maps when reading news articles. In addition, some said reading the comics is an important part of their newspaper reading routine. Some said the addition of short video clips would also help sell them on using an electronic newspaper. In order for an electronic newspaper to compete with printed newspapers, it must contain all the news contained in the printed newspapers it is competing with. Most evaluators said they would be hesitant to replace their daily printed newspapers with electronic newspapers unless they knew they could get local news, weather, and entertainment information, editorials, and indepth reports electronically. More work should be done to simplify or better document the subscribe component of the ENOF user interface. Most of the evaluators said that subscribing to ENOF was confusing, but they were able to figure it out. However, a less computer-literate target audience might have more trouble. The possibility of developing an audio electronic newspaper interface should be investigated. Many members of the target audience get most of their news by listening to the radio while driving. A portable electronic newspaper with an audio interface might serve as a form of on-demand radio. Problems associated with reading text on a computer screen are likely to become less important as people use computers more. Most of the evaluators said they have no preference about reading newspaper text on paper or on a computer screen. Most said they used to prefer paper, but have gotten so used to reading on a screen that they no longer have a preference. 6.2 FUTURE RESEARCH USER INTERFACE This study has shown that a variety of user interface options may be necessary to satisfy all electronic newspaper subscribers. Some incremental improvements to the ENOF interface, such as adding keyboard shortcuts, changing the default shape and location of windows, and making it easier to page through articles, should be attempted. Future user interface research should investigate the following questions: What visual cues used in printed newspapers should be emulated in electronic newspapers? What visual cues can be developed specifically for electronic newspapers? What is the best way of displaying multiple stories on a screen much smaller than a standard broadsheet newspaper page? What is the best way to display a miniature image of an entire electronic newspaper? How can the subscribe process be simplified? TEXT FORMATTING During the design and implementation phase of this project it became clear that electronic news formatting standards would make electronic newspaper development easier. Currently, human writers write stories in formats suitable for human editors, human page designers, or human broadcasters to process. For example, a new paragraph is often marked by two end-of-line characters in a row. A single end-of-line character usually represents an arbitrary line break, and can generally be ignored. However, an article that contains tabular information (as shown in Figure 6.1) might contain a single end-of-line character that represents the end of a line in the table. A human can quickly recognize a table, even if it is not explicitly labeled. Programming a computer to recognize every such text irregularity would be more difficult. To solve this problem, paragraph, table, date, keyword, and header formats should be standardized. [Figure 6.1 -- not shown] Furthermore, special formatting features that would facilitate electronic indexing, filtering, and layout could be added. For example, writers might include codes to indicate placement of hypertext links to previous articles. In addition, they might write the article in distinct sections that would allow a reader to request an article summary, background information, update information (news that did not appear in the last article on the topic), indepth news analysis, interesting quotes, or the entire article. Human editors might add article ratings that subscribers could use to help select which articles to read. As long as humans are involved in the reporting and editing process, it seems appropriate for them to continue to do tasks more easily done by humans then by computers. STILL AND MOVING IMAGES The ability to display still and moving images may be a major selling point of electronic newspapers in the future. Transmission speed, storage space, and the need for text tags bib:Bender in order to index images are currently the main limiting factors to the use of images in electronic newspapers. In addition to continuing to work on the above problems, researchers should determine the best way to incorporate images with text in an electronic newspaper. AUDIO INTERFACE Portable electronic newspapers have the potential of becoming not only a visual medium, but also an audio medium. The addition of an audio interface would allow them to provide radio on demand. People who currently listen to radio news while driving to work might ask their electronic newspapers to ``read'' them the news in the car. The idea of conversing with a computer about the day's news is somewhat foreign. The only experience most people have had talking to a computer involves voice mail and automated phone systems. However, many people have observed humans and computers conversing in science fiction movies and television shows. A variety of questions must be investigated before an audio interface of this type is developed. These questions include the following: Is current natural language processing technology sophisticated enough for humans to converse with a computer for this purpose? How can a menu of articles be represented in an audio format? Does current voice synthesis technology produce a voice that is pleasant to listen to for long periods of time? Do people like listening to newspaper articles read aloud, or must articles be written specifically for this purpose? Do people want to choose the stories they hear on the radio, or do they prefer listening to a packaged newscast? ADVERTISING Advertising may be important to the success of commercial electronic newspapers. In addition to reducing the costs to subscribers, the inclusion of advertising may also attract subscribers who want to read advertisements or use their computers to order products. Research should be done to determine the best way to format advertising to the satisfaction of the advertisers and the subscribers. COMMERCIAL PRODUCT An electronic news system similar to ENOF could potentially be commercially successful within a few years. The following is an outline of how ENOF could be implemented as a commercial electronic news service. The ENOF company buys specially-formatted electronic news, images, and advertisements from several wire services. As new articles arrive, the company archives them in a permanent collection and distributes them over the Internet to subscribers' computers. Subscribers' computers store and filter new articles as they are received. All filtering is done on subscribers' local computers using software provided by ENOF. The ENOF software also includes a news reader interface and an indexing program. Subscribers can save and index as much news as they have room for on their local computer systems. Subscribers normally read news directly on their local computers. However, when a subscriber requests an article that is no longer available on the local system, the ENOF program automatically connects to the ENOF archive and searches for the requested articles. Subscribers are charged on a monthly basis according to how many wire services they wish to receive. They can pay an additional monthly fee for access to the ENOF archive, or they can be charged a small fee for each article they down load from the archive. Discounts are given to groups of subscribers who receive their news on the same computer. The ENOF software includes a user interface similar to the current ENOF interface. It might also offer an alternative interface which more closely resembles a printed newspaper. Other electronic news providers might compete with ENOF by offering different interfaces, wire services, or payment structures. SUMMARY A working prototype Electronic Newspaper Of the Future (ENOF) was designed and implemented. The ENOF system allows subscribers to define custom newspapers containing personalized sections. ENOF uses a coarse filtering technique to select articles for inclusion in each subscriber's newspaper. The system was intended for a target audience of people who work full time in a university or research environment. ENOF was designed to filter news precisely enough that subscribers find the service useful, but not so precisely that subscribers miss important stories or are not able to browse. Subscriber preferences were assessed qualitatively through user diaries, logs, and interviews. The results indicated that ENOF's coarse filter is appropriate for most members of the target audience. The evaluation also revealed other conclusions that may help guide future research in this area. REFERENCES bib:Allen Allen, Robert B. 1990. User models: theory, method, and practice. Int. J. Man-Machine Studies. 32:511-543. bib:Atkin Atkin, Charles. 1973. Instrumental utilities and information-seeking. In P. Clarke (ed.), New Models for Communication Research. Beverly Hills: Sage Publications. bib:Battelle Battelle, John. 1993. The newspaper industry gets its first R lab. Wired. 1(1):20. bib:Bender Bender, Walter, Hakon Lie, Jonathan Orwant, Laura Teodosio, and Nathan Abramson. Newspace: mass media and personal computing. Paper presented at USENIX, Nashville, June 1991. bib:Blumler Blumler, G. 1980. Information overload: is there a problem? In E. Witte (ed.), Human Aspects of Telecommunication. New York: Springer-Verlag. bib:Davison Davison, W. Phillips, James Boylan, and Frederick T.C. Yu. 1982. Mass Media Systems and Effects , New York: Holt, Rinehart and Winston. bib:Dertouzos Dertouzos, Michael L. 1991. Communications, computers and networks. Scientific American . 265(3): 62-9. bib:Fischer Fischer, Gerhard and Curt Stevens. 1990. Information access in complex, poorly structured information spaces. University of Colorado at Boulder Technical Report CU-CS-461-90. bib:Foltz Foltz, Peter W. and Susan T. Dumais. 1992. Personalized information delivery: an analysis of information filtering methods. Communications of the ACM. 35(12): 51-60. bib:Graber Graber, Doris A. 1988. Processing the News: How People Tame the Information Tide. 2nd ed. New York: Longman. bib:Hoffert Hoffert, Eric M. and Greg Gretsch. 1991. The digital news system at EDUCOM: a convergence of interactive computing, newspapers, television and high-speed networks. Communications of the ACM. 34(4): 113-6. bib:Huser Huser, Christoph and Anja Weber. 1992. The Individualized Electronic Newspaper: An Application Challenging Hypertext Technology. Arbeitspapiere der GMD 664. Darmstadt: Gesellschaft Fur Mathematik Und Datenverbeitung MBH. bib:John John, Thomas. 1993. personal correspondence. (10 February). bib:Kahle Kahle, Brewster. 1989. Serial Indexer Overview. (documentation for Thinking Machines Corporation's Wide Area Information Server. bib:Katzman Katzman, Natan. 1974. The impact of communication technology: promises and prospects. Journal of Communication. 24: 47-58. bib:Lie Lie, Hakon and Walter Bender. 1991. The electronic broadsheet: all the news that fits the display. In C. Vanoirbeen and G. Coray (eds.), Electronic Publishing '92. bib:McKnight McKnight, Cliff, Andrew Dillon, and John Richardson. 1991. Hypertext in Context. Cambridge: Cambridge University Press. bib:Newspaper Newspaper Design 2000 and Beyond. 1988. Reston: American Press Institute. bib:Pao Pao, Miranda Lee. 1989. Uses and users. In Concepts of Information Retrieval. Englewood: Libraries Unlimited, Inc. pp.40-53. bib:Patton Patton, Michael Quinn. 1980 Qualitative Evaluation Methods. Beverly Hills: Sage Publications. bib:Pulp Pulp Paper. 1987. Newsprint: Sellers' market through 1989, as capacity catches up with consumption. (December): 13. bib:Spafford Spafford, Gene. 1991. USENET software: history and sources. Article posted to USENET group news.announce.newusers. bib:Stadnyk Stadnyk, Irene. 1993. personal correspondence. (25 January). irene@cmi.com. bib:Stephenson Stephenson, William. 1964. The ludenic theory of newsreading. Journalism Quarterly. 41: 367-74. bib:Sun Sun Microsystems, Inc. 1991. OpenWindows Developer's Guide: Programmer's Guide to the XView Toolkit Code Generator. Mountain View: SunSoft. bib:Templeton Templeton, Brad. 1992. ClariNet Electronic Newspaper Manual. Cupertino: ClariNet Communications Corp. bib:Weber Weber, Anja. 1992. personal correspondence. (23 March). weber@darmstadt.gmd.de. bib:WeberNeuhold Weber, Anja, and Eric J. Neuhold. 1991. Distributed publishing of electronic newspapers and mailorder catalogues. Arbeitspapiere der GMD 574. Darmstadt: Gesellschaft Fur Mathematik Und Datenverbeitung MBH. bib:Wiegner Wiegner, Kathleen K. 1990. All the news that fits. Forbes. (30 April): 174-5. bib:World The World Almanac and Book of Facts. 1992. New York: Newspaper Enterprise Association. bib:Zerbinos Zerbinos, Eugenia. 1986. Information Seeking and Information Processing in Two Media -- Print and Electronic Print. Doctoral dissertation, Michigan State University. ACKNOWLEDGMENTS I would like to thank my advisor, Dr. Mark Frisse, for his guidance throughout this project. Thanks are also due to the other members of my committee: Dr. Robert Morgan and Dr. Robert Benson. I am most indebted to Scott Hassan for writing the ENOF program, responding to my numerous design changes, following up on all my bug reports, and providing me with bagels and smiles. This project would not have been possible without his help. I would also like to thank all those who helped debug and evaluate the ENOF program: Chuck Cranor, Dr. Martin Dubetz, Dr. Anne Johnstone, Kay Komotos, Dr. Stan Kwasny, Dr. Ronald Loui, Keith Marrs, Paul Schoening, and Zubin Dittia. Thanks to Dr. John Schnase for proof reading this thesis and making suggestions for improvements. Thanks also to Chuck Cranor for proof reading and for providing me with the thesis style files. I am responsible for all errors which remain. Washington University's Center for Intelligent Computing Systems and Southwestern Bell Technology Resources, Inc provided generous support for this project. Thanks also to all those in the Informatics Laboratory and the Computer and Communications Research Center for letting me share their office space and computer resources. Finally, I would like to thank my parents, Drs. Michael and Judy Ackerman, and my fiance, Chuck Cranor, for all their encouragement and support. VITA Lorrie Faith Ackerman Education: Washington University, St. Louis, Missouri, B.S. in Engineering and Public Policy, May 1992 Washington University, St. Louis, Missouri, M.S. in Technology and Human Affairs, expected May 1993 Honors and Awards: Washington University Engineering and Policy Departmental Fellowship Washington University Langsdorf Finalist/Woodward Engineering Fellow Senior Award for Service to the Engineering and Policy Department and Professional Excellence National Merit Scholarship Finalist Missouri College Newspaper Association Award for 2nd Place in Feature Writing Columbia University's Gold Circle Award for 2nd Place in the Nation in Features Profiles Writing Scholastic and Professional Experience: IEEE Intern, Washington Internships for Students of Engineering, summer 1992 Editor, Student Life Newspaper, Washington University, 1989-1992 Health and Environmental Sciences Intern, American Petroleum Institute, summer 1991 Professional Societies: Association for Computing Machinery Computer Professionals for Social Responsibility Institute of Electrical and Electronics Engineers National Association of Science Writers Society of Professional Journalists Society of Women Engineers Electronic Mail Address: lorracks@vault.wustl.edu May 1993