Community-centered Development and Action Research Testing of Online Social Networking Prototype

There is an increasing array of information and communications technology (ICT) aimed at improving societal well-being. As Gurstein and Horan (2005) note, these community information systems (CIS) are designed and built for use by members of a community to support a host of different social, economic, and cultural goals, . This community can be based on a physical location, such as a system built for all the citizens in a particular town; or it can be virtual, such as a system built for everyone who likes a particular music group. When a community wishes to increase its members’ commitment and participation, many times a CIS is proposed as a possible part of the solution. However, previous research has shown that the use of the Internet does not necessarily have a positive effect on the social capital of a community. (Kraut et al., 2002 and Nie & Erbring, 2000) These communities are therefore presented with a dilemma: how do they approach the development and implementation of an online system for their community? What are the success factors for these systems? What design process should be used? Which features should be included in the system?

This paper first reviews the Information Systems Design Theory (ISDT) framework and then creates a framework for applying it to the development of a community information system. Finally, it presents the field testing of a real-life CIS through the lens of ISDT and gives specific, concrete design recommendations for community information systems design and development.

An Information Systems Design Theory is “a prescriptive theory based on theoretical underpinnings which says how a design process can be carried out in a way which is both effective and feasible” . It can be thought of as a complete package of guidance for designers facing particular sets of circumstances containing three interrelated elements: 1) a set of user requirements, 2) a set of system features, and 3) a set of principles deemed effective for guiding the process of development .

The purpose of an ISDT is to support the achievement of goals. Because of this, it is crucial that the desired goals for the system being designed are well understood. How these goals (i.e., user requirements) are best gathered and understood is part of the ISDT. Because a design theory must be subject to empirical refutation, prototype construction is generally an output of design theory research. The use of the prototype will be the ultimate test of whether the information system meets its goals.

There are two aspects of design theory, the design product (what are we designing?) and the design process (how do we design it?). The components of an ISDT and their relationships are shown in Figure 1.

When developing an ISDT, the first question that must be asked is “what is the goal of our system?” This will inform the specification for the design product and the selection of the kernel theories needed to complete the design theory. The general scope of this current study is aimed at improving ‘community’ through community informatics. In this context, the focus is on the sense of community experienced (in terms of social capital) within a defined place-bound physical community The intellectual rubric for the analysis is ‘community informatics’, . Community informatics represents a portfolio of approaches and tools for using information and communication technology to enhance communities, including but not limited to: creating telecentres to provide access for those who do not have it, providing community information via the web, providing online public services, providing education and training online, providing tools for community and regional planning, and so forth. As noted by Keeble and Loader (2001), a common driver for these improvements is community need, which can include economic, social and cultural goals. In this way civic goals may readily be translated into the drivers for community informatics systems.

Once the specific system goal(s) are identified, design theory then requires that we identify one or more “kernel theories” upon which the system is to be built. These kernel theories are the basis for the design of the system. In the sections below, we outline three cornerstone kernel theories—one related to the substance or essence specified by community development and two related to the community informatics processes for ensuring valued systems.

One of the most important needs of any community is social capital. Social capital, like its counterpart, physical capital, is understood as a way of improving the life of the members of the community. Specifically, it is the “connections between individuals—social networks and the norms of reciprocity and trustworthiness that arise from them” . Two categories of social capital are recognized: bonding social capital, which is the connections between members of the same community; and bridging social capital, which is the connections between members of different communities. It is bridging social capital which allows different communities to interact and is absolutely essential for a diverse well-functioning society incorporating multiple communities.

Social capital is transferred within social networks. A social network is described in the scholarly literature as a set of people (or organizations) connected by social relationships, such as family, friendship, who are co-working or involved in information exchange. When interconnected computers connect people, a social network is formed that can increase social capital .

The study of the impact of technology on social networks is well-documented. Initial studies such as the Homenet study and the Stanford Internet Study supported the idea that technology, specifically the Internet, led to a weakening of social networks. Further research, however, disputed the findings of these studies. Research done by the Pew Internet and American Life Project and later work by Kraut and colleagues suggests a much more differentiated impact of the internet on social participation. The emerging middle ground is that the impact of social networks is very much a function of both intended design and subsequent use of such networks .

In her book, Online Communities: Designing Usability and Supporting Sociability, Preece introduces the idea of “community-centered development” (CCD). CCD is a methodology that involves the members of the community in a participatory design process with the developers. It is an evolutionary process that allows for the system to change as the community changes. As Preece describes it, CCD has five distinct phases :

When developing an information system for a community, this Community-Centered Development approach can be used as a guideline for the design process.

During the spring of 2005, a community information systems prototype was developed at a small Los Angeles-area university. The community was comprised of the undergraduate commuter students at the university, which represented approximately one-third of the three thousand undergraduates registered for the 2004-2005 academic year. This community of students generally felt isolated due to the fact that a large majority of students are residents at this particular university. Working with the Director of Commuter Life and her team, the goals of the system were quickly identified: increase the social capital of the commuter students through the use of an online social networking system.

Using Preece’s Community Centered Development (CCD) methodology, the system designer met with members of the community to identify their specific needs. Working to design the system to meet the community’s goal of increased social capital, several features were identified that would be included in the system. These included the ability to see pictures of each other, track activities and classes, and to identify friends. Interestingly, the community members did not want a message board created. Once the system was developed and tested, it was rolled out to members of the community as a prototype. Seventy-eight students registered to participate as research subjects in the project, a smaller than hoped-for sample. These volunteers were randomly assigned to one of two groups: one group used the online system; a second group used a paper-based directory. Surveys were distributed before the system was rolled out and again six weeks later. The surveys were used to assess the impacts of the system with two questions each on the components of social capital as defined by Putnam: social networks, trust, mutual assistance, and norms of reciprocity. The principal investigator also interacted with the users during this time to understand how the system design affected the dynamics of community interaction.

This project was intentionally designed for study within the ISDT framework. By purposefully defining the goals of the system (increased social capital) and using kernel theories from the literature (social capital, CCD), a testable prototype was developed and the success of the system could be measured.

During the initial two weeks of the project, the usage of the system was extremely low. Inquiries by the principal researcher determined that this was due to the lack of a “critical mass” of users. The decision was made to go ahead and allow all students at the university to begin using the system. While this was not ideal, it was determined to be the course of action that would provide the best outcome to both the community and this research project. By expanding our user base, it was hoped that both user registrations and system use would increase.

To encourage these new users to participate, the principal investigator followed a framework laid out by Andrews, specifically to “build trust in the online community through building alliances with existing organizations” . This was done for commuters through an association with the Collegium, a physical location on campus set aside for commuter students. For these new potential users, the researcher used the following methods to build their trust and encourage use:

The actions had their intended effect: each built the user base and eventually increased overall system registration to 273 users. As can be seen below, each intervention (with the exception of the pizza party^¹) led to a noticeable increase in user registration and login activity. However, only one of the interventions led to a sustained increase in login activity: the scavenger hunt.

The plans for the scavenger hunt were simple: each night at midnight for five nights, a set of clues would be posted on the site using the information entered in the online system. The clues were written in such a way as to force the users to use all of the features of the system: user profiles, friends, communities, and classes. Users did not have to log in to the system in order to see the clues. Using these clues, a user would need to log in and deduce the correct answer, which they could then email to a drop box as their entry. Two winners were picked per day: the first correct answer submitted and one winner selected randomly from all the other correct answers. The full set of rules was posted on the site as follows:

Not only did the scavenger hunt increase system usage, it also encouraged users to explore all features in the system. This led to an increase in the number of users in the system (30%) and, more interestingly, a dramatic increase in the number of friends identified (74%)! During the week of the scavenger hunt, the number of friendship pairs created went from 345 to 600. This was the single largest increase in friend identification during the research period.

Once the six week research period ended, the follow-up survey was given to the original volunteers again; both to those using the system and to those using the paper directory. Eight of the survey questions were designed to measure the four components of social capital. In comparison to those who did not use the system, those who participated in the online social network showed positive changes in two of the four components (social network strength and mutual assistance). See the table below for a summary of the changes found.

+ = positive trend in social capital component shown in online group over paper group

The results from these surveys indicate that an online social network does impact social capital creation. The use of an online social network increased the size of social networks. The system allowed users to manage more friendships at a deeper level than those who did not use the system. It was also indicated that the use of the system improved trust and community involvement.

The online social network system allowed users to submit feedback when they encountered problems or had a suggestion. Analyzing this feedback gave the researcher the ability to understand how different product features were utilized. Sixty-one feedback messages were received during the research period of which forty-five were reporting system problems and sixteen were suggestions.

In conducting this research as an action research project, the researcher had the opportunity to interact with the community during the design and implementation of this system. This interaction allowed the principal investigator to observe firsthand, the impacts that the system was having. During the research period, many of the components of social capital were increased, though the small sample size hindered the ability to declare any direct impact by the system. More interestingly, this research provided insight into the design of community information systems for both the design process and the design product through the framework of information systems design theory. By using the framework of an ISDT in this research, the kernel theories used as part of the process were tested and new theories presented themselves.

This research project provides promising but limited evidence that community information systems can influence the experience of social capital within a community. However, this impact is not necessarily direct: only increases in some components of social capital were found. Instead, what became clear was the existence of a more nuanced impact on how social capital is formed using such systems. In other words, using this social networking system, elements of social capital were formed and strengthened between students, but the existence of the system alone did not fully account for this. Instead, the use of the system gave the students a channel to build the social capital that would have been built anyway.

In the terms of Information Systems Design Theory, then, the use of social capital as a kernel theory is supportable, as is the idea that a computer network is a social network through which social capital it transmitted. However, it should be understood that a CIS does not necessarily directly impact social capital but, instead, affects the dynamics of how it is created.

For example, the survey results showed that the users of the system tended to identify friends from the pool of other students in their same group (residents identified residents, commuters identified commuters). Interestingly, even though commuter students identified fewer friends than residents overall, the number of users who identified commuters was approximately the same. This system, then, did have a positive effect on the social networks of its users by exposing them to the larger community as a whole.

In order to drive up system usage, an online scavenger hunt was held for one week in April, 2005. This scavenger hunt was meant to be more than just a way to garner additional logins; it was meant to encourage users to utilize all aspects of the system. By using features in the product as part of the implementation process, the usage of the system went up dramatically. It also drove up user registrations. During the week of the scavenger hunt, the system usage logs showed a sustained increase in usage.

This has implications for the interaction of the design product and design process. By developing features in the system which enabled a game such as the scavenger hunt, the dynamics of the implementation process were affected. As developers work with community members to co-design community information systems, thought should be given to how activities that involve members with both the online system and the physical community can be implemented. This is supported by the work done by Casapulla et al. on the use of interactive game-like activities to encourage community participation. The key is to make the use of the system a habit for the users by giving them a reason to log in day after day. A new kernel theory presents itself here: the design process of community information systems should include steps to design features into the product that promote habitual use by their users.

Throughout the co-design process, the members of the community made it very clear that they did not want this new system to contain message boards. However, once the system was in place, the one feature that the users consistently asked for was… message boards. When asked in a survey during a pizza party, “What would be the best feature(s) to add to the system?” fifteen percent responded with messaging. Additionally, in tallying the comments and requests generated directly from the system by clicking on the “feedback” link, more than half of the suggestions for improvement were requesting a messaging feature.

What can we make of this? Did the Community-Centered Design (CCD) process that was used fail to recognize this as a requirement of the system? The answer may lay in the fact that it was only after the users began using the system that they understood their true desires. The CCD methodology acknowledges that this may happen by specifically creating a phase of the development process to “refine” the prototype:

However, in this case, the requirement culled out of the previous phases of the co-design process was to NOT have message boards! The users did a complete about-face on this requirement after seeing the system in place.

The answer may be that it is very difficult for users to conceptualize the need for a specific feature until that need is visualized in front of them through the use of a prototype. Another important factor to consider is that as a community information system matures, its purposes may change. Over time, the user base will require that the system change to meet new goals. As is the case here, the initial requirements to support the commuter community were met. As the system began to be used (and the user base broadened), the users began to identify new goals (such as communication) that the system now needed to meet.

This is supported by the work done by Bieger, et al, (2005) in their paper on stated versus revealed preferences. The implications they found for product development were that stated preferences (what the users say they want) don’t provide a satisfactory foundation for decisions. Instead, other approaches should be used to understand the true, or revealed preferences of the users. They recommend using a specific type of survey instrument to understand these revealed preferences.

To apply this back to the design of community information systems, it is important to create a mechanism for continued evolution of the system even after implementation. One key concept used as part of this project was to provide a place on every screen where the user could recommend changes to the system (or report errors). In order for this to be effective however, the user community must see that their ideas are taken seriously and that the changes are reviewed regularly and implemented on a timely basis. Trust must continue to be built through concrete actions in the system, responding to users’ requests in a timely manner as they are presented.

From an ISDT perspective, the inclusion of Preece’s CCD as a kernel theory was justified, but that it was only effective at developing the initial design. A more complete kernel theory would include the concept of designing for redesign and the understanding that the system would be changing substantially over time. Adding Andrews’ principles for encouraging system use as a kernel theory is also justified.

In short, this experience has highlighted how community information systems are both a design process and a design product and that design theory can provide a framework for designing, testing and modifying such systems. As such, it represents a promising hybrid of social and community approaches, linked with systems design issues and approaches. As Gurstein and Horan show, the design of community information systems is different from the design of management information systems. Much as with information systems to support business processes, further research in this area is needed so that a comprehensive set of theories can emerge to support the development of effective community information systems.

Andrews, D. C. (2002). "Audience-Specific Online Community Design." Communications of the ACM 45(4): 64-68.

Bieger, T., C. Laesser, et al. (2005). The relevance of revealed preferences in market oriented innovations. Innovation in Tourism - Creating Customer Value, AIEST (St. Gallen): 31-49.

Bourgeois, D. T. (2006). Building Social Networks: Action Research on Design and Testing of Online Systems in a University Setting. Information Systems and Technology. Claremont, CA, Claremont Graduate University: 210.

Casapulla, G., F. d. Cindio, et al. (2001). Community networks and access for all in the era of free Internet: 'Discovering the Treasure' of community. Community Informatics: Shaping Computer-Mediated Social Relations. L. Keeble and B. D. Loader. London, Routledge.

Gurstein, M. (2003). "Effective use: A community informatics strategy beyond the Digital Divide." First Monday 8(12).

Gurstein, M. and T. Horan (2005). Why Community Information Systems Are Important to the Future of Management Information Systems and The Field of Information Science (IS). Gordon Davis Series on the Future of Information Systems Academic Discipline: Opportunities and Directions.

Kraut, R., V. Lundmark, et al. (1998). "Internet Paradox: A Social Technology That Reduces Social Involvement and Psychological Well-Being?" American Psychologist 53(9): 1017-1031.

Kraut, R., Kiesler, S., Boneva, B., Cummings, J., Helgeson, V., & Crawford, A. (2002). Internet Paradox Revisited. Journal of Social Issues, 58(1), 49-74.

Markus, M. L., A. Majchrzak, et al. (2002). "A Design Theory for Systems That Support Emergent Knowledge Processes." MIS Quarterly 26(3): 199-232.

Nie, N. H. and L. Erbring (2000). Internet and Society: A Preliminary Report, Stanford University: 49.

Preece, J. (2000). Online Communities: Designing Usability and Supporting Socialbilty. New York, John Wiley & Sons.

Putnam, R. D. and L. Feldstein (2003). Better Together: Restoring the American Community. New York, Simon & Shuster.

Rainie, L. (2000). Tracking Online Life: How Women Use the Internet to Cultivate Relationships with Family and Friends. Washington, D.C., The Pew Internet and American Life Project.

Shklovski, I., R. E. Kraut, et al. (2004). "The Internet and Social Relationships: Contrasting Cross-Sectional and Longitudinal Analyses." Journal of Computer-Mediated Communication 10(1).

Walls, J. G., G. R. Widmeyer, et al. (1992). "Building an Information System Design Theory for Vigilant EIS." Information Systems Research 3(1): 36-59.

Wellman, B. (2001). "Computer Networks as Social Networks." Science 293: 2031-2034.

1The pizza party was primarily used as a feedback gathering mechanism and was not used to encourage additional registrations or logins.

2These results were found by doing a paired t-test within each group for each question, searching for a significant change in the components of social capital between the initial and follow-up surveys. Unfortunately, due to attrition, the number of valid surveys returned dropped to the point that showing significance between groups was very difficult.Further, the short timeframe of the study limited the ability to analyze fully the effects of the system. However, the results can still be used to analyze trends and understand the impacts of the system.

Social Capital Component	Question Type	Change in Social Capital	t-score level of significance
Social Network	Identification of other participants	+	0.065
Social Network	Depth of friendship with other participants	*	*
Trust	Perceived trust (bonding)	*	*
Trust	Perceived trust (bridging)	+	0.015
Mutual Assistance	Activities with other members of the community	+	0.078
Mutual Assistance	Desire to be involved with community	+	0.090
Norms of reciprocity	Perceptions of caring (bonding)	*	*
Norms of reciprocity	Perceptions of caring (bridging)	*	*