Participation and Deliberation on the Internet: A case study on Digital Participatory Budgeting in Belo Horizonte

 

Rafael Cardoso Sampaio

Universidade Federal da Bahia. Center for Advanced Studies on Electronic Government and Digital Democracy (CEADD)

 

Rousiley Celi Moreira Maia

Universidade Federal de Minas Gerais. Research Group in Media and Public Sphere (EME)

 

Francisco Paulo Jamil Almeida Marques

Universidade Federal do Ceará. 

 

 

 

Abstract: This paper aims to examine how political conversations take place on the digital discursive tools offered as part of the Digital Participatory Budget (OPD) in Belo Horizonte (Brazil). The authors propose an analytical model based on deliberative theories in order to investigate the discussions over this participatory program. The main sample consists of the messages posted by the users (n=375) on the commentaries section. The results show that reciprocity and reflexivity among interlocutors are rare; however, the respect among the participants and the justification levels in several arguments were high during the discussion. The authors conclude that, even in a situation in which there is no empowerment of the digital tools, the internet can effectively provide environments to enhance a qualified discursive exchange. In spite of low levels of deliberativeness, the case study shows that there are important gains concerning social learning among the participants. 

Keywords: Online Deliberation, Digital Democracy, Brazilian Participatory Budgeting.

 

1. Introduction

In recent decades, there is an increasing discussion about models of democracy which emphasize popular sovereignty. The debate seeks to overcome the idea that the ordinary, nonorganized citizen is apathetic or unable to influence the political system in order to improve it. Those views are concerned with the current ideal of more participatory democracies, and especially, more deliberative ones (Habermas, 1996).

Having this in mind, several discursive experiments have been conducted, focusing on an inclusive and egalitarian public deliberation, with real effects on the political decision-making. That is the case of initiatives such as Deliberative Polls ®, Citizen's Jury, Consensus Conferences, AmericaSpeaks and many others (Goodin, Dryzek, 2006). All these projects make efforts to test experiences designed to include citizens in the discussion about public matters. They also help the improvement of a relevant set of participatory mechanisms which are likely to be employed in mass democracies.

‘In the Brazilian case, the most prominent experience is the participatory budgeting (PB), which started in the 90s. This kind of program has as goal to share the decision power with citizens, regarding issues that affect them directly. On one hand, the sphere of citizenship has the opportunity to perform extensive discussions and negotiations, seeking to indicate what would be their most important needs. On the other hand, the State, by encouraging participation and deliberation among citizens, offer to the people a share of its power. It is argued that its ultimate objective is to achieve fairer and more legitimate political outcomes (Fung, 2007).

Over the last years, an increasing number of Brazilian political institutions have conducted experiments of participatory budgeting, emphasizing the use of internet. In order to understand what are the main characteristics and effects of this kind of opportunity of political participation, we propose to examine one of the most prominent cases of online PB in Brazil. Specifically, we seek to identify whether Internet use had positive effects on how public deliberation has been developed among citizens.

Specifically, we examine the online discussion boards available at Belo Horizonte’s of digital participatory budgeting website in order to understand some of the aspects that may have produced impact on the results of that deliberation among citizens. In the first part, the article briefly reviews Habermas' concepts of public deliberation. Our intention is to expand parts of the conceptual discussion around this issue. Second, the paper presents a set of studies on online deliberation, as well as some of the key items that should be considered in order to methodologically observe discursive manifestations within the digital environment. In the third part, we indicate the empirical object to be studied, by characterizing its context and its digital tools. The fourth section outlines the methodology applied to understand those events, regarding online deliberation. Finally, we present our main findings and conclusions which can be drawn from this case study.

2. Public Deliberation

There have been different traditions of deliberative democracy, which are influenced by philosophers, such as Aristotle and Hannah Arendt; by pragmatists, such as John Dewey and George Mead; or even by liberals, such as John Rawls. For our purposes, we are going to use the idea of deliberative democracy proposed by the German philosopher, Jürgen Habermas. Besides being the model used by most studies of online deliberation, some of its specificities are relevant to us, as it will be seen below.

Habermas (1996) argues for a true popular sovereignty, and through his deliberative model, he explains how communicative power is related to the administrative power. The author's goal is to provide conditions for the legitimate genesis of the laws, through a heuristic effort dedicated to think about more participatory forms of democratic practices. Habermas‘s fundamental idea is the exchange of reasons among political actors, fostered by a set of discursive processes which are able to echo on the state structure.

In discourse exchanges, participants thematize requirements for validity claims, and try to either retrieve or criticize them by means of arguments. Its strength is assessed in a context created by the warranty of reasons. Because participants’ arguments are submitted to evaluation and criticism, it is believed that rational expressions can also be corrected and improved through discursive exchanges (Habermas, 1996, p.173-4).

Deliberation would be the search, through discursive practices, for the "best", or yet, the most valid, fairest and truest solution. It proposes to bring ways of dealing with conflicts for which, otherwise, would be difficult to find a solution. The process of opinion formation and deliberation influence the preferences of participants, as it allows them to filter issues, contributions, information and arguments in dispute. 

Furthermore, deliberation would present an extra advantage because, even in cases in which it does not succeed, it remains open for future contributions, as deliberation is continuously under evaluation by concerned people.

However, despite being extremely influential, Habermas' discursive model has been intensively contested. In the search for alternatives to Habermas' model of deliberation, many scholars contend that deliberation is not solely constituted by reason. According to John Dryzek (2007), for instance, deliberation models based on excessive rationality and on search for justification by reasons that all could accept would be highly demanding. Such models tend to ignore aspects such as coercion, deception, manipulation and strategy, which are common elements in deliberative processes. A more defensible version of deliberation includes negotiation and bargain. Under certain circumstances, negotiation and bargain, as long as not based on coercion, may help participants to better understand other people's interests and even their own. If the idea of common good is related to the best solution for the greatest number of people, bargaining and negotiation should be important elements of reciprocal communication, in order to forge self-interest and detect claims for the common good (Mansbridge, 2007, p. 264).

Mark Warren also tries to expand the conditions to accomplish deliberation. This scholar argues that it is reasonable to expect participants to enter communication with strategic intentions. However, participants, regardless their original intentions, may need to craft good arguments and reframe issues in order to persuade others or to exert influence in the course of a discussion. Warren’s key argument is that those concerned with democratic institutions, “should be more interested in the outcomes of communication than communicative intent” (Warren, 2007, p. 278) Therefore, institutions should provide opportunities and incentives for dynamics that are “deliberative in function” (Warren, 2007, p. 278).

2.1 Online deliberation

Deliberative democracy, according to Dryzek (2007, p.237), is “the most active area” in contemporary political theory and there is also a growing interest to increase the discursive component within the daily performance of political activities. This trend is also found in studies on Internet and politics. Several studies investigate how digital ICTs could improve deliberative democracy or help citizens engage in a qualified public deliberation.

Since the Internet allow people to talk to one another and hear different points of view, without constrains of time or space, it can also be very valuable for political expression, deliberation and even decision-making (Davis, 2005). However, several studies on online deliberation showed negative results in terms of deliberativeness, and a lack of willingness from users to participate in exchange of reasons. This is the case of Jankowski’s and Van Os’ research (2004). By studying forums of Hoogeveen digital city (Netherlands), they found that online discussion had not presented sufficient features which would lead them to identify traits pertaining to a deliberative debate. The discussions were dominated by a small group of participants; there were some restrictions in relation to the several issues; there were few expressions of mutual interests and reciprocity.

According to Wilhelm (2000), deliberation is linked to the diversity of ideas and sources. His studies show that Usenet [1] online forums are, by contrast, very homogeneous, and were more likely to gather like-minded people. Such approach, according to this scholar, would lead to balkanization of discourses. In other words, certain types of forums would be inadequate to promote citizens' exposure to different political perspectives. The findings of Wilhelm (2000) also indicate that threads have a short life, since participants come and go, just lurking the development of debates, and not entering the discussion.

Davis (2005), by examining the Usenet forums, is also emphatic in excluding the possibilities of an effective online deliberation. He states that online discussion lacks an effective search for solutions, which is the ground of the idea of deliberation. The Internet – so the argument goes - has only served to public expression of private positions, which are not necessarily confronted. The ideal public space should offer not only the chance to discuss about topics, but also let people learn about issues. Citizens, in turn, should also be willing to listen to and to find solutions to their dilemmas of political nature (ibid.).

However, we argue that these three researches mentioned above are, on the one hand, based on a concern excessively directed to evaluate the behavior of participants of online forums. On the other hand, they focus on the Internet just as a tool. Such studies conclude in general that users have no interest in entering qualified discussions, and also that new media do not have adequate tools or ways to give support to such discussions.

In order to examine the digital discussion tools of participatory budgeting, we endorse a different perspective for conducting a study on online deliberation, through the inclusion of three items: (I) the context in which the discussion has been developed, (II) the structure or design of digital communication tools, and (III) methodological strategies employed to understand deliberation.

According to Janssen, Kies (2005), context is defined basically by cultural differences identified among users, by the type of political actor who is hosting the debate, by the ideology of the participants, and by the topic of debate. Such characteristics allow a more qualified understanding of online deliberation, considering the participants, and also the political actor to be fostering discussion.

Communicative structure, also according to Janssen, Kies (2005), is related to the characteristics of the online forum's digital tools, namely: need to identify or not the person who presents arguments, opening of the forum to different audiences, freedom of agenda setting discussion, moderation, and empowerment. Through such analysis, it is possible to assess constraints and possibilities the online forum management may have on participants. It also enables one to assess the position of the political agent who is hosting the debate.

On this second aspect, it is possible to include the way these tools had been developed (Wright Street, 2007) and their effective functionality. Thus, several types of software design can be developed with the intention to focus more or less on public participation, civic conversation, or on the promotion of information within the digital environment, showing also the behavior of the political agent responsible for the digital space development (Gomes, 2005; Marques, Miola, 2007; Marques, 2008; Jensen, Venkatesh, 2007; Salter, 2004).

The third aspect, concerning the methodological model to understand deliberativeness, is the most important one. Some previous studies concluded that the lack of deliberativeness detected in several cases may be linked to the strategies employed for empirical analysis. Papacharissi (2004), for instance, attempted to call into question studies which point out the flames as harmful to democracy. The author makes an important distinction between impolite and uncivil posts. Whereas impolite posts result of human emotions only (open to public apology), uncivil ones offend the dignity of the interacting actors, and consist a serious threat to democracy and political discussions. In her case study, Papacharissi found polite and civilian online discussions, and evinced that not every rude message is necessarily uncivil.

In another example, Wright and Street (2007) replicated Wilhelm's procedure (2000), when evaluating the same conditions of deliberation in the online forum "Futurum", used for political discussions about the European Union. The research indicated that the forum was highly interactive and that the threads were longer than those identified in Wilhelm's research. In addition, participants engaged in the discussion and, in most cases (75%), presented external data to support their ideas. Beyond a more suitable design for deliberation, the authors ascribe this difference in the outcome to the methodology.

In the same vein, Janssen, Kies (2005) seek to discuss methodological problems of studies on online deliberation. Besides the criteria being significantly different among various studies, many investigations tend to ignore several aspects which may shape or affect deliberation in the forums. Based on these researchers’ developments, we present the model of deliberation which was used in our research designed to study the DPB of Belo Horizonte.

3. DPB

In 2006, the City Hall of Belo Horizonte [2] launched the "Digital Participatory Budget" (DPB), which, differently from the city's regional PB [3] , would not require physical presence of participants, which means that the process would take place only through online voting. Belo Horizonte's City Hall would invest U$ 11.25 million in its nine regions (U$ 44.2 million budget of offline PB were maintained, meaning that the DPB had a different budget from its face-to-face version).

Through the website http://opdigital.pbh.gov.br, any citizen with his or her voter’s registration number from Belo Horizonte could choose 9 out of 36 projects (being one project per region), pre-selected by the City Hall, and by the associations linked to the PB. A peculiar feature concerning the DPB was that the voter, after choosing, could know exactly how many votes each pre-selected project had so far. Citizens should vote at least in one region, but could vote in all nine projects.

To minimize problems related to digital divide in the project, the City Hall established several voting kiosks throughout the city. Associations’ headquarters, cooperatives and schools were also listed as official voting locations. The 2006 DPB website (which was totally apart from the offline PB) presented basic information about each project, such as cost, location and pictures. In addition, online participation tools were also offered, such as e-mail and discussion boards. In the end, the 2006 DPB reached 172.938 participants, representing around 10% of the city’s electorate.

After the end of voting, the 2006 DPB's website continued online. However, the discussion board was closed down to participation, and the posted messages did not remain available for consultation. Only basic information about the winning projects from each region was maintained. Statements from the population were also added, regarding participation in the online program.

The 2008 process, in turn, was very different from the original one. First, votes were no longer occurring by region, but rather, there was a single voting for the whole city. Then, voters from Belo Horizonte should choose one among five pre-selected projects. All the options referred to road projects, with the goal to improve Belo Horizonte's traffic.

The projects of 2008 DPB affected a larger number of people, because they were far more valuable than the projects of 2006 DPB, or than the regional offline PB. Each project of 2008 DPB were worth around U$ 22.2 million versus U$ 11.1 million distributed in all projects of the 2006 DPB. The regional PB, for instance, was worth a total of U$ 44.4 million, but divided among several work projects in nine city regions. A single region usually receives investments either less than or equals to U$ 6.6 million (DPB, 2008; BHCH, 2009).

Regarding the technological aspect, 2008 DPB's website was already launched with all tools available and open to vote (Sampaio, 2010). Officers from the City Hall had neither developed nor managed it: a [4] website development agency was hired to accomplish those tasks. In the electronic 2008 DPB's website, the City Hall presented full details of each project, pointing out the impact, costs, benefits for the traffic, and number of beneficiaries per project.

Besides the Internet, a free phone number was provided in order to allow voting for one of the projects [5] . As it happened with the Internet, the citizen should have been registered to vote in Belo Horizonte and needed his or her voter’s registration number. 11.483 voters used the phone service, which represented about 10% of the total amount of votes. 

The DPB project presented several innovations in terms of digital tools and content. A major improvement was the inclusion of pictures showing the current roads "before" and "after" reconstruction. Educational videos were also offered, explaining the impacts and benefits that would occur when each project was concluded. Another innovation was the use of virtual maps [6] to facilitate the projects' location, as well as the voting spots, as it is shown in the appendix 1.

Finally, DPB’s website also expanded their participatory tools. Besides the reactivation of the discussion board, two new features were implemented. The first one was the possibility of posting online "comments". For each project, there was an option to leave a message without having to register or even to enter identification. The second feature was a chat, opened on previously scheduled dates, in which representatives of "Planning, Budget and Information" bureau, responsible for the PB, would receive questions, suggestions and criticisms from participants. There were four chat sessions during the voting time. Apart from those dates, the chat was not available.

In the end, 124.320 citizens voted for digital PB of 2008. After the process, its board and chat were closed. The discussions held through those two tools were also not available. It was possible to see the posted comments, but one could not post new messages any longer.

Table 1: Participatory Budgeting x Digital Participatory Budgeting 2006-2008

Version	2006 offline	2006 online	2008 offline	2008 online
Participants	33.643	172.938	44.000	124.320
Budget (US$ million)	44,4	11,1	44,4	22,2

4. Methodological Procedures

As stated earlier, the present study was conducted using three levels of analysis. First, we assess the context of the online participatory program. However, among the parameters of Janssen’s and Kies’ study (2005), we are going to evaluate only the profiles of the topic of debate, and the political actor who hosts the debate. Due to the difficulty of contacting a representative number of participants (as the website neither would identify the users nor would register some kind of email or contact form), we did not conduce a survey with users.

Second, we will make a brief qualitative assessment of the website, pointing out its communication infrastructure, and design of its tools. Among the items, we selected openness and freedom of participation and conversation within the forum, identification, moderation, and empowerment. Regarding the design, the tool to post comments is evaluated, and is also the basis to assess the third aspect.

The third level of analysis is the study of online deliberation. In the five forums (tools to post comments), 1209 messages were posted, showing support, criticisms, protests and several discussions. Initially, we selected all messages related to the projects #4 (306) and #5 (544), as they were the most commented and voted works. We tabulated and read all messages from the two forums (850). However, among this corpus, we only analyzed messages that referred to the two most commented issues [7] : "Slow traffic and the project serves the region" (196), and "beneficiaries of the project" (179).Those messages (n = 375) were analyzed according to the model below. For the analysis, we used "Atlas TI 5.5" software.

4.1 Model of Online Deliberation

In our analysis, we partially followed Lincoln Dahlberg’s (2001a, b, c, 2002).studies on online deliberation. He is one of the first researchers who have converted Habermas' theory into consistent analytical indicators, which can be applied to the discursive resources of the Internet. Dahlberg highlights various characteristics ascribed to Habermas' ideal public sphere, and translate them into qualitative indicators for empirical assessment of conversations. According to Dahlberg, high presence of such categories indicates greater degree of deliberativeness in the case at stake. In other words, better will be the quality of the discussion under investigation, according to deliberative theory.

We must acknowledge that such analytical strategy assumes an ideal model of debates. However, the purpose of this paper is not to seek a normative approach to deliberation and then become frustrated if it does not occur. High expectations might lead us to conclude that citizens are not being able to deliberate, or that there is a lack of deliberative spaces on the Internet. Our goal is, rather, to use deliberation as an “evaluative-descriptive” concept (Neblo, 2007, p.528) to analyze conversation held in a specific online environment, in an attempt to understand the strengths and weaknesses of this discussion. And to some extent, we also seek to point out some technological and political aspects that may influence those outcomes, in order to highlight the contributions of Belo Horizonte’s online participatory program.

The model of online deliberation which has guided our empirical examination is shown as follows.

I) Thematization and reasoned critique of problematic validity claims

The first criterion, aiming at measuring the degree of deliberativeness, is called "Thematization and reasoned critique of problematic validity claims" (Dahlberg, 2002). The goal is to understand if the positions have been presented and critically discussed. The arguments of the contributors must be supported by reasons.

Initially, we assess Reciprocity (1), that is, if users demonstrate to be reading messages and responding to them. It is a more elementary level, which requires no formulation of response, but rather, just the act of responding to another user, or to the subject of discussion. Messages that do not have answers will be classified as monological (2). Here is an example of the lack of dialogue:

Project 5 Ana Paula (11/13/2008 01:40:29) Excellent opportunity for traffic improvement. It will bring better access and movement to the regional traffic (OPD, 2008).

After, we assess validation, that is, if participants give justifications for supporting their argumentations. Jensen (2003) presents three possibilities of validation: the External validation (3) is when the citizen uses external sources to maintain his or her argument, such as facts, data, news, etc., as we can see through the example below:

Project 5 Eder (11/17/2008 01:36:07) This work is essential due to heavy traffic at the Avenues Ivaí and Abilio Machado, that flows only through Pará de Minas narrow street. It is going to benefit all residents from the northwest region, and also in Contagem, from Ceasa area (DPB, 2008).

The Internal Validation (4) is based on the debater's own point of view, which explicitly uses his or her standards, values and personal experience (such as testimonies) to support his or her argument. The message below is an example of it:

Project 5 Antonio Machado de Jesus (11/13/2008 12:39:25) In my opinion, this project is a great improvement, as I live in Coqueiros' neighborhood, and face this chaos every day, spending more than half an hour in the traffic, from Dom Bosco church until São Vicente Square, it's an absurd (DPB, 2008).

Finally, Jensen argues that it is possible to occur an allegation, that is, the person express his or her position, but does not provide validation, or any other justification. We will call that Position (5), as in the utterance below, which affirms that the government work is going to improve the traffic, without issuing any reasons for supporting that.

Project 5 Eduado (11/12/2008 04:44:16) This work is going to improve the traffic flow in the area, not only for the neighborhoods, but also for those who use the ring road on daily (DPB, 2008).

 

II) Reflexivity

Dahlberg's second criterion (2002) is the notion of "reflexivity".  Participants should be willing to assess the position of others, and revise their initial opinion, once persuaded by the power of other perspectives.

According to Jensen (2003), reflexivity can be measured in three ways. Persuasion (6), when there are explicit evidences that the user feels persuaded by the argument of another participant or by the discussion in general. Progress (7), when the user considers another post, replies to it with new arguments or information, or even tries to create a synthesis of the arguments. Or Radicalisation (8), when the participant reacts negatively to another post, and makes his or her previous point of view more extreme, not being open to other possibilities. According to our view, a message could only be considered as reflexive if it also shows reciprocity. Here is an example of each criterion (Persuasion, progress and Radicalisation respectively):

Project 4 Alessandra (11/13/2008 11:23:25) [...] I agree with Pedro, we have to think about the traffic in the metropolitan region as a whole: Nossa Senhora do Carmo, Raja, BR, and the Ring suffer from the bottleneck around the complexity of BH Shopping (where traffic gets to be restricted to ONE lane , and many citizens are harmed (DPB, 2008).

 

Project 4 Helen (11/14/2008 10:50:40) To those who say this work is going to benefit Nova Lima only, I just have to say they don't know the reality of the region that separates the two municipalities. The truth is that many local residents work in BH, and some from BH work in Nova Lima. So, if the project is intended to benefit the population of BH, who is now badly damaged by the constant traffic jams in the area, there's nothing more ok than doing it independently of the municipality, and benefitting the neighbor population. We need to have collective sense. (DPB, 2008).

 

Project 4 Ellen Andrade (11/24/2008 09:17:09) It's USELESS to invest in the "exit" from BH to Nova Lima, if within the city is still a chaos, because there are places with much greater needs! Honestly, this work won't benefit most of the citizens from BH, but only a few who travels through this area ... Meanwhile, the Ring Road, that is a quick traffic route, remains crowded, huh? An absurd! (DPB, 2008).

 

III) Ideal Role Taking

Dahlberg's third criterion (2002) is called "Ideal Role Taking." This aspect considers listening respectfully and giving appropriate attention to the position of other participants; it also requires an ongoing debate which does not end abruptly.

According to this criterion, we use the division proposed by Steenbergen et al (2003): the Implicit Respect (9) occurs when there is neither negative nor positive positions, but when a group of concerned citizens is defended. The Explicit Respect (10), when there is at least one clearly displayed positive position on groups, regardless of the presence of negative positions. Below, there is an example of implicit respect, in which other group has been defended, without appealing to values; and after, there is a quote of explicit respect in which the citizen champions the rights of the same group:

Project 5 Luiz (11/25/2008 06:06:54) I totally agree with the conclusion of the work, but I have an objection; Residents from Contagem also go through Vicente Square, and as the votes for participatory budgeting are only valid for BH residents, we will be harmed because the number of residents from Contagem is too large and that would yield many votes for this project (DPB, 2008).

Project 5 Cândido (11/24/2008 11:27:14) The problem is that MOST people HARMED by the traffic at São Vicente Square are residents from Contagem, which directly affects the vote; those people should have the right to vote!!!!!! (DPB, 2008, highlights in original).

Regarding the lack of respect, we believe the work of Papacharissi (2004) is suitable for additional analysis. This scholar distinguishes between uncivil and rude messages, as explained before. In this sense, we would also have two forms of lack of respect. The No Respect: rude message, in which there are insults, sarcasm, personal attacks etc., but that does not attack democratic values; and the No Respect: uncivil message, in which there are discourses of prejudice, racism, hatred or attacks on democratic principles. Here is an example of rude message:

Project 4 Leone (11/27/2008 04:30:25) He is right to defend the region where he lives, citing Patio Savassi and BOURGEOIS surroundings, as he may be a playboy or any of our business elite's son. I advise you to go through the region of project 05, in the suburb neighborhood called Alipio de Melo, at peak time, to better understand the reality of the city where you live (DPB, 2008).

The second aspect concerning the ideal role taking criterion is an ongoing debate, that is, their regularity throughout the days, which indicates an interest in keeping the discussion, aiming at a common understanding. We will initially assess the average messages per day (total number of messages divided by days of discussion), and compare it with the number of messages per day. This comparison will allow us to examine whether there are irregularities in the flow of messages, whether there were either days of several discussions or days of neglect, with regards of the amount of messages posted in the tool to post comments.

IV) Inclusion and Discursive Equality

Dahlberg's fifth criterion is called "inclusion and discursive equality". The debate should be open to all concerned citizens, and all participants must have equal opportunities to express themselves.

Thus, in relation to the item "inclusion and discursive equality", we will analyze especially the idea of digital divide, by presenting some data related to the number of people with Internet access in southeastern region of Brazil [8], as well as the implications of this issue for online deliberation. Our goal is far from conducting a complex analysis of digital divide, which is influenced by several aspects, but rather to evidence that this aspect should be considered when carrying out online participatory programs.

Subsequently, we will analyze if only a few users had dominated the discussion, since this aspect may inhibit the participation of other citizens (such as through the formation of closed groups, which ignore the messages coming from "outsider" users).  We presume, from our first analysis, that most individuals posted just one message. According to Jensen (2003), the fact that people post only once (one-timer effect) limits the complexity of the debate, especially when assessing the discussion in terms of criteria such as reciprocity and reflexivity.

Thus, as stated before, we have quantitative and qualitative indicators. All of them are going to serve as basis for assessing the level of deliberativeness of messages. Below, we present the table which explains our typology. The numbered indicators will be applied directly on each analyzed message.

Table 2: Criteria for deliberativeness

 

The other criteria which will be applied to each message are: anonymity (13), identification (14), (user's) name (15), and date of posting (16). Even those who posted just a personal name are going to be classified as identified.

5. Results

5.1 Context

I) Agent to host the discussion

Belo Horizonte's DPB was created and maintained during the administration of Fernando Pimentel, the city's mayor from PT (Workers Party). Pimentel conducted a series of activities organized to implement the PB throughout the nearly 12 years of his administration (four as vice mayor and eight as mayor). PT is also the party of Brazil's current president, Dilma Roussef and the former president Luis Inácio Lula da Silva. He is recognized, among other reasons, for having created many social programs, such as Bolsa-família (an income transfer program), Fome Zero (with the goal to extinguish hunger), ProUni [9] (a grant Program which allows poor class students to study in Brazilian private universities). PT was also responsible for creating the participatory budgeting in the country. PT can be considered a center-left party, since liberal parties and/or right wing parties are included in his base of supporters

It is important to note that we did not identify any manifestation by the municipal administration in our analysis of the messages which compose the empirical corpus of this study. The discussion continued freely, without restrictions. If observation of party-affiliation of political actors may be essential for understanding their utterances in certain policy forums, within DPB, there was no strong evidence that this aspect was determinant. In other words, there is no indication that such aspect changed or shaped significantly the behavior of users.

It is worth to note, however, that the City Hall remained "neutral". As already discussed, no State officer expressed whatsoever their political positions. There is no mention to PT or even to BH's mayor, Fernando Pimentel, but rather, only to the "City Hall" as an institution. There is propaganda of the City Halls' achievements, but that did not mean any appeal to public officers’ or the mayor's positioning.

As we shall discuss below, the actions of the Municipality have shaped participation of citizens in several ways. For instance, there was no official encouragement for discussions about traffic. However, since only road projects could be chosen, local power-holders’ public expressions about traffic policies impacted on the nature of the dispute. The choice of interactive tools can also facilitate or inhibit certain discussions, as well as the type of moderation selected. So, our key argument is that, in the discussions analyzed, there was no officer identifying him or herself as either moderating, or as belonging to the executive, who somewhat tried to direct the issues or the debates.

II) Topic of Debate

People of all socioeconomic levels are somehow affected by traffic problems, in various ways. Daily experience with traffic difficulties thus encourages the search for knowledge and practical solutions. In some areas, citizens may be better positioned in order to evaluate the impacts of policies, as well as to express their views to the representatives (Fung, 2007). This situation was evident when assessing the messages of most participants of both online forums: the users presented a position (40.3% of messages), and validated their opinions (internal + external validation = 50% of messages).

Since the topic under discussion affects virtually the entire population, we may consider – here following Dryzek’s notion of "meta-consensus" – that individuals may not agree on what should be done, but recognize the legitimacy of the disputed values (Dryzek, 2007; List, 2007). According to our findings, the values of respect in both implicit and explicit ways were quite high, exceeding 40% of the overall posted messages, while the presence of rude messages were only 6%.

Most of the individuals who posted messages defended a particular project; yet most of users recognized the importance of other projects, as well as the necessity to attend other areas. That is, they argued that certain locations would need more attention, but did not claim that other regions did not need it either, or that there would not be reasoned arguments on the other side.

5.2 Communicative Structure and Design

I) Identification

Users did not have to enter any identification in the DPB’s website to post comments. However, most of them did it so, by choosing to use either the full name or just the first name. The ones who chose to present only a first name could be considered "anonymous", as Janssen, Kies (2005) indicate. However, some scholars argue that the use of a name tends to indicate that there is a real person committed to the debate, who is not hiding behind that comment, since anonymity is often related to a lack of commitment to the discussion (Davis, 2005; Wilhelm, 2000).

In the analyzed forums, although the identification was not mandatory, we noticed that most participants sought to present arguments in the discussion. An evidence of this is that the number of irrelevant messages (those that did not contain demands, opinions or clear positions) did not exceed the average of 5% in both forums [10]. On the other hand, there is not any indication that anonymity helped to improve discursive equality.

II) Openness and freedom

The discussion tools of DPB allowed enough openness and freedom to the users. Apart from the vote, and the discussion boards, the site was completely open to any visitor, and identification was not mandatory. There were neither tools to control posts by time, nor any type of issue restriction. However, the discussion boards did not contain even 10 messages, while the tool to post comments showed over a thousand collaborations. One possible explanation for this lies in the fact that the discussion boards demanded a registration, while the tool to post comments did not require even identification. Concerning the online environment, registration and identification may serve as cost and hindrance to political participation. This hypothesis, however, would need to be tested by means of surveys with users in future research.

III) Discussion Agenda

The discussion agenda was also wide open. The City Hall officers did not get involved in directing the discussions or topics on the agenda. There is obviously an initial agenda set by the City Hall, Public officers, for instance, defined that all 2008 DPB projects would be related to roads, they set the traffic as the issue of the program. However, no municipal officer acted to either frame it or to feed it. We noticed that the initial agenda was influential since most of the people’s messages focused on traffic issues, even with neither promotion nor pressure from the municipal administration to forward those discussions.

IV) Moderation

According to Nitrato communication agency – the firm which developed and managed digital resources of the DPB website – all interactive tools were pre-moderated, that is, the moderator had to release the message in order to appear on the website (Nitrato, 2009). Pre-moderation can obviously disrupt the rhythm of any discussion, especially if the moderator takes too long to release the messages (Wright, Street, 2007). When analyzing the message flow, we noticed that the moderator released the messages every single day, which certainly eases the problem.

In the case of DPB, moderation existed only in order to avoid rude or uncivil messages. On the other side, the moderation has not manifested his or her point of view in the debate at all. That is probably explained by the fact that webmasters were hired by the City Hall to accomplish such task. The goal, apparently, was only to avoid flames. Indeed, this choice influenced the low number of rude messages detected, and the amount of zero uncivil messages, which - if they existed whatsoever - did not reach the public forum. This may have facilitated the presence of respectful messages, because it is easier to be respectful in a friendly environment than in a disturbed one (flamewars).

However, this choice has some disadvantages. For instance, it is possible to realize a lack of discursive interaction between officers and citizens, since the method leaves the users as the single responsibles for the discussion. The chat room was an exception, as it involved the presence of municipal secretaries. After all, the City Hall seemed not take full advantage of the interaction space, in order to better understand citizens' needs and desires. We may then assume that the representatives were not willing (or even interested) to know citizens' discursive inputs.

V) Strong or weak Public Space

There is no indication, on the DPB website, that the forum or the comment  tool are strong (empowered) spaces, that is, there was no proof that the messages would be considered or even read by the institutional officers. However, the content of the messages shows that people acted with the hope that their message would have some impact, either on political representatives, or on the other citizens. 

But there is a peculiarity concerning the Digital Participatory Budgeting. The discussion tools operated side by side with the voting tool. This tool showed great empowerment, as the citizens' vote decided the project to be implemented. The importance of voting and the considerable empowerment of citizens may have been factors that explain why the discussion tools had become strong spaces for discussion.

At this point, it is interesting to note the peculiar ways in which participants use the tools available. On the one hand, speaking in normative terms, the tool to post comments was a weak space of discussion, because the City Hall had not empowered it. On the other hand, participants acted as if this discursive space was strong, as the users noticed an increasing flow of messages and, consequently, more chances of having their comments read and considered by other citizens. Hundreds of people entered that discursive space not only to talk, and to present their positions, but also, and largely, to try to convince other people about the best option. Or yet, the empowerment of the voting tool echoed in the comments tool.

5.3 Deliberativeness

The overall result of the level of deliberativeness is presented in the table below. We introduce the number of messages classified in each indicator according to the forum and to the original issue; we also indicate the percentage of those messages posted in a specific issue and forum. In the last right column, we show the number of all classified messages according to each indicator, and also its percentage. 

Table 3: Results of Deliberativeness in DPB’s comments tool

It is worth to note that certain indicators, such as "dialogical" and "monological" are exclusionary, while other items can coexist, that is, can be simultaneously present in the same message, as for instance, "external validation" and "internal validation". Moreover, we point out that certain codes reach 100% as a whole (such as "anonymous" and "identified"), because no message could be classified some other way. Furthermore, the indicators that represent respect, validation and forms of reflexivity do not appear in all messages and do not reach 100% altogether. 

I) Thematization and reasoned critique of problematic validity claims

About 40% of the participants claimed to be either in favor or against the project execution, or the process itself (position). Other members (40.8%) not only claimed, but also offered rational arguments to validate their claims, based on what was presented as facts and data. Finally, 12% of the participants validated their claims by making use of narratives and personal experiences and testimonies, which may also contribute to deliberation (Young, 1996).

On the other hand, a smaller amount of users were willing to reply to messages. Less than a third of the assessed messages made reference to another message or to the discussion itself (although 30% means a significant amount). Perhaps it is overly demanding that the tool to post comments presents high levels of reciprocity, dialogue and other similar traits. We are going to return to this discussion later.

II) Reflexivity

If reciprocity, which means the simple act of responding, was low, reflexivity, which is an even more complex process, proved to be a rarer phenomenon in the discussion, reaching about 20% of the total of messages. On the other hand, if we consider the amount of reflexive messages (87) within the set of messages that simply respond to the discussion somehow (112), we find that around 77% of users, who responded to the forum, did so somehow in a reflexive way. In other words, most citizens responded either to other participants or to the subject in a reflexive way, by presenting new arguments, by explaining why they agreed with some other argument already placed. 

Apparently, due to the simplicity of the tool, the biggest challenge was reading the other posts, but those who attended the discussion, actually did so in a reflexive way.

III) Ideal Role Taking

A) Continuous Dialogue

Figure 1: Posts per day

Assessing the chart, we can notice that it worked in the form of "waves", that is, the exchange of messages was higher on certain days, and was almost absent in other ones. However, apart from a few days, in which the number of posts drops considerably, the average number of messages approximates the weighted average, which would be 14.6 posts per day. We can also notice the rare existence of several following days with less than 10 messages. Besides that, in the last four days of voting, all posts were below the average. This result can be explained by the fact that project 5 was already 10 thousand votes ahead of the project 4 in the last week of the process.

The one-timer effect was quite high, since reciprocity was low. Around 70% of participants left only a single message, and never returned to the topic [11], meaning a low willingness to dialogue. The idea of leaving a comment, expressing a particular point of view, but not necessarily demonstrating willingness to dialogue, was strongly adopted by the users. 

On the other hand, an ongoing dialogue is a complex indicator to be assessed on the internet. As Janssen, Kies (2005) recognize, the problem of several studies on online forums is that they tend to believe that the discussion happens solely and exclusively within the assessed tool. They end up ignoring the multiplicity of simultaneous tasks allowed by new technologies. For example, it is perfectly possible that one enters the DPB website and post only once, but after, he or she writes something in his or her Twitter, weblog and in his or her Facebook profile. In each of these tools, he or she can either mobilize other potential voters, or enter real argumentative disputes.

Moreover, the validity of presenting such data is that the citizens were willing neither to respond nor to maintain an ongoing dialogue, or that DPB digital tool was not sufficiently interesting to keep the discussion on its website.

B) Respectful Listening

Messages that showed some form of explicit respect to individuals or groups were nearly 15% of the total analyzed in the two forums. Messages that showed some attention, care or implicit respect in relation to the others reached 28% of the total, which means that about 40% of the messages showed some form of attention and respect to those affected by the DPB projects. The lack of respect was also low (6%), which may be due to pre-moderation. As already mentioned, we believe that the issue has also a direct influence on this result, as it facilitates someone putting him or herself in someone else's place, as well as it tends to increase respect for other opinions.

IV) Inclusion and Discursive Equality

One aspect to be considered when discussing the interface between Internet and democracy is the digital divide. Difficulties of access involve a considerable part of the Brazilian population. This problem is particularly worrying if we consider the Digital Participatory Budgeting’s aim to expand the number of citizens who are integrated into the decision-making process. At this point, the question refers not only to differences among users, but also to the equipment needed to access the digital environment. Besides that, we cannot neglect the existence of distinct cognitive repertoires, peculiar to each individual citizen, which end up echoing in the ways of using devices available for online participation (Salter, 2004).

Naturally, one must recognize the importance of the digital divide in a country like Brazil. We are not claiming that one cannot talk about digital democracy while everyone is not adequately equipped and able to make use of a computer connected to the Internet. We consider this stance a step backwards, since several aspects of contemporary governance, such as transparency in public affairs, preservation of freedom and rights, as well as electoral participation, have been improved due to the use of digital media (Maia, 2008; Marques, 2008). Moreover, several inequalities have traditionally characterized Brazilian society and national political culture, and yet, it has been the adherence to democratic values which has enabled political innovation and alleviation of social inequalities and injustices.

In this sense, we consider valuable additional studies aimed to understand the complex chain of aspects of digital divide, which intersect political participation on the Internet. We believe such investigations may help to understand, for example, the reasons why online participation were greater than offline in the case of Belo Horizonte's Digital Participatory Budgeting, even though most of resources was associated with the PB face-to-face type.

While making these assumptions, our goal, by including such an item proposed by Dahlberg (2001a) and Janssen, Kies (2005), is to think digital barriers as an extra item to be considered in the design of digital tools, and in the setting of online participatory programs.

Thus, we used the data from ICT Centre for the Study of Information Technology and Communication - CETIC.br, which is in charge of producing indicators and statistics on the availability and use of Internet in Brazil, held in 2009. As there is no research available about Belo Horizonte, we used statistics from Brazilian southeastern region, where the city is located.

In southeastern Brazil, 34% of households have a computer; 26% of all households also have internet access. 37% of Internet users use the phone line connection (slow), and 53% use some kind of either high speed or broadband connection. However, when analyzing it according to social class [12], differences become evident: 93% of A Class people access the Internet, 59% of B Class, 17% of C class, and only 1% of D and E classes, which are below the poverty line. Necessary skills to use an online discussion boards are also low, reaching only 24% of the population of the region [13]. Regarding several skills to use the Internet, inequality also grows according to education and social class (Tic, 2009).

Similar to Wilhelm's findings (2002), the digital divide is still very present, and tends to reinforce other social-economic inequalities. In other words, if the number of Internet users in Brazil is rising, it grows in an extremely unequal way among different educational and social classes (Tic, 2009).

In this sense, DPB's discursive equality is seriously a priori affected. The provision of free Internet access points, as it was the case of DPB, mitigates that inequality, but in a very narrow way. The TIC's research itself shows that only 1% of Brazilians utilize free Internet access points to make use of any electronic government service, against 46% who use the services at home, 23% in paid hotspots, and 16% at work (Tic, 2009). We support that free access points may facilitate voting, since the process is relatively simple and fast. However, an online discussion, according to the various requirements for deliberation, would demand more time and user’s proper motivation. .

DPB's tool to post comments, despite being simple, was extremely open and receptive to participation. It did not require any registration or identification. In addition, it allowed quick reply, posting through only one click (that is, it required a few skills to use Internet). Janssen and Kies (2005) suggest that technical constraints may inhibit participation and deliberation. In this sense, the mentioned tool appears to have been a major impediment to achieve a qualified deliberation, but it certainly did not impede participation in the discussion.

Among those who have posted, there was no evidence that the debate was dominated by a few participants. The first explanation for this lies in the fact that most people posted only once, but even among the most active ones, there was no user who dominated the debate.

6. Discussion and Conclusions

Participatory budgeting is one of the few Brazilian digital programs that uses the Internet to empower citizens’ decisions. It is also innovative in relation to the several participatory and interactive tools it offers, when compared to other formal political institutions in the country (Marques, Miola, 2007; Marques, 2008). The DPB has achieved great success by engaging a significant proportion of the population of Belo Horizonte in the process of choosing the options offered by the City Hall.

In the first level of analysis, we noticed that the political actor to be hosting the debate may sometimes be a weak or neutral agent. In the case of DPB, the intervention of City Hall's political officers was scarce; thus, the analysis did not identify any connection with the party or the government which created the website. Moreover, the topic of discussion was a significant factor for explaining certain outcomes, such as over-validation and respect within the messages.

On the second level of analysis, the main conclusion is that discussions with little control can generate good results, with high levels of validation, respect and identification. It is obvious that the presence of a moderator in charge of promoting deliberation would increase the values found for reciprocity and reflexivity. However, pre-moderation here only prevents from disrespect and incivility. It does not generate the respect we found in this online forum; this result tends to challenge some of the findings of Wilhelm (2000) and Davis (2005) on the individual inclinations for deliberation. People can be respectful without a previous guidance.

Another finding we consider interesting was that a not empowered forum generated a reasonable number of argumentative exchanges. Apparently, the empowerment there was related to two issues: a) the impression that the messages would be read - even if they were not read by institutional agents, but rather, by other citizens; and b) the fact that the same website contained the voting tool, which makes the forum a way to mobilize other citizens to vote or to try to convince them to make a choice.

In the final analysis plan, building a model to understand online deliberation has proved productive. Overall, as well as in other researches (Davis, 2005; Wilhelm, 2000), data indicate that people are more willing to express their thoughts than to listen to other citizens' considerations. It is also necessary to consider the context of a participatory program at a municipality's website whose main objective is to decide on certain projects, as well as the fact that the analyzed messages were in a comment tool. In theory, it is not unusual to find little dialogue in such environment. In fact, rates are worth to be considered, especially if the digital tool makes messages available in chunks without any kind of organization, and does not provide an option to reply a message directly to a participant.

Thus, considering those technical obstacles, as well as the lack of debate orientation by the municipal administration, we can say that the values of respect and argumentation were above expectations. As we stated before, we believe that the main reasons for that lies in the fact that the discussion tools are located on the same website of the voting tool, as well as the issue itself, once traffic distresses all inhabitants of a city.

Taking into account the asynchronous nature of the Internet, these results may have a great political value. The messages remain available for other people who visit the website later. For this third citizen (who visits the forum after the discussions), two initial arguments, which do not respond to each other, but present opinions based on reasons, may constitute a "debate" within the cognitive process of the "third party" involved. Soon, even those who just watch the debate, the lurkers, could benefit from reading the messages.

Finally, even neither encouraging nor empowering the discussion, DPB has created a space in which it can happen. If most individuals chose neither to read nor to respond to other participants, at least, it was possible for citizens to express their opinions about the participatory program, and about their needs and desires. Although we cannot be certain of it, nothing prevents the City Hall officers from keeping such discussions, and subsequently from using them as a feedback of the participatory process. As Warren (2007) states, one can open possibilities to institutionalize deliberation by capturing speeches which are not deliberative in intention, and by producing dynamics that are deliberative in function.

It is worthy to note that some of these conclusions cannot   be generalized to all those discursive events and phenomena which may take place on the Internet. However, it seems clear that such experiences are important to the idea of democracy qualified with discursive features. As stated by Dryzek (2007), deliberation may have several tasks, but it does not need to fulfill all of them simultaneously. Soon, there will seldom be instances of deliberation, which fully present rates of deliberativeness at its maximum. Low rates are also important to identify deficits that either citizens or deliberative programs themselves present. As we sought to highlight in the studied case at stake, the high points of deliberativeness demonstrate that individuals can indeed enter discursive disputes with high levels of arguments, and respect for one another. Regarding the low deliberativeness degree of some aspects, we conclude that there is a need for improvement related to the level of sophistication of the digital discursive tools, as well as a major performance of an institutional officer, who would not only consider the inputs of the participants, but also encourage discussion among citizens themselves.

Notes:

[1] Usenet (Unix User Network) is a medium where users post messages ("articles") in forums which are grouped by topics (newsgroups).Articles posted in newsgroups can be forwarded through an extensive network of interconnected servers. Source: http://pt.wikipedia.org/wiki/Usenet . Accessed in: March, 2nd, 2010.

[2] It is the capital of the State of Minas Gerais, Brazil. It has 2.412,937 inhabitants. Gross Domestic Product R$ 32.725 billion, GDP per capita R$ 13.636. Source: IBGE, 2007 (PBH, 2010).

[3] Face-to-face PB occurred also during the two years of Digital PB implementation, that is, all offline PB items proceeded normally, such as agenda, budget, structure, organization, rules, etc. There was no connection between the two processes.

[4] Nitrato agency was responsible for managing the website and for moderating the discursive tools available. http://blog.nitrato.com.br/ . Accessed in: August, 25th, 2009.

[5] The telephone cannot be considered an online technology, but its use is common and encouraged in participatory programs on the Internet. About the use of mobile technologies in other digital PBs, see: Peixoto (2008).

[6] Using the technology of Google Maps. See example: http://maps.google.com.br/maps?hl=pt-BR&tab=wl . Accessed in: February, 10th, 2010.

[7] The classification of issues was performed in both forums through qualitative reading. We generated 10 different issues in the online forums of both projects. The methodology of this analysis is explained in Sampaio (2010).

[8] There was no recent survey data from Belo Horizonte. We are using data on the southeastern region of Brazil.

[9] See http://www.brasil.gov.br/sobre/citizenship/social-inclusion/insertion-programs?set_language=en to obtain more information about Lula's social programs. Text available in English. Accessed in: February, 10^th, 2010.

[10] 46 posts out of the original 850 from both issues.

[11] As identification was not mandatory, it is not possible to say whether any user posted more than one message using a different name. The result is based on how users are identified.

[12] The criteria used for classification takes into consideration the household's education and the ownership of several household items, relating them to a scoring system. The sum of the points reached by a household is associated with a specific socioeconomic class (A, B, C, D, E).

[13] The research describes, among the skills: using a search engine to find information, sending e-mails with attached files, sending messages in chat rooms and forums, using the Internet to make phone calls, using file sharing programs, creating a web page, downloading and installing softwares. http://www.cetic.br/usuarios/tic/2009/rel-habil-03.htm . Accessed in: January, 10th, 2010.

 

 

Figure 1: online map to locate the projects

 

 

Figure 2: Tool to post comments

References

BHCH. (2010). Belo Horizonte’s City Hall Website. Available: <http://portalpbh.pbh.gov.br/pbh/>. Acessed: February, 27^th, 2010.

Dahlberg, L. (2002). Net-Public Sphere Research: Beyond the 'First Phase'. Euricom Colloquium: Electronic Networks and Democracy. Nijmegen, The Netherlands: 14.

Dahlberg, L. (2001a). Computer-Mediated Communication and the Public Sphere: A Critical Analysis. Journal of Computer-Mediated Communication, vol. 7, n. 1, p. 1-26.

Dahlberg, L. (2001b). Extending the Public Sphere through Cyberspace: The Case of Minnesota E-Democracy. First Monday, vol. 6, n. 3.

Dahlberg, L. (2001c). The Internet and Democratic Discourse. Exploring the prospects of online deliberative forums extending the public sphere. Information, Communication & Society, v. 4, n. 4, p. 615–633.

Davis, R. (2005). Politics Online: Blogs, Chatrooms and Discussion Groups in American Democracy. Routledge, London and New York.

Dryzek, J. S. (2007). Theory, Evidence and the Tasks of Deliberation. In: Rosenberg, S. W. (Ed.). Deliberation, Participation and Democracy: Can the people govern? New York, Palgrave Macmillan.

Fung, A. (2007). Minipublics: deliberative designs and their consequences. In: Rosenberg, S. W. (Ed.). Deliberation, Participation and Democracy: Can the people govern? New York, Palgrave Macmillan.

Gomes, W. (2005). A democracia digital e o problema da participação civil na decisão política. Revista Fronteiras – estudos midiáticos, v.7,  n. 3, p. 214-222. (Digital democracy and the problem about civil participation in political decision making).

Goodin, R. E.; Dryzek, J. S. (2006). Deliberative Impacts: the macro-political uptake of mini-public. Politics Society, vol. 34, p. 219-244.

Habermas, J. (1996). Between facts and norms: Contributions to a discourse theory of law and democracy. Cambridge: Polity Press.

JANKOWSKI, Nicholas W.; VAN OS, Renée. Internet-based political discourse: a case study of electronic democracy in Hoogeveen. In: SHANE, Peter M. (Org.) Democracy Online: the prospects for political Renewal through the Internet. Nova York: Routledge, 2004, p. 181-194.

Janssen, D.; Kies, R. (2005). Online Forums and Deliberative Democracy. Acta Politica, 40, p. 317–335.

Jensen, J. L. (2003). Public Spheres on the Internet: Anarchic or Government-Sponsored – A Comparison. Scandinavian Political Studies, Vol. 26, N. 4.

Jensen, M. J.; Venkatesh, A. (2007). Government Websites and Political Engagement: Facilitating Citizen Entry Into the policy process. Center for research on information technology and organizations, paper 399. Irvine: University of California.

List, C. (2007). Deliberation and Agreement. In: Rosenberg, S. W. (Ed.). Deliberation, Participation and Democracy: Can the people govern? New York, Palgrave Macmillan.

Maia, R. C. M. (2008). Democracia e a Internet como Esfera Pública Virtual: Aproximação às Condições da Deliberação. In: Gomes, W.; Maia, R. C. M (Eds.). Comunicação e Democracia – Problemas e Perspectivas. São Paulo: Paulus, p. 277-292. (Democracy and internet as Virtual Public Sphere).

Mansbridge, J. (2007). “Deliberative democracy” or “democratic deliberation”? In: Rosenberg, S. W. (Ed.). Deliberation, Participation and Democracy: Can the people govern? New York, Palgrave Macmillan.

Marques, F. P. J. A. (2008).  Participação política e internet: meios e oportunidades digitais de participação civil na democracia contemporânea, com um estudo do caso do estado brasileiro. 2008. Unpublished doctoral dissertation, Federal University of Bahia, Salvador, 2008. (Political Participation on the Internet: A Study on How Opportunities of Political Participation Are Offered by Institutions of the Brazilian State Through Digital Communication).

Marques, F. P. J. A.; Miola, E. (2007). Internet e Parlamento - Um estudo dos mecanismos de participação oferecidos pelo Poder Legislativo através de ferramentas online. E-Compós, Brasília, vol. 9, p. 1-20. (Internet and parliament – a study on the mechanisms of participation offered by the Legislative websites in Brazil).

Neblo, M. A. (2007). Family disputes: Diversity in defining and measuring deliberation. Swiss Political Science Review, 13(4): 527-557. 

Nitrato. (2009). Interviews with managers of Digital Participatory Budgeting Website. Held in 13^th November 2009.

DPB. (2009). Digital Participatory Budgeting Website. Available: <http://opdigital.pbh.gov.br>. Acessed: December, 29^th, 2009.

Papacharissi, Z. (2004). Democracy online: civility, politeness, and the democratic potential of online political discussion groups. New Media and Society, London, v.6, n.2, p. 259-283.

Peixoto, T. (2008) E-Participatory Budgeting: e-Democracy from theory to success?. E-Working Papers.  Disponível em: <http://edc.unige.ch/edcadmin/images/Tiago.pdf>. Acesso em: 10 abr. 2009.

Rawls, J. (2005). Political Liberalism. New York: Columbia University Press.

Salter, L. (2004). Structure and Forms of Use. A contribution to understanding the ‘effects’ of the Internet on deliberative democracy. Information, Communication & Society, Vol. 7, No. 2, pp. 185–206.

Sampaio, R. C. (2010). Participação e deliberação na internet: um estudo de caso do Orçamento Participativo Digital de Belo Horizonte. Unpublished master thesis. Federal University of Minas Gerais, Belo Horizonte, 2010. (Participation and deliberation on the internet: a case study of the Belo Horizonte’s Digital Participatory Budgeting).

Steenbergen, M. R. et al. (2003). Measuring Political Deliberation: A Discourse Quality Index. Comparative European Politics, 2003, vol.1, p. 21–48.

TIC. (2009). <http://www.cetic.br/usuarios/tic/2009/rel-habil-03.htm>. Accessed in: January, 10th, 2010.

Warren, M. E. (2007). Institutionalizing Deliberative Democracy. In: Rosenberg, S. W. (Ed.). Deliberation, Participation and Democracy: Can the people govern? New York, Palgrave Macmillan.

Wilhelm, A. (2000). Democracy in the digital age: challenges to political life in cyberspace. New York: Routledge.

Wright, S.; Street, J. (2007). Democracy, deliberation and design: the case of online discussion forums. New Media Society, London, vol. 9, p. 849-869.

Young, I. (1996). Communication and the Other: Beyond Deliberative Democracy. In: BENHABIB, S. (Ed.). Democracy and Difference. Princeton: Princeton University Press.