Archiving Twitter With WordPress

Yesterday I had a spare couple hours and decided to follow Doug Bowman’s example and set up a self-hosted archive of my Twitter stream with WordPress. You can see the finished product of that here.

There was some interest expressed on Twitter of others wanting to do something similar so I thought I’d help out by making what I did available for download. You can grab a copy of the theme and required plugins which will provide pretty close to a turn key solution for getting this running.

I highly suggest following Bowman’s tutorial for downloading and importing the initial archive of previous tweets. Once you have that done and the plugins and theme are installed there’s a couple things you’ll want to do:

  • Replace the profile_image.jpg file in the theme folder with your own profile image.
  • Head to your profile page within your WordPress installation. You’ll see two new fields, one for the url of your Twitter profile and the other for your Twitter username. These power the text in the header so just fill them both out and the header text will be linked to your profile.
  • The tagline below the username in the header is pulled from your blog’s tagline so fill that out in General tab underneath Settings.
  • Setup Twitter Tools to create a new blog post every time you tweet. You can find more information about doing that at the WordPress plugin directory.
  • Run two queries using the Search Regex plugin (for more info on these queries read the original source). This will link up all the @usernames and #hashtags from your tweets.
    • For @usernames enter /(^|s)@(w+)/ into the Search Pattern field and then enter 1@<a href="">2</a> into the Replace pattern field. Check the Regex box.
    • For #hashtags enter /(^|s)#(w+)/ into the Search pattern field and then enter 1#<a href="">2</a> into the Replace pattern field. Check the Regex box.
    • In both cases I suggest running a Replace before running Replace & Save. This will allow you to look everything over before making changes that will affect your database.

That’s it. After doing those steps you should have a searchable, self-hosted archive of everything you’ve posted on Twitter. If you run into questions or problems feel free to fire away in the comments.

Update 3/23: Emily Ingram pointed me to a plugin that will achieve the same auto-linking of @replies and #hashtags that the regex calls do. It’s a super simple solution and can be downloaded from the WordPress directory. Sounds like it works quite well.

Redefining political participation

The technological capacity of individuals to publish information online expanded tremendously over the last decade. Free platforms for public expression and publication of ideas proliferated, most recently through services like WordPress and Twitter. There are now an incredible variety of tools available to people that allow any individual writer to potentially have global reach.

Many of these tools for publication have an inherently social aspect. More than simply tools for publication and the broadcasting of information they have conversation at the heart of their technology. Communication online necessitates more than just publication. With modern communication tools there as much emphasis is placed upon what happens with the information once put online.

The last few years have seen the astounding expansion of three key technologies: near real-time communication, relatively easy self-publishing, and powerful data aggregation. The rise of Twitter and related technologies of the real-time web, such as the Tornado Web Server, allow for messages to be sent, received, and replied to in mere seconds. Blogging, and self-publishing in general, has developed remarkably powerful tools as well. What started with tools like Blogger and LiveJournal has now exploded with software such as WordPress, Tumblr, Posterous, and Movable Type. Finally, with all of this information being published at a more rapid rate our tools for aggregation have improved immensely. RSS readers such as Google Reader and Fever present the ability to categorize, filter, and rank information from many sources. Furthermore, software developers have begun to leverage algorithms to analyze, sort, and rank news items in a way that allows the software to filter the important items out of the noise. 1

The internet and its associated technologies have been around for decades and seen a vast array of literature devoted to their political potential. What is it that makes the current moment different? What about the political potential of these specific technologies lacks precedence? Continue reading Redefining political participation


  1. A definition of noise will be important to keep in mind throughout these essays. In this discussion I am using Matt Pearson’s definition “Irrelevant & unwanted data; anything that is out of context [to an individual] given the intended signal of the medium in question.” It is important to maintain that noise is relevant to individuals and particular contexts. For these essays there is no objective, universal definition of noise.

Participation Through Publication

As communication online continues to grow we must ensure that there are solid tools providing all with the ability to publish their voice. The ability to make one’s opinions known in a public forum is a requirement of a democratic political system. This right can be traced all the way back to Athenian democracy. Under this system all citizens came together in the Ekklesia to discuss and vote on issues of political importance.1 This can be seen in traditional spaces like town hall meetings, political rallies, and in newspaper editorial sections. The expansion of a desire to make one’s opinions known online signals the most recent manifestation of citizens’ desire to make their thoughts known in a public forum.

The current software available to people wanting to publish online allows for remarkably powerful publishing to occur. Numerous professional-level platforms are offered to any user for free. These tools allow for users to publish their thoughts through free and easy to use software in a public-by-default manner. Furthermore, a growing selection of tools allow for people to publish to a global audience from nearly anywhere. A stationary location with a full-featured computer is increasingly no longer a necessity to partake in online publishing. The ability to publish has been extended to anybody with a mobile phone.

The modern tools that have been developed for publishing online give more people a greater ability to make their voice heard from an expanding range of places. WordPress and Twitter take the ability to publish online and make it something that is accessible to a greater portion of the population. The political potential of the millions of people expressing their voice online can have a tremendous expansionary effect on participation in United States politics. Continue reading Participation Through Publication

Real-Time Politics

Communication has been able to happen nearly instantaneously over the web for years now. Technologies like push email have previously opened channels through which information can be transmitted in real-time. Today’s real-time web are different because of the public-by-default nature of messages. Communication through tools like Twitter allows for people to communicate in a matter of seconds and creates a public facing forum that allows any other user to add their voice to the discussion. The public nature of all this communication means that now any person can instantaneously communicate with any leader (be that politician, celebrity, or renowned professor) and engage in substantive discussion.1

How did we get here?

The year 2006 can be seen as an inflection point for what is now termed the real-time web. That year Twitter launched. Suddenly what we had grown accustomed to with email (waiting a few minutes for an update to arrive) seemed like an eternity when there was a service that provided for updates to stream in microseconds. The fact that Twitter limited these messages to 140 characters came to be overshadowed by the sheer rapidity of information transmission. The real-time web became less about reflecting with examined thoughts and more about spreading what was happening right now.

This trend toward short, instantaneous updates has continued with the launch of FriendFeed in 2008 and the open sourcing of its base web server technology (known as Tornado Web Server) in 2009.  A single company owns the technology behind Twitter but the server technology that powers FriendFeed differs. FriendFeed accomplishes the same rapidity of flow that Twitter popularized but does so with a web server that is open. This means that any developer can use the base layer of technology that FriendFeed open sourced and leverage it as a platform from which any forum for real-time communication could be built.2

These technologies provide a stream through which information can spread globally at an unprecedented rate. Messages can be sent, replied to, and echoed by millions of users within seconds. Most importantly this information is not limited in subject matter. The flow of information makes no distinction between a celebrity death and news of electoral protests in Iran. One service ends up being the focal point for news about the latest celebrity gossip as well as the locus for breaking political and economic events. Judgement is not made about the information that passes through Twitter’s channels, the channels simply exist to broadcast that information as quickly as possible to an audience that is now in the tens of millions.

This lack of distinction made between messages posted on Twitter arguably does add to the noise and presence of non-political information; however, this should not be seen as detracting from its political importance. Later, we will see how modern tools for aggregation are allowing for individuals to filter out the noise, but the mere presence of noise is a political benefit. If tools like Twitter were to restrict published information they would be making an explicit statement upon the political nature and source of information. In an open political society the judgement as to what constitutes noise must take place after publication and, thus, after everybody is able to let their voice be heard. Anything else restricts political dialogue that prevents certain people from participating.

The speed at which all types of information can be disseminated holds tremendous political potential within the United States. Our current political structure has served us well in an age when information traveled through a few select channels that were broadcast throughout the country as part of commercial media companies. As citizens we understood that we would have to wait for the nightly newscast or the morning’s paper to find out about the day’s important events. These media kept us informed in a world where news traveled in hours.

The instantaneous dissemination of information is a reality in 2010 and political participation needs to be reframed in order to take advantage of these tools. Ultimately the real-time web has created an ecosystem of communication that can be used to expand and redefine political participation. In an era that prizes the now, political participation must be reconceptualized as a continuous process.

These technologies are being leveraged to create a forum in which citizens can express their opinion at anytime. The political system of the past segmented participation to occur once every year, or even once every four years. Participation in a real-time political system allows for citizens to be involved every month or week, or possibly every day.

What is the real-time web?

In order to understand the political ramifications of all this technology we must first understand the real-time web. In August of 2009 ReadWriteWeb published a three-part series of articles explaining various aspects of the real-time web. In the first part Ken Fromm writes that the real-time web is,

a new form of communication [that] creates a new body of content [which] is public and has an explicitly social graph associated with it.

This characterization embodies the core of what these technologies accomplish. Twitter and the technology behind FriendFeed embody a combination of the elements outlined by Fromm. FriendFeed and Twitter have an inherently social element to them and both have allowed for a new form of communication that has effectively created a new body of content that did not previously exist. When these technologies are combined with the three elements of the real-time web that Fromm describes the potential arises to achieve a notion of political participation defined by constant citizen involvement.

Real-time politics

Three key areas of this technology hold the greatest impact in terms of political participation. A web that allows for instantaneous communication through the mediums detailed above redefines traditional notions of group formation and the political impact of direct citizen input. These two concepts will be explored at length below but in general the real-time web holds the potential to so drastically shift our conceptions of these actions that a radically different notion of political participation is needed.

The way in which groups form and eventually disband is an aspect of the modern American political system that fundamentally differs in a world where Twitter and FriendFeed exist. Politics in the United States has long been about gathering people together through shared opinions and concerns. In the early years of the nation this was primarily done through political parties. Thomas Jefferson wrote of the process of party formation and division in a letter to John Adams on June 27 of 1813,

Men have differed in opinion, and been divided into parties by these opinions, from the first origin of societies and in all governments where they have been permitted freely to think and to speak.3

While political parties characterized groups formed around shared opinions in the early years of the nation, more recently we can see this same effect in such organizations as, PETA, and the NRA. These interest groups arose out of situations in which political parties are no longer affective enough for citizens. Writing in the early twentieth century P.H. Odegard claimed that,

direct democracy falls down in the face of increasing numbers. The individual plain man, swallowed up in a sea of highly differentiated human beings, finds it necessary to organize with others of a like mind so that by concerted action they may bend the state to their will…It is this situation which has engendered the pressure group.4

These pressure groups, now better known as special interest groups, were the twentieth century’s solution to the problem of scale in a democracy as large as the United States. Throughout the last century not every citizen could realistically make his or her claims upon their government. As such they came to band together just like Odegard describes. The result was organizations like PETA and the NRA that mobilize people behind common interests for shared political action.

Not only do interest groups serve to mobilize citizens but they also play a large role in informing their political views. Phillip Agre writes in “Real-Time Politics: The Internet and the Political Process” that,

Political parties and legislatures, for example, do not simply transmit information; they actively process it, especially by synthesizing political opinions and interests into ideologically coherent platforms.5

The role of groups like, PETA, and the NRA as information centers makes older interest groups outmoded. With how information and communication flows on the real-time web these old institutions and structures no longer represent the most efficient outlets for information. In addition, as will be covered later, the reliance of citizens upon interest groups’ ability to process information is no longer a necessity.

The real-time web provides a toolset that alters the role that organizations like play in political mobilization. Furthermore, the technology behind the real-time web provides a partial solution to the problem of scale inherent in twentieth century efforts to involve a greater percentage of the populace in the decision-making process. Finding effective means toward disseminating political information for the goal of organizing political actions no longer hinges upon the abilities of interest groups. The real-time web allows for individuals to track flows of public information on their own and modern tools of data aggregation allow them take control of the processing of this information as well.

Defining participation through the real-time web

Group organization and action is another foundational aspect of politics that becomes transformed by communication through the real-time web. Clay Shirky writes in his recent book, Here Comes Everybody, that,

Group action gives human society its particular character, and anything that changes the way groups get things done will affect society as a whole.6

Shirky holds that group action represents a vital part of not just politics, but human society in general. The development that Shirky points to as creating change in group action is the same social graph that Fromm characterizes as an inherent part of the real-time web. Shirky claims that with tools based around social interaction,

We now have communications tools that are flexible enough to match our social capabilities…we are living in the middle of a remarkable increase in our ability to share, to cooperate with one another, and to take collective action, all outside the framework of traditional institutions and organizations.7

This increase in our ability to share and cooperate with one another forms the basis for a conception of political participation not constrained by the same problems as that of the twentieth century. When American citizens organized together over the last 100 years they largely did so under the auspices of special interest groups.

These interest groups were organizations that were governed by a board of directors or a similar group of full-time employees working in the best interest of the organization’s many members. This structure mirrors that of the political system at large where citizens communicate with their representatives through well defined channels.

Previous writers have remarked that the breakdown of these channels may hold negative ramifications for democracy. Writing in Radical Democracy and the Internet John Downey explains that,

The public sphere might be both more participative and deliberative [as a result of online communication] but there might not be a democratic bonus if the channels between the public sphere and representatives are severed.”8

What Downey fails to realize is that disruption of traditional channels does not necessitate complete destruction. The mass availability of the ability to communicate in real-time through any number of mediums means that anybody, from an individual that makes up the “public sphere” to a city mayor, can participate. The real-time web only destroys the connection between the public and their representatives if their representatives fail to adapt to a changing landscape of communication.

With information from millions of users being transmitted every minute only a small portion of that information needs to be political for its ramifications to be widespread in American politics. Twitter and the open-source technology behind FriendFeed allow for communication to happen in an inherently social medium. This medium is not limited in its applications. Communication can happen between any user with an account. There are no preferred channels. There are no appointment requirements. A citizen just needs a few short second to type their thoughts and click “Update” to communicate with their representative.

Conceptualizing the real-time citizen

We have long possessed tools that allow for citizens to communicate with representatives, but the real-time web changes the nature of this communication. While it can be argued that the ability to communicate through channels like Twitter merely iterates upon our decades long ability to write letters and emails to representatives this misses the central point about the real-time web: the instantaneous communication that occurs in public-by-default forums.

We finally have a software platform from which we can build a conception of political participation unconstrained by annual or quadrennial elections. This is participation for the real-time citizen.

A political process is an inherently iterative one. Bills are presented, refined, compromised, and eventually voted upon. Traditionally this has happened in the secluded halls of Washington and state capitals. The agents of iteration have been representatives that have been selected by the people but the real-time web provides an opportunity for individual citizens to become engaged in this process. Not only does it allow individuals to be involved in this process but it changes the very nature of participation. Participation becomes open to all and, more importantly, becomes something public to all.

Political participation must no longer confined to election cycles. Yes, election cycles need to play a role in our representative democracy, but we have technology that allows for something more engaging. Leveraging technologies of the real-time web politicians can present ideas to the public and receive immediate feedback. Furthermore, this garnering of feedback would be done with very little overhead. There would be no organizations that would have to mobilize, no buildings to rent, or speaking tours to arrange. The entire process could fit within a representatives current schedule and could take place from wherever a politician was at the moment.

Finally, political debates could use some recent conferences as a model and project a backchannel of discussion during sessions. This could bring a real-time stream of feedback into a legislative discussion. Particularly when combined with a live broadcast of the debate this method would allow for citizens to listen in on and speak up at important legislative events.

All of these potential avenues could be explored to accomplish a singular goal: reframe political participation as something that occurs in small pieces throughout the course of every day for every citizen. The technology has shown that there are millions of people who are willing to produce short pieces of information and convey brief opinions as a part of their everyday life. The only thing left is to incorporate this technology into our ideas of political participation.

  1. For an example of this type of communication see the following exchanges of messages on Twitter between Daniel Bachhuber, a 22 year-old entrepreneur, and Jay Rosen, a journalism professor at New York University. Jay posted a short message with a link to a longer article. Daniel posed a question in response to that post. Jay then proceeded to respond to Daniel’s question in two later posts. The entire conversation took place in less than 30 minutes.
  2. One recent example of this Quora, a real-time question and answer application that uses Tornado as its base.
  3. Jefferson, Thomas. The Life and Selected Writings of Thomas Jefferson. Ed. Adrienne Koch and William Peden. New York: The Modern Library, 2004. 574
  4. Jordan, Grant and William A. Maloney. Democracy and Interest Groups: Enhancing Participation? New York: Palgrave Macmillian, 2007. 1
  5. Agre asks for the copy of this essay that appeared in The Information Society journal to be cited but for reasons of accessibility I have cited the linked essay since it is freely available online.
  6. Shirky, Location 335-343.
  7. Shirky, Location 299-307.
  8. Downey, John. “Participation and/or Deliberation? The Internet as a Tool for Achieving Radical Democratic Aims.” Radical Democracy and the Internet. Ed. Lincoln Dahlberg and Eugenia Siapera. New York, Palgrave Macmillian, 2007. 111.

How we can participate

As technology and the tools for communicating online become more mature and accessible, some general trends emerge. First, with the rise of the real-time web and services like Twitter communication online happens faster. It is also coming from millions of individuals who can publish from nearly everywhere using the mobile capacities of WordPress, Twitter, and similar software. Finally, this increase in speed and quantity of communication fuels the development of sophisticated tools for aggregation and filtration of information flows. All of these tools are usable and deployable by individuals. What is possible with all this technology is a radical shift toward individual control and influence in political participation. Political participation need no longer be something mediated through interest groups and representatives. Instead, technology has allowed for the potential for individuals to play their part in the broader political arena.

We must keep in mind that as revolutionary as all of these technologies can seem they are no guarantee of expanded participation. Technology in and of itself does not determine politics. Application and adoption by citizens determines political impact. Yochai Benkler makes a similar point in the introduction to The Wealth of Networks when he writes that,

Neither deterministic nor wholly malleable, technology sets some parameters of individual and social action. It can make some actions, relationships, organizations, and institutions easier to pursue, and others harder.1

Communication platforms like the real-time web, Twitter, WordPress and consumption mechanisms like Google Reader and Fever drastically altered the parameters of potential individual and social action. Individual citizens have such significant opportunity at their fingertips that the boundaries of political participation have expanded significantly. None of this is assured, rather it is potential that we must put into political practice; however, some recent events can provide optimism.

The parameters that have been expanded through online communication are twofold. First, our ability for individual action and autonomy expands through these tools than previous methods of publication. Second, this individual autonomy allows for more independent group formation that maintains the identity of individual citizens.

Having the ability to publish to a potential global audience was something open to only a select few in a pre-internet age. To globally distribute information was something restricted to mass publishing houses and mainstream media publications. The democratization of publishing has changed this dynamic. If an individual wants a mass audience the potential exists to have one. The physical limitations of printing presses and capital resources to disseminate information have largely dissappeared. As Clay Shirky said at Web 2.0 Expo in 2008,

the internet introduced post-Gutenberg economics. The cost of publishing has fallen through the floor.

The availability of this type of publishing to individuals represents a remarkably political event. Through these tools individuals have tremendous power to publish their viewpoints and through software like Google Reader and Fever they can aggregate information from other individuals to find areas of common interest and shared concern. This ability to publish, aggregate, and organize presents a unique opportunity of group formation and mobilization online that is not the same under a traditional political system. Groups can come together as true collections of individuals who all have access to public-facing communication channels. Symbolic leaders are not needed to relay information and tell members what is important, this process can all be done by individuals.

Ultimately, though, as powerful as this technology is nothing will change by itself. In order for political participation to truly be revolutionized it will rely upon citizens taking advantage of the tools available to them and beginning to publish online and aggregate sources together into a personalized information flow. None of the potential matters if we, as political citizens, obstinately refuse to change our habits. If we continue to give precedence to organizations that do the aggregate, filter, and publish information for us then the potential of all these technologies disappears. However, if we decide to take individual ownership over the publication of our opinions and seek to construct personalized information streams, then the potential of these technologies will become fully realized in revolutionary political change. Through the political application of these technologies we have the ability to gain individual control over our information consumption and publication. We can organize rapidly as individual to undertake collective political action. Ultimately, we can transform political participation from a slow, occasional process that happens at the government’s convenience to something defined by small actions taken as part of a continual process that works toward iterative political change.

  1. Benkler, 17.

Making sense from the noise

As the technology to publish on the web becomes more streamlined and the process from brainstorm to publication shortens the ability to aggregate and filter will be immensely important. In order to make sense out of all this information published and distributed in real-time online we need a solid set of tools to filter the important information from the noise.

In a world with widely available online publishing tools we need ways of seeing relationships between data. From a political standpoint the ability to filter for relevant data and relationships is invaluable. We finally have tools developed enough to allow us to aggregate and filter information to find these important relationships. More importantly, the tools that we now possess are in the control of individuals who can now aggregate and filter their own personalized information flows. Among the many data aggregation tools out there two are particularly useful in their application to political participation: Google Reader and Fever.

The modern tools that have developed around aggregating and filtering data are tremendously flexible and powerful. These technologies empower more refined aggregation that allows for a more informed public. Information is the key to well formed political decisions and tools like Google Reader and Fever give the American public the opportunity to partake in more informed political participation.

The technology of aggregation

The mainstay of aggregating data online has, since its invention in the late 1990s, been Really Simple Syndication. This technology, commonly known as RSS, allows for users to automate the subscription process to streams of information online.1 Any site that provides an RSS feed of content can be subscribed to by a user using an RSS client like Google Reader or Fever. After subscribing all content feeds automatically update after publication.

This technology is already a default standard on news sites, blogging platforms, photography sites, and social applications like Twitter. Furthermore, it is an open technology that can be implemented free of charge which means there are more client applications than could be covered. Due to this massive number of client applications this essay will focus upon the two pieces of software that hold the greatest application to political participation in the United States: Google Reader and Fever.

Fever, an RSS reader designed and developed by Shaun Inman, was released in the summer of 2009, It took a different approach to reading information through RSS. While the previous paradigm of RSS subscriptions had been to treat them like an email inbox, Fever approaches the model from a different standpoint. Instead of displaying a list of unread items, similar to an inbox, that only disappear when all are read Fever allows for users to focus on just what is important to them while not feeling like they will miss other important news.

By moving past the inbox mindset of RSS reading, Fever changes the way users discover and read feeds on the web. Instead of having to make a choice between subscribing to a plethora of feeds (thus overwhelming themselves) or subscribing to a select few feeds (potentially missing important news), Fever approaches the problem by encouraging users to make a distinction between essential and supplementary information sources while still subscribing to both. Essential feeds are marked as “Kindling” while supplementary feeds are put into a “Sparks” folder.

These two designations of content are, together, the source of Fever’s most politically important aspect: the Hot List. Shaun Inman describes the Hot List on Fever’s site by saying that,

Fever reads your feeds and picks out the most frequently talked about links from a customizable time period. Unlike traditional aggregators, Fever works better the more feeds you follow.

Fever analyzes the links of sources in both the Kindling and Sparks folders. Through this analysis it then presents the most popular stories as determined by the sources one follows. This Hot List can be narrowed down to a range of days or for the most recent week. Furthermore, not only does it show the most linked to items but it also shows the originating sources for those items. Thus, it allows a user to see the most popular items and how they relate to the information sources that he or she follows.

While Google Reader does not provide the same type of personalized Hot List as Fever, it nonetheless represents an important web application for data aggregation. While Fever presents some compelling features, Google Reader is the dominant market leader for RSS readers. In February of 2007 this market share measured 59% of the web-based RSS reader market.

With this dominant market share Google has integrated a significant toolset of social features into Google Reader. Most important are the personalization features that have been built into Google Reader. The two main aspects of this are the recommendation engine and the individual social tools.

The most politically compelling feature of Google Reader is its recommendation engine. Upon launching the new feature Beverly Yang, a Google employee, wrote,

to make it easier to find interesting feeds, we’re moving recommendations into the new Explore section and giving it a new name — “Recommended sources.” Like always, it uses your Reader Trends and Web History (if you’re opted into Web History) to generate a list of feeds we think you might like.

Simply put, Google Reader has the ability to track a user’s reading habits and use that as the basis for suggesting additional content either popular across the web, or particularly interesting to that user’s interests. This takes the burden of categorizing information sources off of the user. Anyone can start with the information sources that they know they want to read and partially rely upon Google Reader to find similar sources from around the web.

Another important aspect of Google Reader’s features is its ability to share and recommend items to people within your existing social circle. Google Reader allows users to share articles with other users and to comment on these articles when doing so. This allows for users to discover new articles and new sources of information by either leveraging the algorithm behind Google’s recommendation engine or through their social network of friends, coworkers, and contacts.

One final feature important to keep in mind about Google Reader is the way it tracks a user’s reading habits and displays this data in accessible charts. These reading trends allow any user to automatically see which information sources garner their attention most consistently. Ultimately the ability to track reading habits without having to rely upon what one remembers reading can allow for a user to analyze their own reading habits and perhaps restructure their information intake accordingly.

Overall, two key aspects of Fever and Google Reader are important to keep in mind for a discussion their political potential. Fever provides analysis of information sources already familiar to a person. This allows them to sort through massive amounts of information and leverage an algorithm to filter for importance. Google Reader, then, provides the ability to use a social network of contacts as well as an algorithm to find new information flows and news items. Most importantly from a political standpoint, all of these technologies are able to used and structured by an individual user. Individuals determine the structure of an information flow in both Fever and Google Reader.

Translating aggregation to politics

In a political system grounded in the involvement of the general populace the relative education and knowledge of the citizenry is crucial to the legitimacy of the political decisions made. Thomas Jefferson recognized this when writing “A Bill For The More General Diffusion of Knowledge.” In this he writes that,

even under the best forms, those entrusted with power have, in time, and by slow operations, perverted it into tyranny; and it is believed that the most effectual means of preventing this would be, to illuminate, as far as practicable, the minds of the people at large.

Jefferson saw the education of the citizenry as a foundational guard against the threat of tyranny. Having what he terms an illuminated citizenry is the best protection against undemocratic elements that would oppress a people. Having the ability to aggregate information sources together using tools like Google Reader and Fever then provides many opportunities for citizens to take the initiative and inform themselves of political matters. The test of how well these technologies expand avenues for political participation, however, relies heavily upon the ability of users to filter out the important information from the irrelevant noise.

In order for filtering through online applications like Fever and Google Reader to be more effective at informing the citizenry than traditional media we need a solid technological response to an online world in which the many can publish. In The Wealth of Networks Yochai Benkler refers to early critiques of the democratizing effects of information online by describing the Babel objection. In Benkler’s words,

According to the Babel objection, when everyone can speak, no one can be heard, and we devolve either to a cacophony or to the reemergence of money as the distinguishing factor between statements that are heard and those that wallow in obscurity.2

A cacophony in which all publish but none consume would certainly represent an issue for a political system grounded in common movements for change. In a democratic political structure where majorities, whether they be citizens or representatives, determine decisions the ability of people to converse with one another about topics of mutual concern is paramount. In order for this conversation to happen we need some element of information consumption; people need to have consumed information to have a basis for discussion. In short, we need consumers of information and sharers of information just as much as we need the publishers.

Despite the critical importance of ensuring the consumption of information, Benkler views the cacophony of an open publishing world as not inherently debilitating. He writes that,

The Babel objection may give us good reason to pause before we celebrate the networked information economy, but it does not provide us with reasons to celebrate the autonomy effects of the industrial information economy.3

While the concern over a world of pure projection is important it should not be so debilitating that it makes political life in the United States complacent. In a talk at the Web 2.0 Expo in New York in 2008 Clay Shirky points to those proclaiming the disastrous effects of a world in which everybody speaks as missing the more significant, underlying problem: the filter. In this talk he says that,

Thinking about information overload isn’t actually describing the problem and thinking about filter failure is.

In an online reality where the many possess the ability to publish the weight of keeping the public informed and, in Jefferson’s words, politically effectual is not a problem of too much information, rather it is a problem with a filter that is not refined enough. With Google Reader and Fever we have tools that become increasingly more effective at filtering out the noise in information streams and allowing citizens to more efficiently stay informed. By leveraging the recommendations detailed above, Fever’s link analysis or Google Reader’s socially-powered recommendations, it becomes easier for people to take in a wide array of information and filter it for the most politically important and relevant material.

A final aspect of aggregation technology that improves the potential avenues of political participation is the way that Google Reader, Fever, and any other RSS reader pools together various sources into a single, contained information flow. In an information economy where print signifies the most technological option available information comes to people in distinct packets. The raw form can come in primary source documents that present something closer to plain data while the aggregation happens through secondary source material where the author must carefully aggregate and curate the information referenced. Data aggregation through RSS places this power in the hands of the people. Any individual can subscribe to feeds of various news or data sources and have all of that information flow into a central repository. Lastly, when combined with the type of recommendations and social features from Google Reader or the Hot List feature of Fever this places the power of finding relationships between information sources in the control of an individual. Each individual who uses a tool of data aggregation can compare various sources to find similarities, differences, and contradictions. Ultimately it greatly increases individual autonomy by allowing each person to receive information straight from the source and serve as their own filter without having to trust in the filtering abilities of a third party.

Aggregated Participation

Aggregated raw information and information recommendations from a personal social circle and a refined algorithm expand the potential information to which each person has access. Individuals can now personalize an information flow and leverage their wide ranging social networks, as well as the skills of software developers, to find additional sources of data. All of this serves an important function for expanding access to data and information, both of which serve as key foundational elements to political participation in the United States. These technologies can provide something greater though as well. Not only can they aggregate information and increase the information flows used as a basis for participation but they can also aggregate and filter participation.

The acts of political participation occurring through the real-time web combine with the maturation of self-publishing tools and advanced aggregation technology to provide a powerful redefinition of public political participation. This definition hinges upon individual activity aggregated in the collective to find relationships between opinions that help spur political action.

The most important aspect of being able to aggregate expressed political opinions using these technologies is the accessibility of these tools to the individual. As Dave Winer, one of the founding developers behind RSS, says in the aforementioned BBC article,

RSS makes it possible for information to flow to you.

The content of this information can be anything but ultimately it all flows to a single individual who then can make judgements based upon it. This is not political information streaming through an interest group’s filter. It is not news being restricted to what passes the mainstream media filter. It is not political opinion coming down to citizens from a presumably trustworthy representative. Rather, it is information flowing directly to those who ultimately have to make political decisions: individuals.

The independence of individual judgement and information intake is important for participation to effectively be extended to a mass of individual citizens. By allowing individuals to aggregate their own information and come to their own judgements concerning it the political system can become reflective of what individuals actually desire from their government. Instead of relying upon a mass media outlet to aggregate and curate information users can now use something like Fever’s Hot List to independently view what the important stories of the day, week, or month are. Furthermore, with all of the social features packaged into Google Reader any user of that RSS reader could quickly and easily leverage his or her existing social network to inform their decisions.

Finally, all of this individual control over information consumption contributes to a political arena in which individuals are actually informed about the political actions they take. Information plays a central role in political actions. Whether that action is organizing a movement for political change, voting for a candidate, or simply discussing political issues over with family members information lies at the center of everything. The ways in which individuals gather and sort this information signals an inherently political act. That information informs political participation and discussion.

By placing the onus of informed participation upon the individual these tools allow for individuals to come to their own conclusions concerning information. A greater reliance upon a personally customized river of information means that individuals can come to rely upon other packaged versions of information less. By cutting out a middle step of interpretation individuals can learn to process ideas and knowledge themselves in such a way that it informs their participation in politics.

For a democratically-based political system like the United States the freedom and independence of individuals is vitally important. Modern tools of data aggregation like Fever and Google Reader provide the ability for people to take a faster flowing stream of data published by the many and turn it into their own, independent source for informed participation.

  1. This process is effectively explained in a 2005 article published by the BBC.
  2. Benkler, 10.
  3. Benkler, 171.

A design critique of Publish2

I have written about Publish2 before but if you are not familiar with it its a great service that allows users to curate and share the best of the news they read from around the web. The sort of cliché way to think about it is a Delicious for journalism.

As great a tool as Publish2 is I have always been frustrated by its design. It has always felt stuck somewhere between the late 1990s and early 2000s where web design was in its infancy. Being that it is an entirely web-based tool for journalism I was also a bit boggled at how its design lagged behind other web-based applications. I finally got a chance to put some of my thoughts about it down though so what follows is a bit of a design critique.

In short, I see Publish2 as lacking in two key areas: user interface and site design. These two aspects are closely linked but for the purposes of analysis I will treat them as separate. First, some specifics. Publish2 has a nifty bookmarklet that allows users to add links to the system from anywhere. This is where I will focus my critique of the user interface because it is the tool that I have the most interaction with. The site design was something that was recently updated, at least for the homepage. While it is a definite improvement it is still lacking in my eyes.

Interacting with the bookmarklet

The bookmarklet is perhaps my greatest frustration. Every time I load it I wonder if it is really even worth the effort to share anything. In an age when so many web services make forms fun to fill out this just makes me think of standing in line with forms at the DMV or filing my taxes.

There are so many ways to enhance forms with jQuery, Uniform for example, and other technologies that it boggles my mind how this bookmarklet, the main source of interaction with Publish2 that most users have, is so completely undesigned.

When I started using Publish2 I was not really sure where to start with the bookmarklet. The various elements are not really given any sense of priority, besides being stacked one on top of the other. No field seems particularly necessary to fill out and perhaps that is why so many of the links on the newswire look like those that I was frustrated with back in June.

The newswire and collaborative elements of Publish2 all rely upon users inputting detailed data and yet the method of getting that information in is filled with friction.

The homepage

The homepage of Publish2 (which, frustratingly, can only be accessed when logged out) recently got quite an update and while it is a massive improvement upon the old one it is still lacking in a lot of areas.

First, my biggest complaint is the sheer amount of text on the page. If I am a journalist looking to use Publish2 where do I go to see how it works? All of the links on the Journalism half of the page (more on that in a minute) point me toward examples of how data looks coming out of the software but a prospective user needs to be able to see how they interact with the tools.

On a side note I am a little confused by the new homepage. Is Publish2 focusing on tools for individual journalists and news organizations or marketers? Maybe there is not yet a focus? Are the links that I put into the system going to be used as a way to drive marketers to the service? Either way it is unclear to me what the focus is with the way Journalism and Marketing are given equal weight on the homepage.

Finally, both of the above critiques get at a central point that I still feel is lacking with Publish2’s recent design updates: I am still not sure what the focus of the software is. With an application that provides for everything from link sharing to newsgroup collaboration to WordPress plugins a driving focus is essential.

Emphasizing certain aspects of the software makes it easier for users to understand and ultimately helps them use a tool like the bookmarklet. Both of those things help improve the end product.

…and the rest of the site

The last bit of the site design is perhaps the most puzzling. Like I mentioned above you can only see the main homepage when logged out. All logged in users are redirected to their profile pages which consist of the most recent links they have saved. Here is the confusing part: why in the world do I want to see my personal links upon logging in?

This is akin to logging into Twitter and seeing nothing but what I have posted. That is not at all why Twitter is useful and the links I save to Publish2 is not at all the reason why I want to be using it. Seeing a list of my links does not promote collaboration; instead, it makes me think of myself, not others. I want to see a newswire of my network links. Where do I find that in the current design? All the way at the bottom of the page.

Approaches that work

There are no shortage of web applications out there that have taken complex problems and turned them into simple solutions. Just take a look at Droplr, Dropbox, or Tumblr for an example of what I mean.

Notice anything similar about all of those successful applications? They all present a clear and concise description of what they do. Hell, Tumblr boils their homepage down to one sentence. I can already see a Publish2 homepage adopting that model. Publish2: The best way to find, save, and share news.

Droplr and Dropbox even provide videos to introduce users to how the software works and what it could do for them. They urge people to start envisioning themselves as users. Most importantly, they keep the focus on what the user is doing and how they are interacting with the application.

Furthermore, they all have boiled their user interfaces down to the essentials. It is very clear what I need to interact with to make effective use of those web apps. Droplr, Dropbox, and Tumblr all present polished user experiences that make it so easy to share files, keep things in sync, or publish online.

Ultimately, Publish2 is news software that isn’t acting as software. The way I see it, the power of Publish2 lies in the data that users put into the system and yet so much of the service puts the focus on the data that comes out of the software. If it really wants to be powering the next generation of news (like its homepage claims) then it is crucial to get people actually using it and using it effectively. The way to do this is to present a simple, clear message. Show people how your software works. Let them envision themselves as users.