TOPP Labs
Open Source Civic Hacking @ The Open Planning Project
The Civic Hacker
A Community Blog
There are a number of sites now that invite “participation” by proposing or collecting ideas and having the “public” vote on those ideas.
The supposition here is that there is a politician somewhere just desperate to base decisions not on their own belief and intuition, not even on general surveys of public opinion, but to base decisions on the preferences of a self-selected group of people who invested enough time to hit an up or down arrow on a site.
In another post I’d like to offer another rational (but cynical) reason for building these voting systems, but for now I’ll take them at face value.
Take, for example, Stimulus Watch. This is a recently built site that has data on the projects proposed for the upcoming stimulus bill. It has a large list of projects with the money required for the project and the estimated number of jobs created. Where it gets interesting is that it also involves collaborative research, where people can post details about a proposal, and argue points for and against items. People have found newspaper articles, even apparently called the individuals involved in the proposals to get more information. Very useful!
The site has two other notable components: comments and voting.
Let’s look at a proposal that has received a surprising amount of attention: Doorbells. It currently is voted at -7858 and has 249 comments. I think this page is a good demonstration of the problem of shallow involvement.
Actually reading the proposal I think it’s quite reasonable. It’s a fairly small item (just about $100k), it’s job-to-cost ratio is quite good ($50k/job). The actual proposal is to install doorbells in a federally-funded elderly housing facility, including doorbells on the inside of units that the residents can use to call for help. Someone noted in the description that at $155/doorbell that it might be expensive, but honestly this seems reasonable to me depending on the details of the project. The whole proposal seems fine. To voters it’s apparently one of the worst proposals out there. To commenters… well, a few examples:
My doorbell is busted, can I get $100,000 to fix it?
Zing! In case you were wondering, this is about as funny as the comments get.
Doorbells?? Why is this called Stimulus? How is this going to help our economy? This is just more idiotic, wasteful government SPENDING. That is all the government knows how to do – spend our hard earned tax dollars (that they extort from each one of us) on projects that do not generate any revenue. Why can’t the government JUST focus on U.S. infrastructure and security? If they could leave everything else to the private sector, our taxes would be manageable and our economy would not be in this mess!
I am going to send this website to all of my friends that are just as irritated by all this HOAX as I am.
Important political analysis of how this fits into a larger context! Of course probably this work would be performed by a private company, so technically the analysis is incorrect, but that probably wasn’t the point of the comment.
Individuals can buy doorbells for themselves if they want doorbells. No real jobs created.
Indeed, let the old people pull themselves up by their bootstraps!
Hey underprivileged, go to home depot and pay the $20 and spend the afternoon installing like the rest of us working folks. Put down the crack pipe and the 40 oz.
Fair and balanced! I can sense the country becoming post-racial right in front of us.
Of course I don’t (in any way) blame Stimulus Watch for the content of these comments. If I was genuinely offended or even bothered by these comments, it would be a sign that I hadn’t spent much time on the internet. It’s just disappointing… the voting and the comments distract from the good stuff Stimulus Watch is doing. It also means that in a page sorted by activity this stupid Doorbell debate hovers at the top while a huge number of other proposals remain empty and under-described. The voting in turn feeds nonsense indignation, instead of people actually coming up with a thoughtful analysis (and very possibly thoughtful indignation).
There’s tremendous potential if Stimulus Watch can bring together people to do research on these proposals and to help identify the best and worst proposals. But by engaging people in trivial ways I think it is working against itself. Even the Doorbell item could be usefully expanded. What kind of building are they installing doorbells into? Are there special requirements for the hardware or installation? But leaving a comment on that item would be useless, it will be lost in a stream of pointlessness.
Something more like Wikipedia might be a better model (even though Stimulus Watch actually uses the Wikipedia software as one of its components). The site already adopts Wikipedia’s standard of a neutral point of view. This represents a kind of consensus, achieved through iterations of editing. Discussion also exists, but in Wikipedia it is relegated to a secondary page. The discussion can also be edited, and is actually a bit awkward to participate in. Discussion is really for people that have tried editing but need further direct communication to move that forward. If it isn’t relevant to editing the main article, the discussion is useless and not appreciated. And there is no voting. Let’s make more of that.
+1
I wonder how the blame breaks down here between process (voting, shallow participation options) and content. The doorbells proposal is a close relative of the “bikeshed” in software development – it’s such a simple idea (with a non-astronomical price tag) that anyone feels qualified to comment, so it becomes an especially attractive place for people to get on their soapbox. Wikipedia has outlier “lightning-rod” pages, too.
There are enough different factors here that I wonder where “voting sucks” sits on the list of top take-home points. (So yeah, I’m building a site – carrotproject.com – that includes voting as one participation channel.) There’s putting enough of a roadblock in front of potential users to screen out some proportion of those with no intention of constructive involvement: Metafilter costs $5, stackoverflow.com requires a minimum karma level to vote, Stimulus Watch doesn’t even require that you register. Also the question of whether you’ve got enough of a community that cares in the first place to apply significant social pressure against useless behaviors (the prerequisite for a functioning wiki).
@Carl – absolutely people hooked onto that post because it seemed easy to understand, with an obvious name and a seemingly large price. There’s lots of less simple proposals that aren’t getting any notice at all.
Now, of course it would be terribly hypocritical of me if I was to, say, lead with a sensational headline when there’s more going on here than just voting. But, while I couldn’t *possibly* have overstated my case in the title, let’s imagine maybe I did… and maybe the more measured statement would be “collecting votes without knowing what those votes are worth isn’t smart”. No one is at a loss for opinions on the internet. At least with a comment there can be some clear intrinsic value (or lack thereof) in the comment — it’s often fairly obvious whether the comment is dumb or not. A vote usually has no clear way to value it. You can’t tell if it was valid at all (fraudulent internet voting being common), or if the person read through the material before voting, or if the person actually cares about what you are talking about, and there’s no way to weigh people’s input into any of this. Metafilter and stackoverflow and some other sites ensure at least some quality of input. Open (not anonymous) voting is another way to retroactively apply weighting without putting up barriers. There’s probably other good ideas, and using them I agree voting doesn’t have to suck.
So when we started Stimulus Watch, I loaded up all of the projects and stubbed each one with a little template wiki page. It was otherwise a blank slate. Soon after starting it, the infamous “Doorbells” project rocketed up seemingly out of nowhere. It had no information at all other than the title and the cost.
Two things happened at this point. First, since it’s very easy to vote on the site, the Doorbells project started getting lots of negative votes. Someone dug it up out of the 18000 projects and to most people it must have sounded fishy. Second, people actually researched it and put meaningful detail into the wiki page. So I was impressed that A) someone found it and B) some more people cared to document it. It’s unfortunate that it spent some time completely unresearched gaining negative attention. Upon reflection, I wonder if we should have limited voting before any meaningful content had been posted to the corresponding wiki page. I don’t know if that would have reduced or increased the participation, but it would have meant (potentially) more informed voting.
Another way to raise the bar to trivial (and less-productive) participation might be to allow users to weight their own votes. Users would gain points / creds / whateverterm by engaging more deeply – filling out additional information, fleshing out pros and cons for projects, or other behavior you want to encourage.
Users could still express their opinions on a project without additional effort, but you’d be less likely to have skewed results from drive-by posters.
Comments are tougher. I think Ian might be onto something with de-emphasizing comments. The last thing StimulusWatch needs is to have their comments degenerate into YouTube-caliber name-calling.
A few thoughts:
Maybe Stimulus Watch could have had a vote along the lines of “I think this item deserves attention”. There would be no down vote. Doorbells could have attention brought, and then people can add the description. Maybe the importance of those votes would decay after an edit (if people didn’t keep voting up Doorbells after an explanation, maybe it’s been explained). Maybe Reddit’s vote scoring would be a fit (it has a time component). For better or worse, that kind of vote wouldn’t be as satisfying or attractive as an up/down vote.
Another improvement might have been better (or worse ;) sorting — bringing items of interest to people’s attention, but with enough randomness that other items can float into view, at least occasionally. With a big database of items, as soon as anything starts floating to the top, most of the stuff is stuck in a boring middle.
For comments I think different standards might accomplish this. I was talking to a guy at PyCon (wish I could remember who, but I talked to too many people to keep it straight) about comments, and we (or maybe he, again I can’t remember ;) came up with the idea of a “citation needed” button on comments. It’d be like “flag as inappropriate”. Once the citation challenge came up, the person should respond and edit their comment to cite something, or delete their comment (something that actually gets you a badge on stackoverflow!), or maybe it’d expire and the comment would be hidden. Obviously moderation would be appropriate. And if people aren’t willing to follow up with their comments, then there’s no reason anyone else should bother reading them.
A lot of the issued raised here seem to point toward the potential for inline commenting/annotation – http://ideas.topplabs.org/wiki/Collaborative_Annotation
The annotation model intrinsically requires at least some articulate reading of the content that’s being addressed. It therefore also provides contextual relevance and the highlighting “heat-map” displays this pretty clearly. This model certainly makes a lot more sense to me than MediaWiki’s discussion pages.
[...] Take Stimulus Watch, a great project which furthers these two goals, but—as Ian points out in his recent post— assumes that politicians will take the initiative to visit the site and base their decisions on [...]