How do Wikipedia’s processes for creating and modifying articles ever lead to high-quality results? In other words, since anyone can easily edit Wikipedia, how is it that good (and usually accurate) content emerges?
The case doesn't present a proper mechanism to quantify a reasonably accurate answer to the question posited. Anecdotal evidence of error rates in one study on a minute portion of the sites articles (42 of over 3 million) is not statistically significant. Though the idea of collective wisdom has power that perhaps wikipedia has developed a process of "quality" that is greater than traditional authoritative hierarchies cannot be discounted. They say that journalism represents a "first draft of history." The more important question here is: "Has wikipedia designed a process that represents a "continual draft" of history."
To be sure, the model has advantages over the traditional storage of mankind's knowledge. 1. It can't be easily lost (i.e. the Library of Alexandria) 2. It's less likely to be biased by the original writers of "history" (i.e. "the winners) 3. It can be corrected quickly, cheaply and continually. There are more major benefits. The trade-offs of accuracy vs cost/ speed of value delivered to reader cannot be ignored.
The marginally lower costs of quality (if they are in fact borne out by more studies) can be easily mitigated. Some articles rate of change (velocity) are bound to lower than others. A simple visual cue to discern the difference could easily add value to readers. Perhaps a "green" color for articles about less contentious subjects, like the Periodic Table or the Arabic Numeral System (which is really Hindu) Presumably, this would be less volatile than the recent Sendai Earthquake in Japan (pun intended) - that could be colored "Red". In the former, the debates are limited to highly technical matters. In the latter, the information is more fluid and dynamic. Does that mean it's not worth capturing? In fact, some subjects are much more inclined to have embellishment over others. Subjective topics such as music, movies, books, etc... will always be based more on opinion than "fact." Eventually mistakes will be weeded out and reflected in a reduction in rate of article change. Quantifying that in effect "qualifies" the article.
The allusion to a "Bolshevik Soviet" system smacked of an overly-socialist inclination bias that marked this article of lesser quality. The evidence of an equally brutal, bureaucratic system was weak. I've been an infrequent editor of wikipedia over the past 5 years. Though my contributions are mostly frivolous, my non-scientific test was to see how long egregious comments would be removed (in MOST cases VERY quickly) Though I have learned if a user is only making stupid comments those are easy to identify and quickly remove with scanning technology. SO I've adapted and occasionally make some meaningful edits (mostly deleting irrelevant info) Find me a system that ISN'T trying to be gamed, be it overly academic or free to the "common man" to alter.
If we stick to the anecdotal approach to answering the question posited I'll present the following evidence. I've run a football pool that directly measures "collective" intelligence both BEFORE and DURING the entire NFL season. I've done this for over 15 years and have over 23,000 data points over that span. In one game, players pick every football game for the entire season before a single ball is kicked. The "collective wisdom" or accuracy of those picks is not only better before a ball is kicked, it gets better as the season progresses, injuries and poor performances apparently insignificant. It's not always right and doesn't beat the "best" player or win the pool. But it's usually better than 80% of the "NFL experts" and has an impressive track record for a crude, limited version of a "crowd-sourcing" model.
There's a larger irony here to point out. The insulting arrogance of the Ivory Tower (those Phds that produced 12 articles in 18 months on Nupedia) is rooted in the simple danger that the transparency of wikipedia marginalizes their expensively acquired efforts. A distributed knowledge creation system may just be more efficient than "traditional" higher-education systems (no, it doesn't replace the need for doctorates, just diminishes and is better suited for some categories, not all) Given this phenomenon the cost / benefit for higher education is a critically re-framed question for those most heavily vested in the process. OF COURSE they will complain on marginal quality over marginal cost. Anything to challenge the unsustainable increases in tuition costs represents a direct threat to their way of life. They should be worried Zuckerburg dropped out of Harvard. More worried that there is no monetization model for wikipedia. The marginal knowledge added by this select, overly-qualified community is diminishing. Data in social networks much more powerful. But that's a question for next week's case. :)
I think your opening sentence hints at a central problem of quantifying, and then comparing & contrasting accuracy. George Will famously said that he would rather be ruled by the first 2000 names in a phonebook than by the entire Harvard faculty, arguing that the former would yield superior policies. There is certainly a case to be made about the superiority of collective wisdom over authoritative knowledge in the realm of statecraft, but does the superiority of collective wisdom in discerning the best policies extend to discerning facts? Political Science is more of an art than a science, since pubic policy is generally not conducive to rigorous mathematical proofs, e.g. you can't "prove" that a gas tax hike is "bad" policy, you can merely argue its various pros and cons. On the other hand, historical events are generally facual. We can say with certainty that Hitler invaded Poland in 1939 or that Donald Rumsfeld was SecDef under Ford. Such facts don't need to be validated by the authority of collective wisdom to withstand scrutiny, as say a public policy debate might. To establish accuracy in such uncontroversial matters, it is sufficient for a journalist or a historian to state and source a historic fact. And until Wiki came on the scene, it was conventional wisdom that the art of ascertaining, documenting and verifying facts is best left to the experts, not the collective. The fact that Wikipedia has apparently shattered this theory is indeed a phenomenon, if a poorly understood one.
ReplyDeleteWikipedia has long ticked me off with it's various mistakes. But this is a model where one can take direct action to correct "mistakes" For a long time, before even being nominated to the Football Hall of Fame, Rod Woodson was described as a "lock in the Hall." However one thought about Mr Woodson's prospects to make it, it certainly wasn't a fact, but a prediction. Though he did in fact make the HOF, it was not fact at the time. one can only assume that a bitter Patriot fan or some other jealous type properly removed the reference.
ReplyDeleteI would link to the bad reference, but it seems wikipedia doesn't have an easy way to search history. A feature it might consider adding in the future.
Another neat feature would be automatic updates to articles based on time. Many articles post the age of a person, or the relative cost of a commodity (adjusted for inflation) the updating of these trivial items is both tedious and time consuming. This auto-updates for wikipedia. This would obviously have to be controlled by the administrators (not simply editors)
ReplyDeleteAlso, defining a common layout / templates for similar types of information. For example, biographies would all have same look/feel. Data that was missing would be marked clearly and allow other people to fill it in. This is just one small example.
Regarding the Ivory Tower's hatred of their control of knowledge. Is it too far a stretch to liken their obsessive control of "knowledge" to the Vatican's control of teaching literacy? Surely most would agree that the spread of literacy was beneficial for all mankind (at the cost of a decreasing relevance of the Church) Giving wikipedia control to the "heathen" mindless, uneducated masses scares them since it makes their societal function less relevant. Ah, how the Ivory Tower crumbles........
I don't know if this is necessarily a question of marginal cost vs. marginal quality. In my view, Wikipedia is simply another form of expression in the marketplace of ideas. Censoring or condemning Wikipedia because of "marginal quality" also runs the risk of violating free speech. Rajwoodson make the argument that "eventually mistakes (on Wikipedia edits) will be weeded out and reflected in a reduction in rate of article change." Therefore, it's up to the reader to make a judgement on the veracity of the edits.
ReplyDeleteHere's an example of an automated wiki tool: http://en.wikipedia.org/wiki/WikiScanner
ReplyDeleteAs a member of your football pool, I will argue that it is possibly statistically insignificant, definitely has a sampling error, and likely has significant overconfidence in the members ability to predict results.
ReplyDeleteAn interesting argument against ivory towers, or a case of buyers remorse?
Any way you cut it, wikipedia remains my World Book of mostly facts and I just aspire to the day where I can become judged 'significant' by the wikicommunity.
I did note that the football pool was anecdotal evidence. However I think in your case it also represents some buyers remorse! ha! It seems there is an opportunity for you to contribute as well. There is a dearth of wiki articles on survivor strategy. Time to add an entry for the $teel Curtain?
ReplyDelete