Wikipedia Watch: Follow-Up to the Michael Scott Endorsement
Office' fans flock to edit Wikipedia
By ANICK JESDANUN, AP Internet WriterWed Apr 11, 5:37 PM ET
In the NBC series "The Office," the boss Michael Scott turned to Wikipedia for tips on fending off an employee's request for a pay raise. Viewers quickly flocked to the online encyclopedia and added their take to its entry on negotiations.
Administrators at Wikipedia had to limit editing of the entry, most recently late Tuesday, placing it in "semi-protection" mode. That meant users couldn't make changes anonymously or from accounts fewer than four days old — to discourage those drawn to the site specifically because of the broadcast.
The site imposed similar restrictions on the entry twice before, only to see vandalism continue after they were lifted.
Wikipedia is a collaborative reference site where anyone can add, change or even delete entries, regardless of expertise. The thinking is that the collective wisdom results in a better product overall, and members of the community can watch for any vandalism and reverse it.
In the case of the "negotiation" entry, viewers quickly added phony tips in response to clueless advice from Scott, played by Steve Carell, in last week's episode.
One edit simply replaced the entry with a statement praising the television program. That was followed by the insertion of Scott's tips for getting the upper hand, including "suddenly changing the location" and "refusing to talk first."
Users made more than 100 changes, including those to reverse the vandalism, before the site imposed the latest restrictions on revisions.
Wikipedia does face vandalism from time to time as a result of high-profile mentions.
Fans of Stephen Colbert's Comedy Central show "The Colbert Report" flocked to Wikipedia to alter articles on elephants after he said on the program, "all we need to do is convince a majority of people that some factoid is true — for instance, that Africa has more elephants today than it did 10 years ago."
Changes aren't always noticed and fixed immediately.
In late 2005, prominent journalist John Seigenthaler, the former publisher of the Tennessean newspaper and founding editorial director of USA Today, revealed that a Wikipedia entry that ran for four months had incorrectly named him as a longtime suspect in the assassinations of President John F. Kennedy and his brother Robert.
The last paragraph says it all. And people wonder why I don't take the Wikipedia seriously?
Wikipedia Watch: The Michael Scott Endorsement
Wikipedia Watch: Editor Who Posed As Professor Is Kentucky Dropout
Above Cartoon by Peter Steiner. The New Yorker, July 5, 1993 issue (Vol.69, no. 20) page 61
From today's Courier Journal (Louisville, Kentucky):
He touted himself as a tenured professor with doctorates in theology and canon law.
But the volunteer editor and fact checker for the free online encyclopedia Wikipedia turned out to be a 24-year-old dropout from Centre College and Lexington Community College.
In a long feature last July, The New Yorker described how Ryan Jordan -- identified online and in the magazine article as Essjay -- was renowned for rooting out errors and obscenity from Wikipedia, whose entries are contributed and corrected by readers and volunteer editors.
Last week, however, after Jordan's credentials were exposed as fictitious, the magazine ran a correction -- and yesterday Jordan, who had written or edited about 16,000 entries, resigned.
Wikipedia has said no questions have been raised about the accuracy of his work.
Critics of Wikipedia said the deception was fitting, given the site's history of errors that include reporting that the prime minister of Norway was a pedophile.
Devoted contributors, including Louisville computer programmer Steve Magruder, who has written many entries about the city, said they feared Essjay's deceptions could damage the encyclopedia's credibility. [Rick chuckles..."You think?"]
Another Wikipedian, as they call themselves, denounced Jordan last week on the site, saying: "I hope you understand how you are –––– all over the thousands of people who have made a real effort to turn Wikipedia into a credible source."
Responding on his Wikipedia page, Jordan initially defended his deceptions by saying he had to protect himself from online stalkers. He told The New Yorker he routinely got death threats from people he banned from the site.
Wikipedia's co-founder, Jimmy Wales, also said at first on the site that he had no problem with Jordan's invented persona.
But yesterday, Wales asked for Jordan's resignation.
Read the entire article by Andrew Wolfson.
Ahh... the democratization of knowledge...
You know the issue is not that Jordan is a dropout. There are lots of intelligent, well-educated, successful folks who never finished college (consider Bill Gates, Steve Jobs, and many of my friends and family). No, the fact that Jordan is a dropout only adds to the irony that he was impersonating a tenured professor with multiple doctorates. And this adds to the growing concern from myself and others regarding the use of the Wikipedia as a serious source of information (I think it's great for pop-culture references).
And even if Jordan's work was 100% accurate, his subterfuge calls everything he's written/edited on the Wikipedia into question. Someone will have to go behind him to verify his work. He was responsible for cleaning up the nonsense that any bozo can add/edit on the Wikipedia. But you'd think they at least check out the checkers!
I can guarantee this wouldn't happen at Britannica.com...
Wikipedia Watch: Not Satisfactory for College Papers
College: Wikipedia Not Source for Papers
Middlebury College in Vermont Says History Students May Not Cite Wikipedia As a Source
The Associated Press
MIDDLEBURY, Vt. - Middlebury College history students are no longer allowed to use Wikipedia in preparing class papers.
The school's history department recently adopted a policy that says it's OK to consult the popular online encyclopedia, but that it can't be cited as an authoritative source by students.
The policy says, in part, "Wikipedia is not an acceptable citation, even though it may lead one to a citable source."
History professor Neil Waters says Wikipedia is an ideal place to start research but an unacceptable way to end it.
Copyright 2007 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten, or redistributed.
Copyright © 2007 ABC News Internet Ventures
I know that I'm often hard on the "democritization/relativization of knowledge" inherent in the Wikipedia. But in reality, Middlebury College's decision isn't really so much a slam against the Wikipedia as a reaffirmation that encyclopedic references are not suitable for college papers. Any encyclopedia, whether the Wikipedia or Britannica are beginning sources for research that should point to other, more detailed information that would be appropriate in a college paper.
When I was in college, I would have ever used an encyclopedia as a reference, but I often I did stop there first in my research. Could it be that the internet has just made some students a bit lazy?
Irony of Ironies: Britannica Won't Function on My New MacBook
In the end, I only had to reinstall two programs: Microsoft Office 2004 (I was getting an error message about missing files even though the software would function), and Symantec Antivirus (I know, I know--no viruses on a Mac; I'm just paranoid after working in the past as a system administrator).
Criminy. Don't tell David Ker about any of this.
Well, I've already removed the Britannica software from my Applications folder. There's no need for it to take up space if it can't be used. I'll dig through my CD's later. If I remember correctly, the CD I have can be loaded in either Windows or Mac OS X, and I did install Parallels Desktop yesterday to run the occasional Windows program (Parallels is much faster than VirtualPC ever was, I might add). But I don't know. Having to run Britannica in Windows somehow takes all the fun out of everything.
At least I still have my hardbound set which is platform independent.
Related Reading (in case you're just tuning in and don't know why this is ironic):
Is the Wikipedia the Hitchhiker's Guide to the Galaxy?
The Sum of Human Knowledge
Why Britannica Trumps the Wikipedia
The Wikipedia Is Not Enough
Martin Luther Was Excommunicated on this day and Why I Love the Encyclopedia Britannica
Wiki-FoxTrot
Is the Wikipedia the Hitchhiker's Guide to the Galaxy?
From The Hitchhiker's Guide to the Galaxy (that is, the book by Douglas Adams), ch. 2:
Here's what the Encyclopedia Galactica has to say about alcohol. It says that alcohol is a colorless volatile liquid formed by the fermentation of sugars and also notes its intoxicating effect on certain carbon-based life forms.
The Hitchhiker's Guide to the Galaxy also mentions alcohol. It says that the best drink in existence is the Pan Galactic Gargle Blaster.
It says that the effect of drinking a Pan Galactic Gargle Blaster is like having your brains smashed out by a slice of lemon wrapped round a large gold brick.
The Guide also tells you on which planets the best Pan Galactic Gargle Blasters are mixed, how much you can expect to pay for one and what voluntary organizations exist to help you rehabilitate afterward.
The Guide even tells you how you can mix one yourself.
Take the juice from one bottle of the Ol' Janx Spirit, it says.
Pour into it one measure of water from the seas of Santraginus V--Oh, that Santraginean seawater, it says. Oh, those Santraginean fish!
Allow three cubes of Arcturan Mega-gin to melt into the mixture (it must be properly iced or the benzine is lost).
Allow four liters of Fallian marsh gas to bubble through it, in memory of all those happy bikers who have died of pleasure in the Marshes of Fallia.
Over the back of a silver spoon, float a measure of Qualactin Hypermint extract, redolent of all the beady odors of the dark Qualactin Zones, subtle, sweet and mystic.
Drop in the tooth of an Algolian Suntiger. Watch it disolve, spreading the fires of the Algolian Suns deep into the heart of the drink.
Sprinkle Zamphuor.
Add an olive.
Drink...but...very carefully...
The Hitchhiker's Guide to the Galaxy sells rather better than the Encyclopedia Galactica.
Now hold that thought.
For those of you just tuning in, this post is part of an ongoing conversation between myself and David Ker (host of Lingamish) regarding the benefits and limitations of encyclopedias such as the Wikipedia and Encyclopedia Britannica. David is bonkers over the Wikipedia. I'm much more trusting of the kind of editorial controls found in a more traditional source like Britannica.
Having said that, I use the Wikipedia regularly. I have multiple links to it here on my blog. Encyclopedias are the starting point of research, and as such, both the Wikipedia and Britannica are good sources of information. But in the end, for serious starting points, I simply favor Britannica and its editorial controls.
In David's last post, "The Flattening of Knowledge," he spoke of the benefits of collaboration on the internet. He noted that more people have access to information than at any other time in the history of the world, and more people have an opportunity to be heard than ever before. Both of these ideas are very true, and the Wikipedia embodies them both.
I'm not opposed to access to information by everyone; nor am I opposed to voices being heard. But as I have mentioned before, I have problems with the Wikipedia's absence of--for lack of a better term--quality control. I have already discussed my concern, not with the democratization of knowledge, but the democratization of truth. The advantage of the Wikipedia is also its curse: anyone can contribute. And regardless of whether or not information is true, it can remain in an article unless it's challenged. But even if it's challenged, if the original contributor is persistent enough or if enough other contributors agree (or can be convinced to agree), then the posting becomes fact. This is a problem, no matter how much we revel in all of our voices being heard. And in my opinion, that makes the Wikipedia an unreliable source because information becomes far too fluid.
A second concern of mine has to do with the Wikipedia's lack of editorial discretion. What I mean by this is that there's really no control over what information is included, even insignificant details about a subject or even insignificant subjects themselves.
For instance, the Wikipedia boasts 1,300,000 articles in English. Now the online version of Encyclopedia Britannica only has about 100,000 articles. Does the Wikipedia therefore trump Britannica? I don't think so. On any Wikipedia page, there's a link in the top left side called "Random Article." Here are five clicks I made. I give you my word that these are the articles I received in five successive hits and I haven't fudged the list in any way.
1. British School - Muscat = "a school in Muscat, Oman, catering primarily for the British expatriate community, but containing many students of many different nationalities."
2. Asherah = (from Hebrew אשרה), generally taken as identical with the Ugaritic goddess Athirat (more pedantically but accurately ʼAṯirat), was a major northwest Semitic mother goddess, appearing occasionally also in Akkadian sources as Ashratum/Ashratu and in Hittite as Asherdu(s) or Ashertu(s) or Aserdu(s) or Asertu(s)."
3. Pedro Dimas = "a Mexican violinist, guitarist, composer, and preservationist of traditional music from the Purépecha, an indigenous culture in the Mexican state of Michoacán."
4. Sumed Ibrahem = "(born December 30, 1980 in Tamale, Ghana) is a Ghanaian soccer player, who, as of 2005, plays midfield for the Harrisburg City Islanders of USL Second Division."
5. Abaújvár = "a village in Hungary, next to the Slovakian border. It lies 72 km northeast of Miskolc."
Of these entries, only the second one is also included in the Encyclopedia Britannica. Does this make Britannica inferior? I don't think so. I guess it's good to have all this other information in one handy place, but I think it demonstrates that the sheer number of articles in Wikipedia doesn't mean much as some are so obscure they are essentially irrelevant for the average person. There are articles for everything form comic book characters to individual episodes of television shows. Thus, the Wikipedia is a bit like the Hitchhiker's Guide to the Galaxy as it contains information on just about everything. I suppose that this is not a bad thing, but it does not make it superior to Britannica which by necessity should be more selective in its content.
Another problem with this lack of editorial discretion is the kind of information found in many of the articles. Let me illustrate. Last week, Kathy and I were at home watching the news on one of the cable channels, and after the serious events of the day were exhausted, a report was aired about Paris Hilton having to go to the emergency room in the middle of the night because she was bit by her pet kinkajou. I had never heard of a kinkajou and announced as much to Kathy.
"You've never heard of a kinkajou?" she asked as if surprised.
"No," I replied. "What's a kinkajou."
"Everyone knows what a kinkajou is. It's an exotic pet. Paris Hilton has one."
Not impressed with her explanation, I decided to look kinkajou up for myself in my 2006 edition of Britannica that I have loaded on my PowerBook. I got this short article:
Well, not too bad. It gave me everything I need to know and then some. In preparation for this blog entry, I decided to look up kinkajou in the Wikipedia. To its credit, I got pretty much the same information--at least in regard to the main facts. But I nearly fell out of my chair when I noticed this sentence: "Paris Hilton's Kinkajou, "Baby Luv" has been in the media twice in the past two years; once when he scratched her and the CA Fish and Game Dept issued her a warning for illegally transporting the animal, and again in August 2006 when he bit her arm. Both injuries were minor. Owning a kinkajou in CA is illegal." Is this really important information to put an article about this animal? Will this be relevant in 50 years? In 50 days? [Note: it seems that evidently, there is a Wiki-squabble going on and various forms of the Paris Hilton incident have been added and removed over the last few days. If you decide to look up the kinkajou article and you think I made the whole Paris Hilton insertion up, please look at the history of the article. All I know is that I expect Google hits to my website to triple now that I've used the name "Paris Hilton" five times in one blog entry.] Regardless, this is a prime example of the lack of a final editorial control over the Wikipedia's content. Certainly, someone may remove objectionable content, but there's nothing to keep someone else from putting it back in. Such self-regulation appears very democratic on the surface, but it seems to me that there needs to be a final authority who can simply freeze content.
As an experiment of how easy it is to manipulate an article, I added a reference to myself in one. Now, I didn't add fraudulent information, but I did add irrelevant information. Under the famous residents section of the Wikipedia article on Shreveport, Louisiana, I listed myself as having been born there (which I was). I'm going to paste it here below because I don't think it's ethical for me to leave it there more than about 24 hours just to prove my point.
My name is in red because I created a stub for my name so that it could be edited, and thus a new article about yours truly could be created. Don't worry, as I said, I'm going to remove the whole thing. This is just to prove why I believe a peer-edited resource like Britannica is a much more reliable source of information than the Wikipedia.
David, I will concede to you a major point. You wrote in your last post about this subject:
We can talk all we want about how wonderful the editors and writers are for EB but the simple fact is this:
Virtually no one has access to the Encyclopedia Britannica in the 21st century!
So if you consider that to be a problem (I do!), then our primary concern should be creating access to the EB. If the publishers don’t fix that problem, their excellent information is going to become rapidly irrelevant.
Frankly, I can only assume that the rise of the personal computer and especially the internet has hurt Britannica sales dramatically. If video killed the radio star, will the internet kill the bound encyclopedia? Maybe. Britannica is available in three forms: (1) the print bound set, (2) computer editions, and (3) all articles are available via the internet. But unlike the Wikipedia, none of these are free unless you count access to the bound sets at your local library. The fee-based internet editions are primarily sold to schools. The online edition is updated daily, but not by just anyone, and inclusions of pop-culture figures and events are weighed very carefully. But if the PTB at Britannica decided to simply give this information away for free to everyone over the internet, how would they stay in business? This is something I don't have an easy answer for.
Fortunately the Hitchhiker's Guide to the Galaxy says, DON'T PANIC.
The Sum of Human Knowledge*
Well, he didn't quite put it in those terms, but that's the feeling I got from him in our friendly little debate
Be sure to read David's entire blog entry, but here are a few main points:
The bigger picture here is globalization and democratization of knowledge. Wiki knowledge fits within the framework of a new way of organizing knowledge in the modern world. The era of printed collections of information gathered by an elite group of experts is slowly coming to an end.
I see googlization as a positive trend but it brings with it a big change in how we access information. I remember writing a paper on the subject of royalty in the history plays of Shakespeare. In order to find the quotes I needed I had to actually... gasp... read the plays! Over and over again. Can you imagine! What would you do today if you were writing such a paper? You'd google the topic of course! You'd start googling combinations of keywords until you were pointed to an online information source that perhaps allowed you to search Shakespeare's plays. In the process you'd probably come across other writers who had tackled the subject and use them in your bibliography.
So, Rick, I respect your concerns. We're all concerned about Wikipedia being abused for political and ideological reasons. But frankly, Encyclopedia Britannica has its own political and ideological aims as well. The difference is that with Wikipedia you have a collective mind composed of hundreds of editors "mediating truth" rather than some ivory tower cabal working at EB. Rather than shooting down Wikipedia, I think the real discussion we need to have is how can trusted sources of information like Encyclopedia Britannica embrace Wiki forms of information gathering and disseminating in order to stay dynamic.
Wikipedia is a new kind of encyclopedia for a new kind of mind. The global citizen is going to need global information and it is unlikely that this knowledge is going to carry the name of an 18th century colonializer.
That last line was quite a cheap shot, don't you think?
Anyway, taking the high road, I ignored the ad-biblionem attack, and gently responded with these words in the comments:
No, I don't want Britannica updated by Wikipedia's methods.
Here's the deal...I'm not completely against the Wikipedia. I use it regularly and have quite a few links to it on my blog. I created a link to it in the blog I wrote today.
But I have trouble being confident in the Wikipedia for any kind of serious investigation of a subject. I might go there, but I don't know who wrote the information and I don't know how accurate the information is. At least with Britannica, I can at least know it was written by an expert in the area. Granted, "experts" can have bias, too, but at least the information in Britannica is not a moving target.
As for research as in regard to my students, the real problem is that the average student--high school, college, and even higher--does not adequately know how to discern good sources from bad sources.
I've watched as students run searches in Google and immediately run to the top selections regardless of whether they are actual good sources or not.
I'm not afraid of the democratization of knowledge, but more of the democratization of truth. That's what Colbert was driving at. If enough people think it's true, it becomes fact. And unfortunately, too many people don't know the difference.
Then in a counter-point to my counterpoint to his blog entry which was a counter-point to my blog entry, David wrote the following:
1. Determining truth. You see the concept of an "expert" being a safe-guard against falsehood, while I see the concept of "democratic editing" being that safe-guard. Maybe another way of looking at this is that it is an authority question.
2. Old vs. new media. Wikipedia and EB represent two very different forms of publishing. It seems to me that the rate of information growth and change in this century makes the thought of waiting 20 years to get an update unthinkable. At the same time, hyper-editing at Wikipedia seems fraught with danger.
My response: Point #1. It's not merely the "concept of an 'expert' being a safe-guard against falsehood," but rather an expert who is peer-reviewed by other experts (i. e., Britannica's editorial board). Granted, any such individual or group can have agendas or political biases, but at least I know my source since the larger and more significant articles in Britannica are signed. Yes, it certainly is an authority question, because if I read something on the Wikipedia, I have to ask, "Says who?"
Point #2. Britannica editions are actually becoming updated more quickly. For the print edition, a new 16th edition was recently released, plus they release updates annually in their Book of the Year 20XX. And as for the online edition, they update the content daily (okay, I'll admit that I learned this last fact from the "Encyclopedia Britannica" entry on the Wikipedia. So?).
And, David, your last sentence: "...hyperediting at Wikipedia seems fraught with danger." That's entirely my point--couldn't have said it better myself. The problem with the Wikipedia as a consistently serious source of factual information is that the content is--or can be--a moving target.
And David, just remember that every time someone reads your anti-Britannica blog and decides to forego a purchase of the print set in favor of a relativistic, knowledge-and-truth-democratized online source, well... that's 32 extra days in purgatory for you after you die (one day per each volume in the current set).
*The phrase "Sum of Human Knowledge" comes from a 1913 advertisement for the 11th edition of the Encyclopedia Britannica.
Image Source: Wikipedia
Why Britannica Trumps Wikipedia (Thank-You, Stephen Colbert)
The satirical current events show The Colbert Report, which airs Monday through Thursday nights on Comedy Central, has a history of messing with the mainstream media to comedic ends. But last night, host Stephen Colbert went after Wikipedia. The results, as always, were hilarious.
Check out the video for the Wikiality segment on YouTube. Colbert encourages his viewers to change the Wikipedia entry for elephant so that it says the number of African elephants has tripled in the last six months. The result? Various Wikipedia articles referring to elephants, African elephants, African Bush elephants, African Forest elephants and the like were immediately moved to semi-protected status by the site's administrators. Pages with the semi-protected designation can only be edited by registered and trusted users. Colbert's Wikipedia user account was also blocked from making edits.
Watch the segment for yourself:
Sometimes Colbert offends me, but sometimes his use of insincerity gives a potent voice to my feelings about a subject. I've had a deep gnawing problem with the Wikipedia for a while--specifically the democratization of knowledge and the potential for the relativistic abuse of information. Here are some of the best quotes from the video clip linked above:
"I love the Wikipedia. Any site that has a longer entry on 'truthiness' [a term coined by Colbert] than on Lutherans has its priorities straight."
Regarding the mechanics of the Wikipedia: "Any user can change any entry and if enough users agree with them, it becomes true."
"Who is Britannica to tell me that George Washington had slaves? If I want to say he didn't, that's my right. And now thanks to Wikipedia [types on his laptop keyboard], it's also a fact.
"Together we can create a reality that we can all agree on: the reality we just agreed on."
You see, this is exactly why the Encyclopedia Britannica is better than the Wikipedia. I have a 15th edition, 1995 set at home and the 2006 electronic edition on my PowerBook. It goes with me just about everywhere. Britannica's entries are written by experts in the field and they are subject to a review board. With the Wikipedia the articles can be written or altered by just about anyone. And who knows if what's written there is true, false, accurate, or inaccurate?
If I want to write (or change) an article on the Wikipedia, I can just log in (or act anonymously) and write whatever I want. If I want to write an article in the Encyclopedia Britannica, I have to spend years becoming an expert in my field, become widely published and recognized for my expertise in the subject and then, maybe I will be allowed to write for Britannica.
The Wikipedia is great for information about anything relating to pop-culture, such as if I want to read about the latest theories and analysis regarding the television show, Lost. But if I want a starting place to seriously investigate a subject--history, science, literature, biography, philosophy, and even religion-- with an article written by a known expert, with an extensive bibliography for further research, I'll go to the Encyclopedia Britannica--every time.
Related previous posts:
- The Wikipedia is Not Enough
- Martin Luther Was Excommunicated on This Day and Why I Love the Encyclopedia Britannica
The Wikipedia Is Not Enough
Douglas Groothuis has written a blog entry entitled, "Against Wikipedias (For Scholarship)."
Groothuis writes
Wikipedias are so much the rage that students are citing them in a philosophy papers. This occasioned a mini-sermon from me in one of my classes. The key excerpt was, “Never, ever, ever cite a Wikipedia in a philosophy paper.” Wikis (as they are abbreviated; our culture abbreviates everything) are unedited and unauthorized internet articles in which anyone can contribute, delete, or alter previous material. They are unregulated and ever-changing amalgamations, contingent configurations. Some of them—as least some of the time—may be well written and knowledgeable; but they lack any editorial protocol to insure decent material that conforms to standards. Thus, they are worthless as sources that one would cite in a paper or article of a scholarly nature.
I couldn't agree more. However, I would extend that not to just philosophy papers, but any papers. Further, I would extend the prohibition not just to the Wikipedia, but any encyclopedia. These are not adequate sources for a paper written at the college or graduate level.
That doesn't mean that I don't like encyclopedias, including the Wikipedia. I very much do. I have a link to the Wikipedia on the toolbar of my web browser. I use it all the time. Also, I have the entire Encyclopedia Britannica loaded onto my PowerBook, which I've written about on this blog before. I refer to it all the time to get initial information on a subject. Sometimes that information is all I need--I've been able to educate myself on something I didn't know about before or couldn't remember. But I wouldn't quote from it in a paper if the same information were available elsewhere.
The truth is that encyclopedias of any kind are merely starting points for research, not the end goal. A good encyclopedia will include a bibliography of major works about a subject. That's one reason I like the Encyclopedia Britannica. It gives me direction for further research. The downside is that all articles aren't as current as others. I use the Wikipedia for quick information on a subject, and to its credit, there are often valuable links or other sources listed at the end of the articles.
Encyclopedias were great sources for that 8th grade report on the marsupials of Australia, but they aren't adequate as a final source for information in serious research.