3/22/2007

FAQ Friday: Evaluating sources -- the good, the bad, and the worthless

The end of spring semester looms in the not-to-distant future. You are a seasoned researcher, having mastered the art of locating resources – the library catalog holds no secrets, databases are a breeze, ILL is your new best friend and you have a stack of resources all ready to be transformed into the best darn paper your prof has every read.

You may be a lean, mean researchin’ machine, but before you set the margins and start typing and citing yourself silly, it’s a good idea to take a few minutes to analyze those sources you worked so hard to track down. What sounds good – or at least good enough – when it’s very late, when you’re very tired and desperate for that last article in support of your thesis, may not be so great after all. Even the best researchers end up wearing beer goggles every now and then – figuratively, of course.

Because FAQ cares about you, really cares, today’s feature is a quick primer on identifying and evaluating sources. These tips can be adapted to suit any kind of source – books, articles, websites, etc.

Let the critical thinking begin!

Scholarly vs. popular sources
In terms of articles, “scholarly” generally refers to articles published in journals that are peer-reviewed or refereed. This means the articles are reviewed by other prominent scholars in the same field of study prior to publication in the journal. Some journals are considered academic, which means the articles are written by experts but not necessarily peer-reviewed prior to publication.

Popular articles can be found in newspapers and popular magazines (think waiting rooms and newsstands). The articles are written for a general audience, and often the writer does not have an academic background related to the subject of the article. Although the articles are edited and fact-checked (well, most of the time), the process is not equivalent to peer review.

Use similar criteria to determine if a book is scholarly or popular. In general, scholarly books are published by academic presses, have little or no cover art and contain an extensive bibliography in addition to an index. Popular books are often published by mass market publishers (think Random House) and authors may display a corporate affiliation instead of academic credentials.

Guidelines for recognizing scholarly articles:

  • Peer review
  • Author credentials included
  • Literature review
  • Reference list/bibliography/works cited included
  • Published by an academic organization or university press
  • No advertisements for consumer products (baby food, cleaning products)
Examples: Behavioral Neuroscience, The Auk, Conservation Biology, Journal of Analytical Chemistry, Journal of Learning Disabilities

Guidelines for recognizing popular articles:

  • No peer review
  • Author may not be listed
  • No references listed
  • Many advertisements for variety of products
Examples: People, Time, Good Housekeeping, Ebony, Vanity Fair

Guidelines for recognizing scholarly books:

  • Published by an academic publisher or university press
  • Author lists academic credentials/affiliations
  • Extensive references/bibliography and index

Guidelines for recognizing popular books:

  • Mass market/trade publishing house
  • References/index not extensive (or not present at all)
  • No academic credentials


Identifying sources as primary or secondary
Primary and secondary source identification can vary depending on the discipline, so check with your professor if you are uncertain about a particular source. The social sciences and in particular the study of history have very specific guidelines in the use of primary vs. secondary sources.

The following general guidelines apply in most disciplines:

Primary sources are:

  • Original documents (letters, diaries, manuscripts)
  • Public records, official documents and other archival materials
  • Literary works
  • Original research studies

Secondary sources are:

  • Reviews/critiques of author’s work
  • Analysis of original documents
  • Analysis of research studies

For example, an original Civil War diary would be considered a primary source. An article about the symbolism of spelling errors contained in that diary would be a secondary source.

Evaluating sources – essential questions to ask about every resource.

  1. Is it relevant to your topic?

Read the abstract, scan the index, review the research methodology and conclusions.

  1. Is the author credible?

Is an author listed? Are the authors’ credentials listed? Is this author quoted/referenced by others in the field?

  1. Is the article recent/timely?

How long ago was the material published, and is it still relevant to the field of study. This may or may not be an issue; it depends on the requirements of your project.

  1. Is the data valid and/or accurate?

Are sources provided to back up the data? Does it appear to be a valid research study?

  1. Identify the question or argument posed by the author.

Analyze the argument, supporting data, analysis and conclusions. Is the logic sound, does it all make sense based on the material presented? Are varying interpretations considered or presented?

  1. Is the topic coverage

Did the author reference sources current to the period of publication, or are the sources outdated in the field of study? Does the author ignore certain aspects of the subject?

  1. Does the author present a bias?

Is a particular bias stated in the work? Do certain statements or omissions indicate an unstated bias?

Caveats about web resources

This set of standard questions may be used to evaluate resources in any format. If you are evaluating a web site, pay particular attention to questions of credibility, currency of information and accuracy. Almost anyone can throw up a website and post whatever comes to mind, so take the time to evaluate who is presenting the information and why (credibility), how you can contact them (again, credibility), the posting date (currency of information) and of course relevancy. The internet is full of information, but in most cases it’s not peer reviewed. It’s up to you to do the detective work. FAQ will devote a future posting specifically to evaluating web resources, but in the meantime be smart about using internet sites. Be certain you know who created the site, why they post what they post, how to contact them, and of course whether or not there is an obvious bias.

Lest you think FAQ is a crackpot, the following sources were consulted for the preparation of this blog entry:

Keys for Writers: A Brief Handbook / Raimes
PE1408 .R24

MLA Handbook for Writers of Research Papers, 6th ed. / Garibaldi
REF LB2369 .G42

Schaum’s Quick Guide to Writing Great Research Papers / Rozakis

eBook, accessed via library online catalog


No comments: