Open Access Sting Reveals Peer Review Problems

Science Magazine recently wrote a completely bogus article about lichen extracts that combat cancer, came up with some fake scientist names, gave them some made up credentials, and then submitted the lot of it to 300 open access journals. Though many of the journals boasted peer review boards with prestigious-sounding names and titles, the article was accepted by over half of the journals. Some of the journals did ask for edits, but the authors claim the paper was so bad that mere editing would not have fixed the fatal flaws of the research, methods, data, and conclusions.

It gets worse. According to the authors:
The paper was accepted by journals hosted by industry titans Sage and
Elsevier. The paper was accepted by journals published by prestigious
academic institutions such as Kobe University in Japan. It was accepted
by scholarly society journals. It was even accepted by journals for which
the paper’s topic was utterly inappropriate, such as the Journal of
Experimental & Clinical Assisted Reproduction.

So from whence did this problem arise? Is it merely the case that hucksters realized that academic publishing was a good scam, or is there a more nuanced take on it? According to Bauerlein, Grody, McKelvey, and Trimble’s recent We Must Stop the Avalanche of Low-Quality Research, greater access to publishing shares no small culpability. They suggest that increased access serves only to obscure truly brilliant work. Because so much work –much of it redundant, dim, or otherwise inconsequential– is published now, it is easy to miss the good stuff! The authors state that the amount of research makes it impossible to ensure that it is all accurate:

The surest guarantee of integrity, peer review, falls under a debilitating crush of findings, for peer review can handle only so much material without breaking down. More isn’t better. At some point, quality gives way to quantity (Par. 15).

I blogged here about the problem of open source journals demanding payment for publication, and there I suggested that students be told to use library databases instead of Google Scholar and the open internet. Because a new wrinkle has been added, I redouble my efforts here. PLEASE PLEASE PLEASE tell your students to use the library databases.

Over the Transom

Though she’s rocking a sabbatical, Adriana Tapanes-Inojosa hasn’t forgotten about the rest of us. This week she sent along a bunch of stuff to check out:

~This about Digital Scholarship and the Humanities

and

~This research on school reform from the CPS Teacher’s Union called, “The Schools Chicago’s Students Deserve”

PLUS, I received this link featuring AACU published research on VALUE rubrics (for Valid Assessment of Learning in Undergraduate Education) from Rock’in Tha Shoe, who writes, “Sadly, you can see that all research references are done by the Lumina foundation (mmmh??- here we go again with the business trying to dictate educational parameters).”

PLUS, Matt Usner sent this link awhile back on the power of nudges toward environmentally responsible behavior.

PLUS, while getting ready for the archiving, I found this email from Michal Eskayo from last November which had a link to this great article about students from China.

Enjoy, and h/t’s to all for the pointers.

Distant Reading

Speaking of reading, this is an interesting look at the new kinds of research going on in the Humanities at places like the Stanford Literary Lab:

As its name suggests, the Lit Lab tackles literary problems by scientific means: hypothesis-testing, computational modeling, quantitative analysis. Similar efforts are currently proliferating under the broad rubric of “digital humanities,” but Moretti’s approach is among the more radical. He advocates what he terms “distant reading”: understanding literature not by studying particular texts, but by aggregating and analyzing massive amounts of data.

We need distant reading, Moretti argues, because its opposite, close reading, can’t uncover the true scope and nature of literature. Let’s say you pick up a copy of “Jude the Obscure,” become obsessed with Victorian fiction and somehow manage to make your way through all 200-odd books generally considered part of that canon. Moretti would say: So what? As many as 60,000 other novels were published in 19th-century England — to mention nothing of other times and places. You might know your George Eliot from your George Meredith, but you won’t have learned anything meaningful about literature, because your sample size is absurdly small. Since no feasible amount of reading can fix that, what’s called for is a change not in scale but in strategy. To understand literature, Moretti argues, we must stop reading books.

Check out the rest HERE.

Think, Know, Prove: Research Papers

Think, Know, Prove is a regular Saturday feature, where a topic with both mystery and importance is posted for community discussion. The title is a shortened version of the Investigative Mantra: What do we think, what do we know, what can we prove? and everything from wild speculation to resource referencing fact is welcome here.

So, I’ve been reading a lot about reading and writing this summer, while trying to revamp the things that I do with respect to both. Along the way, I’ve seen story after essay ripping on the idea of assigning research papers (and even argumentative or analytical essays of any sort, i.e., academic forms of writing in general) as here and, again, here (there were others but I can’t seem to find them now). Over and over again, I’ve come across ideas for alternatives (does anyone assign book or album or movie reviews as papers? I thought of that as a possibility for both pre-writing reading (since examples are easy to find) and then writing assignments after reading this article by Robert Pinsky (!) on the “rules” for good book reviews and this fun to read example (sorry Kamran; I know you liked the book–hope this doesn’t ruin it for you).

And it seems like I can’t read a piece about writing without it mentioning this hunk of research from Stanford (of which,for all it’s appeal and purported influence, I can’t seem to make heads or tails in terms of concrete applications in my classroom).

Then, recently, the research paper complainer cited above had some second thoughts (published here), which has me all ambivalent about teaching academic writing all over again.

And so I ask, when it comes to teaching writing (specifically in terms of forms of writing): What do you think? What do you know? What can you Prove?

Reading Strategery

So, while I still don’t know exactly why it happened, the reality for our department (Humanities) is that we have a whole new set of prerequisites for our classes, and given those, I’ve been thinking all summer about how to better support the students coming in who are not as far along, preparation-wise, as the students I’ve had over the past few years.

While poking around, I found some good stuff, like this, which led me to this research on effective strategies for reading:

Text Comprehension Instruction

Comprehension is defined as “intentional thinking during which meaning is constructed through interactions between text and reader” (Harris & Hodges, 1995). Thus, readers derive meaning from text when they engage in intentional, problem solving thinking processes. The data suggest that text comprehension is enhanced when readers actively relate the ideas represented in print to their own knowledge and experiences and construct mental representations in memory.

The rationale for the explicit teaching of comprehension skills is that comprehension can be improved by teaching students to use specific cognitive strategies or to reason strategically when they encounter barriers to understanding what they are reading. Readers acquire these strategies informally to some extent, but explicit or formal instruction in the application of comprehension strategies has been shown to be highly effective in enhancing understanding. The teacher generally demonstrates such strategies for students until the students are able to carry them out independently.

The literature search identified 453 studies that addressed issues and topics relevant to text comprehension since 1980. Studies published between 1970 and 1979 were added if they were of particular relevance, resulting in 481 studies that were initially reviewed. Of these, 205 studies met the general NRP methodological criteria and were then classified into instructional categories based on the kind of instruction used. Application of the more specific review criteria precluded formal meta-analyses because of the large variation in methodologies and implementations used. The Panel found few research studies that met all NRP research methodology criteria. Nevertheless, the Panel employed the NRP criteria to the maximum extent possible in its examination of this body of literature. (See the Comprehension section of the Report of the National Reading Panel: Reports of the Subgroups.)

In its review, the Panel identified 16 categories of text comprehension instruction of which 7 appear to have a solid scientific basis for concluding that these types of instruction improve comprehension in non-impaired readers. Some of these types of instruction are helpful when used alone, but many are more effective when used as part of a multiple-strategy method. The types of instruction are:

  • Comprehension monitoring, where readers learn how to be aware of their understanding of the material;
  • Cooperative learning, where students learn reading strategies together;
  • Use of graphic and semantic organizers (including story maps), where readers make graphic representations of the material to assist comprehension;
  • Question answering, where readers answer questions posed by the teacher and receive immediate feedback;
  • Question generation, where readers ask themselves questions about various aspects of the story;
  • Story structure, where students are taught to use the structure of the story as a means of helping them recall story content in order to answer questions about what they have read; and
  • Summarization, where readers are taught to integrate ideas and generalize from the text information.

Definitely good stuff, worth reading and adapting to your classroom, if you don’t do all of this already, that is.

A Viewpoint on the Reinvention Data

I haven’t yet linked to the CCC Reinvention: The Truth blog in part because I haven’t been quite sure of it in a variety of ways (plus I knew that people could find it through the Truman Lounge if they really wanted to).

Still, I’ve been poking my head in now and again and gradually gained confidence that it is, at the least, an expression of the perspective of some CCC faculty (if not always mine). When I came across this post yesterday, I figured it was time to feature the post and the blog, just in case anyone out there hadn’t seen it yet.

The post is about the contrast in the data used for “The Case for Change” (and more saliently, the imprecision in the way that data is used and promulgated) and some other data collections that seem to be at odds with the much repeated numbers of “The Case for Change.”

I’d been wanting to post some sort of response to the “White Paper” on Reinvention (have you read it?), but the unnamed author put together way better work than I can muster at this point, and in multiple ways better than anything I could put together under good circumstances. In other words, it’s worth reading.

PS: I’m going to add it to the BlogRoll, too, since it is a faculty blog.