Open Access Sting Reveals Peer Review Problems

Science Magazine recently wrote a completely bogus article about lichen extracts that combat cancer, came up with some fake scientist names, gave them some made up credentials, and then submitted the lot of it to 300 open access journals. Though many of the journals boasted peer review boards with prestigious-sounding names and titles, the article was accepted by over half of the journals. Some of the journals did ask for edits, but the authors claim the paper was so bad that mere editing would not have fixed the fatal flaws of the research, methods, data, and conclusions.

It gets worse. According to the authors:
The paper was accepted by journals hosted by industry titans Sage and
Elsevier. The paper was accepted by journals published by prestigious
academic institutions such as Kobe University in Japan. It was accepted
by scholarly society journals. It was even accepted by journals for which
the paper’s topic was utterly inappropriate, such as the Journal of
Experimental & Clinical Assisted Reproduction.

So from whence did this problem arise? Is it merely the case that hucksters realized that academic publishing was a good scam, or is there a more nuanced take on it? According to Bauerlein, Grody, McKelvey, and Trimble’s recent We Must Stop the Avalanche of Low-Quality Research, greater access to publishing shares no small culpability. They suggest that increased access serves only to obscure truly brilliant work. Because so much work –much of it redundant, dim, or otherwise inconsequential– is published now, it is easy to miss the good stuff! The authors state that the amount of research makes it impossible to ensure that it is all accurate:

The surest guarantee of integrity, peer review, falls under a debilitating crush of findings, for peer review can handle only so much material without breaking down. More isn’t better. At some point, quality gives way to quantity (Par. 15).

I blogged here about the problem of open source journals demanding payment for publication, and there I suggested that students be told to use library databases instead of Google Scholar and the open internet. Because a new wrinkle has been added, I redouble my efforts here. PLEASE PLEASE PLEASE tell your students to use the library databases.

Over the Transom

Though she’s rocking a sabbatical, Adriana Tapanes-Inojosa hasn’t forgotten about the rest of us. This week she sent along a bunch of stuff to check out:

~This about Digital Scholarship and the Humanities

and

~This research on school reform from the CPS Teacher’s Union called, “The Schools Chicago’s Students Deserve”

PLUS, I received this link featuring AACU published research on VALUE rubrics (for Valid Assessment of Learning in Undergraduate Education) from Rock’in Tha Shoe, who writes, “Sadly, you can see that all research references are done by the Lumina foundation (mmmh??- here we go again with the business trying to dictate educational parameters).”

PLUS, Matt Usner sent this link awhile back on the power of nudges toward environmentally responsible behavior.

PLUS, while getting ready for the archiving, I found this email from Michal Eskayo from last November which had a link to this great article about students from China.

Enjoy, and h/t’s to all for the pointers.

Distant Reading

Speaking of reading, this is an interesting look at the new kinds of research going on in the Humanities at places like the Stanford Literary Lab:

As its name suggests, the Lit Lab tackles literary problems by scientific means: hypothesis-testing, computational modeling, quantitative analysis. Similar efforts are currently proliferating under the broad rubric of “digital humanities,” but Moretti’s approach is among the more radical. He advocates what he terms “distant reading”: understanding literature not by studying particular texts, but by aggregating and analyzing massive amounts of data.

We need distant reading, Moretti argues, because its opposite, close reading, can’t uncover the true scope and nature of literature. Let’s say you pick up a copy of “Jude the Obscure,” become obsessed with Victorian fiction and somehow manage to make your way through all 200-odd books generally considered part of that canon. Moretti would say: So what? As many as 60,000 other novels were published in 19th-century England — to mention nothing of other times and places. You might know your George Eliot from your George Meredith, but you won’t have learned anything meaningful about literature, because your sample size is absurdly small. Since no feasible amount of reading can fix that, what’s called for is a change not in scale but in strategy. To understand literature, Moretti argues, we must stop reading books.

Check out the rest HERE.

Think, Know, Prove: Research Papers

Think, Know, Prove is a regular Saturday feature, where a topic with both mystery and importance is posted for community discussion. The title is a shortened version of the Investigative Mantra: What do we think, what do we know, what can we prove? and everything from wild speculation to resource referencing fact is welcome here.

So, I’ve been reading a lot about reading and writing this summer, while trying to revamp the things that I do with respect to both. Along the way, I’ve seen story after essay ripping on the idea of assigning research papers (and even argumentative or analytical essays of any sort, i.e., academic forms of writing in general) as here and, again, here (there were others but I can’t seem to find them now). Over and over again, I’ve come across ideas for alternatives (does anyone assign book or album or movie reviews as papers? I thought of that as a possibility for both pre-writing reading (since examples are easy to find) and then writing assignments after reading this article by Robert Pinsky (!) on the “rules” for good book reviews and this fun to read example (sorry Kamran; I know you liked the book–hope this doesn’t ruin it for you).

And it seems like I can’t read a piece about writing without it mentioning this hunk of research from Stanford (of which,for all it’s appeal and purported influence, I can’t seem to make heads or tails in terms of concrete applications in my classroom).

Then, recently, the research paper complainer cited above had some second thoughts (published here), which has me all ambivalent about teaching academic writing all over again.

And so I ask, when it comes to teaching writing (specifically in terms of forms of writing): What do you think? What do you know? What can you Prove?

Reading Strategery

So, while I still don’t know exactly why it happened, the reality for our department (Humanities) is that we have a whole new set of prerequisites for our classes, and given those, I’ve been thinking all summer about how to better support the students coming in who are not as far along, preparation-wise, as the students I’ve had over the past few years.

While poking around, I found some good stuff, like this, which led me to this research on effective strategies for reading:

Text Comprehension Instruction

Comprehension is defined as “intentional thinking during which meaning is constructed through interactions between text and reader” (Harris & Hodges, 1995). Thus, readers derive meaning from text when they engage in intentional, problem solving thinking processes. The data suggest that text comprehension is enhanced when readers actively relate the ideas represented in print to their own knowledge and experiences and construct mental representations in memory.

The rationale for the explicit teaching of comprehension skills is that comprehension can be improved by teaching students to use specific cognitive strategies or to reason strategically when they encounter barriers to understanding what they are reading. Readers acquire these strategies informally to some extent, but explicit or formal instruction in the application of comprehension strategies has been shown to be highly effective in enhancing understanding. The teacher generally demonstrates such strategies for students until the students are able to carry them out independently.

The literature search identified 453 studies that addressed issues and topics relevant to text comprehension since 1980. Studies published between 1970 and 1979 were added if they were of particular relevance, resulting in 481 studies that were initially reviewed. Of these, 205 studies met the general NRP methodological criteria and were then classified into instructional categories based on the kind of instruction used. Application of the more specific review criteria precluded formal meta-analyses because of the large variation in methodologies and implementations used. The Panel found few research studies that met all NRP research methodology criteria. Nevertheless, the Panel employed the NRP criteria to the maximum extent possible in its examination of this body of literature. (See the Comprehension section of the Report of the National Reading Panel: Reports of the Subgroups.)

In its review, the Panel identified 16 categories of text comprehension instruction of which 7 appear to have a solid scientific basis for concluding that these types of instruction improve comprehension in non-impaired readers. Some of these types of instruction are helpful when used alone, but many are more effective when used as part of a multiple-strategy method. The types of instruction are:

  • Comprehension monitoring, where readers learn how to be aware of their understanding of the material;
  • Cooperative learning, where students learn reading strategies together;
  • Use of graphic and semantic organizers (including story maps), where readers make graphic representations of the material to assist comprehension;
  • Question answering, where readers answer questions posed by the teacher and receive immediate feedback;
  • Question generation, where readers ask themselves questions about various aspects of the story;
  • Story structure, where students are taught to use the structure of the story as a means of helping them recall story content in order to answer questions about what they have read; and
  • Summarization, where readers are taught to integrate ideas and generalize from the text information.

Definitely good stuff, worth reading and adapting to your classroom, if you don’t do all of this already, that is.

A Viewpoint on the Reinvention Data

I haven’t yet linked to the CCC Reinvention: The Truth blog in part because I haven’t been quite sure of it in a variety of ways (plus I knew that people could find it through the Truman Lounge if they really wanted to).

Still, I’ve been poking my head in now and again and gradually gained confidence that it is, at the least, an expression of the perspective of some CCC faculty (if not always mine). When I came across this post yesterday, I figured it was time to feature the post and the blog, just in case anyone out there hadn’t seen it yet.

The post is about the contrast in the data used for “The Case for Change” (and more saliently, the imprecision in the way that data is used and promulgated) and some other data collections that seem to be at odds with the much repeated numbers of “The Case for Change.”

I’d been wanting to post some sort of response to the “White Paper” on Reinvention (have you read it?), but the unnamed author put together way better work than I can muster at this point, and in multiple ways better than anything I could put together under good circumstances. In other words, it’s worth reading.

PS: I’m going to add it to the BlogRoll, too, since it is a faculty blog.

Weekend Reading

I don’t know what’s in it (yet), but I’ll be reading pieces of this promising looking report over the weekend when I get tired of reading student essays on Categorical Syllogisms, Pornography, and Ralph Waldo Emerson.

In response to these issues, MDRC launched the Opening Doors Demonstration in 2003 — the first large-scale random assignment study in a community college setting. The demonstration pursued promising strategies that emerged from focus groups with low-income students, discussions with college administrators, and an extensive literature review. Partnering with six community colleges across the country, MDRC helped develop and evaluated four distinct programs based on the following approaches: financial incentives, reforms in instructional practices, and enhancements in student services. Colleges were encouraged to focus on one strategy but to think creatively about combining elements of the other strategies to design programs that would help students perform better academically and persist toward degree completion.

Opening Doors provides some of the first rigorous evidence that a range of interventions can, indeed, improve educational outcomes for community college students.

Click HERE to read the Policy Brief inspired by the research (it has descriptions of the programs and their effects).

Systematic Change

Another version of the playbook.

The issue of teacher effectiveness has risen rapidly to the top of the education policy agenda, and the federal government and states are considering bold steps to improve teacher and leader effectiveness. One place to look for ideas is the experiences of high-performing education systems around the world. Finland, Ontario, and Singapore all have well-developed systems for recruiting, preparing, developing, and retaining teachers and school leaders, and all have attained high levels of student performance and attribute their success to their teacher-effectiveness policies.

 

WHM Special: Girls, Women, Men, and Math

I thought this research on was absolutely stunning when I saw it on Slate last week.

In a more ambitious experiment organized with the university’s math department, the psychologists evaluated how undergraduates performed when they had male or female math professors.

They measured, for instance, how often each student responded to questions posed by professors to the classroom as a whole. At the start of the semester, 11 percent of the female students attempted to answer questions posed to the entire class when the professor was male, and 7 percent of the female students attempted to answer questions posed to the entire class when the professor was female. By the end of the semester, the number of female students who attempted to answer questions posed by a male professor had not changed significantly: Only 7 percent of the women tried to answer such questions. But when classes were taught by a woman, the percentage of female students who attempted to answer questions by the semester’s end rose to 46.

The researchers also measured how often students approached professors for help after class. Around 12 percent of the female students approached both male and female professors for help at the start of the semester. The number of female students approaching female professors was 14 percent at the end of the semester. But the number of female students asking for help from a male professor dropped to zero.

Finally, when Stout and Dasgupta evaluated how much the students identified with mathematics, they found that women ended up with less confidence in their mathematical abilities when their teachers were men rather than women. This happened even when women outperformed men on actual tests of math performance.

Think about that. On objective measures of math performance, these women were outscoring men. But their identification with mathematics was not tied to their interest, determination, or talent. It was connected to whether their teacher was a woman or a man.

So, what would it take to get an experimental section or two of all women students taught by a woman teacher?

Google Art Project

Well, I’ve been told that The Lounge is celebrating an anniversary this week and I was asked if I’d like to contribute. The answer is yes, and I will share a link that was brought to my attention by our esteemed colleague, Armen.

Google Art Project

I’ve not had an opportunity to view the entire site, but from what Armen tells me, it will be yet another great resource to our Art, Architecture, and Humanities courses. Here’s a link to ‘Learn More‘ about how to use and view the images. I hope you find the site of interest and of good use in your classes, regardless of discipline.

A big congratulations to The Lounge on this milestone. I’m sorry that my posts have been far and few this past year. I will not say the same regarding my visits and the time I spend reading the posts and replies.

Again, congratulations to The Lounge. It’s a great resource!

Experiments Related to Philosophies of Ed

I found this one (“When Teaching Restrains Discovery”) accidentally when I was reading about the test anxiety study, and I liked the experiments, the results, and the description of both.

[T]his experience is relevant a longstanding debate about the best way to teach children, especially very young ones. One camp believes that children learn mostly through teaching and direct instruction. The other says that children learn mostly by exploring and figuring things out for themselves. To them, formal instruction is too passive, and makes for children that receive knowledge without engaging with it. On the other hand, people who favour more direct teaching argue that children need more guidance. Leaving them to explore on their own, through so-called “discovery learning”, is inefficient and ineffective. These are, of course, extreme positions and the debate is more subtle. Both approaches have their merits and good teachers face the challenge of finding a happy medium.

That’s never been clearer than in a new study by Elizabeth Bonawitz from University of California, Berkeley. Through two experiments with pre-schoolers, Bonawitz has found that teaching can be a “double-edge sword”. When teachers provided specific instructions about a new toy, children learned how to play with it more efficiently. But the lessons also curtailed their exploratory streak. They were less likely to play with the toy in new ways. Ultimately, they failed to find all of its secrets.

But it isn’t presented as a stark either/or. There are nuances considered, too, and suggested reasons for what is going on in the kids heads. Later, the author of the article writes:

Context clearly matters. When the apparently knowledgeable teachers in the experiments provide a seemingly complete lesson about the toy, the children deduce that there is a no more to learn. If the lesson is interrupted, or if the instructor seems like a novice, the child deduces that there is more to discover. Bonawitz thinks that these abilities start from a very early age, when children are still in pre-school or kindergarten.

As a bonus, it’s a short piece, too.

Rehabilitation Options

Apropos of nothing in particular, I thought this article was interesting and potentially useful across lots of disciplines.

What works and what doesn’t work to solve a social problem is often no mystery.  The mystery is why we so often persist in doing what doesn’t work.  The topic of Tuesday’s column — prisoner re-entry into the community — offers myriad examples.   One is the practice of dropping people getting out of jail or prison right back into the neighborhoods where they got in trouble in the first place.  Intuition tells us that this is a bad idea: the old street corners and the old friends seem like a recipe for the old troubles.  Research on this idea is rare and hard to do — it’s tough to get around the problem that the person who chooses not to go home may have other qualities that make him successful.

Prisoners are often aware of the temptations they will face upon resuming their old lives.  Nearly half of the prisoners in Illinois surveyed by the Urban Institute said they didn’t want to go back home upon release.  But states not only encourage people to go home again, some of them demand it — in most states, prisoners released on parole are legally required to go back to their county of last residence.

This rule is one of many protocols for dealing with former prisoners that seem to make little sense.  Many prisoners are sent home to arrive in the middle of the night with only a few dollars in their pockets.  Virtually no one in prison in the United States today can get methadone maintenance therapy, the gold standard drug treatment.  Prisoners are no longer eligible for the grants that used to make getting a college education in prison possible. This system is designed to fail.  And it does.

It is not failing quietly.  On the positive side, there are programs all over the country that recognize that helping prisoners remake their lives is both humane and cost-effective.

Test Anxiety Remedy

I know I posted on this sort of thing before, but I happened to see this U of C research about how to effectively relieve test/learning anxiety through a simple technique over the winter break and saved it. With Week 4 practically upon us and so the first round of tests for a lot of classes, I thought it would be worth posting. It’s a different study than the one I first posted about, but the findings are the same:

A team of University of Chicago psychological scientists found that high school and college students who jotted down their worries for 10 minutes before exam time avoided choking under the pressure. In fact, they performed markedly better.

In a study released Thursday in the journal Science, Sian Beilock and Gerardo Ramirez asked half a class of freshmen facing their first final exams to write down their concerns about the upcoming test while other students journaled about an unrelated topic.

To a teen, students who wrote about their stress scored as well or better than those who didn’t.

Check out the rest (including suggestions for why it works) here.