Faked journals -- Tip of the iceberg?

Through some investigative reporting, a magazine covering the scientific community has provoked a publisher of scientific journals into admitting it created six "fake" journals between 2000 and 2005 that were essentially secret ads for pharmaceutical companies.

One fake journal reprinted real articles from other sources, virtually all of them favorable to the drug manufacturing giant, Merck, according to magazine The Scientist, which broke the story.

Merck appears to have secretly sponsored the creation of that fake journal, The Australasian Journal of Bone and Joint Medicine, which not only did not have the same controls and editorial oversight as real journals, but also did not in any way disclose its connection to the manufacturer. The superficial impression was that it was a real, independent journal.

The publisher of the fake journals, Elsevier, publishes many other real journals, and is a well-known firm. Many of them might carry ads from manufacturers. But in those legitimate cases, the ads themselves reveal any possible financial entanglements, and the editors of the journal are generally assumed to be independent, relying on peer review rather than corporate input to determine which articles to publish. The six fake journals, on the other hand, appear to have been secretly sponsored by companies that determined their contents, even if some of the articles were real and had been previously published by real journals elsewhere.

Elsevier does not appear to have publicly identified which companies sponsored the other five journals, and the only reason we know about Merck's involvement in the sixth is that The Scientist (the magazine that broke the story) had already identified that relationship in an earlier news story. Elsevier has, however, said that all of the fake journals were published out of its Australian office during a period of five years by people no longer with the company, and the company maintains that this sort of practice does not represent its usual priorities.

Nevertheless, reader comments appearing below the article in The Scientist reveal that the discovery has touched off or renewed some serious concerns: Have we in fact identified all of the fake journals? Many small journals exist for short periods of time. Others held by Elsevier -- or even by other companies -- might be similar to the Australasian Journal of Bone and Joint Medicine. And what of the "legitimate" journals? How neutral are they, really?

Writes one reader, TS Raman:

For all practical purposes, a journal that merely looks like it is a "peer-reviewed" is not different from one that has real peer review, but of a very poor quality. I say this because there are scores of journals published by professional or scientific societies, and in-house journals published by institutions, laboratories, etc., all of which are virtually "captive journals". There is a plethora of such journals in India. They are all nominally peer-reviewed, but the review is often a complete farce. The reviewers may not have expertise in the field of study to which the paper pertains: for instance a person whose [nominal] speciality is nuclear physics, may review a paper on biochemistry.

Both are legitimate concerns, but they are neither new nor unique to the scientific community: Readers have always, and will always, need to be able to discern the difference between suspect and trustworthy material. It can be done.

In one 1SC class I am teaching, a team of students gave a presentation on a journal dealing with climate change. A student in the audience asked whether the journal gave much attention to skeptical arguments (that is, arguments that maybe global warming is a natural phenomenon, not the result of pollution). Even as the student asked his question, it was clear he had reached the same conclusion I had, just from listening to the presentation: No, the journal was strictly in the believer camp. If a scientist or lab had data undermining the consensus on global warming, that would not be a good journal to submit to. The journal was real, in the sense that it had real scientists as editors and authors, and used peer review, but it was pretty easy to tell what its biases were, just from a description of its history, its mission, and a list of articles it had recently published.

This is often the case.

We can all tell when we've flipped the channel to an infomercial, too, without necessarily being able to spell out exactly how we know that's what we're seeing.

Similarly, when one is looking at a Web site or a "news" publication or a white paper that is really advertorial (that is, composed of articles that look real but are basically ads for a product), it seldom takes investigative reporting to smell out the advertising. If you read very often, you start to notice little things like ...

• The presence of trademark symbols (like the R in a circle, or the letters TM superscripted), which only the original company ever includes. No, Merck probably didn't include these in the faked journal, but you'll see this all the time in press releases and "white papers" that are really marketing gimmicks. Why? It all boils down to Kleenex. I'll explain: When we say we are Xeroxing something, or need a box of Kleenex, or rode a Jet Ski, or got into a Jacuzzi, or drank a Coke, we might not in fact be using devices made by those companies. There are personal watercraft that Jet Ski didn't make. There are photocopiers not made by Xerox. There are hot tubs not made by Jacuzzi. All of those are brand names that have become "generic." Having a name become generic seems like a good thing, but there's a nasty consequence: When you go to court later to try to get someone to stop using your name to sell their product, the court considers whether you have been consistently enforcing your trademark. If you have made it clear that the term can only be used by you, then you can win that case. If you have allowed your term to become generic, then you no longer really "own" the terms, and you might lose. For this reason, legal departments of big companies spend a lot of their time putting trademark symbols (R and TM) next to their names on all of their materials, and sending nasty letters to anyone who uses the terms generically. All of this means that when the marketing department cooks up a really convincing fake document that says XYZ Corporation's new BackShaver is amazing, the legal guys are going to fight to put a little TM next to that word, and when they win that fight, they tell the rest of us that the article was written by the guys who make BackShaver. No one else cares about putting the TM there, and no one else is required to do it.
• The constant repetition of the company's name or the product name, often starting in the first sentence. Marketing people learn that repetition builds name recognition, and that name recognition builds sales. So they make sure they use those names far more often than a person would in natural conversation or real writing. I bet the Merck journal did do this. If it made any changes to the articles or titles of articles, in fact, my bet would be that they inserted company and product names.
  
• The absence of snide, sass, and complexity. Real, professional writers are always trying to sell you on their objectivity; marketing departments are always trying to sell you on their product. These lead to important differences in the copy. For instance, real writers often take little shots at everything and everyone, just to let you know that they're independent. These shots might be as simple as little disclaimers or qualifiers ("The new BackShaver is great at removing back hair, but I really hated it when I had to clean the device") or harmless little jabs at the Marketing Deparment itself ("I wish, though, that they'd given the BackShaver a better name -- who wants to stand in line at a store with a box that says 'BackShaver' on it?") Marketing Departments generally cannot stand these sorts of comments. If it never teases, and never has a complicated opinion involving some sort of negative, it's probably an ad in disguise.
• Dwelling on details of success that neutral writers care little about. Marketing people know that customers will buy things they think are popular, so they rarely can resist pumping even their faux materials full of user statistics ("90 million copies sold") and testimonials ("'I use it all the time,' says Suzy, a freshman at UCLA"). They also try to build up connections between the current product and previously successful products made by the same company, leading to weird paragraphs in which they say that the 2007 version was great, but the 2009 version is better. A real writer might also talk about popularity, but doesn't necessarily see popularity as a good thing (and often throws in a plug for an underdog -- the software reviewer will talk about Microsoft's popularity, but mention he uses Linux himself, for instance). When describing a new edition or upgrade that improves on the 2007 version, he'll say the obvious: that it fixes problems that the 2007 version had. Marketing people won't ever describe it that way: As far as they're concerned, the old 2007 version was also perfect, just not as super perfect as the 2009 version.
• Presence of contact information. If it has information for how to contact someone in the company, it's probably an ad, even if it doesn't look like one. (Exception: If it says "Call Bob Smith in the legal department at 111-555-1213 to complain about this critical safety issue!" then it's obviously not an ad.)
  

A few closing thoughts on this issue:

1. Don't assume that because Merck and Elsevier's Australian branch did this, that they're "getting away with it" -- or that this means you should be unethical too, in order to get ahead. Generally, the corrupt people you hear about who seem to be getting away with things aren't. You hear about what they did, but you don't see the negative consequences. (See my earlier post on this subject.)
2. A very enterprising student could do some cool detective work on this sort of thing and then report the results in a paper: Through textual analysis, study a handful of other small, short-run journals published by Elsevier with the word "Australasian" in the title (quite a few exist -- see the reader comments for the article I linked above), and look for signs of shadiness. Have the articles previously appeared elsewhere? (Web of Science will tell you.) Does one company's name or product name keep coming up, even though that company doesn't appear as a sponsor of the journal? Or what about the other five journals? We know their names (see the article), but not who sponsored them. Can you figure out which companies were involved in them by looking at the articles they published? (The enterprising student wouldn't have to cover all of this -- even covering one journal could be interesting.) Why is this a worthwhile project for a student? Among other things, discovering a fake journal while you're an undergraduate would probably lead to real publication for you (or, at least, some news stories about your discovery), and give you something to put on your CV.
3. Be on the lookout for fakery. The Internet has made this easy, because Web sites are cheap. Faking an actual book or physical journal isn't usually cost-effective, due to printing costs. But Web sites are easy to fake. (In fact, if you plan to do what I describe in #2 above, you might narrow your search to journals that don't have print versions.)
   

Churning

"But I worked sooo hard on this paper!" a student says to me, upon seeing her grade.

I believe her. But I also think she's wrong. That sounds like a contradiction, but it isn't, really. I used to tell students that there's a difference between work and what I call "churning" -- an activity that feels like work, is very unpleasant, but doesn't really do anything. When you should be working on a paper, but you keep sorting your notes, checking your email, alphabetizing your snacks, and complaining to your friend about how this paper is killing you, you're churning, not working on the paper. It's a natural inclination; I do it, too. And it feels like work, because it takes up a lot of time, and isn't fun. But it doesn't accomplish anything, other than using up your time.

Cal Newport, an MIT graduate student, calls churning "pseudowork," but we're pretty much talking about the same thing. His column (see previous link) is well worth reading, since it explains -- much better than I ever have -- why straight A students tend to spend less time studying and writing than other students.

If you've ever heard the expression "work smart, not hard," that's what he's talking about. His points are spot-on, and I strongly suspect that a lot of students would enjoy their academic lives more if they were exposed to his advice, which he sets up to be pretty easy to follow. His basic mission is to show students how to simplify their lives, have more time for enjoyment, and do better, all at the same time. It is possible. And it isn't difficult.

I used to have a whole speech about churning that I'd give students in my 1A classes, but now I'll probably just have them read Newport's post. It's pretty good.

- GS

"Margaritaville"

Following my recent lectures about popularization and accommodation, Cody Lewis (a 1SC student) emailed me about an episode of South Park, titled "Margaritaville," in which the South Park team takes a shot at explaining the recent economic meltdown, albeit crudely (pardon the double-entendre). Mr. Lewis pointed out that it was a fairly good popularization (and I, having also seen it, concurred), despite being dressed up as fiction.

Over the course of a few emails, we discussed the state of journalism today, with Jon Stewart and the creators of South Park somehow seeming -- despite the humor -- to be more like investigative journalists than our journalists are. Since I used to be a journalist, and a newspaper editor, I have a lot to say on this topic, and after telling a true story to Mr. Lewis that I thought captured the current journalistic mindset pretty well, I decided to share it here on my blog.

Here it is:

Many years back, I was a newspaper editor for a business journal. I had assigned one of my reporters a pretty standard business story -- a local company had filed for bankruptcy, and it was a big enough deal to warrant an article. He turned in his first draft, and it didn't have a lot of information in it: He said the company wouldn't return calls, so he was stuck with what he could find in the court filing itself. It took a lot of work on my part to get him to do any real digging -- to call people other than company representatives, for instance.

At one point, I asked him this: "How many creditors does the company have?"

Him (sighing): "I don't know. I'll go try to find out."

Later, he returned: "The court filing doesn't say, and the company won't call me back, so I don't know."

Me: "But you have the court filing, right?"

Him: "Yes."

Me: "And it has a list of the creditors, right?"

Him: "Yes, it does, but it doesn't say how many there are."

Me: "No, of course it doesn't. But it has a list."

Him: "Yeah, so?"

Long, long pause, as I waited for him to figure it out. He didn't. So ...

Me: "Count them."

Him (shocked): "You want me to count them?"

That's what I dealt with, pretty much every day, in a newsroom. When I read or watch the news, I still see that basic attitude, that same overreliance on pre-packaged information. Many (not all) reporters seem inclined to lean back in the child-seat and wait to be spoon-fed pre-digested, infantile matter. Didn't use to be like that, but it is now.

And that's why Jon Stewart can show them up. He knows how to count, and isn't afraid to use his fingers to do it.

- GS

Getting Away With It

I've written this blog entry as a kind of "open letter" to good, honest students who get frustrated because they think cheaters, scammers, and BS artists are doing half the work and getting the same grades. Throughout this entire posting, I'm going to assume that anyone reading it is an honest, hard-working, ethical, concerned student, and that he or she won't take offense at anything written here about dishonest students, whom I hope are not reading. (It's usually a safe assumption, but I thought I ought to state it outright.)

Anyway ... Let's start with a story.

Several years ago, three of the best students I had at the time teamed up to write a paper together. The paper did well, and they were happy. But then a couple of weeks later, they were reading papers that other students had posted online, and they stumbled across one that had ripped off roughly three of their paragraphs.

Ticked and bemused, they came to me.

"Mr. Scott, we really hate to rat on another student, but this is really bothering us. Chuck* plagiarized our earlier paper, and we thought you should know about it."

(* No, his name's not Chuck. All the specifics here are tweaked, to protect his identity -- an issue that actually ties into the point I hope to make with this blog entry.)

"Really?" I asked. "Show me."

They did. They even went so far as to print out the two versions and highlight all of the similarities in carefully documented notes.

I called Chuck in for a conference, showed him the two papers, got him to admit that he'd copied their papers, had him fill out Student Judicial Affairs paperwork, and told him he was getting an F in the class. He asked whether he should keep attending.

I said, "Well, even if you do, you'll have an F in the class, so there wouldn't be a lot of point in it."

Usually, when I say that, students get the hint and take the rest of the term off (at least, from me).

Chuck, however, stuck around. He kept showing up to class, and participating, apparently hoping to change my mind by showing me that he wasn't so easily discouraged. After he'd done this for a while, one of the students who'd turned him in came to see me, looking simultaneously sheepish and rather annoyed.

"I know you're the teacher, and that what's going on with Chuck is none of our business at this point, but I just wanted you to know that we think it's really unfair that he might still pass this class, given that he was ripping off work from other students. We're doing our own work, and working hard, and for all we know, he's just stealing stuff from other students now, instead of from us. I guess we were just wondering why you're giving him a second chance?"

Well, I wasn't, of course. I'd already failed him. But I couldn't say so -- student privacy rules prohibit me from telling you that Bob got an F on a test, or that Susan got an A, or that Chuck cheated and is failing the course. (Interesting note: If your parents call me and ask me how you're doing, I can't tell them. The same laws kick in there. Naturally, there are weird exceptions, and there are situations in which there's no helping it -- when someone on a team paper contributes plagiarized material, I often have to talk to the whole team about it, which means there's some privacy leakage along the way.)

At any rate, what this meant was that I couldn't tell Concerned Good Student that she was wrong, that Chuck had failed the class, and that he was still showing up for reasons totally unfathomable to me.

Instead, I had to say, "Well, thank you for letting me know about that. I assure you I'm taking the matter very seriously."

She snorted, like what I'd just said was PR-ese for "I don't care at all; stop bothering me." And I don't really blame her. But I couldn't tell her. Those are the rules.

Here's another rule that applies to this situation: Unless Chuck threatened to knife someone (or did something similar), I couldn't tell him to stop showing up. He'd paid his tuition. If he wanted to put in all of that work for an already promised, guaranteed F, that was his business.

As far as the three Concerned Students know, Chuck passed the class with flying colors. He didn't, of course. But as far as they're concerned, he "got away with it."

This story is not at all unusual. I see this same basic narrative repeated several times a quarter.

Speculation by good students that such-and-such bad student is "getting away with it" is common. And it's almost always wildly wrong.

I've had students come to me all worked up because they're sure the team flake who never showed up to class and never contributed to the team project is going to unfairly get the same grade as the rest of the team. If I'm aware of the behavior, he doesn't. But I can't tell a student that Bob got an F on the project, unless that student is Bob. So I say, "Thank you for the information. I'll take it into consideration." And that's pretty much all I can say.

A similar story, this time about BS: I had a student -- perhaps the best writer I've had in a class all year -- write in a blog last term that, after reading material by some fellow students, she had decided many of her fellow students were BS artists who throw together long strings of big words in an attempt to impress, even though their sentences say little, or are vague, or are empty. She concluded that most of them would probably get A's, and she'd get a B, even though she was pretty sure she wrote better than they did. She figured she'd keep writing simply, even if it meant a lower grade, out of principle.

I knew which students she was talking about, and they weren't getting A's. In some cases, they were far from it -- and for many of them, long strings of BS were the primary reason they were struggling. She, on the other hand, was a fabulous writer (still is), and should have known it by then, since that was her second class with me.

I'm not sure where these "getting away with it" narratives come from, but they're persistent -- and they're so often wrong, that I finally decided to address the matter here. These narratives are wrong for several reasons, but I'll focus on three:

1. With very rare exceptions, we're not blind, stupid, or inexperienced.

Most of us can see the obvious. Most of us have been teaching for a while. All of us, before we taught, were students: We sat in those uncomfortable chairs; tried to figure out how to arrange things on those fold-out "desktops" so that our arms and notes and other gear could all fit; watched people pass notes in front of us; felt the person behind us kick our chair repeatedly; and watched the clock a lot if the lecturer tended to drone. Outside of class, we liked to think we were super-scholars, capable of acing classes without always showing up, doing the readings, or following instructions closely -- believing this sort of thing gave us more time for dating and playing networked videogames in the dorm hallways. We all had at least one buddy who liked to brag that the 6-page paper he just turned in was "total BS, with nothing comprehensible at all." We are full people, and have histories that are much like yours, but longer. Those histories include awareness of the sorts of things that other students do. Also, from the front of the room, our view of the scene has improved a lot more than you'd think, and we see a lot more than we comment on aloud.

2. "They" are bad at gambling.

Really. Maybe they're okay in Vegas, but they're lousy gamblers in a classroom. Dishonest students have a staggeringly high likelihood of getting caught, partly because most types of cheating are easy to catch, and partly because cheaters invariably (due to random chance) make some sort of dumb error after a while. And the payoff is limited: That paper from an online essay site probably doesn't match the assignment quite right, so it's doomed at the start to receive a low (and possibly non-passing) grade; all of the screwy citation habits that go into trying to "cover up" plagiarism stand out in a paper and can cost the author points even if the plagiarism itself isn't detected.

(Think about it this way: If you don't cite your sources, you can get nailed for not citing sources. If you cite the sources you plagiarized from, the grader will notice the crime if he or she looks them up. If you cite other sources as a smokescreen, you get nailed for fabrication -- for citing a source that really didn't say what you say it said. That's a kind of academic dishonesty, too, and every bit as serious as plagiarism. If you make up sources -- as I've had a few students do -- those are the easiest to catch of them all. The whole citation thing is designed to make verification of your research possible. Any attempt to mess with that verification makes it unverifiable -- and the paper gets a lower grade because of its unverifiability. This is a very hard game to win, if one treats it like a game, which is why I say these are bad gamblers.)

What about BS? We all know BS, pretty much, when we read it. (A quick definition: BS occurs when an author is so unsure of his knowledge, understanding, or ideas, that he fills his page with verbal fog, with sentences and phrases that mean nothing to him, but which he hopes will fool others into believing an idea was present.) Teachers tend to feel insulted when they see BS. Because they want to be fair to the occasional student who has ideas but is actually unclear, they try to give some benefit of the doubt, but they still grade the paper down for poor articulation, poor grammar, poor word choice, poor style, or any number of other features that tend to go hand-in-hand with BS. That is, the BS tends to punish itself, and most teachers will simply let it do so.

3. Long-Term Ramifications

Finally, there is something like a tortoise-and-hare effect involved here. The good, honest, struggling student, who scrapes by with an honest and bloody C, might have great reasons for resenting the flakey BS-er who managed a B- simply because he did a couple of assignments well, when he cared. But over the long haul, I'd bet my money on the C student doing better.

Why? Well, that brings me to another story.

Several years ago, while I was working on my Ph.D., I enrolled in a series of undergraduate statistics classes: three quarters' worth. The first class had about 350 students in it; the second 110; and the third about 30. I hadn't taken an undergraduate-level class, with undergraduates, for a very long time, and found the experience ... fascinating.

Just before class in both of the first two courses, if homework was due, students would be madly swapping answers with each other, sometimes in full view of the teachers, who pretended to ignore the answer market. These answer swaps seldom dealt with why or how the answers worked - they just involved trading of answers, blindly. When tests came up, similar trading produced community notes for last-minute cramming sessions. Shortly after tests and homework were completed, students forgot most of what they were supposed to learn.

But each class built on the one before it. By week two or so of the third class, there was this huge gap between those of us who had tried to understand the material and those who had gamed their way through. The latter students were baffled most of the time, I'd say, and did terribly. A lot of them dropped or withdrew. If they needed the third class for graduation, I don't know how they managed it, since they'd already "passed" the previous two and thus could not retake them to learn the stuff they'd missed.

A lot of the world works like this, particularly in a university setting. Bad habits smile at you now, but kill you later. Every once in a while, a former student asks me for a letter of reference. Not surprisingly, they're all hardworking, honest students who have some reason to imagine I'd say nice things about them. The bad apples that you kind, hard-working, Concerned Students worry so often about have limited options in similar situations -- they've burned those bridges. They're trying to figure out how to pass their upper-division classes with skills that atrophied while they were faking their way through lower-division units, and they're scrambling to find anyone to write them a letter. Many of them do find three letter-writers, but they never compare well to the letters and letter-authors of the honest students: The bad apple gets a vague and bland letter from a former TA ("I can confirm that I had Chuck in a literature discussion section during the spring of 2006, and that he showed up for discussion sections a few times"), while the ethical hardworker gets a letter that says she's an ethical hardworker, written by a recognizable name in the field.

In short, don't worry about the trolls out there -- they seem tough, but tend to quietly expire off-stage. In a few years, you'll look around, and wonder where a lot of them went.