Wow. What a strange advice. All it takes to be an expert on anything is labeling yourself as such. I'm an expert on a lot of things.

Being competent at something, on the contrary, means actually going there, doing the stuff and learning the craft.

If the guy trying to teach me mechanics doesn't have dirty hands and nails, I'm not very interested in what he has to say.

Sometimes outside perspective is important.

  --  Niels Henrik David Bohr

The Copy & Paste one is rubbish. We say copy & paste is bad because 99% of the time you see it, it is bad.

The authors blithely ignore this to make an intellectual point that there are occasional uses to cloning code. Of course there are. A complete waste of words.

"For example, one way to evaluate possible new features for a system is to clone the affected subsystems and introduce the new features there, in a kind of sandbox testbed. As features mature and become stable within the experimental subsystems, they can be migrated incrementally into the stable code base; in this way, the risk of introducing instabilities in the stable version is minimized."

One might say, branching? Indeed, the paper mentions "forking" and boilerplate code. Many examples are poor, where a better language would be able to abstract at a higher level and not require "cloning". One example required "cloning" because the developer didn't have write access to the section he wanted to fix.

As far as real "copy and pasting": "Common examples include the initial lines of for loops".

But hey, I don't have a good survey to back up the fact that most of the real "copy and paste" I see in programming is laziness or poor platform limitations that end up being a pain in the ass and introducing more bugs.

Makes it sound like another candidate for new abstraction facilities (not really all that new -- see APL and its descendants).

But further than that, truth is they are not the first one to come up with those results. Following is a blatant selection of near verbatim quotes from Code Complete (the author makes a great job covering the subject).

"Microsoft's applications division has found that it takes three hours to find and fix a defect by using code inspection, a one-step technique, and 12 hours to find and fix a defect by using testing, a two-step technique (Moore 1992)."

"Collofello and Woodfield reported on a 700,000-line program built by over 400 developers (1989). They found that code reviews were several times as cost-effective as testin - a 1.38 return on investment vs. 0.17."

"[...]the Software Engineering Laboratory found that code reading detected about 80 percent more faults per hour than testing (Basili and Selby 1987). "

"A later study at IBM found that only 3.5 staff hours were needed to find each error when using code inspections, whereas 15-25 hours were needed to find each error through testing (Kaplan 1995)."

    Table 20-2. Defect-Detection Rates Removal Step
    				 Lowest Rate Modal Rate Highest Rate
 
    Informal design reviews			25%	35%	40%
    Formal design inspections    		45%	55%	65% 
    Informal code reviews			20%	25%	35% 
    Formal code inspections			45%	60%	70% 
    Modeling or prototyping			35%	65%	80% 
    Personal desk-checking of code		20%	40%	60% 
    Unit test		        		15%	30%	50% 
    New function (component) test		20%	30%	35% 
    Integration test			        25%	35%	40% 
    Regression test				15%	25%	30% 
    System test			        	25%	40%	55% 
    Low-volume beta test (<10 sites)    	25%	35%	40%
    High-volume beta test (>1,000 sites)	60%	75%	85%
     
    Source: Adapted from Programming Productivity (Jones 1986a), 
    "Software Defect-Removal Efficiency" (Jones 1996), and 
    "What We Have Learned About Fighting Defects" (Shull et al. 2002).

"one study of "junior and senior computer science students" (described in an article I have to pay $19 to read) doesn't make somebody an expert on anything"

... is really wrong.

[Edit] Formatting nightmare

Your additional citations are irrelevant because "testing" and "TDD" are not the same thing.

* The size of the company something is being developed for. IBM is not the same environment as some startup, and what works for one may well not work for the other.

* The economic goals and risks of the system: in other words, like someone else mentions, flight control systems have different incentives than a web page. What works for one doesn't work for the other (if you develop a web page as slowly and carefully as the flight control thing, your competitors will eat you alive).

* A long term look at how the code lives and evolves. Perhaps some things are quicker to code up. How do they stand up to maintenance, and adding new features with time? And as new team members are added?

As an example the TDD study you mentioned compared the defect rates of a new software developed once with TDD and once with Code Review. At work we do TDD mostly to help us developing (faster feedback if the code does what i want, running it on the target needs 10+min), to have an example how the code should be used and to now that a refactoring broke something unrelated.

If it helps us with a refactoring or reduces the defect reduction rate that is a nice benefit but not the main reason why we use TDD.

So it depends on what you do if a study is applicable or not.

The study is much more thoroughly debunked in the comments than I will attempt to reproduce here. Important points: 1) The study's hypothesis was that TDD would produce software with equal, or fewer defects than inspection. That would be one, and depending who you talk to, by no means the most significant benefit of TDD. 2) The actual data collected by the researches showed no statistically significant difference in the two approaches.

Reading studies about the effectiveness of different tools and methods is a great idea. Treating them uncritically is not. :)

John Graham-Cumming, the author of the article submitted here, has a review of this book on Amazon.com:

http://www.amazon.com/Making-Software-Really-Works-Believe/d...

"This isn't a book about evangelizing the latest development fad, it's about hard data on what does and does not work in software engineering."

"Are you going to believe me, or what you see with your own eyes?"

Besides the assorted cognitive biases that lead people to be convinced of the (non-)existence of $DEITY (and the absolute superiority of $EDITOR), there's the risk of deliberate trickery by salesmen and consultants who've been studying stage magicians.

One fine day a consultant arrived and announced that she was going to institute a new development paradigm. It was going to be world-class cool and whizzy, and improve our productivity and reduce our bug count.

The process? It consisted of a whiteboard mounted in the engineering area, with everyone's name and some heiroglyphs by the names, and some dates.

"Huh?" we said.

"It's the new (whatever her last name was) software development process. We put your name up here, with these symbols that tell us whether you're behind or ahead of schedule that get updated every day by me."

"WTF?", we said.

"I'm doing this with you guys, for free, because I want to get a business process patent out of it."

She lasted two days.

[1] Any solution that might work, except good software engineering and project management practices, that is. Sigh. There are no silver bullets.

Up to this point, I thought you were going to tell us the new process turned out to be a genuine improvement.

IME, there are few ways to improve productivity in a small development team with a better ROI than providing vast amounts of whiteboard space right next to where people work.

A bonus point is awarded for each different colour of pen.

Ten bonus points are awarded for any technology that can immediately capture the current state of the board and save it in a standard graphics file format for future reference or wider circulation.

I just take photos with my phone and email them around. No need for expensive captures.

[Within minutes of our first meeting with the aforementioned short-timer consultant we were calling the whiteboard a "wall of shame"]

Then these people get up on their high horse about how they've got the solutions to programming and would you idiots just listen to this wisdom and why aren't you listening and come on man up this needs to be made scientific come on. Sorry, no, your wisdom is paper thin to the point that it can't even support its own weight when you pick it up, let alone try to actually apply it to anything. It's orders of magnitude of value away from being strong enough to support you standing on it and preaching.

Yes, the preaching makes people a bit grumpy.

I find that too. Most of them seem so incommensurate with what they purport to be studying as to be nearly trivial. The researchers rarely seem to address (or even be aware of) the assumptions they're making, and their assumptions are usually significant enough to dominate the data.

That's not to say one can't extract value from such studies, but what the value is is so open to interpretation that everyone ends up relying on their pre-existing preferences to decide the issue, which defeats the purpose.

Edit: that study on code cloning that AngryParsley cited above is an example. They make some good distinctions among reasons why programmers duplicate code. But their empirical findings are dominated by their own view of what's valuable vs. not. They admit as much:

Rating how a code clone affects the software system is undoubtedly the most controversial aspect of this study, and also the most subjective.

I have mixed feelings about this study. On the one hand, it's good to see people working diligently to study real codebases. At least someone is trying to look at data. On the other hand, how they're interpreting it is no different than what we all do when we argue this shit online or over beers or – more to the point – when hashing out a design decision. This isn't science, it's folklore with benefits. The problem is that it's being shoehorned into a scientific format it can't live up to.

Their title, by the way, is a straw man. What they're really arguing is that not all forms of code duplication are equally bad, that some are good choices under certain circumstances like platform and language constraints. That's reasonable (if bromidic) and even interesting, but it's just musing. It's not at all up to the authoritative status that AngryParsley gave it; it merely looks that way because it was published in a journal. The reality is that they have an opinion and looked at some code. At least they did look at some code.

Nobody is going to change their mind because of such work, nor should they. It isn't nearly strong enough to justify throwing out one's own hard-won opinions-based-on-experience-to-date. The net result is that everyone will look at it and see what they already believe. For example, I look at it as a Lisp programmer and the examples seem almost comedic. It's obvious that in a more powerful language, you could eliminate most if not all of that duplication, so what the paper really shows is that language constraints force programmers into tradeoffs where duplication is sometimes the lesser evil. Exactly what I already believed.