How different guitar woods sound? - Online Tests

[Samuele Carcagno]

As part of an ongoing study on the preferences of guitar players for different woods used for the back and side plates of guitars we’re conducting two online tests.
The tests, designed by the hearing lab at Lancaster University, are available here:

http://www.psych.lancs.ac.uk/hearing/th ... ine-tests/

a description of each test is provided on the above webpage. Our main aim is to assess the impact that the wood used for the back and side plates
has on the perceptual characteristics of the guitar sound. The tests are based on recordings made a professional player of 6 guitars identical in
shape, and differing only in the wood used for the back and side plates (Brazilian Rosewood, Claro Walnut, Indian Rosewood, Mahogany, Maple, Sapele).

If you are a guitar player living in the Lancaster (UK) area, you can also take part in our laboratory tests in which you can play the guitars yourself:

http://www.psych.lancs.ac.uk/hearing/th ... xperiment/

Cheers,

Sam

[Alan Carruth]

Good luck with this.

I published an article in the 'Journal' of the Catgut Acoustical Society back in May of '98 comparing two matched guitars, one in oak and the other in BRW. Since then I've tried other matched pair experiments, and decided that I'm not sure I know enough yet to make really matched instruments. It seems to me that in the absence of the ability to do that we really can't isolate variables well enough to speak with any great assurance about the results of any such experiment.

Given the wide variety of woods that you're using, I'm sure you'll get some sort of result, and it will probably broadly mirror the usual ranking of these woods. However, in cases where it's a close call I'm not sure how valid the result will be. It would not surprise me at all it the Indian rosewood guitar is judged 'better' than the Brazilian, for example, and soft maple and walnut can be nearly identical in their properties. In that case most of the differecne would be in the tops. Even if they're all from the same flitch there could be enough difference to override moderate differences in the B&S woods.

[Ryan Mazzocco]

This test seems extremely problematic.

[Doug Shaker]

I think if I were trying to do this, I would punt on the idea of trying to build identical tops. Very hard to do, harder to prove one has done it. Will not inspire confidence in the study if one claims to have done it. This means:

• use the same top for all of the guitars
  use a removable neck/fretboard design
  use a reversible glue for the top-to-side joint and for the bindings
  cycle the top through the bodies more than once, so you can see if the top is changing as the experiment continues

Might work. Might not. Has a chance, though. But, heck, I'm not doing the work, so my advice is cheap.
Good luck to the courageous folks trying to make this work.

[Adam Savage]

I suspect that if a synthetic top material was used then idiosyncrasies of different tops would be essentially removed. Perhaps not sounding entirely 'normal' acoustically, however.
A possible control for the experiment could be to have twelve samples recorded, each assigned a separate id code, but have only 6 guitars make the recordings without the listeners knowledge. Further, if a second trial was run with the guitars mis-labelled, then a perception bias could be removed or minimised.
Interesting.

Adam S

P.s, living near Stirling, I'm a wee bit too far north to just pop in, but would love to give it a shot otherwise.

[Eric Baack]

you would probably want some automated method to strum and finger chords too. So you take the human element out of that as well.

[Samuele Carcagno]

Some further info on the construction of the guitars is available here:

http://www.fyldeguitars.com/news0914.html

the guitars were carefully built to be as similar as possible except for the back and sides wood. This doesn't mean that they're identical, but I think that residual differences are unlikely to be significant. I'm not a luthier, so I can't comment much further on these issues (I'm involved in the perceptual side of the study). Ideally, we would have liked to have 5 or even 10 guitars for each wood (5 Brazilian rosewoods, 5 Mahoganys, 5 Sapeles etc..), so that variations on the tops or other aspects would average out across the samples, but that was simply not possible with our budget.

Alan, I'd be very interested to read about your study. Could you send me a link to the paper or to the abstract?

[Samuele Carcagno]

I suspect that if a synthetic top material was used then idiosyncrasies of different tops would be essentially removed. Perhaps not sounding entirely 'normal' acoustically, however.

nice idea, but yeah I guess it wouldn't sound really like a normal acoustic guitar

A possible control for the experiment could be to have twelve samples recorded, each assigned a separate id code, but have only 6 guitars make the recordings without the listeners knowledge. Further, if a second trial was run with the guitars mis-labelled, then a perception bias could be removed or minimised.
Interesting.

sorry I don't follow this, the identity of the guitars is not known to the listener. In the online tests the recordings are not labelled, and in the lab tests the guitar players are doing the tests in a darkened room while wearing welder goggles, they can't recognize the guitar they're playing, they're just barely able to see their fingers on the fingerboard.

P.s, living near Stirling, I'm a wee bit too far north to just pop in, but would love to give it a shot otherwise.

that's a shame!

[Samuele Carcagno]

you would probably want some automated method to strum and finger chords too. So you take the human element out of that as well.

that's a possible way to run the test, but we wanted a more realistic setting (can guitar players hear the differences between guitars when a song is being played?). The guitars were recorded by a professional player (who is also an artist), and each track was recorded several times. Each time the survey is taken a different recording (of the same tune) is presented. There could be small accidental variations due for example to the fingernail angle, but these variations are random (on one recording it may be that guitar X was made slightly brighter by a slightly different fingernail angle, but on another recording it may be that it was made slightly duller), so on average we expect that if guitar X is for example brighter than guitar Y this will be reflected in the survey results.

[Ryan Mazzocco]

They're trying to build guitars "exactly" the same except for the back and sides, but not enough budget to have a big enough sample of each species used to get a really good idea of what each wood offers. I get that. But why try to build them? Why not just borrow a big enough sample number of guitars from each back/sides variant? Sure, the specs, construction methods/skill and properties of the neck, top, bracing etc, etc would be different... but if you have 10 east indian rosewood guitars and compare them to 10 Maple guitars I think you would get a much better idea of what affect the wood species has on the sound of the guitar than just one sample each.
I'm interested in what you're trying to do and I'll probably participate in the test to see what we can learn... I hate to sound cynical, I just think the sample size is way too small to get an accurate conclusion.

[Samuele Carcagno]

They're trying to build guitars "exactly" the same except for the back and sides, but not enough budget to have a big enough sample of each species used to get a really good idea of what each wood offers. I get that. But why try to build them? Why not just borrow a big enough sample number of guitars from each back/sides variant? Sure, the specs, construction methods/skill and properties of the neck, top, bracing etc, etc would be different... but if you have 10 east indian rosewood guitars and compare them to 10 Maple guitars I think you would get a much better idea of what affect the wood species has on the sound of the guitar than just one sample each.
I'm interested in what you're trying to do and I'll probably participate in the test to see what we can learn... I hate to sound cynical, I just think the sample size is way too small to get an accurate conclusion.

thanks for your comments. We're renting the guitars for one year in order to run our lab tests, unlikely we would find many people willing to just lend us their guitars for such a length of time ...
Might have worked if all we were doing was only recording the guitars.

[Randy Roberts]

I applaud your intent,
but a first read of the description on your link leads me to expect this study will just add to the vast mass of misinformation that currently exists pertaining to guitars and sound.

The back and side materials are just one of a myriad of factors that result in the sound heard by a listener, and a relatively small one relative to the other factors that are variables rather than constants in the study as you describe it.

I’ll attempt an analogy:

Picture your side and back woods as spices; Brazilian Rosewood = salt, maple = pepper, Claro walnut = garlic, etc.

Your guitars are different meals; Guitar 1 = Salmon, Guitar2 = filet mignon, Guitar3 = Lasagna, etc. (your 6 “same shape” guitars are really this different.)

The study is the equivalent of taking one of these spices and using it in one of the varying dishes; Salt in the Salmon dish, garlic in the filet mignon, pepper in the Lasagna, etc.

Then your subjects taste, compare, and assess the effect of the respective spices.

Your results obviously are; Salt results in a “fishy” taste, Garlic gives you a “meaty” taste, pepper gives you a rich “tomato-y, cheesy” taste, etc.

The recipes in your cookbook then are obviously; if you use more salt it will give your soup a fishy taste, Use garlic to get a meat flavor, and pepper to make your pizza more tomato/ cheese flavor. It’s obvious because your "scientific" test says so.

Unfortunately your cookbook is going to mislead both cooks and diners. Same problem as claims made for "health" foods, "natural" medical treatments etc.


Also, "sound" as an entity does not exist, it is just an imaginary construct each individual"s brain creates to make usable sense of the impact of air molecules upon the tympanic membrane. As such, each and every person has a different construct of the same "sound". No two of your subjects is hearing the same thing from each instrument. You might want to do some reading in the field of psycho-acoustics. (Daniel J. Levitan's work is a good start)

Just to put a cherry on the top, this also results in us having no common language to describe what we are hearing. "Bright" or "warm" means something different to each listener. This has been universally a handicap to us since the first string was plucked, and brings to mind the old saw "Americans and Englishmen, forever separated by a common language".

I'm afraid you have plunged into a pond that is far deeper and darker than it appears from standing on the dock. <g>

[Bryan Bear]

I agree that this topic is well beyond the scope of this study but that doesn't mean this should not be done, only that conclusions cannot be drawn from this study alone. The early studies in any given field really serve as stepping stones to the field of study inspiring further investigation. Unfortunately people latch on to the boiled down conclusions (often misinterpreted) of single studies and run wild with them.

I understand that it was impractical to have 10 guitars of each back and side wood made for this study. However, IMHO that would have been the minimum requirement to say you are trying to account for variability in other aspects of the construction of the guitars. Even with 10 of each, at best you would be able to say that the results seem to be X for guitars of this style made with the approach and techniques of this luthier. You could easily have the same study design but use guitars made by a different hand or in a different style and have very different conclusions.

I think this undertaking is at least one step ahead of itself. You can't try to prove out that variable X can be manipulated to result Y or Z without first demonstrating that variable X has a meaningful (consistent) contribution to Y and Z. I feel like this study would have been far better served to compare two back and side options. Two that you expect to have measurable differences and try to prove that they exist. Say 15 rosewood guitars and 15 mahogany guitars. How would listeners score them? Do those scores predict the actual back and side material. Even then, you still only can say that mahogany v. rosewood has an effect on the tone of this type of guitar made in this way, but it is a start.

[Alan Carruth]

" I think that residual differences are unlikely to be significant."

That's where I disagree.

The last time I actually tried to make matched pair of guitars was for a 'playing in' experiment. I built two Classicals of redwood and mahogany. The tops came from the same plank, which was amazingly consistent. The backs and sides were also cut 'in flitch', as were the necks and the bridges. Plate weights were matched within three grams. The first nine 'free' plate modes of the top and of the back were matched within three Hz. The masses and fundamental bar modes of the bridges were matched. The lowest four modes of the assembled tops and backs were matched within a couple of Hz.. as were the lower order 'air' modes. In the end they sounded very similar, but one was clearly better than the other, and everybody who tried them agreed on that and on their preference. Since the experiment relied on having two guitars that sounded 'the same' at the outset, that was the end of that.

At the time I did not have the wherewithal to run impulse spectra, so I can't be sure, but I'm reasonably confident that the lower order modes were sufficiently 'alike' that the spectra below, say, 500 Hz would have been hard to distinguish. It's really hard to do much with Chladni patterns above 500 Hz; the power required (and noise level!) gets high, but, more the the point, you're approaching the 'resonance continuum', where small differences loom large.

And that's the point. You have to remember that we're looking at a very highly developed design here. In any such case the difference between 'good' and 'great' can be so small as to be imponderable. I think I know what caused those two guitars to be different, but until I can repeat the experiment and make two that actually sound 'the same' I can't be sure. One of these days I'll find the time to do that project. Then, of course, I'll have to do it again a couple of more times to prove that it wasn't a fluke.

If I (or somebody) can do that, then we'll be in a position to start looking at variables. I've got a red cedar top and a Red spruce one that have exactly the same density, and Young's modulus along and across the grain, Essentially, they're identical except for the damping factor. It's an experiment that's crying to get done, but it won't mean much if I can't first make 'identical' guitars with 'the same' wood.

The bottom line is that his is 'way more complicated than it 'should' be, and that severely limits the sort of conclusions you can draw from any such experiment. The most viable alternative, IMO, would be to get access to a really large number of guitars made in a factory like Taylor or Martin, where the parts are made on CNC machines so that the dimensions are as nearly 'identical' as possible. In that case you could take readings on the wood properties, and, with a sufficiently large sample, attempt to draw some conclusions as to the main variables. Either the 'careful luthier' or 'brute force' approaches have drawbacks, of course, which anybody could point out. I can't think of a third way. In this case, I'm not sure you've been 'careful' enough, and the sample is 'way too small to be valid as a 'brute force' experiment.

[Alan Carruth]

I remembered another point that needs to be made.

In an article in the Catgut 'Journal' in May of '97, Joe Curtin talked about the difficulties of evaluating an instrument based on the tone a player gets. Basically, a poor player won't get good tone out of anything, and part of the professional competence of a good player is the ability to get the tone they want from any instrument. They do this automatically: you can't get them to turn it off. He, along with physicist Gabriel Weinrich, came up with a 'reciprocal bow' test for violins that allows one to 'play' the instrument without a player in the loop. This is described in the article, and an example is given of a fiddle that nobody liked even though it sounded wonderful.

This is only one of several methods of standardizing the way the instrument is driven. I've used a mechanical plucker, based on the fact that fine magnet wire is so consistent that it breaks at almost exactly the same load every time. You loop a length of #44 wire under the string and pull upward until the wire breaks. thus controlling the force, location, and direction of the 'pluck'. It has been suggested that .3mm pencil lead could be used in the same way, and it might be more convenient. The sound can be recorded and played back for comparison. With some effort you could even assemble a short musical selection.

All of the Catgut Journals are archived, and available on line as .pdf files. I don't have the URL for that, but it should be available through the Violin Society of America, which took over the Catgut Society in 2004. If memory serves, the archive is at Stanford University, and you could try there if a Google search doesn't turn it up.

[Ryan Mazzocco]

I kinda feel sorry for the guy. It's like he just jumped into a shark tank.
We're not trying to attack you. It's cool that you're trying to scientifically quantify something like this, even if it's unquantifiable. It's just that this sort of thing has been an on going discussion in this forum (and others like it) for years now.
Everything that has been said here is exactly what most everyone who builds guitars already knows.
We really hate to poo-poo on your study, especially considering how much work has already gone into it. But the problems really don't even end with the guitars themselves. I participated in both tests last night. I found my personal results to be predictable, which is good news for what I think you guys are trying to prove. But especially in the second test. Sample 1 and 2 play the same tune on different guitars and then you have to pick out which guitar is playing sample 3. Maybe my ear just isn't as good as I had always thought, but I found it very hard to figure out. I expected that I may have difficulty telling the difference or trying to identify which wood was used on which samples. But here's where I started having problems. I would listen to, say, sample 2 and it sounded very similar while sample 1 as very different. So I would answer that sample 2 is the matching guitar. So many times this happened that I was absolutely, without a doubt sure of what I was hearing. Yet, I was wrong! So, back to where I started... The problem doesn't end with the unintended differences with the guitars. You also have to consider the recording equipment, the microphone (it's sensitivity and responsiveness), mixer, EQ settings, preamps, software, even cables; as well as the vast variation in playback set up and equipment. Are the participants listening through noise canceling headphone? earbuds? PC speakers? stereo speakers? What are the audio output settings on their computer? Think about the recording itself. Many of the times I got fooled it was because I was listening for specific low end responses. The same guitar sounded different in different recordings. You had the same skilled musician playing all tracks as perfectly exact as humanly possible, set up exactly the same, same room, same mics and equipment... But what if in one recording he is sitting an inch further away from the microphone? The proximity affects the listeners perception of loudness, sustain and definitely the low end.
Like I said before, I love what you guys are trying to do. But just look at Alan's post. It's been done and done to death with very little concrete results. Everything involved is too subjective.

[Bryan Bear]

Hopefully Samuele doesn't feel like he's been thrown to the Sharks. This is akin to peer review. Any study design is to be critiqued as part of the process. It is very unlikely that any study design will be able to control for every possible pitfall given the resources available. Acknowledgement of the potential shortcomings goes hand in hand with attempting to draw conclusions. As I said before, just because it is very difficult to design a study like this doesn't mean it shouldn't be done. It just shouldn't be used as the sole support for any claim. More experiments will come and the results of this one will be one of the building blocks for a better understanding. The periodic table of elements wasn't organized after one lab test. . .

[Samuele Carcagno]

Hi guys,

first of all, thank you to all of you who have taken the tests, and have commented on the thread. Criticism is welcome, of course I appreciate if the tone of the discussion is civilized (it's been civilized on this forum, but less so on other forums). These discussions are indeed similar to peer review, I'm used to that, and there are lots of interesting ideas that have come up from the discussions. There are surely things that could have been done differently, but before embarking on a large-scale study with dozens of guitars that could cost hundreds of thousands of pounds I'm very confident that several useful results will come up from this smaller-scale study.

Back to the discussion. Our aim is not to test whether guitar X sounds better, or brighter, or warmer than guitar Y when they're both being played by a robot in a perfectly controlled environment. Our aim is to test this in a quasi-realistic setting. Personally, if I walk into a shop and have to choose between a £6,000 guitar X and a £2,000 guitar Y I wouldn't be interested in knowing whether in a perfectly controlled environment people are barely able to hear a tiny difference between the two guitars, I would rather be interested in knowing whether when played many times by a competent player, competent listeners can consistently judge guitar X to sound better, brighter, warmer, ect... than guitar Y.

Also, generally speaking, a lot of attention has been given to the physical aspects of guitar making, but the perceptual side of things has not been given much attention. If a precise measuring instrument can detect a difference between guitar X and guitar Y in a given physical characteristic, is this difference perceptually important? In other words, can competent listeners consistently hear it under blinded conditions (i.e. without knowing the identity of the guitar)? I see lots of claims about possible small physical differences in some attributes of the guitars (e.g. the tops, which were carefully selected as to be as similar as possible but may nonetheless show some variation) as being huge confounds in the study, but I haven't seen any hard evidence of this.

[Samuele Carcagno]

All of the Catgut Journals are archived, and available on line as .pdf files. I don't have the URL for that, but it should be available through the Violin Society of America, which took over the Catgut Society in 2004. If memory serves, the archive is at Stanford University, and you could try there if a Google search doesn't turn it up.

Thanks Alan, I was able to find the article at this URL:

https://stacks.stanford.edu/file/druid: ... gb0065.pdf

[Bryan Bear]

BTW Samuele, I haven't yet taken the test because I am waiting for a chance to give it my attention and use descent speakers but I will get to it eventually.

"I see lots of claims about possible small physical differences in some attributes of the guitars (e.g. the tops, which were carefully selected as to be as similar as possible but may nonetheless show some variation) as being huge confounds in the study, but I haven't seen any hard evidence of this."

But you can't produce evidence of it being or not being a confounder unless you test to control for everything but that (impossible I know). If it is present in an experiment to test another variable, you will never be able to say definitively that it does or does not influence your results. You have an anecdotal claim from Al that says it did (in that case) influence tone to a noticeable degree. This is where boosting your n will help but the water gets muddy that way too.

Also, before I take the tests, is there a concern with having participants know about the design? I suspect not but wanted to ask. I would also admit that I don't have as well developed ear for tone as you might like in your subjects; you say competent listeners how are you defining that (I very well may not be one). I'd hate to waste one of the participant slots the IRB approved <G>

Sorry to hear that you have encountered less than civil discussions on other fora. The internet is a strange place. . .