Why not do a real scientific test?

Ed. Stoebenau wrote:

James Higby wrote:

The old-timers (gotta be careful when I use that term) like to go around
saying, "Well, in my day, a coin was either uncirculated or it wasn't." But
let's take all those guys in a time machine back to 1955 and set them down
at a table upon which are laid out 500 uncirculated silver dollars. Then
let's instruct them to sort those coins into two piles, the ones above
average and the ones below average. Very easily done, no doubt in my mind.
Then let's instruct them to sort each of those piles into several sub-piles
until they had a total of eleven piles, in ascending order of desirability.
My money bets on them being quite capable of completing the task. Thus the
monster has always existed. The only thing different is that it now has a
name and a face.

Sure, and everyone's piles will all be different from one
another. Ask them to do the task again, and they'll be different
from what they got before. There are gradiations within grades,
but it's a multidimensional continuum and not something which
anyone can do at the precision assumed in the current number of
uncirculated grades.

It would be quite easy to do a scientific test that shows how much
precision an experienced coin collector or dealer actually has.

Have someone put some sort of numbered label on each coin, covering
up the label if the coins are in containers (post-it-notes?), writes
down which number is which grade. The numbers should be assigned
by a dice roll or by getting numbers from the hotbits website.

That person goes away with the list so as to make it a double-blind test.

Then the person being tested tries to grade the coins, writing down
the results.

Compare the two lists, and it's a simple matter of statistics to find
out that that person has, say, a 95% chance of being within 2 grades
of the label. This is, of course the combined error of the testee
and the original grader.

Repeat for several coin collectors / dealers, and you will know that
most coins are plus or minus N grades from what is on the label.
Then publish it as an article in a cion magazine, inviting readers
to repeat the test and send you the results.

You could even find out which of you is the best and the worst at
grading coins!

snip
Repeat for several coin collectors / dealers, and you will know that
most coins are plus or minus N grades from what is on the label.
Then publish it as an article in a cion magazine, inviting readers
to repeat the test and send you the results.

You could even find out which of you is the best and the worst at
grading coins!



Did anybody happen to save the Coin World slabbing test printed in the
November 17th 2003 issue? I've googled everything I could think of and can't
find it online. The PNG/ICTA survey is easily found however.

wrote in message
...



Ed. Stoebenau wrote:

James Higby wrote:

The old-timers (gotta be careful when I use that term) like to go around
saying, "Well, in my day, a coin was either uncirculated or it wasn't."
But
let's take all those guys in a time machine back to 1955 and set them
down
at a table upon which are laid out 500 uncirculated silver dollars. Then
let's instruct them to sort those coins into two piles, the ones above
average and the ones below average. Very easily done, no doubt in my
mind.
Then let's instruct them to sort each of those piles into several
sub-piles
until they had a total of eleven piles, in ascending order of
desirability.
My money bets on them being quite capable of completing the task. Thus
the
monster has always existed. The only thing different is that it now has
a
name and a face.

Sure, and everyone's piles will all be different from one
another. Ask them to do the task again, and they'll be different
from what they got before. There are gradiations within grades,
but it's a multidimensional continuum and not something which
anyone can do at the precision assumed in the current number of
uncirculated grades.

It would be quite easy to do a scientific test that shows how much
precision an experienced coin collector or dealer actually has.

Have someone put some sort of numbered label on each coin, covering
up the label if the coins are in containers (post-it-notes?), writes
down which number is which grade. The numbers should be assigned
by a dice roll or by getting numbers from the hotbits website.

That person goes away with the list so as to make it a double-blind test.

Then the person being tested tries to grade the coins, writing down
the results.

Compare the two lists, and it's a simple matter of statistics to find
out that that person has, say, a 95% chance of being within 2 grades
of the label. This is, of course the combined error of the testee
and the original grader.

Repeat for several coin collectors / dealers, and you will know that
most coins are plus or minus N grades from what is on the label.
Then publish it as an article in a cion magazine, inviting readers
to repeat the test and send you the results.

You could even find out which of you is the best and the worst at
grading coins!

My premise did not address the issue of "how well" anyone could grade, nor
"how close" anyone could come to someone else's opinion on grade. It
addressed the issue of the folks who claim that there is only one version of
UNC. That they would be proven wrong seems intuitively obvious to all but
the most casual observer. It matters not whether they could distinguish
three versions, eleven versions, or a hundred versions.

James

The PNG survey was neither scientific nor valid in respect to grading
accuracy. It actually measured the personal preferences of the
respondants and the market acceptance of brands of slabs rather than
grading accuracy.

The reason you will not see a scientifc grading study is because it
would prove what everyone already knows - that there is no single grade
everyone will assign to the same coin on any reliable basis.

The test procedure you suggest would not be very scientific. Basically
you are confusing several different issues in measurement and grading.

First is the concept that everyone's grading standards are the same.
Although we all wish they were, we all also know that they are not. So,
a coin that is graded MS 63 by one person and MS 67 by another may have
been properly graded according to the standards of the individual
graders. Before we can conduct any type of "scientific" analysis there
has to be a uniform grading system that can be uniformly applied by all
persons. As long as the concept of "market" grading is in place (where
you are essentially grading the coin to what you think it is worth and
not to its appearance of wear, strike, luster, etc) uniform grading
will never exist.

Second, you are confusing the statistical, and scientific, concepts of
accuracy and precision. Accuracy is how close a measuring system comes
to the "true" value. Because of the problems with the market grading
system I doubt that we can ever assess grading accuracy (averaging
several grading attempts is not a way to assess the "true" grade -- see
my earlier posting on this topic or my editorial in Coin World).
Precision is how close a measuring system produces the same value each
time it is used. You can be very precise (same value each time) while
being very innaccurate (way off the true value). Similarly, you can be
very accurate (one measurement hits the true value dead on) but very
imprecise (not able to acheive the same value each time).

Since we all know that we cannot impose the same grading standards on
all slabbers and collectors, it is pretty useless to try. We can,
however, assess the precision of each grading company. Accumulate a
bunch of coins and send them to a grading company. Get the results,
crack them out of their slabs and send the into the same company again.
Do this three or four times. Eventually you will get a reasonable
assessment of how precise that grading company is in applying their own
standards. Now do the same thing for another company. With the results
for both companies in hand you could make a statement that one company
is more, or less, precise in its grading standards than onther. You may
even find that precision will vary as to coin type (ANACS claims to be
the dollar experts in their ads -- maybe they are, in which case ANACS
would have the highest precision for grading dollar coins, although
perhaps a lower precision for other denominations).

Thomas Greiner

The test procedure you suggest would not be very scientific. Basically
you are confusing several different issues in measurement and grading.

First is the concept that everyone's grading standards are the same.
Although we all wish they were, we all also know that they are not. So,
a coin that is graded MS 63 by one person and MS 67 by another may have
been properly graded according to the standards of the individual
graders. Before we can conduct any type of "scientific" analysis there
has to be a uniform grading system that can be uniformly applied by all
persons. As long as the concept of "market" grading is in place (where
you are essentially grading the coin to what you think it is worth and
not to its appearance of wear, strike, luster, etc) uniform grading
will never exist.

Second, you are confusing the statistical, and scientific, concepts of
accuracy and precision. Accuracy is how close a measuring system comes
to the "true" value. Because of the problems with the market grading
system I doubt that we can ever assess grading accuracy (averaging
several grading attempts is not a way to assess the "true" grade -- see
my earlier posting on this topic or my editorial in Coin World).
Precision is how close a measuring system produces the same value each
time it is used. You can be very precise (same value each time) while
being very innaccurate (way off the true value). Similarly, you can be
very accurate (one measurement hits the true value dead on) but very
imprecise (not able to acheive the same value each time).

Since we all know that we cannot impose the same grading standards on
all slabbers and collectors, it is pretty useless to try. We can,
however, assess the precision of each grading company. Accumulate a
bunch of coins and send them to a grading company. Get the results,
crack them out of their slabs and send the into the same company again.
Do this three or four times. Eventually you will get a reasonable
assessment of how precise that grading company is in applying their own
standards. Now do the same thing for another company. With the results
for both companies in hand you could make a statement that one company
is more, or less, precise in its grading standards than onther. You may
even find that precision will vary as to coin type (ANACS claims to be
the dollar experts in their ads -- maybe they are, in which case ANACS
would have the highest precision for grading dollar coins, although
perhaps a lower precision for other denominations).

Thomas Greiner