Chapter 23

SOURCES OF ERROR IN JUDGING INTELLIGENCE

ARE INTELLIGENCE TESTS SUPERFLUOUS? Binet tells us that he often encountered the criticism that intelligence tests are superfluous, and that in going to so much trouble to devise his measuring scale he was forcing an open door. Those who made this criticism believed that the observant teacher or parent is able to make an offhand estimate of a child's intelligence which is accurate enough. "It is a stupid teacher," said one, "who needs a psychologist to tell her which pupils are not intelligent." Every one who uses intelligence tests meets this attitude from time to time.

This should not be surprising or discouraging. It is only natural that those who are unfamiliar with the methods of psychology should occasionally question their validity or worth, just as there are many excellent people who do not "believe in" vaccination against typhoid and small pox, operations for appendicitis, etc.

There is an additional reason why the applications of psychology have to overcome a good deal of conservatism and skepticism; namely, the fact that every one, whether psychologically trained or not, acquires in the ordinary experiences of life a certain degree of expertness in the observation and interpretation of mental traits. The possession of this little fund of practical working knowledge makes most people slow to admit any one's claim to greater expertness. When the astronomer tells us the distance to Jupiter, we accept his statement, because we recognize that our ordinary experience affords no basis for judgment about such matters. But every one acquires more or less facility in distinguishing the coarser differences among people in intelligence, and this half-knowledge naturally generates a certain amount of resistance to the more refined method of tests.

It should be evident, however, that we need more than the ability merely to distinguish a genius from a simpleton, just as a physician needs something more than the ability to distinguish an athlete from a man dying of consumption. It is necessary to have a definite and accurate diagnosis, one which will differentiate more finely the many degrees and qualities of intelligence. Just as in the case of physical illness, we need to know not merely that the patient is sick, but also why he is sick, what organs are involved, what course the illness will run, and what physical work the patient can safely undertake, so in the case of a retarded child, we need to know the exact degree of intellectual deficiency, what mental functions are chiefly concerned in the defect, whether the deficiency is due to innate endowment, to physical illness, or to faults of education, and what lines of mental activity the child will be able to pursue with reasonable hope of success. In the diagnosis of a case of malnutrition, the up-to-date physician does not depend upon general symptoms, but instead makes a blood test to determine the exact number of red corpuscles per cubic millimeter of blood and the exact percentage of hæmoglobin. He has learned that external appearances are often misleading. Similarly, every psychologist who is experienced in the mental examination of school children knows that his own or the teacher's estimate of a child's intelligence is subject to grave and frequent error.

THE NECESSITY OF STANDARDS. In the first place, in order to judge an individual's intelligence it is necessary to have in mind some standard as to what constitutes normal intelligence. This the ordinary parent or teacher does not have. In the case of school children, for example, each pupil is judged with reference to the average intelligence of the class. But the teacher has no means of knowing whether the average for her class is above, equal to, or below that for children in general. Her standard may be too high, too low, vague, mechanical, or fragmentary. The same, of course, holds in the case of parents or any one else attempting to estimate intelligence on the basis of common observation.

THE INTELLIGENCE OF RETARDED CHILDREN USUALLY OVERESTIMATED. One of the most common errors made by the teacher is to overestimate the intelligence of the over-age pupil. This is because she fails to take account of age differences and estimates intelligence on the basis of the child's school performance in the grade where he happens to be located. She tends to overlook the fact that quality of school work is no index of intelligence unless age is taken into account. The question should be, not, "Is this child doing his school work well?" but rather, "In what school grade should a child of this age be able to do satisfactory work?" A high-grade imbecile may do average work in the first grade, and a high-grade moron average work in the third or fourth grade, provided only they are sufficiently over-age for the grade in question.

Our experience in testing children for segregation in special classes has time and again brought this fallacy of teachers to our attention. We have often found one or more feeble-minded children in a class after the teacher had confidently asserted that there was not a single exceptionally dull child present. In every case where there has been opportunity to follow the later school progress of such a child the validity of the intelligence test has been fully confirmed.

The following are typical examples of the neglect of teachers to take the age factor into account when estimating the intelligence of the over-age child:--

_A. R. Girl, age 11; in low second grade._ She was able to do the work of this grade, not well, but passably. The teacher's judgment as to this child's intelligence was "dull but not defective." What the teacher overlooked was the fact that she had judged the child by a 7-year standard, and that, instead of only being able to do the work of the second grade indifferently, a child of this age should have been equal to the work of the fifth grade. In reality, A. R. is definitely feeble-minded. Although she is from a home of average culture, is 11 years old, and has attended school five years, she has barely the intelligence of the average child of six years.

_D. C. Boy, age 17; in fifth grade._ His teacher knew that he was dull, but had not thought of him as belonging to the class of feeble-minded. She had judged this boy by the 11-year standard and had perhaps been further misled by his normal appearance and exceptionally satisfactory behavior. The Binet test quickly showed that he had a mental level of approximately 9 years. There is little probability that his comprehension will ever surpass that of the average 10-year-old.

_R. A. Boy, age 17; mental age 11; sixth grade; school work "nearly average"; teacher's estimate of intelligence "average."_ Test plainly shows this child to be a high-grade moron, or border-liner at best. Had attended school regularly 11 years and had made 6 grades. Teacher had compared child with his 12-year-old classmates.

_H. A. Boy, age 14; mental age 9-6; low fourth grade; school work "inferior"; teacher's estimate of intelligence "average."_ The teacher blamed the inferior quality of school work to "bad home environment." As a matter of fact, the boy's father is feeble-minded and the normality of the mother is questionable. An older brother is in a reform school. We are perfectly safe in predicting that this boy will not complete the eighth grade even if he attends school till he is 21 years of age.

_F. I. Boy, age 12-11; mental age 9-4; third grade; school work "average"; teacher's estimate of intelligence "average"; social environment "average"; health good and attendance regular._ Intelligence and school success are what we should expect of an average 9-year-old.

_D. A. Boy, age 12; mental age 9-2; third grade; school work "inferior"; teacher's estimate of intelligence "average."_ Teacher imputes inferior school work to "absence from school and lack of interest in books"; we have yet to find a child with a mental age 25 per cent below chronological age who _was_ particularly interested in books or enthusiastic about school.

_C. U. Girl, age 10; mental age 7-8; second grade; school work "average"; teacher's estimate of intelligence "average."_ Teacher blames adenoids and bad teeth for retardation. No doubt of child's mental deficiency.

_P. I. Girl, age 8-10; mental age 6-7; has been in first grade 2½ years; school work "average"; teacher's estimate of intelligence "average."_ The mother and one brother of this girl are both feeble-minded.

_H. O. Girl, age 7-10; mental age 5-2; first grade for 2 years; school work "inferior"; teacher's estimate of intelligence "average."_ The teacher nevertheless adds, "This child is not normal, but her ability to respond to drill shows that she has intelligence." It is of course true that even feeble-minded children of 5-year intelligence are able to profit a little from drill. Their weakness comes to light in their inability to perform higher types of mental activity.

THE INTELLIGENCE OF SUPERIOR CHILDREN USUALLY UNDERESTIMATED. We have already mentioned the frequent failure of teachers and parents to recognize superior ability.[7] The fallacy here is again largely due to the neglect of the age factor, but the resulting error is in the opposite direction from that set forth above. The superior child is likely to be a year or two younger than the average child of his grade, and is accordingly judged by a standard which is too high. The following are illustrations:--

[7] See p. 13 _ff._

_M. L. Girl, age 11-2; mental age "average adult" (16); sixth grade; school work "superior"; teacher's estimate of intelligence "average."_ Teacher credits superior school work to "unusual home advantages." Father a college professor. The teacher considers the child accelerated in school. In reality she ought to be in the second year of high school instead of in the sixth grade.

_H. A. Boy, age 11; mental age 14; sixth grade; school work "average"; teacher's estimate of intelligence "average."_ According to the supplementary information the boy is "wonderfully attentive," "studious," and possessed of "all-round ability." The estimate of "average intelligence" was probably the result of comparing him with classmates who averaged about a year older.

_K. R. Girl, age 6-1; mental age 8-5; second grade; school work "average"; teacher's estimate of intelligence "superior"; social environment "average."_ Is it not evident that a child from ordinary social environment, who does work of average quality in the second grade when barely 6 years of age, should be judged "very superior" rather than merely "superior" in intelligence? The intelligence quotient of this girl is 140, which is not reached by more than one child in two hundred.

_S. A. Boy, age 8-10; mental age 10-9; fourth grade; school work "average"; teacher's estimate of intelligence "average."_ Teacher attributed school acceleration to "studiousness" and "delight in school work." It would be more reasonable to infer that these traits are indications of unusually superior intelligence.

OTHER FALLACIES IN THE ESTIMATION OF INTELLIGENCE. Another source of error in the teacher's judgment comes from the difficulty in distinguishing genuine dullness from the mental condition which results sometimes from unfavorable social environment or lack of training.

_V. P. Boy, age 7._ Had attended school one year and had profited very little from the instruction. He had learned to read very little, spoke chiefly in monosyllables, and seemed "queer." The teacher suspected his intelligence and asked for a mental examination. The Binet test showed that except for vocabulary, which was unusually low, there was practically no mental retardation. Inquiry disclosed the fact that the boy's parents were uneducated deaf-mutes, and that the boy had associated little with other children. Four years later this boy was doing fairly well in school, though a year retarded because of his unfavorable home environment.

_X. Y. Boy, age 10._ Son of a successful business man, he was barely able to read in the second reader. The Binet test revealed an intelligence level which was absolutely normal. The boy was removed to a special class where he could receive individual attention, and two years later was found doing good work in a regular class of the fifth grade. His bad beginning seemed to have been due to an unfavorable attitude toward school work, due in turn to lack of discipline in the home, and to the fact that because of the father's frequent change of business headquarters the boy had never attended one school longer than three months.

Another source of error in judging intelligence from common observation is the tendency to overestimate the intelligence of the sprightly, talkative, sanguine child, and to underestimate the intelligence of the child who is less emotional, reacts slowly, and talks little. One occasionally finds a feeble-minded adult, perhaps of only 9- or 10-year intelligence, whose verbal fluency, mental liveliness, and self-confidence would mislead the offhand judgment of even the psychologist. One individual of this type, a border-line case at best, was accustomed to harangue street audiences and had served as "major" in "Kelly's Army," a horde of several hundred unemployed men who a few years ago organized and started to march from San Francisco to Washington.

BINET'S QUESTIONNAIRE ON TEACHERS' METHODS OF JUDGING INTELLIGENCE.[8] Aroused by the skepticism so often shown toward his test method, Binet decided to make a little study of the methods by which teachers are accustomed to arrive at a judgment as to a child's intelligence. Accordingly, through the coöperation of the director of elementary education in Paris, he secured answers from a number of teachers to the following questions:--

[8] See p. 169 _ff._ of reference 2, at end of this book

1. _By what means do you judge the intelligence of your pupils?_ 2. _How often have you been deceived in your judgments?_

About 40 replies were received. Most of the answers to the first question were vague, one-sided, "verbal," or bookish. Only a few showed much psychological discrimination as to what intelligence is and what its symptoms are. There was a very general tendency to judge intelligence by success in one or more of the school studies. Some thought that ability to master arithmetic was a sure criterion. Others were influenced almost entirely by the pupil's ability to read. One teacher said that the child who can "read so expressively as to make you feel the punctuation" is certainly intelligent, an observation which is rather good, as far as it goes. A few judged intelligence by the pupil's knowledge of such subjects as history and geography, which, as Binet points out, is to confound intelligence with the ability to memorize. "Memory," says Binet, is a "great simulator of intelligence." It is a wise teacher who is not deceived by it. Only a small minority mentioned resourcefulness in play, capacity to adjust to practical situations, or any other out-of-school criteria.

Some suggested asking the pupil such questions as the following:--

"Why do you love your parents?" "If it takes three persons seven hours to do a piece of work, would it take seven persons any longer?" "Which would you rather have, a fourth of a pie, or a half of a half?" "Which is heavier, a pound of feathers or a pound of lead?" "If you had twenty cents what would you do with it?"

A great many based their judgment mainly on the general appearance of the face and eyes. An "active" or "passive" expression of the eyes was looked upon as especially significant. One teacher thought that a mere "glance of the eye" was sufficient to display the grade of intelligence. If the eyes are penetrating, reflective, or show curiosity, the child must be intelligent; if they are heavy and expressionless, he must be dull. The mobility of countenance came in for frequent mention, also the shape of the head.

No one will deny that intelligence displays itself to a greater or less extent in the features; but how, asks Binet, are we going to _standardize_ a "glance of the eye" or an "expression of curiosity" so that it will serve as an exact measure of intelligence?

The fact is, the more one sees of feeble-minded children, the less reliance one comes to place upon facial expression as a sign of intelligence. Some children who are only slightly backward have the general appearance of low-grade imbeciles. On the other hand, not a few who are distinctly feeble-minded are pretty and attractive. With many such children a ready smile takes the place of comprehension. If the smile is rather sweet and sympathetic, as is often the case, the observer is almost sure to be deceived.

As regards the shape of the head, peculiar conformation of the ears, and other "stigmata," science long ago demonstrated that these are ordinarily of little or no significance.

In reply to the second question, some teachers stated that they never made a mistake, while others admitted failure in one case out of three. Still others said, "Once in ten years," "once in twenty years," "once in a thousand times," etc.

As Binet remarks, the answers to this question are not very enlightening. In the first place, the teacher as a rule loses sight of the pupil when he has passed from her care, and seldom has opportunity of finding out whether his later success belies her judgment or confirms it. Errors go undiscovered for the simple reason that there is no opportunity to check them up. In the second place, her estimate is so rough that an error must be very great in order to have any meaning. If I say that a man is six feet and two inches tall, it is easy enough to apply a measuring stick and prove the correctness or incorrectness of my assertion. But if I say simply that the man is "rather tall," or "very tall," the error must be very extreme before we can expose it, particularly since the estimate can itself be checked up only by observation and not by controlled experiment.

The teachers' answers seem to justify three conclusions:--

1. Teachers do not have a very definite idea of what constitutes intelligence. They tend to confuse it variously with capacity for memorizing, facility in reading, ability to master arithmetic, etc. On the whole, their standard is too academic. They fail to appreciate the one-sidedness of the school's demands upon intelligence.

In a quaintly humorous passage discussing this tendency, Binet characterizes the child in a class as _dénaturé_, a French word which we may translate (though rather too literally) as "denatured." Too often this "denatured" child of the classroom is the only child the teacher knows.

2. In judging intelligence teachers are too easily deceived by a sprightly attitude, a sympathetic expression, a glance of the eye, or a chance "bump" on the head.

3. Although a few teachers seem to realize the many possibilities of error, the majority show rather undue confidence in the accuracy of their judgment.

BINET'S EXPERIMENT ON HOW TEACHERS TEST INTELLIGENCE.[9] Finally, Binet had three teachers come to his laboratory to judge the intelligence of children whom they had never seen before. Each spent an afternoon in the laboratory and examined five pupils. In each case the teacher was left free to arrive at a conclusion in her own way. Binet, who remained in the room and took notes, recounts with playful humor how the teachers were unavoidably compelled to resort to the much-abused test method, although their attempts at using it were sometimes, from the psychologist's point of view, amusingly clumsy.

[9] See p. 182 _ff._ of reference 2 at end of this book.

One teacher, for example, questioned the children about some canals and sluices which were in the vicinity, asking what their purpose was and how they worked. Another showed the children some pretty pictures, which she had brought with her for the purpose, and asked questions about them. Showing the picture of a garret, she asked how a garret differs from an ordinary room. One teacher asked whether in building a factory it was best to have the walls thick or thin. As King Edward had just died, another teacher questioned the children about the details of this event, in order to find out whether they were in the habit of reading the newspapers, or understood the things they heard others read. Other questions related to the names of the streets in the neighborhood, the road one should take to reach a certain point in the vicinity, etc. Binet notes that many of the questions were special, and were only applicable with the children of this particular school.

The method of proposing the questions and judging the responses was also at fault. The teachers did not adhere consistently to any definite formula in giving a particular test to the different children. Instead, the questions were materially altered from time to time. One teacher scored the identical response differently for two children, giving one child more credit than the other because she had already judged his intelligence to be superior. In several cases the examination was needlessly delayed in order to instruct the child in what he did not know.

The examination ended, quite properly for a teacher's examination, with questions about history, literature, the metric system, etc., and with the recitation of a fable.

A comparison of the results showed hardly any agreement among the estimates of the three teachers. When questioned about the standard that had been taken in arriving at their conclusions, one teacher said she had taken the answers of the first pupil as a point of departure, and that she had judged the other pupils by this one. Another judged all the children by a child of her acquaintance whom she knew to be intelligent. This was, of course, an unsafe method, because no one could say how the child taken as an ideal would have responded to the tests used with the five children.

In summarizing the result of his little experiment, Binet points out that the teachers employed, as if by instinct, the very method which he himself recommends. In using it, however, they made numerous errors. Their questions were often needlessly long. Several were "dilemma questions," that is, answerable by _yes_ or _no_. In such cases chance alone will cause fifty per cent of the answers to be correct. Some of the questions were merely tests of school knowledge. Others were entirely special, usable only with the children of this particular school on this particular day. Not all of the questions were put in the same terms, and a given response did not always receive the same score. When the children responded incorrectly or incompletely, they were often given help, but not always to the same extent. In other words, says Binet, it was evident that "the teachers employed very awkwardly a very excellent method."

The above remark is as pertinent as it is expressive. As the statement implies, the test method is but a refinement and standardization of the common-sense approach. Binet remarks that most people who inquire into his method of measuring intelligence do so expecting to find something very surprising and mysterious; and on seeing how much it resembles the methods which common sense employs in ordinary life, they heave a sigh of disappointment and say, "Is that all?" Binet reminds us that the difference between the scientific and unscientific way of doing a thing is not necessarily a difference in the _nature_ of the method; it is often merely a difference in _exactness_. Science does the thing better, because it does it more accurately.

It was of course not the purpose of Binet to cast a slur upon the good sense and judgment of teachers. The teachers who took part in the little experiment described above were Binet's personal friends. The errors he points out in his entertaining and good-humored account of the experiment are inherent in the situation. They are the kind of errors which any person, however discriminating and observant, is likely to make in estimating the intelligence of a subject without the use of standardized tests.

It is the writer's experience that the teacher's estimate of a child's intelligence is much more reliable than that of the average parent; more accurate even than that of the physician who has not had psychological training.

Indeed, it is an exceptional school physician who is able to give any very valuable assistance to teachers in the classification of mentally exceptional children for special pedagogical treatment.

This is only to be expected, for the physician has ordinarily had much less instruction in psychology than the teacher, and of course infinitely less experience in judging the mental performances of children. Even if graduated from a first-rank medical school, the instruction he has received in the important subject of mental deficiency has probably been less adequate than that given to the students of a standard normal school. As a rule, the doctor has no equipment or special fitness which gives him any advantage over the teacher in acquiring facility in the use of intelligence tests.

As for parents, it would of course be unreasonable to expect from them a very accurate judgment regarding the mental peculiarities of their children. The difficulty is not simply that which comes from lack of special training. The presence of parental affection renders impartial judgment impossible. Still more serious are the effects of habituation to the child's mental traits. As a result of such habituation the most intelligent parent tends to develop an unfortunate blindness to all sorts of abnormalities which exist in his own children.

The only way of escape from the fallacies we have mentioned lies in the use of some kind of refined psychological procedure. Binet testing is destined to become universally known and practiced in schools, prisons, reformatories, charity stations, orphan asylums, and even ordinary homes, for the same reason that Babcock testing has become universal in dairying. Each is indispensable to its purpose.