The practices of the professional Board emerged as more of a problem than those of the psychologist they investigated.
Dr. Batoff was reprimanded and fined by the Board, which concluded that Batoff's training and education did not qualify him to provide certain psychological services. In its order, the Board noted that psychological evaluations prepared by Batoff were "clearly substandard" and showed that he failed to recognize the limits of his competence. He allegedly prepared a report on one patient based on a Minnesota Multiphasic Personality Inventory in which 158 of the 588 questions were left blank. In another instance, the Board said, Batoff had administered the California Achievement Test, a test given to elementary school children to assess academic performance, to a four-year-old.
Holding: The court reversed, noting that an expert witness for Batoff had testified that his reports "were within the standard of competency despite their quality." Moreover, the Board did not credit either of the two expert witnesses who spoke on Batoff's behalf before a hearing examiner. Simply put, "the Board does not explain why the five untainted Board members who decided this case were independently qualified to substitute their opinions for those given by the expert witnesses."
Furthermore, the Board's allegations were based on incorrect information. The patient who took the MMPI "indeed completed the test, although not in one sitting." After answering the bulk of the questions, the patient finished the test one or two days later. The incomplete answer sheet was never used.
In the case of the four-year-old, Batoff actually administered the Children's Apperception Test. A typist transcribing Batoff's treatment notes was responsible for the mistake.
The court noted, 'The facts that purportedly underlie portions of the Board's decision are so inadequate and forcefully contradicted by uncontroverted facts of record that the administrative findings become mere conjecture."
 | James Seward, Ph.D. Forensic Psychologist The Forensic Panel |
Dr. Seward comments: Abuses of psychological testing can occur in all three phases of the evaluation process: the choice of tests, test administration, and test scoring and interpretation.
The choice of tests presents four potential problems: (i) the use of outdated versions; (ii) projective tests that are not validated; (iii) tests that are not adjusted for demographics; and (iv) using tests on populations for which they are not normed.
Abuses in test administration include deviating from the specific directions for administration (e.g., instructions to the patient, order of test administration, qualifications of the examiner), or not using the script that is meant to be read to the patient verbatim.
In my experience, it is common for psychologists to deviate from standardized procedures by administering only portions of tests, using photocopied forms (in violation of copyright), or modifying instructions. When challenged, these psychologists will often defend their practice on the basis of "clinical judgment." However, there is a wealth of literature documenting the superiority of actuarial versus clinical decision-making, and this advantage is lost when the standardized procedures are not followed.
During the development of a test, it is administered to the normative sample following specific instructions. Thus, any deviation from the standardized procedures may make the published norms inapplicable and invalidate the testing.
Test manuals provide information on the reading level necessary for completing measures that require reading and responding to statements. I was recently involved in a competency hearing in which the psychologist hired by the defendant utilized the Minnesota Multiphasic - Inventory-2 (MMPI2) in his evaluation. This recent revision of the MMPI requires an eighth-grade reading level. The psychologist failed to assess reading level and instead attributed the defendant's chaotic performance on the MMPI-2 to acquired "brain damage," when in fact the defendant had a history of poor educational performance (having failed several grades) and had left school in the seventh grade. Of course, the psychologist should assure that the patient has adequate sensory (e.g., visual acuity) and motor abilities to complete the test.
There are two ways to guard against this abuse in litigations: examining the raw data, and observing the evaluation itself. The American Psychological Association's Ethics Code prohibits releasing raw test data to persons who are "not qualified," but sending the data to another psychologist is acceptable. When requesting this material, attorneys should also request interview notes and any other documents relied upon by the psychologist to reach his or her decision.
As for test scoring and interpretation, people have a tendency to find what they are looking for. This is referred to as confirmatory bias. In narrative test reports, this is expressed as a tendency to gloss over (or even ignore) test findings that do not support the psychologist's agenda, while emphasizing those that do. A thorough examination of the raw data is a means of detecting potential bias.
Psychologists often ignore basic statistical and/or psychometric principles. One example is regression to the mean. Very high scores will tend to become lower on repeat administration of a test, and very low scores will tend to become higher. Therefore, high and low scores do not necessarily indicate areas of cognitive strengths and weaknesses. Likewise, the base rate of the condition being investigated should be considered. The rarer a condition or occurrence, the more difficult it is to detect; most psychological tests do not have sufficient sensitivity or specificity to detect rare conditions.
Psychologists often do not consider alternative explanations for- test findings, such as level of motivation and malingering. This last condition 'is of immense importance in forensic evaluations. All psychological tests can be faked; the issue is whether or not this faking can be detected.
Many psychologists depend primarily or solely on test results when formulating their conclusions. I believe that this is a mistake, and that the test results should be integrated with data from a variety of sources, including the patient's records, information from significant others, clinical interview, and observations of the patient during not only the testing itself but at breaks and other down time.
Errors in Choice of Tests
- Use of outdated versions
- Projective tests not validated
- Not adjusted for demographics
- Not adjusted for language fluency
