Evaluation Revalued

Evaluation Revalued

Robert E. Stewart, D.D.S. *

In any sound program of research and development, whatever the intended goal, there must inevitably come a time when extensive evaluation of the product is indicated. Less than fifty years ago, systematic tests of new concepts were performed more or less routinely by private inventors dedicated to proper self-appraisal as occasion warranted. In a period less sophisticated technologically, this fashion in science served its purpose adequately and well. But with the growth in a more modern era of the large and vastly more complicated system of scientific inquiry, such as we know it today in government and industry alike, the requirement for periodic assessment of experimental results has led to the development of the independent testing laboratory, either as a part of the basic organization or as a separate contracting institution. So indispensable has this phase of technical investigation become that now large sums of money are spent annually in support of evaluation groups who themselves commonly engage at least in part in research aimed at improving their own methods and techniques.

With respect to these matters, the Artificial Limb Program has exhibited ostensibly no basic deviation from the general pattern now characteristic of other broad exploratory projects involving the cooperation of various specialists in otherwise distinct disciplines. But because of the peculiar nature of the amputee problem, the particular state of the art of limb prosthetics, especially in the upper extremity, and the demands of rather unusual external influences of one kind or another, the approach to systematic evaluation has in this case evolved out of a unique history and has, consequently, given rise to some valuable results in research and education of which the influence was not fully anticipated in the beginning.

Although in that portion of ALP devoted to the upper extremity much of the initial investigation was directed toward all-purpose, or "ideal," prostheses for selected levels of arm amputation, it was soon recognized that the desired objectives would be served more effectively were a variety of components made available for assembly into various combinations the better to provide for the particular needs of the individual patient. As these components were developed, prototypes and, later, production units were subjected to systematic testing by the Prosthetic Devices Study, an organization established for this specific purpose within the Research Division of the College of Engineering of New York University.

At this point, evaluation generally furnished much needed data concerning the usefulness and reliability of individual units in direct comparison with previous similar parts but without regard for the influence of socket fit, type of harness and harness adjustment, type and extent of training, individual amputee preference, and other factors. Because methods suitable for the evaluation of techniques had yet to be introduced, early evaluations of components brought with them the subtle dangers of misinterpretation owing to the indirect effects of pre-existing errors in socket or harness, to say nothing of the possibility of the influence of one component upon the performance of another used in conjunction. In these circumstances, a great deal was left to be desired in reference to the over-all problem of upper-extremity prosthetics.

To fill the gap, there was initiated in 1950, in the Department of Engineering at the University of California at Los Angeles, the so-called "Case Study," with the purpose of bringing together all available information, of viewing systematically the results obtained by use of various combinations of devices and techniques, and thus of developing a set of general principles of management for the upper-extremity amputee. As the Case Study progressed, there arose an increasing awareness of the necessity for teamwork in the proper application of such knowledge as there was, and by 1952 the Prosthetic Devices Study was called upon to conduct an evaluation of the results of the UCLA Case Study.

It was obvious that, if such an evaluation were to be conclusive, large numbers of cases under varying geographical conditions would be needed for observation and that therefore the services of a number of clinic teams throughout the country would be required. Although the Prosthetic and Sensory Aids Service of the Veterans Administration, long the chief sponsor of the Artificial Limb Program, had already established some thirty prosthetic clinic teams, and although these groups were readily available for participation, it was patently mandatory that they be trained in the latest methods before any reliable program of evaluation could be initiated. Accordingly, short-term courses for clinic team members physicians, therapists, and prosthetists were organized and conducted at UCLA beginning in 1953. The formation of new clinic teams outside the VA framework was encouraged, and these, along with a few private clinic teams already in existence, were invited to participate.

The education program leading to the Upper-Extremity Field Studies, the name applied to this part of the NYU evaluation work, proved to be a pioneering effort in its own right. While results of research were being made available to clinic teams for general use in a remarkably short time after the initiation of laboratory work, the continued association of clinic personnel with the research program through participation in the Field Studies had a definite impact on those responsible for amputee care. Thus the Field Studies came to be a series of complex investigations designed not only to evaluate the usefulness of available upper-extremity prostheses but also to determine the effectiveness of the management procedures elucidated by the UCLA Case Study. Simultaneously, and almost unavoidably, the process of accumulating voluminous clinical data on one segment of the population led to a general upgrading of industry practices in amputee service and furnished the basis for further research into the needs, physical and mental, of the armless.

Because the NYU Field Studies represent the first, and thus far the only, attempt in the United States to appraise the status of upper-extremity prosthetics directly and on such a broad scale, and because the results present such a wealth of information not available elsewhere, this and the following issue of Artificial Limbs are given over to presentation of a series of summary articles divided into two parts the first (this number) concerned with the educative aspects of the work, the second (Autumn 1958, Vol. 5, No. 2) with the research implications. For those who would undertake further study and interpretation in the interest of scholarship, the original data, far too detailed for thorough analysis by other than biostatisticians, are available in the College of Engineering of New York University, New York City.

In reviewing the material offered here, it is appropriate to keep in mind that the Field Studies constituted a new voyage into an area in which both subject matter and method of approach were uncharted and unexplored. Understandably beset by all the problems of design, organization, and execution typical of adventures into the unknown, they now reveal certain deficiencies most readily viewed with benefit of hindsight. In all probability, the true value of the Field Studies remains to be had in the further application of the principles not only in the field of limb prosthetics but in other, more general areas of physical handicap as well.