1* Ogechi MaryJane, Nwosu
M.Ed., (In V.), Dept. of Voc. Tech. Edu., Faculty of Education, Niger Delta University
*2 Michael O'Jules
M.Ed., (In V.), Dept. of Voc. Tech. Edu., Faculty of Education, Niger Delta University
2* Department of Vocational and Technology Education,
Faculty of Education,
Niger Delta University,
Nature of programme evaluation: Summary of the evaluation process
Evaluation is a very key element in educational processes. Without evaluation, it is difficult for educators to know the extent of educational objectives that has been attained. Therefore, this paper on “designing and developing evaluation system. Nature of programme evaluation: Summary of the evaluation process” aim to elucidate the nature and nitty-gritty of the evaluation process.
Keywords: Evaluation, Scope, design, education.
Download in Doc here: Michael and Maryjane – Evaluation Process
During the period of designing educational structures, curriculum, determining managerial hierarchical order and even within the process of teaching and learning, the designers are fully aware that along the line, there may be need to come back to check either as the process is still ongoing (formative) or at the end of the programme (summative) how far the previous arrangement has met the targeted objectives. This is because complexities and numerous intervening variables which the designers did not initially put into consideration may influence the programme either positively or negatively. Thus, evaluation has become every part of educational endeavour to streamline or to keep check of predetermined targeted objective.
Evaluation in this context according to UNESCO (2016) is “the systematic and objective assessment of an ongoing or completed project, programme or policy, its design, implementation and results, with the aim to determine the relevance and fulfillment of objectives, development efficiency, effectiveness, impact and sustainability”. The above definition mentioning ongoing is indicative of the fact that evaluation can be carried out intermediately; this is known as formative evaluation. Additionally, this definition also implies that evaluation can also be carried out as at when the programme is completed; this refers to summative evaluation. In any case however, both types of evaluation are important if educational programmes are to succeed.
In this paper however, the authors looked into two specific areas of evaluation – the nature of programme evaluation and summary of the evaluation process. Nonetheless, the next subheading will look into the concept of evaluation before the two areas were delved into.
Concept of educational evaluation
There are various concepts in educational evaluation that guides the understanding of what evaluation truly is. For instance, Educational evaluation is broader in scope and more objective than educational measurement. It is also the process of carefully appraising a programme/individual to form a variety of information giving device. Besides testing and other tools of measurement, evaluation seeks additional evidences from various sources of information supplementing each other: like interviews, questionnaires, anecdotal records, cumulative records, case conferences, mechanical or electronic recorders, case studies or projective techniques, etc. and the selection, through careful analysis of data, most pertinent to a wise just and comprehensive interpretation to make value judgment of the programme under evaluation.
In addition, evaluation as mentioned above can also be carried out intermediately of after the completion of the programme, depending on the discretion of the evaluator, the environment and the nature of the programme under evaluation. All these are done with the sole intention of checking how far the programme have achieved its aim.
Educational evaluation is a systematic process which means it omits casual, uncontrolled, unsystematic ways and means of judging programme’s progress. Secondly, educational evaluation strives to judge programme’s progress towards objectives in the curriculum. It necessitates that objectives in the curriculum i.e. educational objectives should be previously determined. This is because it is not possible to judge the growth and progress of programmes towards objectives or values without previously determining the educational objectives.
Educational evaluation is a comprehensive process in the sense that it has a bearing on all the stages of the process of education right from beginning with the determination of objectives through planning of the curriculum/teaching and learning process till testing. It is comprehensive again in the sense that it takes into consideration all the objectives of the programme and not only achievement. Educational evaluation is a continuous process. It is because programmes’ growth and development occur throughout the year, therefore the single act of testing growth and development at the end of an academic year is not enough to know the true and complete account of the changes taking place in programme. It is, therefore, necessary to judge their progress from time to time throughout the year. It is continuous also in the sense that it does not end with testing. It strives to interpret the results obtained and in the light of interpretation either continues or modifies educational objectives, curriculum, methods of instruction or tools and techniques of evaluation themselves and the process goes on.
Definition of educational evaluation
It is virtually impossible to restrict evaluation into one universally accepted definition, however, the following are widely accepted definitions within the educational domain.
For Stake (1967), evaluation is implicitly “the description and judgment of an educational programme which encompasses the role of an evaluator”. Unfortunately, the authors found this definition incomplete since it fails to elucidate the roles of the evaluator. At best, this definition only brought more complexities into the already complex environment as it leaves the reader with the impression of going further to look for what an evaluator is. Contrastingly, Stufflebeam (1971) and Alkin (1969) were more meticulous in their definition when they define evaluation as the “process of alienating, obtaining and providing useful information for judging useful information so as to determine alternative course of action”. This definition reveals that during evaluation, information about a programme is collected first, then studied, before judgments can be passed.
Scriven (2009) saw evaluation from a tripartite view: (1) “evaluation is the process of determining merit, worth, or significance (abbreviated as m/w/s). In many but not all contexts these three properties are roughly the same as, respectively: quality, value, and importance. (2) “an evaluation is the product of that process”. (3) “an evaluand is whatever is evaluated. (The term evaluee is often used for human evaluands.) This third triparte however refers to evaluation as having a subject and a predicate.
Educational evaluation is a systematic process of determining the growth and progress of programes towards objectives or values in the curriculum. In other words, it is a systematic process of determining the extent to which educational objectives are achieved by pupils.
Nevertheless, the writers see evaluation as a process that includes measurement and possibly testing but it also contains the notion of value judgment. If a teacher administers a test to a class and computes the percentage of correct responses, it is said that measurement and testing has taken place. The scores must be interpreted which may mean converting them to values like As Bs Cs and so on or judging them to be excellent, good, fair or poor. This process is called evaluation. So we can say, evaluation is concerned with making judgments about things. When we act as evaluators, we attribute ‘value’ or ‘worth to behavior, objects, processes and programmes.
Nature of programme evaluation
Evaluation is a continuous process of inquiry concerned with the study, appraisal, and improvement of all aspects of the education program. It is a process carried out cooperatively by all concerned with the growth and development of the programme. It is the process of determining the changes in the programme during the course of implementation and the intervening variables that has influenced it during the period. It is a systematic attempt at ascertaining the amount of progress made in the implementation and the extent of realization of predetermined objectives. It is an act of judging the programme’s improvement in terms of teaching and learning quality, teacher quality, students’ achievements, administrative competence, and numerous other dimensions. On the basis of the information about the growth and development of the programme, suitable evaluative procedures and instruments can be prepared by all concerned to find out the effectiveness of the whole educational program in terms of meeting the needs of the individual learners, teachers, stakeholders, government and the community as a whole. Evaluation can be formative or summative (Joseph, 2019).
A formative evaluation (sometimes referred to as internal) is a method for judging the worth of a program while the program activities are forming (in progress). Here, the focus of the evaluation is on the process. They can be conducted during any phase of the programme process. This part of the evaluation focuses on the process. Thus, formative evaluations are basically done on the go. They permit the designers, learners, instructors, and managers to monitor how well the instructional goals and objectives are being met. Its main purpose is to identify deficiencies as soon as possible so that proper learning interventions can take place that allows the learners to master the required skills and knowledge.
Formative evaluation is also useful in analyzing learning materials, student learning and achievements, and teacher effectiveness . . . Formative evaluation is primarily a building process which accumulates a series of components of new materials, skills, and problems into an ultimate meaningful whole (Guyot, 1978). In addition, prototyping is used in formative evaluations to test a particular design aspect by using one or more iterations.
A summative evaluation (sometimes referred to as external) is a method of judging the worth of a program at the end of the program activities (summation). The focus is on the outcome. According to Scriven (1967), “all assessments can be summative (i.e., have the potential to serve a summative function), but only some have the additional capability of serving formative functions”. The various instruments used to collect the data are questionnaires, surveys, interviews, observations, and testing. The model or methodology used to gather the data should be a specified step-by-step procedure. It should be carefully designed and executed to ensure the data is accurate and valid. Questionnaires are the least expensive procedure for external evaluations and can be used to collect large samples of information. The questionnaires should be tested before using to ensure the recipients understand their operation the way the designer intended. When designing questionnaires, keep in mind the most important feature is the guidance given for its completion. All instructions should be clearly stated . . . let nothing be taken for granted.
General principles of evaluation
The greatest benefit can be secured from an evaluation program by following certain general principles played out by Clark (2012):
- Evaluation should be in terms of the extent to which the programme has attained the objectives of education. These objectives include not only mastery of subject matter but also the growth in physical and mental health, ability to get along with others, use of critical thinking to solve problems, efficiency in using skills, competency in the creative arts, wide interest in many fields of human endeavor, ability to use knowledge gain.
- Objectives should be defined in terms of subject matter behavior. This refers to a behavioral analysis of what students should be able to do after successfully completing units of instructions. Behavioral analysis is the breaking down of higher level objectives into component parts, each of which must be mastered to eventually lead to a final behavior expected of the student.
- Evaluation is an integral part of the educative process. The teacher should guide every experience in terms of the needs and interests of the group as seen in life situations. Evaluation forms the basis for decisions as to the nature of the next experience needed by the learner. The day-to-day appraisal helps the teacher in deciding how to guide on-going experiences and when to introduce new ones.
- The evaluation program should be cooperative. Learners, teachers, and parents should participate in the process of evaluating educational objectives.
- Records should give a complete picture of each child. Growth as a continuous process can be seen only when adequate records give a complete picture of the child as he develops under the care of the teacher. Records, however, should not be considered as ends but as service tools to help the teacher understands the learners to interpret behavior, and to define immediate and long-term needs.
- Evaluation should be comprehensive. It should take into account the learner’s individual character, his background, and the immediate environmental factors. Data should include the individual’s health physiologic needs, emotional adjustments, mental characteristics, talents and aptitudes, values and attitudes, social relationship and competence, ability to function effectively in his environment and in the whole realm of his interest, aspirations, and goals. Records should show the learner as a developing personality, including both positive and negative aspects of the learner’s development.
- Evaluation uses a variety of instrument, tools, and techniques. These instruments should be valid, reliable, and practical from the standard points of time, effort, and facilities of the school. There are teacher made and standardize tests, anecdotal records, rating scales, samples of students’ work, sociograms, diaries, and journals. The teacher should choose the technique suited to the individual student concerned and to the specific purpose for which the evaluation is being made.
- Objective measurement and subjective judgment are both essential in evaluation. Records should be specific and as, far as possible objective. Subjective estimate are made objective by the inclusion of specific incidents and illustrations.
- Diagnosis and remedial work are phase of the evaluative process. Test results should be used for the improvement of instructions. Results should be carefully interpreted and necessary follow-up work should be done accordingly.
- Evaluation should be descriptive. Although the uses of terms like superior, good, average, and poor is better than the use of figures, these terms still leave much to be desired from the standpoint of evaluation. A descriptive concrete statement about the child is more meaningful and significant to teachers, to parents, and to children than any blanket judgment that merely indicates that the child has passed.
Kirkpatrick’s Four Level Evaluation Model (1959)
Perhaps the best known evaluation methodology for judging learning processes is Donald Kirkpatrick’s Four Level Evaluation Model that was first published in a series of articles in 1959 in the Journal of American Society of Training Directors (now known as T+D Magazine). The series was later compiled and published as an article, Techniques for Evaluating Training Programs, in a book Kirkpatrick edited, Evaluating Training Programs (1975).
However it was not until his 1994 book was published, Evaluating Training Programs, that the four levels became popular. Nowadays, his four levels remain a cornerstone in the learning environment. While most people refer to the four criteria for evaluating learning processes as “levels,” Kirkpatrick never used that term, he normally called them “steps” (Craig, 1996). In addition, he did not call it a model, but used words such as, “techniques for conducting the evaluation” (Craig, 1996).
The four steps of evaluation consist of:
- Step 1: Reaction – How well did the learners like the learning process?
- Step 2: Learning – What did they learn? (the extent to which the learners gain knowledge and skills)
- Step 3: Behavior – (What changes in job performance resulted from the learning process? (capability to perform the newly learned skills while on the job)
- Step 4: Results – What are the tangible results of the learning process in terms of reduced cost, improved quality, increased production, efficiency, etc.?
Kirkpatrick’s concept is quite important as it makes an excellent planning, evaluating, and troubling-shooting tool, especially if we make some slight improvements.
From the above, it can be said that evaluation is an important aspect of the educational system. Without evaluation, growth can be impossible. This is because it is the level of growth that determines the next course of action. Therefore, where there is no assessment/evaluation, deciding the next level can be difficult. Accordingly, this paper has made us to know that evaluation can be carried out either as the programme is ongoing (which is called formative evaluation in this paper – process focus) or as the programmes concludes (which is summative-outcome base). It is however worthy of note that while the summative has been widely adjudged to be very important, the formative evaluation is also very key to evaluation. This is because assessment done at the end of a programme may omit several points that are invaluable to the overall growth of the educational system being evaluated.
Alkin, M. C. (1969). Evaluation theory development. Evaluation Comment, 2, 2–7
Clark, D.R. (2012). Types of evaluations in instructional design. Retrieved from http://www.nwlink.com/~donclark/hrd/isd/types_of_evaluations.html
Craig, R. L. (1996). The ASTD training: Development handbook. New York: McGraw-Hill.
Guyot, W. M. (1978). Summative and formative evaluation. The Journal of Business Education. 54(3),127-129.
Joseph, E. (2019). Meaning and nature of evaluation. Retrieved from https://www.scribd.com/doc/26092878/Meaning-and-Nature-of-Evaluation-Evaluation-Is
Kirkpatrick, D. L. (1959). Techniques for evaluating training programs. Journal of American Society of Training Directors. 13(3), 21–26.
Scriven, M. (2009). The nature of evaluation. Retrieved from http://www.rismes.it/pdf/Scriven_domain_evaluation.pdf.
Scriven, M. S. (1967). The methodology of evaluation. In R. E. Stake (ed.) Curriculum evaluation. AERA Monograph Series on Curriculum Evaluation (Vol. 1). Chicago: Rand McNally
Stake, R. E. (1967). The countenance (sic) of educational evaluation. Illinois: Center for Instructional Research and Curriculum Evaluation University of Illinois. Retrieved February 26th, 2019, from https://pdfs.semanticscholar.org/b07e/5b61cde550bfb0b64e895674a236c9003335.pdf
Stuffiebeam, D. L. (1971). The relevance of the CIPP evaluation model for educational accountability. Journal of Research and Development in Education, 117-214.
UNESCO, (2016). Designing effective monitoring and evaluation of education systems for 2030: A global synthesiss of policies and practices. UNESCO Education Sector Division for Policies and Lifelong Learning Systems (ED/PLS)/Section of Education Policy (ED/PLS/EDP). Retrieved from http://www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/ED/pdf/me-report.pdf