Chapter 10           Outcomes Assessment/Institutional Effectiveness

overview

Outcomes assessment and institutional effectiveness are integral components of Howard’s commitment to provide a quality education to its students.  The Middle States Association of Colleges and Schools visiting team reported in 1989 that the measurement of outcomes at the University was not well developed.  It was noted that although outcomes assessment is a general practice at the University, there is limited systematic, University-wide assessment activity, especially in areas that cut across all academic and non-academic units.

In 1990, a “Howard Commission” was formed to recommend future courses of action for program effectiveness at the University.  The Commission report recommended that “the University... establish outcome measures for students, faculty and programs, and it should use the results of assessment to improve student learning and program effectiveness, and to justify and support continued accreditation.”  This recommendation was consistent with the Standards for Accreditation of the Middle States Association of Colleges and Schools, which states: “The deciding factor in assessing the effectiveness of any institution is evidence of the extent to which it achieves the goals it has set for itself.”

In Howard’s 1996 report[1] to the Middle States Association of Colleges and Schools, the president reiterated the institution’s commitment to a centrally coordinated campus-wide assessment initiative.  In that report the University indicated that the appointment of a provost as the University’s chief academic officer would mark the start of the initiatives to provide the organizational structure needed to develop a truly comprehensive, university-wide outcomes assessment program. When fully operational, quantitative and qualitative data about outcomes and institutional effectiveness will be systematically collected, analyzed and used in institutional planning and improvement, and as indicators of progress towards achieving the mission, goals and objectives of the University.  In this chapter an overview of the current status of Howard’s assessment of its educational programs and services as related to its mission, goals and objectives will be examined. 

assessment of Learning and teaching

Assessment activities at Howard University have two specific, interrelated purposes: (1) to review, evaluate and improve student learning, achievement and performance; and (2) to continue to improve and modify academic programs, program planning and program development. Howard University assessment activities are intended to be an integral component of the University’s efforts to improve the quality of educational research and service to the internal and external communities. 

The University’s commitment to evaluate and improve institutional effectiveness is operative throughout teaching and learning activities and processes.  Assessment activities existing within the University have focused on assessing student learning and academic achievement, student success rates, retention, student employment opportunities and student academic satisfaction.  These activities are central to the University’s mission and core values.  

Howard University’s outcomes assessment activities are intended to provide information on institutional activities and their effectiveness.  Such activities have influenced the following actions: (1) assessing and revising the academic mission and goals of the colleges/schools, as well as the mission and goals of other University units; (2) designing/reorganizing academic programs and majors; (3) modifying student assessment plans, policies and practices; (4) developing a University-wide general education curriculum; (5) creating new and evaluating current student out-of-class learning experiences; and (6) redesigning/modifying instructional/teaching methods and practices; and (7) modifying student academic support services.  The primary goal of these activities is to build on what currently exists.  Information obtained from these efforts provides assistance to academic and non-academic units for improving their effectiveness and increasing commitment to the University’s mission and goals.

An important element of outcomes assessment at Howard is the appraisal of students’ academic experience in the classroom.  The assessment tools vary significantly depending upon the academic field and level of study.  The assessment instruments used most frequently are the standard ones, i.e., examinations, homework, quizzes, clinical demonstrations, portfolios, projects and reports.  The primary goal of these assessment activities is to determine the final course grade.  Many Howard professors are aware of the national paradigm shift from an emphasis on instructional delivery towards an emphasis on student learning.  They are placing more emphasis on using these classical instruments to provide immediate feedback to students to facilitate improved learning. In addition, the results are used to revise and improve course content.

Academic units at Howard University use many different assessment methods.  As with teaching and classroom assessment methods, they reflect differences in (1) school/program practice; (2) discipline; (3) faculty members; and (4) practical constraints, such as the number of student majors and typical class size.

Assessment of curricula at the department level is designed to improve the curricula and strengthen the academic program.  Table 10.1 provides a summary of the assessment tools used throughout the campus.  A description of the tools follows:

Standardized Tests: Tests developed by national scholarly associations are used and the results are compared to national norms.

            Departmental Tests: Comprehensive/preliminary examinations are developed by the department faculty to measure the student’s mastery of discipline-specific knowledge.

            Student Portfolios: Collections of students’ written, scientific or creative works are collected and collated throughout students’ matriculation in specific departments.  Faculty supervise the development and evaluation of portfolios.

            Alumni Surveys: Alumni are surveyed to determine their satisfaction with their educational experience and to solicit recommendations for curricula reform.

            Exit Interviews: Interviews are conducted with degree candidates by the department chair, student advisor or instructor to determine the quality of the


Table 10.1 Assessment Tools Used in Academic Programs

ASSESSMENT

TOOL

UNDERGRADUATE PROGRAMS

GRADUATE PROGRAMS

PROFESSIONAL PROGRAMS

Standardized Tests

Economics, chemical engineering, civil engineering, electrical engineering, mechanical engineering, physical education, systems and computer science, and division of Allied Health Sciences

Biology, mathematics, pharmacology and philosophy, physical education

Medicine

Departmental Tests

African studies, Afro-American studies, anthropology, biology, chemistry, classics, comprehensive science, economics, English, mathematics, modern languages, music, political science, human development, early childhood and elementary education, philosophy, physical education, physics, psychology, secondary education, sociology, theatre arts and division of Allied Health Sciences

Afro-American studies, African studies, anthropology, biology, chemical engineering, chemistry, classics, civil engineering, comprehensive science, economics, education, electrical engineering, English, graduate school, math, mechanical engineering, music, pharmacology, philosophy physical education, physics, psychology, sociology, systems and computer science, social work and theatre arts

 Medicine, divinity

Student Portfolios

Art, biology, English, physical education, theatre arts and division of Allied Health Sciences

Art, biology, education, English, pharmacology, philosophy, physical education, social work, and theatre arts

Medicine

Alumni Questionnaires

Art, biology, English, political science, division of Allied Health Sciences.

Art, biology, English, math, pharmacology and philosophy.

Medicine, divinity

Exit Interviews

Art, biology, political science, human development, early childhood and elementary education, secondary education, chemical engineering, chemistry, civil engineering, electrical engineering, mechanical engineering, philosophy, systems and computer science, theatre arts and division of Allied Health Sciences, communications.

Art, education, English, chemistry, math, pharmacology, philosophy, physical education, social work and theatre arts

Medicine, divinity

Capstone Experience

 Anthropology, art, biology, classics, economics, English, mathematics, music, political science, human development, early childhood and elementary education, secondary education, chemical engineering, civil engineering, electrical engineering, mechanical engineering, philosophy, psychology, sociology, systems and computer science, and division of Allied Health Sciences, communications.

 Anthropology, art, biology, chemical engineering, civil engineering, classics, economics, electrical engineering, mechanical engineering, systems and computer science, economics, education, English, math, mechanical engineering, music, pharmacology, philosophy and social work and sociology

Medicine


            student’s academic experiences.

            Capstone Experiences: Courses for senior-level students that are designed to measure mastery of essential theoretical and methodological issues associated with a given discipline. Capstone experiences are valuable because they can measure the systematization of departmental curricula, since students cannot perform well in these courses unless they have acquired the basic educational skills and comprehensive knowledge of the discipline reflected in the major sequence courses. 

One of the University’s leaders in implementing a systematic approach to outcomes is the Graduate School with its regular program review assessments.   Since 1976, the Graduate School has conducted program reviews utilizing methods recommended by the Council of Graduate Schools  (CGS). Graduate program reviews are conducted in each department every five to seven years.  Areas of assessment that the Graduate School examines include program effectiveness, research productivity, student retention, graduation rates, post-graduate career success, student services and infrastructure. 

The assessment plans currently being implemented by graduate programs across the country originated from a variety of changes in statewide mandates, regional accrediting organizations and legislative interests.  The Graduate School has recognized that many internal and external factors exist that impact directly on teaching, learning, research productivity and, ultimately, the success of Howard graduate students.  The Graduate School has committed itself to the development of an outcomes assessment program that can be used as a model for the University-wide community, and which will provide data and information that can be utilized effectively for the enhancement of academic programs.  This approach has evolved from careful review and continual modification since 1976 and incorporates the practices of several graduate schools at Research I universities.

The professional schools and a number of other programs use the pass rate on licensure and certification examination as assessment tools.  The results also are being used in the review and modification of curricula.  Examples are provided in Table 10.2.

Table 10.2 Licensure/Certification Examinations

PROGRAM

EXAMINATION

Clinical Laboratory Sciences

ASCP Certifying Examination

Dentistry

Northeast Regional Board

Education

Praxis Core Battery Test

Law

BAR Examination (Maryland, Virginia, DC)

Medicine

Medical Licensing Examination (USMLE)


Nursing

NCLE Examination

Nutritional Sciences

National Registration Examination for Dieticians

Occupational Therapy

Occupational Therapy Certification Examination

Pharmacy

NABL Examination

Physical Therapy

Physical Therapy State Licensing Examination

Physician Assistant

National Certifying Examination For Physician Assistant

Radiation Therapy

American Registry of Radiological Technologists

 

One of the University’s newest initiatives, The Leadership Academy — an outgrowth of the Framework — has incorporated outcomes assessment from its start.  Course evaluation data are systematically compiled, analyzed, summarized and formally incorporated in a course improvement process.

Assessment of non-academic services

Non-academic services provided to students by the University have an impact on student outcomes and learning as well.  These services are pivotal to students’ successful matriculation at the University.  Most of these services provide for the compilation, analysis and dissemination of data about students.  The primary goal for many of these offices is to function as a conduit for communication of outcomes assessment activities occurring within the units, and to provide timely and accurate information to the University community for decision making. Many of the units are moving toward electronic dissemination of information to students, faculty and staff.

The University provides non-academic student services in the following areas: (1) enrollment management, e.g., financial aid, records and articulation, and registration; (2) student affairs services, e.g., housing, counseling, career planning, special student services, student life and activities, dining services, international concerns, intramural sports, and recreational activities and intercollegiate athletics; (3) health services; (4) religious life; and (5) campus services.  These services are pivotal to students’ successful matriculation at the University.

Many of the non-academic service units utilize surveys, questionnaires and anecdotal information to measure their effectiveness in meeting the needs of students.  Continued improvement of services requires that all units develop outcomes assessment programs to obtain systematic feedback, particularly as the University implements changes in operation.

Outcomes Assessment Summary

Outcomes assessment activities are taking place at Howard in a variety of ways.  These activities, although not coordinated, produce a wealth of data that when analyzed and evaluated can be shared among units and used for the improvement of programs and services.  While increased attention to (1) the analysis of currently available data, (2) the sharing of best practices, and (3) the development of outcomes assessment activities where they currently do not exist is needed, the Office of the Provost is taking the lead to assure that continuous evaluation of student performance, curricula, retention and time-to-degree occurs.  As such, plans are being made to establish a permanent University-wide Office of Outcomes Assessment and Institutional Effectiveness to be administered by a senior level and seasoned assessment professional.  This office should report, the Task Force believes, directly to the President and coordinate its activities with the Office of the Provost and Chief Academic Officer — especially when the focus is on academic programs and curricula quality, faculty performance, student learning outcomes — as well as with the Office of Institutional Research.  This Office will provide technical assistance to academic and non-academic units and coordinate and monitor all student outcomes assessment activities at the University.

In addition, each school/college will establish an outcomes assessment and institutional effectiveness committee consisting of faculty, students, staff, administrators and alumni to plan and implement all assessment activities at the departmental and school levels.  One person from the committee will serve on a University-wide Advisory Committee, established to advise the Office of Outcomes Assessment and Institutional Effectiveness on all aspects of assessment activity at the University.  Similarly, the non-academic units should establish outcomes assessment committees as needed. 

When fully established, the Office of Outcomes Assessment and Institutional Effectiveness would gather, analyze and report information regarding the assessment policies, practices and results of outcomes of academic and non-academic units, as well as provide expert advice to these units, and organize and direct University-wide assessment initiatives.  It will also provide periodic recommendations to the president, provost, academic and service units, the Faculty Senate, Howard University Student Association, Howard University Support Staff Organization and the University community for University improvement and enhancement.

The surveys utilized in this self-study and the results from those surveys are indeed a type of University-wide outcomes assessment.  The current instruments incorporated many items from the surveys performed as part of the earlier 1989 Self-Study.  While the current instruments were developed for particular constituent audiences — students, faculty, staff, senior administrators — many items were consistent across instruments.  Further, data collection incorporated methodology that allows for the compilation of findings by various status characteristics of the respondents (e.g., school and department affiliation, gender, tenure status, etc.).  The surveys examined perceptions of program quality and unit-level performance, as well as satisfaction with the delivery of services across many areas of the University.  Because large, representative samples were used, this survey data constitutes a baseline for use in subsequent assessment initiatives.  Therefore, the Task Force recommends that the various University units adopt and modify the survey instruments, as appropriate, and use this methodology as part of the effort to compile, on a systematic basis, outcomes assessment data.

 



[1] “Report Pursuant to the Special Staff Visit of the Commission on Higher Education, Middle States Association of Colleges and Schools,” March 15, 1996.