Narrative:  

Troy University is in compliance with Comprehensive Standard 3.3.1 and employs a process of institutional effectiveness across all academic, support, service units, and research and community service areas. This narrative will illustrate how the University makes use of defined outcomes, the extent to which outcomes are achieved, and the process by which improvement based on analysis of the results occurs in the institutional process of academic programs, academic support units, service support units and community/public service units.

The merger of the separately accredited campuses in 2005 gave Troy University the opportunity to consolidate the best practices in institutional effectiveness from each campus into a new holistic model. This model builds upon twenty years of an evolutionary practice of institutional effectiveness. Given the variety of programs and locations in which Troy University provides service, it was important to build the institutional effectiveness model around best practices that would enable the institution to evaluate its performance in many areas. The overall process for institutional effectiveness is defined in detail, and updated each year, in the Institutional Planning and Effectiveness Handbook (p. 11).

Troy University developed the Performance Effectiveness Dashboard System (usually referred to Dashboard) as an online system that tracks strategic planning, annual planning, and data related to institutional effectiveness. The section of Dashboard that deals with institutional effectiveness is called the Performance Effectiveness Reports, or PERs. The overall planning process for Troy University, and the use of Dashboard in that process, is described in Core Requirement 2.5.

The Program Effectiveness Reports in Dashboard contains an area for institutional effectiveness information for each academic program and all major support and administrative units in the university. Each PER includes a description of the purpose of the program, its relationship to the University’s mission, a set of expected outcomes, and actual assessment data. The expected outcomes are determined based upon a review of the assessment results from previous years, requirements of specialized accrediting agencies, expectations of affiliated agencies or partners, outcomes of benchmarked peers, the Troy University mission statement, and the Troy University strategic objectives.

Student learning outcomes representing the expected outcomes in academic programs are expressed in Dashboard. Where possible, student learning outcomes are based on performance, on licensing examinations or student performance on major examinations that provide national comparative data, such as in nursing, education, human resource management and risk management. Major field tests form the basis for student learning outcomes in biology, criminal justice, history, literature, political science, and psychology. The academic programs that do not have these types of comparative performance data will generally base their expected student learning outcomes on student performance in capstone courses. Faculty members are then able to consider the holistic performance of the student within the context of the entire program, consistent with the SACS perspective that assessment must occur at the program level. Dashboard does not attempt to track or provide assessment data that individual faculty may use to drive the continuous improvement of specific courses, with some notable exceptions such as COM 2241.

Expected outcomes in support programs and administrative areas include two types of expected outcomes. Some are related to how well the unit is meeting key objectives in the University’s strategic plan or in the unit’s annual plan and are goal oriented in nature. Most are related to data from annual surveys of students and alumni that are conducted by the Office of Institutional Research, Planning and Effectiveness and from other surveys that the support organization or administrative unit may administer on its own, providing a perspective as to how stakeholders or users of these processes perceive the quality of performance. Overall results from these surveys are provided in the SACS reference area, such as the most recent annual Graduating Student Survey, Alumni Survey, International Student Survey, and Employer Survey.

The academic dean or the head of a support or administrative organization is responsible for monitoring the effectiveness of his or her unit with respect to that unit’s function in the delivery of the University’s mission and ensuring that assessment data related to expected outcomes is provided in the Dashboard system.

Assessment results may identify an opportunity for improvement in an academic or administrative program. The responsible manager may elect to wait for an additional round of data to ensure that the results are not due to special causes of variation. In some cases, the dean or responsible manager may conclude that the expected outcomes were unrealistic and may make adjustments. In other cases, the responsible manager will prepare a Plan for Improvement which will become part of the organization’s annual plan.

Each year, each college prepares an Annual Achievement Report that summarizes its accomplishments in terms of both its annual plans and institutional effectiveness information. The online availability of assessment results across all units of the University has allowed institutional effectiveness and program effectiveness to be continuously monitored. Annual Achievement Reports are available for review.

To provide for ongoing improvement of the institutional effectiveness process, Troy University established an Institutional Effectiveness Committee that consists of faculty, at least one dean, and representatives from administrative organizations. The committee has established a schedule for the periodic review of the institutional effectiveness process for each academic and administrative program. Reports from the Institutional Effectiveness Committee for the review of the College of Education and for the Information Technology department and Human Resources are provided. The committee is also responsible for the review of expected outcomes and methods of assessment for any newly proposed academic program.

3.3.1.1 Educational Programs

Every academic program at Troy University has defined its purpose, described how it is related to the mission of the University, and established student learning outcomes (SLOs) and program objectives that are documented in that program’s Program Effectiveness Report (PER) available online in Dashboard. In 2006 and 2007, workshops were conducted by the Institutional Research, Planning and Effectiveness office to deploy this system across the University. Program coordinators and discipline faculty were trained on the technological use of Dashboard, the essential elements of program SLOs, and the appropriate selection of assessment tools. The academic deans have reviewed all academic program SLOs for inclusion of well-defined expected outcomes. College deans/designees also confirmed the assessment tools used to evaluate outcomes were appropriate. The Institutional Research, Planning and Effectiveness office also reviews the current status of PERs each term and provides feedback to deans for follow up. The deans also receive feedback from the Institutional Effectiveness Committee regarding the quality of the student learning outcomes and assessment results for the programs in their colleges.

The following links will take the reviewer to files that show the Program Effectiveness Reports for a wide variety of academic programs at Troy University.

The following examples illustrate how academic programs have used the institutional effectiveness process to improve student learning outcomes.

1. Sport and Fitness Management. Low pass rates (54 percent) on departmental exit exams in academic year 2004-2006 were noted as an area of concern. Given that there had been several revisions of the undergraduate sport and fitness management curriculum since the semester conversion in 2000 without a revision in the exit exam, faculty work groups revised the exit exam to more accurately reflect the objectives of the current curricula. The revised exit exam was given to students for the first time in the fall of 2005. First time pass rates improved from 54 percent in 2004-2005 to 90 percent in 2005-2006.
   
   
2. School of Nursing. Declining graduation and retention rates of Bachelor of Science in Nursing (BSN) students indicated a need to explore strategies to improve the success of students. Students were experiencing the greatest difficulty in the first and last semester of the program. Approximately 50 percent of students were failing skills performance check-offs in NSG 3314 Nursing Concepts I Practicum, even after mandatory practice time was instituted in the course. Students were requesting to take NSG 4406 Public Health Nursing in the summer to minimize their academic load in the subsequent fall or spring semester. NSG 4406 is a higher level course designed to teach students how to care for the community as the patient. To address these concerns, the BSN curriculum committee and faculty reviewed the overall BSN curriculum in an attempt to improve student retention and graduation rates. First, the BSN curriculum was revised in relation to the new general studies guidelines. A new course, NSG 3306 Perspectives of Professional Nursing, was added to Semester I, and NSG 3315 Pathophysiology was moved to the general studies requirements for admission to the BSN clinical nursing sequence. NSG 3323 Maternal Infant Health Nursing, NSG 3334 Psychiatric Mental Health Nursing, NSG 3332 Child Health Nursing and NSG 4405 Public Health Nursing were changed from two to three hour courses. Also, an additional hour was added to NSG 3326 Nursing Concepts II Practicum. The sequencing of courses in the BSN clinical nursing sequence was revised for completion over five rather than four semesters. These changes were also submitted to the National League for Nursing Accrediting Commission (NLNAC) for approval. The new plan did not require students to complete NSG 4413/4414 Concepts III, NSG 4405/4406 Public Health Nursing, and NSG 4421 Preceptorship, NSG 4407 Clinical Nutrition, and NSG 4417 Senior Seminar in one semester. Additionally, NSG 3314 Nursing Concepts I Practicum underwent a major revision with faculty leading and directing simulation labs and skills performance. Faculty guide, supervise, and evaluate groups of ten students each in the simulated lab setting. A phase-in of the revised curriculum began in fall semester 2005, and results indicate an improved progression rate. Approximately 77 percent of the students admitted in fall 2005 were successful in progressing to Semester II, up from 50 percent the previous fall. Only one student during the 2005-2006 academic year was unsuccessful in completing the skills requirements for NSG 3314.
   
   
3. Athletic Training Education. Athletic training education students were not performing at the desired level on the written simulation portion of their National Athletic Trainers’ Association Board of Certification (NATABOC) exams. It was hypothesized that athletic training education students have less than needed skill or practice in critical thinking and problem solving. A plan to include critical thinking and problem solving elements into all athletic training courses was developed by the faculty and implemented in spring 2006. Additionally, instructors taught reasoning skills along with a logical progression of problem solving. Instructors increased student engagement in lectures by asking “What would you do? Why?” National boards pass rate improved from 80 percent in fall 2006 to 87 percent in fall 2007.
   
   
4. Chemistry. The chemistry academic discipline committee established a 60 percent demonstrated competency level for the student learning outcome that students completing CHM 1143 Introduction to Chemistry II would be able to relate chemical behavior and physical properties to intermolecular forces. This student learning outcome was assessed through selected questions on the final exam. In the fall of 2006, only 55 percent of students correctly answered questions related to this student learning outcome. As part of the faculty-developed plan for improving student learning outcomes, a new textbook was adopted in the fall of 2007. Included with the new text were online homework and tutorials to aid student understanding of the topic.

Assessment and Outcomes in Distance Learning

Troy University has been a national leader over the past decade in providing distance learning programs, primarily to adult learners. As part of the 2003 SACS review of Troy State University, the University provided SACS with extensive evidence from assessment of 752 distance learning courses (offered online through the Blackboard Courseware Classroom) that demonstrated that student performance and learning in distance courses compared favorably to student performance and learning in the traditional classroom setting. (Source: Distance Learning Opportunities at Troy State University: Undergraduate and Graduate Degree Programs, 2004)

Troy University continues to assess student learning outcomes in distance learning programs and has further improved its ability to compare the outcomes of distance learning to traditional programs through the implementation of the Measure of Academic Proficiency and Progress (MAPP) Report for assessing general studies, and the use of major field tests as the basis for assessment of many academic programs.

The University uses the MAPP test to assess student learning in general studies, focusing on reading, critical thinking, writing, and mathematics. MAPP data enables the University to compare the performance of students in online venues, such as eArmy and eCampus, with students who take their general studies courses in a traditional classroom format on the residential campus in Troy, Ala. Table one provides data from the fall of 2007 MAPP exams on overall scores. Table two provides data from the fall of 2007 on skills sub scores and context-based sub-scores

TABLE ONE  MAPP - Overall Scores - Fall 2007
National Average	eArmy	eCampus	Troy Campus
444.6	447.5	461.75	444.43

TABLE TWO  MAPP - Fall 2007 Skills Sub-scores and Context-Based Sub-scores
	eArmy	eCampus	Troy Campus
Critical Thinking	114.13	117	112.29
Reading	119.5	122	118.48
Writing	114.88	119	114.75
Mathematics	113.5	119.25	113.05
Context-Based Sub-scores: Humanities	116	116.5	115.35
Social Science	114.25	118	113.73
Natural Science	117.63	120.75	115.69

While it is tempting to suggest that the data infers that online courses are more effective than traditional classroom delivery, it is more likely that the students taking online courses are more mature, have a broader set of life experiences, and are perhaps more focused on their academic work.

Comparison of student performance on major field tests provides further evidence of the efficacy of online learning compared to traditional classroom delivery. Not all academic programs at Troy University can be assessed with major field tests, and of those that can be, not all of these programs are offered through distance learning.

However, the data for those undergraduate programs that use major field tests and are also offered online suggests that student learning in online distance learning programs is comparable to student learning in traditional classroom programs. Tables three through seven demonstrate results for Summer 2007 – Spring 2008.

TABLE THREE Major Field Test - Sorrell College of Business Summer 2007 - Spring 2008
	Alabama Campuses	eArmy	eCampus
Average Score	147.3	146.4	147.4

TABLE FOUR Major Field Test - History Summer 2007 - Spring 2008
	Troy Campus	eCampus
Average Score	144.6	147.8

TABLE FIVE Major Field Test - Criminal Justice Summer 2007 - Spring 2008
	Troy Campus	eCampus
Average Score	144.6	147.8

Similarly, graduate programs use major field tests and capstone exams to assess program effectiveness. Table six provides comparative data on Major Field Test scores for the Master of Business Administration students who took their coursework in the traditional classroom setting on the four Alabama campuses and those students who took their MBA courses online through eCampus.

TABLE SIX Major Field Test for Master’s in Business Administration Summer 2007 - Spring 2008
	Alabama Campuses	eCampus
Average Score	244.4	248.8

Table seven provides comparative data for graduate students in the Master of Science in Management between students working online through eCampus and those taking courses in classroom settings in the Atlantic, Southeast, and Western Regions using the Major Field Test in Business.

TABLE SEVEN Major Field Test for Master of Science in Management Summer 2007 - Spring 2008
	Atlantic	Southeast	Western	eCampus
Average Score	236.4	234.8	239.3	241.0

In summary, evidence suggests that student performance on student learning outcomes is comparable between courses taken in traditional classroom settings and those taken online.

3.3.1.2 Administrative Support Services

Institutional effectiveness for Troy University’s major administrative support programs is likewise managed through the Dashboard process in which each program defines its purpose, its connection to the mission of the University, its expected outcomes, and assessment information related to expected outcomes. All of this information informs the annual planning process for support organizations and contributes to defining the annual plans for each organization.

The following links will take the reviewer to the Program Effectiveness Reports for a variety of administrative organizations as examples of how their effectiveness is assessed.

The following are examples of how administrative organizations have used their assessment data to drive
continuous improvement:

1. Sponsored Programs. Plans and strategies directed toward increasing competitive proposal writing included emphasis on faculty training and increased staff. After increased professional development of faculty and the hiring of two half-time staff positions and a full-time clerical assistant, competitive proposals increased by 40 percent between academic year 2005-2006 and academic year 2006-2007.
   
   
2. Cash Management. At the close the fiscal year 2004-2005m Troy University reported $1,366,484.65 in interest income with an average interest rate of 2.63 percent. With the development of a Treasury Services division and the hiring of a Director of Cash Management, there was a person assigned to monitor the cash accounts. The Director learned some of the University’s cash accounts did not receive any interest, and some were receiving only .002 percent. The Troy University Treasury Services division determined that interest revenue could be maximized by converting commercial checking accounts to higher interest bearing governmental checking accounts. A plan was developed to convert all bank accounts not receiving interest to interest bearing accounts. During the 2005-2006 fiscal year, accounts were systematically converted to government interest-bearing checking accounts. By the end of fiscal year 2006, interest income had increase to $3,487,421.45, and the average interest rate on University checking accounts was 4.42 percent. Between fiscal year 2004-2005 and fiscal year 2005-2006, this plan increased the revenue earned by University checking accounts by 150 percent.
   
   
3. Physical Plant. The consolidation of three separately accredited universities into Troy University in 2005 suggested a need to develop a common standardized operating procedure and method of documentation of physical plant activities. In 2005, the physical plant established an expected outcome of 85 percent implementation of the new operating procedures at all Alabama campus locations by 2007. This was achieved.

3.3.1.3 Educational Support Services

Consistent with the practice for academic programs and administrative programs, Troy University employs the Dashboard System to manage institutional effectiveness information for educational support services and establishes expected outcomes and assessment of these outcomes for these services.

The following links will take the reviewer to PER information that provides the purpose of each educational support service, its link to the mission of the University, expected outcomes and assessment information.

The following are examples of how educational support organizations have used their institutional effectiveness process to drive continuous improvement.

1. Library. During the 2002 accreditation of Troy University’s Long School of Music, the National Association of Schools of Music recommended that the library increase its collection of music compact disks (CDs). In 2003, the library began dedicating approximately $1000 to be spent annually in upgrading its music CD collection. The amount allocated for CDs was increased to $2,500 in fiscal year 2007 and has continued at that level for fiscal year 2008. The collection has risen from almost non-existent in 2002 to well over 900 CDs at the end of fiscal year 2007. Additionally, in the spring of 2007, the library acquired a machine that can convert long playing disks (LPs) into CDs. The library has a collection of approximately 9,000 LPs most of which are not currently available as CDs. The equipment acquired will allow the University library to convert LP to CD upon request by faculty or student. In addition, five new dedicated music listening stations have been added in fiscal year 2007. No library holding deficiencies were noted during the 2007 reaccreditation of the Troy University Long School of Music.
   
   
2. Department of Student Development/Counseling. The university established a goal of retaining 80% of first time freshmen on the Troy campus. Retention rates as determined by the Office of Institutional Research, Planning and Effectiveness Non-Returning Report indicated that retention for the 05/06 AY was 71%. Analysis indicated that the retention rate data were was in part confounded by international students who are enrolled in non-credit English as a second language courses, and are therefore not progressing to sophomores in the next academic year. Retention rates of native students were 76% in 05/06 and 77% in 06/07. In the fall of 2007, Troy University established a College of First Year Studies and hired a First Year Studies dean who is responsible for the coordination of general studies courses, freshman orientation, and freshman support services.

3.3.1.4 Research within the educational mission

Troy University values research by both students and its faculty. Although Troy University has been traditionally considered a teaching university, the University places value on research as part of the overall educational experience as demonstrated in the Troy University mission statement.

A. Faculty Research.

Troy University recognizes that in higher education both teaching and research are essential to a vigorous institution and a sound curriculum. The Faculty Handbook Section 3.4.5 states that “a requisite for effective teaching is the active involvement in the intellectual and scholarly developments of an individual’s [faculty member’s] field”. It is incumbent on each faculty member to maintain his or her competence by keeping abreast of developments in his or her own field. Troy University expects each faculty member to devote a reasonable amount of time to research and scholarly / creative activities. Troy University identifies expected research/scholarly/creative outcomes, assesses those outcomes, and provides evidence of improvement in significant ways.

1. Annually, each faculty member lists research and scholarly/creative activities that have been accomplished during the preceding year in his or her year end self evaluation. Supervisors evaluate the extent to which a faculty member has complied with expectations. When the faculty member does not meet expectations, the supervisor recommends this area as an area needing professional growth. Likewise, the faculty member will include recommended growth in scholarly / creative activities in the upcoming year’s Professional Development Plan.
   
   
2. The annual plans and evaluation of a faculty member’s scholarly / creative work are accumulated and used as supporting documentation for the tenure and review process. The tenure and preview process is explained in greater detail in Sections 3.5 and 3.6 of the Faculty Handbook.
   
   
3. Recognizing that scholarly research and activities can take different forms in different disciplines, the five colleges of Troy University have established guidelines to help faculty meet expectations of research and creative pursuits within each college. College of Arts and Sciences guidelines, College of Health and Human Services guidelines, Sorrell College of Business guidelines, College of Education guidelines, and College of Communication and Fine Arts guidelines are provided.
   
   
4. Troy University’s Institutional Review Board (IRB) provides oversight for all institutional and individual research that is performed by administrators, faculty and students. The major function of the IRB is to ensure protection of the rights of human subjects who participate in research endeavors conducted by Troy University, faculty, professional staff and students. The policies and regulations of the IRB are guided by federal rules and regulations, and are based on the Protection of Human Subjects Code of Federal Regulation, and the Belmont Report. According to 45CFR 46 - Protection of Human Subjects, at institutions receiving federal funds, all research involving human participants must be reviewed and approved by an IRB. Information on the role, policies and procedures of Troy University’s IRB is published online.
   
   
5. Several departments and colleges have created their own research colloquia. Included in this category of research activity is the Sorrell College of Business Annual Business Research Symposium, the eCampus E-Colloquium, the National Institute on Social Work and Human Services in Rural Areas, Sorrell College of Business Montgomery Research Seminars, and Department of Psychology’s Annual Research Colloquium. These events provide opportunities to researchers within a community to share and develop research and are open not only to Troy University participants but also to participants from other academic entities.
   
   
6. Troy University faculty members are encouraged and supported in research and scholarly/creative activities. Departments annually request a travel budget which is used to support attendance at research-based professional events. Additionally, University Faculty Development Funds are available to assist faculty in travel /costs associated with research and professional development. The types of grants available and the policies and procedures used in awarding these grants are explained in detail on the Faculty Development Web page. Details regarding the University’s commitment to support faculty research-based professional activities are provided in Comprehensive Standard 3.7.3.

B. Student Research.

The Troy University Graduate Catalog, p. 4 specifically notes that one of the primary goals of graduate studies at Troy University is the preparation of students for research. Graduate students are expected to perform research in a manner that contributes to their respective fields as a demonstration of appropriate preparation. All graduate programs require certification of the student's ability to do research in a specialization. This requirement is met by achieving a grade of B or better in an approved research course in the student's program. Students must repeat the research course if a grade of C or below is attained. Approximately 23 percent of (13 of the 58) graduate programs offered at Troy University offer a thesis option for graduate students within the curriculum.

Troy University Graduate Thesis Guidelines are used in preparing and reviewing theses. Thesis guidelines have been established by graduate faculty across disciplines and coordinated through the Graduate School. The guidelines are reviewed and updated every two years. Individual departments, schools and colleges may impose additional requirements or may specify requirements in greater detail. The graduate student has a responsibility to learn what, if any, special departmental/school/college requirements may apply.

All students who prepare a thesis or field project must pass a final examination covering the thesis or field project. All field projects or theses are proposed well in advance and approved by the advisory committee assigned to the student. The student’s major professor (a member of the student’s major department and the chair of the student’s advisory committee), assisted by the committee, is responsible for directing all aspects of the thesis or the student’s field project. The completed, final thesis or field project will be submitted in appropriate form and in sufficient copies to the advisory committee for approval and then to the Dean of the Graduate School.

Troy University evaluates its research activities to assure a common process of assessment that supports the University mission and assists in the design and improvement of educational experiences to enhance student learning.

3.3.1.5 Community/public service within its educational mission

Troy University engages in community and public service related to its educational mission. The largest activity that the University supports in this area is the operation of a public radio network and a cable television channel that give students an opportunity for experience in broadcast journalism. The public radio station reaches listeners from central Alabama to western Georgia. Institutional effectiveness information for radio and television is included in the Dashboard system.

1. Radio & Television. Eighty-four percent of listeners indicated on a 2006-2007 annual survey that the quality of the Troy University Public Radio reception was either good or excellent. This assessment outcome fell below the expected outcome of 90 percent stated in the Program Effectiveness Report. In order to improve the quality and reliability of radio reception across the service area, it was determined that backup generators and additional antenna were needed. During 2007-2008, a new antenna was placed at the WRWA Wicksburg site, a rectifier was replaced in Phenix City, a new generator was purchased for placement in Pine Level, and the Uchee Rectifer plant was rebuilt. These actions have resulted in improved signal strength and radio reception as documented by the 2007-2008 annual survey of Listeners with over 90 percent responding positively.
   
   
2. Rosa Park’s Children Wing. Troy University has maintained the Rosa Parks Library and Museum on the Montgomery campus since Dec. 1, 2000. Since that time nearly 200,000 children and adults have visited the museum. Approximately 95 percent of the feedback was positive, however, requests were made for a historical educational component directed to school aged children to help them understand why Rosa Parks and the 55,000 citizens who walked for 382 days as part of the Montgomery Bus Boycott did what they did. After considerable research and obtaining of a federal grant, and with help from the city of Montgomery, Troy University was able to add a Children’s Wing. The Children’s Wing opened for visitors in 2006. Components of the Children’s Wing include the Cleveland Avenue Time Machine that puts children visually back in a time of segregation and helps them to experience the emotional and social implications of segregation. The Children’s Wing also provides a research center for information related to civil rights and the Montgomery Bus Boycott. The Children’s Wing won the prestigious THEA Award for 2007 from the Themed Entertainment Association, as well as several other recognitions by major organizations. Attendance in 2006 was 11,651. Attendance increased in 2007 to 16,111.

Conclusions

Troy University has effectively merged its separately accredited campuses and has forged a single system for uniform and consistent management of institutional effectiveness. Academic programs, administrative functions, student support organizations, and community/public service programs are all included in the Dashboard system that provides access to information on expected outcomes and assessment information in all of these areas. Troy University is in compliance with this Comprehensive Standard.