Assessing Tommies

Opus College of Business Assurance of Learning Program

Fall 2017

By Dr. Marcella de la Torre (Director, AACSB Administration, Opus College of Business) and Dr. Michael Garrison (Senior Associate Dean / Professor Ethics and Business Law, Opus College of Business)

Background

The Association to Advance Collegiate Schools of Business (AACSB), the premier international accrediting body for collegiate schools of business, has been instrumental in the movement to student learning assessment in management education. Under the accreditation standards adopted by AACSB in 2003, business schools are required to have a systematic and ongoing process to assess student learning and to make curricular improvements in response to assessment results, a process referred to as “assurance of learning” (AOL).

The AOL process begins with the identification of mission driven goals and culminates with the critical closing-the-loop stage, which is the development and implementation of curricular changes to improve student learning. AOL involves a cycle of continuous improvement of curricula, with schools regularly assessing and improving their programs to ensure essential student competencies. It focuses on outputs of the school’s programs (knowledge and skills of students) rather than inputs (faculty qualifications and content). Under AOL standards, assessment must be conducted at the degree program level; faculty ownership of the AOL process is critical; and schools must document and report their assessment activities in improving student learning.

The OCB Philosophy and Approach to AOL   

The development of the Opus College of Business (OCB) mission-centered assurance of learning program began in 2006, when the chair of the OCB AACSB committee worked with the program directors and the Curriculum Committee to identify program-based learning goals and objectives. It was decided that all OCB programs would develop and use a common set of mission-centered AOL goals and that all programs would generally assess a common set of objectives within each goal.  AOL differentiation between undergraduate and MBA programs would be based on the level of demonstrated competency for each goal/objective and the focus or context of application.

Originally, the OCB faculty assigned responsibility for the implementation of the Assessment Plan to a subcommittee of the Curriculum Committee, but as the faculty gained experience with a complex AOL system, it became apparent that a standing faculty committee was needed to set the standards for and effectively manage all aspects of the assurance of learning process. An AOL Committee was established in 2009 to oversee the AOL process. As presently constituted, the AOL Committee includes all faculty program directors, the department chairs, and the chair of the OCB Curriculum Committee. The Senior Associate Dean acts as the convener of the committee and serves as an ex-officio member, along with the Director of AASCB Accreditation.  

Student Learning Goals and AOL Process

AACSB standards require schools to adopt mission-based goals and objectives. The OCB mission reads as follows:

Inspired by Catholic intellectual tradition, the Opus College of Business develops effective, principled business leaders who think globally, act ethically and create enduring value for society.

Based on our college mission, the college adopted four key learning goals for all of our programs and students: (1) knowledge of current business practice and theory (business acumen), (2) leadership, (3) critical thinking, and (4) ethical decision making. The UST Business Professional diagram represents the integrated nature of these goals: 

 

Under the college-wide assessment plan, the college follows a two-year AOL cycle. Data are collected and course-embedded measures scored on two goals in all programs over an entire academic year. Program faculty groups review the assessment results and initiate any necessary curricular interventions to close the feedback loop as soon as practicable in the year following the data collection.

AACSB standards generally prefer business schools to employ “direct” measures of student performance, such as examinations and other course assignments. As a result, the college primarily utilizes so-called “course-embedded measures” in the AOL process, including cases, papers and other student work. The college also employs national standardized exams to assess business acumen (Goal 1) in the undergraduate and MBA programs, specifically the ETS ® Major Field Tests. Indirect measures – surveys of students and alumni – have also been utilized

AOL Successes and Challenges: Closing the Loop

Closing the loop is the most important and yet the most challenging stage of the AOL process. Corrective measures need to be developed, implemented and then judgments need to be made as to whether the curricular interventions were successful, which is generally a multi-year process. Recent successful closing the loop measures adopted by OCB programs include changes in courses and course content, program-level changes and other measures designed to improve student learning. Examples include: 

  • The Full Time MBA program responded to a weakness in students’ ability to bridge the gap between theory and practice that was identified in leadership and ethics assessments. Accounting and ethics faculty altered assignments requiring students to focus on decision implementation, consider objections to proposed actions, and determine how realistic assumptions were by considering actual costs. 
  • The MS in Accountancy program responded to a weakness in advanced technical knowledge by developing course content in the international accounting course requiring students to identify the public interest issues in an international accounting situation, and to make recommendations to protect the public interest.    
  • The Part Time Flex MBA program provided training by a business ethics faculty for course instructors on the frameworks for ethical decision making to improve classroom discussion of ethics.
  • The Executive MBA program responded to a weakness in leadership by redesigning the first five-day residential component of the program to focus on organizational behavior and executive development. Also, when the program was redesigned, a new series of six one-credit leadership development courses was developed for the program.  

AOL Successes and Challenges: Learning Objectives and Measures  

Throughout the development of the college's AOL system, faculty have learned that objectives must be clear and measurable, and course-embedded measures have to be carefully developed to accurately assess the learning objectives. For example, the original leadership objectives focused on understanding leadership perspectives, and in developing and communicating effective strategies. One problem was the overlap with critical thinking and ethical decision-making objectives. A more fundamental problem was the emphasis on knowledge and analysis rather than the key dimensions of “being” a leader. Consequently, the college adopted new leadership objectives that focus on taking initiative and influence, the capacity to bring others along to achieve positive results for the organization. Similarly, programs have sometimes developed assessment measures that were not closely aligned with the objectives making it difficult to assess student work. Obviously, without valid assessment data, improving student learning through effective closing the loop measures is virtually impossible.