SUNY Orange General Education Assessment Plan - Cycle 2 Competency: Critical Thinking* Year: AY 2007/2008 SUNY Student Learning Outcome(s): Identify, analyze, and evaluate arguments as they occur in their own or other�s work Develop well-reasoned arguments SUNY Orange assessed critical thinking by administering the CAAP Critical Thinking Test by ACT during the first GE assessment cycle. Results indicated that while SUNY Orange students scored slightly higher than the national mean, there was room for improvement in students� critical thinking skills. In response, the General Education Committee in collaboration with the Coordinator of the CTL and the Academic Affairs Leadership Team Cycle 1 created opportunities for college-wide discussions and strategy development sessions on Assessment critical thinking issues. Additionally, the assessment results have been disseminated and discussed at several critical thinking workshops. Recommendations from workshop participants, faculty, and administration are being formalized and will continue to be addressed through a collaborative college-wide approach. The General Education committee has incorporated the feedback in the development of the SCBA for Critical Thinking. The newly formed Assessment Advisory Committee will work collaboratively with Academic Affairs and Department Chairs representing each of the 10 GE areas to review the General Education curriculum in order to designate courses as �Critical Thinking Intensive.� These courses will be representative of the 10 General Education knowledge and skill areas Courses and will provide students with the opportunity to practice/develop critical thinking skills, particularly demonstrated in a written format. This designation process will formally begin in Fall 2006. A random selection of �Critical Thinking Intensive� courses will be selected to participate in the assessment to ensure that at least 20% of the total students enrolled in the intensive Sampling courses will be assessed. The sample selected will be representative of SUNY Orange�s Information General Education Curriculum. The Assessment Instrument & Methods: A �Critical Thinking Intensive,� course must identify written student work that follows the Standards and is able to be scored using the SUNY developed Scoring Rubric. Student work may include a variety of written formats depending from which of the ten General Education areas it originates. The written assignment may include essays, papers, portfolios, experiments, etc. Assessment Methods Specific parameters for the critical thinking assignment/instrument will be developed during & the Fall 2006 semester as courses are designated as �Critical Thinking Intensive.� The Externally assessment methods will be determined during the Spring 2007 semester, following a pilot Referenced (See validity section). Measures Externally Referenced Measures: SUNY�s Critical Thinking Standards and Scoring Rubric: All critical thinking assessments will be scored using the Standards and Scoring Rubric developed by the SUNY discipline panel. Scores will be aggregated for SUNY reporting purposes. Students participating in the assessment will be asked to provide self-reported demographic information. Information to be collected may include, full-time vs. part-time enrollment status, major, developmental course history, utilization of campus writing center, etc. Student demographic data will provide valuable information for many areas of the college to more effectively improve student learning and critical thinking skills, review the General Student Education curriculum, create professional development opportunities, and increase student Participants engagement, etc. The embedment of the critical thinking assessment within �Critical Thinking Intensive� courses will address concerns of student motivation. Students will also be given the scoring rubric prior to the assessment. In order to demonstrate consistency in scoring the assessments, two readers/raters will score each assessment included in the 20% random sample with the first reader/rater being the course instructor. Inter-rater reliability: To ensure inter-rater reliability, a representative group of faculty will conduct norming sessions prior to the administration of the assessment to obtain agreement on the scoring definitions. Norming sessions will result in faculty agreement of the scoring Reliability definitions at least 90% of the time. Ongoing norming sessions will be held to ensure consistency in the scoring of student assessments. If discrepancies occur in the scoring, a third faculty reader/rater will score the assessment. SUNY Orange faculty request compensation information from SUNY for both the norming process as well as for the second reader/rater for the 20% random sample. The Assessment Advisory Committee will work with Academic Affairs and Department Chairs as well as faculty members representing each of the General Education areas to develop parameters for the assessment as well as to review the instrument(s) for its validity. This process will ensure face validity in that the assessment is accurate and appropriate in Validity measuring each of the student learning outcomes. Additionally, a pilot will be conducted during the Spring 2007 semester whereby a representative sample of student work from designated �CT Intensive� courses will be scored using the Standards and Scoring Guide to ensure both reliability and validity of the instrument. The standard that defines what level of student performance the faculty considers as �exceeding, meeting, approaching, and not meeting� is outlined in the Scoring Rubric developed by the SUNY discipline panel. SLO 1 SLO 2 Criteria Percentage (%) Percentage (%) Criteria "Exceeding" "Meeting" "Approaching" "Not Meeting" After the assessment has been completed and the results analyzed, a report will be created to Results & document the instrument, methods, and findings. This report will be shared among SUNY Dissemination Orange stakeholders including all academic Departments, student development services staff, Assessment Advisory Committee, additional governance committees, the Center for Teaching and Learning, and college administration. The findings will also be shared with students; however, creative and effective methods will need to be determined. The findings of the assessment will be relevant to many facets of this college all of whom have a stake in improving student learning, particularly in moving students to higher level thinking skills. Findings will also be used to assist the Center for Teaching and Learning in creating specific professional development opportunities for faculty and staff that address critical thinking issues. Both the assessment process and the findings will assist the college in continuing to promote critical thinking across the curriculum. Additionally, a review of this assessment process will be conducted and facilitated by the Assessment Coordinator and supported by the Assessment Advisory Committee in order to identify areas of improvement in conducting future assessments. *Indicates Strengthened Campus Based Assessment This plan was developed through of a collaborative effort of the General Education Committee, the Assessment Coordinator, as well as faculty representing all 10 General Education areas. The plan has been approved by the General Education Committee. Upon receiving feedback from GEAR, the implementation of the pilot and plan will be led by the newly formed Assessment Advisory Committee. (This comprehensive assessment committee has assumed the responsibilities of the General Education Committee.)