Assessment of Student Learning and e-Scholar Program


Action Project


 
Institution: University of Wisconsin-Stout
Submitted: 2002-09-23

Timeline:
Planned project kickoff date:
Target completion date:
Actual completion date:
A. Give this Action Project a short title in 10 words or fewer:
Assessment of Student Learning and E-Scholar program at UW-Stout
 
B. Describe this Action Project's goal in 100 words or fewer:
The goals of this project are to develop an overall assessment framework for the campus and to improve the methods by which student learning outcomes are measured in major programs, in the general education program and through the new e-scholar program at UW-Stout. This will allow the university to enhance our premier reputation as a “hands-on, minds-on” university, and use the e-scholar program and its benefits as a brand image for our campus.
C. Identify the single AQIP Category which the Action Project will most affect or impact:
Primary Category: Helping Students Learn
D. Describe briefly your institution's reasons for taking on this Action Project now -- why the project and its goals are high among your current priorities:
UW-Stout implemented assessment requirements for all academic programs in 1992 and a systematic review process for general education in 1998. However, over the past decade, the quality of the assessment processes implemented has varied considerably.  The e-scholar (laptop) program was implemented in the fall of 2002; therefore this program has not yet been assessed to determine its impact on student learning.  Although there are numerous assessment initiatives at UW-Stout, the campus lacks an overall framework for student learning assessment. These related issues were discussed at the Chancellor’s Advisory Council retreat in the summer of 2000 and brought forward as a university priority for 2001-2002. Although initial discussions have been taking place on improving assessment, much more effort is needed in this area.
E. List the organizational areas - -institutional departments, programs, divisions, or units -- most affected by or involved in this Action Project:
This project will involve faculty and staff in several areas of the campus.  All of the academic departments and colleges will be affected; as well as several areas in support services. Support units such as Learning Technology Services and Technology and Information Services will be heavily involved in this project. Program directors, associate deans and deans will have key roles in this project. Several existing governance committees will be involved.
F. Name and describe briefly the key organizational process(es) that you expect this Action Project to change or improve:
Critical processes include the annual “assessment in major” process, the academic program review process; the general education review process, the e-scholar implementation and evaluation processes; and the faculty/staff training and development processes
G. Explain the rationale for the length of time planned for this Action Project (from kickoff to target completion):
UW-Stout would work through existing organizational and governance structures to keep the institution focused on this project. Special taskforces have been convened to work with development of the assessment framework and the e-scholar implementation and evaluation.  A structure to support continued assessment efforts must be implemented; an assessment coordinator may be needed.
H. Describe how you plan to monitor how successfully your efforts on this Action Project are progressing:
Number of faculty/staff that attend meetings or training sessions on these topics.  Assessment committee is formed and develops campus framework for assessment. Number of program directors that submit assessment reports that include data/evidence of student learning outcomes.  Improved processes for general education assessment developed. A plan for evaluating the e-scholar program is developed and implemented. The number of web-enhanced courses continues to increase.

Process assessments related to the Primary criteria identified above will include monitoring the frequency, availability and comprehensiveness of supporting training programs for faculty, staff and students.  Major stakeholder groups will be polled regularly to guide the ongoing training and professional development opportunities.  Process assessments related to the four Related criteria will assess institutional progress toward developing the most efficacious assessment strategies for our diverse stakeholders.  Measures will track the validity, adequacy, timeliness and coordination of assessment content and mechanisms.
I. Describe the overall "outcome" measures or indicators that will tell you whether this Action Project has been a success or failure in achieving its goals:
Annual assessment reports from all majors demonstrate evidence that data are collected and utilized to improve courses and programs. A systematic process has been implemented to review and improve the general education program. An e-scholar evaluation has been developed and results are reported annual. Student competencies are measured annually and the results are reported internally and externally.
J. Other information (e.g., publicity, sponsor or champion, etc.):
2002-2003Assessment framework is developedModels for general education assessment are developed and shared with faculty/staffCurrent “assessment in major” processes are reviewed and feedback is provided E-scholar evaluation is developed2003-2004Revised assessment process for general education is piloted Data are collected on e-scholar program effectiveness2004-2005Revised assessment process for general education is implemented100 percent of program directors submit appropriate “assessment in major” reports

Last Action Project Update: 2004-09-08
A. Describe the past year's accomplishments and the current status of this Action Project.
Substantial progress was made on each of the project goals including:

a) Assessment of student learning outcomes in major programs
Each undergraduate and graduate program is required to submit an annual report of how assessment of student learning is conducted in that program, how the results of these assessments are shared with faculty and staff, and how the results are used to improve teaching and learning within the program.  Ninety percent of program directors submitted reports for the 2002-2003 academic year.  Each report was shared with the newly-formed campus assessment oversight committee; and the Assistant Chancellor for Assessment and Continuous Improvement provided individual feedback to each program director.  The Assistant Chancellor is currently in the process of reading and reviewing the reports submitted for 2003-2004. One of the goals of this project was to streamline the reporting process through the use of web-based format and create an electronic database of assessment information so program directors can review approaches utilized by other programs. The web-based format was made available in the spring of 2004 and an electronic record of program assessment information is currently being compiled by the Provost’s Office and the Center for Assessment and Continuous Improvement. In addition, the use of student portfolios as an assessment tool was implemented in all teacher education programs.

b) Assessment of student learning outcomes in general education
After extensive research, discussion and review of best practices at other institutions, a new model for the assessment of general education that utilizes course-embedded assessment and standardized examinations, including the ACT CAAP, was developed by the General Education Assessment Taskforce and approved by the General Education Committee (standing committee of the Faculty Senate) in September, 2003. During 2003-2004, the taskforce worked with the Provost’s Office and the Center for Assessment and Continuous to implement the model. The ACT CAAP was given to 444 upperclassmen in March of 2004. UW-Stout’s overall scores in 2004 are comparable to national scores, as are the scores for the junior and senior students, when examined individually (see results below). The course-embedded assessment process is scheduled to begin in all approved general education courses in the fall of 2004. Initial training was provided to general education instructors and department chairs on September 1, 2004, with follow-up training provided to individual colleges and departments during the fall semester.

UW-Stout Budget, Planning and Analysis Office, 2004.

2004 ACT CAAP Scores, UW-Stout Summary
        Writing  Math   Reading Critical Thinking   Science
UW-Stout  63.4  58.1  62.5        62.7       61.1
National  64.5  58.1  62.6        62.1       61.1

c) Assessment of student learning outcomes through the e-Scholar program
In the fall of 2002, UW-Stout began its e-Scholar (laptop/digital campus) program. An assessment taskforce was formed, an assessment plan was developed and a significant amount of initial data have been collected on the effectiveness of the e-Scholar program.  The evaluation is focused on four areas:  student learning and academic performance; student engagement; student satisfaction; teaching and instructional practices.  Evaluation methods include student surveys, student focus groups, faculty surveys, faculty focus groups and behavioral data (including pass rates and grades in specific courses), and freshman to sophomore retention rates.  Results of these assessments are shared widely with the campus.
In the spring of 2004, a student questionnaire developed at Valley City State University was conducted at UW-Stout to obtain comparative data on student learning outcomes in a laptop environment.  Six hundred UW-Stout students participated in the online survey which included items dealing with computer use, student learning, courses, online interactions, interactions with faculty and the importance of various technology elements. When compared to the benchmark, UW-Stout students scored high in items related to working in groups, critical thinking and using laptops outside of the classroom. UW-Stout students reported the lowest ratings on questions dealing with computer use during class and online interactions. These results were used to create a series of professional development workshops on “teaching in the mobile classroom” that will be delivered beginning in September 2004.  A summary of all of the data collected to date, and actions taken based on the results, may be found at http://www.uwstout.edu/bpa/ir/laptop/indexlap.htm
B. Describe how the institution involved people in work on this Action Project.
Several offices, individuals and groups have been involved in this project including all of the academic colleges; the School of Education; all undergraduate and graduate program directors; the General Education Assessment Taskforce and General Education Committee; the e-Scholar Assessment Taskforce; the Budget, Planning and Analysis Office; the Provost’s Office; and the Assessment and Continuous Improvement Center as well as all of the faculty, staff and students who provided data for the various assessments.
To support this action project, the Chancellor created a new office, the Center for Assessment and Continuous Improvement, in 2003. This office is headed by an Assistant Chancellor and includes a half-time program assistant and a graduate assistant. The Provost’s Office continues to provide clerical support for the assessment in the major and general education assessment processes. The Office of Budget, Planning and Analysis has also been instrumental in providing assistance with data collection, data analysis and developing executive summaries of assessment reports for distribution.
  To keep the campus focused on this initiative, periodic progress updates on this project have been provided to several groups on campus including the Chancellor’s Advisory Council, the Provost’s Council, the Academic Affairs Administrative Team (which also serves as the campus assessment oversight committee), the Faculty Senate, the Senate of  Academic Staff, and the various college/school councils.
C. Describe your planned next steps for this Action Project.
This project will be officially retired following the submission of this report so that the campus can move forward on new action projects generated through the recent campus-wide strategic planning process.  However, UW-Stout will continue to assess student learning outcomes in the areas described above (major programs, general education curriculum and   e-Scholar outcomes) and use the results to improve teaching, learning, programs and services.  Oversight of these activities has been incorporated into the job description of the Assistant Chancellor for Assessment and Continuous Improvement, with continued support from the Provost’s Office and the Office of Budget, Planning and Analysis.
UW-Stout recently received a TITLE III grant of $1,750,000 to strengthen assessment, advising, retention and technology integration initiatives across the institution. This funding will allow the campus to expand the e-portfolio project to all undergraduate majors and provide additional extensive training in course-embedded assessment to all general education instructors.
D. Describe any "effective practice(s)" that resulted from your work on this Action Project.
Assessment of the e-Scholar program is comprehensive and could be a model for other campuses interested in implementing a laptop program. This has generated a great deal of interest from other institutions and could be presented at the North Central Association Higher Learning Commission’s annual meeting. The session could include preparation for a laptop environment, faculty training and development as well as assessment. For details on the assessment plan and results, see http://www.uwstout.edu/bpa/ir/laptop/indexlap.htm.
E. What challenges, if any, are you still facing in regards to this Action Project?
Because of continued budget reductions, workload for faculty and academic staff involved in the assessment of student learning has increased over the few past years. Release time allocations were reduced for some program directors, and other instructors have increased class sizes or are teaching additional sections. This has brought about increased concerns regarding workload and less time and enthusiasm for assessment-related activities, such as attending training sessions or developing common instruments.  However, the recent TITLE III funding will allow the campus to provide release time and/or stipends to faculty and academic participating professional development activities in this area.
F. If you would like to discuss the possibility of AQIP providing you help to stimulate progress on this action project, explain your need(s) here and tell us who to contact and when?