Using
Lesson Study to find evidence of Student Learning
Submitted
by:
Len
Bogner
Submitted
for:
Scholarship
of Teaching and Learning Project 2004-2005
A. REFLECTION ON THE PROJECT
Which practices did I employ to
encourage deeper learning?
I
used a technique called Lesson Study in my research to not only encourage
deeper learning but to also look for evidence of this learning.
What is the difference between a teaching and a learning question?
A
teaching question focuses on what the instructor is doing. A learning question
focuses on the student and their understanding of the lesson. With a learning
question we focus on the actions of the student. We try to look through their
eyes to better understand our practice. What evidence can we find to prove that
they not only understand the concept, but also can apply it in their
profession?
What does it mean for a student to have learned deeply in the course?
In
looking at the evidence of student learning we find that this question has
several answers. Using Bloomıs Taxonomy we found that students are at different
levels of comprehension, and I would define deeper learning to be associated
with the higher levels of application, analysis and synthesis. Our goal as
technical instructors should be to get the students to the level of application
and hope to push them to the analysis and synthesis stages. In this study we do
find evidence that the students all reach application and the some did obtain
the levels of analysis and synthesis.
What will I do to continue my investigations?
Because
there are a limited number of universities that have Graphic Communications
programs, and I would like to extend the Lesson Study to those schools.
B.
REPORT ON THE STUDY
Introduction to the Study
The idea of Lesson
Study is not complicated (Lewis, 2002).
Logically, collaboration with colleagues to plan, observe, and reflect
on lessons to improve teaching. Although the idea is straight forward, the
process involves a commitment to agreed upon goals, conscientious collection of
data regarding student learning, and processes that promote constructive
discussion of complex issues.
Lesson Study is a
technique that has been used in the Japanese K-12 educational system for over a
century (
I was first
introduced to the idea of Lesson Study when Cerbin and Kopp presented at the
There were several reasons for conducting this research. First I teach for the Graphic Communications Management program, which has a narrow focus, and there are no more then 15 universities that offer this specialized degree. So in order to form a Lesson Study group with similar backgrounds I wanted to extend my possible populations to other programs. Because there are so few programs the ability to share ideas on teaching and learning with peers in our discipline becomes limited. These facts help me develop the following research questions:
How successful would a Lesson Study be that was conducted by instructors from like disciplines at different universities?
Would Lesson Study provide an instrument to create collegiality with individuals at other institutions to create conversations about teaching and provide evidence of deeper student learning in technical courses?
The Lesson Study
conducted began at Pittsburg State University (PSU), which is located in
The Lesson Study
was repeated a week later at the
Method
The process of a
Lesson Study is fairly simple. While different authors suggest slightly
different models, they all consist of the same premises. A group of educators
and teachers connecting with one another, administrators, specialists, and even
with other schools come together over a period of time to work on a single or
multiple lesson plan(s) and to analyze the student population (Buckwalter,
2002). The group starts with lesson plans because they are the building blocks
of any course. The group then moves to the classroom and as one member teaches
the class, the non-teaching members participate as researchers by recording
student reactions in order to document student thinking. The job of an observer
is to examine students for learning, thinking, and confusion. They note student
comments, their levels of collaboration and engagement, and the resulting work
produced from the lesson (
Cerbin with the
Lesson Study project he completed at the
The first step is to form a Lesson Study team (Cerbin, 2004a). Typically, a Lesson Study group is made up of a few teachers (three to six) whose goal is to improve student learning and their teaching. The team participants come from the same discipline to collaborate in developing the lesson, observing the classroom, and analyzing what worked.
The second step is to develop student learning goals (Cerbin, 2004a). The essence of a lesson is the learning goal since it provides the purpose for teaching and observing it. The team first selects a subject within a course to study. The goals are then stated in terms of what they want the students to be able to do and understand. They are to specify the students' learning, thinking, engagement, and behavior. The lessons tend to have several goals, which may include larger long-term outcomes.
The third step is to plan the research lesson (Cerbin, 2004a). In higher education lesson planning is generally performed individually, without participation from colleagues. Sharing knowledge is one benefit from lesson planning. Team members share teaching philosophies and styles as well as stories on what has worked and what has not worked in their classrooms. The group attempts to look at the lesson through the studentsı eyes, what Cerbin has dubbed "cognitive empathy," the goal of which is to understand how the students will absorb the lesson plan.
Step four is to teach and observe the lesson (Cerbin, 2004a). One group member teaches and the others sit in attendance to ³observe and collect evidence of student learning, thinking and engagement² (p. 8). The team develops data collection procedures to determine if the learning goal was met. Evidence based on observation is the most common collection device. Students are told about the research lesson and the observers are introduced before class begins. The observation is focused on the students and what they are doing in response to the instruction. Outside observers may be invited in at this step.
The fifth step is to analyze evidence of learning (Cerbin, 2004a). The three questions addressed by the analysis phase include ³In what ways did students accomplish the lesson goals? How could the lesson be improved? [and] What did we learn from this experience?" (p. 10). When the class ends the group meets to debrief. Members talk about their observations and spend time evaluating the lesson. The member's notes are then compiled looking for evidence of student learning.
Step six is to
revise the lesson and report the results (Cerbin, 2004a). "Lesson Study
usually involves more than one iteration of the lesson" (p. 12). After meeting and looking at the data the group looks at
ways to improve the lesson plan. If major revisions are made, steps three
through five are repeated. If the groups decides the goals of the Lesson Study
were met they create a research lesson report. In this report it gives an
account of what the group learned and provides shared information to help other
instructors improve their teaching.
I have decided to report the project as
one Lesson Study that is repeated. I reported the findings by following the six
steps listed above. The Lesson Study began at PSU and was repeated at Stout a
week later.
The Lesson Study begins at
Pittsburg State University
The Lesson Study would focus on Estimating Systems. The instructors wanted to incorporate Microsoft Excel into the lesson plan to help illustrate how Estimating Systems are developed and to give the student a better understanding of where the numbers used to estimate cost and price are derived.
The Lesson Study group reviewed the course objectives for the estimating lesson and discussed what the instructors wanted to accomplish. The Lesson Study learning goals were then quickly agreed upon. The lesson plan was then created to reflect the steps to achieve the goals. Because the Lesson Study was to be deliver over two class periods the group created a timeline. This timeline was to help keep the instructor on message and on time.
Evidence of Learning was gathered through observation and notation. The group members who were observing were instructed to write down what they saw and then note the time of the observation. After the evidence was gathered the members explained and interpreted their notes to the group as a whole. After looking at and analyzing the evidence some minor changes were made to the lesson plan.
1. Form a Lesson Study Group
A Lesson
Study group usually consists of several teachers (3-6) who are interested in
working together to improve their teaching and student learning.
The Pittsburg
State University Lesson Study group was made up of three instructors in the
Graphic and Imaging Technologies Department at PSU and the instructor of the
Estimating course at Stout. The teaching experience of the instructors ranged
from five years to thirty plus years. All of the instructors have worked in the
graphic communications industry. One instructor holds an Ed.D in Vocational
Education; one holds an Ed.S in Technical Educations and the others posses
Master Degrees. All are graduates of Pittsburg State University. The instructor
from Stout delivered the lesson plan.
2. Develop Student Learning Goals
The
learning goal is the backbone of a lesson and provides the ³reason² for
teaching and observing it.
The primary instructors of the estimating courses took the lead in creating the learning goals for the Lesson Study. Through cooperative research and over a period of eight weeks they looked both Graphic Communications Management programs and where the estimating courses fell in there programs then isolated the objectives or outcomes of each estimating course that represented the estimating software lesson. The Lesson Study group then developed the following goals:
The students will:
1. Understand how to use a Computer Aided Estimating System
2.
Be able to create an Estimating System using Excel
3. Plan the Research Lesson
Teachers design a lesson to achieve the learning goals, anticipating how students will respond.
Lesson Plan
150 minutes total (2-75 minute classes)
1st Class period
5-10 minutes Introduction of the Instructor
Background
Describe their Cost Estimating course
Explain the Lesson Study (Video)
Game Plan for next two class periods
5-10 minutes Defining Cost Estimating:
What is Cost Estimating?
Review the Estimating formula of BHR's x STP's + Materials + Buyouts
What is the difference between Cost and Price?
What is a Spread Sheet Program?
15-20 minutes demonstrating an estimating software (Power Quote)
20-30 minutes showing basic Excel Software and formulas
Break into groups of two
Let them have the rest of the time to work
2nd class period
10 minutes to review project and answer questions
50 minutes to work in groups
15 minutes to review the good the bad and the ugly of the project.
Work on the project outcomes
4. Gather Evidence of Learning
One team member
teaches the lesson while others observe, collecting evidence of student
learning.
Evidence of Learning
Answering Questions
Opening files on the computers
Flipping through the books
Watching others work
Bouncing ideas back and forth
Observing others in the group
One group worked on two computers
Students getting frustrated
Cutting and Pasting Formulas
Manipulating Formulas
Slow progression not a quick hit
Fighting Formulas
Fighting the Spreadsheet
Taking Notes
Listening
Different Learning Styles (students asking questions)
Students asking if you can customize the software (Led in question)
Answering questions from material from previous classes
Having files from a previous class
Who took the leadership role, and if it changed (working well in groups)
Working through a problem (Management skill)
Taking separate responsibilities (using their strengths)
Different styles in doing the project
Working on the project outside of class
Looked at their notes
Used a calculator to double check figures
Checked the Blue Book
Talking and explaining where numbers should go
Heard a student respond ³Yeah² made a positive gesture
Student raising their hand then waving me off
A student expressed excitement to the group
No side conversations
No snoozing
Other Observations
Yawning
Chewing Gum
Chewing Fingernails
Playing with Hair
Squinting
Not asking from input from other group members
Students that were anxious
Better students in the program
Confusion
Instructor assisted students raising their hands
Students believing they were done when they were not
A student arguing their point to the group
5. Analyze Evidence of Learning
The team
discusses the results and assesses progress made toward learning goals.
The learning goals:
1. Understand how to use a Computer Aided Estimating System
2. Be able to create an Estimating System using Excel
The Lesson Study group reviewed the notes taken by the observers and analyzed the findings. In general most of the observations were positive. Notes from the first day reflected that the students were engaged in the discussion but showed some apprehension to the project. Evidence of learning from the first class period included comments like: students were answering questions, students were taking notes and students were asking questions. These were constant themes, that were mention in all observersı notes, and the group believed these actions should occur during a lecture.
The evidence from the second class period seemed to be much more reveling as to the students understanding of the lesson goals. The class was divided up into three groups, listed as groups A, B & C to work on the project, and each group approached the assignment differently. But each showed a deep understanding of the lesson goals.
Group A consisting of two members each used a computer. One kept the template given to them open while the other manipulated it. This allowed them the make mistakes without the fear of losing the original file. The members talked back and forth and figured out how to copy and paste formulas for efficiency. This group expressed that they were finished with the project, but when the instructor review their submission they had not completed all the parts. They resumed based on the instructorıs comments and completed the project.
Group B which consisted of three members took turns leading and inputting information into the computer. The original leader showed frustration and handed the keyboard to another member. However the original leader kept involved by using a calculator to double check figures and kept referring to the notes taken during the lecture. The group did complete their project in the time provided.
Group C also consisted of three members and used previous assignments from another class to complete the project. Two members of this group were quite vocal and often debated each other as to what direction to proceed with the project. The third member was often silent and gave very little feedback, and although the group did complete their project in the time provided, it was believed that not all the students in this group achieved the same level of understanding.
The Lesson Study group decided that the lesson goals were achieved. All groups completed a functional estimating systems within the time allotted. Using Bloomıs taxonomy the group believed that by completing the projects the students had reached the application stage. They demonstrated understanding of the lesson by completing the project. We also believe that some of the groups reach the analysis stage because they were able to explain what they were creating to others and connect how this project related to the class as a whole. There was some debate that one group reached synthesis because they were able to integrate materials from a previous class into this project.
6. Repeat
The group
revises the lesson, repeating steps 2-5 as necessary, and shares findings.
This
was the first time the lesson plan was executed and as suggested by Cerbin and Kopp
this is the stage were changes should be made before repeating the lesson plan.
After evaluating the evidence of learning several changes were made to the
lesson plan. The project was changed to reflect the equipment students have
used in the graphics labs at their school. It was hoped that this would give
the students a better understanding of estimating because they would have real
life experience with the equipment and the production workflow. The timeline
for the lesson plan was adjusted to try and give the students more time to work
on the project. The biggest adjustment was to shorten the demonstration of the
estimating software and try to give more time to explaining the project and how
to use Excel. The group size was adjusted from three to two, it was thought
that this would allow the students to become more involved and this would also
allow more hands on computer use. Finally a review was added to look at what
was accomplished in the first period and set the stage for the second period
before dismissing them. The same review was added at the start of the second
period.
There
were other suggestions discussed by the group that were not added to the lesson
plan. It was suggested that the Excel formulas use in the project be given to
the students in paper form. At first the group agreed with the paper idea but
decide that the formulas were available to them in a digital form and that they
could retrieve them if needed. The idea of giving the students a simple
assignment to complete between the two class periods was proposed, but the
group could not agree on what content was needed and how much benefit it would
add, so it was decided not to add the assignment for the Stout Lesson Study.
Some
of the teaching lessons learned from the PSU Lesson Study included the bias of
the regular instructor of the course. They knew the students well and had
preconceived ideas about their learning. The students also possessed a bias
about each other in the fact that they knew each other and what their roles
were in the class. This is to be expected in any classroom setting where the
students have spent a majority of the semester together. The instructor should
always be watching for lead in questions from the student and to have the
freedom to redirect the lesson plan from the questions. Along with lead in
questions the instructor should be aware that they could answer student
questions with redirected questions. An example given was if a student would
ask, ³What number goes here?² the response should be ³It depends, what are you
trying to accomplish?²
The Lesson Study is repeated at
the University of Wisconsin-Stout
The Lesson Study learning goals were not adjusted. It was felt that they were still valid. The changes to the lesson plan and timeline were presented and agree upon.
Evidence of Learning was once again gathered through observation and notation. The group members who were observing were instructed to write down what they saw and then note the time of the observation. After the evidence was gathered the members explained and interpreted their notes to the group as a whole. After looking at and analyzing the evidence, the findings were reported.
1. Form a Lesson Study Group
A Lesson
Study group usually consists of several teachers (3-6) who are interested in
working
together to improve their teaching and student learning.
The University of Wisconsin-Stout Lesson Study group was made up of three instructors from the Apparel and Communications Technologies Department at Stout and the instructor of the Estimating course at Pittsburg State University. Two of the instructors had participated in the PSU Lesson Study. The teaching experience of the instructors ranged from five years to thirty plus years. Two of the instructors have worked in the graphic communications industry before teaching. One instructor has twenty plus years experience teaching graphics at the secondary level. One instructor holds an Ed.D in Vocational Education and the others posses Master Degrees. Two are graduates of Stout. The instructor from PSU delivered the lesson plan.
2. Develop Student Learning Goals
The
learning goal is the backbone of a lesson and provides the ³reason² for
teaching and observing it.
The students will:
1. Understand how to use a Computer Aided Estimating System
2. Be able to create an Estimating System using Excel
3. Plan the Research Lesson
Teachers design a lesson to achieve the learning goals, anticipating how students will respond.
Lesson Plan
150 minutes total (2-75 minute classes)
1st Class period
5-10 minutes Introduction of the Instructor
Background
Describe their Cost Estimating course
Explain the Lesson Study (Video)
Game Plan for next two class periods
5-10 minutes Defining Cost Estimating:
What is Cost Estimating?
Review the Estimating formula of BHR's x STP's + Materials + Buyouts
What is the difference between Cost and Price?
What is a Spread Sheet Program?
15-20 minutes demonstrating an estimating software (Power Quote)
20-30 minutes showing basic Excel Software and formulas
Break into groups of two
Let them have the rest of the time to work
Review
2nd class period
10 minutes to review project and answer questions
50 minutes to work in groups
15 minutes to review the good the bad and the ugly of the project.
Work on the project outcomes
4. Gather Evidence of Learning
One team member teaches the lesson while others observe, collecting evidence of student learning.
Evidence of Learning
Students answered when called upon
Advised instructor on Stout Presses
Nods of Agreement
Engaged with project specs.
Laugh at jokes
Some looked up answers in their notes
Answering question on estimating process (same students)
Looked up equipment for the project
Asked instructor questions
Turned on music
Having fun (Loud and Vocal)
Groups working with other groups
Student expressing ³It works!²
Groups helping other groups
Students liked the relation to real life
Makes BHRıs make sense
Engaged in Laptop discussion
Instructor working with groups
One group member explaining to the other what they was doing
Interactive learning
Student frustration
Lessons on what to look for
Yawning and Crossing Arms
Laughing not Writing
No answer to instructors question
Using Hand Lotion
Eyes Closing
Instructor had to push for response
Fidgeting
Aware of being observed (looked at camera) (looked at me)
Whispering
Chewing on Nails
Started working in groups before instructor was finished
Some left early
One group thinking they are done when they are not
Students waiting for instructor to answer a question
Isolation of one group
Students still working on projects after instructor asked for laptops down
Open up their laptops quickly after good, bad, ugly
Losing Data (Save, save, save)
Only one surfing
Understanding that their profit margin was too big
5. Analyze Evidence of Learning
The team discusses the results and assesses progress made toward learning goals.
The learning goals:
1. Understand how to use a Computer Aided Estimating System
2. Be able to create an Estimating System using Excel
The Lesson Study group reviewed the notes taken by the observers and analyzed the findings. There were more negative comments than the PSU Lesson Study. Some of the comments noted seemed to focus on issues with teaching style and not student learning. There also seemed to be more of a realization that the students knew they were being monitored. Notes indicated students looking at the camera or looking back to the regular instructor for guidance. However the notes indicating evidence of learning seemed to reflect the notes from PSU. These comments included students answering questions, students asking questions and students looking up answers in their notes. However there were no comments on students taking notes from the lecture.
As with the Lesson Study at PSU the evidence of learning collected from the second class period show signs of deeper learning. There were eight groups in the Stout Lesson Study the will be listed as groups 1,2,3,4,5,6,7 &8.
Group 1 seemed to excel in the project and was the first group to complete. The members of this group were active in answering questions from the first class period. The members often debated about how to complete the project. One member expressed ³It worked!² in a loud voice. The interesting thing about this group was after they completed they helped other groups complete their projects.
Group 2 actively asked the instructor questions. One member of this group was very active in answering questions from the previous class period. This group seemed to work well together, dividing task to complete the project. They completed their project within the time provided.
Group 3 had a dominant and a passive member. The group members worked independently on their laptop. However, it appeared from the observations that the dominant member appeared to dictate how the project was to work and explained it to the other member. They completed their project within the time provided.
Group 4 isolated themselves from the other groups. They were quite and appeared to work well with each other. The stated that they were complete, but when the instructor reviewed their project they were not done. They went back to work redirected by the instructor. They completed their project within the time provided.
Group 5 appeared to have trouble with the project. One member was absent for the first class period and the other was noted as being disengaged. The instructor sent the majority of the time working with this group. There were notes indicating that this group looked confused and frustrated. This group did not complete the project in the allotted time, but did submit a completed project after class.
Group 6 seemed to work well with each other but it was noted that they did not talk to each other much. They asked the instructor several questions, and while they waited they did not work on the project. This group did not complete the project in the allotted time, but did submit a completed project after class.
Groups 7 and 8 worked as one big group, each group submitted a finished project. They often helped each other out in figuring out the project. This group played music and was heard laughing and joking around. They did ask the instructor a question. Both groups finished the project in the allotted time.
Once again, just like at PSU the Lesson Study group decided that the lesson goals were achieved. All groups completed a functional estimating system. Using Bloomıs taxonomy the group believed that by completing the projects the students had reached the application stage, however, group 5 was boarder line. They demonstrated understanding of the lesson by completing the project. There was clear evidence that some of the groups reach the analysis stage. After completing their project they actively helped other groups complete the assignment. Students also commented at the end of the second class period that this project helped them connect and explain the objectives of the whole course, including Budget Hourly Rates (BHR), Standard Production Times (SPTıs) and cost versus profit.
6. Report
The group revises the lesson, repeating steps 2-5 as necessary,
and shares findings.
After evaluating the evidence of learning
some suggestions were made to modify the lesson plan. There was an issue with
using the laptops and other technology in the classroom, depending on if and
where the Lesson Study is repeated this is a topic that should be addressed at
the beginning of the study. It was also suggested to add some hands on
activities to the first class period, because as the lecture went on the notes
reflected disengagement. It was also suggest using one of the students to
demonstrate the estimating software or to get each student a copy of the
software.
The changes made to the lesson plan as
suggested at PSU seemed to be successful. Changing the equipment to reflect the
labs received a positive response from the students. The timeline adjustment
were positive, the majority of the students reported that the time allowed was
adequate. The group size adjustment to two appeared to be a benefit.
Some of the teaching lessons learned from the Stout Lesson Study included the difference in having students work on laptops in groups of two versus desktops in groups of three. It was felt that that the small groups and flexibility of laptops at Stout did help in delivery of the lesson plan, however it was noted that the students at PSU all completed the project successfully on time as well. The instructor from PSU noted the change in the students attention when their laptops were closed. Having established groups that have worked with each other throughout the semester also seemed to be an advantage.
Conclusions
-How successful would a Lesson Study be that was conducted by instructors from like disciplines at different universities?
The feedback received suggested that the Lesson Study was very beneficial to the instructors that participated in the study, especially the two that participated at both sites. These two instructors felt that the Lesson Study created a sense of collegiality within the Graphic Communications discipline. It allowed the primary instructor of the course the opportunity to become an observer of their own classroom. It became an eye opening experience to look for evidence of learning from the back of the room. Often we find our focus to be on the students that are participating, and there is no good way to take a look at the class as a whole while instructing. It also created a challenge to the instructor that was delivering the lesson plan because they were place in an unfamiliar classroom with a new group of faces. But at the same time they were dealing with a clean slate, not knowing what to expect from the students. A negative aspect of the Lesson Study is that it is a very time consuming process. The group members should plan to volunteer at least five hours toward the project. The instructors of the lesson plan will spend up to twice that amount of time in preparation, organization, and planning. There is also the expense of travel involved, which has limited funding opportunities.
-Would Lesson Study provide an instrument to create collegiality with individuals at other institutions to create conversations about teaching and provide evidence of deeper student learning in technical courses?
Both groups found Lesson Study to be a benefit to a better understanding of teaching and learning. Therefore, I believe that Lesson Study is a research instrument that can be used for both creating conversation about teaching and looking for evidence of deeper learning in students. In step five which is analyzing the evidence of learning, many conversations developed over what is ³evidence². It is obvious that yawning or closing of eyes is evidence of disengagement but how do we know when the student is learning. This discussion alone shows the value of Lesson Study.
This experience has helped me become a better instructor. By taking the time to look for evidence of student learning, I have a better understanding of what signs to look for in the classroom. I can modify my lesson plans to reflect what I have learned and create new activities to challenge my student to reach for a deeper understanding of the subjects that I teach. It is my hope that I will have the opportunity to continue this Lesson Study at other campuses.
References
Buckwalter, W. (2002). Lesson Study brings rewards and challenges. Currents, V(2), 3-4.
Cerbin, B. & Kopp, B. (2004a). The Lesson Study for college teachers online guide.
Retrieved November 24, 2004, from http://www.uwlax.edu/sotl/lsp/intro.htm
Cerbin, B. & Kopp, B. (2004b). Japanese Lesson Study in the college classroom.
Paper presented at SoTL Spring Conference, Madison, WI.
Lewis, C. (2002). Lesson Study: A handbook of teacher-led instructional change.
Philadelphia, PA: Research for Better Schools, Inc.
Richardson, J. (2004,
February/March). Teachers learn how to improve instruction. Tools
for Schools. Retrieved November 24, 2004, from http://www.nsdc.org/library/
publications/tools/tools2-04rich.cfm
Richardson, J. (2001, December/January). Japanese method benefits all teachers. Results.
Retrieved November 24, 2004, from
http://www.nsdc.org/library/publications/results/res12-00rich.cfme