Study Information

Study Information

Goal of study:

To answer the following research questions:

What instructional techniques engage students?
What is it like to be engaged in learning?

Use of results – immediate

To fulfill the charge in University Priority C:

“Identify best practices of teaching, learning and engagement which are to be shared campus-wide.”

The results will be used by faculty to develop and implement instructional techniques that students identify as being engaging. The end result will be improved student engagement.

Results will be shared with the Teaching and Learning Center in May 2006 and with faculty participating in the Engagement Institute, summer 2006. The purpose is to give faculty the opportunity to translate the study findings into techniques they can incorporate into their instructional practices to improve student engagement.

Use of results – future

Results from the engagement study, along with feedback from the TLC and the 2006 Engagement Institute participants will be used to develop a survey to go to a wider audience about engagement and instructional techniques that engage students. The purpose of the survey is to acquire knowledge about student engagement at UW-Stout that is generalizable to the UW-Stout student population. The survey will be administered fall semester 2007.

Faculty will be asked to implement the identified instructional techniques in Spring 2007 and an evaluation will be conducted to determine if there is an impact on student engagement.

Method

In February 2006, an e-mail was sent to all faculty participants in the 2005 Engagement Institute describing the proposed study of student engagement, and asking them to return a list of students who they would recommend for participation. They were asked to forward names of students who they felt were engaged in learning. From the list of applicants submitted by faculty, 20 students were randomly selected for invitation.

Those students were sent an e-mail from Dan Riordan briefly describing the study. They were then contacted by one of the three interviewers asking them if they would be willing to participate. If they agreed to participate, a time was arranged for an initial interview. Thirteen students, nine female and four male, were interviewed for the study.

Two interviewers were faculty volunteers, and one was a BPA member. All three had some experience in qualitative analysis and interviewing. Training in use of the digital recording equipment and interviewing techniques were provided to the interviewers by a member of BPA.

For the interviews, a semi-structured format was employed, using questions developed by the TLC board (See Appendix A). Interviewers could add questions to the format if they chose to, so long as they did not lead the participant towards a particular response.

Each student was interviewed twice by the same interviewer within two weeks of the initial interview. Interviewers reviewed notes and/or transcripts of the initial interview, and either developed a new set of questions for the second interview, or asked probing questions based on the responses of the first interview. All interviews were recorded and were later transcribed by student workers.

For their participation in the project, each student received a jump drive.

The transcripts were read in their entirety multiple times by one of the investigators, who extracted comments related to engagement from the transcripts. Each comment was recorded and given equal weight (horizonalization). The extracted comments were independently analyzed by 1) the BPA office, 2) a faculty member who was also an interviewer in the project, and 3) a graduate student with research training. Each extracted comment was first evaluated to see if it met the following criteria:

Does it contain a moment of the experience that is a necessary and sufficient constituent for understanding it?
Is it possible to abstract and label it?

Comments that did not meet the above criteria were eliminated.

The remaining comments were organized into themes. The themes were representative of all participants. Common themes were compared from all three people who performed the qualitative analysis (See Appendix B). The BPA office was responsible for the final selection and description of common themes. To be included as a primary or secondary theme, at least five responses were needed. From the common themes, a list of formulated meanings about engagement was developed. This list of statements (See Appendix D) was sent back to all study participants to read and validate (member checking). Eleven of the thirteen participants responded, and none made any suggestions for changing the definitions. The significant statements were refined by the BPA office and the faculty investigator into a narrative description of engagement (See Appendix E).

Other comments extracted from the definition were determined to describe the experience of disengagement. These comments were grouped separately into the same common themes as those used in the engagement definition. Common themes that were found to contain no comments were removed from the analysis (See Appendix F).

Study Design

The study design used for this study is known as a phenomenological study. As defined by Creswell (1998), a phenomenological study “describes the meaning of the lived experiences for several individuals about a concept or the phenomenon.” Creswell also notes that a phenomenological study:

Determines what an experience means for the persons who have had the experience and are able to provide a comprehensive description of it.
Derives general or universal meanings, or in other words;
Defines the essences of the structure of the experience.

Analysis of Phenomenological Data

Essential to the analysis of phenomenological data, according to Moustakas (1994) is the process of Epoche, a Greek word meaning to refrain from judgment, to abstain from or stay away from the everyday, ordinary way of perceiving things…Epoche requires a new way of looking at things…a preparation for deriving new knowledge but also as an experience in itself, a process of setting aside predilections, prejudices, predispositions, and allowing things, events, and people to enter anew into consciousness, and to look and see them again, as if for the first time…(this is) a difficult task and requires that we allow a phenomenon or experience to be just what it is and to come to know it as it presents itself.

The next step in the analysis of phenomenological data is phenomenological reduction.

Phenomenological reduction, according to Moustakas, is “not only a way of seeing, but a way of listening with a conscious and deliberate intention of opening ourselves to phenomena as phenomena, in their own right”. It contains the following elements:

Horizonalizing
Clustering the horizons into themes
Organizing the horizons and themes into a coherent textural description

In horizonalizing, each statement is given the same weight as it is extracted from the transcript. No judgment is made on the value of the statement. Next, overlapping and/or repetitive statements are removed, and clusters/themes are developed. From the themes, an exhaustive narrative description is developed, and from that, a coherent textural description will emerge.

Results

A qualitative analysis was performed on the transcriptions of the thirteen initial student interviews and the follow-up interviews. To measure inter-rater reliability of the analysis, independent analyses of the data were conducted by three people with experience in working with qualitative data. Common themes were identified by all three.

Every attempt was made to maintain the integrity of the responses of the participants as responses were extracted and clustered by themes. At least five responses were necessary for a theme to be identified. Occasionally it may seem that a response could be included in multiple categories. Some responses addressed multiple issues, and a choice was made by the investigator to include the response with one theme vs. another, based on the context in which the statement was made. Additionally, with responses related to engagement vs. disengagement, a choice was again made by the investigator about the context in which the statement was made.

Themes

Six primary themes were identified. They are:

§ Relationships

§ Empowerment

§ Application

§ Passion of instructor

§ Asking questions

§ Openness to experience

In several cases, secondary themes were also developed from the primary themes.

See Table 1 (below) for a summary of themes.

Engagement Primary and Secondary Themes
Primary Theme	Secondary Theme
Relationships
	Knowing others
	Interaction
	Availability
	Being known
	Attentiveness
	Feedback
	Lecture
	Respect
	Reflection
	Class size
	Trust
Empowerment
	Interest level
	Personal responsibility
	Having choices
	Know more about subject
	Structure

Application
	Relevance
	Understand Content
	Hands-on
	Analyze Content

Passion of Instructor
Ask questions
Openness to Experience

Table 1

Engagement – Primary and secondary themes identified

The first theme related to student engagement in learning is relationships. This theme addresses the relationships that occur during the experience of engagement. Relationships include: the relationship of instructor to student, student to instructor, and relationships between students. These relationships occur inside and outside the classroom, and include group and peer learning. Within the theme of relationships, eleven secondary themes are identified, listed below.

§ Knowing others

o The relationships students have with each other in classes

§ Interaction

o The back-and-forth between students and instructor in class

§ Availability

o How accessible students perceive the instructor to be

§ Being known

o The relationship of instructor to student, and how the student perceives that relationship

§ Attentiveness

o How engagement is affected by paying attention

§ Feedback

o How engagement is affected by feedback from peers and instructors

§ Lecture

o How engagement is affected by the lecture component of class

§ Respect

o How engagement is affected by the respect of student for instructor and instructor for student

§ Reflection

o How students respond to feedback and how they may engage others in the future

§ Class size

o How engagement is affected by class size and the classroom setting

§ Trust

o How engagement is affected by the trust between student and instructor, and between students

Examples of statements made by students regarding the primary theme of relationships in student engagement in learning include:

§ They have kids that are our age or older that help. They’re tutors so they go around the classroom and they, they help you whenever you can and sometimes it’s, sometimes they can explain it better than your professor can; somebody your own age.

§ To have your voice heard or to feel like you are having more of a conversation with the professor versus a lecture.

§ With the teachers that impacted me, I thought it was just unbelievable how they changed my point-of-view for English and math courses and it’s just helped me out tremendously and now I don’t feel overwhelmed when I have to do a paper or I have a math test. I just relax and remember what they told me and it makes it a lot easier on a student when the teacher helps you like that.

§ I think it helps when the instructor knows your name to help you feel more connected with them. It’s hard for some instructors because they have bigger classes, but when its classes within my major like my math class and my computer science class, it’s nice that the teachers can know who you are. It makes you feel that there’s a bond and it’s not just a student and a teacher.

§ An engaged student, I think, would be one that is paying attention.

§ Helps a lot cause you get the feedback from not only the instructor but from your group members as well and you’re all working towards a common goal. You have everyone else’s thoughts and ideas and inputs.

§ I see in professors who have been a part of situations where professors have changed their traditional “chalk and talk” approach and what they have done is that they have made you the facilitator as well of the information.

§ It starts with again, that relationship. And that respect factor. I feel that I’m much more willing to dive head first into a class if I feel that respect from and to the instructor.

§ I think that the reflection helps both the student and the instructor.

§ To me, the smaller the better. If I’m with 10 or 12 of my peers I feel like I can become engaged a lot more, especially in discussion than in large lecture halls.

§ I guess it made me trust people a little more. Because we did have to rely on each other to do well.

The second primary theme identified is empowerment. This theme addresses what happens to engagement in learning when students are active participants in the learning process vs. being passive bodies in their chairs listening and taking notes. Secondary themes identified for empowerment include:

Interest level in the subject being learned

How engagement is affected when students have an interest in the topic

Personal responsibility

How engagement is affected by the work ethic the student brings to class

Having choices

How engagement is affected when students have self-determination for the work they do in class

Know more about the subject

How engagement is affected when students develop an interest in learning more about the subject than what is expected in class

Give students some structure

How engagement is affected with the instructor as “the guide on the side”

Examples of engagement in learning related to the theme of empowerment include:

§ And I can see it in everybody else’s face too because they’re all actively involved and they’re all anxiously waiting to speak their mind and I don’t think there’s a class that goes by that not everybody in class speaks.

§ Where the professor gives you that share in what we’re learning. They almost give you a part of it and say, “This is yours to nurture also and if your fellow student drops their ball, you’re going to lose the game too.”

§ It makes me feel better. You know, it makes me feel that I’m actually doing what I’m supposed to do. I’m supposed to go to college and, just by being involved, it makes you feel like you’re actually doing what you need to get done instead of just sitting there and watching the clock go by, because you want to get out and do something else.

§ It makes me feel grown up too. You’re out on your own and you’re doing your thing and you definitely mature over just a short period of time because you have to.

§ I know that if I don’t go, I’m going to miss out on a lot.

§ Here’s the subject. You get to choose how you want to go about doing this project.

§ It made me very engaged in it in terms of participation and discussions but also beyond that, I wanted to know more.

§ I feel that being engaged in learning means that the teacher is there to direct you.

The third primary theme identified is application. This theme addresses what happens to engagement in learning when students see “the big picture”. Secondary themes identified include:

Relevance

What happens to engagement when students see how courses relate to other courses, to their field of study, to their future job plans.

Understanding the content

What happens to engagement when students “get it”.

Hands-on

Part of application, but students used the words “hands-on” to describe the experience.

Analyzing the content

What happens to engagement when students make the leap from hearing information to analyzing what they hear.

Examples of engagement in learning related to the primary theme of application include:

§ He related it towards us, towards the student. It put things into perspective for me more than sitting in a class and saying, “When the heck am I ever going to do this again?” And so for me, that was a class that I could really get engaged in and focus on.

§ Where I understand what the teacher is talking about and I’m remembering it to where I can use it later on and describe it to my friends or my parents and how they also can see how passionate I am about learning this or how interested I was and how now I feel like I’m a little bit smarter.

§ I also think that the engagement is hands-on, minds-on basically, where you’re given an opportunity to actually practice what you are taught.

§ Make a lot of class discussions. And if kids start to disagree with each other let them go, I think. We can see both sides of what they’re saying and it opens our eyes up as well.

The fourth primary theme identified is the passion of the instructor. This theme addresses what happens to student engagement when students perceive the engagement of their instructor. There are no secondary themes identified with passion of the instructor. Some examples of student engagement in learning related to the passion of the instructor include:

You can just tell when a professor really likes what they do...My psychology professor, he comes to class each day with a positive attitude and even though he may be having a very bad day, he still leaves it at the door. He’s still like “How are you guys doing today, we are going to talk about this.“ And he makes it...you can just tell by the tone in his voice that he’s happy to be there, he’s happy to be teaching what he is teaching. It makes us feel he’s really positive about this. He really wants us to get it.

A lot of the times a professor can make you want to jump in and do stuff and actually have that motivation to get it done because one of your professors puts you in a better mood.

The one thing that makes being passionate so complex is that there is so much that goes along with being passionate about what you’re teaching.

They want to see you succeed and develop as a student and as a person and as a professional. I mean that’s passion about what they’re doing. It’s not what you’re teaching. It’s because a teacher really just helps you grow.

I enjoy the class more because of how this teacher has taught it.

The fifth primary theme identified is asking questions. This theme addresses how student engagement in learning is affected when students ask questions inside and outside of class. There are no secondary themes identified with asking questions. Some examples of student engagement in learning related to the primary theme of asking questions include:

If your students get you off track by asking you questions, let them get off track because they are still learning because they are asking you the questions.

That time period between asking a question and allowing a student to answer is very important because that student needs time to reflect on the question and form an answer.

And I will admit to my students the first day, I don’t know everything. You may ask me a question and I will flat out tell you, “I don’t know. But I can certainly look for the answer as much as you can or I can direct you to find the right answer.”

When you are more engaged you are more likely to ask questions. If I’m in an environment where I feel comfortable asking a teacher questions it’s easier for me.

That made all the students around you feel “I can ask anything and I’m not going to feel like ‘oh, that was a stupid question’” or whatnot.

The sixth and final primary theme identified in the analysis is openness to experience. This theme addresses what happens to student engagement when students perceive that they and their instructors are able to “go with the flow” of whatever occurs in class. There are no secondary themes identified with openness to experience. Examples of student engagement in learning related to the primary theme of openness to experience include:

It’s openness. That’s very important. Openness… an environment has to be set up as such where the players involved feel that their voice matters or that their voice is heard

You don’t want to shut down persons in terms of, “Oh, I don’t want to hear that,” and so there’s an openness and respect for the environment that comes when people feel respected.

A lot of kids are too afraid to bring up their ideas; they’re too modest. And we do have some great minds that never grow because they’re so afraid to speak out.

“We’re studying a field that’s unproven. I don’t have all the answers; you don’t have all the answers. No one has all the answers.”

(All comments by theme can be found in Appendix B.)

Further analysis

When the responses were all clustered into themes, a list of formulated meanings about student engagement in learning was developed (Appendix D). Formulated meanings, according to Creswell, are designed to get at the meaning of the students’ statements in their original context.

Formulated meanings about engagement were created from all statements made by students, and were sent to all participants for their feedback. From the list of formulated meanings, a narrative description of engagement was created, meant to convey the essential structure of student engagement. The description is as follows:

The student’s perception of engagement in learning occurs when relationships develop inside and outside the classroom in which they depict themselves as: a peer, a colleague, an apprentice, and a friend. The relationships are open, caring, non-judgmental and respectful. They allow for choices, and provide authentic listening to all voices. The content becomes meaningful, interesting, and the application to future work becomes apparent. Learning, whenever possible, is experiential.

When instructors are perceived by the student to be passionate and positive about what they do, their students and their students’ learning, engagement in learning is significantly enhanced. Instructors are available, accessible, and take time for their students.

Appendix A

Interview questions for Engagement Study

First interview

What does being engaged in learning mean to you?

What is your definition of an engaged student?

How do you know when you are engaged in learning?

Describe a situation in which you felt very engaged in learning. What occurred in that situation that led to you feeling engaged?

Anything specific about instructional practices?
Anything specific about the classroom environment?
Anything specific about interactions with other students/instructors?
Anything specific about your attitudes?

Describe a second situation in which you felt very engaged in learning.

How was this situation similar to the last one you described?
How was this situation different from the last one you described?

In these situations where you felt very engaged in learning, what was the impact on your learning? Your interest in the course?

Provide specific examples

You’ve mentioned that when your instructors do XXXXXX (mention some of the things the student mentioned earlier in the interview), you feel very engaged in learning. What other instructional techniques have your instructors used that helped you to feel very engaged in learning?

What do you do to engage yourself in learning?

What are student techniques for increasing engagement in learning?

Thinking ahead to the future, what instructional techniques do you wish your instructors would use that would help you to be more engaged in learning?

Any additional questions related to teaching, learning and engagement could be added by the interviewer, so long as they are not leading. We want the students to define for us what engagement is, what learning is, etc… We do not want to show them a definition of engagement, or propose our own instructional techniques that we think are engaging, etc…

Second interview

Questions asked in the second interview should be follow-ups to things the students spoke about in the first interview. What wasn’t clear? What do you want them to explain in more detail? Were there any specific questions that they didn’t really answer? What new questions have arisen now that you’ve had the opportunity to think back on the interview?