Video Self-Assessment for Language Learners

=123) viewed both videos in December and completed identical questionnaires with regard to both videos. The questionnaire sought to elicit students’ (1) satisfaction with their English ability, (2) interest in speaking English, (3) ability to interact in English, (4) enjoyment of communication in English, and (5) confidence in speaking English. Mean scores for all items were higher (all statistically significant) for the December videos. In a similar survey comparing students’ perceptions of improvement during their eight months of study, learners participating in the video treatment (

culture, Little (2011) made a very compelling argument for self-assessment, but offered only one new instrument, the ELP.Other recent articles by scholars in the field either focused on assessment of "autonomy" (Benson, 2010) or, as in an article entitled Managing self-access language learning: Principles and practice, made no mention of "assessment" at all (Gardner & Miller, 2011).It was thus, with scant attention given to self-assessment in language learning, that the authors undertook the present research.
This pilot study aims to explore the possibility of using video assessment for teachers, learning advisors, and students to effectively monitor progress in language learning.For students, self-assessment is viewed as an invaluable way of involving students in the learning and evaluation process, enabling students to become more autonomous and self-directed learners, and giving students the skills to make the most of language learning opportunities (Little, 2005;Ross, 2006).For teachers, learning advisors, and administrators, the videos can similarly be used as tangible products for demonstrating gains.
Video-Stimulated Recall (VSR) has long been used in teacher training and development (Calderhead, 1981;Reitano, 2006).Though a 100% purely objective method of data collection does not exist (Pirie (1996) outlined numerous potential biases), VSR has advantages in being able to record at least some aspects of classroom performance and enables the viewer(s) to revisit this data and reflect upon performance, decisions taken, and emotions felt.Reitano (2006) highlighted some of the limitations of VSR, including embarrassment, a fixation on one's physical appearance, and a firm mindset whereby objective observations are challenging.From the discipline of business management, there have been concerns as to the accuracy of selfassessment, with research suggesting self-assessment is more strongly tied to affective factors than to cognitive ones (Sitzmann, Ely, Brown, & Bauer, 2010).For language learners, however, it is precisely these affective factors that are among the most important (Arnold, 2009), perhaps even more so for the independent language learner (Hurd, 2008).Affective factors aside, Ross (2006) found that through proper training, student self-assessment can be both valid and reliable, and can contribute to greater learning outcomes.Reitano (2006) concludes that VSR "has been shown to be a most effective tool for teachers to reflect on their knowledge in action and to promote professional growth" (p.10).The authors of the current study believed that perhaps the very same tool of video reflection could be used for language learners as well.

Literature Review
MacIntyre, Clément, Dörnyei, and Noels (1998) wrote that the "primary goal of language instruction" (p.545) is to facilitate communicative use of the second language (L2).It is not a stretch to see that communicative use of an L2 necessarily involves a certain amount of autonomy.Littlewood (1999) explains: If we define autonomy in educational terms as involving students' capacity to use their learning independently of teachers, then autonomy would appear to be an incontrovertible goal for learners everywhere, since it is obvious that no students, anywhere, will have their teachers to accompany them throughout life.(p.73) An integral part of autonomous learning, regardless of how one may define the term (for definitions see Little, 2007;Littlewood, 1999), is some measure of autonomous assessment or self-assessment.Little (2005) explains that a learner-centered curriculum is incomplete without self-assessment and shared responsibility.Chen (2008) describes the merits of self-assessment in assisting "students to develop knowledge of standards of good work" (p.238), identifying performance in relation to these standards, and making appropriate choices for further study.
Both Little (2005) and Chen emphasize the role of self-assessment in enabling learners to reflect on their strengths and challenges, and consequently develop as informed learners.Chen (2008) specifies the opportunities for growth in the phrase "learning to assess and assessing to learn" (p.254), whereas Little (2005) describes the process of self-assessment as enabling "learners to turn occasions of target language use into opportunities for further explicit language learning" (p.

322).
In the field of teacher training and development, there have been numerous studies investigating the practice of VSR.Reitano (2006) notes five advantages of VSR that may also apply to the current study: (1) allowing for the reliving of specific episodes in context, (2) enabling both self-reflection and input from others, (3) giving adequate time for reflection, (4) putting the subject in control, and (5) enabling subjects to make explicit what may have previously been understood only implicitly.Though these advantages of VSR are expected to cross over from the realm of teacher training to that of language learning, the authors of the current study were unable to find any research literature focusing upon video use for student selfassessment and learning.

Research Questions
The following research questions were thus proposed: 1. How do students perceive their progress in spoken English after 8 months of formal study?
(Progress, in this study, is understood to encompass not only communicative ability, but also the important affective considerations of interest, enjoyment, and confidence.)

How do the above perceptions compare with those of students who have not been video
recorded?

Participants
The participants were drawn from 11 intact classes of first year students at a Japanese university of sciences and engineering.All participants had two 90-minute periods of English per week for two 15-week semesters.None were English language majors.All participants undertook the same English curriculum taught by the three instructor-researchers.
Two survey measurements were completed, details of which are explained later in this section.For the first survey instrument, a paired samples t-test was used to analyze the two iterations.After removing participants who had been absent for one or more of the video recordings, or for one or more of the video-viewing and questionnaire completion sessions, the number of participants stood at 107.
For the second survey measurement, analyzed via an independent samples t-test, there were 143 participants in the experimental group and 107 participants in the control group (N=250).

Conditions
The participants in the video treatment group were video-recorded twice during the academic year: once at the outset in April and once towards the end of the year in December.
The time of recording was chosen to provide students and their teachers with viewable data from approximately the beginning and end of their first-year university English studies.The aim of the above research questions was to assess students' progress in spoken English (understood to be communicative).The participants were therefore asked to record interactions in pairs such that elements of conversation, including interactive ability and confidence in communicating with a partner, could be evaluated.The April video recording entailed pairs of students asking each other about their identity pages (see Appendix A).The December video recording involved pairs of students speaking about common topics (see Appendix B).Following completion of the second video recording in December, the students watched the two videos in successive classes, immediately completing the same questionnaire after each video, comprising the questions indicated in the next section.
In a third class, the video treatment group participants completed an additional questionnaire after watching the two videos together.The control group responded to the same questionnaire (instrument two) in the absence of any videos.Control group participants experienced the same classes, with the same curriculum taught by the same instructors.The sole difference was the absence of video activities.

Survey Instrument One
The survey instrument, completed twice by the treatment group (N=123), comprised five items translated into Japanese and back-translated for accuracy.The items were chosen and constructed in an effort to assess a robust definition of progress in spoken English comprising both elements of production (general English ability, ability to interact in English) and elements of affect (interest, enjoyment, confidence).Each item was followed by a space to optionally record any comments.The survey instrument was administered online, and the respondents could answer in Japanese or English.The items are shown in Figure 1.

Survey Instrument Two
The survey instrument completed by both the treatment and control groups (N=250) comprised five items translated into Japanese and back-translated to ensure accuracy.The items were constructed to enable comparison between the treatment group, which had recorded, watched, and evaluated their videos; and the control group, which had no video activity.The items are shown in Figure 2.

Results
The descriptive statistics for survey instrument one are presented in Table 1.The average ratings of the participants (N=123) were all slightly higher for the survey administered following viewing of the December video as compared with the survey following viewing of the April video.A paired samples t-test, with values reported in Table 2, indicates that all December gains were statistically significant at the .05level.
2. Looking at this video, I can see that I have an interest in speaking English.
3. Looking at this video, I feel that I have the ability to interact in English.
4. Looking at this video, I can see that I enjoy communicating in English.
5. Looking at this video, I feel that I have confidence in speaking English.
II.Any comments for each of items 1-5.
2. Since beginning university, my interest in speaking English has increased.
3. Since beginning university, my ability to interact in English has improved.
4. Since beginning university, I enjoy communicating in English more.
5. Since beginning university, I have more confidence in speaking English.

Qualitative Data
For the purposes of triangulation, participant comments were solicited in the two administrations of survey instrument one.Participants were encouraged to make comments in either English or Japanese, with a professional translator performing translations of the Japanese comments.The comments section was both optional and open ended.Thus, while definitive conclusions cannot be made about the comments of students who elected to respond, the comments can give a broader indication of student perspectives than by using the Likert-scale data alone.
The comments for each of the five questions on iterations one and two of survey instrument one were first independently coded by the three researchers, discussed, and finally coded together by the team.Comments for items 2 (interest in speaking English) and 4 (enjoy communicating in English) generally fell along the spectrum of interest in the subject matter (clearly evident, clearly lacking, or not explicitly mentioned).Statements such as "I'm not sure that other people can see, but I'm very interested" were categorized as the student demonstrating an interest in English language learning.Conversely, statements such as "I don't like English very much" were clearly indicative of students who lacked interest.Given the open-ended nature of the comments, many, such as "I thought English is difficult" could not be categorized on a scale of interest.One commonly uncategorizable type of comment had to do with smiling.In the two iterations of the survey, there were twenty instances of smile, smily, or smiling recorded, only two of which were coded for participant interest: "When I speak, I smile and look interested" and "I looked that I was having fun, because I was smiling".Other comments such as "I was smiling", "I'm talking with a smile", and "I could talk with smile" would tend to indicate interest in the subject matter, but without greater context could not be coded as such.Smiling, particularly in the Japanese context, could be construed as a sign of embarrassment or discomfiture (Andrade & Williams, 2009).The three comments, "I'm smiling foolishly", "I'm smiling bitterly", and "I don't have smile" might indicate negative feelings, but again, without greater context could not be coded.Comments for items 1 (satisfaction with English ability), 3 (ability to interact in English), and 5 (confidence in speaking English) generally aligned along two different dimensions: a) interest in the subject matter, and b) satisfaction with English ability.
Comments that did not reference interest or satisfaction with English ability were left uncoded.
Table 5 displays the results of the qualitative portion of this study.Bearing in mind that comments were optional and that their coding can be problematic based on limited context, the results are nonetheless promising.In all instances, satisfaction and interest were registered in higher percentages for the December video iteration than for the April iteration, and dissatisfaction and disinterest in equal or lower percentages for the December iteration.
Particularly noteworthy is the large percentage of student responses for the December video indicating interest for items 2 and 4. of the students did clearly express interest in studying English throughout their responses to the five survey items.Often-recorded responses such as "I'm interested in English but I can't really speak English" and "I want to improve my English and gain confidence" underscore this point.

Limitations
As with any research into language learner perceptions, there are a number of limitations inherent.First, there is the concern that students may not be particularly accurate assessors of their own progress.Sitzmann et al. (2010) have called into question whether self-assessments are in fact more of an affective judgment than a cognitive one.Given that the statistically significant results in this study were indeed for affective factors rather than those concerning ability, perhaps this limitation is not as worrying.In future studies, it may be useful to provide thorough training in self-assessment such that students can achieve greater validity and reliability in their own assessments (Ross, 2006).
A second concern is that all students were clearly aware of which video was taken in April and which was taken in December.There thus exists the possibility whereby student expectations led them to assume improvement when, in fact, none may have existed.Having invested eight months of study into their English, it may be difficult for learners not to give themselves higher scores on the second iteration of survey one.On the contrary, however, there is also the possibility that non-English major students who have been required to take English might have been disaffected with the subject matter and may have given themselves equal or lower scores in the second iteration of survey one, regardless of any possible improvement.As there is little possibility of controlling for students' knowledge of which video was taken when, the best way of ensuring valid and reliable self-assessments is, again, through comprehensive self-assessment training.
A third limitation relates to the timing of both iterations of survey one.The recorded videos from April and December were both viewed and evaluated by students in December in successive classes.The proximity of the viewings and the elapsed time from the April recording may have affected students' evaluations.The authors' follow-up research will have students view and evaluate their performances shortly after the recordings.
There are finally practical matters to be considered.The April recordings were made in the classroom (multiple pairs at once) with no external microphone and the audio quality was often problematic.The December recordings were instead made in auxiliary rooms and were much more audible.The next study will see both sets of recordings made in auxiliary rooms for maximum clarity.

Conclusion
The above limitations notwithstanding, this study has gleaned some important and tangible results.Taken together, the qualitative and quantitative data point to a pattern whereby, over the eight month course of study, students appeared to be developing a greater interest in and enjoyment of communicating in English.Slightly increased levels of confidence were also apparent.Particularly noteworthy was that participants in the video treatment group were able to perceive gains in interest, enjoyment, and confidence that the control group participants did not.
What has not been sufficiently demonstrated is a perception of gain in satisfaction with general English ability or in the belief that the student has an increased ability to interact in English.The results for video treatment group participants were not statistically different from those of the control group for those measures.Frequent survey responses such as "I can't communicate well yet" suggest that, although students may not appreciate gains in their ability, they may have made gains in terms of their interest in the subject matter.
There are a number of potential explanations for students being unable to perceive gains in actual ability.Perhaps the most obvious (and the most disconcerting for teachers) is that there was no gain.The authors believe, however, that gains in communicative ability in a short term EFL setting are rather difficult to pinpoint, especially in the absence of any quantifiable measurement.Students, particularly in the absence of training, may be unable to objectively measure their own gains (Sitzmann et al., 2010).A child, by way of analogy, may have experienced vertical growth in an eight-month period.With no chart to assist her, however, that growth may be imperceptible, particularly if the child's friends are growing as well.
What this study has clearly demonstrated is that intermittent video recordings can assist students in identifying gains of interest in, enjoyment of, and confidence with using English.
Particularly in the absence of other concrete measures of demonstrating gain, these video recordings may better enable learners to realize the often elusive progress they are making in their language studies.This progress, even if it is more affective than cognitive, can provide a substantial boost in motivation for students.

Appendices
Appendix A Identity Pages Illustrated by Students Prior to the April Recording

Table 1 .
Descriptive Statistics for Survey Instrument 1.

Table 2 .
Differences in Means for Survey Instrument 1.The descriptive statistics for survey instrument two are presented in Table 3.The experimental video group scored marginally higher on all items regarding improvement.An independent samples t-test, with values reported in Table 4, was performed.Levene's Test for

Table 4 .
Differences in Means for Survey Instrument 2.
Note: *Differences in means were statistically significant at p ˂ 0.5.

Table 5 .
Analysis of Comments.+ andrefer, respectively, to presence and absence of the coding dimension.unc indicates responses that were uncategorizable with regard to the coding dimension. Note: