Speaker
Description
This presentation explores what a failed vocabulary test can reveal about research planning in classroom-based studies of incidental vocabulary learning. The original project aimed to measure learners’ gains in vocabulary knowledge after exposure to target items in on-demand instruction. However, weaknesses in test design, limited construct alignment, and inaccurate assumptions about students’ prior knowledge compromised the study’s data. In particular, the notion of “our students’ level” operated as an unexamined assumption that shaped item selection and test construction more strongly than empirical validation.
The presentation argues that this failure was not merely technical but also interpretive. Teacher identity and insider positioning influenced how student ability was imagined, how the test was designed, and how research decisions were justified. By reflecting on these processes, the talk shows how familiarity with a teaching context can both support and distort research planning.
Using this case, the presentation highlights the value of piloting instruments, validating assumptions about participant knowledge, and critically examining the role of researcher identity in study design. It concludes that unsuccessful projects can still offer useful methodological lessons, particularly when they reveal hidden assumptions that influence educational research from the beginning.
ORCID: 0000-0002-4985-3237