IPIC (Intercultural, Pragmatic, and Interactional Competence Assessment)
This project aims to develop, pilot, and validate a digital simulation instrument to help educators assess second language learners’ pragmatic, interactional, and intercultural competence. Through scenarios with varying degrees of social and individual factors (e.g., gender, social distance, power), the assessment tool hopes to reflect the individualized, multilingual interactions learners are likely to engage in real life. The IPIC project is a partnership with the Center for Applied Second Language Studies (CASLS) at the University of Oregon. Read our research brief for an accessible overview of current IPIC research.
Study of high-impact student learning outcomes assessment factors
Dr. Subtirelu will conduct research to investigate national and local efforts to promote the Seal of Biliteracy; the impact of the Seal on enrollment in and articulation to foreign language courses; and the degree to which the Seal is accessible to groups whose existing language abilities often go unrecognized or who have not been adequately represented in foreign language education. This research will lead to the development of materials for ensuring that the Seal of Biliteracy is implemented in a way that expands opportunities for language study, promotes rigorous assessment of language skills, and effectively highlights the value that foreign language learners and heritage language users can offer to employers and higher education institutions. Read our research brief for an accessible overview of current Seal of Biliteracy research.
AAPPL (Assessment of Performance toward Proficiency in Languages)
Georgetown and ACTFL will work to identify the most critical languages to add to ACTFL’s AAPPL assessment via census data and surveys of the field. This project will support future efforts to help students meet the Seal of Biliteracy. Read our research brief for an accessible overview of current Seal of Biliteracy research.
Developing a Turkish C-test for Research Purposes
Şeyma Toker, Ayşenur Sağdıç, Derek Reagan, Meg Malone
This project aims to expand and validate an existing short-cut proficiency measure of Turkish. The existing texts will be reviewed for their alignment with ACTFL levels, and domain experts, Turkish native speakers, instructors, and learners of Turkish will be included in the development stages for feedback and quality control. Think-aloud data will be collected to inform native and learner processes of linguistic items. The results of the C-test will be further correlated with learners’ performance on ACTFL tests. Five optimal texts of high reliability, good separation indices and fit statistics will be selected to inform future L2 research.
Assessment for Language Instructors: The Basics
Investigators: Meg Montee, Melissa Cheston
Through the AELRC, CAL offers a free online training course that teaches world language instructors about the fundamentals of assessment. The asynchronous course provides world language instructors with an understanding of the fundamentals of assessment by presenting specific examples of how each applies to real classroom assessment situations. The course lasts five weeks and includes five units that require two hours each. Units focus on: (1) an introduction to assessment, (2) validity, (3) reliability, (4) practicality, and (5) impact. Recruitment for courses during the 2018-2022 funding cycle will include targeted outreach to educators working in community colleges and minority serving institutions.
Workshop on Language Assessment and Understanding: Application and Use
Investigators: Meg Montee (CAL), Meg Malone
This hands-on assessment workshop covers key concepts in formative, classroom-based language assessment with a focus on how educators can integrate language assessment with daily instruction. The workshop will provide educators with a foundational understanding of language assessment as well as principles and strategies for implementing assessment. Participants will come away with an understanding of practical ways to use formative assessment to better understand students’ language proficiency and their language development.
STARTALK: Understanding the ACTFL Guidelines
Investigators: Meg Malone, Amy Kim, Yiran Xu, Ayşenur Sağdıç, Margaret Borowczyk, Rima Elabdali
The goal of this project, now funded for its third year, is to work with instructors of critical languages in community colleges and higher education to take part in a professional development workshop on how to incorporate the ACTFL Guidelines into teaching and assessment. The project promotes foreign language educators’ understanding of ACTFL proficiency levels, which in turn can improve curricular design and implementation.
East Coast Organization of Language Testers (ECOLT) annual conference
Investigators: AELRC, ILR
The AELRC supports this conference, initiated in 2002, to provide a forum for the dissemination of high-quality, peer-reviewed research on language assessment to national educators, scholars, students, and other professionals. Pre- and post-conference workshops will be offered to enhance language teacher assessment literacy. Approximately 140 language testers participate annually, and post-conference evaluations reveal high satisfaction with the conference.
Summer Assessment Institutes
Investigators: AELRC, CASLS, local education agencies, community colleges
The AELRC conducts summer institutes on the basics of language assessment, rating oral proficiency assessments, and developing assessment tasks. Two upcoming institutes will focus on the IPIC and its uses for teaching intercultural communication in the classroom. The AELRC and the Center for Applied Second Language Study (CASLS) will each support a summer institute on assessing intercultural competence. Institutes are designed to be applicable to key constituencies such as K-12 teachers and community college and heritage language program instructors and to date have served over 100 such language teachers and instructors.
Consortium on Useful Assessment in Languages and Humanities Education (CUALHE)
This annual conference continues as a vehicle to bring foreign language education stakeholders together and create opportunities for disciplinary dialogue on topics related to student learning outcomes assessment. Project work will include an annual summit on student learning outcomes assessment in college foreign language programs, with round tables on best practices, paper/poster sessions, and colloquia addressing student learning outcomes assessment in college foreign language education.
Conducting and assessing the impact of professional development in student learning outcomes assessment
The AELRC will investigate the impact of professional development in student learning outcomes assessment, as provided through ACTFL Oral Proficiency Interview training, on instructors in these contexts, including the apparent effects on language education and what activities might help or hinder the usefulness of student learning outcomes assessment in such contexts. Dr. Ann Aly of ACTFL will schedule four-day OPI workshops and work with Georgetown to conduct follow-on research on the influences of these workshops for teaching and learning, specifically curriculum development and assessment practices. Findings will lead to the development of recommended practices, procedures, and educational materials. Other outcomes will include presentations at the ACTFL annual conference and a publication in a journal such as Foreign Language Annals.
Heritage Teacher Perceptions Article
Investigators: John Chi, Meg Montee
This study investigates language teachers’ perceptions of heritage language learners on a large scale using survey methodology. The study is inclusive of teachers of different grade levels, of different types of classrooms, and different languages. By looking at a wide array and large number of participants (N=325) this study provides robust data to answer the overarching question: How do language teachers perceive their HLLs in the classroom? Using both qualitative and quantitative data from the survey, the findings will provide insight on teachers’ views of heritage language learners’ dialects, their expectations of the learners, and the assessment needs of the learners, as well as their overall perceptions of the learners. A manuscript is currently in preparation for publication.
Foreign Language Assessment Directory (FLAD)
The AELRC updates and continually maintains this free, online, searchable directory of information about currently available language assessments, hosted by CAL. The FLAD also includes a free, web-based tutorial designed to support educators in selecting an assessment. The existing FLAD resources focus on university audiences; the AELRC is developing new web-based materials for heritage, K-12, and community college educators with information about how to use the FLAD to select assessments that meet their needs.
Expanding the Foreign Language Data Sharing System
Investigator: Bradford Salen
The AELRC is developing a database (N=~1,200) of pre-rated simulated oral proficiency tests in seven languages from a previous study of pre- and post-study abroad students. We will support this effort by working with Georgetown graduate students, internal researchers, and external researchers to further anonymize the data for national research, transcribe it, and make it available for widespread use.
Examining the Differential Difficulty of Languages
Investigators: Meg Malone, Yiran Xu, Charlene Polio (MSU)
To date, claims of differential difficulty have neither been substantiated through empirical research nor fully supported in the SLA literature. This pilot study represents an initial attempt to investigate these claims via both quantitative and qualitative methods, examining the progress of complete beginners in a summer university-level immersion program that included a promise to use the target language (Chinese, Russian or Spanish) exclusively. Participants took standard, large scale listening, reading, and oral interview (OPI) tests after six to seven weeks of study. Interviews with instructors and directors, as well as classroom observations, were conducted to determine the comparability of instruction across languages and how perceptions of language difficulty influenced instruction.
Evaluation of Project-Based Language Learning Efforts
Investigators: AELRC, NFLRC
For the past eight years, the National Foreign Language Resource Center (NFLRC) at the University of Hawai’i has highlighted Project-Based Language Learning (PBLL) in its offerings. AELRC researchers will collaborate with NFLRC colleagues to investigate the efficacy of their current PBLL efforts, identify best practices within their offerings, and provide input on ways to improve. This effort will allow us to update current frameworks, methods, and instruments. We will present findings at the annual ACTFL and local FL conferences, disseminate to community colleges, develop both a research brief and an applied research brief and produce both a short white paper and a publication in a refereed journal.
Survey of K-12 program evaluation needs
The AELRC will conduct a study of the needs of K-12 programs to improve program evaluation. This national study will elicit input from teachers and administrators on their “evaluation literacy” and determine needs for this population going forward. The survey will include a questionnaire about participants’ perceptions of evaluation, as well as the needs for such projects in their schools.