Heritage Language Evaluation & Assessment


Annotated Bibliography: Overview of Evaluation and Assessment in Heritage Language Learning
Young A Son

The following annotated bibliography presents a synopsis of studies, from as early as 1989 to the present, which address different issues related to heritage language assessment and program evaluation. It is composed of 8 parts: (a) Reviews; (b) Surveys; (c) Assessment (placement and other tests); (d) Assessment for the purpose of differentiating HLLs and non-HLLs; (e) ACTFL Proficiency guidelines for assessing HLLs; (f) Evaluation of HL programs; (g) Sociocultural aspects (identity, attitudes, and motivation); and (h) Key background and theoretical papers on HLLs. Throughout this annotated bibliography Heritage Language Learners will be referred as HLLs and Foreign Language Learners as FLLs.

Please cite the use of the bibliography as follows: Son, Young A (2015). Annotated bibliography: Overview of evaluation and assessment in heritage language learning. Washington, DC: Assessment and Evaluation Language Resource Center.

REVIEWS

The entries in this section take the form of broad HLL research overviews and reviews of HLL assessment and evaluation. Most of the reviews examine the main points of concern in research on heritage language learners and offer solutions to these issues.

Campbell, R., & Christian, D. (2003). Directions in research: Intergenerational transmission of heritage languages. Heritage Language Journal, 1(1), 1-44.

Keywords: Second National Conference on Heritage Languages in America; summary; HL maintenance; HLLs’ identity, motivation, language use.

This article summarizes the topics discussed at the Second National Conference on Heritage Languages in America. It is composed of brief summaries of the presentations by eleven researchers: Terrence Wiley discusses HL language maintenance; Lily Wong Fillmore talks about external and internal factors to stop speaking the HL; Reynaldo Macías explain the role of school and policy in HL maintenance; more relevant to this annotated bibliography is Kiyomi Chinen and G. Richard Tucker describe the role of Saturday-schools and offer suggestions for which tools to use in order to assess HL learners’ identity, motivation, development, vitality, and language use; Sarah Shin discusses the role of parents in HL maintenance; Christine Sims explains the role of policy in Native American Language maintenance; Norma González describes the role children’s language ideologies in HL maintenance; Guadalupe Valdés presents the view on language ideologies in Spanish Departments, as well as monolingualism and bilingualism ideologies; Joseph Lo Bianco talks about minority language policies discourse; and finally Nancy Hornberger discusses language policy of minority languages.

 

Kagan, O. (2005). In support of a proficiency-based definition of heritage language learners: The case of Russian. International Journal of Bilingual Education And Bilingualism, 8(2-3), 213-221.

Keywords: HLL placement; HLL profiles                                                 

This review study discusses some of the previous literature on HLLs and offers suggestions on how to profile (and assess) this learner group for purposes of instruction and placement. It focuses on Russian HLLs. According to Kagan, Russian HLLs can be classified into 3 groups based on their biographical data: (1) those who graduated from high-school in a Russian speaking country and have a native-level grammar, vocabulary, and knowledge of culture; (2) those who had 5-7 years of formal education with a fairly complete competence on grammar but not as much in vocabulary and are unfamiliar with sociocultural conventions; (3) the ‘incomplete acquirers’ or ‘forgetters’—those who emigrated after attending elementary school or pre-school or were born in the US. Overall, Kagan suggests that placement could be done on the basis of HLLs background or profiles as well as their proficiency. In terms of instruction, it is recommended to follow a macro-approach (i.e., global or top-down), as opposed to micro-approach (i.e., separate elements of the language with a gradual increase in complexity), building on initial abilities in speaking and listening. Finally, Kagan also suggests considering HLLs’ motivation in the design of curriculum and teaching materials.

 

Kagan, O., & Dillon, K. (2008). Issues in heritage language learning in the United States. In N. Van Deusen-Scholl and N. H. Hornberger (Eds.), Encyclopedia of language and education 2nd Edition, Volume 4: Second and Foreign Language Education (pp. 143-156). New York: Springer Science + Business Media LLC.

Keywords: HLL classification; macro-approach; micro-approach; HLL research agenda

After a comprehensive review of the literature on the definition of HLLs and the history of research on HLLs, the researchers offer an overview of the main issues concerning research and instruction of HLLs, such as profiling individuals, differences between FLLs and HLLs language skills, and challenges in the research on HLLs. First they provide a description of how HLLs can be classified based on Valdés’ (2001) classification according to time of arrival and Kagan’s (2005) grouping. Similar to Kagan (2005), they propose a four-group categorization, separating Kagan’s (2005) original group 3 (see above) into two separate groups—those emigrating after elementary school and those emigrating after pre-school or being born in the US. Furthermore, the researchers compare HLLs’ and non-HLLs’ pedagogical needs, suggesting a macro-approach (i.e., global or top-down) rather than a micro-approach (i.e., individual elements or bottom-down). Finally, some of the challenges are raised in HLLs research and suggestions for its future research agenda are provided. Main foci include (a) HLLs’ placement and assessment studies; (b) study abroad studies focusing on HLLs; (c) the cultural, historical, and linguistic contexts that define HLLs; (d) the reasons for some HLLs to retain and others not to retain the HL; and (e) the conditions that cause the shift from speaking the HL to only speaking the host language.

 

Kondo-Brown, K. (2003). Heritage language instruction for post-secondary students from immigrant backgrounds. Heritage Language Journal, 1(1), 1-25.

Keywords: HLL early research; gaps in the HLL literature

As the lead article in the very first issue of the Heritage Language Journal, this article carries out the task of summarizing early research on HLLs and raising awareness on some of the most important gaps in the HLLs literature. Kondo-Brown states that there is lack of research in the evaluation of HL programs. She asserts that little attention is given to the HLL education in terms of policies in support of HL maintenance. Furthermore, she discusses the lack of empirical research on the differences between HLLs and FLLs. Although previous research has elucidated some of the linguistic differences between both learner groups, the researcher asserts that there is very little empirical evidence. In addition, the article brings forward the issue of the appropriateness of using ACTFL proficiency guidelines to assess HLLs, previously raised by Valdés, and the lack of empirical research investigating this issue. As the guidelines have been created to assess FLLs and not HLLs, Valdés questions their appropriateness in discriminating HLL proficiency levels. Kondo-Brown underscores the need for more empirical research that closely examines this problem. Another understudied area is effectiveness of formal instruction on registers and vocabulary for HLLs and the relationship between socio-psychological factors, such as motivation, and language proficiency development.

 

Kondo-Brown, K. (2010). Curriculum development for advancing heritage language competence: Recent research, current practices, and a future agenda. Annual Review of Applied Linguistics, 30, 24-41.

Keywords: HLL curriculum; issues with HLL research

Kondo-Brown reviews theoretical and practical studies on HL curriculum development from numerous language groups. In addition, she offers suggestions for the future research agenda on HLLs. In regards to curriculum development, the researcher proposes three frameworks: (a) learner-centered, (b) content-based, and (c) outcome-based. She underscores the need for more research on these frameworks, particularly the last two, indicating that very little empirical research has been done on content-based instruction and assessment tools to measure HLLs’ proficiency. The review also offers an overview of issues concerning pre-college- and college-level HL research. Some of the issues discussed include two-way immersion programs for the purpose of HL maintenance, placement tests for HLLs, formal instruction for HLLs with unbalanced development of literacy skills, individual differences between FLLs and HLLs, and best vocabulary instruction practices for HLLs.

 

Malone, M., Peyton, J. K., & Kim, K. (2014). Assessment of heritage language learners: Issues and directions. In T. G. Wiley, J. K. Peyton, D. Christian, S. C. Moore, & N. Liu (Eds.), Handbook of heritage, community, and Native American languages in the United States (pp. 349-358). New York: Routledge.

Keywords: HLL assessment; five absences; recommendations

Unlike previous reviews of HL literature, this recent article provides a comprehensive summary of the literature focused mostly on HLLs assessment. It is organized in terms of what the authors call the “five absences,” which represent major gaps in the HLLs literature. These absences include: (1) the absence of research on HL proficiency standards; (2) the lack of consensus in the definitions of HL and HL proficiency; (3) the limited understanding of language variation; (4) the lack of assessment for measuring HLLs’ linguistic skills; and (5) the dearth of research on HLLs assessment, particularly oriented to less commonly taught languages. In addition, the researchers conclude with recommendations for future research on HLLs such as: (1) more research focused on less commonly taught language in order to be able to generalize more findings; (2) research on integrated methods of assessing HLLs as opposed to tests that only measure grammatical knowledge and speaking ability; (3) development of an assessment framework for HLLs that will define standards for HL proficiency and help create valid measurements; and finally (4) further research on HLLs assessment processes and design of tests to investigate the usefulness of the already existing tests for HLLs.

 

Peyton, J. K. (2008). Spanish for native speakers education: The state of the field. In D. Brinton, O. Kagan, & S. Bauckus (Eds.), Heritage language education: A new field emerging (pp. 243-256). New York: Routledge.

Keywords: Spanish HLLs; National Endowment for the Humanities (NEH) institute; HLL education; HLL programs; HLL assessment

This article summarizes the discussions between Spanish HLL instructors during a meeting in summer 1999 at the National Endowment for the Humanities (NEH) institute. In this meeting, teachers of Spanish for native speakers in K-12 programs met to discuss issues related to Spanish HL education and exchange ideas about best practices. The paper is divided into six sections: (a) benefits and challenges of HL education, (b) characteristic of HLLs, (c) HL teacher qualification and training, (d) HL programs and instruction, (e) HL assessment, and (f) HL education policy. Each section includes a list of recommendations on how to address some of the main issues in HL education. In the first section, Peyton describes some of the main benefits of HL education, which include literacy development in two languages and more business and professional opportunities among others. Peyton moves on to describe some of the most common characteristics of Spanish HLLs in terms of their proficiency in Spanish and English as well as their socio-economic background. Next, she discusses the state of teacher training and qualification as well as the HL programs at the time. Peyton underscores the need to develop more pre- and in-service training courses, setting up standards for both teacher qualification and HL program in general, familiarizing staff with HL instruction, materials, and curricula. Most relevant to this annotated bibliography is the assessment section, where Peyton describes the status of HL testing and program evaluation at the time. She describes the most common types of test uses, namely, diagnostic, placement, and achievement tests. She emphasizes the need to establish more standards and more uniformity among these tests. In terms of evaluating HL programs, Peyton argues that there are no systematic and rigorous evaluations of HL programs in K-12. Some of the recommendations to address this issue include: (a) having staff that is knowledgeable HLLs assessment; (b) developing Spanish HL achievement tests; and (c) developing guidelines for assessment and program evaluation. Finally, in regards to HL education policies, Peyton suggests developing policies that support separate instruction of foreign languages for HLLs and dissemination research that could inform policy makers and educators.

 

Sekerina, I. (2013). A Psychometric Approach to Heritage Language Studies. Heritage Language Journal 10(2), 203-210.

Keywords: assessment tools; HLL development; HLL standards

This article discusses the need for assessment tools that can identify the weaknesses and strengths of HLLs through their HL development and process of reaching ultimate attainment. Sekerina indicates that the lack of standards that are unique for HLLs is the main problem of HL research. The article begins by offering a brief overview of some of the most important SLA research on ultimate attainment and the differences between HLLs and FLLs in reaching higher levels of proficiency. She describes that unlike L2 learners, for HLLs age of arrival is more a critical factor than age of acquisition in predicting ultimate attainment. Next, she mentions a few standardized tests that are being used to assess HLL proficiency, including the Bilingual English Spanish Assessment test, the Russian language proficiency test for multilingual children which is part of the Bi-SLI, the Russian Language Assessment (RuLA), and the Bilingual Russian Language Assessment (BiRuLA). She only describes the latter, explaining that it is composed of 24 tests which belong to three different categories, namely, production, naming, and comprehension. The tests measure pronunciation, morphology, syntax, and vocabulary, among other skills. Finally, she also refers to different assessment tools that measure HLLs cognitive ability, such as, working memory tests that measure reading span, eye movement, and processing speed. Sekerina concludes the article calling for research that uses more sophisticated psychometric measures other than ANOVAs and t-tests.

 

Survey Studies

The following entries describe studies that have used surveys in order to collect general information about HLL programs at the university level as well as HLL programs in community language schools.

 

Beaudrie, S. M. (2011). Spanish heritage language programs: A snapshot of current programs in the southwestern United States. Foreign Language Annals, 44(2), 321-337.

Keywords: Spanish HLL program; Southwest US universities; enrollment; course description; program limitations

This is a survey study on Spanish HL (Spanish Heritage Language) programs in universities throughout the southwest of the US. The data were collected in two phases: (1) extensive website research and (2) email surveys targeted to universities with and without HL programs. The findings for the study are based on 173 universities with at least 5% of Hispanic student enrollment. Overall, the results indicate that there is a large variation in terms of the number of universities with a SHL programs and the number of courses offered. On average, 38% of the surveyed universities offer SHL programs composed mostly of 1 or 2 courses. Furthermore, the likelihood of there being a SHL program increases with the size of Hispanic population. However, there were still universities with large Hispanic population without a SHL program. The 48 universities without an HL program stated they have lack of funding, low enrollment, staffing problems, and they wish to avoid discrimination. Moreover, a thorough analysis of the HL course descriptions shows that they are mostly targeting only SHL learners with high oral skills and poor formal instruction. Most of the instructional goals of these courses include writing, grammar, speaking, spelling, vocabulary, and culture.

 

Carreira, M., & Kagan, O. (2011). The results of the national heritage language survey: Implications for teaching, curriculum design, and professional development. Foreign Language Annals, 44(1), 40-64.

Keywords: national survey; university-level HLL programs; less-commonly taught languages; Community-Based Instruction (CBI)

Carreira and Kagan summarize the findings of a national survey carried out to examine several HL programs from different universities across various regions in the US. It includes information from 22 different languages programs and 1732 students. This is one of the very few survey studies that focus on less commonly taught languages, including Russian, Korean, Vietnamese, Tagalog, and Persian. The findings are organized in terms of language usage and proficiency, HLLs attitudes and goals towards learning their HL, profiling HLLs (individual differences) according to their biographical data—language spoken at home, place of birth, and age on arrival. Finally, the researchers also provide pedagogical implications and propose Community-Based Instruction (CBI) (i.e., an approach where learners are taught skills needed to function within heritage language communities), as an option to designing HL curricula.

 

Ingold, C. W., Rivers, W., Tesser, C. C., & Ashby, E. (2002). Report on the NFLC/AATSP survey of Spanish language programs for native speakers. Hispania, 85(2), 324-329.

Keywords: Spanish HLL programs; Spanish for Native Speakers; HLL program challenges

This early survey study reports on 240 randomly selected Spanish programs in higher education across the US. With a 60% response rate, the results were based on 146 Spanish HL programs—referred to in the survey as Spanish for Native Speakers (SNS). Findings indicated that out of the 146 Spanish programs, only 17.8% (N = 26) cited having an SHL program. In addition, 93% of the institutions without a HL program stated insufficient enrollment, funds, and staff as reasons for not having a program. The institutions with a SHL program reported problems with lack of information on HL courses, interest from HLLs, and effective placement procedures.

 

You, B. K., & Liu, N. (2011). Stakeholder views on the roles, challenges, and future prospects of Korean and Chinese heritage language-community language schools in phoenix: A comparative study. Heritage Language Journal, 8(3), 67-92.

Keywords: Korean; Chinese; HLL programs; Community Language schools; challenges

The paper reports on the results of a survey and interview study exploring the roles, major challenges, and prospects of Korean and Chinese HL and Community Language (CL) schools in Arizona. The questionnaire was taken by 124 Koreans (i.e., 99 parents and 25 teachers) and 135 Chinese (i.e., 102 Parents and 33 teachers). Four principals, 20 teachers, and 24 Chinese and Korean parents were also interviewed. You and Liu explain some of the challenges that these schools face, which include: (a) lack of teachers, (b) limited parents’ involvement, (c) lack of teaching materials and resources. When comparing both HL groups, there are differences in the challenges that they face as well as the degree of parents’ involvement as teachers and as motivators. In the Korean community, the role of the church is crucial for HL programs, as they offer financial aid and a meeting place for HLLs. Moreover, unlike the Chinese HLLs, Korean HLLs indicated being greatly motivated through the Korean pop culture. Both Korean and Chinese communities have an optimistic view of the HL and CL schools. The authors also collect micro (i.e., internal factors) and macro (i.e., external factors) rationales of the HL or CL program stakeholders that could predict the future of the programs.

 

Assessment (Placement and Other Tests)

This section presents studies that describe the design and implementation of HLL tests such as placement tests for university-level HLL programs, self-assessment tests, and various formative assessments.

 

Beaudrie, S. M., & Ducar, C. (2012). Language placement and beyond: Guidelines for the design and implementation of a computerized Spanish heritage language exam. Heritage Language Journal, 9(1), 77-99.

Keywords: computerized placement test; Spanish HLLs; demographic survey; recommendations

Beaudrie and Ducar provide a step-by-step process of designing and administering a computerized placement test for Spanish HLLs at the University of Arizona. The test is composed of two parts: (1) demographic survey, which has the purpose of separating HLLs and FLLs; and (2) Spanish HL test, which separates learners into 4 instructional levels. Only those who answer positively to 3 or more of a total of 10 questions in the survey are qualified as HLLs and directed to part 2. The test consists of multiple-choice items assessing tense, morphology, spelling, and vocabulary. In terms of reliability measures, the researchers report Cronbach alpha as a measure of internal consistency for each test from the year 2005 to 2011 with a range of 0.881 to 0.936. Furthermore, Classical Test Theory (CTT) analysis is conducted to calculate item difficulty and identify misfitting items for each of the tests. The authors also discuss some of the challenges they faced when implementing the test, such as enrollment issues, misplacement, student underestimation of their abilities, and the lack of more advanced proficiency courses. Finally the researchers conclude with some recommendations for administration and design of HL placement tests, such as, involving all the stakeholders in the process of designing and administering HL placement test, promoting HL courses to increase enrollment in all levels, and continuing the process of placement test revisions.

 

Burgo, C. (2013). Spanish in Chicago: Writing an online placement exam for Spanish heritage speakers. Borealis–An International Journal of Hispanic Linguistics, 2(1), 199-207.

Keywords: online HLL placement test; Spanish HLLs; methodology pitfalls

After providing a brief summary of the current state of the Hispanic community and HL programs in Chicago, Burgo reviews previous research on HL placement tests and discusses some of the pitfalls of previous methods of placing HLLs into courses (e.g., self-placement tests). Furthermore, the paper provides a description of the author’s experience in collaborating in the design of an online placement test for Spanish HLLs.

The test was composed of three parts which assessed (a) grammar, spelling, and accent marks; (b) vocabulary and registers; and (c) reading and writing. The first part was graded through the computer whereas part 2 and 3 were graded by instructors. The researcher asserts that one of the limitations were the instructor-graded sections (i.e., writing section). Even though Burgo provides some revisions made on the test and shows a few examples of test items, no results or validation measures were reported.

 

Carreira, M. M. (2012). Formative assessment in HL teaching: Purposes, procedures, and practices. Heritage Language Journal, 9(1), 100-120.

Keywords: formative assessments; summative assessment; HLL instruction; assessment tools

In this article, Carreira emphasizes the importance of formative assessments in enhancing the instruction of HLLs. She starts off by reviewing previous studies on formative assessment in general as well as explaining the differences between summative and formative assessment. She then describes a Spanish course for HLLs at California State University, Long Beach and illustrates how they use various assessment tools (e.g., computer-graded exercises, homework exercises, reading journals, exit cards, quizzes, and dictations) to monitor the degree to which instruction and curriculum are meeting HLLs’ learning needs according to their language abilities. With each explanation, Carreira also provides detailed and concrete examples of each assessment activity. Finally, she offers recommendations on how to implement these practices in real-life classrooms.

 

Donovan, A., Di Silvio, F., Malone, M., & de la Torre, J. (2013, April). Assessment of heritage language learners: A pilot study of self-assessment and learner beliefs. Paper presented at the 16th National Conference of Less Commonly Taught Languages (NCOLCTL), Chicago, IL.

Keywords: Mandarin HLLs; self-assessment; Oral Proficiency Interview (OPI); ACTFL Guidelines 

This conference presentation discusses the results of a pilot study investigating Mandarin HLLs’ self-assessment of oral proficiency and their perception of the OPI tasks. For this purpose, three separate instruments are used, namely, (a) a self-assessment based on the ACTFL Proficiency Guidelines (1999), (b) a cognitive laboratory consisting of interviews and surveys about HLLs’ perceptions of OPI tasks and of response samples, and (c) the ACTLF OPI. A total of 10 Chinese HLLs participated in this study. The results indicate that HLLs with higher levels of proficiency are able to adequately self-assess their oral proficiency. On the other hand, the lower level HLLs tend to underestimate their speaking abilities. In addition, the responses to the cognitive lab indicate that many HLLs perceived Novice level tasks to be inadequate, as they believed the questions were too easy. In contrast, they thought the Advanced and Superior level tasks are effective in assessing speaking proficiency. Finally, participants’ views of the sample responses mostly agree with the ratings given by trained raters. HLLs’ rationale for their decisions of the appropriateness of sample responses largely agree with some of the criteria cited in the ACTFL Guidelines. Although still in the piloting stage, this study offers insights into HLLs’ ability to self-assess their proficiency, which is part of an understudied topic within HLL research.

 

Fairclough, M. (2006). Language placement exams for heritage speakers of Spanish: Learning from students' mistakes. Foreign Language Annals, 39(4), 595-604.

Keywords: computerized placement test; Spanish HLLs; dialects; cloze test; essays; recommendations

This study describes the process of designing a computerized placement test targeted for Spanish HLLs at the University of Houston. The researcher first examines placement tests administered over two academic years in order to identify gaps in the HLLs language and the differences in the language variation of HL spoken at home and the standard dialect. The test is composed of two parts: (1) a cloze test, which measured verb morphology; and (2) four short essays, which assessed basic types of discourse. Fairclough provides only one example of an item for Part 1 but explains what type of verb tenses were assessed. Results are reported in terms of practical issues, such as how many students actually took the test, and the effectiveness of the test in separating HLLs into levels. Furthermore, findings show that for the verb morphology section, differences in tense and moods imposed different difficulty levels on test-takers. Examples of the test-takers’ responses in terms of the types of errors are also reported. Finally, Fairclough offers recommendations for those interested in creating a HL placement test.

 

Fairclough, M. (2011). Testing the lexical recognition task with Spanish/English bilinguals in the United States. Language Testing, 28(2), 273-297.

Keywords: lexical recognition test; placement test; Spanish HLLs; Spanish-English bilinguals

This study examines the effectiveness of using a lexical recognition test as a placement test for L2 and HL learners in the University of Houston. A total of 330 participants are assessed, out of which 183 were HLLs and 147 were L2 learners. Each of the learner groups is already divided into four instructional levels. Additionally, a control group composed of 16 graduate students who are Spanish-English bilinguals is included in order to examine the baseline validity of the task. The lexical recognition test consists of 120 words selected from a list of 5000 most frequent Spanish word lexicon, and 80 pseudo-words were also added. Forty-four words (24 real words and 20 non-words) are chosen for each of the 4 levels. As points of comparison, half of the participants take a cloze test, which consists of one paragraph that omitted every fifth word and the other half take a multi-task test, composed of several productive tasks. Test-takers are asked to mark the words for which they could explain the meaning. The ratio between raw hits (i.e., proportion of words recognized) and false alarms (i.e., proportion of non-words marked as recognized) is calculated for the scoring. The results indicate that the test could differentiate L2 learners and HLLs, as well as discriminate different proficiency levels for L2 learners. However, the test is not as efficient for discriminating HLLs with advanced proficiency levels.

 

Fairclough, M. (2012). A working model for assessing Spanish heritage language learners' language proficiency through a placement exam. Heritage Language Journal, 9(1), 121-138.

Keywords: placement test model; Spanish HLLs; program characteristics; recommendations

Fairclough proposes a model for designing and administering a placement test to university-level Spanish HLLs (SHLLs). The article discusses current L2 theory and research on HLLs, in order to describe what and how SHLLs should be assessed. As the target is a placement test, Fairclough provides some of the preliminary programmatic considerations that should be taken before designing the test, which include (a) the mission of the program, (b) the program and learner characteristics, and (c) the course content. In addition, she discusses what language skills should be tested and how. The researcher suggests that receptive, productive, creative skills, and background information about the HLLs should all be assessed. She also provides recommendations of the different criteria that should be taken into consideration when assessing SHLLs’ language proficiency in a placement test. Finally, the article provides useful considerations in terms of the implementation of the test.

 

Fairclough, M., & Ramírez Vera, C. J. (2009). Lexical decision test as a tool for Spanish university students placement. Íkala, Revista De Lenguaje y Cultura, 14(21), 85-99.

Keywords: pilot study; Lexical Recognition Test; placement test; Spanish HLLs; cloze test; proficiency levels

This study appears to be a pilot study for the same Lexical Recognition Test used in Fairclough (2011) (see description above). The goal of the study is to examine the effectiveness of this test as a placement tool. The test was designed for Spanish HLLs and L2 learners from the University of Houston. A total of 106 participants, 51 SHLLs and 55 L2, take the test. A cloze tests consisting of one paragraph with omitted words is used as a point of comparison. The Lexical Recognition Test is designed the same way as described in Fairclough (2011). Similar results are found as in the more recent study, which showed that the test was effective in differentiating between L2 and HLLs. Furthermore, it distinguishes up to 4 different levels for the L2 learner group but it does not prove to do the same for the HLL group. Accordingly, the researchers suggest that considering the first 5000 most frequent words might not be sufficient to differentiate levels within this learner group.

 

Fairclough, M. A., Belpoliti, F., & Bermejo, E. (2010). Developing an electronic placement examination for heritage learners of Spanish: Challenges and payoffs. Hispania, 93(2), 273-291.

Keywords: computerized placement test; Spanish HLLs; steps of design and implementation

This study describes the design and piloting of a computerized placement test for Spanish HLLs at the University of Houston. It discusses some of the challenges the researchers encountered during the process. Overall, the design and implementation of the placement test is carried out in five different steps: (1) listing learning outcomes of each of the Spanish levels; (2) selecting the software; (3) creating a test blueprint according to previous examinations, textbooks, and previous research; (4) piloting the test; and (5) implementing the actual examination. The test is composed of different sections:  (a) questionnaire, which separates HLLs from L2 learners; (b) lexical recognition test used in Fairclough (2009); (c) partial translation, which assesses students knowledge of general forms of Spanish and their difference to expressions commonly found in US Spanish; (d) dictation; (e) grammar, which consists of multiple-choice items of the most problematic structures for Spanish learners; (f) verbs, which asks test-takers to fill in the blank with the correct verb conjugation; (g) oral test, administered through a computer; (h) reading and writing. Examples of the items of all sections except the last two are provided. The test is designed in a way such that only the SHLLs who pass one section can advance to the next. Results are analyzed using ANOVAs, which indicate that there are significant differences between all levels except for the last two (i.e., levels 3 and 4).

 

MacGregor-Mendoza, P. (2012). Spanish as a heritage language assessment: Successes, failures, lessons learned. Heritage Language Journal, 9(1), 1-26.

Keywords: Spanish Placement Test (SPT); appropriateness; Spanish HLLs; FLLs vs. HLLs

Research on HL has provided ample evidence demonstrating that there is a difference between HLLs and L2/FLLs. Nevertheless, in practice, both learner groups are still being instructed and assessed as one single group. MacGregor-Mendoza provides an example of a placement test, the Spanish Placement Test (SPT), which is being utilized for both learner groups. She analyzes the appropriateness of using this test for assessing Spanish HLLs at New Mexico State University (NMSU). She also examines its effectiveness in differentiating HLLs from L2 learners, as well as identifying HLLs language skills and discriminating different proficiency levels in the HLL group. In order to do so, she analyzes the results of 4764 test-takers, estimates item difficulty measures, and examines the content of the items. The analysis of the results demonstrated that the SPT is not appropriate for assessing SHLLs’ language skills or classifying them in different levels. According to MacGregor-Mendoza, the problems with the SPT stem from the fact that it is being used for a different purpose from which it was originally designed (i.e., to assess L2/FLLs). The researcher concludes with recommendations for the improvement of assessment measures for SHLLs.

 

Otheguy, R., & Toro, J. (2000). Tests for Spanish-for-native-speaker classes. In G. Valdés (Ed.), AATSP Professional Development Series Handbook for Teachers K-16, Vol. I: Spanish for Native Speakers (pp. 1–20). New York: Harcourt College Publishers.

Keywords: Spanish HLLs; elementary; middle school; high school; placement tests; research organizations; experts

This short article provides a list of different resources that could be useful for Spanish HLL teachers who need to place their students into different courses. It focuses on Spanish HLLs of elementary, middle school, and high school. This list includes the most recommended placement tests according to the authors, such as, Prueba de ubicación para hispanohablantes, the Leamos program placement test, and the Riverside Performance Assessment series. No specific details are provided about these tests. In addition to placement tests, the article also provides names of organizations who do research on Spanish HLLs (e.g., CAL, CLEAR), names of experts in the field and some of their published articles. Finally, the authors conclude by offering a list of other placement tests with a brief description in terms of age group, the level it can classify, the skills being assessed, and the target learner group (i.e., bilinguals, Spanish HLLs, ESL students). Although useful at the time, the resources may be outdated as the article was published 15 years prior to this annotation.

 

Potowski, K., Parada, M., & Morgan-Short, K. (2012). Developing an online placement exam for Spanish heritage speakers and L2 students. Heritage Language Journal, 9(1), 51-76.

Keywords: online placement test; Spanish HLLs; non-HLLs; adaptive test; colloquial vocabulary

This article reports on the process of designing and piloting an adaptive, online placement test for SHLLs at the University of Illinois at Chicago. Researchers wanted to design a computerized placement test that would reliably separate L2 and HLLs, placing them into the appropriate level courses. For this reason they design an adaptive test where all learners (i.e., HLLs and non-HLLs) start off from the same entry point and depending on their scores advance or not to the subsequent test. In other words, there are five tests in total: (1) Test B, or entry point; (2) Test A for those who have very low scores; (3) Test C, which have the goal of separating HLLs from FLLs and is taken by those who pass Test B; (4) Test D for L2 learners; (5) Test D for HLLs. The content of the test is based on class material. Researchers conduct three different pilot tests. After the first pilot test, it is found that colloquial vocabulary is very effective in terms of differentiating HL and L2 learners. In regards to spelling, accent, and grammar, the advanced L2 learners appear to outperform the HLLs. The items are reviewed, and a second pilot test is conducted. Additional modifications are made to problematic items and a third pilot test is conducted. Researchers conclude that the test is successful in differentiating all levels except of SPAN 2 and SPAN 3 (i.e., L2 courses).

 

Sohn, S., & Shin, S. (2007). True beginners, false beginners, and fake beginners: Placement strategies for Korean heritage speakers. Foreign Language Annals, 40(3), 407-418.

Keywords: placement test; Korean HLLs; auracy skills; literacy skills; recommendations

This article is one of the few that focuses on the design of placement tests for HLLs from a language other than Spanish. Sohn and Shin describe the process of analyzing the effectiveness of a placement test for Korean HLLs at the University of California, Los Angeles. The test consists of two sections: a multiple-choice section that assesses listening, grammar, and reading skills, and a composition section that assesses writing skills. A total of 107 Korean HLLs take the test. The results indicate that test-takers have different levels of proficiency in each of the skills. In other words, while HLLs are proficient in auracy skills, they are not as proficient in literacy skills. The disparity of the scores between the receptive and productive modes makes it difficult to place students effectively. Researchers provide recommendations to cope with these problems, such as setting high standards for speaking and listening and low standards for reading and writing, when examining advanced HLLs. In terms of instruction, the researchers suggest concentrating more on reading and writing rather than speaking and listening.

 

Titus, J. (2012). Creating an online assessment test for heritage learners of Russian. Heritage Language Journal, 9(2), 85-107.

Keywords: online achievement test; Russian HLLs; ACTFL proficiency guidelines

Titus reports on the design and implementation of an online assessment tool to determine the achievements of 8 Russian HLLs at the end of two semesters. The test has the purpose of assessing listening, reading, and writing skills based on the ACTFL scale. The first two skills are assessed through multiple-choice questions, and writing is examined through three written assignments. HLLs also complete a questionnaire, which require them to self-assess their language proficiency. The test-takers are in the Intermediate level of speaking in ACTFL at the time they take the test. Results indicate that HLLs have advanced levels of listening skills. In terms of reading, their ability varies depending on the genre. Finally, results of the writing samples indicate that the HLLs show difficulties with word choice, register, the ability to support an argument through appropriate linking devices, and spelling. Overall, the test result suggests that there are differences between HLLs’ aural and written skills, with aural skills being more developed. In addition, the questionnaire reveals that HLLs tend to overestimate their listening skills. Titus concludes with curricular implications for Russian HLLs.

 

Wilson, D. V. (2012). Developing a placement exam for Spanish heritage language learners: item analysis and learner characteristics. Heritage Language Journal, 9(1), 27-50.

Keywords: online placement test; Spanish HLLs; graphical analysis; statistical analysis; item types

The purpose of this study is twofold. First, it examines different methods of item analysis—graphical analysis and statistical analysis—to identify good and poor items in an online placement for Spanish HLLs in the University of New Mexico. Second, it analyzes which items are the most effective in discriminating between HL and non-HL learners. Two pilot tests are conducted with 507 students participating in the first one and 330 in the second. This allows the researcher to test as many items as possible and to eliminate poor items. In regards to the statistical methods of item analysis, Wilson utilizes item facility indices, point biserial correlation coefficients, and Cronbach’s alpha.

According to the results of the item analysis, Wilson recommends using different types of items for L2 and HL learners (e.g., items related to object pronouns or subjunctive verbs functioned well when discriminating HLLs but not for L2 learners). Some items, however, are found to be effective for both groups (e.g., gender/number concordance items).

 

Assessment for the Purpose of Differentiating between HL learners and non-HL learners

The entries in the following section describe studies of the design and implementation of tests with the sole purpose of differentiating HLLs from non-HLLs.

 

Davidson, D. E., & Lekic, M. D. (2013). The heritage and non-heritage learner in the overseas immersion context: Comparing learning outcomes and target-language utilization in the Russian flagship. Heritage Language Journal, 10(2), 88-114.

Keywords: Russian HLLs; study abroad; language gains; Russian Overseas Flagship program; Oral Proficiency Interview (OPI); American Councils Standardized Reading and Listening Proficiency Tests; Test of Russian as a Foreign Language (TORFL); self-assessment questionnaire

Although HLLs have received increasing attention in the language acquisition research community, there remains a dearth of research on HLLs studying abroad. Davidson and Lekic examine this particular issue. The researchers report on a study that compares the results from outcomes assessment and language gain assessment of HLLs and non-HLLs in a Russian Overseas Flagship program (i.e., a year-long immersion program abroad). The assessment instruments included (a) the Oral Proficiency Interview (OPI), (b) American Councils Standardized Reading and Listening Proficiency Tests, (c) the Test of Russian as a Foreign Language (TORFL) administered pre- and post-program, and (d) a self-assessment questionnaire administered at the end of the program. Findings indicate that HLLs improved from level 2 to level 4 in the OPI scale, which was more than non-HLLs. In terms of language use, no differences were found between both learner groups. Moreover, the self-assessment questionnaire suggested that HLLs are more self-critical about their language abilities. In addition, the activities identified as helping HLLs improve their language skills are different to those helping the non-HLLs.

 

Di Gennaro, K. (2008). Assessment of generation 1.5 learners for placement into college writing courses. Journal of Basic Writing, 27(1), 61-79.

Keywords: 1.5 generation learners; HLLs; L2 learners; native speakers; academic writing skills

Di Gennaro reviews previous studies concerning different types of L2 learners, which include generation 1.5 (G1.5) learners. She describes the distinctive characteristics of this specific learner group. In this article, they are defined as those HLLs who immigrated at a school age and who completed their secondary education in their host country. They are perceived as having different writing skills than traditional international learners and native speakers. Even though G1.5 learners may be fluent and show advanced levels of oral and aural skills, some lack academic writing skills, which traditional L2 learners have acquired in their L1. By describing these differences, Di Gennaro also draws implications for the design and implementation of placement tests. She does not support having two separate tests targeting each learner group, as it is impractical. However, she advises that the writings should be scored considering that these two groups are different.

 

Di Gennaro, K. (2009). Investigating differences in the writing performance of international and generation 1.5 students. Language Testing, 26(4), 533-559.

Keywords: tradition L2 learners; 1.5 generation students; writing test; Rasch measurement model

Using a Rasch measurement model, this article examines writing samples elicited through a writing test assessing traditional international (IL2) and generation 1.5 (G1.5) students. The study attempts to provide evidence of the difference between the writing performance between both types of learners when considering them as a single group and as separate groups. The test-takers are 97 students (i.e., 43 G1.5 and 54 IL2) and the raters are three L2 writing instructors. The test consists of 1 prompt, which is chosen by the test-takers out of 3. Raters assess the writing samples with an analytic rubric composed of 5 different criteria—cohesive control, grammar, rhetoric, sociolinguistic control, and content. Results of the Rasch analysis, when looking at both groups together, indicate that G1.5 students performed significantly better than the IL2 students in certain components(e.g., rhetorical control), but not in others (e.g., sociolinguistic control). When analyzing the groups separately, the perceived difficulty for the content criterion is found to be different, with G1.5 perceiving it as the most difficult while IL2 as the easiest component.

 

ACTFL Proficiency Guidelines for assessing heritage language learners

The following entries summarize studies that examine the appropriateness of using the ACTFL proficiency guidelines for assessing HLLs.

 

Alonso, E. (1997). La evaluación de la actuación oral de los hispanoparlantes bilingües mediante las directrices de ACTFL. Hispania, 80(2), 328-341.

Keywords: ACTFL Proficiency Guidelines; Spanish HLLs; non-HLLs

The purpose of this study is to closely examine the ACTFL Proficiency Guidelines and to determine the extent to which it can be used as a valid measurement to assess Spanish HLLs’ language proficiency. For this purpose, it qualitatively analyzes the speaking performance of 17 Spanish learners based on the ACTFL guidelines. It focuses on the criteria of content and accuracy, which includes fluency, grammatical, pragmatic and sociolinguistic competence, pronunciation, and vocabulary. First, it compares the performance of heritage and non-heritage language learners. Second, it describes the linguistic characteristics that HLLs posses in 4 different levels of the ACFTL proficiency guidelines, namely Intermediate-mid, Intermediate-high, Advanced, and Superior. Finally, it examines the possibility of using these guidelines to assess HLLs. Alonso concludes that even though the HLL and non-HLL groups present different linguistic characteristics in their speech performance, the ACTFL is a valid measurement to assess HLLs as it separates learners into different levels.

 

Draper, J. B., & Hicks, J. H. (2000). Where we’ve been; what we’ve learned. In J. B. Webb & B. L. Miller (Eds.), Teaching heritage language learners: Voices from the classroom (pp. 15–35). Yonkers, NY: ACTFL.

Keywords: HLL education; National Foreign Language Standards; ACTFL proficiency guidelines; HLL assessment

This review study offers an overview of the history of minority languages and early research of HLs the US. It starts of by briefly describing the history of HL education and the roles that teachers, parents, and the community in general have in HL maintenance in the US. Then, it describes HL teachers’ and learners’ attitudes about the HL and how this factor affects the success in the HL acquisition. Most relevant to this annotated bibliography, Draper and Hicks also discuss some of the limitations of the National Foreign Language Standards (1995) and the ACTFL Proficiency Guidelines (1999) in terms of their inappropriateness to assess HLLs. They argue, as Valdés (1989), that these standards were designed to assess FLLs and not HLLs, and therefore they are not appropriate for that purpose. The language developmental hierarchy described in the Guidelines does not fit the HLLs’ language developmental sequence. Furthermore, these standards are not useful when it comes to improving HL education. To conclude, the authors provide suggestions of different types of assessment tools that could be used specifically for the purpose of assessing HLLs, such as portfolios and dialogue journals. They underscore the need for tools that can take into account the heterogeneity of HLLs and that can be used for measuring their language development.

 

Ilieva, G. N. (2012). Hindi heritage language learners' performance during OPIs: Characteristics and pedagogical implications. Heritage Language Journal, 9(2), 18-36.

Keywords: Hindi HLLs; ACTFL Oral Proficiency Interview; HLLs vs. non-HLLs

Ilieva compares Intermediate Mid or Intermediate High Hindi foreign and HL learners’ speaking ability by using the ACTFL Oral Proficiency Interviews. The participants are 43 Hindi learners (i.e., 36 HLLs, and 7 non-HLLs). Comparisons between the speech samples of both learners groups are made in terms of fluency, salient linguistic features (e.g., adverbial phrase, adjectival phrases; intonation contours), cultural competency, personalization, and use of English. Results indicate that HLLs outperformed the FLLs in terms of fluency, pronunciation, intonation, and cultural competency. Nevertheless, FLLs are more accurate in terms of grammar use than HLLs. Furthermore HLLs tend to overuse the colloquial register and personalize their speech. The study suggests that a differentiation between both learner groups should be made in the guidelines by separating those features that are characteristic just to HLLs.

 

Kagan, O., & Friedman, D. (2003). Using the OPI to place heritage speakers of Russian. Foreign Language Annals, 36(4), 536-545.

Keywords: ACTFL proficiency guidelines; Russian HLLs; educated native speaker; standards; Oral Proficiency Interview (OPI); Simulated Oral Proficiency Interview (SOPI)

This study explores the extent to which the ACTFL guidelines can provide an accurate description of Russian HLLs as well as the extent to which the “educated native speaker” standard can be applied to describe HLLs of languages with minimal dialectal variation, as Russian. It appears to be a response to the article by Valdés (1989) (see below). A total of 11 Russian HLLs of mostly intermediate-low and intermediate-mid proficiency levels participated in the study. A combination of the OPI and the SOPI is used to assess their oral proficiency. HLLs’ speech samples are compared with the 1999 ACTFL guidelines which measure (a) global tasks or functions, (b) contexts and content areas, (c) accuracy, and (d) text types. Findings indicate that while their fluency and pronunciation are almost native-like, their grammatical competence greatly vary across participants and their vocabulary range, although broader than traditional FLLs, shows a great number of calques and code switching. The authors conclude that the ACTFL guidelines pose no problems when assessing HLLs from a global perspective. It could become an issue only if raters take into consideration discrete linguistic features, which according to the authors is not the main focus of the ACTFL guidelines. They advise, however, the use of more than one assessment measure, such as bibliographical information and a written test. In addition, Kagan and Friedman support assessing Russian HLLs according to the “educated native speaker” standard as most of the HLLs come from educated families and unlike Spanish HLLs, there are only very minimal dialectal varieties among this group of Russian learners.

 

Martin, C. (2010). Assessing the oral proficiency of adult learners, "heritage" and "native" speakers using the ILR descriptors and ACTFL guidelines: Considering the challenges. Russian Language Journal, 60, 167-181.

Keywords: ILR descriptors; ACTFL proficiency guidelines; HLLs; Foreign Language Learners; “native” speaker standard

This article examines whether ILR descriptors and ACTFL guidelines are appropriate to assess HLLs. Martin begins by describing the types of test-takers who are currently being assessed with the ILR descriptors or ACTFL guidelines. She states that FLLs, Native Speakers, who are bilinguals or multilinguals, as well as HLLs are being assessed under the same standards. Then she thoroughly examines each of the descriptors in the ILR and the ACTFL guidelines in order to determine how test-takers’ performances are being compared to native speakers (NS). The researcher explains that no criterion compares the test-takers’ performance to an idealized performance or a “native” speaker. The reference to NS in ILR descriptors appear only from level 3+ and more advanced levels. According to Martin, there is no consensus on the fact that there needs to be different set of assessment or criteria to assess HL and FL learners. However, she argues that if the criterion were that the speaker be functionally able to perform certain communicative task and not to match what NSs can do, then there’s no need to have separate tests. She maintains, nonetheless, that there needs to be a different criterion or set of assessments only for Superior or very advanced-level HLLs.

 

Martin, C., Swender, E., & Rivera-Martinez, M. (2013). Assessing the oral proficiency of heritage speakers according to the ACTFL proficiency guidelines 2012–speaking. Heritage Language Journal, 10(2), 211-225.

Keywords: ACTFL proficiency guidelines; computerized Oral Proficiency Interview (OPIc); Spanish HLLs; Russian HLLs

This article discusses the appropriateness and effectiveness of using the ACTFL proficiency guidelines to assess HLLs’ speaking abilities. It examines what kind of information the OPI can or cannot provide about learners who have been rated at different levels. In addition, it discusses some of the results of the joint NHLRC/ACTFL study. This investigation tries to analyze which HLLs’ features prevented them from being rated one level higher. The participants are intermediate and advanced level HLLs (i.e., 41 Spanish and 50 Russian HLLs). They are administered the computerized version of the Oral Proficiency Interview (OPIc). Raters are asked to provide detailed analyses on the examinees’ abilities to address topics, complete tasks, organize extended discourse, and use vocabulary. Researchers conclude that even though HLLs’ profiles clearly differ from the FLLs’, using the ACFTL guidelines could still be useful in assessing the functional proficiency of HLLs. However, they assert that the results of the OPIc should be interpreted with caution, and they should only be used for purposes for which the test was designed. Finally, the article concludes by making some preliminary recommendations about HLL instruction.

 

Swender, E., Martin, C. L., Rivera‐Martinez, M., & Kagan, O. E. (2014). Exploring oral proficiency profiles of heritage speakers of Russian and Spanish. Foreign Language Annals, 47(3), 423-446.

Keywords: Russian HLLs; Spanish HLLs; ACTFL proficiency guidelines; survey; self-assessment; computerized Oral Proficiency Interview (OPIc)

This article is a full report of the Martin et al.’s (2013) study involving 53 Russian and 41 Spanish HLLs assessed using the ACTFL Proficiency Guidelines (see above). The purpose of this study was twofold. It first investigates the different language abilities that Russian and Spanish HLLs in the Intermediate Mid through Superior level showed when taking the OPIc. Then, it also examines the different factors that could be associated with a higher language proficiency level, such as, language background, use, and instruction. The instruments that are used to conduct the study include (a) several online surveys to screen for HLLs and collect data about their language background and experience; (b) a self-assessment tool based on the ACTFL Guidelines; (c) the OPIc. The results of the study are reported in terms qualitative analyses of the weaknesses and strengths of HLLs belonging to different proficiency levels, such as their pronunciation and fluency, use of vocabulary, and ability to organize discourse for different themes (e.g., abstract topics). The results also suggest that the factor that is most strongly associated with higher proficiency is formal instruction and use of HL. However, no correlation coefficients are reported. The authors conclude by providing implications for HLL instruction and offering suggestions for improving the curriculum.

 

Valdés, G. (1989). Teaching Spanish to Hispanic bilinguals: A look at oral proficiency testing and the proficiency movement. Hispania, 72(2), 392-401.

Keywords: ACTFL proficiency guidelines; Hispanic bilinguals; analysis of descriptors; language varieties; “educated native speaker”; standards

This study examines the extent to which the ACTFL Proficiency Guidelines can be used as a valid description of Hispanic bilinguals, and the degree to which the OPI can be used as a valid tool to assess bilinguals’ language competencies. Valdés conducts a thorough analysis of the descriptors and the language developmental sequences assumed in the guidelines. The analysis demonstrates that the guidelines are not useful in describing HLLs. For example, in terms of comprehensibility, the guidelines do not provide information on which level Spanish HLLs should be in the continuum. In addition, there is little information about the comprehensibility in terms of the different language varieties used. Furthermore, the language development sequence assumed by the guidelines is based on FLLs and thus it is unsuitable for describing HLLs (e.g., the ability to engage in social exchanges is in the higher end of the continuum for FLLs but this ability is observed in HLLs of low-proficiency levels). Finally, the article criticizes the “educated native speaker” standard, as it seems to imply that the non-standard varieties of the language would be considered to belong to a lower-level proficiency. Valdés concludes by calling for a modification of the guidelines or the use of a separate set of standards to assess HLLs.

 

Evaluation of HLL programs

The entries in the following section present studies that describe the steps in which different HLL programs were evaluated. In this case, evaluation is defined as a systematic way of collecting various types of data for understanding and improving HLL educational programs.

 

Elder, C. (2005). Evaluating the effectiveness of heritage language education: What role for Testing? International Journal of Bilingual Education and Bilingualism, 8(2-3), 196-212.

Keywords: HLL program evaluation; Chinese-English bilingual program; Arabic-English bilingual programs; Vietnamese-English bilingual program; language proficiency gains

This study summarizes the challenges of performing an evaluation based on 4 HLL programs. The programs evaluated are in Australian primary and secondary schools: (1) Chinese-English bilingual program for both HLLs and non-HLLs, (2) Chinese-English bilingual program for 1st and 2nd generation Chinese HLLs from diverse dialects, (3) Arabic-English bilingual program for 2nd generation immigrants in a secondary school, and (4) Vietnamese-English bilingual program for recent arrivals from Vietnam in a secondary school. Several tests are created to measure HLLs’ language proficiency gains. These tests are designed in the form of general proficiency tests, consisting of both writing and speaking tasks. Elder reports some of the pitfalls of these tests and critiques the way in which proficiency gains are used as the sole evidence of program effectiveness. She concludes with some recommendations of more effective uses of tests for HLLs. Some of these include: developing a bank of sample assessment tasks that could be used by HL teachers for achievement tests, developing validated standardized tests in different HLs and proficiencies, and collecting longitudinal data that show the learning patterns of HLs and FLLs.

 

Elder, C. (2009). Reconciling accountability and development needs in heritage language education: A communication challenge for the evaluation consultant. Language Teaching Research, 13(1), 15-33.

Keywords: HLL program evaluation; Vietnamese-English bilingual program; Mandarin-English bilingual program; Arabic-English bilingual program; language proficiency gain; stakeholder engagement; accountability measures

This article describes the process undertaken to evaluate three of the same HL programs mentioned in Elder (2005) (see above): (1) Vietnamese-English bilingual program, (i.e., Rosemont), (2) Mandarin-English bilingual program (i.e., Luxton), and (3) Arabic-English bilingual program (i.e., Seaview). It also highlights the major concerns related to the context, communication, and process of conducting the evaluation. For Rosemont, an external consultant evaluation assigned Elder and her colleague to be in charge of designing and administering tests that would measure students’ Vietnamese proficiency at two different stages 2 years apart. No other type of evidence, such as classroom observations or discussions with stakeholders, is collected for this evaluation. Moreover, even after the inconclusive results of the proficiency tests, it is reported that the program should undergo only minimal changes. In contrast to Rosemont, the evaluation conducted for Luxton collects evidence from difference sources, such as parent and teacher surveys, classroom observations, and proficiency tests. This evaluation focuses on the effectiveness of the program as well as the extent to which the school and community were supporting the program. The results indicate positive outcomes of the program and identified some of the problems in the program. Finally, for Seaview, a team of consultants that included Elder and two Arabic-speaking academics conduct the evaluation. Elder points out that, unlike the previous evaluations for Rosemont and Luxton, there is a greater degree of engagement between the stakeholders and the evaluators. Thus everyone involved in the program is able to provide feedback, which contributed to the success of the evaluation in terms of uptake of formative recommendations. Overall, this study offers suggestions to reconcile the need for accountability measures and evaluators’ willingness to conduct evaluations that are not solely based on proficiency gains and can lead to program improvement. It also identifies the challenges associated with the status of the evaluator vis-à-vis the heritage language program (i.e., external evaluation).

 

Nocus, I., Guimard, P., Vernaudon, J., Paia, M., Cosnefroy, O., & Florin, A. (2012). Effectiveness of a heritage educational program for the acquisition of oral and written French and Tahitian in French Polynesia. Teaching and Teacher Education, 28, 21-31.

Keywords: French-Tahitian bilingual program; HLL program evaluation; longitudinal design; language proficiency gains

This article reports on the evaluation of the effectiveness of a French-Tahitian bilingual program by comparing an experimental group receiving instruction in the minority language, Tahitian, and a control group taught in French. The purpose of the study is to examine whether instruction in the HL (i.e., Tahitian) would disrupt the acquisition of the dominant language (i.e., French) or vice versa. The Polynesian languages and culture (PLC) program teaches HLLs in Tahitian for 300 minutes per week while the regular program is taught in French with only 160 minutes of instruction in Tahitian. A total of 125 elementary school students with Tahitian as their HL participated in this study. The study uses a longitudinal design, assessing participants five times throughout three years of formal education. Both groups are compared in terms of both language proficiencies. The assessment instruments include: (a) for the French tests, two vocabulary tests (receptive and expressive), one oral comprehension test, and one oral cloze test for kindergarteners, and a word recognition test for first graders; (b) for the Tahitian test, the four French tests for kindergarteners are adapted in Tahitian, as well as the reading test. Results indicate that the Tahitian group performs better than the French group in the Tahitian reading test but does not for the French reading test. In addition, it is found that instruction in Tahitian does not disrupt learners’ French reading proficiency.

 

Shin, S. J. (2006). High-stakes testing and heritage language maintenance. In K. Kondo-Brown (Ed.), Heritage language development: Focus on East Asian immigrants (pp. 127-144). Amsterdam: John Benjamins.

Keywords: No Child Left Behind (NCLB) policy; HLL maintenance; East Asian minority groups

This study examines the effects of the No Child Left Behind (NCLB) policy on HL maintenance with a specific focus on East Asian minority groups. After a detailed overview of the NCLB, Shin describes the possible repercussions of this policy on HL education, such as the discouragement to teach and learn languages other than English, decrease in the instruction time allotted for FLs, the loss of the HL proficiency, among others. In addition, Shin reports on the findings of three individual interviews with an ESL community outreach specialist, ESL instructor, and ESL family liaison from a public school in Maryland which has a large East Asian immigrant population. Furthermore, she conducts a group interview with four HLL parents from a church. Her findings indicate that although some of the interviewees showed a positive attitude toward the NCLB policy, all of them also reported knowing the negative consequences that it has on HL maintenance. Shin concludes by offering suggestions to raise awareness of the importance of HL maintenance and improve HL education.

 

Wright, W. E. (2007). Heritage language programs in the era of English-only and No Child Left Behind. Heritage Language Journal, 5(1), 1-26.

Keywords: English-only education policies; No Child Left Behind Act (NCLB); repercussions; HLL programs; elementary and middle school; bilingual programs

Wright describes the repercussions of English-only education policies, such as the No Child Left Behind Act (NCLB) of 2001. Such policies deter the development of programs that could address American’s needs to be proficient in languages other than English. In this article, Wright provides several cases of in-school and after school HL programs in elementary and middle schools that ended due to the pressure caused by accountability programs and high stakes exams aimed at increasing English proficiency. Some of these include a Cambodian bilingual program (California), bilingual programs for Latinos and Native Americans in charter schools (Arizona). In addition, there are some bilingual programs in Texas, in particular the non-Latino bilingual communities, which were not yet terminated but expressed concern that they would be in the near future. HL programs are considered a low priority compared to the school’s need to increase test scores for accountability evidence.

 

Sociocultural aspects – Identity, Attitudes, Motivation

The following section presents studies that examine the relationship between HLLs’ sociocultural aspects (e.g., identity, attitude, and motivation) and their HL proficiency.

 

Alarcón, I. (2010). Advanced heritage learners of Spanish: A sociolinguistic profile for pedagogical purposes. Foreign Language Annals, 43(2), 269-288.

Keywords: sociolinguistic profile; Spanish HLLs; advanced-proficiency; sociocultural factors; attitude; motivation; linguistic choice; diagnostic test; questionnaire

This survey study examines the sociolinguistic profile of advanced-proficiency Spanish HLLs in order to determine what social and cultural factors influence their attitudes, motivations, and linguistic choices. It also tries to compare some of these factors to the ones described in lower-proficiency learners in previous literature. Furthermore, it draws pedagogical implications in terms of curriculum design and instructional practices. Five advanced-proficiency Spanish HLLs participate in this study. Their proficiency is determined by a diagnostic test that is used to place students in different levels of instruction. The questionnaire consists of 56 items about background, language behaviors, attitudes, and self-assessments. The results indicate that advanced HLLs are proficient in both receptive and productive skills. They also report using the HL at home and being knowledgeable of the standard variety. Moreover, there are differences between the advanced- and lower-proficiency HLLs. Therefore, Alarcón suggests targeting their pedagogical needs differently.

 

Beaudrie, S., & Ducar, C. (2005). Beginning level university heritage programs: Creating a space for all heritage language learners. Heritage Language Journal, 3(1), 1-26.

Keywords: beginner-level Spanish HLLs; attitude; linguistic behavior; HLLs contact; self-assessment; HL exposure

Beaudrie and Ducar assess the attitudes and behaviors of beginner-level Spanish HLLs at the University of Arizona. In particular they attempted to define HLLs at the beginning-level, examine their thoughts about the HL and culture, as well as identify the factors that motivate them to study the HL. The study is conducted in two phases. For the first phase, 20 Spanish HLLs answered an extensive survey. For the second phase, 8 of the 20 participants from phase 1 are interviewed. The survey consists of questions that address HLLs contact with the language, their attitudes towards the HL and its varieties, their self-assessed proficiency, and general background information. Results suggest that most of the HLLs use of the HL is limited to listening rather than speaking. The great majority reported having contact with the HL at home with family members. Moreover, even though a great percentage indicates having a strong positive attitude towards Spanish, they state being more comfortable and interested in listening than in speaking Spanish. According to their answers, they appear to be highly motivated to learn their HL and have both extrinsic and intrinsic motivating factors. The results of the self-assessment demonstrate that they lack confidence in their speaking abilities and the prestige level of the variety of HL that they use. The authors suggest a new and broader definition of HLLs, which would include learners with exposure to the HL through family as well as those exposed through someone outside of family (e.g., nanny). Finally, they provide pedagogical implications for beginner-level HLLs.

 

Ducar, C. M. (2008). Student voices: The missing link in the Spanish heritage language debate. Foreign Language Annals, 41(3), 415-433.

Keywords: Spanish HLLs; perception; attitudes; Spanish language varieties; HLL pedagogy

This survey study investigates Spanish HLLs’ perceptions of and attitudes toward Spanish language varieties. It answers four research questions: (a) what types of Spanish language varieties do students bring to the classroom? (b) what types of language variety do students believe their instructors have? (c) do the students perceive they are being corrected? and (d) do they want to be corrected? 152 students respond to an extensive survey with both close- and open-ended questions focusing on their beliefs about Spanish varieties. The results indicate that most students use the Mexican or Mexican-American variety of Spanish. Moreover, students perceive most of the instructors to have a different language variety than theirs, namely the academic variety. They indicate that even though they are being corrected in the classrooms they do not feel their language variety is being disrespected. Finally, an overwhelming majority of students show their desire to be corrected. Ducar concludes with some recommendations about HLLs pedagogy. Even though this study is not directly relevant to the assessment and evaluation of HLLs, it raises questions about the language varieties that should be taken into account when setting standards to assess HLLs’ language proficiency.

 

Jensen, L., & Llosa, L. (2007). Heritage language reading in the university: A survey of students’ experiences, strategies, and preferences. Heritage Language Journal, 5(1), 98-116.

Keywords: university-level HLLs; reading experience; Korean; Russian; Thai; Vietnamese; self-assessment

This survey study examines the general profiles of university-level HLLs focusing on early reading experiences, the processes and strategies that they use, and their preferences and goals when reading in their HL. A total of 128 HLLs from different language backgrounds, including, Korean, Russian, Thai, and Vietnamese participate in this study. The survey examines HLLs’ personal background, their HL use, and their self-assessed proficiency of 4 skills, and their reading experience, strategies and preferences. The results indicate that the majority of HLLs use English more than their HL but that it varies depending on HL group. For example, Koreans report speaking in both their HL and English. The majority of the HLLs consider maintaining their HL to be important. The reasons they provide in order of importance are: identity, communication with their family members, teaching their future children, and getting jobs. In terms of the results of the self-assessment, HLLs report feeling more confident in their proficiency in listening followed by speaking, reading, and writing. As to their reading experiences, 40% indicate not reading in their HL at all and 58% answer that they read. When comparing the language groups, Thai HLLs are the ones reading the most in their HL followed by Koreans, Vietnamese, and Russians. Finally, the authors indicate that there appears to be a mismatch between the willingness of HLLs to achieve college-level reading abilities with the actual time spent by them in reading. As a result, the researchers suggest that at each instructional level, HLLs should be provided with relevant reading materials that would motivate them.

 

Kang, H. S., & Kim, I. (2012). Perceived and actual competence and ethnic identity in heritage language learning: a case of Korean-American college students. International Journal of Bilingual Education and Bilingualism, 15(3), 279-294.

Keywords: ethnic identity orientation; Korean HLLs; self-perceived competence; ethnic identity scale; Foreign Language Oral Skills Evaluation Matrix (FLOSEM); speaking test; writing test

This study examines the relationship between perceived and actual competence and ethnic identity orientation of 30 second-generation Korean-Americans. It explores whether there are biases of self-perceived competence and ethnic identity orientation. For this purpose, several instruments are used, including an ethnic identity scale (adapted from Lee, 2002), Foreign Language Oral Skills Evaluation Matrix (FLOSEM, Padilla, et al. 1997), and speaking and writing tests. The writing test is composed of 4 prompts and rated by 2 native speakers on the amount and quality of the writing sample. The results are based on correlations coefficients between speaking and writing tasks output (i.e., actual competence) and the HLLs’ self-perceived competence as well as the answers on the ethnic identity scale. Statistically significant correlations are found between competence and ethnic identity, with correlation coefficients ranging from 0.61 to 0.62. However, there are differences in the results of speaking and writing. While there is a statistically significant correlation between HLLs’ actual competence in speaking ( ) and ethnic identity, no statistically significant results are found between actual competence in writing and their ethnic orientation ( ). Regressions also demonstrate that speaking and writing tasks predicted learners’ self-assessed scores. In addition, HLLs with a strong Korean ethnic identity tend to overestimate their ability in their HL while those with a weak Korean identity tend to do the opposite. Moreover, HLLs underestimate their writing ability regardless of their ethnic identity orientation.

 

Kim, E. J. (2006). Heritage language maintenance by Korean-American college students. In K. Kondo-Brown (Ed.), Heritage language development: Focus on East Asian immigrants (pp. 175-208). Amsterdam: John Benjamins Publishing Co.

Keywords: Korean HLLs; age of immigration; length of HLL education; HL use; motivation; HL maintenance; Korean language proficiency test

This study investigates the extent to which age of immigration, length of Korean language education, degree and frequency of HL use, cultural identity, and motivation could be attributed to the maintenance of Korean as a heritage language (HL). One hundred twenty Korean-American college students are administered a background survey and a Korean language proficiency test. The instruments used in the study include a test of Korean proficiency and an extensive language background questionnaire. Multiple regressions as well as a qualitative analysis of the questionnaire are conducted to examine which of the factors best predicts the Korean language maintenance. The results indicate that “heritage language use” was the strongest predictor of HL maintenance.

 

Kondo-Brown, K. (2005). Differences in language skills: Heritage language learner subgroups and foreign language learners. Modern Language Journal, 89, 563-81.

Keywords: Japanese HLLs; Japanese FLLs; language skills; HL-speaking parents; family; HL competence

This article reports on a study examining different subgroups of Japanese HLLs and FLLs. The purpose of the study was twofold. First, it aims at identifying the subgroups of JHLLs that show distinct language behaviors from JFLLs. Second, it explores the language skills that specifically differentiate JHL and JFL learners. A total of 185 Japanese learners participated in this study. They are categorized in four groups: (1) JHLLs Descent Group (N=66), Japanese descent born in the US without Japanese-speaking parents; (2) JHLLs Grandparent Group (N= 47), learners of Japanese descent born in the US with at least 1 Japan-born grandparent; (3) JHLLs Parent Group (N= 30), learners born in Japan or the US with at least 1 Japanese-speaking parent; (4) JFLLs group (N=42), learners self-reported not to be HLLs. The instruments include a multiple-choice test assessing their Japanese grammar, listening, and reading proficiency. In addition, a questionnaire examined their use of Japanese in various contexts, their language choice or dominance in different situations, and their self-ratings of ability to carry out oral tasks. Profile analyses are conducted according to these data and the results showed that JFLLs, JHLLs descent, and JHLLs grandparent are found to be similar, whereas the JHLLs parent group is found to be different from the other three groups. Accordingly, Kondo-Brown suggests that HLLs who are in the competent JHLLs group and those who are in the JHLLs identity group should be differentiated. In other words, a simple dichotomous differentiation between HLLs and FLLs is too simplistic and not accurate.

 

Kondo-Brown, K. (2006). East Asian heritage language proficiency development. In K. Kondo-Brown (Ed.), Heritage language development: Focus on East Asian immigrants (pp. 243-258). Amsterdam: John Benjamins.

Keywords: East-Asian HLLs; HL proficiency development; HL use; socio-psychological factors; HLL parents’ role; biliteracy development

This review study summarizes a number of articles that investigate different factors that could be associated with East-Asian HLLs’ proficiency development. These studies focus on three main types of variables: (a) L1 background and HL use and contact; (b) socio-psychological factors, namely, motivation, attitude, ethnic orientation, identity, vitality perception, peer influence, and length of HL instruction; and (c) HLL parents’ role on HLLs’ HL proficiency development and their perceptions on biliteracy development. Kondo-Brown synthesizes the studies based on (a) the type of study, that is, whether it uses quantitative, qualitative, or mixed methods; (b) the type of instruments used, such as self-rating questionnaire, interviews, or proficiency tests; and (c) what types of variables they explores. Finally, she concludes with suggestions for future research on East-Asian HLLs, which include conducting more research based on observation data, implementing curriculum development research that takes into consideration socio-psychological factors, researching how teachers and school administrators address the needs and concerns of HLLs’ parents, and examining cross-national policies on HLLs, among others.

 

Oh, J. S., & Au, T. K. F. (2005). Learning Spanish as a heritage language: The role of sociocultural background variables. Language, Culture and Curriculum, 18(3), 229-241.

Keywords: Spanish HLLs; sociocultural background; HL competence; Multigroup Ethnic Identity Measure (MEIM); language ability test

This study aims at identifying sociocultural background variables that would predict high competence in the HL of college-level Spanish HLLs. The variables in consideration include cultural identification, cultural participation, and use of Spanish outside of the classroom situation. Proficiency in Spanish is determined by native-like accent and grammar. The participants are 55 Latino college students, with at least one Latino parent (N=27) or with both Latino parents (N=28). Several instruments are used for the purposes of this study, namely: (a) a Multigroup Ethnic Identity Measure (MEIM; Phinney, 1992); (b) a questionnaire consisting of 5 times for cultural identification, 5 items for cultural participation; and (c) a language ability test assessing narrative accent, phoneme accent, and grammar. Results indicate there are significantly positive correlations between accent and cultural identification, cultural participation, Spanish use in high school and Spanish use currently, with coefficients ranging from 0.34 to 0.48. In terms of grammar in narratives, only Spanish use is positively correlated, with coefficients ranging from 0.29 to 0.44. Note that, even though the correlations are statistically significant, the reported coefficients are relatively weak.

 

Shibata, S. & Koshiyama, Y. (2001). Social influences in the acquisition and maintenance of spoken Japanese as a heritage language. Japan Journal of Multilingualism and Multiculturalism, 7, 18-37.

Keywords: Japanese HLLs; Japanese Oral Proficiency Interview (OPI); social factors; language use; HL instruction; heritage culture interest; language history; language experience

This study explores the influence of social factors, such as, language use, HL instruction, and heritage culture interest on Japanese HLL’s proficiency. A total of 30 JHLLs participate in this study and they are asked to take a Japanese OPI as well as to respond to an extensive questionnaire about their language history and experience with the HL. ANOVAs are conducted to examine the differences between the proficiency scores in the OPI between different types of groups. For example, in order to determine whether HLLs’ parents language influence their proficiency, four groups were compared: (a) both parents speaking Japanese, (b) one parent speaking Japanese, (c) parents using both English and Japanese, and (d) parents speaking only English. Similarly, the language used by the HLLs to talk to their parents is considered as another factor that could influence the HLLs’ proficiency. The results indicate that when both or at least one parent speaks to the HLL in the HL, their proficiency is significantly different to those who where only spoken English to. Likewise, HLLs who speak to one or both parents in the HL or a combination of English and HL are significantly more proficient than the HLLs who only speak English. Next, HL instruction is another factor that is considered in combination with parents’ language use. Findings show that HLLs, who had contact with HL either through their parents or through going to HL schools, differ significantly with those who did not have contact at all. No effect sizes are reported for any of the ANOVAs conducted. Finally, the relationship between HL proficiency and (a) number of years in HL school, (b) whether or not HLLs had HL-speaking relatives, (c) frequency of visits to Japan, (d) whether or not they participated in activities with HL, and (e) their interest in Japan through media is analyzed through Pearson correlation coefficients or point biserial correlations. Significant positive correlations are only found between proficiency and factors a (r=.487), b (rpbi = .667), and e (rpbi =.608).

 

Key background and theoretical papers on HLLs

The following studies are not about assessment or evaluation, but they provide important frameworks that might inform that kind of work.

 

Benmamoun, E. & Kagan, O. (2013). The administration of heritage language programs: Challenges and opportunities. Heritage Language Journal, 10(2), 143-155.

Keywords: university level HLL programs; administration; Q&A

This paper provides a summary of a Q&A session with Abbas Benmamoun and Olga Kagan, two well-established researchers of Heritage Language Learners (HLLs). They discuss their opinions on important issues about HL programs, including the challenges that HLLs face in order to achieve advanced levels of HL proficiency. Key issues are (a) offering advanced language courses specifically designed for HLLs, (b) best institutional conditions for HL programs as well as the need for a separate track from the foreign language (FL) programs, and (c) overall practical challenges of administering HL programs (e.g., staffing, teacher training, and budgetary concerns). Their answers are based on their experiences not only as researchers but also as instructors/administrators. Accordingly, this article offers a practical view of the challenges and opportunities of HL programs, not just based on empirical evidence but on the reality of administering language programs at the university level.

 

Lee, J. S., & Shin, S. (2008). Korean heritage language education in the United States: The current state, opportunities, and possibilities. Heritage Language Journal, 6(2), 153-172.

Keywords: Korean HLLs; community schools; HL maintenance

As one of the biggest HL communities in the US, there has been increasing interest in Korean HLLs. Lee and Shin provide a general overview of research investigating this specific HL community. After providing census-based data on the Korean population in US, and Korean community schools, they provide an overview of studies on Korean HLLs. First, in addition to summarizing various HLL definitions proposed by several researchers, they specifically define Korean HLLs. Second, they describe Korean language and the structures that may be challenging to HLLs, such as honorifics. Third, they make recommendations in terms of the areas in need of more research, including HLLs’ attitude, identity shift, and learning opportunities. They point out the lack of research on “intergeneration transmission” (Campbell & Christian, 2003), language maintenance, and study abroad. Finally, they offer suggestions on how community schools could work together with public schools in order to support Korean language maintenance.

 

Montrul, S. (2010). Heritage language programs. In M. H. Long  & C. J. Doughty (Eds.), Handbook of second and foreign language teaching (pp. 182-200). Malden, MA: Wiley-Blackwell.

Keywords: HLLs’ linguistic development; cognitive studies; HLLs vs. non-HLLs

Although this is not a review focused on assessment and evaluation of HLLs, this article is included in this annotated bibliography as it can inform assessment and evaluation researchers on HLLs’ linguistic development and how they differ from first and second language learners in their skills and acquisition of the HL. The review provides a thorough overview of earlier and relatively recent empirical research on HLLs, focusing mainly on cognitive studies that compare HL and non-HL and HLLs’ language abilities as seen through their skills in phonetics and phonology, vocabulary, morphosyntax, and syntax. It summarizes the findings of previous research on certain linguistic structures that have been incompletely acquired by the HLLs (e.g., relative clauses in Spanish and Russian). In addition, Montrul provides a description of how some of these studies have followed different theoretical approaches such as formal linguistics (e.g., UG and the critical period hypothesis), cognitive linguistics (e.g., neurolinguistic studies), and emergentistm (e.g., constructions). Finally, the researcher concludes with some suggestions for a future HLL research agenda.