To Advanced Proficiency and Beyond — facebook.com/LinguaLIB


Чтобы посмотреть этот PDF файл с форматированием и разметкой, скачайте его и откройте на своем компьютере.
To Advanced Prociency and Beyond
TO ADV
NCED PROFICIENCY
ND BEYOND
Theory and Methods for Developing Superior
Second Language Ability
Tony Brown and Jennifer Bown, Editors
GEOR
ETOWN UNIVERSITY PRESS
ashington, D.C.
Shifting Focus from Intermediate Skills in Classroom Training
6. Taking on the “Ceiling
ffect” in Arabic
R. Kirk Belnap and Khaled Abuamsha
Context of the Federally Supported Overseas
anguage Training
NATIO
xpertise:
vii
Ray Cliord, Director of the Center for Language Studies, and John
Rosenberg, Dean of the College of Humanities, both of Brigham Young University,
for sponsoring a symposium in 2012 that precipitated this volume. Agnes Welch of
the Center for Language Studies provided invaluable behind-the-scenes assistance
in the way of planning and carrying out the symposium. We also wish to thank Mel
Thorne of Faculty Editing Services at Brigham Young University for providing
critical input and direction in the nal stages of format editing.
The contributors to this volume made editing it a true pleasure. We express
our sincere thanks to each of them for their thoughtful input, graceful acceptance
David Nicholls, acquisitions editor, Deborah Weiner, former editorial and produc
tion manager, and Glenn Saltzman, editorial, design, and production manager.
Outside readers provided important organizational, compositional, and stylis
tic comments that enhanced the volume as a whole. We extend our sincere thanks
to them for their careful reading of the manuscript and quality feedback.
Lastly, we wish to thank our spouses, Emily Brown and Tom Bown, for their
ACIE
American Councils for International Education
ACTFL
American Council on the Teaching of Foreign Languages
ACTR
American Council of Teachers of Russian
AH
Advanced High
Advanced Low
Advanced language ability
Advanced language capability
Advanced Language Performance Portfolio System
ALT
Adaptive Listening Test
Advanced Mid
AOF
Arabic Overseas Flagship
ART
Adaptive Reading Test
Bachelor of Arts
BYU
Brigham Young University
Center for Applied Linguistics
Center for the Advancement of Language Learning
CASA
Center for Arabic Study Abroad
CATRC
Computer Adaptive Test for Reading Chinese
CCALT
Chinese Computer Adaptive Listening Test
CDLC
Coalition of Distinguished Language Centers
Continuing Education
Content and Language Integrated Learning
Critical Languages Scholarship
DA
Diagnostic Assessment
DLAB
Defense Language Aptitude Battery
DLIELC
Defense Language Institute English Language Center
DLIFLC
Defense Language Institute Foreign Language Center
DLNSEO
Defense Language and National Security Education Oce
DLPT
Defense Language Prociency Test
DTRA
Defense Threat Reduction Agency
ECA
Educational and Cultural Aairs
English for Heritage Language Speakers
event-related potential
English for Specic Purposes
OCQ
Overclaiming Questionnaire
OFC
Overseas Flagship Capstone
OIG
Oce of the Inspector General
OPI
Oral Prociency Interview
Open Source Analytical Research Project
Ohio State University
PDF
Portable Document Format
PLUS
Partnership for Language in the U.S.
Question and Answer
Reading Comprehension
ROF
Russian Overseas Flagship
Superior
SEALang
Southeast Asian Languages
Second Language Acquisition
SOPI
Simulated Oral Prociency Interview
Speaking Prociency
Science Technology Engineering Mathematics
TBLT
task-based language teaching
TOEFL
Test of English as a Foreign Language
University of North Florida
US
United States
Writing Prociency Test
Zone of Proximal Development
xiii
Intermediate Skills in Classroom Training to Advanced/Superior
A L. MART
THE YEAR 2012 MARKED
thirty years since the initial development of the
American Council
on the Teaching of Foreign Languages (ACTFL) provisional prociency guidelines
for speaking (as part of the ACTFL Language Prociency Projects in 1982), and
twenty years since the launch of ACTFL’s ocial testing oce, Language Testing
International (in 1992).
The original framework for the
ACTFL Prociency
Guidelines was based on the
Interagency Language Roundtable (ILR) Descriptions
for Speaking, which remain in widespread use by US government agencies.
the rst two decades subsequent to the initial publication of
ACTFL’s provisional
guidelines, a growing prociency movement transformed the foreign language
classroom and curricular planning in the United States in ways many had hoped
for, but no one had imagined. This transformation took place, as one might expect,
mostly at the lower levels of instruction in what is generally thought of as Novice and
Intermediate levels. During the 1980s and well into the 1990s, the focus remained
have placed a notion of what students can
with their foreign language, rather than
what they
The prociency movement has been so successful in transforming the US eld of
foreign language instruction that university language programs commonly expect to
graduate students with a minimum Intermediate level of
oral prociency. However,
xiv
Cynthia L. Martin
of languages at these levels for purposes of national security, but learners themselves
have sought higher prociency levels in order to take advantage of growing oppor
tunities to use language skills in an era of rapid globalization in ways unimaginable
just a few decades ago. This increase in demand for Advanced-level or Superior-level
speakers across a wide range of specializations has naturally expanded the need for
and use of a reliable and valid assessment instrument that uses a standardized pro
tocol across languages in order to assess how well and in which contexts a speaker
is able to function in a language. The most widespread and familiar assessment tool
for speaking has become the
Oral Prociency Interview (OPI), rated according
to either the ACTFL Prociency Guidelines
2012—Speaking or the Interagency
Language Roundtable Descriptors—Speaking.
De�ning Our Terms: What Does a Rating of
Advanced (ILR Level 2) or Superior (ILR Level 3)
on an Oral Pro�ciency Interview (OPI) Tell Us?
The chapters in this volume use the ratings of Advanced or Superior according to
the ACTFL Guidelines as reference points for discussing higher levels of language
use; therefore, it is important that the reader understand (1) how an OPI is admin
istered, (2) what a rating in the Advanced or Superior range says about a speaker, and
(3) what rating criteria are used for each level. Note that OPIs conducted by govern
ment agencies and rated according to the
ILR Skill Level Descriptors are similar to
the ACTFL OPI in terms of the assessment criteria and functional expectations,
but since there is no standard ILR OPI administered by one central organization,
each government agency uses its own testing protocol. Therefore, the explanation
below is limited to the ACTFL OPI, as it uses a standardized protocol for elicitation
in addition to standardized rating criteria.
An ocial ACTFL OPI is conducted by a certied ACTFL tester either over
IN
TRODUCTIO
Intermediate speakers can use language to engage in simple conversation, ask ques
tions, and handle a simple social transaction. Advanced speakers can narrate and
describe in major time frames and handle a situation with an unexpected compli
cation. Superior speakers can discuss abstract issues extensively, support opinions,
hypothesize, and handle linguistically unfamiliar (i.e., not previously encountered)
xvi
Cynthia L. Martin
Type, paragraph-type oral discourse is required for a rating in the Advanced range
because that is the text type required by the Advanced functions (e.g., narrations and
descriptions) and, therefore, would not apply to a speaker capable of only producing
simple sentences.
A word about the
sublevel ratings: a speaker rated Advanced Low can perform
the functions of Advanced while maintaining the other assessment criteria, albeit
minimally. At Advanced Low, a speaker does not usually show much evidence of
Superior-level criteria. A speaker rated Advanced Mid is a speaker who performs the
functions of Advanced while sustaining the other assessment criteria with appropri
ate quantity and quality across a range of topics. A speaker rated Advanced Mid is
likely to show some evidence of performance or linguistic features associated with
the Superior level. An Advanced High speaker is able to perform the functions of the
Superior most of the time, but is not able to sustain that performance across a range
of functions or topic areas. A speaker with a rating of Advanced High is not just a
superstar when performing Advanced functions; rather, an Advanced High speaker
should be thought of as an emerging Superior.
What does a rating of Superior mean?
The Global Tasks and Functions criteria that a speaker must fulll to achieve a rat
ing of ACTFL
Superior include discussing an issue at the abstract level, supporting
opinions, hypothesizing, and handling linguistically unfamiliar situations (usually
indicated by the ability to circumlocute and to deal with any low-frequency or unfa
miliar linguistic material). A Superior speaker is one who can think aloud in the tar
IN
TRODUCTIO
xvii
our collective goals. Individual chapters in this volume align with these three broad
Dening Characteristics of High-level Language Learning and Learners
Approaches to Maximizing Language Gain at Home and Abroad
Future Directions in Assessment, Program Design, and National Policy
I. De�ning Characteristics of High-level
Language Learning and Learners
One thing we know for certain is that all language learners are unique and do not
comprise a homogenous group. That being said, learners who achieve high levels
of prociency in a second or foreign language they begin studying as adults may
share common characteristics. In chapter 1, “Experience with Higher Levels of
xviii
Cynthia L. Martin
and organizing and evaluating ideas. The results from language prociency
that included these aligned language prociency and cognitive skill development
expectations, along with the results from an intelligence test, are used to report
IN
TRODUCTIO
xix
adult learners who plan to pursue professional careers with the US government. The
chapter draws on the authors’ experiences as directors of the English for Heritage
Language Speakers (EHLS) program, which prepares native speakers of critical lan
Cynthia L. Martin
Another approach to facilitating high levels of prociency by integrating state
side and in-country study is presented by Matthew Christensen and Dana Bourgerie
of BYU in chapter 5, “Chinese for Special Purposes: Individualized Instruction as a
Bridge to Overseas Direct Enrollment.” The authors discuss their experience with
an individualized instructional approach used in the Chinese Flagship Program at
BYU that prepared students for direct enrollment in content classes in a Chinese
university. This approach is predicated upon the assertion that in order for stu
dents to maximize in-class instruction abroad, they must be prepared in three ways:
IN
TRODUCTIO
xxi
Scholarship program (CLS), and the national Flagships. The reports provide here
tofore unreported information on the impact of instruction in the less commonly
taught languages undertaken in the overseas immersion context for learners’ cohorts
ranging from 16-year-old students with no prior training to young adults seeking to
advance their linguistic and cultural prociency at the Intermediate, Advanced, and
Superior levels. Davidson’s study documents and compares the outcomes of immer
sion study across dierent languages and for students of dierent ages at dierent
stages of their language-learning career. While clearly demonstrating the advantages
of an early start in the study of a critical language, exemplied by the results of the
NSLI-Y program, the author also provides strong evidence of success at reach
ing the Superior level for students who begin their study at the college level with
access to appropriate levels of overseas immersion training, such as those provided
by the Critical Language Scholarship Summer Institutes, Fulbright-Hays, and the
Language Flagship Program.
III. Future Directions in Assessment, Program
Design, and
ational Policy
The growing body of evidence about successful learners, programs, and approaches
at the advanced levels has promoted thinking about new directions in assessment,
teaching, and learning related to professional language use. In chapter 8, “From
xxii
Cynthia L. Martin
HR-based assessment tools could enhance educators’ ability to graduate foreign lan
IN
TRODUCTIO
xxiii
of such language features leads to accelerated and deeper acquisition, provided that
learners are able to immerse themselves concurrently in an authentic lingua-cultural
context.
Additionally, Jackson discusses work by Bernhardt (2005), who pointed out
that the
Foreign Service Institute’s (FSI) understanding of ILR Level 3 language
prociency has been deepened and rened in recent years, resulting in improved
training. Jackson also discusses work by Byrnes (2006a; 2006b) and several col
leagues, who have proposed a model based on multi-literacies. These multi-literacies
are, in turn, based on a recognition that various communicative contexts exist and
that certain contexts, particularly the public context, require the language user to
xxiv
Cynthia L. Martin
Though a level (called Distinguished) has been added to the 2012 guidelines above Superior, the
current ocial OPI does not test for Distinguished. A brief discussion of the Distinguished level
and considerations for assessment can be found at the end of this introduction.
6.
ACTR/ACCELS administers Advanced and Professional-level language instruction at overseas
universities and immersion centers that currently support more than 1,500 Americans annually,
including the State Departme湴’s NSLI-Y program for high school students and the undergradu
ate overseas CLS summer institutes, the Department of State’s Title VIII training program,
US Department of Education’s Fulbright-Hays Group Project Abroad, the Defense Language
and National Security Education Oce’s (DLNSEO) African Languages Initiative (AFLI), and
Overseas Flagship programs, which include direct enrollment and internships in Arabic, Chinese,
Persian, Russian, Swahili, and Turkic languages.
References
Byrnes, Heidi, ed. 2006a.
Advanced Language Learning
. London: Continuum Press.
Bynres, Heidi. 2006b. “Locating the Advanced Learner in Theory, Research, and Educational Practice:
An Introduction.” In
Educating for Advanced Foreign Language Capacities
, edited by Heidi Byrnes, Heather
Part One
Language Learning and Learners
AV
SINCE 1983, US GOVERNMENT
Betty Lou Leaver and Christine Campbell
EXP
ER
CE
WIT
HIG
HER
EL
S OF
PR
OFI
to support programs seeking to produce ILR-4 outcomes, and (2) a precursor of
the Flagship programs, the
Center for the Advancement of Distinguished Language
Prociency at
San Diego State University, funded by the
National Security Education
Program (NSEP). The program was established to train teachers of high-prociency
programs in pragmatic techniques for achieving near-native levels of foreign lan
guage skills. With the appearance of these two organizations, publications and con
ferences of the
Betty Lou Leaver and Christine Campbell
motivational, demographic, and learning strategy traits, including an intense desire
to share their “story.” Data were collected in more than one hundred categories per
skill (reading, writing, listening, speaking) and in more than twenty-eight demo
graphic areas. Among the learners were polyglots who shed much light on the vari
ability in language acquisition at higher levels.
The
NFLC-DLIFLC study was followed by another and conducted on learn
ers of English at the University of Jordan who had achieved Level 4. That study,
funded by the National Teachers of English’s Conference on College Composition
and Communication, compared Level 4 achievers with Level 3 achievers and looked
mainly at demographics (Jasser, Al-Khanji, and Leaver 2005).
These studies distilled eight typical core characteristics of a
language learner
able to achieve
near-native levels of foreign language prociency, some of which are
counter to those exhibited by good language learners at lower levels of prociency:
tenacity;
a good ear;
desire to do a good job (a job that requires L2), i.e., instrumental motivation;
upbringing in a multilingual neighborhood;
background in a multilingual home;
experience abroad, particularly in going to school with native speakers and
being treated as a native speaker or in holding a job abroad in the same capacity
as a native speaker;
a native-speaking spouse or signicant other (or equivalent friendships); and
access to classroom instruction at levels 2+ and 3 (Leaver 2013).
It is important to note that 75 percent of the students had received upper-
EXP
ER
CE
WIT
HIG
HER
EL
S OF
PR
OFI
remembered them. Most respondents reported actively seeking out access to the
culture; e.g., they went to public places like restaurants and cultural events where the
language was spoken. The study abroad programs they joined provided an experi
ence in which the students could attend class alongside native speakers, not foreign
ers. They became tenacious and were able to motivate themselves when they fell
into a slump.
Surprisingly, aside from the typical need for an instructor’s insights into the less
obvious aspects of sociolinguistics, there was no clear path to the top. Students made
Betty Lou Leaver and Christine Campbell
conducted in the
Russian language, and was open to the Russian-speaking com
munity in the Washington area. Students presented speeches, conducted debates,
and held roundtable discussions on typical State Department topics in Russian, with
the public asking unanticipated questions that students had to handle in an erudite
manner. This conference was open to students in the basic course, as well, and often
pushed students over the ILR-2+/3 threshold.
Today, the program has taken a new turn. Called the
Beyond Three Course,
there is no guarantee that students will reach ILR-4, although many do, and much
more responsibility for learning is left to individual students, with emphasis on
strong reading of classics in the language. D椝erent approaches with similar goals
have seen nearly thirty years of success in teaching upper levels of prociency at FSI.
Along the
ay:
ther
rograms
There are other programs that have pushed an occasional student to ILR-4. These
include astronaut training, study abroad sites, teacher training programs, some
Flagship programs, and some university centers.
The Russian and English program developed for the
International Space Station
in 1998 saw the development of ILR-4 prociency by several astronauts at
NASA
and a cosmonaut or two at Russia’s
Gagarin Cosmonaut Training Center. The train
ing program, like the DLIFLC and FSI programs, is content-based and uses real-life
tasks and authentic materials—an approach that can be very challenging for foreign
language teachers since none of those hired to date at either institution have been
EXP
ER
CE
WIT
HIG
HER
EL
S OF
PR
OFI
and focused on linguistics, sociolinguistics, culture, literature, and other high-level
topics needed to prepare teachers for instructing all levels of prociency. Although
no prociency testing was used, four forms of feedback were used to assess the suc
cess of these programs: native-speaker instructor observations (conclusion: language
prociency noticeably improved in both uency and accuracy), formal participant
feedback to the Institute (conclusion: without exception, participants stated that
Betty Lou Leaver and Christine Campbell
who contributed to this volume, the current limited amount of information on
how to move forward in terms of theory, practice, and research (every one of these
domains being critical) will be substantially supplemented and enhanced.
A Recent Example:
he Defense Language
nstitute
oreign Language Center
The
Prociency Enhancement Program at the DLIFLC has as its ultimate goal to
produce linguists with the highest possible levels of prociency in its basic, inter
mediate, and advanced courses. Currently, for basic (initial acquisition) courses the
goal is ILR-2+ in listening comprehension (LC), ILR-2+ in reading comprehen
sion (RC), and ILR-2 in speaking prociency (SP) or Advanced High/Advanced
High/Advanced, respectively, on the ACTFL scale. The
graduation requirement is
ILR-2/2/1+, or Advanced/Advanced/Intermediate High, respectively, although with
the recent emphasis by the military services on the ILR-3 as the career goal for lin
guists the goal may at some point become the requirement.
For intermediate courses, as noted above, the graduation requirement is
2+/2+/2, and for advanced courses, the graduation requirement is 3/3/2+. However,
for the past ve to six years, DLIFLC has nearly continually produced linguists
at ILR-3 and above in both the intermediate and advanced courses at DLIFLC’s
Directorate of
Continuing Education (CE), using a strategic approach that pro
motes teacher autonomy and innovation in the areas of curricula, assessment, and
faculty development. The approach is informed by the survey research on learner
characteristics and teaching experience described in the historical background and
literature review section above. CE empowers its faculty to do more than teaching:
it asks them to create their own curricula, to develop their own assessments, and to
seek out opportunities to develop professionally and share that knowledge with col
leagues as they collaborate to construct knowledge, ultimately forming a “commu
nity of understanding and practice” (Bailey and Freeman 1994). This approach has
been a key factor in an increasing number of students who improve as much as 1.5,
or even two prociency points in the skills of listening, reading, and speaking (writ
ing is taught but not tested) over the course of eighteen, thirty-six, or forty-seven
weeks, depending on the category of the language. Below are the entry requirements,
although, given that students who are working military jobs need to improve their
prociency, these requirements are often waived:
Intermediate: ILR-2 in LC and in one other skill; minimum graduation require
ment—ILR-2+/2+/2 (exception: the
Defense Threat Reduction Agency
EXP
ER
CE
WIT
HIG
HER
EL
S OF
PR
OFI
Course lengths vary according to language category.
Below, the lengths of
courses in weeks are given by language category:
Intermediate: eighteen weeks for a Category 1 language (
Spanish); thirty-six for
Category 3 (
Russian,
Serbo-Croatian,
Hebrew,
Persian/
Farsi); forty-seven for
Category 4 (
Arabic, Chinese, Korean);
Advanced: eighteen weeks for a Category 1; thirty-six for Category 3; forty-
seven for Category 4.
As the curriculum and approach to upper levels have matured and improved
over time, results have improved from a low of 33 percent of the students reaching
igure 1.1
CE
verall
raduation
ates
ote a:
otal number of students from 2008 through 2013 is 850.
ote b:
eference legend: Ss = students; %
rad = number of students achieving graduation requirements (
-2+/2+/2 for
ntermediate students and
3/3/2+ for Advanced students);
xceed = number of students with scores higher than graduation standards (
-3, 3+, and 4 for
ntermediate students and
-3+ or 4 for Advanced students). About 75 percent of those who exceed
-3 achieve
-3+ and 25 percent
-4.
Betty Lou Leaver and Christine Campbell
speaking scores, students take an oral prociency interview. Although the
DLPT5
is a multiple-choice test of listening and reading skills, CE conducts a diagnostic
assessment (DA), which is an interactive, full-range, formative prociency assess
ment, on each student. Graduation results have been, for the most part, consistent
EXP
ER
CE
WIT
HIG
HER
EL
S OF
PR
OFI
student is most ready to learn next, which will vary by student. Assessors (teach
ers) provide learners with a learner prole identifying their strengths and weak
nesses in the three skills and those linguistic items that fall within their ZPD; this
prole will guide their classroom and homework activities, as well as their inde
pendent study time. The team of teachers develops a class prole (a picture of the
linguistic and cognitive strengths and weaknesses of the group as a whole) in order
Betty Lou Leaver and Christine Campbell
in the United States, specically in English; then they practice interviewing using the
EXP
ER
CE
WIT
HIG
HER
EL
S OF
PR
OFI
CE teachers accompany students and each evening prepare them for the classes
they will attend the next day (see description below);
learners live the campus experience;
peer tutors are available;
excursions and social activities relate to the lesson topics;
learners have real-life experiences;
learners write daily reports;
homestays are arranged; and
learners make a formal presentation about their immersion experiences upon
Betty Lou Leaver and Christine Campbell
students go on an excursion directly tied to the morning lecture. If, for example,
the lecture is about law, the students may visit a law oce and, in the case of Arabic
countries, ask the lawyers how they integrate government law and
law. If the
lecture is about the military, in some countries it is possible to visit a military post
and compare foreign and US military practices. If the lecture is about a social phe
nomenon, the excursion might be to the oce of a journalist writing about the topic.
Of course, there are also historical and cultural tours, trips to restaurants, shopping,
and other everyday activities. The goal is to provide students with real-life experi
EXP
ER
CE
WIT
HIG
HER
EL
S OF
PR
OFI
An alternative assessment approach promoted by Wiggins (1998, 12) in the late
educative assessment,” is also practiced at CE. Educative assessment has two
underlying principles:
First, assessment should be deliberately designed to teach (not just measure)
by revealing to students what worthy adult work looks like (oering them authentic
tasks). Second, assessment should provide rich and useful feedback to all students
and to their teachers and it should indeed be designed to assess the use of feedback
by both students and teachers.
Two forms of formative assessment are used on a regular basis, as well: DA (as
described above) and
Betty Lou Leaver and Christine Campbell
present these projects both orally and in writing and are evaluated by the instruc
tional st愝. Daily quizzes and the like are rarely used; the time in the classroom is
spent on learning, and the independent time is spent not in test preparation but
on project preparation. As stated above, formal presentations are also made upon
EXP
ER
CE
WIT
HIG
HER
EL
S OF
PR
OFI
desired and a greater range of practices is always welcome, in the past thirty years,
since the rst introduction of a high-level prociency course at FSI, much has
become known about how teachers can actually “teach” to these higher levels; how
ever, if a greater number of students are to achieve near-native prociency, more
research—both empirical and action based—is needed. Chance and in-country
Betty Lou Leaver and Christine Campbell
EXP
ER
CE
WIT
HIG
HER
EL
S OF
PR
OFI
Y, RAY C
express frustration over the reality that their limited
second language (L2) skills prevent them from full participation in intellectual dis
cussions, and some have exclaimed, 鍈ey, 䦒m smarter than I sound!” Those who
Dan Dewey, Ray Clifford, and Troy Cox
Context for
Assessing Cognition and
Foreign
Language Pro ciency
To assess language prociency, we opted to use the ACTFL
Oral Prociency
Interview (OPI) and the ACTFL
Writing Prociency Test (WPT), two of the
most prominent and widely accepted measures of language prociency used in the
L1, L2,
COGNI
IVE DEVE
MEN
PL
ING RE
LAT
IONSHI
areas by undergraduate students during their college careers. Perkins’s data are out
dated, but the more recent work by Conley (2009) suggests that this lack of formal
reasoning skills is still an issue of concern today. Given the limited reasoning abili
Table 2.1
ACTFL
Hierarchy of
ommunication
asks and the
elated
evelopment of
ognition and Knowledge
Level
ACTFL Proὣiency Guidelines
Corresponding
Elements from Bloom’s
Revised Taxonomy
Cognitive Process
Dimension
Cognitive Abilities Expected
of US Undergraduates
(based on Conley 2009 and
Perkins 1985)
Speaking Tasks
Writing Tasks
Novice
Can communicate short mes
sages using memorized words
and phrases.
Can produce lists, notes, and limited
formulaic information on simple forms
and documents. Writing is typically lim
ited to words, phrases, and memorized
material.
Remember
(Retrieve from memory)
Intermediate
Can recombine learned
material to express simple
statements/questions about
familiar topics of a personal or
predictable nature.
Can meet a range of simple and practical
writing needs, e.g., simple messages and
letters, requests for information, notes,
etc. Can communicate simple facts and
ideas in a loosely connected series of
sentences on topics of personal interest
and social needs, primarily in the pres
ent. Because of vocabulary limitations
and errors in basic structures, writing is
comprehensible to those accustomed to
the writing of nonnatives.
Understand
(Construct meaning)
Memorize and state facts
Advanced
Can describe people, places,
and things in the major time
frames of past, present, and
future.
Can provide detailed, concrete,
factual narrations about events
and activities.
Can give coherent directions or
instructions.
Can write routine, informal, and some
formal correspondence, narratives,
descriptions, and summaries of a factual
nature in all major time frames in
connected discourse of a paragraph in
length. Writing is comprehensible to all
native speakers due to breadth of generic
vocabulary and good control of the most
frequently used structures.
Apply
(Carry out, use)
Exchange ideas (when more
concrete topics and factual nar
rations are involved)
Conduct experiments (largely
involves following factual
directions, acting questions
about procedures, etc.)
Level
ACTFL Proὣiency Guidelines
Corresponding
Elements from Bloom’s
Revised Taxonomy
Cognitive Process
Dimension
Cognitive Abilities Expected
of US Undergraduates
(based on Conley 2009 and
Perkins 1985)
Speaking Tasks
Writing Tasks
Superior
Evaluate ideas and provide
structured arguments to sup
port one’s opinions.
Develop hypotheses to explore
alternative possibilities.
Discuss topics from an abstract
perspective.
Can produce informal and formal
writing on practical, social, and profes
sional topics treated both abstractly and
concretely.
Can present well-developed ideas,
opinions, arguments, and hypotheses
through extended discourse.
Can control structures, both general
and specialized/professional vocabulary,
spelling, punctuation, cohesive devices,
and all other aspects of written form and
organization with no pattern of error to
distract the reader.
Analyze
(Distinguish, organize)
Evaluate
(Judge appropriateness)
Create
(Hypothesize)
Engage in reasoning, argumen
tation, and proof
Defend a point of view
Description of a point of view
(abstract topic/perspective)
Evaluation of a variety of
perspectives
Analyze con icting explana
tions, supporting arguments,
Dan Dewey, Ray Clifford, and Troy Cox
L1, L2,
COGNI
IVE DEVE
MEN
PL
ING RE
LAT
IONSHI
and listening comprehension (principally receptive skills) to evaluating the ability to
perform in real-life situations. This desire to see learners’ performance in real-life
situations eventually resulted in the US Foreign Service Institute (FSI) OPI, an
oral test that involved describing pictures, giving a short talk without preparation,
and responding to recorded audio prompts. Results were then rated according to
a checklist of features that included accent, grammar, vocabulary, comprehension,
and uency. This test ultimately transitioned into a face-to-face interview designed
to elicit speech samples according to a hierarchy of increasingly dicult, real-world
tasks. One impact of this task hierarchy was that it created a situation in which the
cognitive demands placed on the examinees made monitoring language use progres
sively dicult, resulting in the elicitation of speech samples that reected the exam
inee’s internalized language system and nonrehearsed material.
As described elsewhere in this volume, the transitions in government testing
were eventually reected in an ACTFL-led initiative to produce a similar global
Dan Dewey, Ray Clifford, and Troy Cox
which “involves considering a claim and seeking reasons with a nonformal bearing
on the claim, pro or con, in an attempt to resolve the truth of the claim” (562).
Further, in informal reasoning, “reasons typically occur on both sides of the case, no
L1, L2,
COGNI
IVE DEVE
MEN
PL
ING RE
LAT
IONSHI
The learners’ L1 and L2 prociencies;
The cognitive diculty of the communication tasks presented and the amount
of language produced when performing those tasks in the learners’ L1 and L2;
and
The learners’ level of cognitive development and the learners’ L1 and L2 test
performance.
Participants
Participants consisted of 108 learners of Spanish as a second language (88 male,
㈰�female, mean age 21.7 years,
= 1.54 years) enrolled at Brigham Young University
(BYU) in Spanish 321: Third-Year Spanish Reading, Grammar, and Culture. Spanish
321 is a required course for all Spanish majors and minors and the rst class that
Dan Dewey, Ray Clifford, and Troy Cox
“What are the Writing Prociency Tests and the Business Writing Tests?”, para
graph 1). The WPT consists of four or ve written prompts in English “dealing with
practical, social, and/or professional topics that are encountered in informal and
formal contexts” (paragraph 2). Prompts are selected largely based on a learner’s
L1, L2,
COGNI
IVE DEVE
MEN
PL
ING RE
LAT
IONSHI
Results
In this section, we compare results for L1 and L2 OPI and WPT
Figure 2.1
ral Pro ciency
atings for Subjects in
nglish (
ative
anguage) and Spanish (Second
anguage)
Dan Dewey, Ray Clifford, and Troy Cox
English and Spanish (2 at the Advanced High level and 1 at the Advanced Mid level)
and no instances in which someone scored higher in Spanish than in English.
For the WPT, the mean score on the English tests was 9.56 (
= 0.64), with
median and mode at the Superior level (see Figure 2.2). For the Spanish, the mean
was 7.07 (
= 0.89), with the median and mode also at the Advanced Low level. A
Wilcoxon Matched-Pairs Signed-Ranks Test was likewise conducted with the WPT
for both languages. For the WPTs, the English scores were also signicantly higher
than the Spanish (
= –8.944;
0.001), with 102 instances of the subjects scor
ing higher in English. There was only one instance in which a subject had the same
rating in English and Spanish (Advanced Mid) and no instances in which someone
scored higher in Spanish than in English.
Figure 2.2
Writing Pro ciency
atings for Subjects in
nglish (
ative
anguage) and Spanish (Second
anguage)
�able 2.2
riting Pro ciency Test Word Count by Prompt Type
Intermediate
Advanced
Advanced High
Superior
English
Spanish
English
Spanish
English
Spanish
English
Spanish
102
100
103
100
103
100
103
100
Mean
178.04 146.10
367.36
236.12
368.74
262.45
385.96
297.12
SD
56.4
40.57 103.56
69.71
125.25
83.48 120.81
112.2
Mean Di erence between English and Spanish
31.94
131.24
106.29
88.84
Dan Dewey, Ray Clifford, and Troy Cox
of dierence, with the subjects writing on average 31.94 more words in English
than in Spanish. The largest mean dierence (di) was with the Advanced prompt
⡤椟�= 131.24), followed by the Advanced High prompt (di = 106.29), and then the
Superior prompt (di = 88.84). These data were somewhat surprising because the
�able 2.3
hi-Square Results for L1 English OPI (Pro ciency) and OCQ (IQ)
Variable
Pro ciency
High
Low
High
3.90
0.047
Low
Totals
15
able 2.4
hi-Square Results for L1 English WPT (Pro ciency) and OCQ (IQ) Results
Variable
Pro ciency
High
Low
High
2.91
0.076
Low
1313
Totals
36
22
L1, L2, AN
COGNI
VE DEVE
PMEN
NG RE
ONSHIPS
intelligence/cognitive ability and L1 speaking prociency. Although the connection
�able 2.5
hi-Square Results for L2 Spanish OPI (Pro ciency) and OCQ (IQ) Results
Pro ciency
Variable
High
Low
High
15
1.24
0.198
Low
Totals
able 2.6
hi-Square Results for L2 Spanish WPT (Pro ciency) and OCQ (IQ) Results
Pro ciency
Variable
High
Low
High
15
0.121
0.468
Low
15
Totals
2632
Dan Dewey, Ray Clifford, and Troy Cox
In light of previous ndings, it is encouraging to see higher-level reasoning skills
consistently present in the L1 for a great majority of these language learners. The
fact that learners did so well on the L1 measures contributed to a ceiling eect (i.e.,
L1, L2,
COGNI
IVE DEVE
MEN
PL
ING RE
LAT
IONSHI
have relatively high cognitive abilities, given their prole upon entrance. B奕’s 2013
acceptance rate was 56.2 percent, and incoming freshmen in that same year had an
average grade point of 3.82 on a 4-point scale and an average ACT score of 28.52
(near the 90
percentile). Depending on their admission policies, other universities
may have students demonstrating a broader range of cognitive abilities.
In addition to a more varied L1 prole, recruiting subjects with a broader range
Dan Dewey, Ray Clifford, and Troy Cox
References
American Counsel on the Teaching of Foreign Languages (ACTFL). 2013. “Testing for pro
ciency.” Accessed October 22. http://www.act.org/professional-development/
certied-prociency-testing-program/testing-prociency.
American Counsel on the Teaching of Foreign Languages (ACTFL). 2012. “Oral Prociency Interview
L1, L2,
COGNI
IVE DEVE
MEN
PL
ING RE
LAT
IONSHI
Spolsky, Bernard. 1995.
Measured Words: The Development of Objective Language Testing
. Oxford, UK: Oxford
University Press.
Weatherford, H. Jerold. 1986.
Personal Benets from Foreign Language Study
. (ERIC Document Reproduction
Services 276305.) Washington, DC: ERIC.
Part Two
Workforce
professional prociency (Interagency Language Roundtable
ILR] Level 3 prociency) in a second or additional language is the goal of many
adult language learners, particularly those who intend to use the linguistic and cul
tural knowledge in their professional careers. Business, industry, and government
all have need of personnel with professional-level skills in two or more languages.
In particular, many branches of the US government actively seek to employ persons
who possess general professional prociency in both English and another language
and thus can conduct research and analysis that is informed by deep cultural and
linguistic knowledge.
This chapter describes the
English for Heritage Language Speakers (EHLS)
Deborah Kennedy and Christa Hansen
the establishment of an English language instructional program that would provide
such skills. The legislation specied that the program would be housed at an institu
tion of higher education and would enable naturalized US citizens with native or
Superior-level heritage language prociency in a language critical to national secu
rity to develop the professional English skills needed to fulll vital positions in fed
eral government agencies.
The overall design for the program was developed by the Center for Applied
Linguistics (CAL) in 2005–2006. CAL distributed a survey on language use and
language needs to senior personnel in the Department of Defense, the Department
of State, and the Intelligence Community and interviewed government and private
DEVELO
ING SU
ERIOR
ANGUAGE PROFI
IEN
Y AN
NALYTI
AL S
ILL
oundations of Program Design
The overall design and the instructional curriculum for the
EHLS program at
Deborah Kennedy and Christa Hansen
DEVELO
ING SU
ERIOR
ANGUAGE PROFI
IEN
Y AN
NALYTI
AL S
ILL
decisions, understanding the essence of challenges, stating and defending policy,
Deborah Kennedy and Christa Hansen
DEVELO
ING SU
ERIOR
ANGUAGE PROFI
IEN
Y AN
NALYTI
AL S
ILL
Develop a systematic editing and revision process;
Practice paraphrasing, quoting, and citing sources;
Deborah Kennedy and Christa Hansen
comprehension activities, analytical discussion, oral briengs, written and oral sum
maries, analytical reports, vocabulary building exercises, and language analysis activi
DEVELO
ING SU
ERIOR
ANGUAGE PROFI
IEN
Y AN
NALYTI
AL S
ILL
Deborah Kennedy and Christa Hansen
The instructional program also incorporates an extensive amount of individu
alized instruction to address the specic needs of each participant (
CDLC 2008).
Individualized instruction occurs in three formats:
Oral Communication Tutorial
: A weekly one-hour tutorial with a
Professional Oral
Communication
instructor for extensive feedback and individualized instruction
on all aspects of oral communication (listening, speaking, intercultural compe
tence) as a complement to the
Professional Oral Communication
course. Feedback is
based on a review of participant-produced audio journals and video recordings
DEVELO
ING SU
ERIOR
ANGUAGE PROFI
IEN
Y AN
NALYTI
AL S
ILL
activities that will promote the desired learning outcome. In the backward design
Deborah Kennedy and Christa Hansen
achievement assessments. End-of-session assessments range from timed reading
and writing activities in the
Reading and Writing for Professionals
course, to assessment of
application of frame analysis concepts in
News Analysis
, to an interactive team project
Professional Oral Communication
. This design gives the instructor multiple data points
DEVELO
ING SU
ERIOR
ANGUAGE PROFI
IEN
Y AN
NALYTI
AL S
ILL
Money Laundering Enterprises: Afghanistan.
Money laundering in Afghanistan involves
four key groups, with the current government playing a dominant role; ndings
also identify three primary modes of cash transfers. Research was conducted
in English, Dari, Pashto, and Russian using websites, international reports, and
Afghan media and government documents.
China’s Public Position on the South China Sea.
China’s public position on the South
China Sea, albeit ambiguous, becomes clearer with careful examination of mes
sages underlying the use of key terms. Examination of Chinese news media, o�cial
statements, and academic publications indicates that China may be adopting a
new approach in order to enhance its bargaining power in the region.
Deborah Kennedy and Christa Hansen
At the inception of their tenure in the
EHLS instructional program, all instruc
tors received eighty hours of in-house training in the curriculum, programmatic
DEVELO
ING SU
ERIOR
ANGUAGE PROFI
IEN
Y AN
NALYTI
AL S
ILL
59
To be admitted to the
EHLS program, a candidate must receive ratings at or
above ILR Level 2/ACTFL Advanced Low in all four modalities in English. This
requirement ensures that participants will have the skills needed to participate eec
tively in all instructional activities, particularly the capstone OSAP. The same tests
(in B forms for the listening, reading, and writing) are administered at program exit.
The listening, reading, and writing tests each have two parallel forms: Form A, given
at entry, and Form B, given at exit.
The total number of participants admitted to the program each year depends on
the funding level available. Each participant receives a scholarship that includes full
tuition and a living stipend for the duration of the program; receiving a scholarship
is conditional on each participant agreeing to work for the federal government for
one year in a position with national security responsibilities.
Cumulative
esting
utcomes
Renement of the instructional curriculum and teaching approach over the years
has resulted in progressive improvements in language prociency outcomes. The
goals for the program, as identied by the funder, are for 50 percent of all exit test
ratings to be at ILR Level 3 and 75 percent to be at ILR Level 2+ or higher after
eight months of instruction (six months intensive and two months part time). The
2009
2010
2011
2012
28 participants
37 participants
35 participants
29 participants
Entry
Exit
Entry
Exit
Entry
Exit
Entry
Exit
Total—all modalities
ILR 3
19.6
28.6
31.1
28.4
34.3
41.0
39.1
58.3
ILR 2+ or higher
50.0
61.6
63.5
68.9
63.6
72.7
71.3
81.7
ILR 2 or higher
92.9
91.1
95.9
98.6
93.6
98.6
100.0
100.0
ILR 1+ or lower
7.1
8.9
4.1
1.4
6.4
1.4
0.0
0.0
* Formal reporting of speaking scores used the ACTFL pro ciency scale. The corresponding
ILR scale ratings are used in this table for purposes of summarization. Correspondence as fol
lows: ACTFL Superior = ILR 3; ACTFL Advanced High = ILR 2+; ACTFL Advanced Mid and
Advanced Low = ILR 2; ACTFL Intermediate High = ILR 1+.
Deborah Kennedy and Christa Hansen
table. Strict adherence to the minimum entry requirement in 2012 contributed to
the progra涒s ability to create a learning environment with a higher level of chal
lenge and thus achieve higher outcome levels in that year.
Oral Skills Outcomes
In listening and speaking, outcomes showed marked improvement in the 2012
program year, particularly with regard to the percent of participants who attained
ILR Level 3 (Table 3.2). Adherence to the minimum requirement of ILR Level 2
prociency at entry contributed to this outcome. Additionally, in preparing for the
2012 instructional program, the instructors revised the listening comprehension
and related oral communication activities to ensure that the activities demanded
the levels of listening discrimination and speaking nesse that characterize Superior
prociency. The revised activities increased the progra涒s ability to promote the
development of the desired levels of prociency in the oral skill modalities.
In listening, a closer look at participant outcomes illustrates the challenges that
the program has faced in addressing this modality, as well as the improvement that
resulted from the curricular changes made in 2012 (Table 3.3).
In Table 3.3, the shaded cells indicate the number of participants each year who
2009
2010
2011
2012
28 participants
37 participants
35 participants
29 participants
Entry
Exit
Entry
Exit
Entry
Exit
Entry
Exit
Listening
ILR 3
7.1
17.9
10.8
10.8
20.0
34.3
27.6
69.0
ILR 2+ or
higher
39.3
64.3
64.9
64.9
80.0
74.3
72.4
90.0
ILR 2 or higher
100.0
96.4
100.0
100.0
100.0
100.0
100.0
100.0
ILR 1+ or lower
0.0
3.6
0.0
0.0
0.0
0.0
0.0
0.0
Speaking
ACTFL
Superior
14.3
25.0
27.0
21.6
34.3
45.7
31.0
58.6
ACTFL
Advanced High
or higher
32.1
42.9
59.5
73.0
54.3
71.4
65.5
79.3
ACTFL
Advanced Low
or higher
75.0
75.0
83.8
94.6
88.6
97.1
100.0
100.0
ILR 1+ or lower
25.0
25.0
16.2
5.4
11.4
2.9
0.0
0.0
DEVELO
ING SU
ERIOR
ANGUAGE PROFI
IEN
Y AN
NALYTI
AL S
ILL
decreased, and the cells to the right of the shaded cells provide numbers of partici
pants whose ratings increased.
The majority of participants each year demonstrated no change in rating for
listening. Multiple noncurricular factors may explain this outcome:
The
ILR prociency levels are broad, meaning that a learner must acquire a
great deal of language ability before learning is reected in a level change.
The test contains only 65 items; one right or wrong answer can mean the dier
Exit (ratings)
ILR 1+
ILR 2
ILR 2+
ILR 3
Entry (ratings)
Listening 2009
ILR 1+
ILR 2
ILR 2+
ILR 3
Listening 2010
ILR 1+
ILR 2
ILR 2+
ILR 3
Listening 2011
ILR 1+
ILR 2
ILR 2+
ILR 3
Listening 2012
ILR 1+
ILR 2
ILR 2+
ILR 3
Total Listening
ILR 1+
ILR 2
15
ILR 2+
21
ILR 3
Deborah Kennedy and Christa Hansen
The content of the
test is not aligned closely with the focus of EHLS instruc
tion. EHLS activities use topics of relevance to the federal government context,
whereas the test contains general interest topics.
Program participants have varying educational backgrounds, professional back
grounds, and learning goals and motivations.
*CAL does not have access to the technical manual for the test and therefore is
unable to ascertain its Standard Error of Measurement.
However, the outcomes for 2012 also seem to show that the changes made to
the curriculum in that year did have an eect. In that year, more than half of the
participants improved their listening ratings, a far higher percentage than seen in
previous years. In 2012, ten of the thirteen participants who had entered at Level 2+
increased their ratings to Level 3, whereas in 2011, only six of twenty-one did so. In
addition, in 2012, for the rst time no participants received a lower listening rating
at exit than at entry (see Table 3.10 in Appendix 1). These dierences can be attrib
uted, in part, to curricular changes that aligned listening practice activities more
closely with the specic skills and abilities that characterize the ILR Level 3 listener.
A paired sample
Year
Paired Differences
Sig.
(2-tailed)
Effect
Size
Mean
Std.
Deviation
2009
LExit - LEntry
0.423
0.703
3.070
0.005
28
0.602
2010
LExit - LEntry
0.061
0.659
0.529
0.601
0.092
2011
LExit - LEntry
0.143
0.591
1.279
0.212
0.242
2012
LExit - LEntry
0.810
0.602
6.167
20
0.000
1.346
Overall
LTotalExit
- LTotalEntry
0.315
0.692
4.725
107
0.000
129
0.455
= 0.05
DEVELO
ING SU
ERIOR
ANGUAGE PROFI
IEN
Y AN
NALYTI
AL S
ILL
observed for the years 2010 and 2011. Cohe溒s eect size values for the listening
= 0.46) suggested a moderate practical signicance (Cohen 1988).
In speaking, the trend across all years has been for the majority of participants to
maintain a prociency level, with most of the rest achieving a rating increase and a few
experiencing a rating decrease (Table 3.5). Because Language Testing International
provides the speaking tests, speaking ratings are reported using the ACTFL scale.
For the sake of comparison with the ILR scale, in Table 3.5, the ACTFL Mid and
Low levels are collapsed.
able 3.5
Speaking
odality,
ntry/
xit
atings by
umber of
articipants
Exit (ratings)
ACTFL IH
ACTFL AM
ACTFL AH
ACTFL S
Entry (ratings)
Speaking 2009
ACTFL IH
ACTFL AM
ACTFL AH
ACTFL S
Speaking 2010
ACTFL IH
ACTFL AM
ACTFL AH
ACTFL S
Speaking 2011
ACTFL IH
ACTFL AM
ACTFL AH
ACTFL S
Speaking 2012
ACTFL IH
ACTFL AM
ACTFL AH
ACTFL S
Total Speaking
ACTFL IH
ACTFL AM
21
15
ACTFL AH
15
ACTFL S
28
Deborah Kennedy and Christa Hansen
However, 44.8 percent of participants achieved a rating increase in 2012, com
pared with ㌰–35 percent in the three preceding years (see Table 3.11 in Appendix1).
In particular, data for those who entered with a speaking rating of Advanced High
show that, whereas only three of ve moved from Advanced High to Superior in
2009, as did two of twelve in 2010 and three of seven in 2011, fully eight of ten in
2012 were able to demonstrate such growth in speaking prociency. This improve
ment may, in part, reect changes to the 2012 curriculum and overall stronger cohort
in terms of speaking prociency at entry. Because no participants entered at a pro
ciency level below Advanced Low, the instructors were able to challenge participants
in ways that they could not in earlier years when they had to ensure that lower pro
ciency participants were able to keep pace.
A paired sample
Year
Paired Differences
Sig.
(2-tailed)
Effect
Size
Mean
Std.
Deviation
2009
SExit - SEntry
0.292
0.751
1.904
0.070
28
0.389
2010
SExit - SEntry
0.444
0.698
3.309
0.003
0.637
2011
SExit - SEntry
0.565
0.590
4.596
22
0.000
0.958
2012
SExit - SEntry
0.650
0.671
4.333
0.000
0.969
Overall
STotalExit
- STotalEntry
0.479
0.684
6.790
0.000
129
0.700
= 0.05
DEVELO
ING SU
ERIOR
ANGUAGE PROFI
IEN
Y AN
NALYTI
AL S
ILL
Writing Outcomes
In reading, obtaining a fully accurate estimate of program participants’ ability and
progress presents a challenge. As Table 3.7 shows, reading is the strongest skill
modality overall for EHLS program participants, all of whom have tested at ILR
Level 2+ or higher at both entry and exit in every program year since 2009. Because
the ELPT test used for entry and exit testing does not provide ratings above Level 3,
it may not provide actual reading prociency levels for some program participants.
The strength of the entry ratings and the limitations of the reading test do not allow
for meaningful conclusions to be drawn. Therefore, no further analysis has been car
ried out on the reading outcomes.
Achievement of the designated exit prociency goals in the writing modality
has presented the biggest challenge to the
EHLS program. Not surprisingly, this
is the modality in which entry ratings are lowest in all years, with more than half
of participants entering at ILR Level 2 or lower each year (Table 3.8; Table 3.12 in
Appendix 1). The data provided in Tables 3.9 and 3.10 demonstrate the diculty of
moving participants to ILR Level 3 in this modality. The 2012 cohort was stronger,
overall, than those of previous years, with higher percentages of 3 and 2+ ratings at
entry. In that year, for the rst time, all participants who entered at 2 or 2+ either
maintained or increased their rating at exit. However, three of the ve who entered
with Level 3 ratings saw their prociency ratings decrease by the end of the program.
Table 3.8 provides a graphic representation of the writing outcomes.
A paired sample
-test was conducted on the writing outcomes, using the same
2009
2010
2011
2012
28 participants
37 participants
35 participants
29 participants
Entry
Exit
Entry
Exit
Entry
Exit
Entry
Exit
Reading
ILR 3
50.0
57.1
73.0
64.9
77.1
71.4
79.3
79.3
ILR 2+ or higher
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
ILR 2 or higher
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
ILR 1+ or lower
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
Writing
ILR 3
7.1
14.3
13.5
16.2
5.7
11.8
17.8
25.0
ILR 2+ or higher
28.6
39.3
29.7
37.8
20.0
44.1
46.4
57.1
ILR 2 or higher
96.4
92.9
100.0
100.0
85.7
97.1
100.0
100.0
ILR 1+ or lower
3.6
7.1
0.0
0.0
14.3
2.9
0.0
0.0
Deborah Kennedy and Christa Hansen
= 2.45,
= 0.69),
(114) = 4.85,
= 0.000. The results showed a sig
Exit (ratings)
ILR 1+
ILR 2
ILR 2+
ILR 3
Entry (ratings)
Writing 2009
ILR 1+
ILR 2
ILR 2+
ILR 3
Writing 2010
ILR 1+
ILR 2
ILR 2+
ILR 3
Writing 2011
ILR 1+
ILR 2
ILR 2+
ILR 3
Writing 2012
ILR 1+
ILR 2
ILR 2+
ILR 3
Total Writing
ILR 1+
ILR 2
ILR 2+
ILR 3
DEVELO
ING SU
ERIOR
ANGUAGE PROFI
IEN
Y AN
NALYTI
AL S
ILL
Though writing represents a major focus of both the six-month intensive por
tion of the program and the two-month part-time session that follows, the devel
opment of professional prociency in writing evidently may require more time
and sustained practice than the EHLS program has been able to provide to date.
However, the 2011 and 2012 data suggest that programmatic changes, such as a
closer alignment of instructional design, activities, assessment, and performance
Year
Paired
Differences
Sig.
(2-tailed)
Effect
Size
Mean
Std.
Deviation
2009
WExit WEntry
0.154
0.613
1.280
0.212
28
0.251
2010
WExit - WEntry
0.188
0.535
1.982
0.056
0.350
2011
WExit - WEntry
0.424
0.708
3.440
0.002
0.599
2012
WExit - WEntry
0.333
0.565
2.892
0.008
0.590
Overall
WTotalExit
- WTotalEntry
0.278
0.615
4.853
114
0.000
129
0.453
= 0.05
Deborah Kennedy and Christa Hansen
federal agencies and government contractors. The program has produced a total of
215 graduates from its inception in 2006 through 2012.
DEVELO
ING SU
ERIOR
ANGUAGE PROFI
IEN
Y AN
NALYTI
AL S
ILL
Deborah Kennedy and Christa Hansen
Shekhtman, Boris. 2003.
Working with Advanced Foreign Language Students.
Salinas, CA: MSI Press.
2009
2010
2011
2012
Entry Score = 3
7.1
10.8
20.0
27.6
No rating change*
0.0
5.4
17.1
27.6
Decrease 3 to 2+
7.1
5.4
2.9
0.0
Decrease 3 to 2
0.0
0.0
0.0
0.0
Entry Score = 2+
32.1
20
54.1
21
60.0
44.8
No rating change
21.4
32.4
34.3
10.3
Increase 2+ to 3
10.7
5.4
17.1
34.5
Decrease 2+ to 2
0.0
16.2
8.6
0.0
Entry Score = 2
60.7
35.1
20.0
27.6
No rating change
32.1
18.9
17.1
10.3
Increase 2 to 3
7.1
0.0
0.0
6.9
Increase 2 to 2+
17.9
16.2
2.9
10.3
Decrease 2 to 1+
3.6
0.0
0.0
0.0
Total Participants
28
Total no rating change
15
53.6
21
56.8
24
68.6
48.3
Total increased rating
35.7
21.6
20.0
15
51.7
Total decreased rating
10.7
21.6
11.4
0.0
*The maximum rating on the test is 3, so an increase is not possible for an examinee who receives an
entry rating of 3.
DEVELO
ING SU
ERIOR
ANGUAGE PROFI
IEN
Y AN
NALYTI
AL S
ILL
able 3.11
EHL
S Speaking
utcomes, 2009–2012 (number and percentage of ratings)
2009
2010
2011
2012
Entry Rating = Superior
14.3
27.0
34.3
31.0
No rating change
10.7
17.9
34.3
27.6
Decrease Superior to
Advanced High
3.6
17.9
0.0
3.4
Entry Rating = Advanced
High
17.9
32.4
20.0
34.5
No rating change
3.6
24.3
11.4
3.4
Increase Advanced High
to Superior
10.7
5.4
8.6
27.6
Decrease Advanced High
to Advanced Mid or
Advanced Low
3.6
2.7
0.0
3.4
Entry Rating = Advanced
Mid or Advanced Low
4.3
24.3
34.3
34.5
No rating change*
21.4
10.8
17.1
17.2
Increase Advanced Mid or
Advanced Low to Superior
3.6
2.7
2.9
3.4
Increase Advanced Mid
or Advanced Low to
Advanced High
10.7
10.8
14.3
13.8
Decrease Advanced Mid
or Advanced Low to
Intermediate High
7.1
0.0
0.0
0.0
Entry Rating = Intermediate
High
21.4
17.9
11.4
0.0
No rating change
14.3
2.7
2.9
Increase Intermediate
High to Advanced High
0.0
2.7
0.0
Increase Intermediate
High to Advanced Mid or
Advanced Low
7.1
8.1
8.6
Entry Rating = Intermediate
Mid
3.6
2.7
0.0
0.0
No rating change
0.0
0.0
Increase Intermediate Mid
to Intermediate High
3.6
2.7
Total Participants
28
Total no rating change
50.0
51.4
65.7
48.3
Total increased rating
35.7
32.4
34.3
44.8
Total decreased rating
14.3
0.0
0.0
6.9
*Includes scores that moved from Advanced Mid to Advanced Low and vice versa.
Deborah Kennedy and Christa Hansen
able 3.12
Title
EHL
S Writing
utcomes, 2009–2012 (number and percentage of ratings)
2009
2010
2011
2012
Entry Rating = 3
7.1
13.5
5.7
17.2
No rating change
7.1
10.8
5.7
6.9
Decrease 3 to 2+
0.0
0.0
0.0
6.9
Decrease 3 to 2
0.0
2.7
0.0
3.4
Entry Rating = 2+
21.4
16.2
14.3
31.0
No rating change
10.7
5.4
11.4
20.7
Increase 2+ to 3
7.1
5.4
2.9
10.3
Decrease 2+ to 2
3.6
5.4
0.0
0.0
Entry Rating = 2
67.9
70.3
65.7
15
51.7
No rating change
46.4
20
54.1
45.7
37.9
Increase 2 to 3
0.0
0.0
2.9
3.4
Increase 2 to 2+
14.3
16.2
14.3
10.3
Decrease 2 to 1+
7.1
0.0
2.9
0.0
Entry Rating = 1+
3.6
0.0
11.4
0.0
No rating change
0.0
0.0
Increase 1+ to 2+
0.0
5.7
Increase 1+ to 2
3.6
5.7
Entry Rating = 1
2.9
0.0
No rating change
0.0
Increase 1 to 2
2.9
Total Participants
28
Total no rating
change
64.3
70.3
22
62.9
65.5
Total increased
rating
25.0
21.6
34.3
24.1
Total decreased
rating
10.7
8.1
2.9
10.3
FOR CENTURIES, DEBATE HAS
been a major component of universities throughout Western
Europe and the United States. Aside from educating students about signicant
social and political issues, debate fosters critical thinking and analytical skills, not to
mention respect for opposing opinions and an increased capacity to relate to indi
viduals of dierent persuasions. In a debate, every claim is subject to questioning,
Tony Brown, Jennifer Bown, and Dennis L. Eggett
university foreign language instruction as a critical step toward achieving Advanced-
and Superior-level prociency.
Debate and Superior-Level Functions—Speaking
Supporting and defending opinions is a core task at the Superior level of foreign
language learning, and the criteria outlined in the ACTFL speaking performance
prole describe qualities emphasized in public speaking and debate (i.e., uency/
integrative abilities, breadth of vocabulary, sociolinguistic/cultural appropriateness,
grammatical accuracy, and debate-related communication tasks).
Fluency/Integrative Abilities
In terms of uency of speech, the Superior-level speaker easily connects extended
breadth of vocabulary facilitates uency and accuracy and expands one’s capacity
to provide uninterrupted, extended, and in-depth discourse on a topic rather than
spotty, fragmented statements that lack transitions, cohesion, and continuity.
Sociolinguistic/Cultural Appropriateness
Parliamentary style debate forums challenge one’s ability to demonstrate intellectual
prowess, cultural sophistication, and wittiness. Unlike American-style debate, parlia
mentary-style debate accords greater value to overall presentation than to the defense
of a position. As such, manner of presentation in the form of idiomatic statements,
NC
ED
FOR
IGN L
NGU
HROUGH
GL
BAL
EBATE
Similarly, a Superior-level speaker may make occasional grammatical errors, but
one does not typically notice a recurring pattern of errors in speech, especially with
regard to basic structures.
Debate and Superior-Level Functions—
Writing
Tony Brown, Jennifer Bown, and Dennis L. Eggett
NC
ED
FOR
IGN L
NGU
HROUGH
GL
BAL
EBATE
Tony Brown, Jennifer Bown, and Dennis L. Eggett
NC
ED
FOR
IGN L
NGU
HROUGH
GL
BAL
EBATE
Homework
Tony Brown, Jennifer Bown, and Dennis L. Eggett
conversation in this way focused students’ attention on lexical items that were intro
duced in a given unit and encouraged their usage in responses to questions posed by
the native speaker. As with feedback on writing assignments, native speakers pro
vided corrective recasts when discussing topics with students as a way of modeling
correct usage of newly introduced items.
ourse
utcomes and
mplications
Figure 4.1
omparison of Pre- and Post-
atings
NC
ED
FOR
IGN L
NGU
HROUGH
GL
BAL
EBATE
Figure 4.2
omparison of Pre- and Post-WP
atings
Table 4.1
Pre–/Post–
ral Pro ciency by
umber of Participants
Post-OPI
ACTFL
rating
Pre-OPI
Table 4.2
Pre–/Post–Writing Pro ciency by
umber of Participants
Post-WPT
ACTFL
rating
Pre-OPI
Tony Brown, Jennifer Bown, and Dennis L. Eggett
�㴜–3.78,
�㴜.0005). In terms of writing prociency, the data likewise suggest that
the higher participants rated on the pre-WPT, the less gain they demonstrated dur
ing the semester (slope/eect size = –0.19,
= –1.47,
= 0.15). Results of carrying
out a paired t-test for overall gain indicated that participants demonstrated signi
NC
ED
FOR
IGN L
NGU
HROUGH
GL
BAL
EBATE
Thus, an approach in which learners are given tasks that are aligned with the
Tony Brown, Jennifer Bown, and Dennis L. Eggett
Ellis, Rod. 1997.
Second Language Acquisition
. Oxford: Oxford University Press.
NC
ED
FOR
IGN L
NGU
HROUGH
GL
BAL
EBATE
Willis, Jane R. 2004. “Perspectives on Task-Based Instruction: Understanding our Practices,
Acknowledging D椞erent Practitioners.” In
Task-Based Instruction in Foreign Language Education: Practice
and Programs
MATTH
university environment alongside native speakers of the respec
Teaching of Foreign Languages (ACTFL)
Superior-level language prociency. Such
an approach to foreign language study diers signicantly from many
Matthew B. Christensen and Dana S. Bourgerie
The Need for
Superior-Level Skills
For years, the expectation of foreign professionals conducting business in the United
States has been that they do so in English. However, Americans conducting business
CHI
NESE
FOR SP
EC
AL
URPO
SES
Matthew B. Christensen and Dana S. Bourgerie
classroom as well. For example, in a traditional Chinese classroom students view the
teacher as the ultimate authority and, as such, asking questions can appear disre
spectful. Likewise, Chinese education tends to emphasize theory over practice and
CHI
NESE
FOR SP
EC
AL
URPO
SES
of devoting an entire term to a specic domain. In recent years, there has been a
marked increase in the development and publication of Business Chinese materials.
Matthew B. Christensen and Dana S. Bourgerie
time since during a typical individualized session, which may last from fteen to fty
minutes, students work one-on-one with a teacher. With respect to speaking, tradi
tional classrooms also present a challenge in that the bulk of communication occurs
CHI
NESE
FOR SP
EC
AL
URPO
SES
Personalized tutoring from Chinese native peers
Cultural preparation course (for direct enrollment and internships)
Direct enrollment at a Chinese university (Nanjing University)
Four- to six-month internship in a Chinese organization (business, industry,
Matthew B. Christensen and Dana S. Bourgerie
Component
Description
Individual tutorials with faculty member
Language tutorial (50 minutes);
focus on linguistic issues
Content tutorial (50 minutes);
focus on content and cultural issues
Weekly group meeting
Focus on cultural literacy, professional
presentation skills, and résumand inter
view skills
Portfolio
Final term paper (in Chinese)
Accumulated domain-speci c lexicon
Article reviews
Final oral presentation
15-minute oral presentation to faculty
and peers
5-minute Q&A
Individual tutoring
Peer tutors (5 hours per week)
CHI
NESE
FOR SP
EC
AL
URPO
SES
Matthew B. Christensen and Dana S. Bourgerie
At the beginning of the semester, students select a research topic within their
domain that both interests them and that will be covered in courses related to their
major. Once the topic has been selected, students search the wiki and other sources
for articles that relate to their topic. Based on students’ individual backgrounds and
previous training, teachers and tutors help them select level-appropriate articles.
With the help of their instructors, students select one article per week and conduct
a careful, close reading of it. Once the lead instructor (content session) approves the
article, it is sent to the other instructor (language session) one week in advance of
the individualized sessions in order to allow time for instructor preparation.
The
purpose of the individualized language sessions is to ensure that students
understand the meaning of the articles they read on their own. To that end, teach
ers focus on grammar, vocabulary, and other structural issues. Students come having
read the article and noted areas where they need help. To expedite the comprehen
sion process, they use electronic dictionaries and annotators.
Graduate students
trained in Chinese language pedagogy usually conduct the individualized language
sessions and help students understand the social and cultural implications of the
language in the article. Naturally, the eectiveness and eciency of these sessions
depends largely on the degree of student preparation.
In addition to answering questions, the instructor asks leading and probing
questions related to grammar and language usage. Every three weeks, students com
CHI
NESE
FOR SP
EC
AL
URPO
SES
Because many of these peer tutors have experience within their Chinese elds—
either through undergraduate study or wor殗they prove invaluable as both cultural
guides and linguistic informants.
Matthew B. Christensen and Dana S. Bourgerie
utcomes and Assessment
Although the Flagship assessment routine has evolved over the ten-plus years of the
progra涒s existence, measuring student performance has always been part of the pro
gram. Accordingly, BYU Flagship has used a range of assessments to gauge student
progress and to inform curricular design. Early in the program, data collected largely
reected exit tests (ACTFL-OPI or Interagency Language Roundtable (ILR)-
OPI), but assessments have been adjusted according to demand and availability.
During the rst three years of the program, students were tested via informal
oral assessments and through interviews, but due to funding limitations, no full,
rated OPIs were administered. In the last seven years, the testing process has been
expanded considerably and students are now selected, in part, according to scores
on standardized prociency testing. To enter the advanced program, potential stu
dents must rate in the ACTFL Advanced range. They must also undergo in-house
CHI
NESE
FOR SP
EC
AL
URPO
SES
Table 5.3
ains Made on
FL-
after
omestic Program (
wo Semesters)
Proὣiency level gains
Number of students
Percentage of students
+1 sublevel
+2 sublevel
+3 sublevel
Null gain (Superior)
Null gain (AM or AH)
阱 sublevel
Total students
100
Total with rounding.
0.00000011
N = 34, excluding a participant who had already attained a Superior rating on entry to the program.
Table 5.4
Cross
abulation of
FL-
ains by Sublevel after
omestic Program (
wo Semesters)
Post-
Pre-
N = 35
Table 5.2
ssessments for the B
YU
Chinese Flagship Program
Entrance Tests
Baseline
on Acceptance
Post-Domestic/
Pre-Overseas
Capstone
Program Exit Tests
ART,
ALT
Informal interview
Writing sample
ACTFL-OPI
ACTFL-OPI
ACIE reading and
listening
ACTFL-OPI
ILR test
ACTFL-WPT
ACIE reading and
listening
HSK
100
Matthew B. Christensen and Dana S. Bourgerie
problems with the
CCALT that may have aected the results on both ends, perhaps
accounting for these outliers. Nevertheless, the pattern of gain seems as clear for
listening as it is for speaking.
Finally, the HSK test data echoe the results for ACTFL-OPI and CCALT. The
HSK tests several skills (vocabulary, listening, reading, and grammar), but Tables 5.7
and 5.8 show the global score on the original HSK test, i.e., they show the nature of
student gains, with all but one of thirteen students who took a pre- and post-test
gaining at least one level on the Intermediate or Advanced HSK (scale of 1–14 cov
ering three subtests).
Despite relatively small samples, outcomes for all three tests demonstrate clear
evidence of the eectiveness of the domestic training with respect to core linguistic
Table 5.5
ains Made on CC
after
omestic Program (
wo Semesters)
Proὣiency level gains
Number of students
Percentage of students
+5 sublevel
6.7
+4 sublevels
13.3
+3 sublevels
6.7
+2 sublevels
26.7
+1 sublevels
20
Null gain
6.7
-1 sublevel
13.3
-2 sublevels
6.7
Total students
15
100
Total with rounding.
0.00157
N = 15
Table 5.6
Cross-
abulation of CC
ains by Sublevel after
omestic Program (
wo Semesters)
Post-
Pre-
The Computer-Adaptive Listening Comprehension Test (CCALT) follows the ACTFL rating
system: Novice-Intermediate-Advanced-Superior, with low, mid, and high gradations. In this table,
we have AH=Advanced High, AM=Advanced Mid, AL=Advanced Low, IH=Intermediate High,
and IL=Intermediate Low.
N=15
CHI
NESE
FOR SP
EC
AL
URPO
SES
101
skills. It should also be noted that the higher learners progress on the scale, the more
dicult advancing to the next level becomes. In other words, crossing the Advanced
High/Superior threshold represents a far greater jump than, for example, crossing
the Intermediate High/Advanced Low threshold or making a sublevel gain from
Advanced Low to Advanced Mid.
Beyond the standardized measures, students have portfolios of work that attest
to the skills and abilities that they develop. These portfolios include video samples
of debate performances and of professional-level presentations to Chinese-speaking
audiences. Additionally, students produce a written term paper involving multiple
revisions, resulting in highly developed prose. Both the presentation and writing
skills become part of a foundation that allows participants to eventually participate
in direct enrollment courses in China and to successfully negotiate an internship in
a Chinese context—the ultimate goal of the program.
Table 5.8
Cross-
abulation of HSK
ains by Sublevel after
omestic Program (
wo Semesters) on Scale of 1–14
Post-
Pre-
The original HSK was divided into three tests: Beginner (1–3), Intermediate (3–8), and Advanced
(9–11). As a reference, a level 6 had been the requirement for a foreigner to directly enroll in a
Chinese university in the social sciences and humanities, whereas level 4 was the requirement for the
sciences. In 2010, the Chinese government released a new HSK with six separate tests.
N=13
Table 5.7
ains Made on HSK after
omestic Program (
wo Semesters)
Proὣiency level gains
Number of students
Percentage of students
+3 levels
7.7
+2 levels
30.7
+1 levels
53.8
Null gain
7.7
Total students
100
Total after rounding.
0.00014
N = 13
102
Matthew B. Christensen and Dana S. Bourgerie
Conclusion
Taking direct enrollment courses at universities in the People’s Republic of China
presents a daunting task, thus making individualized instruction and training prior
to entering the university classroom in China essential for students of the Chinese
individually based on perceptions of their own language ability.
eferences
Christensen, Matthew B. 2013. “Chinese for Special Purposes: An Individualized Approach.” In
Individualized Instruction in East Asian Languages
, edited by Etsuyo Yuasa, 39–59. Columbus, OH:
Foreign Language Publications, Ohio State University.
Christensen, Matthew B., and Xiaoqi Wu. 1993. “An Individualized Approach for Teaching False
Beginners.”
Journal of the Chinese Language Teachers Association
CHI
NESE
FOR SP
EC
AL
URPO
SES
103
Dowling, Carol, and Anita Mitchell. 1993. “Reading in a Specic Purpose Foreign Language Course: A
Case Study of Technical Japanese.”
Modern Language Journal
7: 443–44.
Falsgraf, Carl, and Dana Scott Bourgerie. 2008⸠“The Language Flagship: Multiple Approaches to
Creating Global Professionals.” In
U.S.-China Educational Exchange: Perspectives on a Growing Partnership
edited by Shepherd Laughlin, 83–97. New York: Institute of International Education
Grosse, Christine Uber, and Georey M. Voght. 1991. “The Evolution of Languages for Specic
Purposes in the United States.”
Modern Language Journal
75 (ii): 181–95.
Harlow, Linda. 1987. “Individualized Instruction in Foreign Languages: A Survey of Programs at the
College Level.”
Modern Language Journal
71: 338–94.
Hu, Wenzhong, Cornelius N. Grove, and Zhuang Enping. 2010.
Encountering the Chinese: A Modern Country,
An Ancient Culture
. 3rd ed. Boston and London: Nicholas Brealey Publishing.
Institute for International Education.
2013 Open Doors Report
. http://www.iie.org/Research-and-
Publications/Open-Doors.
Knorr, Walter L. 1977. “Individualized Specialization in Intermediate and Advanced Language Courses
in the Small College.”
105
Taking on the “Ceiling Effect” in Arabic
AS OF 2013, WELL
over thirty thousand students were pursuing the study of Arabic in
US colleges and universities, and extensive surveying indicates that most of those
students want to reach a level of prociency that would allow them to use Arabic
comfortably in their professional activities (Belnap and Nassif 2011, 5).
However,
there is little chance of timely success unless they can increase the amount of time
they spend on task to even more than that typically amassed by foreign language
majors in four years, and thereby break through the ceiling imposed by a typical
undergraduate course of study (Riin 2005). This breakthrough is doable; some
undergraduates are reaching Advanced High- and even Superior-level prociency
in Arabic (Al-Batal 2007–2008). The key? Extended periods of intensive study.
However, not all intensive study programs are created equal.
Time on task is likely to remain the most important factor, but innovative peda
gogical practices that result in higher levels of engagement and, therefore, expedited
learning warrant careful consideration. For example,
剩ṩ溒s analysis of Russian
data from on-campus and intensive contexts shows the latter to result in greater
gains in language prociency (2005). In their landmark study of thirteen hundred
study abroad students,
Vande Berg,
Connor-Linton, and
Paige (2009) found that
students who took content courses during a study abroad program achieved sig
nicantly higher gains in speaking and intercultural understanding than those who
enrolled in on-campus language courses. Highly engaging intensive experiences
presumably contribute to more time
on task
(i.e., to greater focus and actual use of
106
R. Kirk Belnap and Khaled Abuamsha
Pre-
Program Preparation
Participation in B奕’s study abroad program became a core requirement of its new
Middle East Studies/Arabic (MESA) major, launched in 2002, which has become
the most popular area studies major on campus, eclipsing Latin American Studies
in 2007.
Students are told on their rst day of Arabic 101 that in as little as twelve
months they could be stepping 漝 of an airplane to begin an experience designed to
result in the acquisition of Advanced-level prociency in both speaking and read
The nearness of this goal is underscored daily, given that most Arabic 101 and
102 students have an instructor or
AK
ING ON
HE “CEILING
FFE
CT
鐠IN A
RAB
107
they will be reading newspapers in Arabic on a regular basis, midway through Arabic
201, students begin reading daily headlines from the Arab press. Later in 202, short
newspaper readings become a part of the learners’ daily routine and they learn strat
108
R. Kirk Belnap and Khaled Abuamsha
AK
ING ON
HE “CEILING
FFE
CT
鐠IN A
RAB
109
on the few aspects they still do not understand well, but they try to do so rarely.
The last time through, they “pass it o” (i.e., they try to read it with full mastery,
110
R. Kirk Belnap and Khaled Abuamsha
Post-
ntensive Program Study
pportunities
to preceding years. For example, enrollments in fall semesters for 2008, 2009, and
Table 6.1
2011
ral Pro ciency Gains
Pre-Program OPI
Post-Program OPI (change)
Number of Students
Advanced Low
Advanced Mid (+1)
Advanced Low (0)
Intermediate High
Advanced Mid (+2)
Advanced Low (+1)
Intermediate High (0)
Intermediate Mid (–1)
Intermediate Mid
Advanced Mid (+3)
Advanced Low (+2)
Intermediate High (+1)
Intermediate Mid (0)
Intermediate Low
Advanced Low (+3)
Intermediate High (+2)
Intermediate Mid (+1)
Novice High
Intermediate Mid (0)
TOTAL:
AK
ING ON
HE “CEILING
FFE
CT
鐠IN A
RAB
2010, the semester that immediately followed the study abroad experience during
these years, were fty-one, forty-two, and forty-four, respectively. In short, we have
found that BYU students’ experiences during the study abroad program coupled
Course
Students Enrolled
Current Events (two sections)
Advanced Jordanian Arabic
Advanced Grammar (Close Reading)
15
Independent Readings
Classical Arabic Texts
Debate (for Arabic majors)
Tutorial (majors鈠research projects)
TOTAL:
Table 6.1
2011
ral Pro ciency Gains
Pre-Program OPI
Post-Program OPI (change)
Number of Students
Advanced Low
Advanced Mid (+1)
Advanced Low (0)
Intermediate High
Advanced Mid (+2)
Advanced Low (+1)
Intermediate High (0)
Intermediate Mid (–1)
Intermediate Mid
Advanced Mid (+3)
Advanced Low (+2)
Intermediate High (+1)
Intermediate Mid (0)
Intermediate Low
Advanced Low (+3)
Intermediate High (+2)
Intermediate Mid (+1)
Novice High
Intermediate Mid (0)
TOTAL:
112
R. Kirk Belnap and Khaled Abuamsha
December 2011
August 2011 OPI (change)
Number of Students
Advanced Mid
Advanced High (+1)
Advanced Mid (0)
Advanced Low
Advanced Mid (+1)
Advanced Low (0)
Intermediate High
Advanced Mid (+2)
TOTAL:
AK
ING ON
HE “CEILING
FFE
CT
鐠IN A
RAB
113
fellowships oered (from forty-six in 2011/12 to thirty in 2013/14). In addition, the
unrest in Egypt resulted in CASA’s relocating to Jordan in 2013 (see Belnap and
ing deliberate practice in designed practice environments” (2009, 18). Other
114
R. Kirk Belnap and Khaled Abuamsha
may seem exceptional, we are convinced that many students could have similar expe
riences, especially with appropriate guidance.
Findings from this research underscore the importance of extended periods of
quality, intensive learning experiences as a key to students’ achieving a high level of
prociency in a timely manner. With
Pellegrino Aveni (2005),
Vande Berg,
Linton, and
Paige (2009),
Trentman (2012), and others, we emphasize that signicant
pre-program interventions, well-designed overseas learning experiences, and on-site
coaching lead to high levels of engagement, self-awareness, and deliberate practice.
otes
This estimate is based on the Modern Language Association’s 2009 survey and annual reports from
a sample of institutions to the National Middle East Language Resource Center, as discussed in
Belnap and Nassif (2011, 1–2).
In addition to those mentioned by name in this chapter, the authors must also thank Spencer
Scoville and Shereen Salah who oversaw the implementation of B奕’s new Arabic major.
to be key components of the BYU Arabic program.
AK
ING ON
HE “CEILING
FFE
CT
鐠IN A
RAB
115
Pre- and post-program OPIs and signicant attention during the program devotes what the test
measures have helped students focus during and beyond the program and awakened as well as
increased their awareness of what is required to reach the Superior level.
OPIs represent an important part of the program, but the experience has also been frustrating.
Telephone conversations are not ideal, especially when the quality of the connection cannot be guar
116
R. Kirk Belnap and Khaled Abuamsha
Ericsson, K. Anders, Ray S. Perez, David W. Eccles, Laura Lang, Eva L. Baker, John D. Bransford,
Kurt VanLehn, and Paul Ward. 2009⸠“The Measurement and Development of Professional
Performance: An Introduction to the Topic and a Background to the Design and Origin of This
Book.” I渚
Development of Professional Expertise: Toward Measurement of Expert Performance and Design of
Optimal Learning Environments
, edited by K. Anders Ericsson, 1–24. New York: Cambridge University
Press.
Mills, Nicole. A., Frank Pajares, and Carol Herron. 2007⸠“Self-Ecacy of College Intermediate French
Students: Relation to Achievement and Motivation.”
Language Learning
57 (3): 417–442.
Pellegrino Aveni, Valerie. 2005.
Study Abroad and Second Language Use: Constructing the Self
. Cambridge:
Cambridge University Press.
Riin, Benjamin. 2005. “A Ceiling Eect in Traditional Classroom Foreign Language Instruction: Data
from Russian.”
The Modern Language Journal
89 (1): 3–18.
Trentman, Emma Gale. 2012. “Study Abroad in
Egypt: Identity, Access, and Arabic Language Learning.”
117
Federally Supported Overseas Language
Training Programs for Americans
research over the past two decades has demonstrated that sec
ond language (L2) acquisition in the overseas instructed/immersion environment
118
Dan E. Davidson
teachers of the critical languages. None of the new programs include a US govern
ment service requirement.
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
119
language study? How do language gains in reading, listening, and speaking vary
across summer and academic-year program durations for participating students?
To what extent does
initial level
of prociency aect L2 gain in the overseas
NSLI-Y Summer and Academic-Year (AY) Programs (N = 523)
Length of Program
Age
Gender
Prior Language Study
Summer N = 479
Mean Age 16.6
Female 61.0%
Yes 58.9%
AY N = 44
Mean Age 17.5
Male 39.0%
No 41.1%
CLS/Summer Program (N = 620)
Educational Level
Age
Gender
Prior Language Study
2/4-year colleges: 70%
Mean Age 21.1
Female 60%
Yes 61%
Graduate: 30%
Mean Age 23.3
Male 40%
No 39%
Flagship Capstone Program (N = 314)
Educational Level
Age
Gender
Prior Language Study
Undergraduate: 89%
22.9
Female: 52%
Yes 100%
Post-BA: 11%
24.1
Male 48%
No 0%
120
Dan E. Davidson
adaptability, and commitment to future study of the language is taken into consider
ation by external selection committees within the review process. Except for Flagship
Capstone, which requires demonstrated Advanced-level prociency (ILR-2) in at
least two language modalities, selection for the overseas programs is not based on
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
121
(minimum summer-length) in the host country prior to applying for the year-long
program.
Academic Components (Age Appropriate)
Predeparture participant orientation
Intensive language training in small groups (12–15 hours per week)
122
Dan E. Davidson
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
123
actually loses information, in eect introducing a measurement error, in that stu
dents with quite dierent (unobserved) prociency levels may be assigned the same
speaking, reading, or listening prociency scores.
Novice Low is the default prociency assigned to all students entering the
critical languages after two (or more) semesters of regular academic study.
NL – Novice Low
NM – Novice Mid
NH – Novice High
IL – Intermediate Low
IM – Intermediate Mid
IH – Intermediate High
AL – Advanced Low
AM – Advanced Mid
AH – Advanced High
igure 7.1
All
NSLI
Programs Pre- and Post-Program
cores for 2011, 2012, and 2013
ummer Programs
124
Dan E. Davidson
the right one column (or two, or three) indicates a gain of a sublevel or threshold
on the prociency scale. Linear models, such as those used here, do not fully capture
the three-dimensional nature of the inverted pyramid used by specialists to model
the prociency scale and the reality that L2 gains in the upper ranges of the scale are
considerably more dicult to achieve than those at the lower range.
Pre-Program
Speaking
Proὣiency
Level
Post-Program Speaking Proὣiency Level
Total
163
0.60
16.60
36.80
30.10
11.00
4.30
0.60
100.00
22
5.50
40.00
34.50
20.00
100.00
36
86
5.80
45.30
41.90
7.00
100.00
15
23.80
51.30
18.80
5.00
1.30
100.00
42.40
44.10
8.50
5.10
100.00
36.00
52.00
12.00
100.00
37.50
50.00
12.50
100.00
100
100.00
100
100.00
Total
126
131
479
0.20
6.30
18.20
26.30
27.30
13.20
5.40
2.90
0.20
100.00
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
125
remained in the
Novice Mid (NM) range, with only one student failing to register a
measureable gain in speaking.
By comparison, fewer than 10 percent of participants in the Chinese NSLI-Y
summer-program sample entered the program without prior formal study, with
igure 7.2
ral Pro ciency
utcomes in the Arabic
NSLI
ummer Programs
126
Dan E. Davidson
igure 7.3
ral Pro ciency
utcomes in the
hinese
NSLI
ummer Programs
igure 7.4
ral Pro ciency
utcomes in the Persian (
Tajiki)
NSLI
ummer Programs
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
127
those of the NSLI-Y population as a whole, and prepared participants to continue
their study of the language at the second- or third-year level. Two-thirds of the group
Pre-Program
Speaking
Proὣiency
Level
Post-Program Speaking Proὣiency Level
Total
42.90
42.90
14.30
100.00
10.30
37.90
27.60
24.10
100.00
22
24
5.50
40.00
43.60
10.90
100.00
23.50
49.00
21.60
5.90
0.00
100.00
44.20
39.50
9.30
7.00
100.00
29.40
52.90
17.60
100.00
0.60
40.00
100.00
100.00
100.00
Total
209
2.90
8.10
20.10
35.90
19.10
9.10
4.80
100.00
128
Dan E. Davidson
Intermediate range into the Advanced range requires considerable time, as is evi
dent from the rising values of null-gain gures in Table 7.4 for IL, IM, and IH.
igure 7.5
ral Pro ciency
utcomes in the
ussian
NSLI
ummer Programs
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
129
Pre-Program
Speaking
Proὣiency
Level
Post-Program Speaking Proὣiency Level
Total
14.90
47.30
35.10
1.40
1.40
100.00
22
40.90
40.90
18.20
100.00
6.50
54.80
38.70
100.00
28
25.00
57.10
10.70
3.60
3.60
100.00
37.50
56.30
6.30
100.00
42.90
57.10
100.00
50.00
50.00
100.00
0.00
100
100.00
Total
46
181
6.10
25.40
32.60
21.50
8.80
3.30
1.70
0.60
100.00
130
Dan E. Davidson
igure 7.7
Pre- and Post-Program
istening
cores for
NSLI
ummer 2011, 2012, and 2013
igure 7.6
Pre- and Post-Program
eading
cores for
NSLI
ummer 2011, 2012, and 2013
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
131
ro ciency
utcomes in the
LI
cademic-
ear
rograms
The NSLI-Y academic-year programs induct a much smaller number of students
annually than do the summer programs (see Appendix 1 for enumeration, eligibility,
and conditions). Most students in this program enter with prior training and with
prociencies in the NM/NH range. Typically, only one-third of the participants
begin the year of overseas study without any prior study of the language.
The typical post-program-measured oral prociency is Advanced Low (AL)
⠴㠜percent), Intermediate High (IH) (27 percent), and Advanced Mid (AM)
132
Dan E. Davidson
graduate students with the opportunity to undertake or continue study of one of
thirteen critical languages (see Appendix 2 for enumeration, eligibility, and condi
tions) at a structured overseas language and cultural immersion program or summer
igure 7.8
Pre- and Post-Program Pro ciency Test
esults for
hinese and
ussian
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
133
prociency scores remained at the Intermediate level throughout the program typi
cally reveal a remarkable range of dierences in the latter, including more native-
like uency, sociopragmatic skills, cultural referencing, and a considerably expanded
vocabulary evident in the speech production in the post-program learner (Davidson
1982; Magnan and Back 2007; Martinsen 2010).
Table 7.5 presents a cross-tabulation of language-gain data comparing pre- and
post-program OPI scores by participant numbers and percentages (N = 620).
Measured gain for each entering level of L2 prociency can be gauged by the distri
bution of scores to the right (positive gain) or left (negative gain) of the diagonal.
igure 7.9
CLS
Pre- and Post-Program
ral Pro ciency
core
omparisons
134
Dan E. Davidson
re- and
ost-Testing Results: The
verseas
lagship
apstone
rogram
Within the triumvirate of new federal programs supporting the development of US
understanding and expertise in major world languages and cultures, the
Overseas
Flagship Capstone programs (OFC) occupy a special position. Their task is to pro
duce Level-3 speakers in ten critical languages who are well-grounded in language,
culture, and regional knowledge of the host country and can function at a profes
sional level in business, government, research, academia, or other sectors of the
Pre-
Program
Speaking
Proὣiency
Level
Post-Program Speaking Proὣiency Level
Total
8.3
19.0
46.4
22.6
3.6
100.00
20
1.5
9.2
30.8
44.6
12.3
1.5
100.00
46
1.1
18.5
50.0
19.6
9.8
1.1
100.00
28
6.1
41.8
28.6
18.4
5.1
100.00
22
139
21.6
33.8
28.1
15.8
0.7
100.00
22
24.1
40.5
27.8
7.6
100.00
28
19.0
66.7
14.3
100.00
10.5
52.6
31.6
5.3
100.00
100.0
100.00
Total
165
123
108
21
620
1.3
3.7
13.2
26.6
19.8
17.4
14.4
3.4
0.2
100.00
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
135
experience that draws in varying degrees, depending on the host country, on the
full range of intervention strategies and support systems outlined previously. The
OFCs emphasize small-group language training and individual tutorials but place
increased stress on direct enrollment in courses at the overseas university as well as
professional-level internships. The OFC places a special emphasis on in-class and
public presentational or project work.
Table 7.6
2012
CLS
Test
cores
All Languages
Frequency
Percent
Valid Percent
Cumulative
Percent
Loss
0.3
0.3
0.3
Null
12.4
12.4
12.7
Null
12.7
12.7
12.7
1 unit
135
21.8
21.8
34.5
2 units
8.1
8.1
42.6
1 threshold
345
55.6
55.6
98.2
2 thresholds
1.8
1.8
100.0
Total
620
100.0
100.0
igure 7.10
CLS
Gain
ategories
136
Dan E. Davidson
Figure 7.11 presents the pre- and post-program OPI score reports for the several
multi-institutional OFC programs. The time period reected in these data is six
years: 2007–2012.
The dark bars reect measured entering oral prociency of Flagship students,
while the light bars reect the distribution of post-program OPI scores. It should
be noted that in the earlier years of the program, the requirement of 2-level OPI
prociency was not uniformly enforced across all programs. As with other forms of
igure 7.11
Pre- and Post-Program
core
omparisons: Arabic,
hinese,
Persian, and
ussian
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
137
Table 7.7
Pre- and Post-Program
peaking
cores for
NSLI
ummer 2011, 2012, and 2013
Pre-Program
Speaking
Proὣiency Level
Post-Program Speaking Proὣiency Level
Total
63.60
27.30
100.00
28
32.10
25.00
42.90
100.00
106
22
195
6.20
21.00
54.40
11.30
6.70
0.50
100.00
4.30
11.40
58.60
12.90
11.40
1.40
100.00
10.00
70.00
10.00
10.00
100.00
Total
167
22
314
9.90
19.10
53.20
10.20
7.00
0.60
100.00
Table 7.8
verseas
lagship Programs:
eading Pro ciency
cores for All
earners (
= 238), Pre- and Post-Program
ount/
ow Percent)
Pre-Program
Reading
Proὣiency Level
Post-Program Reading Proὣiency Level
Total
25.00
50.00
25.00
100.00
29.40
47.10
23.50
100.00
126
6.30
37.30
39.70
14.30
2.40
100.00
18.30
68.30
10.00
3.30
100.00
7.10
28.60
64.30
100.00
Total
20
103
238
0.40
8.40
31.90
43.30
13.90
2.10
100.00
138
Dan E. Davidson
As noted previously, reading and listening prociency testing of OFC was
introduced beginning in 2009, so the N-value in Figure 7.12 is lower than in the
OPI report. The pre- and post-testing reading prociency results present a pattern
similar to the OPI speaking inputs and outcomes. A slightly larger concentration
(㌲�percent) of scores at the 2+ threshold level is noted for this modality in compari
son to speaking (19 percent). The percentage of OFC students with scores of 3+ or
higher in speaking and reading is 18 percent and 16 percent, respectively.
A total of 59.9 percent of all OFC students achieved Level-3 or higher scores in
reading prociency, including a group of 11 (18.3 percent) students who entered the
program with Level 2+-prociency in reading.
Listening prociency outcomes for all OFC programs are presented in Figure7.13.
Listening prociency outcomes for the OFC groups as a whole parallel OPI
outcomes to a very high degree—much more closely, in fact, than was the case with
igure 7.12
eading Proὣiency
utcomes:
verseas
lagship
apstone Programs
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
139
Given the considerable dierences in local linguistic and cultural conditions
across the major OFCs, breakout data for the
Russian OFC alone are provided in
Figures 7.14–7.16.
The cross-tabulations here present the input and output rates of the Russian
OFC since its inception. The program currently has a null-gain rate of 8.57 percent,
with production rates of 62.86 percent for Level 3 and 27.63 percent for 3+ and
above (see Table 7.9).
Reading and listening prociency results in the Russian OFC generally exceed
those for speaking, with 38 percent of the group achieving 3+ or higher results in
listening and 54 percent achieving 3+ or higher in reading.
The Arabic OFC pre-/post-program oral prociency test results are presented
in Figure 7.17. Over the past six years, 84 percent of students have entered the
program with OPIs in the Advanced-level (ILR-2) range, while 71 percent of all
igure 7.13
Pre- and Post-Program
istening Pro ciency
utcomes:
OFC
140
Dan E. Davidson
igure 7.15
Pre- and Post-Program
eading Pro ciencies:
ussian
OFC
igure 7.14
ussian
verseas
lagship, 2004–2012
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
141
igure 7.16
Pre-and Post-Program
istening Pro ciencies:
ussian
OFC
Table 7.9
ussian
verseas
lagship
peaking Pro ciency
cores for All
earners (
= 105), Pre- and Post-Program
ount/
ow Percent)
Pre-Program
Speaking
Proὣiency Level
Post-Program Speaking Proὣiency Level
Total
1.00
100.00
33.33
16.67
50.00
100.00
7.46
64.18
17.91
8.96
1.49
100.00
65.52
17.24
17.24
100.00
50.00
50.00
100.00
Total
105
1.90
6.67
62.86
17.14
10.48
0.01
100.00
142
Dan E. Davidson
igure 7.18
eading Proὣiencies: Arabic
OFC
igure 7.17
Pre- and Post-Program
ral Pro ciencies: Arabic
OFC
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
143
71 percent of the group achieved the Professional level in speaking, and 70 percent
achieved the Professional level in listening comprehension. In comparison, 51 per
cent achieved the Professional level in reading, while 38 percent tested at Level 2+.
The teaching and testing of Arabic reects both the diglossic and, in some cases,
bidialectal nature of the host country sites where Flagship students are placed. These
complexities are reected in the testing environment for Arabic as well.
Results of the Chinese OFC are presented next; however, for technical reasons,
only one year of program data is available at this time: 2012–2013.
While the prociency measurements of entering Chinese OFC students were
relatively strong in comparison to all other OFCs, the production of Level-3 speak
ers (and above) was somewhat lower than both the Arabic and the Russian pro
grams. On the other hand, no participant in the Chinese OFC scored lower than 2+
in speaking at the close of the program (see Figure 7.20).
Reading and listening prociency ratings for Chinese
OFC students show that
31 percent of the cohort attained Level-3 prociency in reading, while 38 percent
achieved that level in listening comprehension (see Figures 7.21 and 7.22). Entering
levels for both modalities were primarily at 2. A majority of participants in the pro
gram were rated at 2+ on the post-program test in reading, while program-nal lis
tening tests also included a cohort of 25 percent of the learners who scored at Level
2. Seven out of thirty-two students registered null gain in listening. Speaking scores
exceeded reading or listening scores for the group as a whole by more than twenty
igure 7.19
istening Pro ciencies: Arabic
OFC
144
Dan E. Davidson
igure 7.20
Pre- and Post-Program
ral Pro ciencies:
hinese
OFC
igure 7.21
Pre- and Post-Program
eading Pro ciencies:
hinese
OFC
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
145
percentage points. It should be noted that the testing of reading at Level 2+ and
above requires knowledge of both traditional and simplied characters.
Discussion
The data presented here and elsewhere in this volume make clear that the pathway to
igure 7.22
Pre- and Post-Program
istening Pro ciencies:
hinese
OFC
146
Dan E. Davidson
intensive summer immersion model, implemented by the NSLI-Y programs
for seven major world languages. We have seen that the NSLI-Y academic-year
program is producing L2 speakers who test at the Advanced level, a level of
language mastery typical of upperclassmen and graduating seniors at many of
our major universities. We have seen the summer
CLS program for university
students providing a similar opportunity for achieving threshold gains to the
Intermediate or Advanced level for students of a wide range of language-study
backgrounds across thirteen languages. A 50+ percent success rate in produc
ing Intermediate-level speakers from Novices, and Advanced-level speakers
from Intermediates, is a notable achievement in second language acquisition
in the context of a summer program, a time frame that has not previously been
regarded as sucient for threshold-level gains. Finally, in response to the need
to produce greater numbers of Americans capable of functioning as “global pro
fessionals,” we have observed that it is possible to reach Level 3 and above on
a systematic basis, year after year, with US undergraduate students who have
attained Level 2 at the time of their participation in the program in the year-
long OFC model.
We have also seen that reading and listening gains generally align with the acqui
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
147
Richa牤�D⸜Brecht (2013, 4) summarized the views of the conference, starting with
the assumption that all Americans should have access to L2 training and that the
country as a whole will require persons of diering levels of prociency, depending
148
Dan E. Davidson
and Cultural Aairs (ECA) or an employee at an NSLI-Y administering orga
nization whose duties involve the NSLI-Y program
Have not previously traveled outside the United States on a long-term (more
than eight weeks) program sponsored by the Bureau of Educational and Cultural
Aairs, Department of State
Previous NSLI-Y
program participants or participants of ECA-funded
short-term programs are only eligible to apply for a NSLI-Y academic-year
program.
Previous language study is not a requirement. Students of all levels of language
ability are encouraged to apply.
The NSLI-Y program seeks applicants who represent the diversity of the United
TH
LOPM
OF
PR
OFICI
NCY AN
TER
ACY…
149
ppendix 3
The Language Flagship Capstone Program
www.thelanguageagship.org
The
Language Flagship Capstone full-year immersion program is open to all under
graduate students who are committed to attaining Professional- or Superior-level
language prociency through an intensive language training program tailored to
their professional interests and academic specialization. It may occur during the
third, fourth, or fth year of a stude湴’s undergraduate program. The model also
assumes that, in addition to full-year study, some students will require additional
periods of immersion overseas to accelerate their language learning and to accom
modate academic schedules. The program also accepts applications from part-time,
non-degree-seeking students.
Applicants should have a strong academic record, a demonstrated interest in
advancing their Arabic, Russian,
Persian, Chinese,
Hindi/
Urdu, Korean,
Portuguese,
Russian,
Swahili, and Turkish skills and using these languages in their future career,
and a desire to share their understanding of this language and culture within the
larger community.
Undergraduate Program
All students who are enrolled at one of the Domestic Programs and reach the
required prociency level ILR-2 in their language are accepted to the Overseas
Program, upon recommendation of the Overseas Academic Council.
Post-BA Program
This program accepts applicants who did not participate in a Domestic Flagship
program and already have a bachelor’s degree. The participants are selected on the
basis of their language skills, academic merits, work experience, and ability to dem
onstrate how advanced Russian skills are going to help their career plans. Applicants
to the Flagship Post-BA program must either possess a BA degree or expect to
receive one before starting the program. Successful applicants who are not heritage
150
Dan E. Davidson
151
Part Three
153
From Prociency to Expertise: Using
154
Patrick McAloon
FRO
PRO
IEN
ER
ISE…
155
Oral Prociency Interviews appear to be acontextual, meaning that the con
text defaults to that of a tester chatting with an interviewee. For example, in one
sample Advanced-level response, the interviewer asked the interviewee to discuss
what people in White Plains, New York, are talking about.
The most appropriate
conversation on this topic should vary depending on who is talking and why. One
would anticipate that the language used by a congressman from White Plains in a
156
Patrick McAloon
FRO
PRO
IEN
ER
ISE…
157
tion needs and seeks to develop; second, it requires the involvement of more people
than traditional supervisor-subordinate performance reviews, thus complicating the
collection and analysis of data.
The feedback tool must be carefully designed to include the organization’s tar
158
Patrick McAloon
common behaviors. Many vice presidents in banks have few or no people who report
directly to them, whereas vice presidents in manufacturing companies have many.
Their relative needs for negotiating with subordinates versus peers may be quite
dierent, and these dierences should be taken into account when an organization
designs a 360-degree feedback tool. The skill category, in particular, may be highly
considerable amount of time. The raters must be identied and invited to partici
Figure 8.1
Working De nitions for the
ypes of Data Collected by 360-Degree Feedback (from Lepsinger and
Lucia 1997, 10)
FRO
PRO
IEN
ER
ISE…
159
Advantages of
360-Degree Feedback
Despite the time and expense involved in implementing 360-degree feedback, there
are a number of advantages to using such a process for assessing professional per
formance. These advantages include adjusting the attitudes of participants toward
their professional roles and increasing the ease with which the assessments can be
converted into constructive behavioral change.
Perhaps the most important advantage of 360-degree feedback systems, from
the standpoint of foreign language educators, is that such systems help partici
Consider this manager’s effectiveness in the following items. How sati猟ed are you
with the way this manager…
Highly Dissati猟ed
Dissatis ed
Neither Sati猟ed
or Dissatis ed
Satis ed
Highly Sati猟ed
No
Information
Serving Customers
1. Creatively responds
to customers’ needs
2. Responds to
customers’ needs in a
timely manner
3. Works to meet
commitments that
have been made to
customers
Processes Are the Problem, Not People
1. Involves employees
in decisions that a�ect
their work
2. Asks people what
they need to do their
work better
3. Strives for continu
ous improvement
eprinted from
aximizing the Value of 360-Degree Feedback:
Process for Successful
ndividual and
rganizational
Development, p. 79,
xhibit�4.1, with permission from Jossey-Bass Publishers/
Wiley
mprint. (c) 1998.
Figure 8.2
xample 360-Degree Feedback
ating
tems
160
Patrick McAloon
assessments of educators can fall into the same trap as employees assessed exclusively
FRO
PRO
IEN
ER
ISE…
161
test subject’s eld feel about his or her professional performance (what we can call
expertise).
Information regarding how an organization denes expertise can be commu
nicated via assessment tools themselves. AnalogouslyⰠ“expert students” focus their
eorts on mastering material that will be covered on the test, and the same can be
said of aspiring professionals. Three hundred sixty–degree feedback can guide per
162
Patrick McAloon
FRO
PRO
IEN
ER
ISE…
163
In ALPPS, raters evaluate performance samples based on a mastery scale. For
example, after a portfolio evaluator watches a video of an individual giving a speech
Rubric in ALPPS English evaluation
interface
Rubric in ALPPS Chinese
evaluation interface (English translations
in italics added for this book)
Superior Competence/Superior
Performance
表现非常好
Excellent performance
Strong Competence/Strong Performance
表现挺好
Very good performance
Competence/Passable Performance
表现不KI
Pretty good performance
Some Competence/Some Performance
表现一8\
So-so performance
Minimum Competence/Minimum
Performance
表现有进步
Performance shows improvement
No Competence/No Performance
表现差
Poor performance
Figure 8.3
Sample
LPPS
valuation
ubric (
loon 2008, 229)
164
Patrick McAloon
organization’s requirements, “native-speaking colleague” could be further subdi
vided into rater categories such as boss, peer, or subordinate.
Tools like ALPPS are particularly
prospective
employers of foreign lan
guage speakers. The rst piece of information that an employer receives about an
applica湴’s language ability is a self-assessment on the applica湴’s résumé, where self-
promotion may take precedence over accuracy. Some résumés include third-party
ratings such as a
Test of English as a Foreign Language (TOEFL) score for English
Hanyu Shuiping Kaoshi (HSK) score for Chinese. These ratings can estab
FRO
PRO
IEN
ER
ISE…
165
Finally, foreign language learners benet from the use of feedback tools like
166
Patrick McAloon
program. However, at the time Lepsinger and Lucia expressed this opinion, record
ing and replaying videos of professional performances was inconvenient. Now that
videos are more easily shot, stored, and replayed in digital format, establishing a
minimum period of familiarity for portfolio raters may no longer be needed. In fact,
knowing the subjects may be preferable for raters, as personal experience with
them could make already subjective ratings more subjective.
In an ideal world, at least ten evaluators would assess a portfolio evaluation sub
ject and additional evaluations would follow for the duration of the subject’s work
ing life. In the real world, however, evaluators can experience evaluation fatigue if
asked to evaluate more than three people at a time (Lepsinger and Lucia 1997, 121),
and native-speaking colleagues of nonnative speakers may be in such diverse loca
FRO
PRO
IEN
ER
ISE…
167
Arguably, the most eective way to evaluate a learner’s ability to construct a
second-culture worldview and translate it into a culturally appropriate action requires
observing the action itself. With portfolios that hold samples of performances recorded
at dierent times, systems like ALPPS can generate reports that show how portfolio
subjects have progressed in producing performances that align with native audiences’
expectations.
A further advantage of the transparency of multi-rater portfolio systems is that
ratings can be isolated by rater identity. Portfolio evaluation reports can compare
ratings made by industry insiders, such as peers and superiors, against those of indus
try outsiders, such as language instructors. Each of these comparisons can provide
valuable data to program administrators, instructors, employing organizations, and
the portfolio subjects themselves, who generally want to leave a good impression.
If a portfolio syste涒s reports show that instructors are consistently giving learn
ers higher ratings than potential or actual professional colleagues, the instructors
can adjust their program of instruction accordingly, e.g., the addition of domain-
specic eld experience versus additional class work under the direction of language
instructors.
Looking Ahead
Particularly among the less commonly taught languages, the American foreign
language education community has largely left the achievement of professionally
By
ari
oda and Galal Walker.
eprinted by permission of the authors.
Figure 8.4
Walker and
oda’s Cycle of Compilation
168
Patrick McAloon
useful levels of foreign language ability up to individual learners after they gradu
ate. Likewise, responsibility for assessing the uppermost levels of foreign language
ability and their specic professional applications have been left to organizations
possessing a critical mass of foreign language evaluators, such as the Department
FRO
PRO
IEN
ER
ISE…
169
6.
This rubric was taken from a 2006 iteration of ALPPS. Notice that the Chinese and English rubrics
do not exactly match, which would be a cause for concern when one is comparing evaluations sub
mitted by Chinese speakers and English speakers, i.e., nonnative speakers of Chinese.
7.
Since the original publication of the chart, Walker and Noda have since added “themes” to cases
and sagas.
eferences
171
HT, WI
IAM P. RIVE
, JOHN P. RO
ON,
. DAVI
overview of the current state of language education at the
advanced levels in the United States, starting with a brief assessment of the need
for language skills, driven by the decade-long increase in demand in the public and
private sectors for language professionals—individuals with the ability to perform as
professionals in a second language across a wide range of disciplines. As perspective
172
Richard D. Brecht, William P. Rivers, John P. Robinson, and Dan E. Davidson
War and the subsequent
National Defense Education Act of 1958 (Brecht and Rivers
2012). To this end, the US government has built an extensive recruitment and lan
guage training system for defense, diplomacy, and intelligence needs, with announced
goals of producing graduates with professional levels of language prociency (at
or above the 2+ level on the
Interagency Language Roundtable [ILR] Prociency
Guidelines). While a series of
Government Accountability Oce (GAO) reports
and congressional hearings cite linguistic deciencies across the government (see,
for example, United States Government Accountability Oce 2010), there is little
reason to expect this focus on high-level skills to change for the foreseeable future.
The US government has well-documented needs across more than eighty agencies
and for an equally large number of languages. The general requirements are at pro
fessional levels of prociency in two or more languages (English and one other), as
Industry Needs
What has changed dramatically in the last decade or so is the fact that the language
industry, and the major sectors of the US economy that it serves, faces a major talent
gap. The
language industry continues to grow at 8–10 percent per year, some three-
to-four-times faster than the overall US economy, and is worth at least $15billion
per year (Kelly and Stewart 2011). Industry experts and observers expect this growth
to continue, if not accelerate, due to the explosion in content, particularly from social
media and the use thereof by major industry clients. Accordingly, there is an intense
FESSI
LA
GUAG
SK
ILLS
UNPRECEDENTED
ND
ND
SU
PPL
173
least intersects with STEM disciplines in novel ways. Among these intersections are
the increasing use of blended learning integrated into FL programs; the high degree
of technology use in translation, which as an industry relies on terminological data
bases, mixed stochastic and rule-governed machine translation, and crowd-sourced
translations as enablers and adjuncts to human translators; and web-based confer
tion of higher education prior to study abroad. Moreover, as the data collected by
ACTR include the number of years of study, we can gauge an expected prociency
outcome from four years of Russian study. The results of nearly forty years of data
collection indicate a remarkably stable, and disappointingly low, median prociency
of Intermediate High, roughly equivalent to an ILR rating of 1+, across listening,
174
Richard D. Brecht, William P. Rivers, John P. Robinson, and Dan E. Davidson
reading, and speaking (Davidson 2010). Here are the
results of data collected for
the years 1994–2009:
Speaking score after a minimum of six semesters of study or equivalent: Mean
is 4.58 (standard deviation [SD 1.55]) for semester students; 4.78 (SD 1.70) for
academic year students, where
5.0 = Intermediate Low
Reading score after a minimum of six semesters of study or equivalent: Mean is
7.03 (SD 2.27) for semester students; 7.43 (SD 2.46) for academic year, where
7.0 = Intermediate High
Listening score after a minimum of six semesters of study or equivalent: Mean
is 5.72 (SD 1.57) for semester students; 5.80 (SD 1.47) for academic year, where
6.0 = Intermediate Mid
Without
study abroad, US Russian programs have consistently produced
FESSI
LA
GUAG
SK
ILLS
UNPRECEDENTED
ND
ND
SU
PPL
175
a signicant number of students receive basic communication skills in some foreign
language through
Foreign Language Elementary School (FLES) or immersion pro
gramming. Following this initial exposure, secondary schools would be responsible
for providing a majority of their learners with usable language skills. Colleges and
universities would then be in a position to graduate global professionals who are
able to practice across the international environment with adequate language skills.
Finally, a combination of undergraduate and graduate programs in language and
literature would produce the expertise that would enable practice at all the afore
mentioned levels. That this language-education system is possible depends on many
factors, not the least of which is broad-based support from parents and other con
stituents of the voting public, as discussed previously.
Pathway:
High-Quality Intensive Instruction
The characteristics of high-quality intensive instruction are a staple of higher educa
Global Awareness & Appreciation
Figure 9.1
Language
apacity for
lurilingual
itizens: The
yramidal Base
176
Richard D. Brecht, William P. Rivers, John P. Robinson, and Dan E. Davidson
and ecient. These developments, which we will call “enablers,” include (1)broad
public support for language education across the United States, (2)research in cog
nitive science and second language acquisition, (3) language learning technology,
(4) best practices in primary and secondary FL education, and (5) best practices in
higher education.
Public Support
FESSI
LA
GUAG
SK
ILLS
UNPRECEDENTED
ND
ND
SU
PPL
177
Table 9.1
2000–2013
hanges in Foreign Language
olicy Questions
1. Do you favor a law making English the ofὣial language of the United States, or
do you oppose such a law?
2000
Favor: 78%
Oppose: 22%
2008
Favor: 72%
Oppose: 28%
2013
Favor: 81%
Oppose: 19%
2. Please tell us whether you strongly agree, agree, disagree, or strongly disagree
with each of these statements:
a.
Children in the
nited States should learn a second languag攠 uently before
the礠�nish high school.
2000
Strongly Agree: 27%
Agree: 49%
Disagree: 22%
Strongly Disagree: 3%
2008
Strongly Agree: 40%
Agree: 40%
Disagree: 15%
Strongly Disagree: 6%
2013
Strongly Agree: 17%
Agree: 49%
Disagree: 30%
Strongly Disagree: 4%
b.
Bilingual education programs should be eliminated in
merican public schools.
2000
Strongly Agree: 6%
Agree: 16%
Disagree: 50%
Strongly Disagree: 28%
2008
Strongly Agree: 10%
Agree: 13%
Disagree: 41%
Strongly Disagree: 36%
2013
Strongly Agree: 10%
Agree: 20%
Disagree: 49%
Strongly Disagree: 21%
c.
Speaking English as the common national language is what unites all
mericans.
2000
Strongly Agree: 26%
Agree: 50%
Disagree: 21%
Strongly Disagree: 3%
2008
Strongly Agree: 39%
Agree: 38%
Disagree: 17%
Strongly Disagree: 6%
2013
Strongly Agree: 36%
Agree: 44%
Disagree: 16%
Strongly Disagree: 46%
d.
Learning a foreign language is as valuable as learning math and science in
school.
2000
Strongly Agree: 21%
Agree: 43%
Disagree: 31%
Strongly Disagree: 5%
2008
Strongly Agree: 32%
Agree: 36%
Disagree: 24%
Strongly Disagree: 8%
2013
Strongly Agree: 24%
Agree: 47%
Disagree: 25%
Strongly Disagree: 4%
e.
English will be threatened if other languages are frequently used in large immi
grant communities in the
nited States.
2000
Strongly Agree: 9%
Agree: 24%
Disagree: 51%
Strongly Disagree: 16%
2008
Strongly Agree: 16%
Agree: 22%
Disagree: 40%
Strongly Disagree: 22%
2013
Strongly Agree: 17%
Agree: 31%
Disagree: 39%
Strongly Disagree: 13%
f.
Election ballots should be printed in other languages in areas where lots of
people don’t speak English.
2000
Strongly Agree: 17%
Agree: 49%
Disagree: 22%
Strongly Disagree: 12%
2008
Strongly Agree: 18%
Agree: 30%
Disagree: 27%
Strongly Disagree: 25%
2013
Strongly Agree: 11%
Agree: 34%
Disagree: 31%
Strongly Disagree: 24%
178
Richard D. Brecht, William P. Rivers, John P. Robinson, and Dan E. Davidson
Figure 9.2
ublic Opinion Toward Foreign Language, 2000–2013
Combining the “strongly” and “less strongly” positions from the responses in
Table 9.1, close to or more than 70 percent agreed in all three years that English
should be the ocial language of the United States (Q1), that high school students
should become ᭵ent in a foreign language (Q2b), and that FL has the same edu
cational value as math or science (Q2d); some 75 percent also agreed that English
unites Americans (Q2c). In contrast, less than 25 percent agreed that bilingual edu
cation should be eliminated (Q2a), and 32–36 percent agreed that immigrant use of
foreign languages posed a threat to English (Q2e). The overall import of these data
is that public opinion supports language instruction, and that support has in general
held steady for the past decade. With respect to public attitudes toward ocial lan
guage policy, support for foreign language use and instruction does not contradict a
general recognition among the public that English is the
ocial language of
the United States. Moreover, our results consistently show that, while public opin
ion favors making English the
de jure
ocial language of the United States, the use of
other languages is not seen as a threat to English.
In the summer of 2013, a coalition of language organizations coordinated by the
Center for Advanced Study of Language at the University of Maryland produced a
FESSI
LA
GUAG
SK
ILLS
UNPRECEDENTED
ND
ND
SU
PPL
179
180
Richard D. Brecht, William P. Rivers, John P. Robinson, and Dan E. Davidson
Best Practices in Primary and Secondary FL Education
Needless to say, the base of the pyramid described above represents kindergarten,
elementary, and secondary schools across the country. The
Pre-K–12 language
teaching and learning sector is experiencing a period of notable increase in eective
program models and classroom practices, encouraging research results, and broad
ening innovation. One of the most eective modes of providing usable language
skills at the Pre-K–12 level is language-immersion programming. Successful exem
plars of dual-language immersion programs beginning at the elementary level with
articulated sequencing through grade 12 can be found throughout the United States.
A number of states, including California, Florida, Illinois,
FESSI
LA
GUAG
SK
ILLS
UNPRECEDENTED
ND
ND
SU
PPL
181
the curricular resources are available, and that high-level outcomes are possible not
just for a narrow group of linguists but for students in any eld of study.
Finally, there is a budding movement to diuse cutting-edge innovation in
higher-education language learning, which is known as the
Partnership for Language
Figure 9.3
ussian Overseas Flagship
rogram
re- and
ost-
rogram O
PI
cores for 2004–05 through
2012–13
cademic Year
rogram (
= 105)
182
Richard D. Brecht, William P. Rivers, John P. Robinson, and Dan E. Davidson
the primary source of language training for the US military, consistently and rig
orously documents student outcomes across almost two dozen languages, includ
ing the hardest for English speakers to learn, such as
Korean,
Chinese, and
Japanese. From 2007 to 2012, the DLIFLC has contributed to the eld 9,000+
2/2/1+ qualied linguists, 2,884 2+/2+/2 qualied linguists, and 950 3/3/2 quali
ed linguists from the Basic Course Program. The measures for success in the last
two years include not only 2/2/1+ prociency. The data below show the percentage
of students from the entering classes who graduate with 2/2/1+ prociency on the
Defense Language Prociency Test in listening, reading, and speaking, regardless of
attrition:
Results FY12:
Pro ciency: 77.3%
Production: 61.0%
Results FY13:
Pro ciency: 86.4%
Production: 66.1%
These data demonstrate that a disciplined and accountable program can reli
ably take high school graduates to signicant prociency levels in specied periods
of instruction. To be sure, this success is due to a range of factors, including low
student-to-faculty ratios, superior technology (see Enabler 3 above), most students
being selected by language aptitude, native-speaking teachers, constant formative
assessment and mentoring, and intensive course work. On the other hand, the insti
tute works mostly with high school graduates and includes military-dictated sched
uling, rare languages, and a stressful environment for learners who must succeed or
be reassigned or released from service. In sum, we take these data as evidence that
language training has grown in eectiveness, utilizing all emerging advantages, in
spite of signicant challenges.
Conclusion
Language education in the United States has undergone signicant improvement
over the past two decades, making the attainment of high-level Professional lan
guage skills more possible and certainly more accessible. With increasing public sup
port and advances in science, technology, and practice, the future looks bright—that
is, if policymakers, leaders, and managers take full advantage of these enabling fac
tors. Finding common ground among the ve sectors of the “Language Enterprise”
(education, government, industry, heritage communities, and overseas partners) in
vision, message, and strategy for moving forward represents an important step in
realizing such language skills. This joint eort on behalf of language has a new force
behind it: the growing demand on the part of industry for language-enabled global
Notes
The authors acknowledge contributions by the participants in the “Languages for All? The
Anglophone Challenge” eort, including the White Paper and the International Forum held at the
University of Maryland on September 30, 2013.
FESSI
LA
GUAG
SK
ILLS
UNPRECEDENTED
ND
ND
SU
PPL
183
184
Richard D. Brecht, William P. Rivers, John P. Robinson, and Dan E. Davidson
185
ALTHOUGH ENGLISH H
to dominate such elds as commerce, science, technology,
law enforcement, and diplomacy in the international arena, scholars and decision-
makers have documented a national need for individuals who possess advanced abil
ities in languages other than English, as well as a deep understanding of the cultures
in which the languages are spoken (see, e.g., citations in Jackson and Malone 2009).
“Advanced” language capability, as discussed in such documents, most typically
refers to language prociency that is equal to or higher than the abilities described
for Level-3 (“General Professional Prociency”) in the Interagency Language
Roundtable (ILR) Language Skill-Level Descriptions (n.d.a., see also Martin, this
volume.) Over the last two decades, however, some scholars (e.g., Byrnes 2002b,
2006a; Norris and Pfe椞er 2003) have characterized the
ILR-type descriptions of
language ability as too limiting in scope.
The expansion in the understanding of linguistic communication components
in recent years is extremely relevant to program design and instruction for learn
ers who want to achieve advanced levels of language ability. On the most basic lev
els, the
American Council on the Teaching of Foreign Languages (ACTFL) and
associated organizations have revised and expanded upon the inuential volume
National Standards for Foreign Language Learning
(ACTFL 1996). While it still focuses
on K–16 language learning, this inuential volume explicitly incorporates develop
ment of knowledge of cultures, communities, and other disciplines within the lan
guage curriculum.
James Bernhardt (Dubinsky, Bernhardt, and Morley 2005, 25)
pointed out that the understanding of ILR-3 language prociency by senior sta at
the Foreign Service Institute’s (FSI)
School of Language Studies has been deepened
and “rened” in recent years, resulting in improved training. As a consequence, FSI’s
186
Frederick H. Jackson
EXPAND
D UND
RSTANDINGS AND
ROGRAMMATIC
PPROAC
S…
187
What
oe猠“
dvanced Language
bility鐠
ean, and What
oes
ake to
chieve
t?
In the introduction to this volume,
Martin summarizes the
prociency descriptors
for Advanced- and Superior-level speakers according to the ACTFL scale and notes
that they were based on the ILR scale (see respective websites for full descriptors).
The ACTFL denitions are similar to but not isomorphic with the ILR guide
lines. Although both speaking scales emphasize the language of spoken interper
sonal interaction, especially in conversation, in comparison to the ILR guidelines,
the ACTFL Speaking descriptions make little reference to comprehension of spo
ken language. Somewhat surprisingly, the ACTFL descriptions make no explicit
reference to the spoken language of
presentation
(viz. the
National Standards for Foreign
Language Learning
, ACTFL 1996), other than to state that a speaker’s language at the
Distinguished level “typically resembles written discourse,” thereby appearing to
imply (wrongly) that the levels are descriptions of language samples and not of the
abilities of language users.
Only the ILR descriptions at the highest level refer to
“functionally equivalent [prociency] to that of a highly articulate well educated native
speaker.” In contrast, the ACTFL description makes the point that the prociency of a
Distinguished speaker may be characterized by a “non-native accent,” a “lack of native-
like economy of expression,” or “an occasional isolated language error⺔
Byrnes has frequently pointed out (2002a, 2002b, and elsewhere) that the con
struct of an idealized “highly articulate well educated native speaker” does a dis
188
Frederick H. Jackson
EXPAND
D UND
RSTANDINGS AND
ROGRAMMATIC
PPROAC
S…
189
it is possible for feedback and instruction to help some learners notice and attend to
aspects of communication that might otherwise not be observed (cf. Schmidt 2001;
N. Ellis 2005). As Schmidt (1994) pointed out, instruction can help learners “notice
190
Frederick H. Jackson
There are many opportunities for pair- and small-group interactions among
students that are typically rooted in meaningful tasks.
Language is used meaningfully from the beginning of instruction at least 90 per
cent of the time by teachers and students to carry out realistic and authentic tasks.
The
EXPAND
D UND
RSTANDINGS AND
ROGRAMMATIC
PPROAC
S…
191
for what purpose, how well” the language will be used (Byrnes 2003). In this respect,
gram as a whole improve. Language functions that are used to carry out a task are
evaluated at the lexico-grammatical level, and are also evaluated at the sentence and
discourse levels, in terms of accuracy, uency, and (appropriate) complexity. The
192
Frederick H. Jackson
has been arranged for students in various languages who have achieved ILR-3 pro
ciency in training at FSI and can arrange time for further training before taking up
their assignment at the US Embassy or consulate. Prociency levels of ILR-3+ and
ILR-4 have been achieved by motivated students in both of these kinds of programs,
which are described here.
Although FSI training results are expressed in terms of ILR Speaking and Reading
scores for global language prociency, the training itself focuses primarily on preparing
the students to perform the representational duties that their assignment and position
will require. Thus, for example, students in training would not normally be asked to
play a role that they would be unlikely to perform in their real lives.
eneral language learning at F
.
The FSI was established
in 1947 as the training arm of the State Department. At
present, the School of Language Studies teaches more than seventy languages—from
world languages like Spanish,
Arabic, and Russian to national languages such as
Turkish,
Urdu, and
Thai, to regional languages like
Pashto,
Kurdish, and
Malayalam.
EXPAND
D UND
RSTANDINGS AND
ROGRAMMATIC
PPROAC
S…
193
Because of the small class size and the abilities of teachers to dierentiate
States. He read the book and discussed its contents with his teacher and with other
194
Frederick H. Jackson
speech were heavily inuenced by his native English, and, while intelligible, sounded
coarser to the
Thai ear than what would be used by a senior
Thai. In this example,
the student was helped to become more aware of features of formal spoken discourse
Thai and also of ways in which his own speech lacked many of those features. As
part of the work with this student, he was given several opportunities to practice
reading aloud, as if giving public remarks, from short speeches that he had composed
with the assistance of a teacher. This helped him to automatize not only the expres
sions and style of formal
EXPAND
D UND
RSTANDINGS AND
ROGRAMMATIC
PPROAC
S…
195
Germany she had regained much of her uency along with a much more acceptable
control of the grammatical structures of German.
In the last decade, a few language programs at FSI have made it possible for
students, who developed strong language ability toward the end of their assigned
training at FSI, to continue their learning for several weeks at an institution in the
country.
At the request of the US ambassador to Israel, FSI initiated a
Hebrew
program for students who had developed a prociency in reading and speaking the
language of at least ILR-2+ to study at an Israeli
taking up their assigned position at the embassy. The
are schools established
by the Israeli government primarily to teach the
Hebrew language and Israeli social
norms to recent immigrants in the country. Classes are intensive, typically for four or
ve hours, ve days a week, and are oered at three levels of language ability.
The embassy requested this program in order to accustom new ocers to
Hebrew
as it is spoken in the very diverse communities of Israel, where about 40percent of
citizens are native speakers of other languages. The two most urgent concerns of
196
Frederick H. Jackson
quite positive. Out of the seven ocers who participated in it during the rst two
years, all of whom had begun their study of
Hebrew at FSI without prior knowl
edge of the language, all but one had post-program scores on the FSI test that were
EXPAND
D UND
RSTANDINGS AND
ROGRAMMATIC
PPROAC
S…
197
with a business, organization, or agency that works in the stude湴’s eld. Many pro
198
Frederick H. Jackson
The major prociency assessments of Flagship students include pre- and post-
administrations of the ACTFL Oral Prociency Interview (OPI), scored on the
ILR scale; online Reading and Listening tests developed by the American Councils
for International Education; and, at the end of the study, the FSI test of Speaking
and the Defense Language Prociency Tests (
DLPT) for Reading and Listening.
Russian students take the European Union-developed Test of Russian as a Foreign
Language (Davidson 2010, 2012). Chinese summer students are given pre- and
post-program criterion-referenced prociency assessments of reading and listen
ing that were developed at Brigham Young University (Bourgerie p.c.; also http://
chineseagship.byu.edu/node/10).
rticulation from domestic programs to overseas programs
It is obviously critical for the overseas programs to mesh their instruction and learn
ing activities appropriately with the abilities and experience of the students from
the domestic programs. The established benchmarks help in achieving that mesh
EXPAND
D UND
RSTANDINGS AND
ROGRAMMATIC
PPROAC
S…
199
The task requires expert balancing of challenge and support to the learner, who
will invariably experience both cultural shock and cognitive overload (Davidson
onclusion:
ome
ew
esources for
dvanced
Language Learning
The highly motivated language learner now has access online to a rich panoply of
resource tools that can be used to support independent and/or instructed language
study. Among these tools are the guided self-assessment instruments that have
been developed by the ILR (e.g.,
http://www.govtilr.org/Publications/speakingsa.
LinguaFolio
https://linguafolio.uoregon.edu/documents/LFGrid.pdf
), and
ACTFL Progress Indicators for Language Learners (
http://www.act.org/pub
lications/guidelines-and-manuals/ncss-act-can-do-statements
). These give
learners and their teachers clear information about what constitutes functional
prociency.
Malone and her colleagues (2004) have reported that distance learning can combine
200
Frederick H. Jackson
ome
rying
eeds for (
e-)
nvestment
The
SEALang Lab site is not alone in losing funding for the improvement of lan
guage learning and teaching. With the exception of federal funding for STARTALK
and the Language Flagship from the National Security Language Initiative of 2006,
national investment in language education has been very scarce.
Such invest
ment will be critical in order to provide the kinds of advanced language instruc
tion described in this chapter and this book to more learners. Indeed, universities
across the country are continuing to cut back on oering third- and fourth-year so-
called “advanced” courses in many languages in order to reduce costs, which further
reduces opportunities for students to continue developing their language abilities
toward truly advanced levels.
The areas of greatest need include investment in
Beginning use-oriented language instruction in elementary schools and con
tinuing it through high school.
Developing closely articulated learning materials for upper-level students, such
as the early computer-mediated tutorials to teach advanced skills in Russian
that were developed at the
University of Wisconsin–Madison (
http://imp.lss.
EXPAND
D UND
RSTANDINGS AND
ROGRAMMATIC
PPROAC
S…
201
otes
Thanks and appreciation are due to anonymous reviewers for their comments on an earlier draft of
this article, to Sam Eisen of the Defense Language National Security Education Oce for taking
time to deepen my understanding of the Language Flagship programs, and to Heidi Byrnes, Chair
of the GUGD, for sharing her insights into AL2 over several years. It is needless to say that any
202
Frederick H. Jackson
Teachers who are native speakers of the language and who grew to adulthood in a country where it is
the language of education normally do not need further knowledge of the language and understand
ing of the culture. However, what they do need is an understanding of American cultural and profes
sional expectations as represented in their students, colleagues, and the school or college department
where they are employed.
eferences
American Council on the Teaching of Foreign Languages. 1996. N
ational Standards for Foreign Language
Learning
. ACTFL.
http://www.act.org/sites/default/les/pdfs/public/StandardsforFLLexecsumm_
rev.pdf
———. 2012.
ACTFL Prociency Guidelines 2012: Speaking, Writing, Listening, and Reading
http://www.act.org/
publications/guidelines-and-manuals/act-prociency-guidelines-2012
Bourgerie, Dana Scott, Matthew B. Christensen, and Rita Cortex. 2012. “Customizing and
Individualizing Domestic Training at the Advanced Level.” Presented at preconvention workshop
on Increasing Language Prociency at the Postsecondary Level Using Flagship Principles at the
ACTFL Annual Convention, Philadelphia, PA, November 15, 2012.
Byrnes, Heidi, ed. 1998a.
Learning Foreign and Second Languages: Perspectives in Research and Scholarship
. New York:
The Modern Language Association of America.
———. 1998b⸠“Constructing Curricula in Collegiate Foreign Language Departments.” In
Learning Foreign
and Second Languages: Perspectives in Research and Scholarship
, edited by Heidi Byrnes. New York: The
Modern Language Association of America. 26㊖95.
———. 2002a. “Contexts for Advanced Foreign Language Learning: A Report on an Immersion
Institute.” In
Developing Professional-Level Language Prociency
EXPAND
D UND
RSTANDINGS AND
ROGRAMMATIC
PPROAC
S…
203
Ehrman, Madeline. 2001. “Bringing Learning Strategies to the Learner: The FSI Language Learning
Consultation Service.” In
Language in Our Time: Bilingual Education and Ocial English, Ebonics and
Standard English, Immigration and the Unz Initiative
, edited by James E. Alatis and An-Huei Tan, 41–58.
204
Frederick H. Jackson
Spring, Madeline K. 2012a. “Languages for Special Purposes Curriculum in the Context of Chinese-
Language Flagship Programs.” In
Modern Language Journal Special Issue: Focus Issue: Languages for Specic
Purposes: Update on Über Grosse and Voght (1991) in a Global Context
, vol. 96 s1, edited by Barbara A.
Laord, 140–57.
———. 2012b. “Linking Curriculum, Assessment, and Professional Development: Articulation in
Chinese Flagship Programs.” In
Teaching Chinese as a Foreign Language: Theories and Applications
tion.), edited by Michael Everson and Yun Xiao, : 195–236. Boston, MA Cheng & Tsui.
Stevick, Earl W. 1989.
Success with Foreign Languages
. New York: Prentice Hall.
Thorne, Steven L., and Jonathan Reinhardt. 2007⸠“Technologies for Advanced Foreign Language
Prociency.” Position Paper. Center for Advanced Language Prociency Education and Research,
Pennsylvania State University.
http://calper.la.psu.edu/docs/pdfs/alppositionpapers/CALPER_
ALP_Technologies.pdf
Van Olphen, Herman. 2008⸠“A New Model for Teaching South Asian Languages: The University of
Texas
Hindi-
Urdu Flagship⺔
To Advanced Prociency and Beyond:
Charting a New Course in the Twenty-
FirstCentury
in this volume demonstrates the complexity of developing profes
sional levels of foreign language prociency. Foreign language study in the United
States became unprecedentedly urgent in the decade following 9/11.
206
Tony Brown and Jennifer Bown
and curriculum development, including (1) challenges of assessment, (2) immersion
experiences, (3) individualized instruction, (4) curricular materials, and (5) selective
versus open enrollment.
Challenges of Assessment
Findings from this volume underscore a number of challenges associated with ade
quately measuring culturally inected language. For example,
Christensen points
out that the Oral Prociency Guidelines, inasmuch as they emphasize quantity of
speech, reect Western European norms of engagement; however, in Asian cultures
speakers are often expected to say less than more. Such an incongruity is not lim
ited to the testing of Asian languages; rather it reects a deeper concern about the
Oral Prociency Guidelines themselves. The guidelines require greater quantities of
language as a means of dierentiating Superior-level from Advanced-level speech,
a violation of the Gricean norm
for many western and particularly nonwestern
languages. In response to this perceived need to align test design and sociolinguis
tic norms, the American Council on the Teaching of Foreign Languages/Language
Testing International (ACTFL/LTI) recently recalibrated the Oral Prociency
Interview (OPI) for Arabic, which is focused on assessing the balance of colloquial
and Modern Standard Arabic in the speech of interviewees at the Superior level and
Many of the chapters in this volume point to the use of other assessment mea
sures above and beyond the ACTFL OPI and
Writing Prociency Test (WPT).
McAloon, for example, notes that the OPI inadequately addresses speakers’ profes
TO ADV
ED PROFI
IEN
ND BEYOND…
207
the multiplicity of contexts that demand culturally appropriate conduct of a truly
Advanced-level speaker (this volume).
Similarly, in this volume,
Jackson and
McAloon question the criterion of the
ACTFL Distinguished level and of ILR Level 4+ to speak like a highly articulate,
well-educated native, noting that erudite academic speech is not appropriate in all
contexts. Indeed,
Lantolf and Frawley (1985) question the “privileging” of academic
speech in the ACTFL and ILR scales, citing the following admission from the
Foreign
Language Prociency Assessment
manual: “In universities there are Spanish professors who
can speak eloquently about Calderon’s dramas, but who could never read a Spanish
208
Tony Brown and Jennifer Bown
Jackson notes that tailoring learning to students’ specic interests, needs, and
strengths is a necessary component of developing advanced language prociency.
TO ADV
ED PROFI
IEN
ND BEYOND…
209
through Global Debate
represents an important rst step in furthering the Professional
prociency movement.
The increasing availability of language materials on the
210
Tony Brown and Jennifer Bown
this volume), entails a thorough and rigorous selection process involving prociency
testing in each of the skills.
Likewise, one can turn to scaolding theory
and
Lev Vygotsky’s zone of proxi
TO ADV
ED PROFI
IEN
ND BEYOND…
211
Bruner, Jerome S., and Victoria Sherwood. 1975. “Peekaboo and the Learning of Rule Structures.” In
Play: Its Role in Development and Evolution
, edited by Jerome S. Bruner, Alison Jolly, and Katherine Sylva,
277–285. Harmondsworth, UK: Penguin Books.
Byrnes, Heidi. 2002. “The Role of Task and Task-Based Assessment in a Content-Oriented Collegiate
Foreign Language Curriculum.”
Language Testing
19: 425–443.
Cliord, Ray T., and Troy L. Cox. 2013. “Empirical Validation of Reading Prociency Guidelines.”
Foreign Language Annals
46: 45–61.
Davidson, Dan E., and Nadra Garas. 2009⸠“ACTR Census of Russian Programs in the U.S⺔
Language Journa
l 59: 3–20.
213
is a Senior Lecturer of Arabic and Academic Director at the Qasid
Institute in Amman, Jorda渮 His research interests include prociency measure
214
Contributors
Tony Brown
is an Associate Professor of Russian at Brigham Young University, where
he has taught since 2004. Dr. Brown has published articles in the elds of second
language acquisition, language policy, and cultural history and, most recently, coau
thored two advanced-level language textbooks titled
Mastering Russian through Global
Debate
and
Mastering English through Global Debate
215
Contributors
, Associate Professor of Linguistics at Brigham Young University, received
a PhD in Second Language Acquisition from Carnegie Mellon University. Dr. Dewey
teaches courses in linguistics, second language acquisition, language pedagogy, and
language testing. He has published in numerous venues, including
Studies in Second
Language Acquisition, Foreign Language Annals, Innovation in Language Learning and Teaching,
and
Frontiers: The Interdisciplinary Journal of Study Abroad.
216
Contributors
Cynthia L. Martin
is an Associate Professor of Russian at the University of Maryland,
where she teaches courses in language, literature, and culture as well as courses
that are part of the general education program. Dr. Martin was the Director of the
Domestic Russian Flagship Program at UM (2006–2010). Her research interests
217
academic language abilities, 28–29
ACIE.
See
American Councils for International
Education (ACIE)
ACTFL.
See
American Council on the Teaching of
Foreign Languages (ACTFL)
ACTFL Prociency Guidelines, xiii, 98, 153, 155,
2012 revision, xiv
history of, xiii
ACTR.
See
American Council of Teachers of
Russian
Adaptive Test for Listening (ALT), ix, 98, 99, 102
Adaptive Test for Reading (ART), ix, 98, 99, 102
Advanced High (AH), 80
advanced language ability (AL2)
218
BYU Flagship Model, 87
components, 92–93
course goals, 93–94
course materials, 94
219
Defense Language Institute Foreign Language
Center (DLIFLC)
curricula, 12–14
faculty development, 18
Headstart program, 180
historical background, 3–6
immersion experiences, 14–16
Prociency Enhancement Program, 10
student outcomes, 183
Defense Language Prociency Test (DLPT), 9, 11,
12, 17, 155, 198
Defense Threat Reduction Agency (DTRA), 10
direct enrollment
challenges of, 88
immersion experiences, 207
requirements of, 87
220
Foreign Service Institute (continued)
history of, 3–4, 192
prociency tests, 122
Russian program, 7–8, 9
School of Language Studies, 185–86, 190–92,
194–95
Foreign Service Ocer, 7
FORTE, 109
Frawley, William, 207
French, 175
See
὆oreign Service Institute
221
K-8 programs, 182
K-16 programs, 4
Kaplan, Robert, 75, 79
Kennedy, Deborah, 82, 207
Knorr, Walter, 92
Korean
FSI learning programs, 192
investment in teacher education, 200
Language Flagship program, 196
NSLI for Youth program, 131, 147–49
qualied linguists, 183
L2 (second language) prociency
language measures, 121–22
Overseas Flagship Capstone programs, 134–45
pre- and post-program prociency outcomes,
131–33
primary curricular and cocurricular intervention
strategies, 120–21
prociency-based speaking, reading, and listening
scores, 122–28
prociency outcomes, 131
reading and listening comprehension, 128–30
research questions, 118
National Security Language Initiative for Youth
prociency outcomes, 131
prociency testing, 121–22
Russian program of, 127, 128, 131
National Security Language Initiative (NSLI),
xviii, 117
National Standards, 3–4, 186
NATO.
See
὎ational Atlantic Treaty Organization
See
὎ational Endowment for the
Humanities
news analysis, 51–52
See
὎ational Foreign Language Center
(NFLC)
See
὎ational Middle East Language
Resource Center
Noda, Mari, 154
NORC.
See
὎ational Opinion Research Center
Novice High, 118, 123
Novice Low, 123
Novice Mid, 125
NSEP.
See
὎ational Security Education Program
See
὎ational Security Language Initiative
NSLI-Y.
See
὎ational Security Language Incentive
for Youth
OCP.
See
Ὄanguage Flagship Overseas Capstone
Programs
OCQ.
See
὏verclaiming Questionnaire
OFC.
See
὏verseas Flagship Capstone programs
Oce of Science and Technology Policy, 172
Online Activities by Islamist Groups in Egypt:
Evolution and Trends, 56
Open Doors, 88, 117
Open Source Analytical Research Project (OSAP)
2012 projects, 56–57
instructors of, 57
OPI.
See
὏ral Prociency Interview
oral prociency, xiii
levels of, xiii–xv, 123
testing outcomes, 60–64
Oral Prociency Interview (OPI), xiv
cognitive abilities, 29–30
compared with 360-degree feedback, 160
development of, 28–29
foreign language assessment with, 154–55
ILR Level 3 prociency and, 58
L1/L2 prociency and, 24–25, 31, 38–39
Russian Global Debate and, 77
testers, 121
test scores of, 33–37, 98–101
OSAP.
See
὏pen Source Analytical Research
Project
Output Hypothesis, 75
Overclaiming Questionnaire (OCQ), 25, 36–37
Overseas Flagship Capstone programs (OFC)
Chinese, 143–45
listening prociency outcomes, 138, 138–39
oral prociency outcomes, 136
pre- and post-testing results, 134–45
reading prociency outcomes, 136–38
Russian, 139–41
Paige, R. Michael, 105–06, 114
Parkinson, Dilworth, 107
Partnership for Language in the United States
(PLUS), 182
Pashto, 192
Pellegrino Aveni, Valerie, 106阰7, 114
performance, 187
Perkins, David, 29
Persian
Critical Language Scholarship program, 148
DLIFLC course, 11
Flagship Capstone program, 122, 149
Language Flagship program, 196
NSLI for Youth program, 125–26, 147
OPI scores, 136
Pilkenton, W.D., 92
PLUS.
See
ὐartnership for Language in the United
States
Portuguese, 118, 122, 149, 175, 196, 201
Post Language Ocer, 195
223
advantages of, 159–61
disadvantages of, 161–62
in human resources, 156–68
professional prociency
career skills, 52–54
224
Task Force on Global Talent, 173
task performance, 156
See
ὴask-based instruction
TBLT.
See
ὴask-based language teaching
teaching assistant (TA), 106, 107, 109, 111
testing outcomes, 59–67
Test of English as a Foreign Language (TOEFL), 164
Thai, 186, 192, 193, 194, 199, 201
The American Councils for International
Education, 120
theories of practice, 4
The Prociency Enhancement Program, 10
360-degree feedback, 157–62
TOEFL.
See
ὔest of English as a Foreign Language
Trentman, Emma Gale, 114
Tribal Colleges, 119
Turkish, 118, 122, 147–49, 192, 196
ulpan, 195
ulpanim schools, 195
United States Military Academy, 175
University of Aarhus, 5
University of Chicago, 179
University of Jordan, 9, 107
University of North Florida, 179
University of Oregon, 91, 155
University of Wisconsin, 197, 200
upward feedback, 157
Urdu, 118, 148, 149, 192, 196, 204
US-China Clean Energy, 153
Uscinksi, Izabela, 90
Utah Dual Language Immersion programs, 201
Vande Berg, Michael, 105, 114
Varonis, E. Marlos, 76
vocabulary, 74
Voght, Georey M., 88–90
Vygotsky, Lev, 79, 210
Walker, Galal, 154
Webliography, 199
Wesche, Marjorie, 49
West Point, 175
White House Oce of Science and Technology
Policy, 172
Whitmore, Sharon, 153
workplace performance appraisal, 156–62
World War II, 28
Writing Lab, 53, 54
writing prociency
course, 50–51
testing outcomes, 65–68
Writing Prociency Test (WPT), 24, 206
cognitive abilities, 29–30
development of, 28–29
Russian, 77
test scores of, 33–37
Wu, Xiaoqi, 91
Yale University, 9
Yuasa, Etsuyo, 91
zone of proximal development (ZPD), 12, 79

Приложенные файлы

  • pdf 10975120
    Размер файла: 2 MB Загрузок: 1

Добавить комментарий