Delphi Study
Research on Multilingualism and Language Education Identifying Research Needs via The Delphi Method – Some Preliminary Findings
In the following we present some central and preliminary findings from a Delphi study carried out by the Coordination Office for Multilingualism and Language Education (KoMBi) at the University of Hamburg, Germany. This study was focused on the following questions:
- Which research topics are regarded by experts in the field of multilingualism and language education as particularly urgent and should be researched with priority?
- Which research topics are considered by these experts to be sufficiently researched?
1. Method
The Delphi study was conducted in two rounds which assumed an open and closed format respectively. In Round 1, expert participants provided open research questions, under 12 subtopics covering the research field of multilingualism and language education, that they considered pressing. By means of computer-based, qualitative content analysis (Kuckartz 2016) these questions were systematized and summarized in a questionnaire of 199 closed items for the second round. Expert participants were asked to rate the importance of 143 of these closed items on a Likert scale: e.g. to the question “How important is it to research the topic of …”, they could select between 1. very important, 2. rather important, 3. less important or 4. not important. It was also possible to answer “is sufficiently researched” and “unable to assess”.
2. Participating Experts
We consulted a panel of experts active in and familiar with the state-of-the-art research on multilingualism and language education in the German-speaking area. Recruitment took place on the basis of research at universities, and from consulting conference programs and relevant journals. 164 experts took part in the first round, 101 in the second. 42% of these participants are older than 50 years, 24% between 40 and 50 years, and 28% between 30 and 40 years old. Just 6% of participants are younger than 30. A total of 40% of participants have been active in the area of multilingualism and language education for more than 15 years, 17% for 11 years or more, 33% for six years or more, and 11% for five years or less. Around three quarters are employed in academic research and represent the disciplines of education science, linguistics, psychology and sociology. Others are employed in educational practice and administration, or in both research and practice.
3. Selected Findings
Of the 143 items evaluated on the Likert scale, a ranking could be established on the basis of mean values:
Table 1: ‘Top ten’ of 143 ranked items
Question: How important is it to research the following?
Ranking |
N* |
Mean |
ADM** |
|
1 |
The characteristics of lessons aimed at successfully supporting language among newly arrived migrant children |
85 |
1,26 |
0,40 |
2 |
The effectiveness of teaching concepts for the promotion of multilingualism in regular lessons |
86 |
1,28 |
0,42 |
3 |
The skills required by pedagogical personnel to foster multilingualism |
75 |
1,33 |
0,49 |
4 |
How research findings can be successfully transferred to teacher training |
71 |
1,34 |
0,45 |
5 |
The effects of supporting multilingualism on academic language skills |
89 |
1,35 |
0,48 |
6 |
Impact of the inclusion of multilingualism in subject lessons on subject comprehension |
80 |
1,35 |
0,49 |
7 |
The effective design of heritage language lessons given linguistic heterogeneity among pupils |
76 |
1,39 |
0,51 |
8 |
Appropriate concepts for raising awareness of language education among ‘non-language’ teachers |
74 |
1,41 |
0,49 |
9 |
The effects of bilingual subject lessons on subject comprehension |
78 |
1,42 |
0,52 |
10 |
Inclusion of multilingualism in subject-specific teacher training |
75 |
1,43 |
0,55 |
* N totals are different as the options „unable to assess“ and „is sufficiently researched“ were not incorporated when calculating mean values.
** The Average Deviation Index is used as the measure of agreement between participant answers. It is based on deviations of individual assessments from the mean average of the group. Burke and Dunlap (2002) claim a cut-off value of A/6. Values lower than A/6 indicate significant agreement between participants on a particular topic. In this study, values lower than 0.6666 (4/6) indicate satisfactory agreement among participants.
Participants evaluated the topic „The characteristics of lessons aimed at successfully supporting language among newly arrived migrant children” as having highest priority. The high and consistent agreement on this topic can be interpreted as a reaction to the immigration of refugees to Germany that was occurring as this study was carried out. Another research topic perceived to be very urgent concerns the examination of the effectiveness of educational concepts for the promotion of multilingualism in regular subject lessons. The effectiveness of teaching concepts generally plays a central role for participants in the study with related research topics having high mean values (e.g. The effectiveness of teaching concepts for the promotion of multilingualism in regular lessons: 1.28).
A further significant topic concerns research into competences for the promotion of multilingualism among pedagogical personnel. Research efforts should focus in particular on raising the qualifications of and awareness among ‘non-language’ subject teachers regarding multilingualism, as well as the inclusion of multilingualism in subject-specific teacher training. Research into the successful transfer of research findings into teacher training also received high priority (Rank 4).
Other thematic areas that received high priority include the effects of support for multilingualism on academic language skills as well as on subject comprehension.
Table 2: Topic ranking: Effects of supporting multilingualism
Question: How important is it to research the effects of supporting multilingualism on …
Ranking |
N |
Mean |
ADM |
|
1 |
academic language skills |
89 |
1,35 |
0,48 |
2 |
competence in German as language of instruction |
89 |
1,44 |
0,52 |
3 |
social integration |
84 |
1,51 |
0,56 |
4 |
school success |
87 |
1,62 |
0,61 |
5 |
self-perception in relation to education |
83 |
1,63 |
0,66 |
6 |
motivation |
86 |
1,69 |
0,69 |
7 |
the psychological well-being of students |
82 |
1,74 |
0,74 |
8 |
acquiring favorable learning strategies |
82 |
1,78 |
0,74 |
9 |
competence in the heritage language |
88 |
1,81 |
0,62 |
10 |
competence in foreign languages |
84 |
2,29 |
0,60 |
Concerning the effects of supporting multilingualism, overall the experts consider research into academic language skills to be most important, followed by the impact on German as language of instruction. Implications for skills in the heritage language are considered to be less important here. Perhaps somewhat surprisingly, research concerning foreign language skills is considered least important within this particular constellation. Participants also attach importance to non-linguistic and non-performance-related dimensions such as social integration, motivation and psychological well-being, prioritizing their research over research into heritage and foreign language competences.
A significant finding from all questions concerning the effects of multilingual approaches in education is that participants prioritize research into subject comprehension over language skills. Some examples are shown in Tables 3, 4 and 5:
Table 3: Topic ranking: Multilingualism and subject lessons
Question: How important is it to research the effects of multilingualism in subject lessons on …
Ranking |
N |
Mean |
ADM |
|
1 |
subject comprehension |
80 |
1,35 |
0,49 |
2 |
German-language skills |
80 |
1,59 |
0,60 |
3 |
heritage-language skills |
80 |
1,78 |
0,66 |
The analagous question on bilingual models produced similar results:
Table 4: Topic ranking: Bilingual subject lessons
Question: How important is it to research the effects of bilingual subject lessons on …
Ranking |
N |
Mean |
ADM |
|
1 |
subject comprehension |
78 |
1,42 |
0,52 |
2 |
skills in the respective partner language |
76 |
1,76 |
0,50 |
3 |
German-language skills |
77 |
1,88 |
0,53 |
Also, with respect to research concerning heritage language lessons, particular emphasis is placed on subject language comprehension. Here, too, research into motivation and the psychological well-being received high priority.
Table 5: Topic ranking: Heritage language lessons
Question: How important is it to research the effects of heritage language lessons on …
Ranking |
N |
Mean |
ADM |
|
1 |
subject comprehension |
78 |
1,47 |
0,52 |
2 |
motivation/ psychological state of learners |
78 |
1,49 |
0,56 |
3 |
skills development in the heritage language |
78 |
1,53 |
0,56 |
4 |
skills development in German |
78 |
1,54 |
0,59 |
5 |
Skills development in foreign languages |
77 |
2,17 |
0,52 |
4. What has been sufficiently researched?
Overall, participants considered but a few of the listed topics to be sufficiently researched. The greatest consistency in this regard – i.e. topics that were not prioritized for further research – concerned factors influencing the linguistic skills of children and youth. 28% of participants consider the state of research into socio-economic status to be sufficient, 25% with regard to gender; 20% considered research into the influence of age on language skills to be satisfactory, and 19% see little need for further research on family educational experience. 13% saw the influence of parents’ linguistic abilities on children’s competences as sufficiently researched, 13% the influence of cognitive abilities, 10% the influence of previous knowledge of language, and 10% the link between multilingualism and the ability to learn other languages. However, values of 10% do not indicate any level of agreement between participants. Rather, this data can merely illustrate a trend.
Literature
Burke, Michael J. und Dunlap, William P. (2002): Estimating Interrater Agreement With the Average Deviation Index: A User's Guide. Organizational Research Methods 5/2: 159-172.
Kunina-Habenicht, Olga et al. (2012): Welche bildungswissenschaftlichen Inhalte sind wichtig in der Lehrerbildung. Zeitschrift für Erziehungswissenschaft 15: 649-682.
Häder, Michael (2002): Delphi-Befragungen. Ein Arbeitsbuch. Wiesbaden: Westdeutscher Verlag.
Kuckartz, Udo (2016): Qualitative Inhaltsanalyse. Methoden, Praxis, Computerunterstützung. 3. Aufl. München und Weinheim: Beltz Juventa.
Rösselet, Stephan (2012): ExpertInnen machen Schule. Ergebnisse einer Delphibefragung zur Förderung von SchülerInnen mit Migrationshintergrund. Wiesbaden: Springer.