Key factors to evaluate the impact of bilingual programs: Employability, mobility and intercultural awareness

The importance of bilingual education is widely debated in the scientific world; it represents an opportunity for millions of international students in the 21st century, endorsed with varied benefits, including cognitive, neurolinguistic, socio-cultural, academic or labor, among others. The main objective of this work is to evaluate the impact of bilingual education on students who graduated from bilingual programs by measuring three key factors (employability, mobility and intercultural awareness), and comparing the results with those from a group of students who graduated from non-bilingual programs. A factorial analysis was carried out to reduce the size of the original set of variables from 11 to 3, thus constituting a model of scientific efficiency. Finally, this paper shows a statistical evaluation of the real effects that bilingual programs have achieved since implementation, concluding that the three main factors identified here allow the scientific evaluation of the success of bilingual programs.


IntroductIon
The importance of bilingual education has been widely debated and supported by scientific literature, where field research has proven bilingualism to be an opportunity for millions of international students in the 21 st century. Among the features of bilingual education, numerous benefits can be highlighted in different areas: a) cognitive and neurolinguistic (Hanesová, 2014;Baker et al, 2016); b) socio-and intercultural (Dewaele and Wei, 2013;Fielding and Harbon, 2013); c) academic (Hemsley et al., 2014;Lee et al., 2015); and d) employment advantages (Callahan and Gándara, 2014). The research in this field was groundbreaking at the time and marked several essential lines of work. In this light, Pavlenko (2005) stated that being bilingual has a positive impact on critical and flexible thinking. Moreover, Lindholm-Leary (2001) demonstrated that bilingual programs improved academic performance and have positive effects on language learning. Current objective data defend the implementation of this type of programs at all educational levels, including initial stages (Bialystok, 2018).
Investment in the implementation of bilingual programs worldwide is unquestionable and absolutely plausible because, as stated by García and Yin (2017, p. 6): "The interrelationship of bilingual education especially to economic factors is evident in its concern with the neoliberal global economy in which it operates." However, actual profitability of this investment is worth considering at a global level. The most direct way to assess the investment and impact of bilingual programs is through former students, determined by the use of bilingualism in their lives.

theoretIcal Backdrop
Traditionally, specialized literature has identified three models of bilingual education: Transitional, Maintenance, and Enrichment Models (Hornberger, 1991, p. 224). These models are implemented with different contextual and structural characteristics in various countries of the world, which has generated "ninety different variables depending on four intersecting factors" (García, 2009, p. 275). The implementation of bilingual programs consequently entails extensive variation at the international level.
In European countries, the European Commission has endorsed the implementation of an official bilingual education approach: Content and Language Integrated Learning (CLIL), whose legislative documentation is outlined by Eurydice (2006). CLIL is a bilingual approach which refers to educational contexts where subjects (or parts of subjects) are taught through a foreign language with dual-focused aims: the learning of content and the learning of the foreign language. The European educational policy is very clear: bilingual education is a priority for the Council of Europe, which defends the efficacy of bilingual programs: "In dealing with different models of bilingual education, it should be borne in mind that this form of education has proved its effectiveness in numerous situations throughout the world" (Council of Europe, 2003, p. 23).
Thus, bilingual education in Europe is supported by both the scientific community and international educational policies. This commitment is based on the aforementioned scientific evidence of its benefits, in addition to the work opportunities it offers for students who take part in bilingual programs. Research on the implementation of bilingual programs reveals a fruitful trajectory. In Europe, we can find interesting studies on the implementation of bilingual (both CLIL and non-CLIL) programs. For instance, Pilat et al. (2014) studied the efficacy of bilingual programs in Russia considering the factors that influence second language learning and teaching. Moreover, profuse literature is devoted to the efficiency of bilingual (CLIL) programs in Spain (e.g. Cenoz, 2005;Madrid and Madrid-Manrique, 2014;Pérez-Cañado, 2016;Ortega-Martín et al., 2018;Fernández-Sanjurjo et al., 2019;Madrid et al., 2019). Other studies pay special attention to the comparison of data between programs and schools in different European countries. Muszynska and Gómez (2015) carried out a comparative study on the effectiveness of bilingual programs in four European countries (Italy, the Netherlands, Poland and Spain), concluding that these programs have: an urgent need for effective and real institutional support. There should be more assistance provided for the schools wishing to implement a bilingual curriculum in the form of practical publications/guides, as well as personal support from teacher trainers and curriculum developers (p. 115).
Analyses of the effectiveness of bilingual programs at national and international levels analyze specific areas of bilingual education, though many of these studies focus on students' academic or cognitive factors. The studies on national education policies typically focus on specific content subjects, generally combined with educational levels (e.g. physical education or mathematics in primary, secondary, or tertiary education). Moreover, comparisons between international schools, at best, suggest proposals for improvement that are not easily applicable in other contexts due to limitations of the specific study. In this light, Reljić et al. (2015) carried out one of the few meta-analyses at the European level focused on academic results. Regarding the limitations of the study, they highlight: Other limitations to this and other meta-analyses in this field are related to the methodological flaws of the primary studies in bilingual education. Takakuwa (2005) pointed out that among the common mistakes in studies of bilingualism are overgeneralization, invalid use of tests of statistical significance, and inappropriate use of analysis of covariance (Reljić et al., 2015, p. 122).
Therefore, according to the previous quotation, one of the challenges that research in bilingual education faces is the adjustment of the statistical method. Our study addresses this issue to explain the causes that lead to the success of bilingual programs from the perspective of the final user (that is, the graduate). It is a new approach for data analysis based on a completely different model to date, and to our knowledge it entails an interesting advantage: scientific efficiency, which allows researchers to make the most of the study time, reliability, and objectivity of the data.

Methodology
This work is the first in-depth analysis -considered as a pilot study-of the impact that education has had on the students of the University of Córdoba (UCO), Spain, who have taken both bilingual and non-bilingual programs. The main objective of this work is to evaluate the impact of education on students who graduated from both bilingual and non-bilingual programs by measuring and comparing three key factors: employability, mobility and intercultural awareness. To sub-objectives are identified: a) to visualize whether there are significant differences in the participants' responses; and b) to determine whether predicted differences exist in a clear, reliable, and objective way through a factor analysis.
For these purposes, this paper analyses the results of a survey gathered by massive data collection dispersed through the UCO website, comparing the responses of students who have taken part in bilingual programs (BSs) and those who have not (NBSs). The methodology presents an exhaustive statistical analysis in which a confirmatory factor analysis has been developed. This model is based on the principle of scientific efficiency; in other words, it has reduced the number of items to evaluate the essential data and allow researchers to reach relevant conclusions. The confirmatory factor analysis (which has reduced the factors from 11 to 3) and statistical methodology lead to the conclusion that to measure the success of participants of bilingual education programs three factors must be considered: employability, mobility, and intercultural awareness.

Participants
The sampling was non-probabilistic and of the accidental type. The final sample was formed by 746 undergraduate students of the University of Córdoba (UCO): 284 (38.1%) were men, 455 (61.0%) were women, and 7 (0.9%) of participants did not answer this question. All the subjects were enrolled in the Welcome Course (CBM) of the UCO's platform Moodle. The average age of the sample was 20.7 (SD 2.26). This analysis will distinguish between two types of participants, classified according to their answers to the questionnaire: a) students who have been part of a bilingual program, considered in this study as 'formally bilinguals' (BSs -standing for 'bilingual students'); and b) students who, regardless their exposition to the language, have not taken part of a bilingual program (NBSs -non-bilingual students).

Procedure
The data collection process was carried out in November 2019 by CBM; students were invited to participate in the survey through a direct link to the platform Survey Monkey. The questionnaire was conducted in English and Spanish for this research. The total number of questionnaires answered was 1603, out of which 1289 were complete.
After the information was collected, the data were refined in two phases. In the first stage, the cases which presented atypical data were detected and deleted after a process of standardization. In the second stage (and considering that the total sample was large), the questionnaires with missing data were removed instead of applying an imputation procedure. This ensured the integrity of the results.

Instruments and Materials
A web format was chosen to develop the data collection tool, thus ensuring accessibility for all participants. The questionnaire was developed by the researchers of the Project "Facing Bilinguals: study of the results of the bilingual education programs through the massive collection and analysis of data extracted from social networks" (EDU2017-84800-R).
The first step was the literature review on bilingual education. The work presented here is essentially interdisciplinary, and involves four areas of knowledge: Bilingualism, Computer Science, Statistics, and Bilingual Education. Through a Delphi-like process, the basis to define the concept of 'success in the implementation of bilingual programs' was set, identifying the main dimensions involved: employability, mobility, and intercultural awareness. For this purpose, and using repeated rounds, consensus among the team members was reached regarding the items to define these dimensions. This led to the construction of an initial instrument composed of 24 items to measure the 3 dimensions previously identified, which included 13 demographic and generic questions for the respondents. Most questions were made on a 10-point Likert scale (1 = very little, 10 = a lot). For general questions, numeric and binary variables were used.

Statistical Analysis
Data were statistically analyzed on SPSS version 22.0 software. A basic descriptive analysis was carried out to detect deviations (anomalies), as well as to get a general perspective of the data before performing any inferential studies.
A primary objective of this study is to visualize whether significant differences on mobility, employability and intercultural awareness between students who have taken part in bilingual programs (BSs) and those who have not (NBSs) exist. After verifying the assumption of normality in the distributions of considered variables is not supported (sig. < 0.05) by means of the Kolmogorov-Smirnov test with Lilliefors' correction, comparisons between BSs and NBSs were carried out by Mann-Whitney U and Wilcoxon W (UW) non-parametric tests. The reliability or internal consistency of the constructed scale was performed by the Cronbach's alpha index (Cronbach, 1990) both for the whole scale and for each of the sub-scales of the 3 dimensions. To validate the dimensions, a confirmatory factorial analysis was conducted by using the principal component analysis as extraction method with Promax rotation and Kaiser normalization. The suitability of this type of analysis was verified by Kaiser-Meyer-Olkin Measure of Sampling Adequacy and Bartlett's Test of Sphericity.
Finally, when the factorial scores of the subjects that determine the dimensions considered were set, the scores of BSs and NBSs were compared again by UW tests.

FIndIngs
As stated above, the sample was formed by 746 undergraduate students of the University of Córdoba (UCO): 284 (38.1%) were men, 455 (61.0%) were women, and 7 (0.9%) of the participants did not answer this question. The average age of the sample was 20.7 (SD 2.26). A total of 97.1% (N = 724) have their official residence in Spain, while 2.9% (N = 22) officially reside in a foreign country.
Regarding the questions on generic information of the participants, 42.9% of the respondents participated in a bilingual program (N = 320), 29% had taken a training course abroad, 9.9% were currently working, and 8.8% had worked abroad. This information discriminating between BSs and NBSs is shown in Table 1:  Table 2 shows the descriptive information obtained from the analysis, discriminating between BSs and NBSs. As seen below, some of the means differ depending on student participation or lack thereof in bilingual programs, which are corroborated by the results of the corresponding UW tests carried out. BSs' average age is 20.37, and NBSs' is 20.94, so there is no significant difference between the groups. Nevertheless, all scores are higher for BSs than for NBSs (e.g. BSs more frequently use their second language writing and orally, believe that being bilingual helps them learn other languages. Regarding their mobility, they feel more international, adapt more easily to other cultural contexts, want to travel more abroad, and believe that their second language improves their employability, their work relationships, and their salaries). Consequently, the scores are remarkably higher for BSs than for NBSs in the key three dimensions identified: employability, mobility, and intercultural awareness, a fact that is confirmed later. UW tests show that there are statistically significant differences (sig. < 0.05) in most of the scores between BSs and NBSs. As a result, this factor can be considered essential in the responses of the participants. The items without equality of means rejected (sig. > 0.05) are the following: (a) how much do you think your bilingualism / languages favor/s your mobility abroad?; (b) how much does your second language influence your destination when travelling abroad?; and (c) how much do you think your bilingualism / languages can improve your salary?
Regarding the scales, it is necessary to indicate that only items determined by the research team as indicative have been considered. This recommendation is based on consensus among experts, considering the principle of parsimony, and the study of item-factor correlations, eliminating those items that provided less information. Basically, the concept of reliability refers to the degree of accuracy of the measures obtained by an assessment instrument, quantifying the degree of error affecting these measures. The index or the coefficient used to measure this reliability is Cronbach's alpha. The internal consistency of each of the 3 dimensions combined with their items, the correlation item-factor, and the alpha of the subscales (if item removed) are shown accordingly in Tables 3, 4, and 5:   We verified that in the three subscales defined a value of alpha higher than 0.7 is sufficient to validate their reliability (Nunnally, and Bernstein, 1994). Similarly, the reliability of the whole scale is also analyzed; the number of elements is 11 and the alpha value 0.842, which is higher than the threshold value 0.7. To confirm the proposed underlying structures, confirmatory factor analysis has been used. The appropriateness of this type of analysis is confirmed by Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO) and Bartlett's Test of Sphericity. This last test is significant (sig. < 0.05), so the correlation matrix is not the unit matrix, and it also has a very good measure of adequacy among variables (KMO = 0.859). The percentage of the total variance explained by the 3 factors considered is 63.183% (Table 6), a high percentage in relation to the inherent variability in the data: The main component method has been chosen for factor extraction. Moreover, to clarify the factor structure, a rotation of axes using the Promax method with Kaiser normalization has been used, which alters the results of an orthogonal rotation to create a solution with factor loadings as close as possible to the ideal structure. How much do you think your bilingualism / languages make/s you a citizen of the world? .735 How much do you think your bilingualism / languages make/s you competent in different cultural contexts? .650 How much do you use your second language/s abroad when you travel for pleasure? .930 How much do you feel more international due to your second language? .742 How much do you think your second language makes you more willing to travel abroad? .705 How much do you think your bilingualism / languages favor/s your mobility abroad? .400 How much do you think your second language has impacted the development of your job? .895 How much do you think your bilingualism / languages has / have favored your employability? .858 How much do/would you use your second language to communicate with your colleagues at work? .663 As shown in Table 7, the items saturate in all the three factors considered, each one identifying with the dimensions proposed, thus confirming the theoretical components initially proposed. These dimensions or components are correlated with each other, as shown in Table 8: From this confirmatory analysis, the factorial scores of each of the subjects analyzed were obtained. This way, the dimensions of the original problem are reduced from 11 to 3: factor 1 corresponds to intercultural awareness, factor 2 corresponds to mobility, and factor 3 corresponds to employability.
With this information, a new analysis comparing the means between these dimensions was carried out considering students participation or lack thereof in a bilingual program. That is, a new analysis was carried out to compare the means between intercultural awareness, mobility and employability (table 9). The results of UW tests reveal significant differences in the participants' responses (sig. < 0.05). From the results presented in Table 9, it can be deduced (through higher ranges and mean values) that the highest scores correspond to BSs. This confirms the initial assumptions that taking part in bilingual education programs is determinant in the theoretical factors that influence the success of bilingual education: intercultural awareness, mobility, and employability.

dIscussIon and conclusIons
In this study we have analyzed a sample of 746 undergraduate students of the University of Córdoba (Spain) with a double objective discussed below.
This paper is in line with other studies that analyze the intercultural awareness of bilinguals (e.g. Arasaratnam-Smith, 2016;Abduh and Rosmaladewi, 2018;Gómez-Parra, 2020), their employability (e.g. Callahan and Gándara, 2014;Yang, 2015;Jongyeon, 2019), and their mobility (e.g. Pérez-Vidal, 2015). Applied scales have been statistically validated both individually and as a whole in our statistical analysis. This way, this study goes a step further and makes an exhaustive analysis of these three areas as a whole (not isolated), which provides us with valuable crossed information to draw conclusions regarding the intercultural awareness, employability, and mobility of bilingual individuals as key factors to evaluate the success of bilingual education programs. Moreover, it should be borne in mind that our analysis is not focused on students who are currently part of a bilingual program, but on students who were 'formally bilingual', that is, they have completed the educational stages that name them as 'bilinguals.' The first objective of this paper was to measure the efficiency of bilingual education programs by identifying evidence of the existence of significant differences between the perspectives of the two differentiated groups of the sample, that is, students who have taken part in bilingual programs (BSs) and those who have not (NBSs). The three factors identified to evaluate the success of bilingual programs are employability, mobility, and intercultural awareness. Data show clear differences between both groups in these three dimensions. Bilingual individuals believe their employability is better thanks to their bilingual competence, they have higher intercultural awareness, and the fact of speaking two languages enhances their mobility. It is essential to emphasize again that this paper is focused on the analysis of these factors to assess the success of bilingual programs when the students have finished these stages. In other words, the participants, despite being made up of undergraduate students, are no longer enrolled in bilingual programs, but they had been involved in such programs in previous stages (Primary Education, Secondary Education or Baccalaureate).
Regarding the data related to intercultural awareness, findings shown in Table 2 would seem to indicate that BSs think that speaking several languages helps them feel more like citizens of the world than NBSs. Similarly, their adaptation to other cultures, willingness to live abroad, has a positive impact on how they are and how they live, makes them evolve as intercultural individuals, improves competence in different cultural contexts, increases empathy, and gives them access to enjoy more cultural products. Consequently, we can state that bilingualism, according to the bilingual participants in this study (BSs), has a variety of positive impacts on individuals' cultural awareness.
The intercultural axis is essential in numerous models of bilingual education. For instance, it is the central C of CLIL (Coyle, 1999(Coyle, , 2002(Coyle, , 2006(Coyle, , and 2007. Moreover, it is one of the main objectives of Spanish bilingual programs, which mainly follow the CLIL approach: to help students improve (or develop) their intercultural awareness (Coyle, 2009;Méndez-García, 2012), because CLIL has been specifically designed to develop students' intercultural awareness (Méndez-García, 2013). In this light, it is important to verify whether students who have taken part in bilingual education programs have been enabled to develop their intercultural awareness.
Intercultural awareness is confirmed in this study as a value that respondents consider important in the development of their lives as speakers educated in bilingual programs. Our data prove clear evidence that the CLIL programs that the participants of our survey have participated in have been successful in developing their intercultural awareness, as BSs feel interculturally more competent than NBSs. More generally (that is, beyond Europe), interest in promoting intercultural awareness or competence among students at different educational stages coincides with public policies that support their introduction in the curricula. The focus is largely on sustainable development and social inclusion as the pillars for intercultural education (Banks, 2006;Portera, 2008;UNESCO, 2012). Consequently, we can affirm that the results of this work are relevant at a global level.
Furthermore, employability is one of the objectives of the Spanish Organic Law 8/2013, 9 December 2013, closely linked to plurilingualism as Gándara (2015) demonstrates. This is also connected with scientific literature, which has already identified the need to draw relevant conclusions regarding the economic value that bilingualism brings to society (Callahan and Gándara, 2014, p. 4). Data regarding employability show that a majority of BSs do not work (92.5%, see Table 1). This means that only 7.5% are currently working. Nevertheless, 8.4% have previously worked abroad, and 34.7% have attended at least one training course abroad. When comparing these results to those of NBSs, we can see that 11.7% are currently working, while 88.3% do not work; moreover, 9.2% have previously worked abroad, and 24.6% have attended at least one training course abroad. These data correlate with the three items in Table 2, whose differences are not significant (sig. > 0.05) and explain the fact that the respondents did not answer positively to the question that their bilingualism improves their salary, as they mainly do not have work experience (only 9.9.% of all respondents were working at the moment of the survey).
Regarding mobility, literature reveals that this is one of the areas where further research is needed. For instance, Yang (2015, p. 2) emphasized: "to what extent CLIL has contributed to the goal of enhanced mobility and employability is still unknown (Tudor, 2008)." In the dimension related to mobility, our data show higher scores for BSs (see Table 2), as their status as bilingual individuals helps the use of a second language when they travel for pleasure, they feel more international, and are more willing to travel abroad. Consequently, the influence of bilingualism on international mobility for students who have taken part in bilingual education programs is further supported by our data.
The data obtained from the 3 sub-scales (intercultural awareness, mobility, and employability) allow us to say that the highest scores are associated with BSs. This confirms the initial assumptions that these three theoretical factors are important to assess the success of bilingual education programs. This result is directly connected to the second objective of this paper, focused on reducing the size of the problem. It allowed us to determine whether these differences exist in a clear and objective way, looking for the desired scientific efficiency that all researchers seek. The data show reliable values for all the sub-scales and scales, and the different statistical analyses have led us to reduce the dimension of the original problem from 11 to 3 variables: intercultural awareness, mobility, and employability.
Finally, and considering that this study is the first of a series of papers that will continue to analyze the success of bilingual education programs at the global level from the perspective of their former students, we will extrapolate our conclusions to other contexts. Moreover, we will be able to contrast these data with future analysis, which may help improve the implementation of bilingual programs and contribute to the advancement of science in this field.