Software-assisted identification of non-native pitch elements for Russian-speaking learners of Spanish

2.1. Russian and Spanish prosodic systems

In the context of prior contrastive-comparative investigations into the phonetic-phonological systems of Russian or Spanish, noteworthy contributions have been made by García-Riverón (1980García-Riverón, R. (1980). Sistema sobstvenno voprositel´nyj predlozhenii v sopostovlenii s ispanskim. MGU.
, 1987)García-Riverón, R. (1987). La interrogación. Introducción a su estudio. Editorial Científico-Técnica.
, Mazina (1984) Mazina, L. (1984). Методика обучения студентов-иностранцев интонации русского языка (начальный этап контакта испанского языка с русским) (Doctoral dissertation, Московский государственный педагогический институт иностранных языков имени Мори́са Торе́за). Retrieved from https://bit.ly/47tWcge
, Dmítrieva (2017)Dmítrieva, A. (2017). Estudio recíproco de la fonética y fonología del español y el ruso desde el punto de vista de su adquisición y de su enseñanza como lenguas extranjeras (Doctoral dissertation, Universidad Pablo de Olavide). Retrieved from https://bit.ly/47sVi3G
, and Sarymsakova (2022)Sarymsakova, A. (2022). La enseñanza del español a rusohablantes por medio de interacciones profesionales simuladas (role-play): el análisis del conflicto comunicativo e intercultural y su relación con la prosodia y el gesto (Doctoral dissertation, Universidade da Coruña). Retrieved from https://bit.ly/3kAevOf
. The elucidations provided by these authors distinct dissimilarities in the functioning of melodic patterns within the respective linguistic frameworks of Russian or Spanish. As explained in subsection 4.1, we have selected four speech acts representing pragmatic categories identified as challenging by Sarymsakova (2022)Sarymsakova, A. (2022). La enseñanza del español a rusohablantes por medio de interacciones profesionales simuladas (role-play): el análisis del conflicto comunicativo e intercultural y su relación con la prosodia y el gesto (Doctoral dissertation, Universidade da Coruña). Retrieved from https://bit.ly/3kAevOf
. These acts -namely, threat, politeness, irony, and request- have been deemed challenging due to the pivotal role of pitch in activating implicit meaning. Accordingly, we focus on delineating distinctions in the melodic patterns characterizing these utterances within contributions made by García Riverón, Mazina, Dmitrieva, and Sarymsakova, dedicated to the undertaking of contrastive-comparative analyses within the domain of phonetic-phonological systems in Russian and Spanish.

Henceforth, these authors concur in the observed distinctions in melodic patterns within the Russian and Spanish intonational systems. Regarding the melodic characteristics of speech acts such as threat and request, the Russian intonational system manifests an ascending pitch movement at the centre of its intonational construction. As defined by Bryzgunova (1977)Bryzgunova, E. A. (1977). Zvuki i intonatsiya russkoi rechi. Russkii yazyk.
, the centre of its intonational construction signifies a pitch functional element denoting semantic-pragmatic alterations within an utterance, wherein a declarative statement transitions into an interrogative through the modulation of the centre of intonational construction. Nevertheless, the authors mentioned above underline the incompleteness in the characterization of melodic patterns pertaining to irony and politeness within the Russian intonational system.

Moreover, Hidalgo-Navarro (2019)Hidalgo-Navarro, A. (2019). Sistema y uso de la entonación en español hablado: aproximación interactivo-funcional. Ediciones Universidad Alberto Hurtado.
and Cantero-Serena and Mateo-Ruiz (2013)Cantero-Serena, F. J., & Mateo-Ruiz, M. (2013). La entonación prelingüística del español: implicaciones didácticas. In Aportaciones para una educación lingüística y literaria en el siglo xxi, 2-11.
emphasize the employment of circumflex (ascending-descending) tonal movements in the tonemes associated with politeness and irony (which are not entirely delineated within the Spanish system). Additionally, they identify the utilization of plain tonemes in threat expressions and ascending tonemes in politeness utterances.

In summary, the aforementioned authors delineate fundamental challenges encountered by Russian speakers in the acquisition of Spanish melodic patterns. Certain pitch features associated with speech acts, such as irony and politeness, lack a clear definition within the Russian intonational system. While the Russian intonational system exhibits modulation of the pitch functional element known as the center of intonational construction, Spanish pitch features are distinctly specified for particular speech acts.

Drawing upon the abovementioned dissimilarities of melodic patterns of Russian or Spanish and possible challenge for phonetic-phonological competence acquisition among Russian speaking learners of Spanish, we aim to contribute to the field of comparative-contrastive studies of intonational systems of Russian or Spanish by employing the PAFe tool, which is expounded upon below.

2.2. PAFe operation and usage

The PAFe tool has been created as an extension to an existing desktop application for acoustic speech analysis, Praat, developed by Boersma and Weenink (2019)Boersma, P., & Weenink, D. (2019). Praat: doing phonetics by computer (version 6.0. 52) [windows]. Available at: https://www.fon.hum.uva.nl/praat/.
. PAFe consists of a series of scripts in the Python programming language and implements three intonation comparison algorithms of an ELE learner (in Spanish: Español como Lengua Extranjera; in English: Spanish as Foreign Language) and a native Spanish speaker. This software tool is based on the Praat architecture (Boersma & Weenink, 2019Boersma, P., & Weenink, D. (2019). Praat: doing phonetics by computer (version 6.0. 52) [windows]. Available at: https://www.fon.hum.uva.nl/praat/.
), to which a new module (PAFe, as an extension of Praat) is coupled (see Figure 1) made up of Praat scripts, Python code and a PostgreSQL database. Praat, through its scripting, allows command line calls to other systems, as described by Pop and Altar (2014)Pop, D.-P., & Altar, A. (2014). Designing an MVC model for rapid web application development. Procedia Engineering, 69, 1172-1179. doi: https://doi.org/10.1016/j.proeng.2014.03.106
, making it possible to extend the application through the use of other languages and technologies external to Praat.

This new module (PAFe) communicates with the original system by means of new Praat scripts associated with the application’s menu items (see Figure 2), from which these files are executed. Sometimes, the new module dispenses with calls to Praat and generates information windows directly from Python code files.

Natural language processing and audio processing techniques are used in the PAFe tool, taking as the main source the human voice recordings of native speakers and students. This tool allows three different types of analysis: global, tonal tendency, and intersyllabic, as Couto-Fernández et al. (2022)Couto-Fernández, T., Sarymsakova, A., Condori-Fernández, N., & Martín-Rodilla, P. (2022). Plugin for automatisation of phonetic-phonological analysis and obtaining analytical feedback for Spanish learners. In Sepln-pd 2022: Annual conference of the Spanish Association for Natural Language Processing 2022: Projects and Demonstrations, 83-87. Retrieved from https://bit.ly/48yNqz2
describe. In addition, PAFe provides its users with (a) a database containing their user profile, pronunciation exercises and WAV. audios, and (b) a graphical interface including reports on their progress, such as improvement of their pronunciation in Praat.

Concerning its application, the PAFe tool has predominantly undergone testing within the domain of second language didactics. As previously anticipated in Section 1, subsequent evaluations of the PAFe tool underscore its merits as a computer-assisted instrument for enhancing phonetic-phonological competence among both second language educators and learners (Couto-Fernández, 2021Couto-Fernández, T. (2021). Una herramienta de análisis del habla de audio para proporcionar retroalimentación automática a los estudiantes en la pronunciación en español.
; Sarymsakova, 2022Sarymsakova, A. (2022). La enseñanza del español a rusohablantes por medio de interacciones profesionales simuladas (role-play): el análisis del conflicto comunicativo e intercultural y su relación con la prosodia y el gesto (Doctoral dissertation, Universidade da Coruña). Retrieved from https://bit.ly/3kAevOf
). A notable outcome from prior PAFe evaluations, in addition to presenting feedback similarity percentages in intonational production, is the participants’ recognition of its capacity for graphical visualization of native and non-native melodic curves that indicate each syllable tonal movement (Couto-Fernández, 2021Couto-Fernández, T. (2021). Una herramienta de análisis del habla de audio para proporcionar retroalimentación automática a los estudiantes en la pronunciación en español.
). This feature was deemed a valuable resource for aiding in the improvement and acquisition of melodic patterns in the target language.

Following a brief overview of relevant background related to our study and connected to comparative-contrastive studies on Russian and Spanish intonation, as well as aligned with the PAFe operation and testing, the ensuing section will focus on delineating the methodologies and techniques employed in the processing and interpretation of our data.

4.1. Data collection

In accordance to the Wohlin’s stage of experimental design, the preliminary data on assessment of pitch functional elements via PAFe feedback has been obtained through multiple recording sessions with Russian speaking learners of Spanish. In order to carry out these recordings, we have chosen four indirect speech acts that have been identified as “challenging” by Sarymsakova (2022)Sarymsakova, A. (2022). La enseñanza del español a rusohablantes por medio de interacciones profesionales simuladas (role-play): el análisis del conflicto comunicativo e intercultural y su relación con la prosodia y el gesto (Doctoral dissertation, Universidade da Coruña). Retrieved from https://bit.ly/3kAevOf
due to the main role of intonation in activation of implicit meaning. These speech acts (SA) are presented in Table 1 and Table 2 include the following lexical-syntactic structures and pragmatic functions (PF):

In English: SA 1: "And, of course, then you will be responsible"; SA 2: "I have no doubt that you are good guides, I know that"; SA 3: "Saying that I understand that you are very smart to give explanations"; SA 4: "But, that can't happen again".

The recording procedure design consists of the following steps:

4.2. Data analysis outcomes

Hereafter, we present the results obtained after synthesis of collected data. Table 1 shows the outcomes we got after the recordings with our five female non-native speakers (Sp 1F, 2F, 3F, 4F, 5F) and intersyllabic analysis performed by PAFe tool. Table 1 illustrates five Russian female learners took part in the aforementioned recordings, which were used for PAFe intersyllabic tests; the task was to reproduce four indirect speech acts pronounced by a native speaker of Spanish as accurately as possible, conveying the pragmatic feature indicated before (first paragraph of the present section). By comparing the results obtained from PAFe’s automatic calculations when running the intersyllabic analysis algorithm, we observe that the results range for these speakers is from 80.93% to 93.41% of intonation accuracy, which we consider closely similar to the intonation of the native speaker.

Regarding the intersyllabic analysis results of our male non-native speakers (Sp 1M, 2M, etc.), see Table 2. Taking the same considerations about the stages of the recording performance as described above, the similarity percentages for our male Russian speaking learners of Spanish range from 74% to 90.70%. These results show slightly lower similarity in reproducing the intonation of the native speaker compared to the percentage of female learners. However, we still consider them quite satisfactory because none of our participants showed less than 70% similarity. Regarding the pitch difference per syllable of each reproduced speech act, see Figures 3-8Figures 3, 4, 5, 6, 7, 8.

Figure 3 indicates the major difference (from here onwards marked in red for female speakers) for SA 1 produced by female non-native speakers appears in the final inflection (last unstressed syllable) ble (33%). Likewise, in the body of SA 1, the unstressed syllable us in the body of the pitch contour indicates a percentage of differentiation of 18.2%.

Regarding the SA 1 by male non-native speakers, Figure 4 shows the largest difference (from here onwards marked in green for male speakers) in the body, the unstressed syllable se (36%) and the stressed syllable rá (29.5%) of the nucleus of the verbal syntagm.

As for SA 2 with negative politeness pragmatic function, Figure 5 displays the largest syllabic difference for female speakers in the subordinate part, in the nexus que (18.2%) and the unstressed syllable nas (29.6%), both of them included to the body of the SA 2 utterance. Likewise, the male speakers showed 24.5% in the same pitch contour element (syllables na and bue).

The data from Figure 6 presents pitch contour elements which has the major pith difference for SA 3. The ironical speech act by female speakers distinguishes three main tonal deviations: the body of the utterance formed by do (19%) and to (16%), and the final inflection nes (15.8%). On the other hand, the male Russian speaking learners’ of Spanish utterance reveals only two main points of pitch contrast: tien (26.5%) and pa (27.3%) syllables, both from the body of ironical speech act. It should be noted that, in the case of the final inflection nes, the difference shown by PAFe is 0.01 due to imperceptible pith values of this segment (less than 75 Hz, due to PAFe default settings, as an extension of Praat).

Finally, Figure 7 shows the pitch difference in the body of indirect request speech act 4, namely in the unstressed syllable vol (23.8%), and in the final inflection represented by the syllable se (34%). Regarding the male speakers’ tonal contrast found for SA 4, Figure 8 shows it in the first peak (first stressed vowel), specifically in the stressed syllable e of the demonstrative pronoun (33.5%), as well as in the beginning of the body of the same pitch contour, concretely in the unstressed syllable pe (22%).

In order to adequately interpret the provided empirical data obtained via intersyllabic intonation analysis of native and Russian speaking learners of Spanish performed by PAFe tool, we have obtained the pitch functional elements where the most of f ₀ deviations occurred for female and male Russian speaking learners of Spanish. Figure 9 shows the most of the pitch deviations detected by PAFe, in the case of female Russian learners, occurs in the body (10.71%) and in the final inflection (27.60%) of the pitch contour, while our male non-native participants (Figure 10) showed a greater pitch contrast in the body (20.73%) and in the first peak (20.13%).

Nevertheless, Figure 9 and Figure 10 illustrate that there were instances where the comparison was deemed inapplicable, evident by tone values equivalent to 0.01. Given the emergence of such data, statistical testing became imperative. Consequently, we employed the software tool proposed by Rodríguez-Fernández, Canosa, Mucientes, and Bugarín (2015)Rodríguez-Fernández, I., Canosa, A., Mucientes, M., & Bugarín, A. (2015). STAC: a web platform for the comparison of algorithms using statistical tests. In Proceedings of the 2015 ieee international conference on fuzzy systems (fuzz-ieee).
to assess the statistical significance of the experimental data. We opted for the Friedman Aligned Ranks test due to the fact that the experimental data are not normally distributed and the property of homocedasticity is not satisfied. The resulting p-value, determined to be 0.03769 (with a significance level of 0.05), was observed for male first peak values, as well as female body tonal values. This implies that these pitch functional elements bear statistical significance within our study.

After all, the data obtained on Russian speaking learners of Spanish via intersyllabic analysis provided by PAFe tool and tested statistically have revealed the percentage of tonal deviations in functional elements of intonational contour (anacrusis, first peak, body, nucleus and final inflection) and have shown slight differences in those elements in speech acts produces by male and female speakers.