1. INTRODUCTIONTop

By now there is a wide body of evidence revealing that listeners have a considerable ability to distinguish between languages based on supra-segmental cues (Ramus, Nespor, & Mehler, 1999; White, Mattys, & Wiget, 2012). A widely held view is that languages can be grouped by their auditory rhythmic characteristics into categories such as stress-, syllable- and mora-timed languages and that listeners can distinguish between languages of different rhythmic classes but not between languages of the same rhythm class (henceforth: class discrimination hypothesis[1]; Nazzi, Bertoncini, & Mehler, 1998; Ramus, Dupoux, & Mehler, 2003; Ramus et al., 1999). While evidence against this view has recently been provided by White et al. (2012) the aim of the present paper was on the one hand to analyze listeners’ language discrimination ability for more language pairs to obtain more evidence either for or against the class discrimination hypothesis and on the other hand to find what possible roles duration and intonation might play in this process.

The categorization of languages into rhythm classes harks back to early claims on auditory rhythmic differences between various languages by Arthur Lloyd James (1929) who argued that some languages sounded more regularly timed than others (he coined the metaphors machine-gun and Morse-code rhythm). This resulted in the well known dichotomy of “stress-timed” versus “syllable-timed” languages by Pike (1945) to which a third class “mora-timed” languages was later added (Ladefoged, 1975). Languages have since often been classified as belonging to one of these three different rhythm classes, which possibly turned into one of the most highly disputed hypotheses in phonetics since the 1920s. Connected to the rhythm class hypothesis was the ‘isochrony hypothesis’ (Abercrombie, 1967), which clearly stated the assumptions towards syllabic and foot durations in syllable- and stress-timed languages respectively. French, for example, has since often been referred to as a canonical syllable-timed language and English as a stress-timed language (Roach, 1982). Since one of the auditory impressions was that syllables in syllable-timed languages reveal a high rhythmic regularity on the syllabic level that is not present in stress-timed languages, it was assumed that the syllable durations must be more equally (quasi-isochronously) distributed in these languages. In stress-timed languages the percept of regularity between stressed syllables was argued to be perceived more regularly, which is why a quasi-isochronous distribution of durations between stressed syllables (i.e., foot intervals) was predicted. Even though the assumptions are possibly not in line with numerous psychoacoustic results (the perception of duration is not independent of factors like sound quality or its intonation) they have more or less been taken for granted in many studies on speech rhythm. However, numerous experiments analyzing syllable and foot durations in a variety of languages revealed that there is no acoustic evidence for the isochrony hypothesis (Dauer, 1983; Ladefoged, 1967; O’Connor, 1965; Ohala, Riordan, & Kawasaki, 1979; Roach, 1982; Shen & Peterson, 1962).

An innovative approach to the problem was proposed towards the end of the 1990s. Based on arguments by Roach (1982) and Dauer (1983) that the perception of regularity and irregularity in syllable- and stress-timed languages might be related to structural differences in phonological/phonotactic characteristics of these languages, Ramus et al. (1999) argued to observe the durational characteristics of consonantal and vocalic interval durations in speech. The rationale for this was that consonantal intervals are typically more complex in stress- than in syllable-timed languages, hence their durational variability should be higher (e.g., as measured by their standard deviation, ∆C; Ramus et al., 1999). Since languages which are argued to be syllable-timed typically do not reveal the phonological feature of vocalic reductions, speakers should spend more of the proportional duration of time during a speech signal on vowels, so their percentage over which speech is vocalic (%V) should be higher. Evidence for these assumptions was provided in Ramus et al. (1999). It led to a large number of studies during the first decade of the 2000s that aimed at further developing the measurement procedures ∆C and %V (amongst others: Barry, Andreeva, Russo, Dimitrova, & Kostadinova, 2003; Dellwo, 2006). Further, a variety of other languages and dialects were studied (amongst others: Arvaniti, 2012; Dancovičová & Dellwo, 2007; Leemann, Dellwo, Kolly, & Schmid, 2012; Mairano, 2011), which sometimes, in particular in the case of Arvaniti (2012), lead to conflicting results not supporting the rhythm class hypothesis. Alternative approaches in a very similar vein were also present. Grabe and Low (2002), for example, measured the average differences between consecutive consonantal and vocalic intervals (the so called Pairwise Variability Index; PVI).

One of the probably strongest arguments in favor of the rhythm class hypothesis, however, was not the acoustic but the perceptual evidence provided by Ramus and Mehler (1999), Ramus et al. (1999), and Ramus et al. (2003). These studies showed that human adult listeners could distinguish between languages of different rhythm classes (for example English and Japanese, i.e., stress- and mora-timed) but not between languages from the same class (for example English and Dutch, both stress-timed). How was this tested? Ramus and Mehler (1999) followed the rationale they elaborated in Ramus et al. (1999), namely that the durations of consonantal and vocalic intervals contain the key information to a language’s rhythmic categorization. Next to the phonological/phonotactic argument (above) the rationale to this view also derived from evidence provided by Nazzi et al. (1998) revealing that listeners (in this case human newborns) are able to derive rhythmic information from highly low-pass filtered speech (below 400 Hz). This manipulation leaves strong cues to the durations of vocalic intervals in the signal, which, in return, is evidence for the assumption that listeners make use of exactly these cues in language discrimination. Low-pass filtering, however, still leaves rudimentary segmental cues in the signal and in the case of high amplitude voiced consonants (e.g., approximants like /l/ or /w/) the distinction between the vocalic and consonantal parts of the signal is thus not quite clear. In the case of low-pass filtering as applied by Nazzi et al. (1998) also the entire pitch contour was present in the resulting signal. For this reason Ramus and Mehler (1999) developed the so-called “flat sasasa,” an MBROLA based resynthesis method in which consonantal intervals in speech were turned into /s/ sounds and vocalic intervals into /a/ while a flat (i.e., monotonous) pitch contour was applied to the signal resynthesized sounds. This resulted in a speech-like signal, which contained only cues to the durations of consonantal and vocalic intervals in the speech signal.

Using flat sasasa, Ramus and Mehler (1999) tested whether French listeners could distinguish between acoustically modified versions of sentences derived from two exotic languages, “Sahatu” and “Moltec,” which were invented names for the real underlying languages English and Japanese. To prevent listeners from using possible acquired knowledge about theses languages in their decision process, the real names to the languages were kept anonymous. Next to durational cues, listeners’ performance for a variety of other cues like the phonotactic arrangement of speech and intonation was tested. For this experiment, three more modifications were created: (a) “saltanaj”: cues to consonant types and vowels as well as the intonation contour were maintained (replacements: all vowels with /a/, all fricatives with /s/, all stops with /t/, all liquids with /l/, all nasals with /n/, and all glides with /j/); (b) “sasasa”: identical to flat sasasa (above) but the intonation contour from the original speech signal was maintained; (c) “aaaa”: sonorants were replaced with /a/ and linear interpolations between the inter-sonorant-intervals were applied. Given that listeners showed the same (in saltanaj and sasasa) or worse performance (in aaaa) compared to the flat sasasa condition, Ramus and Mehler concluded that intonation free durational cues to vocalic intervals are sufficient for French listeners to distinguish between the languages Japanese and English. To us, however, it seems questionable whether this argumentation is justified because in the aaaa condition the intonation contour was not only extracted, but additional new and probably distracting intonation information was created at the points where the contour was interpolated. Such interpolation intervals should be longer and more variable in the case of English, where typically a higher number of voiceless consonants can be present between two vowels compared to Japanese. It thus seems plausible that, because of various psychoacoustic effects, numerous cues to intonation were destroyed and possible artifacts were created by this method. This might have lead to the poorer listener performance in the aaaa condition. Furthermore, it appears implausible that listeners’ performance in language discrimination ability might drop when intonation is provided. Should this be the case, it would mean that listeners cannot rely on the durational information for language discrimination in real speech, as the intonation is always present and would cover the duration cues. This question was addressed in the present paper (Experiment III) when we added intonation to between and within rhythm class language contrasts (French-English, French-Spanish) to test whether this affects listeners’ identification performance (see introduction to Experiment III for details). In order to collect more evidence about the class discrimination hypothesis, Ramus et al. (2003) replicated the method in Ramus and Mehler (1999) for a diverse range of within- and between rhythm class contrasts. Table 1 shows the results from these experiments.

It is visible that contrasts between rhythm class (English-Spanish and possibly Catalan-English) are significantly different from each other, and contrasts within category (English-Dutch, Catalan-Spanish) are around the A’ chance level (0.5) and are nonsignificant. There are a number of other contrasts between languages that are difficult to categorize on an auditory basis (Polish, Catalan, and possibly Spanish; see below), which then again revealed a significant listener discrimination ability when contrasted. In the present study we tested listeners’ performance for more language pairs that have traditionally been classified as either stress- or syllable-timed (Experiment II) to find out whether or not there might be further evidence for the class distinction hypothesis. By replicating results from Ramus and Mehler (1999) in Experiment I, we tested whether the outcome of our method was comparable to theirs.

In summary, we carried out the following experiments:

• In Experiment I we evaluated whether we can replicate the experiments on the Japanese-English distinction from Ramus and Mehler (1999) with Swiss German instead of French listeners and small modifications in the methodology. This was done to obtain a baseline to make our experimental method directly comparable to previous studies (Ramus et al., 2003; Ramus & Mehler, 1999).
• In Experiment II we used our method to test whether Swiss German listeners can discriminate between languages from canonical stress-syllable-timing contrasts, i.e., French and English, based on flat sasasa. We further tested other between rhythm class contrasts (French-Japanese, Spanish-Japanese) and a within-class contrast (French-Spanish).
• In Experiment III we added intonation to test whether listeners might use these cues to improve their language discrimination accuracy. A between-class contrast (English-French) as well as a within-class contrast (French-Spanish) were tested.

The rationale to the individual studies is further elaborated on in the introductions to the subparts.

2. EXPERIMENT I: THE ABILITY OF SWISS GERMAN LISTENERS TO DISTINGUISH BETWEEN JAPANESE AND ENGLISH IN THE FLAT SASASA CONDITIONTop

The first aim of Experiment I was to replicate the results for flat sasasa condition in Ramus and Mehler (1999), henceforth R&M, for our particular apparatus, the stimuli and listener groups. Further we applied a small change in the procedure. The major changes in our method that might influence the results are the following:

1. Change in listener group: We tested the experiment with native listeners of Swiss German as opposed to French listeners in R&M. R&M chose French listeners in their design which were asked to judge sasasa-speech derived from Japanese and English with the argument that French is argued to reveal rhythmic characteristics (syllable-timing) that are neither like those of Japanese (mora-timing) nor those of English (stress-timing). Our Swiss German listeners speak a native language that would commonly be referred to as stress-timed, i.e., it should be more similar in rhythm to English, rather than to Japanese. It is unclear what the effect of this could be. Given that the listeners might be more familiar with the sound of the rhythm in a language that is rhythmically similar (White & Mattys, 2007), it seems plausible that the English flat-sasasa-stimuli sound more familiar to them, and that they should thus be in the position to distinguish between the two types of stimuli even better.
2. Change in test procedure: During the test session in R&M listeners received feedback after responding whether a stimulus derived from a certain language. This may have facilitated the listeners to acquire some knowledge about the test stimuli during the test phase, which might have boosted their performance. We were thus interested in knowing whether the results could be replicated without the presentation of feedback.
3. Change in sasasa-generation: In R&M the signals for the sasasa-condition were generated using MBROLA speech synthesis. In the present study, we developed a method in which /s/ and /a/ segments are concatenated by an overlap-add method (see method below for details). In order to test this we took the identical sentences from a database by Nazzi et al. (1998) that were used in R&M for Japanese and English, and resynthesized them with our sasasa-generator. As the resulting sasasa-stimuli sounded very similar to the MBROLA based ones from R&M we did not expect this point to affect our results.

2.2. Results

Listeners’ mean %C was 74.2, which was clearly above a chance level of 50%. Mean A’ resulted in 0.81, which, again, was drastically higher than the A’ chance level of 0.5. The data shows that listeners’ sensitivity calculated by A’ is higher in comparison to %C, which means that the poorer performance indicated by %C should be due to a listeners’ decision bias (i.e., listeners being either biased towards a Sahatu or a Moltec response when unsure). The distribution of the A’ results confirms the high performance of listeners; the total range was between 0.6 and 0.98. The distribution of the A’ values can be seen in the box-plot to Experiment II (Figure 1, left plot).

Figure 1: Box-plots showing the distributions of A’ (averaged across subjects) for each language-pair condition, Experiments I and II.

In comparison to the results in R&M where the comparable test group achieved a mean %C of 68.1 in the flat sasasa condition (R&M, Table 1, p. 515), our test group scored about 6 percentage points higher. The total A’ range in the R&M group was between 0.25 and 1, which is a drastically larger range than we obtained. This range also shows that a considerable number of subjects scored around chance level (0.5) and lower, meaning that they systematically confounded the languages in the test condition. It is possible that the larger range occurred as a result of the larger group size in R&M (N = 16 as opposed to N = 6 in our experiment).

3. EXPERIMENT II: TESTING THE CLASS DISTINCTION HYPOTHESIS FOR A VARIETY OF LANGUAGES USING FLAT SASASATop

The class distinction hypothesis (the hypothesis that listeners can distinguish languages between but not within rhythm classes based on durational cues) was challenged in a study by White et al. (2012) where English listeners’ ability to distinguish between English and Spanish was compared to their performance in distinguishing between varieties of English (Orkney and Welsh Valleys English). Results based on a slightly modified method (ABX instead of AX) revealed that listeners showed some capacity to distinguish between varieties of the same language, even if this capacity was rather weak. These results were thus the first evidence that listeners can distinguish between languages of the same rhythm class based on durational cues, even between varieties of the same language. The findings gave rise for our Experiment II in which we contrasted language pairs previously not analyzed to investigate listeners’ discrimination performance. Both White et al. (2012) as well as Ramus et al. (2003) have contrasted Spanish and English as a between-rhythm class example. The attribution of a language to a certain rhythm class based on auditory characteristics is not without dispute. Given the experimental results and discussions provided in Pointon (1980), Toledo (1988) and Almeida (1997) it is questioned whether Spanish really is a pure syllable-timed language, as it contains both audible and acoustic characteristics of stress- and syllable-timing. This means that with Spanish and English, listeners in both Ramus et al. (2003) and White et al. (2012) have actually revealed that within-rhythm class distinction is possible. For this reason we tested a contrast that has since the earliest times of the rhythm class hypothesis been described as one of the most prominent contrasts, namely English as a stress-timed language in contrast to French as a syllable-timed language (see Lloyd James, 1929). We tested this contrast in Experiment II. Given the choice of these two canonical representatives of their class, we expected listeners’ performance to be significantly higher than for previous results of English-Spanish contrasts. On the contrary, when French, a canonical syllable-timed language is contrasted with Spanish for which classification has been argued upon, we would expect equally strong discrimination ability for French-Spanish than we would for English-Spanish. In addition to these two language contrasts, we tested two other between-class contrasts: (a) French and Japanese, another between-class contrast (syllable- and mora-timed) for which we expected a high discrimination ability, and (b) Spanish and Japanese. Given that Spanish might reveal characteristics of a stress- and a syllable-timed language, we assumed that listeners’ discrimination ability for this pair should be close to the English-Japanese performance.

4. EXPERIMENT III: THE INFLUENCE OF INTONATION ON LISTENERS’ LANGUAGE DISTINCTION ABILITY BETWEEN AND WITHIN RHYTHM CLASSESTop

Next to language discrimination tests on human adults (Ramus et al., 2003; Ramus & Mehler, 1999; White et al., 2012), such tests have further been carried out with newborns (Ramus et al., 1999), Tamarin monkeys (Ramus, Hauser, Miller, Morris, & Mehler, 2000; Tincoff et al., 2005) as well as rats (Toro, Trobalon, & Sebastián-Gallés, 2003) and led to the widely accepted conclusion that humans as well as mammals generally seem to be capable of distinguishing languages between but not within rhythm classes. This was taken as evidence for the view that an awareness of the rhythmic organization of speech is present in a pre-linguistic stadium of language acquisition (or non-linguistic in animals) and should thus be important structural knowledge that is functionally preceding the acquisition of higher level speech processing abilities in humans like word or syllable segmentation.

The perceptual experiments with humans and animals, however, revealed high variability in their methodology. Only three studies have thus far experimented with a rhythm-only condition, i.e., flat sasasa (Ramus et al., 2003; Ramus & Mehler, 1999; White et al., 2012). Ramus et al. (2000) worked with normal speech as well as saltanaj speech (see Section 1); Tincoff et al. (2005) worked with non-manipulated speech; Toro et al. (2003) again worked with saltanaj; Ramus (2002) has worked with non-manipulated speech, saltanaj, saltanaj + artificial intonation and flat sasasa + artificial intonation. A more recent study by White et al. (2012) used flat sasasa (this study is further discussed below). As a result of these methods, it is apparent that in almost all experiments intonation was actually present; however, in the interpretation of the data it was attributed little importance. This was possibly due to the fact that rhythmic durational cues have been taken for granted to be responsible for listeners’ language discrimination ability for which the results from Ramus and Mehler (1999) provided strong evidence. The way the intonation contour was treated in that study, however, has already been criticized in the introduction to the present paper. The intonation-only condition (aaaa) Ramus and Mehler might have used highly distorted intonation contour information. Ramus and Mehler argued that intonation might be a necessary cue to distinguish between languages that belong to the same rhythm class, e.g., Spanish and Italian (p. 517), while it would be unnecessary and possibly even irritating for the discrimination of languages that belong to different rhythm classes.

In summary, the contribution of intonation to listeners’ language discrimination ability is still unclear. The aim of the present experiment was to take a between-rhythm class contrast (English-French) and within-rhythm class contrast (French-Spanish) and test whether listeners’ inability to distinguish between these language pairs might be enhanced when presented with sasasa that contains the intonation contour from the speech signal it derived from. Given the argumentation in Ramus and Mehler (1999), we should predict that listeners’ accuracy for the French-Spanish contrast should be enhanced but for the English-French contrast intonation should further distort the durational cues and deteriorate listeners’ discrimination performance. However, given the fact that performance was not above chance level for this group in the previous experiment based on flat sasasa speech, this effect is impossible to obtain.

4.2. Results

Figure 2, top box-plot, shows the distributions for listener sensitivity for the flat sasasa from Experiment I and the intonation sasasa for the French-English contrast. For the intonation sasasa listeners performed notably better than for the flat sasasa. A one-tailed t-test was carried out to test whether performance was higher than chance (Bonferroni corrected alpha: 0.025; 0.05/2 condition). The effect turned out significant for the intonation condition (t[11] = 3.28; p = 0.0037).

Figure 2: Box-plots showing the distribution of listener accuracy for flat sasasa from Experiment I and for intonation sasasa for French-English (top) and French-Spanish discrimination (bottom).

Figure 2, bottom box-plot, shows listeners performance for intonation sasasa for the French-Spanish contrast in comparison to the French-Spanish flat sasasa from Experiment II. It is visible that with intonation sasasa a considerably smaller part of the distribution is in the negative A’ condition but nevertheless the bulk of the distribution is only just above chance level. A one-tailed t-test to test whether the performance is above chance was insignificant (t[11] = 1.68; p = 0.06). The effect and the descriptive impression, however, show a slight tendency to drift towards the A’ area above 0.5. Like in Experiment II, it is possible that with some more participants an effect might have been obtained. But again, the magnitude of such an effect must be expected to be very weak. Given the outlier at about 0.2 A’ it is also possible that more listeners would move the distribution below 0.5. The example shows very nicely that listener performance can vary tremendously for this task. While one listener has a performance close to perfection (A’ about 0.9) the before mentioned listener is at the exact opposite (A’ about 0.2). It is possible that individual listeners reach extremely high or low performance values by chance; but it is also possible that certain listeners are better at such tasks than others because they might pay more or less attention to either duration or intonation cues. It would be interesting to test the listeners with extreme A’ value in more detail in future experiments.

5. GENERAL DISCUSSION AND CONCLUSIONSTop

In the present research we demonstrated that Swiss listeners were able to distinguish between English and Japanese based on durational cues to consonantal and vocalic intervals only (Experiment I). This replicated previous findings for French listeners by Ramus and Mehler (1999). In principle this result also confirmed widely held beliefs that listeners are able to discriminate languages from different but not the same rhythm class based on prosodic cues (class distinction hypothesis). The results from Experiment II, however, provided strong counter evidence. According to the theory we predicted that discrimination between languages is possible when stress- and syllable-timed languages are paired, as in English-French, or syllable- and mora-timed languages, as in French-Japanese and Spanish-Japanese (the rhythm class affiliation of Spanish, however, was argued to be unclear). Our results, however, revealed no discrimination accuracy above chance for these language contrasts. This was also true for the language contrast that was described as one of the most canonical stress-/syllable-timed contrast, i.e., English and French. We must therefore conclude that discrimination between languages of different rhythm classes is not always possible based on durational cues. In Experiment III we added intonation to the previously not solvable discrimination task English-French and we found that listeners then showed discrimination ability that was well and significantly above chance. We did not find this effect for the Spanish-French within-class contrasts. From Experiment III we must conclude that previous hypotheses about the role of intonation in listeners’ language discrimination ability have to be revised. According to our results it seems wrong to assume that intonation may distort listeners’ ability to discriminate between languages from the different rhythm classes and it further seems wrong to assume that intonation might provide necessary cues to discriminate between languages of the same rhythm class. This means that intonation might play a much more important role in listeners’ language discrimination ability than widely assumed based on the rhythm class hypothesis.

What do these results tell us about a possible effect of rhythm class? We see two ways of interpretation: (a) For the researcher who wishes to hold on to the rhythm class hypothesis the results might mean that some between-rhythm class contrasts, like French and English, rely on intonation. In other words, one could hypothesize that the only reason why we hear a more regular timing of syllables in French is because possible psycho-acoustic effects triggered by syllabic intonation movements are a possible reason for the sensation of durational regularity in the signal. Without these intonation movements the perceptual durational regularity would not be present. Support for this theory may be seen in the results for the French-Spanish within-rhythm class contrast, where even addition of intonation did precisely not lead to an improvement in performance. (b) The opponents of the rhythm class hypothesis might argue that the results do not provide any evidence in support of the class distinction hypothesis. One might argue that our languages show a wide variety of characteristics regarding their prosodic structure, which may or may not vary between individual language contrasts, always dependent on which particular languages are being paired. In our case it happened that between English and French intonation is more salient, whereas between English and Japanese durational characteristics are more salient. Between French and Japanese it was not the durational characteristics but possibly intonation. No test results are available for this condition.

At this point we leave it to the reader which point of view he/she wishes to take: the rhythm class supporting or the rhythm class opposing point of view. To answer this question unambiguously, a further study should be conducted in which, for example, more within class contrasts in the presence of intonation would be tested. Should within-class contrasts be found in which listeners’ performance is enhanced by the presence of intonation, it would mean that the pro-rhythm class view cannot be held any longer.