Formant dynamics are believed to carry important acoustic information pertaining to vowel identity, where differences in the trajectories of the first and second formant frequencies (F ₁ and F ₂, respectively) are shown to be important cues for the perception of vowels in a particular language (Nearey & Assmann, 1986; Hillenbrand & Nearey, 1999). For instance, the English vowel /æ/, is characterized by a slow and steady upward F ₁ increase followed by a rapid decrease, and by a steady decrease in F ₂ movement (Nearey, 2013). Anatomically, F ₁ is more closely related to the back and F ₂ to the front cavities of the oral tract, where a constriction in the vocal tract caused by the position of articulators, such as the tongue, dictates the shape of these cavities consequently affecting the values of both frequencies (Fry, 1979). Given this indirect relationship between articulatory position and formant values, the modulation of F ₁ is broadly interpreted as the result of the vertical displacement of the tongue, where vertical tongue position is negatively correlated with this formant. Similarly, changes in the frequency of F ₂ are believed to be more closely related to the anteroposterior tongue movement, where a more fronted tongue position results in higher F ₂ values.

In addition, the inherent spectral changes, occurring in the formants as vowels are being produced, are believed to be a product of co-produced articulatory gestures in constant motion (Carré et al., 2017). As such, formant trajectories are thought to be the direct results of the dynamic nature of speech production and should be regarded and investigated as a dynamic process (Carré, 2009; Carré et al., 2017). However, although formant transitions have been shown to reflect, to some extent, articulatory motion (Lee, 2014; Dromey et al., 2013; Gorman & Kirkham, 2020), more often than not, the relationship between the movement of different articulators and the resulting dynamic acoustic output is proven difficult to be captured (e.g. Wieling, 2016), therefore, not always conforming with the acoustic-articulatory assumptions previously mentioned.

Among the reasons for this lack of clarity in the acoustic-articulatory relationship are the well demonstrated uncertainty related to the contribution of each articulator, or the different parts of a single articulator (e.g. tongue blade and dorsum) in the modulation of formant frequencies, the lack of a one-to-one mapping between acoustics and articulation, tied to the quantal theory of speech (Stevens, 1989), and the individual differences in the acoustic and articulatory domains, pertaining, for instance, to a speaker’s anatomical and behavioral characteristics (Yang et al., 1996; McDougall, 2006; He et al., 2019). Nonetheless, shared associations between formant transitions and articulatory movements were demonstrated by means of correlation coefficients (e.g. Dromey, 2013; Lee et al., 2016), linear and non-linear regression models (e.g. Yunusova et al. 2012; Wieling, 2016), and Gaussian graphical models (Lins Machado et al., 2022), to name a few. Although these methods revealed some observed relationships between the acoustic and articulatory domains, causality between the two cannot be determined. One would think that failing to determine causation in statistics may be due to the variety of ways of thinking about causal relations, or the lack of a statistical syntax and semantics for expressing causality. However, theories such as “causal calculus” proposed by Judea Pearl (2009) offer a formal vocabulary and a collection of mathematical principles that allows the inference of causal relationships from observational and interventional data. Moreover, once a definition of causality is accepted, inferences about the causation between variables can be carried out (Granger, 1980).

In the context of this study, causality is defined and consequently investigated as “temporal (or Granger) causality”, where time is the necessary structure for the definition of causality to hold. Under this structure “the present is caused by the past”, based on the principles that causes occur before their effects and contain specific information about future consequences (Granger, 1980). The fundamental assumption of this definition is that if a time series X “Granger-causes” another time series Y, then past values of X should have information that helps predict Y beyond the information contained in past values of Y alone. To put it more simply, if X causes Y, then changes in X should occur before changes in Y.

When we examine Granger causality in the relationship between formant contours and the movements of different articulators (and parts thereof), we begin to consider that changes in acoustic may not be simultaneous but instead preceded by changes in articulation, even if by an extremely short amount of time. In fact, this is not such a far-fetched notion. The quantal theory of speech (Stevens, 1989) proposes that there are quantal regions in the vocal tract, where the acoustic signal is quite sensitive to relatively small changes in articulation. Thus, as an articulator continuously moves to achieve a certain acoustic output associated with these regions, the movement towards a quantal region can inform the expected changes in the acoustic signal. This, for example, would lead us to expect that tongue movements Granger-cause changes on vowel formants, since the tongue movement towards a vocal tract quantal region can indicate that at that point formants will undergo expected changes.

The problem remaining when trying to account for causality between the acoustic and articulatory domains is that of individual variability in these processes. Depending on the research context (e.g. investigating sociolectal differences or general theories of speech production), these differences tend not to be incorporated in the analyses. That may be due to the higher degree of variability found in the acoustic and articulatory processes, which could be the result of motor equivalences (Hughes & Abbs, 1976) tied to speaker-specific preferred articulatory strategies in the production of a particular linguistic sound (Johnson et al., 1993; McDougall, 2006; Y. Ji et al., 2017; Lins Machado et al., 2022). Yet, considering individual differences in speech production may provide valuable insights into how language is used by individuals, subsequently exposing the underlying structures and patterns of a language not despite individual differences but by considering them (Josserand et al., 2021).

Therefore, the current study seeks to investigate whether a causal relationship between tongue movements and the contours of F ₁ and F ₂ can be found while incorporating the idiosyncratic information present in the articulatory movements and the acoustic output. The extent to which previous assumptions suggesting that tongue height may be considered the primary articulatory movement driving the changes in F ₁, and tongue anteroposterior movement strongly modulating F ₂ may likely be a result of previous investigations overlooking the dynamic element of speech or regarding individual differences as “noise”. Thus, besides investigating a potential causal relation, the secondary aim of this study is to assess the stability of the previous assumptions, while considering the individual differences inherent to both processes. With regard to a possible causal link between articulatory tongue displacement and formant movement, we believe that causality in this link can, to some extent, be associated with tongue movements. However, the strength of this causal relationship will likely be influenced by individual differences pertaining to characteristic articulatory behaviors.

To explore causality between tongue movement and changes in F ₁ and F ₂ while considering individual differences, we adopted a hierarchical Bayesian continuous-time dynamic model. By modeling theories as continuous-time dynamic systems, this approach allows for a more direct connection between parameters and theories, formulating changes in terms of predicted transitions over time rather than direct consequences, and allowing for the representation of theories in a causal sense while taking into consideration the limited knowledge of process dynamics and potential model complexity updates (Driver & Tomasik, 2023). The benefit of this strategy is tied to how time and individual differences are handled. The following section is dedicated to explaining this method in further detail.

In studies investigating dynamic information, the data are usually repeated measurements of the same constructs (concepts and variables under study). For instance, formant contours are characterized by extracting acoustic measurements at multiple time points over the course of a vowel. This sort of measurement allows us to gain insights of our constructs (formant contours) at each temporal interval. However, in many theories of change, it is assumed that the variables under study exist and develop continuously over time, and not solely at the measured occasions (Lohmann et al., 2022). Thus, by statistically modeling these continuously developing constructs we are able to more closely connect models with theories of change and to investigate how dynamic effects may develop (ibid.). The analysis of continuous-time processes and dynamics within and between individuals, is made possible through hierarchical Bayesian continuous-time dynamic models, where the constructs measured repeatedly over time yield a time series that when analyzed in this framework reveal information about a construct’s continuous-time dynamics and trends.

Since continuous-time models treat time as continuous rather than discrete, information on dynamics and trends are not limited by time-interval dependency, but rather, processes are represented on a continuous-time scale and parameters are independent of specific intervals (Lohann et al., 2022). This means that parameter estimates are not solely related to a particular interval, but can be generalized to other time intervals, accounting for the continuous nature of the process under study and eliminating bias related to unequal intervals (Driver & Voelkle, 2018). This can be particularly advantageous when investigating acoustic and articulatory time series, since intervals between the measured instances vary due to differences in the length of a particular sound, or to individual differences, for instance.

Moreover, in a Bayesian hierarchical approach, the model structure is shared across all individuals and model parameters are allowed to vary, enabling subject-specific parameters estimation while fully utilizing participants’ data to improve model estimates (Driver & Voelkle, 2021). These models take into account variations between individuals while employing shared characteristics to improve model estimates. This allows for the understanding of how parameters vary across a population, since the estimation of population-level parameters while accounting for individual differences is supported (Driver & Voelkle, 2018). Model parameter population distributions serve as a prior distribution for subject-level parameters. With this strategy, previous knowledge from all other subjects is used to aid in the parameter estimate for each unique individual. The key advantage of this technique is that variance and mean of the population distribution can be estimated alongside subject-level parameters, offering a good scope for random-effects over all model parameters (ibid.).

Mathematically, hierarchical Bayesian continuous-time dynamic models require differential calculus. Differential equations are the mathematics of continuous change limiting time to infinitesimally small values. This enables the usage of a temporal effects matrix that reflects the impact of a system’s current state on the process’ direction of change (Driver, 2022). In this study, the basic stochastic differential equation used in the statistical analysis can be represented as follows:

The derivative dy(t) provides information on how the latent processes in the vector y are changing at the moment. On the right-hand side, this rate of change is explained by a deterministic term, describing trend components, and a stochastic part, reporting the random fluctuations around the trends. In the deterministic part the drift matrix A represents how the latent state of the system changes over time characterizing the temporal dynamics of the processes under study. This matrix contains auto effects on its diagonals and cross effects on the off-diagonals. Auto effects describe how each system process determines its own future values and cross effects between processes explain how one process affects the future values of another. The continuous intercept b provides a constant fixed input to y specifying the long-term level around which the process fluctuates. Lastly, dt can be thought as a very small step in time.

In the stochastic part, allowing for uncertainty in the direction of change (Driver & Tomasik, 2023), dW(t) represents the stochastic error term in continuous time (i.e. random fluctuations) and G the effect of this system noise on the change in y(t), the process under study. The corresponding variance-covariance (or diffusion) matrix consists of the process error variances on the main diagonal as well as the process error covariances on the off-diagonals. For further conceptual and technical details see Driver and Voelkle (2018).

When the underlying system under study is believed to be continually changing and interacting, a continuous-time method is essential for its investigation. To illustrate this, consider the act of producing the vowel /æ/, where interactions occur continuously between the different parts of the tongue and its directions of movement: For instance, as the tongue moves backwards (x) and downwards (y) these connected movements affect each other (given the hydrostatic nature of the tongue) and in turn affect F ₁ values (z). In this constructed example, the continuous-time temporal matrix A would be:

Where the negative diagonal coefficients indicate that increases in any of the variables exerts a downwards pressure on the same variables in the future. This happens because systems tend to fluctuate around a range instead of stretching to infinity (Driver, 2022). The off-diagonals show where a change in one variable (determined by the column) leads to a change in another (determined by the row). Translating to our example, these cross-effects would indicate that a backward movement of the tongue (x) would elevate its dorsum, and the simultaneous jaw opening anatomically coupled with the tongue (y) would increase the first formant (z). Considering the relationship between these three continuous-time variables in this scenario allows us to analyze the ‘Granger causality’ of these relationships. That is, present formant values are caused by past articulatory movements.

The present study used acoustic measures of F ₁ and F ₂ and kinematic measurements of tongue blade and dorsum displacements in the anteroposterior and superior-inferior directions to investigate a possible causal relation between these acoustic and articulatory variables and the individual dynamics in the production of the vowel /æ/ in a sample of native U.S. English speakers. Although statistically significant indications of causality were not demonstrated, the continuous-time modeling approach provided further insights into the dynamic acoustic-articulatory relationship. Further, by accounting for idiosyncratic information present in both domains the stability of this relationship could be investigated.

Regarding F ₁, the model predictions followed the hypothesis that vertical tongue movement has an opposing relationship to this formant. Moreover, the results also suggested that the anteroposterior movement of the tongue blade may have an effect on F ₁ of similar magnitude. These findings suggest that not only tongue height but also the anteroposterior tongue movement have a predictive effect on F ₁. However, while raising predicts an increase, retraction predicts a decrease in F ₁. These results make sense if we consider that in some varieties of U.S. English the vowel /æ/ has a diphthongal quality (Nearey, 2013), with F ₁ slowly increasing with a rapid final decrease as the result of the compound raising and retracting movements. Further, after considering individual differences, the F ₁-tongue movement relationship remained in line with previous assumptions, suggesting that the dynamic relationship between F ₁ and tongue kinematic variables incorporates both height and retraction movements.

In terms of F ₂, tongue fronting and tongue height inversely predicted this construct. A more overall fronted tongue indicated a subsequent decrease in F ₂ and lower tongue dorsum predicted an increase in this formant. Alone, these results do not follow previous accounts postulating that forward and elevating tongue movements increase F ₂. However, when interpreted in combination, they may be indicative of a possible shift in cavity association. That is, instead of the common association of F ₂ with the front vocal tract cavity, its affiliation is likely to be with the cavity behind the constriction point for this vowel (Fant, 1980). Shifts in cavity affiliation happen due to the change in cavity length and constriction degree. The front and back cavities are connected by a region of significant cross-sectional area making the two interact. The narrower the constriction between these cavities the greater the acoustic impedance “uncoupling” the resonances of each cavity. When the constriction degree is broader, such as in the vowel /æ/, the coupled cavities influence each other’s resonances by reducing or increasing the resonant frequencies. Since these are related to the length of the associated cavity, they tend to be higher for shorter cavities and lower for longer ones, however, acoustic coupling can affect this to some extent. The formants F ₁ and F ₂ are said to have shifted in cavity affiliation due to a vocal tract configuration lowering the acoustic impedance between cavities. For instance, as the constriction location moves backwards, the back cavity becomes shorter than the front. Consequently, the back cavity resonance frequency rises to a certain level higher than the front cavity; at that level the back cavity resonance results in F ₂ and the front cavity resonance results in F ₁. Although coherent, this interpretation remains speculative, since an investigation of vocal tract area has not been carried out. Additionally, individual differences followed the same global tendencies except for the rate of change of F ₂, which seems to indicate that a slower increase in F ₂ is expected for speakers who slowly raise their tongue blade. These individual differences, however, seem to suggest that elevating tongue movements increase F ₂ values of speakers in which this formant may be associated with a smaller front cavity.

Overall, the lack of statistically significant effects of tongue kinematic variables on F ₁ and F ₂ could be due to the effects of other unaccounted articulatory variables that are believed to affect formant  values, such as tongue shape (Lee et al., 2015), and laryngeal movement, which most notably either increases or decreases F ₁ values (Esling, 2005). Furthermore, the individual differences mostly followed previous assumptions and model predictions related to the relationship between tongue movement and formant outcomes while also highlighting the complexity of the associations between acoustic features and articulatory variables in these relationships, which we believe are the result of individual articulatory strategies essentially driven by speaker-specific anatomical characteristics and behavioral preferences (Hughes & Abbs, 1976; He et al., 2019; Lins Machado et al., 2022).

Finally, the major limitation of this study must be addressed, this being what the constructs F ₁ and F ₂ actually relate to. Formants are a result of the deformations in the vocal tract area, and although these are primarily done by the tongue, both the lips and the larynx are known to shorten and lengthen the vocal tract, consequently affecting cavity areas and subsequently the values of both formants. Future analysis should, therefore, try to include measurements of these articulators. Notwithstanding this limitation, the present study is a first attempt at explaining possible causal relationships between tongue articulatory variables and the first two formant frequencies, while accounting for its dynamics and the individual differences therein.