Auditory feedback

Auditory feedback (AF) is an aid used by humans to control speech production and singing by helping the individual verify whether the current production of speech or singing is in accordance with his acoustic-auditory intention. This process is possible through what is known as the auditory feedback loop, a three-part cycle that allows individuals to first speak, then listen to what they have said, and lastly, correct it when necessary. From the viewpoint of movement sciences and neurosciences, the acoustic-auditory speech signal can be interpreted as the result of movements (skilled actions) of speech articulators (the lower jaw, lips, tongue, etc.). Auditory feedback can hence be inferred as a feedback mechanism controlling skilled actions in the same way that visual feedback controls limb movements (e.g. reaching movements).

Auditory feedback allows one to monitor their speech and rectify production errors quickly when they identify one, making it an important component of fluent speech productions.^[1] The role of auditory feedback on speech motor control is often investigated by exposing participants to frequency-altered feedback. Inducing brief and unpredictable changes in the frequency of their auditory feedback has consistently been shown to induce a "pitch-shift reflex", which suggests that this reflex aids in stabilizing voice frequency around the desired target.^[2][3]

However, due to the fact that auditory feedback needs more than 100 milliseconds before a correction occurs at the production level,^[4] it is a slow correction mechanism in comparison with the duration (or production time) of speech sounds (vowels or consonants). Thus, auditory feedback is too slow to correct the production of a speech sound in real-time. Nonetheless, it has been shown that auditory feedback is capable of changing speech-sound production over a series of trials (i.e. adaptation by relearning; see e.g. perturbation experiments done with the DIVA model: neurocomputational speech processing). 10 minutes is typically sufficient for a nearly-full adaptation. Research has also shown that auditory linguistic prompts resulted in greater correction to acoustic perturbations than non-linguistic prompts, reflecting the decrease in accepted variance for intended speech when external linguistic templates are available to the speaker.^[5]

Auditory feedback is an important aid during speech acquisition by toddlers, by providing the child with information about speech outcomes that are used to pick-up and eventually hone speech motor planning processes. Auditory inputs are typically produced by a communication partner (e.g. caretaker) and heard by the toddler, who subsequently tries to imitate them.^[6][7] Children as young as the age of four have demonstrated the ability to adapt speech motor patterns to perceived changes in vowel auditory feedback, which enables them to maintain the accuracy of their speech output.^[8] However, children's speech motor adaption abilities are not fully optimised due to their limited auditory perceptual skills. Thus, improvements in children's ability to perceive relevant acoustic property will usually be followed by an improvement in their speech adaption performance.^[9]

Individuals who are born deaf often fail to acquire fluent speech, further reinforcing how auditory feedback plays a crucial role in speech acquisition and development.^[10]

Delayed auditory feedback experiments indicate that auditory feedback is important during speech production, even in adults. It has been shown that severe disfluencies in speech occur when the timing of voice feedback is delayed for a normal speaker.^[11][12] Individuals who become deaf post-lingually and are unable to receive vocal feedback anymore also typically experience a deterioration in speech quality,^[13][14] highlighting the importance of auditory feedback in speech formation throughout one's lifetime.

Stuttering is said to be due to ineffective monitoring of auditory feedback, mainly caused by a deficit in the cortical auditory system modulation during speech planning.^[15] When fluent speakers detect a sudden irregularity in a specific acoustic parameter of their auditory feedback, they are able to instantly correct the error in their speech production. Individuals who stutter, on the other hand, are found to have weaker-than-normal abilities to correct such errors.^[16] Individuals that stutter hence demonstrate ineffective auditory comparisons of desired speech movements, as compared to fluent speakers.^[17]

Delayed auditory feedback has been found to be an effective treatment for some individuals who stutter,^[18] since extending the time between speech and auditory perception allows for more time to process and correct errors.

It is posited that individuals with apraxia of speech have weak feedforward programs, which results in the disfluencies of their speech.^[19] These individuals hence develop a heavy reliance on auditory feedback to minimize and repair speech errors^[20] even in later stages of their lives, while fluent speakers easily transitions from feedback dependent to feedforward-dominant.^[21] This is not ideal since heavy reliance on mostly auditory feedback is said to be inefficient for the production of rapid and accurate speech.^[22]

Auditory masking has been found to decrease disfluency duration and increase vocal intensity as well as syllable rate in some individuals with apraxia of speech.^[23] Since apraxia of speech is said to be due to weak feedforward programs and high dependence on auditory feedback, auditory masking can be reasoned to increase fluency by decreasing the frequency of a speaker attending auditorily to speech errors, and hence reducing the likelihood of disfluency-generating corrections.

Enhanced auditory processing can be observed in individuals with visual impairment, who partially compensate for their lack of vision with greater sensitivity in their other sensories.^[24] Their increased sensitivity to auditory feedback allows them to demonstrate impressive spatial awareness despite their lack of sight.^[25][26][27]

Studies have shown that when vision is no longer the primary source for obtaining information, focus shifts from vision to hearing in the desktop environment.^[28] Currently, there are assistive technologies such as screen readers, which aids visually impaired individuals in obtaining information on their desktop screens via auditory feedback (E.g. JAWS^[29]). The assistance can come in the form of either speech based auditory feedback or non-speech based auditory feedback. Speech based interfaces are based on human speech, whilst non-speech based interfaces are based on environmental sounds such as music or artificial sound effects.

For the visually impaired, sole reliance on speech based auditory feedback imposes a heavier cognitive load which is irritating for users.^[30] In contrast, non-speech auditory feedback is pleasant and conveys information more quickly, but lacks detailed information in their conveyance and training is required to understand the cues. Hence, the most ideal interface currently is adaptive auditory feedback, which automatically transitions between speech and non-speech cues based on the user state. Such an interface has been found to be more comfortable and generates higher satisfaction among visually impaired users.^[31]

A trial was conducted to explore whether auditory feedback had an influence on learning how to write. It was found that in adults, auditory feedback enabled the writer to better discern their writing motions. This resulted in an increase in flow and quickness of writing when using sounds to learn the writing of new characters.^[32] Subsequent studies then tested the use of auditory feedback as an aid for children with dysgraphia to learn how to write. It was found that after multiple sittings of using auditory feedback while writing, children could write more smoothly, rapidly and clearly.^[33]

Products based on auditory feedback principles have been invented to aid individuals with such writing disorders. Children with speech disorders can also benefit from such products. For example, a headphone called Forbrain^[34] uses a bone conductor and a series of dynamic filters to correct the perception of one's own voice. This improves concentration, attention, speech, coordination, and other sensory functions. It was awarded by the BETT Show^[35] in 2015 in the category "ICT Special Educational Needs Solutions".

Patients with cerebral palsy have little walking capability, due to limitations of their nervous system.^[36] Auditory feedback in the form of periodic audio signals was found to have a significant improvement on the gait of patients, with several explanations proposed. One model argues that auditory feedback acts as an additional information channel for the motor systems, thereby decreasing the onset of motor faults and refining the gait of patients.^[37] Another model posits that audio signals influence the gait of patients by directing motion patterns, such as heel strike timings. By wearing a device that provides immediate auditory feedback on the quality of one's gait, children with cerebral palsy learned to set down their feet in proper ways that avoided the sounds created when negative gaits were detected.^[36]

The use of an auditory feedback-based treatment is found to have improved on the social interaction, mimicking and coordination skills of children with autism spectrum disorder.^[38] This is achieved through a software which uses sensors to track the body motions of children. Each gesture made will activate a voice recording articulating pieces of sentences.^[38] Children then have to reorder these sentence pieces to form a storyline. Different indicators of coordination such as motion quantity and speed were also recorded to keep track of the child's improvement through these auditory cues.^[38]

Auditory feedback is important in the picking up of a new musical piece. By exposing beginner piano players to irregular auditory feedback, they make more mistakes as compared to those who are given logical and anticipatable auditory feedback.^[39] Learning in the presence of auditory feedback also improved one's recollection of the musical piece.^[40]

However, multiple studies have shown that even without auditory feedback, there is not much disturbance to the performance of seasoned musicians.^[41][42] In the absence or delay of auditory feedback, musicians turn to auditory imagery to direct their performance.^[42] Other forms of feedback can also be used in compensation instead, such as visual feedback where musicians look at their hands to lead their performance.^[43] Major disturbances were only seen in the area of pedaling, where results have shown that pianists were prone to stepping the pedal less often in the absence of auditory feedback.^[42]

The importance of auditory feedback in the case of human singing is reviewed by Howell.^[44] In the context of singing, it is important for singers to maintain pitch accuracy, even when they are drowned out by orchestral accompaniment or by fellow singers. Many studies have looked into the effects of both external auditory feedback and proprioception (also known as internal feedback) on pitch control. It has been found that external auditory feedback is crucial in maintaining pitch accuracy, especially for adults without voice training.^[45][46] This is further supported by recent research which revealed how non-professional singers show lower pitch accuracy when they receive lesser auditory feedback. However, the research also highlighted how the pitch of professional singers remains almost unaffected by auditory feedback since they are able to rely on their internal feedback after years of training.^[47]

The role of auditory feedback in the learning and production of bird-song has been studied in several research papers. It has been found that songbirds rely on auditory feedback to compare the sounds that they make with inborn tunes or songs that they memorize from others.^[48] Numerous studies have shown that without the ability to hear themselves, songbirds develop erratic songs or show a deterioration in the songs that they sing after experiencing hearing loss.^[49][50] Several scientific models have been put forward to explain the worsening of birdsongs after the loss of hearing. (E.g. see Brainard and Doupe's (2000) error adjustment channel in the anterior forebrain: auditory feedback in birdsong learning).^[49]

However, the decline of birdsong quality can vary greatly between different demographics. For example, other studies have found the songs of older songbirds remained consistent, or had a slower rate of deterioration after going deaf.^[51] Some researchers have attributed to songbirds learning how to use other forms of non-auditory feedback such as sensory information to maintain the quality of their songs.^[50] This process is called sensory-motor coupling. Others have argued that older songbirds have a longer access to auditory feedback to learn their songs, which results in more practice and thus more stable production of songs even after deafening.^[51]

• auditory processing disorder
• neurocomputational speech processing (DIVA model)
• visual feedback