Massachusetts Institute of Technology researchers have developed the Prune, Adjust, and Re-Prune (PARP) technique to simplify an advanced speech-learning model to learn uncommon spoken languages more easily.
It entails eliminating unnecessary components of the Wave2vec 2.0 neural network, then making small adjustments so it can recognize a specific language.
Wave2vec 2.0 is pretrained to learn basic speech from raw audio, and requires massive computing power to train on specific languages.
The researchers pruned network connections that were unnecessary for learning language, then trained the subnetwork with sets of labeled Spanish and French speech, which had 97% overlap.
PARP outperformed other common pruning techniques for speech recognition, especially when trained on a very small amount of transcribed speech.
From MIT News
View Full Article
Abstracts Copyright © 2021 SmithBucklin, Washington, DC, USA