Internship co-supervised by Naver Labs & SpeechBrain

Joint ASR and Repunctuation for Better Machine and Human Readable Transcripts

We want to build an ASR model that jointly learns the transcription and punctuation/case restoration tasks. Solving such a task efficiently would be beneficial to produce more readable transcripts to humans (for instance automatic minuting) or to machines (for further NLP process such as machine translation).

Currently, punctuation and case are restored through a (pipeline) post-processing process that most of the time takes only the ASR transcript as input. Another approach consists in directly producing the cased and punctuated transcription from speech input but large amount of training data is needed for this. In addition to evaluating those baseline approaches, we aim at a better integration of both transcription and restoration tasks. For this we will investigate multi-task objectives and dual decoders. Such a work could be also extended to joint translation and repunctuation from speech. All the developpements related to this internship will be made within the SpeechBrain toolkit.

This internship is a co-supervision between NAVER LABS Europe and SpeechBrain