Title: DisMo: A Morphosyntactic, Disfluency and Multi-Word Unit Annotator. An Evaluation on a Corpus of French Spontaneous and Read Speech
ArXiv ID: 1802.02926
Date: 2014-03-05
Authors: : Christodoulides, N., & Grosman, B.
📝 Abstract
We present DisMo, a multi-level annotator for spoken language corpora that integrates part-of-speech tagging with basic disfluency detection and annotation, and multi-word unit recognition. DisMo is a hybrid system that uses a combination of lexical resources, rules, and statistical models based on Conditional Random Fields (CRF). In this paper, we present the first public version of DisMo for French. The system is trained and its performance evaluated on a 57k-token corpus, including different varieties of French spoken in three countries (Belgium, France and Switzerland). DisMo supports a multi-level annotation scheme, in which the tokenisation to minimal word units is complemented with multi-word unit groupings (each having associated POS tags), as well as separate levels for annotating disfluencies and discourse phenomena. We present the system's architecture, linguistic resources and its hierarchical tag-set. Results show that DisMo achieves a precision of 95% (finest tag-set) to 96.8% (coarse tag-set) in POS-tagging non-punctuated, sound-aligned transcriptions of spoken French, while also offering substantial possibilities for automated multi-level annotation.
💡 Deep Analysis
📄 Full Content
We present DisMo, a multi-level annotator for spoken corpora that integrates part-of-speech tagging with basic disfluency detection and annotation, and multi-word unit recognition. DisMo is a hybrid system that uses a combination of lexical resources, rules, and statistical models based on Conditional Random Fields (CRF). The system is trained and its performance evaluated on a 57k-token corpus of spoken French.
DisMo is designed to explicitly take into account the particular characteristics of spoken language. In the absence of punctuation, the annotator relies on prosodic features and discourse markers to identify discourse boundaries (cf. Leech, 1997;Mertens & Simon, 2013). Disfluencies, such as filled pauses, repetitions and false starts, affect up to 10% of tokens in natural conversation (Shriberg, 2001:154), while previous work has shown that part-of-speech tagging and downstream processing can be improved by detecting and marking these phenomena (e.g. Liu et al., 2006;Georgila, 2010). Furthermore, integrating multi-word expression identification can improve the performance of a POS tagger (e.g. Constant & Sigogne, 2011). While automatic boundary prediction, multi-word unit identification and disfluency detection have already been applied independently on spoken corpora (particularly in French), DisMo integrates these processing steps and encodes the interactions between them. The system’s architecture is not tied to a particular language; however, the tag-set, lexical resources and statistical models have to be adapted to a specific language. In this paper, we present the first public version of DisMo for French, and the results of its evaluation. This work builds upon an earlier version of the system (Christodoulides & Grosman 2012): the main improvements concern the processing of multi-word units and disfluencies, in addition to the use of a lager corpus for training and evaluation.
DisMo accepts several types of input: for a full analysis, an orthographic transcription aligned at the token level with the corresponding sound files is required. It is possible to use the system without the sound signal, in which case some of the prosodic features are ignored. It is also possible to use a transcription which is aligned at the utterance level only, in which case the resulting tokenisation is only approximately aligned. Annotating dialogues is also supported (either one file per speaker, or multiple speakers’ tiers the same file along with a speaker identification tier).
The input formats may be a set of Praat (Boersma & Weenink, 2014) TextGrids, TranscriberAG (Barras et al., 1998), ELAN (Brugman & Russel, 2004), Exmaralda Partitur (Schmidt & Wörner, 2009), or tab-separated text files. DisMo may add its output as a set of annotation tiers in the above-mentioned formats (within the constraints of each format), and additionally supports outputting XML files, OpenDocument spreadsheets, or updating an SQL relational database in the Praaline (Christodoulides, 2014) format.
DisMo’s output consists of six tiers: minimal tokens (tok-min), POS tag of minimal tokens (pos-min), multi-word units (tok-mwu), POS tag of multi-word units (pos-mwu), discourse markers and related phenomena annotation (discourse) and disfluency annotation (disfluency). Figure 1 shows sample output, in the format of a Praat TextGrid, highlighting the containment relationships between the three different levels: tiers tok-min, pos-min and disfluency are congruent; tok-mwu and pos-mwu are congruent and group minimal tokens into multi-word expressions; and discourse may independently group tokens in order to annotate discourse markers. In this figure, the tier ’transcription’ was the input to DisMo, ‘spk2’ contains the utterances of the secondary speaker and tier ‘speaker’ identifies the current speaker.
In this paper we present the results of training and evaluating DisMo on a corpus of spoken French (Avanzi, 2014) created from PFC material (Durand et al. 2002(Durand et al. , 2009)). The corpus includes 12 regional varieties of French recorded in 3 different countries: 4 varieties spoken in Metropolitan France; 4 varieties spoken in Switzerland and 4 varieties spoken in Belgium. In total, there are 96 speakers in the corpus: For each of the 12 sites, 4 female and 4 male speakers, born and raised in the city they were recorded, were selected. The age of the speakers varies between 20 and 80. It is similar between 1 http://www.ling.helsinki.fi/kieliteknologia/tutkimus/hfst/
2 http://crfpp.googlecode.com/svn/trunk/doc/index.html
the 12 groups of speakers (F (11, 95) = 0.360, n.s.), between male and female speakers (F (1, 95) = 0.82, n.s.) and between male and female speakers across the 12 groups (F (11, 95) = 0.133, n.s.).
The recordings consist in semi-directed sociolinguistic interviews, in which the informant has minimal interaction with the interviewer. In average, three minutes of spontaneous speech for each speaker are orthographica
