Dr. Dan Villarreal (University of Pittsburgh) is visiting the Sociolinguistics Lab in early November. He'll be giving a talk, open to the public, on Thursday November 3, 2022. Dan's presentation is of special interest to us because it's about automating analyses of large-scale datasets. As we build a corpus of Michigan speech in the MI Diaries project, we've been using automatic speech recognition (ASR) to speed up our transcription time, and working with MSU's Institute for Cyber-Enabled Research (ICER) to move some of our data processing to their supercomputer. Dr. Villarreal is also giving a talk to the SoConDi group at University of Michigan on Nov 4th, 2022, 3-4pm. If you are interested in joining that talk, please contact Yongqing Ye (yeyongqi@msu.edu) or Suzanne Wagner (wagnersu@msu.edu) for the Zoom link. Sociolinguistic auto-coding: Applications and pitfalls Dan Villareal, University of Pittsburgh Time: Thursday, Nov 3, 4:30-6:15pm Location: Wells Hall B342 and on Zoom Zoom link: https://msu.zoom.us/j/98418360065 Meeting ID: 984 1836 0065 passcode: sociolab. Researchers in sociophonetics and variationist sociolinguistics have increasingly turned to computational methods to automate time-consuming research tasks such as data extraction (e.g., Fromont & Hay 2012), phonetic alignment (e.g., McAuliffe et al. 2017), and accurate vowel measurement (e.g., Barreda 2021). In this talk, I discuss the advantages and challenges of using sociolinguistic auto-coding (SLAC), a method in which machine learning classifiers assign variants to variable data (Kendall et al. 2021; McLarty, Jones & Hall 2019; Villarreal et al. 2020; Villarreal under review). Villarreal et al. (2020) trained random forest classifiers of two sociolinguistic variables of New Zealand English, non-prevocalic /r/ (varying between Present vs. Absent) and intervocalic medial /t/ (Voiced vs. Voiceless), using over 4,000 previously hand-coded tokens (per variable). Cross-validation revealed accuracy rates of 84.5% for /r/ and 91.8% for /t/. In addition to binary predictions, these auto-coders calculate classifier probabilities: the likelihood that a given /r/ token was Present, or a /t/ token was Voiced. In a listening experiment in which 11 phonetically trained listeners coded 60 /r/ tokens, we found a significant positive linear relationship between classifier probability and human judgments; this indicates that classifier probability successfully captures listeners’ perception of phonetically gradient rhoticity. Finally, auto-coders can report which features were most important in classification, helping to shed light on acoustically complex variables like /r/. In short, SLAC can be used for at least three specific functions: binary coding, gradient 'coding', and feature selection. Like other machine learning (ML) methods, however, there are inherent concerns about SLAC's fairness—that is, whether it generates equally valid predictions for different speaker groups (e.g., Koenecke et al. 2020). First, given that there are multiple definitions of ML fairness that are mutually incompatible (Berk et al. 2018; Corbett-Davies et al. 2017; Kleinberg et al. 2017), fairness metrics must be decided upon within individual research domains; I argue for three fairness metrics relevant to the domain of sociolinguistic auto-coding. Second, I re-analyze Villarreal et al.'s (2020) /r/ auto-coder for fairness; I find poor performance on all three fairness metrics, with women’s tokens coded more accurately than men’s (88.8% vs. 81.4%). Third, to remedy these imbalances,…
