VISTA transcriptions reliability (IN PROGRESS)

Process

We used the same process described in https://cloud.wikis.utexas.edu/wiki/x/jg9rDQ

Samples used for reliability

The samples should be copied here and the transcription for reliabiliy and rely output can be found here

5. Data_Raw 102--Connected Speech Reliability VISTA_Script_Data_2024

What needs to be edited?

  • We decided it should be at least 10% per each timepoint for VISTA samples for both trained and untrained topics, separately in each language (but see more information re: decision, below)

  • There are six trained and two untrained topics per bilingual participant and currently we are transcribing only pre 1, pre 2, post 1, post 2, 3mofu (for older cases, only), 6mofu, 12mofu

    • You can find the training condition here https://app.smartsheet.com/sheets/29c56pPH98JW66r6h7fxp8M3rM5cv6xQJqPJm4m1

    • This should result in one trained and untrained topic being double transcribed at each timepoint within a participant (ends up being 16.7% for trained topics per participant in each language at a timepoint and 50% for untrained topics).

      • It doesn’t matter if the topic is the same or different across languages that are being transcribed

      • We have to ensure we assign the reliability transcription to a transcriber who did NOT complete the original transcription

 

What needs to happen next to take advantage of the time Ana Elisa has with us:

The VISTA reliability folder is here: https://utexas.box.com/s/ez1821b4w7l7h84zm0bs4vt15l221vfh

  1. Copy and blind the audios following the guidance above and ensure there is a key so we can reference it @Andrew Collins

  2. If we have a Whisper transcription for the sample, blind the sample and copy it into the reliability folder @Andrew Collins

  3. Copy and paste from the MB datasheets, the final scripts so that the reliability transcribers (and future first transcribers), know what the target is. They should just reference this prior to transcribing so they are aware of the topic and general content @Andrew Collinsyou can direct @Rylee Manning to assist with this.

  4. @Sonia Marqués is blinding the Smartsheet and creating a report so that the data can be entered to the VISTA Smartsheet

Smartsheets

We use this reports to assign samples for reliability and also keep the progress:

 

 

 

Related pages