Copytalk Voice Recognition (VR) Standards

Quick Navigation

How to QA - Problems? - Efficency Tips - Feedback from Staff - Job Examples - Problem Field Examples

Copytalk Voice Recognition (VR) Overview

Copytalk maintains a proprietary speech-to-text implementation hereafter referred to as Copytalk Voice Recognition or just "VR". This system has been designed as assistive technology for our transcriptionists; instead of every job being transcribed by a person, our desire is for most every employee (if not every employee) to spend most of their time editing as opposed to most of their time doing the comparatively more difficult job of transcribing and typing by hand.

Copytalk VR is:

Not ChatGPT or ChatGPT-like, which is to say it is not a learning model which stores audio and customer data to improve itself on its own (huge privacy red flag right there)
A proprietary system that does not send data to third parties
Hosted on Copytalk equipment, on Copytalk networks, and meets the stringent information security and privacy guidelines Copytalk customers mandate
Technology to make the human component of our transcription jobs easier, enabling us to provide the same world-class transcription we've always provided (just with less need to spend as much time completing it)

Productivity Findings

We have seen that the use of VR has helped employees significantly improve their productivity. Because the current compensation model pays more for additional work completed, this is a win-win: employees are able to process more work and get paid more, and Copytalk gets faster transcription times and overall improved productivity. In general:

Scribing is considered the most difficult task and a scribe's X-Factor is expected to be higher than QA X-Factor.
QAing work which human scribes completed is considered comparatively easier, and in general QAXF should be at least 15% faster than scribing on average (which is to say most jobs should be much faster than scribing, but some jobs will not be much faster).
QAing work which VR generates is considered even easier. A primary design consideration for VR is that it enables folks to have even faster QAXFs.
Restated, we want to implement QAing such that: a 2.0 XF scribe should be able to attain a 1.75 QAXF on human-transcribed work and a 1.5 QAXF or lower on VR-transcribed work.

How To QA Copytalk VR Jobs

For current QAs, this will not change your current role drastically (except making your job easier when you get a VR job to QA). QA jobs will appear in Formalizer as they always have, and you will continue to QA jobs from real scribes as you always have. The main difference is that leaving Problem Fields/corrections will be less time-consuming and the types of errors you'll need to look out for are slightly different.

Problem Fields should be left as normal, however, only the type of the problem field needs to be selected. That is, there is no need to type anything in the comment box.
- Example: When leaving the Common Term PF, it is not necessary to type in the comment what the Common Term is
- Example: When leaving a Major Incorrect PF (or any other PFs), it is not necessary to type in the comments for most of the other PFs
Every VR job is a QA send.
There are no bookmarks in the text, and any customer instructions can be assumed to not have been followed

It is not necessary to leave an excess amount of Problem Fields, and it is not necessary to change every single error. Use your judgment on what an acceptable transcript is. The overall goal of VR is to save time and effort. Leaving an excess number of Problem Fields, or spending an excess amount of time re-listening or making minor edits, are not necessary. You are welcome to QA these VR jobs the way you would QA any other QA job. The intent of this document is to establish some reasons why it is not necessary to use a PF on VR jobs. We hope this will save our QAs time and effort.

Why Problem Fields are Useful for VR

As we work to improve VR, we use problem fields to gauge the corrections per hour (CPH) of each VR engine. This data is useful in both representing concrete numbers on the amount of manual intervention necessary from QAs to clean up VR's inaccuracies but also helps to verify the quality of each VR account. A transcription produced by VR is in no way suitable to be delivered to Copytalk subscribers and our organization is aware of the significant role human quality assurance plays in delivering the accuracy subscribers have come to expect from our transcriptions.

Identifying a VR Job

When you get a QA job, you should check the F2 tab. All jobs which are transcribed by VR will have a comment indicating as much.

Things To Do Every VR Job

Once you have identified the job is a VR job, you should:

Create the [3 and [4 fields as VR does not reliably do this
Pay particular attention to:
- Proper nouns and other names (particularly spelled names)
- Numerical amounts (the VR attempts to infer % and $ amounts but is inaccurate sometimes)
- Any instructions dictated by the customer ("operator", "new line", "new paragraph", "skip a space", etc.)
You should feel free to delete text and add bookmarks where appropriate. Perhaps the VR took a non-sensical guess and the accuracy of the transcript is improved by adding bookmarks. But, we do not want to add bookmarks unless necessary.

What To Do Differently On A VR Job

Leave Problem Fields as usual, with the exception of NOT writing anything in the 'comment' box.
Because the VR does not handle customer instructions very well, most every job should have a "Customer Instruction" PF.

One Customer Instruction PF to be used when adding a subject line if one is requested/implied. This would also cover needing to remove spelling instructions scribed verbatim by VR
One Customer Instruction PF to be used for missing requests for "new line", "new paragraph", etc. Even if there are multiple requests in the job that were scribed instead of being followed by the VR, no need to add additional Customer Instruction problem fields.

We have examples outlined in the "Problem Field Examples" section.

Problems With The VR Job?

Did the VR "give up" and most of the job is blank, requiring almost all of the job to be transcribed from scratch?
Leave the Problem Field "Incomplete Job" and mark the job as an FQA
You do not need to leave a "Reasons for FQA" comment. Simply mark the "Decision was right?" dropdown as "No"

If you encounter other issues with VR, we'd like to know so we can continue to improve the VR experience! Please notify a member of management if you come across any issues not covered in this document.

Tips and Tricks On Maximizing Efficiency

Our earnest hope is that QAing a VR job is less time-consuming and less effort-intensive than other types of work.

Speed up the job to a level you're comfortable with. If you have to slow the audio down or return to normal speed, that's OK. After the tricky section is over, speed it back up.
Remember the keyboard shortcuts to leave Problem Fields! Using these means you don't have to stop editing, listening, and reading while you leave a Problem Field.

Create a Problem Field: (F2, Enter)
Start typing Customer Instructions: (C u s, Enter)
And so on. Use the alt+0 (alt and zero) hotkey to get back to the 'Transcribing Text' field on the left-hand side of the screen

Look ahead for customer instructions and standards you can fix but not too far. Stay present and listen critically for major errors/omissions/insertions. Don't take the VR accuracy for granted!
Know that we are NOT looking for every little change to be made

Feedback From Staff

(In no particular order:)

"I'm really going to miss macros as we gradually move to a smaller and smaller percentage of work scribed by humans, but I know that's aways off and also I can't turn down the opportunity to type less and still get the same (or more) amount of work done."
"Even without macros, I was able to keep around a 1 QAXF for decent jobs with no major errors"
"Insertions of text are a common issue but little worry XF-wise; quick and easy to delete"
"VR edit and QA feel like 2 different modes you have to switch between, not having to QA - no comments, no FDS/FQA consideration; VR often has fewer but more egregious corrections than a new scribe QA. Look for the voice recognition comment and make the mental switch to EDIT ONLY :)"
"I really like the program's assistance in video work and am excited to see what it can do in FMZ!"
"I am so excited about this as a disabled person to not type 100% of the time and I generally enjoy editing so much more."

Some Examples Of Jobs

A (first error)	Customer Instruction	A single PF should be used for both the subject and the text.
A (second error)	(none)	The audio is not very clear whether this is "Actually" or "Ashley" and unfortunately "Ashley" is not dictated again in the job. Both are plausible. If "Ashley" was dictated later, we would know this name to be correct. So for this particular job, this is not a required Problem Field. If the audio was clearly "Ashley", dictated multiple times as "Ashley", and was not "Ashley" at the beginning of the text, this should be a Problem Field.
B (first error)	(none)	"We got to talking about, by the way," to "They got to talking about -- by the way". The original text is still neat and readable. "They talked" vs. "We talked" is insignificant, although we should definitely transcribe the correct version. Stylistically a double dash could be appropriate. No need for PF.
B (second error)	Customer Instruction	Because "USI" was correct, this is not a misspelling. Because this instruction did not need to be transcribed (and was), a PF should be used.
C (first error)	Major Incorrect Text	There's a substantial difference in meaning between a "company" and a "committee", because this relates to the advisor potentially selling a product/service to the aforementioned "company" or "committee", which makes this particularly relevant to the overall meaning of the job.
C (second error)	(none)	Because the spelling was correct, we will remove the phonetic spelling but not mark this with a PF since one was used earlier for "USI".
D	Standards	When customers dictate a date in "number-number-number" format we will scribe this with slashes.
E	Incorrect Text	VR wrote "something that's made come after the IDI" but this should be "something that may come after the IDI". Because "something that's made come after the IDI" is obviously wrong even without listening to the audio, this should be marked as Major Incorrect Text.
F	Your Discretion (But Probably No PF)	This audio is somewhat unclear. If your judgment is that what the VR transcribed is OK (which is appropriate in this case), leave it be and do not PF. If your judgment is that what the VR transcribed does not make sense, change the audio or leave a bookmark and do not PF (unless you feel the VR is very wrong).

Examples Of Problem Fields

Original Text	What We Do	Problem Field	Explanation
The VR transcribes instructions verbatim into the text.	Any customer instructions written out by VR should be removed from the text and formatted appropriately.	Customer Instruction	VR oftentimes will scribe formatting instructions instead of following the directive. This can happen multiple times throughout a job, although only one "Customer Instruction" problem field is needed at the first instance. This is still the case even if different requests are missed later in the same job and wouldn't require additional Customer Instruction problem fields. NOTE: Due to VR's failure to create subject lines, the one problem field requirement is separate from any other formatting issues (e.g. you needed to add the [3 field to create a subject and also deleted each instance of punctuation being scribed verbatim as well as removing spelling instructions for dictated names. In this example, you would need two Customer Instruction problem fields for fixing the subject and another for the punctuation/spelling fixes).
VR often has trouble with comma splices or run-on sentences.	Correct this as usual.	Grammar	Generally no problem field is necessary, unless you deem the mistake to be particularly egregious
The VR transcribes ellipsis (...) when customers trail off or interrupt themselves.	Use the double dash ( -- ) as normal.		No problem field needed, simply correcting the ellipsis to a double dash is all that's necessary for each instance
The VR often infers a dollar or percentage amount and will scribe a $ or % sign even when not dictated.	As long as they are contextually accurate, leaving the $ or % amount (even if not dictated) is acceptable and no problem field is needed. However, if the $ or % was scribed incorrectly on a numerical amount then we'll want to remove the erroneous symbol and use a problem field.	If $ or % wasn't dictated but VR correctly guessed and used a symbol no need to correct or problem field Numerical value should be used if the VR incorrectly added $ or % to an amount.
The VR uses different date and time formatting than our standards specify.	Use the formatting as our standards specify		No problem field needed, simply correct to match our standards. However, if the VR transcribes an amount that is incorrect, or transcribes a time or date when it is not a date or time then we'll need to mark with a Numerical Value problem field.
The VR transcribes "eighty twenty" and similar as "8020" (and similar)	Add a slash between the allocation values to match our usual standards (80/20 in this example)		No problem field needed, simply add a slash unless the VR gets the amount wrong which would require a Numerical Value problem field.
VR often uses excess periods such as, "D.C." or "I.R.A." or "S.E.P."	Follow our normal standards and Common Terms. In many instances, this will result in removing the excess periods (IRA, DC, SEP, etc.)		No problem field needed, simply remove the periods to align with our usual standards.
The VR transcribes possessive nouns like so: "Lois's house", "Chris's account", etc.	Follow our normal standards. Remove the additional S ("Lois' house", "Chris' account", etc.).		No problem field needed, simply remove the extra S to align with our usual standards
VR is unable to add any error codes.	All error codes will need to be entered as usual for hang-ups, abusive customer, etc.	Error Code	Only one problem field is needed, even if the job requires the Other Problem, Process Anyway error code due to multiple error codes being necessary.
"...anywhere between $90,000 to $250,000 a year so for the purpose of the PX show him making $150,000" but the customer only dictated "150"	Since the VR correctly formatted the amount despite not being dictated, this can be left as is.		No problem field needed, simply leave as is.
You leave a bookmark in the subject line, encounter a VR job with profanity dictated, etc.	Follow the guidelines for "Mandatory QA Sends" and send to QAQA as if you were the "scribe" on the job in question.		No problem field needed, simply leave a comment to the QAQA explaining the mandatory QA send reason.

Last updated: 2025-1-15

QA Guidelines Advanced Scribing Techniques Bookmarks Common Terms Compensation Policy CBS Employee Guidelines COSA CRM Standards E-mail Corrections Error Codes Reasons for FDS/FQA Speed and Accuracy Standards Troubleshooting

Revision History:
2024-12-17 - Initial document published
2024-12-18 - Revisions to fix minor errors, add image for F2 tab, nav/anchor additions.
2025-1-15 - Added a table, further explanations, and more examples to Problem Fields section.
2025-02-05 - Added reasons for Problem Field usage and instructions for handling incomplete VR jobs.
2025-02-15 - Clarified that VR jobs may be QA'd as any other job. The VR standards exist as an optional requirement to assist in QAs achieving fast QAXFs.