Real-Time Voice Transcription for DCS Kneeboard using Whisper and VoiceAttack

[bojote]

Hello everyone,

I’m excited to share a tool I’ve been working on that integrates real-time voice transcription directly into your DCS kneeboard using Whisper AI and VoiceAttack. As you know, flying in DCS can require managing tons of information on the fly — coordinates, radio frequencies, headings, or even just reminders. This tool is designed to make that easier by letting you speak and then automatically transcribe that information into your kneeboard in real time.
 

How it Works:
Two Python scripts, combined with VoiceAttack, let you record up to 10 seconds (or more if you edit the script) of audio by pressing a button on your HOTAS or joystick. Once you release the button, the recording is automatically processed using OpenAI’s Whisper model, transcribed into text, and pasted directly into your DCS kneeboard (automatically) — no manual typing required.

Voice Recording with a Safety Mechanism:
When you press a designated button in VoiceAttack, the first script (recorder) starts capturing audio. It records up to 10 seconds (configurable), ensuring that even if you forget to stop the recording, it won’t continue indefinitely.

Automatic Transcription and Paste to Kneeboard:
Once you release the button, VoiceAttack triggers the second script (transcriber). This script sends a signal to stop the recording, processes the recorded audio using OpenAI's Whisper, and transcribes it into text. The transcription is then automatically copied and pasted into your DCS kneeboard using a simulated keyboard shortcut (Ctrl + Alt + P) assuming you have it assigned on your DCS controls UI Controls screen.

Whisper AI for Accurate Transcription:
Whisper is one of the best AI models out there for real-time speech recognition, handling aviation terms, and numbers with remarkable accuracy. Whether you're calling out coordinates, frequencies, or instructions, it catches most of it pretty well. Even if you speak a different language, it will translate the instruction to the kneeboard in plain english.

Why Use This Tool?

• Hands-Free: You don’t need to pause the game or type anything. Simply speak into your mic while flying and it will  AUTOMATICALLY paste the transcribed version of your speech to your DCS Kneeboard on the fly! Coordinates, instructions, 9-Lines, reminders. anything.
• Customizable: You can adjust the recording length, transcription settings, and trigger buttons based on your needs.
• Robust Performance: Both scripts are designed to be reliable under various conditions, with built-in safety mechanisms to avoid race conditions or interruptions.
   

What do you need? (Requirements)

• VoiceAttack: To trigger the recording and transcription scripts with a button press.
• Python Installed: The scripts are written in Python, so you’ll need to have Python installed on your system. Detailed instructions will be provided to set this up.
• Whisper (OpenAI): Whisper is the AI model used to process and transcribe the audio. The model can run on both CPU and GPU (CUDA-supported), but a GPU will significantly speed up transcription. 

How to Set It Up:

Download it from here and check the README file included

Feel free to reach out if you have questions, or if you need help getting it set up!

Fly safe,
Jesus "Bojote" Altuve

Edited October 2, 2024 by bojote
Added images

[MAXsenna]

That's cool! Would've loved to see this replace the VAICOM dictation function!
Wonder what VoiceAttack will use now MS Speech Recognition will be "phased out".
Cheers!

Sent from my SM-A536B using Tapatalk

[bojote]

1 minute ago, MAXsenna said:

That's cool! Would've loved to see this replace the VAICOM dictation function!

I tried, and in fact implemented a solution using dictation in VoiceAttack, but its so bad and the buffers so small that it was highly inconsistent. Got so frustrated that ended creating this as I needed 99.9% accuracy. OpenAI's Whisper is simply in another league

[MAXsenna]

I tried, and in fact implemented a solution using dictation in VoiceAttack, but its so bad and the buffers so small that it was highly inconsistent. Got so frustrated that ended creating this as I needed 99.9% accuracy. OpenAI's Whisper is simply in another league

Yeah! And I assume MS won't really compete on that level anymore, and that's why they're phasing it out. Even removed Cortana, and I had such a crush...

Sent from my SM-A536B using Tapatalk

[fabio.dangelo]

One question but can you transcribe what the AI says as well?

[bojote]

6 hours ago, fabio.dangelo said:

One question but can you transcribe what the AI says as well?

Hi Fabio,

The AI component of this utility (Whisper AI) is meant to 'transcribe' (convert speech to text) as accurately as possible what YOU say into the micropone and convert it to plaint text. Once its coverted to text its pasted to your in-game kneeboard on the fly.

As you are probably aware, one of the most challenging aspects of speech recognition is doing it accurately. Whisper is *THE* most accurate Speech recogniton program in the market, its abiliy to translate your speech to text is remarkable and the accuracy is almost 100%.

My program is NOT an AI assistant or chat with AI, it uses AI simply to translate what you say into text as accurately as possible. Let me give you an example:

Imagine you are returning from a mission,you are flying back to your carrier at night and contact marshall for the inbound. The reply you get is usually a long list of vectors, radials, distances and time to commence your approach, every single value you receive is important. So, in this case while you are receiving the transmission, insteading of getting a pen and writing it all down you simply press a HOTAS/controller button and make a memo.. something like. Final Bearing 170, Mother's Radial 350, DME 22, Angels 7, Comence 48. That 'speech' is converted to 'text' and pasted (automatically) to your kneedboard for you to check later as many times as you want.

Not a lot people know that you can 'paste' plain text to your kneeboard, so my application simply transcribes your speech into text, copies it to the clipboard and then does a "paste" to the kneeboard in DCS.

null

Edited October 3, 2024 by bojote
grammar

[MAXsenna]

@bojote Excellent! Will have to try it out. I assume you just use VoiceAttack for simplicity, right?

Sent from my SM-A536B using Tapatalk

[bojote]

53 minutes ago, MAXsenna said:

@bojote Excellent! Will have to try it out. I assume you just use VoiceAttack for simplicity, right?

Sent from my SM-A536B using Tapatalk
 

Correct! Simplicity and convenience as you need to trigger the scripts with a button press/release. But any program that does this  will work, it does NOT have to be VoiceAttack

you can actually test the scripts stand alone and check how they perform

[MAXsenna]

Correct! Simplicity and convenience as you need to trigger the scripts with a button press/release. But any program that does this  will work, it does NOT have to be VoiceAttack
you can actually test the scripts stand alone and check how they perform

Do you think this could be a viable path for VoiceAttack, and change from Microsoft Speech?

Edit: Just found this.

https://forum.voiceattack.com/smf/index.php?topic=4687.0

Sent from my SM-A536B using Tapatalk

[bojote]

2 hours ago, MAXsenna said:

Do you think this could be a viable path for VoiceAttack, and change from Microsoft Speech?

Edit: Just found this.

https://forum.voiceattack.com/smf/index.php?topic=4687.0

Sent from my SM-A536B using Tapatalk

 

VoiceAttack uses the microsoft SAPI which uses whatever speech engine is installed, so it will use the new speech model

 

now, keep in mind I dont use voiceattack voice model or microsoft’s for that matter, im using whisper AI exclusively. The use of voiceattack is merely for the key assigments!

[MAXsenna]

VoiceAttack uses the microsoft SAPI which uses whatever speech engine is installed, so it will use the new speech model
 
now, keep in mind I dont use voiceattack voice model or microsoft’s for that matter, im using whisper AI exclusively. The use of voiceattack is merely for the key assigments!

Yeah! Got that part. It's just seems to me, if this Whisper thing is so awesome you know... Sometimes the MS Speech does strange things.
One more question. For your app to work. One needs to be online?

Sent from my SM-A536B using Tapatalk

[bojote]

43 minutes ago, MAXsenna said:

Yeah! Got that part. It's just seems to me, if this Whisper thing is so awesome you know... Sometimes the MS Speech does strange things.
One more question. For your app to work. One needs to be online?

Sent from my SM-A536B using Tapatalk
 

No online, whisper is a downloadable module, completely stand alone. It relies on your PC power

[Dunska]

EDIT - I realised I had to download your package from the releases section on github to get the scripts 

Hi @bojote - I'm trying to get this running.  I can install Python 3.11.x, torch and whisper, but I can't find the recorder.py or transcriber.py scripts anywhere.  There is a 'transcribe.py' script under the 'local-packages\Python311\site-packages' folder but no transcriber.py or recorder.py.  Any hints where I might find them?

Edited December 14, 2024 by Dunska

[nikoel]

@bojote - Thank you so much. I have installed it, and run the application (had to reinstall VoiceAttack too)

It's leagues ahead of the 1995 Windows Voice Recognition. I pity anyone who is not using this, the days of me saying 'Gear Down' and Voice Attack interpreting this as 'Eject Eject Eject' are over. Good riddance

It blows my mind that the developers have not jumped on this and integrated it within VoiceAttack by default since this is by far and away it's biggest weakness bar none

Thank you again

[buur]

Hi @bojote, thanks for the tool.

do you know if it is possible to train Whisper AI on words or special characters?

[skypickle]

cannot install torch

typing:

C:\Users\Stefan\Documents\Whisper>pip install torch --index-url https://download.pytorch.org/whl/cu118

gives:

Looking in indexes: https://download.pytorch.org/whl/cu118
ERROR: Could not find a version that satisfies the requirement torch (from versions: none)
ERROR: No matching distribution found for torch

[nikoel]

21 hours ago, skypickle said:

cannot install torch

typing:

C:\Users\Stefan\Documents\Whisper>pip install torch --index-url https://download.pytorch.org/whl/cu118

gives:

Looking in indexes: https://download.pytorch.org/whl/cu118
ERROR: Could not find a version that satisfies the requirement torch (from versions: none)
ERROR: No matching distribution found for torch

I'll quickly jump in and give you an answer as one of the users from our implementation of Whisper ran into the same issue

Make sure you Install from python.org or the Microsoft Store Python 3.11
V3.12 3.13 etc... will NOT work - PyTorch often only provides official wheels for Python 3.8 → 3.11

[skypickle]

That’s probably the issue- I installed 3.13

[intruder11]

On 1/2/2025 at 11:56 AM, buur said:

Hi @bojote, thanks for the tool.

do you know if it is possible to train Whisper AI on words or special characters?

https://platform.openai.com/docs/guides/speech-to-text#prompting (the local version of whisper supports this as well)

[Heling]

Can't install. I've python 3.11.0. In windows terminal I got the error "pip not found". In python terminal "install is invalid syntax". What can I do?

 

[nikoel]

14 hours ago, Heling said:

Can't install. I've python 3.11.0. In windows terminal I got the error "pip not found". In python terminal "install is invalid syntax". What can I do?

 

You likely haven't installed Python as PATH - you should see the following as PATH (google how to check it)

C:\Users\YourUsername\AppData\Local\Programs\Python\Python311\
C:\Users\YourUsername\AppData\Local\Programs\Python\Python311\Scripts\

If you're confident of the above and are sure pip is missing, you can install it using Python's ensurepip module

python -m ensurepip --default-pip

[skypickle]

ok, got python, got torch,etc

where do i find the scripts recorder.py and transcriber.py?

[buur]

vor 16 Stunden schrieb skypickle:

ok, got python, got torch,etc

where do i find the scripts recorder.py and transcriber.py?

you find the link to his git in the first post under "how to set it up" 

There you navigate to the latest release on the right side. Download the latest release and you have the scripts. 

[bojote]

If you are interested in the functionality of this tool I suggest you search for WhisperAttack, another user took over what I started and made it much better!