Please Heatblur, improve the RIO/WSO AI significantly

[twistking]

Hello Heatblur,
my favorite American jets are probably the A-6, the F-14 and the F-4, so i should be very happy with your roadmap...

However, after having tested the Tomcat during a free trial, i did not buy it, even though the visuals and simulation are obviously very good! I would even say, that Jester AI is "good", but it was still the thing that ruined the module for me. Ruined is a strong word and i want to stress that i think Jester implementation is decent in itself and i'm sure it was a lot of work to develop it. For my enjoyment it was simply not good enough... and this was my fear already when the module was still in development. Interacting with a module in DCS is - for me - the most immersive kind of interacting with any game or sim. Interacting with a slightly robotic Ai through a (necessarily) intricate GUI just takes so much away from it, that the magic of DCS just vanishes to some degree for me.

Having a human RIO/WSO does probably turn that upside down, but you at Heatblur figured out early that this is not a satisfactorily solution (and i agree), otherwise you would not have put so much effort into Jester.

I'd therefore wish that you'd improve your RIO AI even further by replacing the GUI with voice recognition and making the AI more human-like so that interacting it with it feels less mechanical. I am aware that thereby you'd push Jester beyond anything we've seen in the gaming/sim space concerning Ai interaction, but i think you are generally capable of pushing boundaries with your modules, while at the same time big advancements in AI make a humanlike virtual flight buddy seem a way more reasonable expectation than only a few years ago.

Thanks for reading!

[MAXsenna]

You do know that VAICOM has gone open source and is now free?

Cheers!

Sent from my MAR-LX1A using Tapatalk

[draconus]

18 hours ago, twistking said:

I'd therefore wish that you'd improve your RIO AI even further by replacing the GUI with voice recognition and making the AI more human-like so that interacting it with it feels less mechanical.

They can't just ditch the GUI for a voice based solution as it would require the users to have the necessary hardware (mic) and a place where they can freely communicate (noise free and allowing loud speaking). Having said that there's already voice option with DCS addon used by many and also the Jester is going to be overhauled to version 2.0, whatever it might be, both for F-14, F-4 and future modules.

12 hours ago, MAXsenna said:

You do know that VAICOM has gone open source and is now free?

Free one is limited to 20 commands though afaik.

Edit: OK, so VAICOM/AIRIO is free but the 20 commands limit applies to free license of VoiceAttack software which is requirement for VAICOM.

Edited June 2, 2023 by draconus

[Lt_Jaeger]

Nope  VAICOM with AIRIO is comepletely free.

OP - try VAICOM with the F-14. I did it from day one and I only talk to Jester. I don't even know how the Jester wheel looks, because I saw it as you, and just used the mentioned APP from day one. Yes, VAICOM has a steep learning curve, as the Tomcat has, but once pieces fall into place, it is very satisfying and would achieve what you are looking for.

 

For me....no VAICOM / AIRIO - no TOMCAT

Hope the community guys find a way to integrate Jester 2.0 as soon as he gets released.

[MAXsenna]

Free one is limited to 20 commands though afaik.

VAICOM is totally free now, with all extensions, but it depends on the full paid version of VoiceAttack, which I should have mentioned.

Cheers!

Sent from my MAR-LX1A using Tapatalk

[twistking]

Thanks for the replies. I know about VAICOM and while i haven't tested it myself, i did have a look at some youtube-content of guys using it with Jester.
Sure, it gets rid of the GUI, but i feel that the interaction is still very robotic. It's not terrible... but i guess "immersion" is very subjective. For me it did not click. On this forum, you'll find many people who think the same by the way, but this is not about right or wrong...
On the other hand, if Heatblur could improve their AI even more, everybody would appreciate that i think. Even those of you, who are already happy with the current version.

By the way, if you look at ED's VoIP FAQ, you'll see that ED are at least investigating native voice recognition in DCS: They list AI not reacting to voice commands as "known issues and future features". Sounds promising. Native voice would maybe allow Heatblur to push it a bit further in that direction. GUI could always be a backup for people without MIC.

[Bosun]

On 6/2/2023 at 6:14 AM, twistking said:

Thanks for the replies. I know about VAICOM and while i haven't tested it myself, i did have a look at some youtube-content of guys using it with Jester.
Sure, it gets rid of the GUI, but i feel that the interaction is still very robotic.

While I hear the desire for more human interaction here - I think it is worth noting a few things:
1. They had a live, real voice actor record the Jester audio.  That isn't a computer speaking, it's a human voice.
2. They've layered and strung the commands together in a way that is far more organic than any simulation I have ever played.  When Jester is already hyped, he has different filler and connecting phrases to link statements and cues.  'Nails, 5 o'clock..uhh, and nails, 6 o'clock!' - While it may sound repetitive after playing a lot, it is worth recognizing how much intricate work was done to make the dialogue cues respond to the situations in a more organic way, as opposed to 'Nails 6 o clock, Nails 7 o clock. Nails 10 o clock. Missile launched, evade." That would be boring and truly robotic.  Instead, you have 'Nails, 5'oclock, and missile missile MISSILE Break right!" 

I think, when you say it is too robotic - what you're really highlighting is the repetitiveness of continual play, hearing the same phrases.  While that's understandable, it is, currently unavoidable with any possible technology, and even in real life, cues and communication in aircraft are both routine, and repetitive - for a reason.  

The only possible way to increase the variety of what you're hearing, currently, would be to have several full sets of Jester cues for each individual call out - and the program would randomly string them together each time the cue was called by the situation.  

That means having 4 or 5 different ways for Jester to say '5 o'clock', and each iteration selected at random each time it was called in game.  That would also require the program to string them together organically as it does, for each iteration.  Those organic styles of strings were crafted individually by the designers across 10,500 individual sound recordings. 

Asking to make it 'more realistic' would require - at minimum - double that number, meaning a designer would have to sift through 21,000 individual sound bytes and program them to work and sound correct in each permutation. 

That would take years. 


Until we get a 'general AI' that has the awareness to properly string these things on their own, and situational context with which to do it that is organic, I do have to say that the current Jester format is about as close to a real human as you're likely to see.  


Can it be improved? Certainly - and they're already doing that.  Can it get a lot less repetitive and respond to your funny jokes?  Likely not.  All in all, they've done a herculean effort in individualizing call outs during game scenarios and situations, especially in the scripted mission campaigns.  I'd be hard pressed to say many folks have their immersion broken by Jester before the plastic joysticks, computer screen border, comfy office chair and pixelation does.  

Looking forward to Jester 2.0.


On another note - Grinds my gears - Calling any computer-controlled unit in a computer game "AI."  

There is no such unit or character in a video game that is computer controlled, and AI.  Doesn't exist.  It can't yet. (Though we're getting closer.) All the computer controlled aircraft, units and characters are simply rules-based trees of logic that have very specific programming allowing them to mimic behaviors seen in real life, without the context, awareness or responsiveness of an actually-aware entity.  

A computer opponent doesn't turn and fire because you're nearby and they see you're a good target.  It does so because you're inside a prescribed range, meet a certain amount of set criteria, and it's own status matches compatible parameters for the series of actions of engagement it's been programmed to perform.  Any algorithms involved are only factoring one or two things a time, like speed and distance calculations, and outputting a number that gets referenced continuously as the engagement goes on to be balanced against criteria for certain triggers to perform certain maneuvers.  There's no "AI Logic", no 'black-box' of calculations that are minimizing deviation from trained results, nothing that would be considered any interation of what we currently call 'AI.'  And it is leagues far and below anything that a 'General AI' would embody.  

And the same goes for Jester - it's not an AI, and we don't yet have the ability and technology to implement Jester as an actual AI - so the closest we can get is this manual approach to fidelity where you need 11,000 sound cues, individually recorded, and painstakingly paired in the game.  

If you can invent a way for an AI algorithm to run the Jester program - that would be the next leap forward.  The AI program could then take those 11,000 recordings, and approximate, interpolate and generate new responses based on them.  Theoretically possible - but I don't think anyone has tried it, and the reason why is computer power.  

You'd be flying that machine on an online server while having massive algorithms running continuously in the background, like having the ChatGPT server running in the background of your machine all the time.  

It would wreck your frames, to say the least.

[draconus]

28 minutes ago, Bosun said:

A computer opponent doesn't turn and fire because you're nearby and they see you're a good target.  It does so because you're inside a prescribed range, meet a certain amount of set criteria, and it's own status matches compatible parameters for the series of actions of engagement it's been programmed to perform.

Exactly what a trained pilot would think and do but you'd call it intelligence in this case?

[twistking]

1 hour ago, Bosun said:

[...]

With robotic i did not have Jester's reactions in mind, but the way you need to "talk" to him: Basically going through menus or hierachies of commands. Sometimes it feels more like programming a VHS recorder than communicating with a person. Also i'm well aware, that something i'd like to see, has not yet been archieved in a videogame. I've written that precisely so that people don't feel the need to go on that hell of a tangent about complexities of AI.

And by the way, videogame "AI" has been called AI since forever. There has never been another word for it really. It's an umbrella term, isn't it? Use AGI, if you want to differentiate from pseudo-intelligents, or use words, that describe the technical foundations of the "AI" you are referring to.

Edited June 8, 2023 by twistking

[Bosun]

1 hour ago, draconus said:

Exactly what a trained pilot would think and do but you'd call it intelligence in this case?

Yes - because a pilot continually evaluates as the situation progresses, whereas the computer-controlled aircraft can only re-evaluate within specific parameters at a much lower fidelity than the human brain can, due to lack of actual awareness, and lack of fidelity being input into the computer controlled aircraft, for it to react too.

For example - a computer controlled aircraft cannot necessarily look at the route you're flying, and make judgement calls on what it believes will be your intended target, or your likely reaction to being engaged.  It can only respond to what you do.  It cannot generate it's own behavior patterns based on subjective analysis, the way a real pilot can.  It cannot use a black-box algorithm of AI programming to 'train' it's responses over and over and be able to approximate that either.  

[Bosun]

3 hours ago, twistking said:

And by the way, videogame "AI" has been called AI since forever. There has never been another word for it really. It's an umbrella term, isn't it? Use AGI, if you want to differentiate from pseudo-intelligents, or use words, that describe the technical foundations of the "AI" you are referring to.

 

I hear you on the navigation side.  As others have commented - VoiceAttack and Viacom are both very good options to mitigate (not entirely get rid of, however, that effect.)  

AI is a general term commonly used. Prior to modern-day, I wouldn't have been bothered by it.  In recent years, however, with the rise of actual-form AI algorithms, I think it is worth noting the differences, and perhaps coming up with a different term to use for the style of Computer Controlled Unit programming you see in most games.  That is only due to the growing prevalence of true-to-form AI, and the ease with which the public is increasingly associating that term with certain styles of algorithmic programming, and possibility.

In short - we should consider evolving the terms we use, based on new and actual AI that is being created and used, so that realistic ideas of what capabilities it has are implied, and inherent, in it's name, and less people will have a disconnect on why it isn't as intuitive as actual AI. 

I can call a house fly a bird all day long - until an actual bird shows up. Then I need to re-evaluate how I label it, so that when I speak of it, the reader or listener has the proper context in which to place their frames of reference. (Crude analogy, sorry!)

[draconus]

2 hours ago, Bosun said:

Yes - because a pilot continually evaluates as the situation progresses, whereas the computer-controlled aircraft can only re-evaluate within specific parameters at a much lower fidelity than the human brain can, due to lack of actual awareness, and lack of fidelity being input into the computer controlled aircraft, for it to react too.

I am only noting that human pilot also operates upon finite number of parameters and information, his training and actions are highly procedural, for a good reason.

[Bosun]

On 6/8/2023 at 2:18 PM, draconus said:

I am only noting that human pilot also operates upon finite number of parameters and information, his training and actions are highly procedural, for a good reason.

While that is true, somewhat, it fails to address an actual human's ability to generate new parameters on their own, without programmer input, based on changing information, and to 'think on the fly', to re-evaluate strategy and approach based on subjective analysis through experience.  Or that the 'finite' number of parameters we operate under much higher, and are structured under heirarchies of priorities that we continually and dynamically change as the situation changes, based on aforementioned subjective analysis.  While there are programs that have approximated this, the depth to which you would need to build the logic trees to really make this approach work well is not reasonable or feasible for most development teams. 

In a turning dogfight, a human can 'assume', or 'guess' what an opponent pilot may due based on the evaluated energy state and previous maneuvers pulled, that they've observed, and fly to better take advantage of that assumed strategy by their opponent.  The computer-controlled pilots cannot 'think ahead' like that.  They only ever react to what you have already done.  None of their parameters include planning out iterations of what you may do, and choosing the best one based on it's own subjective analysis.  

In a BVR situation, a computer can only intercept what course you're on. It cannot know, nor even venture a guess, that you're taking a circuitous route to avoid interception somewhere else, or that your intended target is not in your flight path. That means it can only vector toward your current intercept, and cannot actually think ahead to simply be at your intended target to actually, effectively stop you.  

You can program computer-controlled planes to patrol that target when a plane gets wtihin "X" distance of a spot, as in triggers in the Mission Editor - but you cannot get a computer pilot to intepret the routes they're seeing - surmise likely target destinations, and change their flight goals or mission parameters to reflect that new information...until you do something that they can react too. 


Meanwhile - AI programs are fast-learning to look at sets of parameters and previous outcomes, and build ahead to generate outcomes that will likely be desired, but which have not been programmed as responses by any human input.  IE - they're learning to think ahead, and to create profiles of action or output that are based on what the likely possible outcomes a human would want, would be based on 'learned' or 'trained' experiences.  That's a defining characteristic of AI that the computer-controlled aircraft in this game do not have, and likely will not have any time soon. 

Computer Controlled Example: You are flying north to reach a target to the east, to approach it from the north, and not the west. Each time you turn towards the target after vectoring to the north of it, your flight path falls across the target area.  This triggers the computer pilots to react to you, and they vector from the target area, towards your plane, as they have been specifically programmed to do, based on that criteria.  They cannot deviate from that programming, if there is no further programmed function for them to do so.  No matter how many times you fly that path the computer pilot will never react until your flight path crosses that trigger-defined airspace within a certain range. 

AI Example:  You are flying that same flight path described above.  The 2nd time you fly it, the computer has 'learned' from your previous example that being 'to the north of it', regardless of your course, may indicate that you intend to attack it, and the planes from the target area vector towards you within a certain range of the designated airspace they're protecting, regardless your course.  The 3rd time you fly the route, the AI has picked up that it is your specific plane attacking it, and as soon as you enter in range of their intercept fighters, they vector toward you, even though you're still flying north and not approaching the target, and not within any range of their designated airspace. No one has programmed the AI to respond this way - it has simply generated the responses based on an over-arching goal to protect the target the most efficient way possible. This is much more similar of how humans set and think about goals, and this mimics what an online human player might do, using their experiences playing against another human pilot.

That's the difference between what real AI would be, versus what we have now - computer-controlled. 

Edited June 12, 2023 by Bosun

[Bosun]

Though I'd like to add to the thought process how cool it would be if Jester was an actual AI program.  Each pilot has certain characteristics to our flight and play style that Jester would begin to adapt too.  

For example, with a LANTIRN pod, I always do a right-hand circle around the target area. What if I could just tell Jester to 'Search for Targets" without any previous input on where to start looking, and just because the AI has learned that I normally circle that direction, begins his search in the direction of where I'm likely circling?  

What if Jester learned that when I flip the toggle switch to master arm on, I'm likely going to want him fenced in to countermeasures, and prompts me by asking which mode I'd like? Or if I've already flown in a scenario a few times, maybe he'd confirm he's setting it up the same way this flight?

What if I'm often prone to forgetting to raise the flaps on take off, and one day he suddenly says, "Don't forget the flaps" as we lift off? 

None of these would be programmed responses.  They would be the AI program learning how to prompt me for the most likely outcome I often try to engineer.  

We're a long way from that, but imagine if, on a return from a strike, the AWACS pre-emptively called you to check your fuel state and then informed you it had already vectored a tanker to intercept? Just because it had learned that the strike area was a long way away, and several planes prior that flew that route, had called tankers halfway back?  No one told it to do so, it just picked up that it was common, and incorporated that information in to how it checked in on the planes it was tracking to help generate the most desired outcome that it has been trained on.

This is where AI can get results that are of a much higher fidelity than hard programming could ever approximate.  Say the AWACS example above happened once, and a tanker diverted to help a plane, at the cost of having another aircraft that was vectoring towards it, lose the ability to catch it, and ran out of fuel.  Or had to expend too much energy in catching up to it.  The AI would begin to factor that negative result in to it's continued calculation, and the next time it went to divert a tanker somewhere, it would take in to account other aircraft and their states, before doing so, and possibly choose not to vector based on that evaluative strategy.  Now you've got a very high fidelity model of flight direction and command, and it would only get more complex and dynamic as time went on.  To the point, even, when one day we may ask, "Why is the tanker not responding"? The answer wouldn't be 'Oh, it's a bug." The answer would, at that point, more likely be a real world answer of, "They've probably got more info than we have, and is managing a larger fish than our situation represents."  

Imagine the day where you can curse the command chain of AI for failing to prioritize you, for very real-world scenario reasons of difficult prioritization!  High-Fidelity sim, here we come! haha.

 

Edited June 12, 2023 by Bosun

[wowbagger]

On 6/2/2023 at 10:50 AM, Lt_Jaeger said:

...Hope the community guys find a way to integrate Jester 2.0 as soon as he gets released.

Oh, that's a very good point. I wonder if Heatblur could help them get 2.0 integrated, now that it's community driven. Fantastic bunch of people keeping it alive and improving.

[wowbagger]

Just to add; it would be great if HB could work with the community to find a way that allows the Jester wheel and the AiRio extension to work together, rather than having to choose one vs the other.

 

Cheers

[near_blind]

It's been a long while since I've triffled with it, but I seem to remember hiding the jester wheel was a deliberate decision by the AIRIO folks. I don't remember there being a reason why it has to be removed other than they didn't like it.

[Scotia]

There’s a workaround for AIRIO on the discord to allow it to function with the wheel intact.

[draconus]

9 hours ago, near_blind said:

It's been a long while since I've triffled with it, but I seem to remember hiding the jester wheel was a deliberate decision by the AIRIO folks. I don't remember there being a reason why it has to be removed other than they didn't like it.

Since VA/Vaicom is meant to replace the comms menus/Jester wheel, it's only logical that it's no longer needed (or wanted) to be shown.

Edited June 29, 2023 by draconus

[near_blind]

Not my horse, not my race. Merely explaining its not a HB side decision.

[Ahogephilia]

Another aspect of VA/VAICOM,

that is not friendly to non-native English speakers like me...

 

Shame for me ... but you have no idea how much I struggle to pronounce "F", "TH", "CH", and "V" successfully. Everyone can learn to read and write, but speaking and listening are not so easy.

Edited June 29, 2023 by Ahogephilia

[wowbagger]

Anytime you needlessly limit player options, you are degrading the experience the game can offer. For example, I frequently don't play with headphones on - so no mic.

There are two ways to trick Vaicom into allowing the Jester wheel. 1) Get into the Tomcat  before activating AiRio with Vaicom. 2) There are three files which if you set to 'read only', Vaicom cannot change them and thus cannot turn off the wheel.  Trick #1 is onerous and not worth having to do for every flight. #2 now breaks the Vaicom kneeboard.

In any event, using the wheel along with AiRio can apparently de-sync Jester so you end up getting commands you didn't intend.

To quote Pene from the Vaicom community discord "It is possible to not suppress the wheel, however it means it will pop up on every command. This is due the way the wheel is implemented by HB. So I guess that was the design decision. I think the solution in the e short term will be to allow the deactivation of each VaicomPro extension via the UI which will give back control and choice to the user."

This is why I suggested it would be great if HB might be able to coordinate with the VCP community people to come up with a solution. Simply saying "you don't need the wheel with AiRio" isn't a solution, whatever your personal preferences might be.

 

[Extranajero]

3 hours ago, Ahogephilia said:

Another aspect of VA/VAICOM,

that is not friendly to non-native English speakers like me...

 

Shame for me ... but you have no idea how much I struggle to pronounce "F", "TH", "CH", and "V" successfully. Everyone can learn to read and write, but speaking and listening are not so easy.

 

I'm a native English speaker and speech recognition software doesn't understand anything I say either.
I do have a strong regional accent, but even making the effort to sound more " standard English " doesn't help much at all.
 

[Lurker]

On 6/1/2023 at 4:46 PM, twistking said:

the most immersive kind of interacting with any game or sim. Interacting with a slightly robotic Ai through a (necessarily) intricate GUI just takes so much away from it, that the magic of DCS just vanishes to some degree for me.

 

 

Never ever buy the Apache or the Hind.

[MAXsenna]

Another aspect of VA/VAICOM,

that is not friendly to non-native English speakers like me...

 

Shame for me ... but you have no idea how much I struggle to pronounce "F", "TH", "CH", and "V" successfully. Everyone can learn to read and write, but speaking and listening are not so easy.

To be fair, neither VAICOM nor VoiceAttack does any recognition.
VA records the audio and passes it to the Microsoft Speech Recognition Engine, which then does some trickery speech-to-text, and passes that text back to VA, which compares that text to the VAICOM keywords in the VA profile. If it fits with some of those, it runs that VAICOM plugin, which in turn sends the relevant command to DCS. That's why VAICOM doesn't use the radio menu.
For the Jester part, it actually use the Jester wheel in the background. I've experienced any desync using the wheel by tricks others have explained.

I'm not a native English speaker and have no issues at all, BUT I did get me a better microphone and did the Microsoft speech training 20 times. Yes, it was really tedious, but well worth it. I also use VAICOM in VSPX mode.

Cheers!

Sent from my MAR-LX1A using Tapatalk