DCS Integrator

[ohall]

Hi All,

 

I wanted to let everyone building a sim-pit know about a new application I'm developing. It's called DCS Integrator and the intention is to provide a completely application driven integration option between DCS World (any aircraft with a clickable cockpit) and microcontroller boards. For those who want to go further, there are also base classes that can be extended for custom controls.

 

It's still a work-in-progress but you can keep up with development at http://www.dcsintegrator.co.uk

 

I'm hoping to have a release available for public consumption soon.

 

Best regards,

Oliver

[Retsilf]

Nice idea. I'm doing something very similar. It's written in C#

[draken152]

Hi it is looking promising.:thumbup: I am still not convinced how to get life to my FW 190D cockpit gauges . This will be great option... I wish you luck and patience in your work....What is your ETA for first release...

I am still at mechanical development and construction of my pit so I have time for final decision about electronic:prop::prop:...

[ohall]

Nice idea. I'm doing something very similar. It's written in C#

 

it's encouraging when other people are thinking along the same lines :)

 

DCS Integrator is a C#/.Net/WPF application too and handles all the coms natively, so no need for other utilities/tools to route the data etc.

 

I also read/modify/update (or create if it doesn't exist) the Export.lua hooks too, so I'm trying to get to a stage where a user doesn't need to know any of the dirty details of lua scripts in DCS (unless they have to, or create a custom output script... which is another "output device" on my hit list!)

[ohall]

What is your ETA for first release...

 

I'm really not sure yet - and I don't want to fall in the Eagle Dynamics mold of promising something and then not delivering ;-)

 

I haven't set up the installer yet, but I may get that done fairly soon, and then I'll be able to make Alphas available on the site. you won't be able to do much until I get the Arduino* library/reference code up to scratch (it's currently very much hacked together) but you will be able to play with e Windows application, and IMPORTANTLY, I'm keen to know if there are any issues loading various aircraft definitions.

 

ED (or 3rd party devs) don't appear to follow a consistent approach in the LUA files that define the gauges and clickable cockpits so I've had a couple of failures in loading aircraft info. I've got all of my installed aircraft working now (Ka-50, UH-1H, Mi-8MTV2, TF-51D, Su-25T**) but that doesn't mean the others will work without a few more tweaks!

 

* Hardware-wise, the Arduino is just the easiest platform to prototype on. Once I get the significant things squared away, I'll look at other MCU options (or others can come up with ideas once I publish the DCSI<->board protocol)

 

** The Su-25T doesn't have a clickable cockpit so that's severely limited to stuff like altitude, heading, lat & long.

[Warhog]

If we have several people working on basically the same vehicle, only a different color would it not be more efficient to collaborate and develop one all encompassing program that's does everything one could possibly want except maybe etch the PCB.

 

There is Ian who developed DCS-BIOS which is pretty much a proven entity, then Gadroc who is taking DCS-BIOS and making it into...not sure yet, ArturDCS with his USB program and then you two gentlemen which looks like your doing the same thing but not, however, it sounds like it's going to result in similar functionality.

 

Not being a programmer, I assume a project of this magnitude is great deal of work. A collaboration such as this would certainly have significant benefits. Dividing up tasks to reduce workload. Drawing upon specialties would allow the final project to grow beyond any one persons knowledge base. Several people working to the same goal would allow for more ideas to be tabled. More ideas equates better solutions. With the varying areas of interest, expertise and knowledge it would also be a great opportunity to share ideas and concepts and learn from each other. Certainly a win-win for everyone.

 

On the other hand it may be that programmers prefer to work alone. In that case what I just proposed is moot.

 

I thought I would throw this out for consideration as it sure would be interesting to see what comes out of a collaboration of DCS enthusiasts who have the skill to make computers do wonderful things for us.

 

Thanks for reading. I hope you will all give this some thought and what it would mean to the entire community as a result of your combined efforts.:thumbup:

[ohall]

Hi Warhog,

 

What you're suggesting is certainly the ideal for community developed projects, though that ideal doesn't always work out in practice.

 

I won't make any bones about it, I have some thoughts on how I may be able to monetise the work I'm doing, with a central ethos that the platform is free for hobby use, though not in any commercial activity - so opening up the "platform" project to a collaborative effort isn't part of the plan.

 

Ian's work is very impressive, and I did try out DCS-BIOS and even made a number of contributions to the GitHub repository, but with my pull request still sitting dormant after 8 months - to be frank - I got tired of waiting.

 

That experience was really what lead me to consider my initial proof of concept - reading in DCS aircraft files to avoid the process of having to build aircraft definition files for DCS-BIOS - and having passed that milestone fairly easily, I thought I might as well carry on.

 

I completely understand your point of view, and it is a chunk of work which I hope others will pitch in on (e.g. developing reference code for platforms other than Ardui-**spit!**-no ;)

 

All the best,

Oliver

[ohall]

What is your ETA for first release...

 

Not really a "release", but something to play with - go to downloads @ www.dcsintegrator.co.uk

 

All the best,

Ol

[Gadroc]

@Warhog

 

It's not necessarily that developers like to work alone (on the contrary the best code happens from multiple developers collaborating and keeping each other from taking shortcuts). There are multiple issues when trying to co-ordinate a hobby project like this. First is schedules, for example Ian has school exams and moving recently which left him little time to work things like pull requests. With out being able to communicate closely it was easier for me to split the arduino code and work on my version of it. I'm more than happy to contribute it back. Second is we have different itches that we are trying to scratch. So i'm interested in enhancing the library in different ways. Given sufficient communications that actually is a good thing as we'll make progress in multiple areas at the same time. With out being able to do that we end up recreating work.

 

 

@Oliver

Looks good so far. It's pretty close to what I was doing with EOS. On your post I saw mention of bit banging a RS-485 bus. I'd do the math on timing, in order to keep up with the update rate of a cockpit I don't think it's feasible to run at those kind of baud rates (9600). If a button or switch takes over about 500ms it's noticeable when using the cockpit. Early test may look good, but I would definitely mock up the 10-20 boards it will take for a console before committing fully to an approach. Also make sure you do it with a few things that have a constant stream of data like gauges (VVI, IAS, etc...). I'd also keep a keen eye on memory consumption. You're putting a lot of dynamic things on the ATMEGAs which have very limited RAM.

 

One my lessons from Helios is the "binding" approach is not clear for most users. For developers it's seems easier than code, but the concept is not as clear for most users. I had several power users who got it and made wonderful profiles, I was more inundated with my doesn't it just work questions when configuring profiles. While I'm sure GUI clean up could have helped, the tone of the questions ran deeper than that. It's an up hill battle to do the "make it easy" approach. Pick a target audience and stick to it.

[ohall]

Hi Gadroc,

 

Thanks for the tips - I'm certainly at the early end of the process so I appreciate the heads up on the various things that may bite me as configs grow. I have to admit I don't have any experience with your stuff, as I had understood Helios was geared towards glass cockpit stuff, and I hadn't seen EOS until just now.

 

My hope (and it's just that at the moment) is to ameliorate some of these issues by only transmitting information that matters to the cockpit, and has changed since the last update, rather than have DCS dump the entire cockpit constantly. My testing is currently at 115200 baud with a value update message length of 63 bits (including stop bits). I know I won't get anywhere near (or require!) the theoretical limit of 1828 updates/sec - once I get the code complete I'll start pushing the limits and see where things break down!

 

I've written the firmware for a DMX lighting controller which has some fairly precise timing constraints at 250k baud, and a few other RS485 master/slave devices, though never using Arduino before, so without getting down and dirty with the interrupts I'm not sure how it'll turn out! I noticed you've gone low-level on with EOS - was that because the Arduino libs block?

 

I'm developing everything on a few Nano's at the moment - memory wise I can't see the device hosting more than a handful of devices due to the limit on pins, and each device instance is pretty lightweight - It's a good point though and I'll build a test to see how many devices I can instantiate before it keels over!

 

All the very best & thanks again for the advice,

Ol

[ohall]

Hi Gadroc,

 

You've got me thinking now - do you have any example data for what a decent simpit might have in terms of volume of controls/data points to represent?

 

My perspective has been very much towards the Huey with limited/unsophisticated controls whereas I expect the A10 is far more involved. I just wondered what a good sized cockpit might require?

 

Cheers,

Ol

[Gadroc]

I went low level on EOS to remove some problems with the 1.0x Arduino serial libraries. The flush command did not wait for all UART data to be sent, so you could not implement the transmit pin drop to listen for responses. In addition the old serial libraries used ints for everything so where much slower and consumed significantly more ram. Also I was able to collapse to one buffer for serial reading and protocol parsing.

 

1.5+ Arduino libraries have fixed many of these problems, and since I've moved to a streaming protocol (DCS-BIOS) instead of a pure packet based protocol I don't need two buffers. So my newer DCS-BIOS based Arduino code just uses the existing Stream based Serial objects.

 

I'll check some of my data tonight, but there are roughly 10-15 panels per console (center, right and left consoles). I believe there is somewhere on the order of 400 switch positions that need to be monitors, and 150 indicators lights, 50 indicator needles.

[pappavis]

If we have several people working on basically the same vehicle, only a different color would it not be more efficient to collaborate and develop one all encompassing program that's does everything one could possibly want except maybe etch the PCB.

 

The advantage of few independant devs;

* Know-how is spread across the community i.e. >=1 person knows the technical ins & outs

* If a dev stops, temporarily or forever the community can jump over to the alternative.

 

This, more of the same = in our case = good.

[FSFIan]

For DCS-BIOS, when flying around in a Huey, the typical update has between 110 and 130 bytes.

In an A-10C, it is between 130 and 150, up to 170 bytes if I play with the throttles to make the engine instruments move around a lot.

The initial update (containing the complete cockpit state, not just data that has changed) takes 342 bytes for the UH-1H and 758 bytes for the A-10C.

Keep in mind that this also contains the position of every switch, dial, etc. which you probably don't need to export to your physical panels.

 

Obviously these numbers will not directly apply to whatever protocol you are using, but they should give you an idea about the relative complexity of the Huey vs. the A-10C.

 

 

Regarding duplication of effort: There are also some guys on the German forums who are implementing a new gauge pack for HawgTouch and a program to talk to Arcaze USB boards (German forum thread, GitHub).

 

It is probably inevitable that we will end up with multiple approaches with different design goals. To some extent, this is even a good thing for the end user, because each program will try different things, we can see what works and what doesn't, and a little competition can improve the overall quality.

There is however one part where we should really try to work together, and that is a common Export.lua file and aircraft definitions. As an end user, I'd want to be able to mix and match the programs that best fit my requirements without worrying about which one handles which aircraft and without having three Export.lua files that all export the same data in slightly different formats, each one incurring a performance penalty because it blocks the DCS process for a short time.

 

If I can convince enough developers, I'd like to cooperate to design "the last Export.lua file you will ever need" (which was the original goal of DCS-BIOS). At this point, it is just an idea that has been nagging me for about a week or so, but here's a few requirements that have crossed my mind:

• Simple key/value export format, capable of exporting floats (cockpit arguments) and arbitrary strings (can cover everything else, e.g. CDU content, current aircraft name)
  
• Support UDP and TCP protocols
  
• Provide an input API to trigger LoSetCommand(), performClickableAction(), etc. and the ability to execute a snippet of Lua code in case we miss something
  
• Ability to add aircraft-specific add-on Lua files to export things that are not cockpit arguments (e.g. the CDU), which can be reloaded at runtime to speed up development
  
• Have the developers of at least three projects cooperate on this from the very beginning so it can hopefully become the de-facto standard
  
• Out of scope: TacView-like export of all objects in the game world
  

 

Once we have something like that, the number of applications that consume this data would matter a lot less, because they can run on another CPU core or even on a different machine altogether.

 

The next step might be to come up with a machine-readable representation (XML, JSON or Lua tables, maybe all three) to describe the data that is exported. Things like "this is a three-position toggle switch", "this argument value represents a pointer angle and this is the scale of that gauge (in case you want to display a numeric value on a character LCD)". That could eliminate a lot of the work of adding a new aircraft module, because you would only do it once and have the result work in all programs.

[doveman]

Very happy to hear you're working on this ohall. I can't code myself but a while back I suggested a system with a program that lets the user map physical controls attached to Arduinos to sim cockpit controls, with switchable profiles for different aircraft. Eventually it could even be used for other sims like X-plane, which listens for commands and sends data on UDP.

 

Obviously everyone has different physical controls, so even though the Arduino only has to output a generic signal for the PC program to translate, the Arduino code will be different for each person. I imagine it would be possible to have a program that lets the user fill in the necessary info (e.g. 2-position switch connected to pin 5) and the program generates the necessary code to upload but that's probably the most complicated part due to the number of possible setups it would have to accommodate. There's only a finite number of controls and ways of connecting them though, so it should be possible to identify them all and accommodate them.

[ohall]

Eventually it could even be used for other sims like X-plane...

 

That's feasible, though the death of many a project is scope creep, so I'm going to focus on DCS World for now. I know nothing about X-plane and don't want to dilute the effort at this point.

 

I imagine it would be possible to have a program that lets the user fill in the necessary info (e.g. 2-position switch connected to pin 5)

 

That's exactly what DCS Integrator does, though...

 

and the program generates the necessary code to upload

 

... rather than generate code, the Arduino code generates object models (e.g. a button or LED) dynamically and then configures anything that is control specific (such as the range of an encoder or potentiometer). As you say, there's a bunch of "well known" controls which are provided by default, and then more complex stuff would need a bit of programming-fu to realise (though all the coms is encapsulated in the generic base classes).

[ohall]

Hi Ian,

 

Thanks for the reply. Is it worth creating a specific thread where everyone with a vested interest in this stuff can discuss design ideas?

 

Fundamentally, we appear to have different views on getting data in and out of DCS. As you know, I started off using DCS-BIOS and whilst I think it's a quality piece of work (and I have enjoyed many hours setting up my armament and tuning my ADF using your libraries), there are some basic design principles that I don't agree with.

 

I've stolen this from the storage world, but I'm a firm believer in the adage "the cheapest I/O is the one you don't need to do". DCS-BIOS (to my understanding) is based around a memory mapped block export of everything that changes, regardless of whether the cockpit needs it or not. Although the memory map concept provides a very efficient way of getting the data out, you lose a lot of that efficiency *if* all you want is a subset of the data. Naturally the break-even line moves depending on how complete a cockpit you require.

 

My motivation was to be able to fly different aircraft using a generic set of cockpit controls rather than build a faithful Huey cockpit - I don't have the time, money, space and I might find I'm pushing the tolerance envelope with my wife if I did. DCS-BIOS (again, to my understanding) is currently a one aircraft deal (unless I re-flash all of the Arduinos in between switching aircraft). We did discuss this on GitHub I think, though I believe the issue is that there's currently no way for DCS-BIOS to tell the Arduino to "initialise" at the start of a mission?

 

It's for those reasons (which I appreciate I may have misunderstood, so apologies if I have misrepresented things) that I decided that what I wanted (and thought others might want to) would require dynamic generation of export files, and a configurable mapping model. Managing this in the Export.lua is impractical, so I've gone down a middleware application road.

 

Admittedly, it's another layer between the sim and the hardware, but it provides a layer of abstraction that possibly meets your goal of a single "export" (and control API) - just not the way you've described.

 

Please don't take this as a challenge to DCS-BIOS, as a number of people have said, different requirements need different solutions, and I completely accept that you've been doing this for a long time, and I am still hopping between one hacked piece of code to another so I may find tomorrow that my eyes are opened to some huge over-simplification I'm making... I won't be too proud to admit it if I'm talking nonsense!

 

Sorry - this is the kind of chat that's perhaps better off in a specific thread?

All the best,

Oliver

[Gadroc]

I think you have over coupled DCS-BIOS extractor and the DCS-BIOS Arduino libraries. The extractor makes no assumptions about the hardware interface, it just supplies data. It also supplies all data as it makes no assumptions about the hardware interface. I agree with keeping it that simple as it removes burden from all down stream projects which use it (like mine, I don't want to have the "configure" DCS-BIOS every time I talk to it).

 

All that being said it does mean DCS-BIOS will almost always be paired with a hardware controller component (like DCS Integrator, Helios or a custom one like my cockpit controller). Interceptor could still speak DCS-BIOS to DCS World and translate to your desired protocol model for the hardware integration. I actually have code that does that where I was following your same principle and only transmitting the data to each RS-485 bus needed for that bus. After getting the code stabilized and doing timings the effort wasn't worth the squeeze so I just started wrapping the DCS-BIOS data in RS-485 packets. This brought my code simplicity down significantly.

 

It should be very possible to enhance DCS-BIOS to send the aircraft type as a few bytes at the beginning of each "frame". When the hardware controller component (DCS Integrator application) see this change it can re-initialize it's hardware configuration.

 

The nice thing about this type of design is it moves most of the decision logic out of the flight simulation processing loop. I run the hardware controller software on a raspberry pi so the DCS rig has full CPU to do physics and graphics.

[ohall]

Hi Gadroc,

 

I realise that the two are separate entities, though it doesn't really change the points I was making. I think there might be some misunderstanding of my (somewhat proposed) solution too. So, to "decouple":

 

DCS-BIOS exports out all cockpit data, regardless of what's actually required. I'm just commenting that, given that the previous discussion was somewhat biased towards data rates/volumes, this design decision isn't as "frugal" as it could be.

 

DCS-BIOS currently doesn't indicate the start of a new session and which aircraft is in play - sure, it could be extended to do so. But at present it doesn't and I haven't seen any movement in that direction. Any consumer of the DCS-BIOS data stream has to be told (by something else) how to interpret that data. Don't get me wrong, I know none of this is insurmountable - I'm just trying to find a solution to (what I see as) a limitation.

 

DCS-BIOS Arduino Library:

Assumes a static/single-type cockpit. Each control is bound to a specific aircraft's clickable cockpit element.

 

The nice thing about this type of design is it moves most of the decision logic out of the flight simulation processing loop.

 

I've clearly not explained my angle on what I think is a "better" design properly either. I'm not proposing putting any decision logic on DCS processing loop - in fact, quite the reverse. When a session starts, I take a reference to a lua table that lists the items required for export for that particular aircraft (and nothing else). Only those items are exported, and only when the value changes. After the initial "I am aircraft type: xyz" message, there's no more information in the updates than "device, argument, value" - so it's fairly system agnostic.

 

DCS Integrator (or whatever consumes *those* packets) can sit anywhere on the network - it doesn't have to be on the DCS machine.

 

The practical reality is that DCS Integrator is still in the womb and may be still-born - so this may all be moot anyway!

 

All the best,

Ol

[FSFIan]

Is it worth creating a specific thread where everyone with a vested interest in this stuff can discuss design ideas?

 

Done: Designing a common Export.lua file

 

Let's take the discussion over there (and sorry for derailing this thread :) )

[doveman]

That's feasible, though the death of many a project is scope creep, so I'm going to focus on DCS World for now. I know nothing about X-plane and don't want to dilute the effort at this point.

 

Absolutely agree, I'm just imagining a happy future where we can use one program for all our sims :)

 

That's exactly what DCS Integrator does, though...

 

... rather than generate code, the Arduino code generates object models (e.g. a button or LED) dynamically and then configures anything that is control specific (such as the range of an encoder or potentiometer). As you say, there's a bunch of "well known" controls which are provided by default, and then more complex stuff would need a bit of programming-fu to realise (though all the coms is encapsulated in the generic base classes).

 

I'm not sure from what you've said whether we're talking about the same thing. What I meant is that there needs to be code uploaded to each Arduino for the various controls attached to it and it would be great if this code could be generated by DCS Integrator (or some other program) after the user tells it what controls are attached. If that's the idea though, fantastic!

[ohall]

We are talking about the same end goal, but we're getting there by slightly different routes.

 

What you're saying:

A program would auto-generate Arduino code for upload. This is definitely doable but, without a lot of work/dependencies, would still require you to load the generated code into the arduino ide and compile/upload up a new version if anything changes.

 

What I'm saying:

The "standard" arduino code (providing you're using the "built in" primitive controls) is loaded up to each arduino board once, though the code doesn't "know" at this stage what controls are connect to the various pins. DCSI then talks to the board at startup and tells the board which types of controls are attached where. If you change/add a new control, just restart the DCSI config and the new control should be functional.

 

Gadroc's pointed out that there are potential issues with this approach (memory management) so it's not a panacea, but I'm hoping to be able to manage that with suitable checks and balances. Keep your fingers crossed for me ;)

[doveman]

I'm crossing everything I can ;)

 

Your method of dynamically configuring the boards on the fly does sound much easier (for users). I don't pretend to understand how that's possible but I don't really need to, so don't waste your time trying to explain it to me when you could be coding :)

 

If your approach doesn't work out, you might want to look at the Edtracker GUI, as that is able to upload code without the Arduino IDE. I'm not sure if it's open-source though.

[ohall]

I'm writing unit tests right now ;) ... well, not *right* now... I can't multi-task.

[ohall]

Arduino Library Available

 

Just a quick note:

 

As well as a very minor update to DCSI to allow users to specify the location of aircraft config files (hopefully this will work with the DCS 1.5 beta, though I've not tested it yet) I've also posted an alpha release of the Arduino library which includes support for a number of simple input/output devices and also supports an RS485 bus for multiple boards.

 

All available from www.dcsintegrator.co.uk