LOCERF 10-2 Announcement

[Aliboy]

Yes Its very unfair because I play with a ping of around 180 on the 51st&4c (Euro Server?) and around constant 270 on the 104th&3sq(US server) and I never get accused of warping or stuff like that even when there are 40+players.

[X-man]

Right, but do you also get the point here? Once again black and white, right

and wrong etc....but do you get my point here? Those who failed or were difficult to get started (even though 1.12) WHY?

 

Often due to settings of the server. Like views not being set, server messages being on, ATC being on and a few times LRM. I cant really recall a scrapped event due to mission design. (Usually LOCERF missions were fairly light)

 

Infact, I can only remember a handful of events that was actually canceled due to issues, out of the 30+ events (Not only LOCERF) that were held.... Most, eventhe ones that started off with some tech issues, were flown and finished as intended.

Edited June 22, 2010 by X-man

[SEFalcon]

LOFC is very hard for hosting in any way and client PC to. LOFC server need PC settings (excellent configuration) to work without problems. Clasic server not need sound, not need 3d graphic etc. LOFC need all of that. That is the problem. Many pilots had better pc configuration than every LOFC server machine. What i want to say with this?

 

Maybe guys who make some missios, make it on very very fast PCs and no see any problem. But when same map put on the server (a litle slowly machine) problems sudenly coming. Why? Because this is LOFC and he eat resources, so if you have minimum configuration for server machine...

 

Yes, sterss test is No1. Maybe with clean map with the only planes slots and minimum graphic settings. And step by step put other units on the map and every time test it. Just like as <51> guys. FC1.12 and 2.0 is very different for hosting (i mean every server must be stronger for 2.0 hosting defintley). Or resize map, decrease number of units, trigers, scritps..etc. EVERY server had crashes, maybe just becuse excessive missions and average server machines.

 

I hope so we will have lessons about whole LOFC setings and servers from this. Thing is simple, LOFC 2.0 need super pc (server) for hosting 40+ players on the map with to many units.

 

ED programers?

[X-man]

What is your expierience with active acting or also moving ground units and the amount of them?

My experience from 1.12 is that moving units affect the server a lot more then stationary units. I would assume this is the same for FC2

[Frostie]

Since we tried the RF mission on the 104th server this makes me think, what was different from normal 104th server day to day operations?

 

- Different mission

- Hi group of players in China with high ping

 

One of the biggest differences with a casual server that gradually builds up in player size is the fact that all 60 join at the same time in a LOCERF event.

 

In LOCERF 1.12 everybody joined into the server lobby where a staggered load in into the game ensued, but now in FC2.0 there is no server lobby so we all effectively load into the game as spectators at the same time. So now everybody's tacviews, tracks and whatever else are starting up all creating a bottleneck immediately. Then all thats required is a small measure of time combined with a pinch of sods law and hey presto, implosion of the server.

[104th_Crunch]

One of the biggest differences with a casual server that gradually builds up in player size is the fact that all 60 join at the same time in a LOCERF event.

 

In LOCERF 1.12 everybody joined into the server lobby where a staggered load in into the game ensued, but now in FC2.0 there is no server lobby so we all effectively load into the game as spectators at the same time. So now everybody's tacviews, tracks and whatever else are starting up all creating a bottleneck immediately. Then all thats required is a small measure of time combined with a pinch of sods law and hey presto, implosion of the server.

 

We did though try staggered joining on Blue when we tried on the 104th server for this RF. Then I think most of Red came in close to each other. The lag started to really get bad about 15 minutes in when the first big furball took place, then the server crashed. Just to review.

[SFJackBauer]

One of the biggest differences with a casual server that gradually builds up in player size is the fact that all 60 join at the same time in a LOCERF event..

 

I believe you are into something here. Lock-on way of loading files is weird. Notice when you start the sim fresh and enter the cockpit first time, every degree you look around it stutters because it is loading things, even though the "loading screen" has already passed. Now assuming it happens on the server too, imagine every client and server having to load files for first time, then depending on amount of data to load and disk performance, desyncs are bound to occur.

 

Even though we joined the server gradually as Crunch noted, I remember clearly that in the first kill message I noticed heavy stutter, and other people reported lag on chat.

 

In LOCERF both the server and clients were started clean. In MP sessions the server is running for a longer time, and have in cache almost every data file.

 

Perhaps the server should have a "warmup mission", where lots of events occur like AC being destroyed, ground units spawning etc, so the executable can build its cache before the main mission gets loaded.

[Laud]

@=SE= Falcon: Most of the servers I know beeing used for those events are close to my home setup. Meaning something like a E8400 CPU, 0.5-1gb graphic-card, 4gb RAM,... This shouldn't really be the problem. Especially because they're run on quite low settings: no blur, basic haze, low resolution...

 

Focus should stay on load in sequence, mission design (triggers, moving or static units) and exports.

[4c Hajduk Veljko]

@=SE= Falcon: Most of the servers I know beeing used for those events are close to my home setup. Meaning something like a E8400 CPU, 0.5-1gb graphic-card, 4gb RAM,... This shouldn't really be the problem. Especially because they're run on quite low settings: no blur, basic haze, low resolution...

 

Focus should stay on load in sequence, mission design (triggers, moving or static units) and exports.

Our server is a quad core Q6700 @2.66GHz with 2GB of RAM memory. Video card is GIGABYTE GV-R465OC-1GI Radeon HD 4650 1GB. We usually run our missions at medium settings.

[Moa]

'Mission warm-up' as has been suggested will help stutter on the server but not on clients. When spooling as a client you can see the hills get rendered with progressive levels of detail as more stuff gets loaded in (very clever by ED). However, there is stutter (due to disk IO probably) as this happens - and during loading of new textures and effects for kills etc. As each client stutters it will cause stutter in their network replies to the server (in software not fully threaded). It used to be the case in FC 1 that a long lag in reply could cause problems in the server as it struggled to cope with bad/missing client packets (sometimes such packets were noted in the network log IIRC), occasionally there would be a null pointer caused somewhere which would kill the server. I'm guessing this may still be the case. If so, then that can only be fixed by ED I think - handle all cases of bad/missing network data without letting a null pointer slip past.

[104th_Crunch]

'Mission warm-up' as has been suggested will help stutter on the server but not on clients. When spooling as a client you can see the hills get rendered with progressive levels of detail as more stuff gets loaded in (very clever by ED). However, there is stutter (due to disk IO probably) as this happens - and during loading of new textures and effects for kills etc. As each client stutters it will cause stutter in their network replies to the server (in software not fully threaded). It used to be the case in FC 1 that a long lag in reply could cause problems in the server as it struggled to cope with bad/missing client packets (sometimes such packets were noted in the network log IIRC), occasionally there would be a null pointer caused somewhere which would kill the server. I'm guessing this may still be the case. If so, then that can only be fixed by ED I think - handle all cases of bad/missing network data without letting a null pointer slip past.

 

Good point. Take the fact that when a new client joins and loads in to a server, every client gets a stutter. It was there in FC1, but know it is much worse in FC2.

[Protos]

As each client stutters it will cause stutter in their network replies to the server

 

occasionally there would be a null pointer caused somewhere which would kill the server. I'm guessing this may still be the case. If so, then that can only be fixed by ED I think - handle all cases of bad/missing network data without letting a null pointer slip past.

 

 

:thumbup:

Linux dedicated server client with server side hit detection scheme configurable for timeout/pkl/clientrate .....ect ect

[Moa]

:thumbup:

Linux dedicated server client with server side hit detection scheme configurable for timeout/pkl/clientrate .....ect ect

 

Yes. This makes sense to me. The following could be performed on the server:

* weapon guidance and hit detection scheme server-side (printscreen cheat on clients would not affect missile impact).

* enforce weapon guidance only locked targets or those the pilot is looking at.

* since a second machine is involved the missile guidance calculations could be made far more realistic than at the moment.

* basic fuel, countermeasures, and stores caclulation performed periodically server-side.

* flight path checks, no more clients moving at Mach-25 and pulling more.

 

If it was written in Java (yeah, I'm a pusher, but only 'cause it is a good drug) then you would have:

* Host anywhere: Windows (32- or 64- bit, any version from 2000 onward, without recompile!), Linux, Mac etc.

* Good multithreading support (C++ resource management with threads is just too hard to get right). You get to use all 6 cores dual-threading on that lovely Phenom II x6 1090 T you just bought.

* Users can use free tools without restriction (don't have to buy or steal IDE and SDK like .NET Windows tools).

* Get to use all the good existing libraries (log4j, JUnit, Spring, Metro etc etc) without re-inventing the whole stack from scratch. Java has great built-in libraries but the 3rd party stuff cannot be paralelled in scope.

* Produce TacView-like output server-side only which is later streamed to clients.

* Stop the flamewars over whether and how much Integrity Checking to use - it would no longer be necessary to perform IC.

 

The only thing holding us back is knowing the network protocol of LockOn, otherwise we could produce such a thing and it would scale to many users and more complex battles than we have now. Unfortunately the packet protocol is undocumented :(

 

There are those who argue that weapon collision detection cannot be performed on the server due to lag. Well, the 'truth' has to be determined somewhere and the server is the only trusted and shared place to do it. Plus, the server has a single consistent view of the battle, rather than each client having a slightly different view due to lag. Sure the client will have visual cues that are slightly different from the server's picture, preventing last minute G pulls to avoid missiles, but that is not such a bad thing and you can still think you have dodged fire due to what you see on screen but still die (although usually this is more likely with very low-flying and ground collisions).

 

Since the server wouldn't be running graphics it would do a better job than any client.

 

The other things about not requiring graphics means that the any of the large number of game hosting services can be used to serve a game. At the moment it is very expensive to provision an appropriate server with a good connection since a rackmount is desirable (co-locations like these) but most rackmounts don't come with video cards. In the USA it is not a problem to get these but elsewhere (bordering Antarctica like me) this is an expensive hassle.

 

If you throw in a server that can be run on Linux (as well as Windows, of course) then someone doesn't have to pay the Microsoft Tax to host. You would see a lot more LockOn servers out there as a result.

 

So, the question about dedicated servers really boils down to, is it possible to get or reverse engineer the LockOn client-server protocol? Will ED be so kind as to document this for us, so we can do the rest?

Edited June 24, 2010 by Moa

[Protos]

Good Post .... doubt ED will EVER give that kind of info out. I certainly would never expose my multiplayer code. It can be reverse engineered rather easily .... but without the source its really not worth the effort to fire up wireshark.

 

I have a feeling that ED is eventually going to go the Serverside route sooner rather than later. Its really the only logical choice.

 

It will just take a while, look how many of the forum people here are flying a 'sim' on a laptop that wouldn't run word 2010 properly. The basic issues have to be addressed first. Lack of a 64 bit client being the first (hopefully soon to be corrected) and then more goodies can come.

 

In the end, we are a boutique market and trying to have airquakers and simmers together is not going to work well. So go whole hog and bring in all the hardcore jockeys from F4 and other titles. That along with the new integrated server browser and the IC checks is a really good beginning. Its all up to Wags.

[GGTharos]

ED is working to improve the DCS line in every way; but as has been said before, 'a step at a time'. Things that will be re-written or corrected are chosen based on available resources and other factors.

You've already read about some of things that are coming in A-10C, you can be certain that the module after it will have additional enhancements.

Patience :)

[sobek]

* Good multithreading support (C++ resource management with threads is just too hard to get right). You get to use all 6 cores dual-threading on that lovely Phenom II x6 1090 T you just bought.

 

And you're going to need all those cores because the high abstraction of java comes at a price.

[RvEYoda]

And you're going to need all those cores because the high abstraction of java comes at a price.

 

I disagree with you, but also somewhat with Moa. Java requires nowhere near several times more power.

(I talk about language here, so I am not comparing standard STL classes to the java counterparts

and so on, both are rather crappy though :P )

 

In fact in many situattions java is faster because it _always_ deals with references (pointer

performances) rather than copies, so your programming is often more efficient

unless the C++ programmer is very careful.In pure math, Java is comparable to C++

performance (except for trigonometrics because standard java enforces some special rules,

but this can be circumvented)

 

Second Java nowadays is compiled down to native code by your JVM before execution, and

in many situations it actually beats classic C code. (which didnt have as good compiler optimizations

among other reasons, I'm no expert on these details)

 

Java problems: developer hesitations, non-standard approaches and much more

complicated situations to connect 3rd party libraries such as required for 3d graphics.

 

However a combination such as game in java, graphics in C++ (like IL2 which is both pretty

and runs fantastically smooth).

 

----

 

But, what I've seen as a major flaw when using java so far is the over-abstraction and "cut and

paste" programming present in oh so many 3rd party libraries, for example I was doing

programming in JOGL (that is openGL for java) and there are so many runtime workarounds

and checks to render just 1 frame that in my own light weight program, the overhead to

render each frame actually becomes larger than the graphics itself.

 

But if you write everything yourself, from scratch, I would go for Java no doubt.

Bottom line is you wont need 4 more cores, maybe you will need 0.5 more cores,

maybe 0.5 less :)

 

(Then again there is another point which is that your development and debugging

time when using java will go down by..hmm.....personally I would say 90% ;), and

I was a really strong C++ supported and hated everything that had to do with "java" )

Edited June 24, 2010 by =RvE=Yoda

[Moa]

And you're going to need all those cores because the high abstraction of java comes at a price.

 

Your statement was correct until the middle of the last decade (for server-side/plain Java). Client-side Java was not as good until Java 1.6.0_u10, then all drawing became hardware (shader!) accelerated. That is not to say you don't have to be careful performance-wise with Java, of course you do, but Java is now faster single threaded than C++ and very much faster when used multi-threaded (which is used far more often than C++, much makes it faster). Java is the sensible strategic choice for new projects for the reasons in my earlier post (better tools, more libraries yada yada).

 

Please read the following reports, including the INRIA supercomputing report that says Java is faster than C++, C (!) and approaching FORTRAN (which is fast since it is so simple). The Java libraries aren't so good for intensive networked applications (eg. scientific simulations) but LockOn is using a very slow 128/64 kbs for most clients which is sloth-country in comparison.

 

http://hal.inria.fr/inria-00312039/en

http://www.facebook.com/note.php?note_id=24763131623

http://kano.net/javabench/

 

Also note that IL2 is largely written in Java. This eased porting it to the PS3 where it could tap into the large market of consoles. Like I said, Java is a sensible (safe) strategic choice, and you can always degrade to JNI (or the even easier JNA) when necessary (mostly for specialised hardware support).

 

I used C++ for a decade. Then I used Java for a decade. I know which I prefer to write my applications in. C++ made perfect sense when writing LockOn no doubt at the time. If you are writing new stuff from scratch I don't think limiting yourself to C++ on Windows 7 makes business sense when you could tap far more platforms than that. After all, why support a declining 89% of desktops and ignore consoles when you could deploy to 100% of desktops, and also consoles [compile Java to native using gcj for them] and the smarter phones (which are becoming powerful and have OpenGL ES hardware).

 

Yoda: You are completely right. The problem with 'Java' is not the language but the too-many-layers-of-abstraction libraries written by CS graduates who think that more layers of indirection == better. Write stuff youself and you don't have the problems.

Edited June 24, 2010 by Moa

[Moa]

One thing I should mention that the Java world does bring is Test Driven Development (TDD) and tool support for it. Yes this can be done on C++ but it requires a team with exceptional discipline to do it - and it is even harder if you have a legacy source code base to write coverage for.

 

For me using TDD is about time and money. It doesn't remove all defects but gets you much further down the exponential-decay curve of bugs remaining in code than manual testing. Plus the test automation means you're not afraid to do testing (issue a command instead of days of manual labour running tests, checking results, fixing, re-running the tests ...). You're also not afraid to add enhancements as you notice and fix regressions immediately. I worked on a long-term high-performance C++ project and know all about the paralysis mode where adding features gets very hard and you avoid doing sweeping changes since it is so risky.

 

Of course LockOn is an interactive GUI system which makes it hard to test some parts of it. However, it is an 'iceberg' in the way that some code is dedicated to rendering but far more is submerged and not directly visible to the user. Those parts that can have test automation applied ought to. It will save time, money, and developer frustration. Expensive human tester time ends up dealing only with the interesting cases - not the routine stuff.

 

I have no pecunary advantage in suggesting Java and Test Driven Development. I'm just trying to impart some wisdom from the trenches - since the State of the Art in reliable software development has moved in recent years. It's also that as a consultant developer myself LockOn seems to have many simple bugs that could easily be fixed by adopting a TDD process, and seems to be hard to adapt for multi-core (which is one of the principal reasons I suggest new standalone parts be developed in Java). Once you start using these processed working any other way seems like you were a craftsman in the Dark Ages - even if TDD needs to be applied judiciously and is not a panacea for all developmental issues. I appreciate how hard it must be for the ED team to get stuff going and keep it going as new features are added. It would be nice if new features could be added without too much pain (cost) to ED and in a reasonable time (to placate the unreasonable users :)).

Edited June 24, 2010 by Moa

[104th_Crunch]

ED is working to improve the DCS line in every way; but as has been said before, 'a step at a time'. Things that will be re-written or corrected are chosen based on available resources and other factors.

You've already read about some of things that are coming in A-10C, you can be certain that the module after it will have additional enhancements.

Patience :)

 

Very cool. I hope when there is an aircraft for DCS that allows full player vs player i.e. a fighter, that the focus will be heavy on multiplayer including performance. Still, from what I know, FC2 is the best MP sim going right now.

Edited June 25, 2010 by Crunch

[Moa]

Very cool. I hope when there is an aircraft for DCS that allows full player vs player i.e. a fighter, that the focus will be heavy on multiplayer including performance. Still, from I know, FC2 is the best MP sim going right now.

 

Amen. DCS is the best sim.

[Fudd]

I think what we call agree on is the FC2 behaves the best and has the most stability when running it stock. No ATC, no scripts, no tacview, no server export, ect, etc. Especially if we are going to stress an already unstable platform with 50-60 clients its stupid to try anything else.

 

Want I want to do is test FC2 within its own confines. Standby for dates and times. I've started making missions to see exactly what our server and the software will and will not tolerate. We will also be testing staggered loadins by passing out the passwords in individual studs channels to see if that makes it easier for everyone to join.

 

I think its safe to say though that LOCERF will remain as stock as possible until we iron out all other issues.