Also, please note that my comments weren’t specifically directed at you in person, rather they are general comments about people who are dependent on the source code. Given the extremely limited context your case may or may not have fitted into that situation, but it now appears it doesn’t. No offense intended.

It’s not a bug in the network driver, I was testing over loopback. I guess it could be a bug in the loopback driver, but that seems rather unlikely.

It’s not 16k current connections, it’s 16k sequential connections (the concurrency is more like 20, but it’s irrelevant). It actually happens across process lifetimes too (e.g. make 10k connections, kill, start again, stall at about 6k connections). Since the code I was testing with maintains no state across runs, it’s simply not possible for a bug in my code to cause problems across process lifetimes.

I’m not dependent on having source code available. I’m dependent on the kernel doing the right thing. Fortunately I’m only running xnu on my workstation :)

None of these problems are caused by a lack of source code. In every case, it is possible to diagnose the problem without looking into the kernel source code. In fact the source code may well be a red-herring as in the example you mention with the network card drivers. By looking into the kernel source you only check the drivers, but neglect to check that the actual network card hardware isn’t at fault. A better way to check that the problem isn’t specific to the network card is to switch it with another type - preferably one that is known to work on another OS. Digging through kernel source looking for a potential cause of the problem will take far longer than swapping the network card and testing it again. Even if you don’t have a spare network card, profiling will determine whether or not there is a resource leak of some kind and usually point fairly accurately to any bottle necks, even in third party code which you don’t have the source to.

If you a completely dependent on having source available it is usually an indication that you haven’t developed the appropriate skills with tools like profilers, debuggers and general debugging/problem solving. If you like having the source code because it makes you feel more comfortable or you personally prefer to dig into the source instead of using profilers etc then that’s fine - but having the source code is pretty clearly not required for serious development as the number of applications available for proprietary OS’s clearly indicates.

Did you actually run any tests of your own, or are you just willing to accept that because it’s “grown up software” it isn’t horribly broken? Based on a very long personal experience of Bob’s knowledge of the mac, I’m inclined to trust what he’s saying here - not to mention the fact that I’ve done similar tests myself and OS X really does have atrocious TCP concurrency and throughput support out of the box. Based on very long personal experience of software, if it looks broken, it’s probably broken :).

Keep in mind that Bob mentioned that the same code runs without problems on *multiple* other platforms, so a resource leak is unlikely. It’s possible (though unlikely) that the leak is compensated for by the exact same bugs or non-standardized behavior in other platforms that don’t share code, of course, but without the kernel source to OS X there’s no way to trace the behavior all the way down to the hardware to figure out _why_ it’s crapping out. On linux, even if the problem isn’t my fault, I can see what my code is making the kernel do. This is super important. I’ve never written a driver in my life, and I don’t even compile my own kernel any more, but I still look at the source to linux on a regular basis.

There are dozens of ways Apple might be compensating for this, so that they can run their servers on their own platform and not embarass themselves. Here are a few:

- the web is a pretty low-concurrency network application, since most connections are dropped as soon as they are service. it’s doubtful that they have 16,000 connections concurrently. In other words they may not need to compensate for the problem at all, it’s just not a problem in the applications they’re running.
- they could use banks of front-end servers to avoid heavily loading any of them; after all, it’s pretty cheap for apple to get a big pile of macs.
- they could be using a development or “enterprise” version of the kernel which does not ship to the general public. (This is unlikely, considering their kernel source has been open in the past, but Micrsoft intentionally hobbled its network stack for years before shipping a working one with the non-”server” version of their OS)
- they could be using different network cards than the one Bob has tested with (maybe it’s a driver problem? no way to tell! no source for the driver!)
- they could be using non-mac network appliances on the front end (e.g. a Cisco load-balancer) to handle the sockets or a front-end proxy for static content, and trickling dynamic application connections through to the application servers to keep the load at a fixed level.

A sufficiently dedicated Mac fan might do all of these things on their own site, but why bother, if other platforms do this job better? The software in question has already been determined to run on them.

Blaming the lack of kernel source for not being able to track this down is merely a distraction for the fact that you aren’t correctly analyzing your code.

Needless to say I’m not interested in deploying anything on OS X after this, but I do like to run tests on my workstation and expect them to pass when my code is correct.

Otherwise, if you don’t have a large number of OS X clients machines to deal with, OS X on a server is a pointless waste of time and effort, compared to the free as in beer and software alternatives (Solaris, GNU/Linux, *BSD, …).

On the other hand, I own a mac. The reason that developing software for the mac is a pain is because you have to use a third party tool like ‘fink’ or ‘darwinports’ to install software. You’ve got to use both because some pieces of software are broken on both.

And then you run into compatibility problems with ‘normal’ software (X11, autotools) that just doesn’t work right unless you do the incantation in exactly the right way to build against your fink and darwinports libraries.

And then you run into problems with Apple just Getting It Wrong, like my saga recently with apple ‘fixing’ ranlib.

The Free OSs are great because the package management works, and everything builds.

OSX has neither of those qualities.