If you have reached this site looking for turtles, you are going to
Windows "Keyed" Events, would have been nice...
Way back in Windows XP, MS introduced a new
synchronization primitive called "keyed events". These
were exposed through the undocumented native system calls
NtCreateKeyedEvent, NtOpenKeyedEvent, NtReleaseKeyedEvent,
and NtWaitForKeyedEvent. Keyed events were then used by
MS to implement (or reimplement) higher level win32
When I dropped support for Win2K some time back, I switched
the implementation of some of my own portable synchronization
primitives to use keyed events on Windows. This eliminated some
potential scalability issues and generally simplified things.
It allows sleeping and later waking a thread with a total
of 2 kernel mode transitions, nice.
Well, things are rarely simple. I am now seeing deadlocks at
process exit in one of my projects. The issue is that
NtReleaseKeyedEvent gets stuck in the kernel if the thread that
called NtWaitForKeyedEvent was terminated. A quick internet
search turns up this hit:
It seems that the keyed event implementation was not well
tested in combination with thread termination.
Many win32 gurus will point out that you should not terminate
threads, so this issue is the developers fault and not a fault
of the keyed event implementation. That conclusion is naive.
Terminating threads needs to be dealt with in Windows at least
for two reasons:
- The win32 TerminateProcess implementation implicitly
terminates threads. When building simple single binary
applications it is trivial to
make sure your threads are finished before TerminateProcess is
called by the runtime, but when building DLLs for use by 3rd
party client applications or modules, things are not as simple.
- Some Windows facilities (eg. ReadFile() and WriteFile()
to console handles) have a tendency to get stuck and prevent
process exit, causing problems even in relatively simple
For better or worse, thread termination is something that at
least needs to be dealt with at process exit time, a time
when the resource leak issues associated with thread termination
(Insert tirade here about loader-lock, DLL TLS issues, user-mode
win32 "handles", TerminateProcess implementation, and about
Microsofts inability to fix broken designs.)
The Windows CRITICAL_SECTION and SRW implementations uses keyed
events and are supposed to handle thread termination, so there
are work-arounds. But, if you look at how keyed events work,
the hang is not surprising and shows the keyed events themselves
are probably working as intended. My solution is to drop keyed
events and go back to using a pool of notification events.
ARM 64bit, Apple gets the ball rolling.
Apple does plenty of things I don't really agree with, but
kudos to them for finally getting relevent ARM64 hardware in
the field. I can't help but wonder now if there is a
Rosetta-x86 team at Apple.
Curious that Apple did not file a patent for "64 bit CPU
in a mobile device"? (sarcasm)
Seems like a bunch more ARM64 hardware is in the pipe. I am
looking forward to what new platforms and devices appear
over the next year or two. My wishlist is for a low cost
micro-server that has some actual I/O bandwidth (PCIe, 10GB
Binary Linux SDK.
I have maintained an internal "Linux SDK" (ptlinsdk) for the
last 3 years now, to build dependency controlled linux
binaries. This SDK installs (rather it builds) on any
reasonably modern linux or OSX system, and can then be used
to build X86/x64/arm/ppc linux binaries that will run on most
modern linux systems (5+ years back).
The SDK provides a GCC 4.4.5 compiler and various shared libs
to link against for using the host systems glibc, X11, GTK,
and some other libs. The expectation is that most libs beyond
those included in the SDK will be be built as part of a
project and statically linked, avoiding host system
dependency issues. In total the SDK currently includes 53
separate open source projects, the largest and finickiest
to build being gcc and glibc. Many of these projects needed
significant diagnostic work and patches to get building with
the specific versions of the other projects. Figuring out
the best set of versions of all the projects was a time
consuming endeavor, with lots of trial and error.
I won't say the time cost of the original work setting up the
SDK. There was no rational business motivation to do it, I am
just too bull headed to give up and support only Redhat. Even
tweaking the SDK to keep it working has proven a nuisance.
New build failures seem to magically appear everytime I install
the SDK to a new build system.
I am aware of crosstool and such. These projects do not solve
the specific problems I was targetting with the SDK, and in my
experience they did not work for the range of components and
versions I needed.
File ID support on Windows sucks, and keeps getting worse.
On my current project I need reliable file identifiers, a binary
blob that is unique to each physical file (not each file name)
on the system, including those on local and remote volumes.
On the *nix platforms, the st_dev and st_ino fields work well. At
any point in time on a running system, the st_dev-st_ino
combinatation is guaranteed unique for a given file. Hard links
can reliably be detected. Of course there a few edge cases that
can screw things up, most notably mounting windows file shares.
On Windows, file-id support sucks. The first major problem is
that there is no equivalent to st_dev, anywhere. In kernel mode
you can fake it by hashing the ptr to the volumes VPB or
DEVICE_OBJECT, but in user mode the only thing you can get at
is the name of the object, the volume create time, and the
volume serial number.
The general consensus by Microsoft seems to be that the volume
serial number is what you should use, but this fails pretty
quickly. Network shares return the serial number of the
source file system volume at the _root_ of the share, but you
get duplicate file ids if the share contains junctions or
mount points. Even with local volumes, all you have to do is
copy a VHD file and mount both copies to show how the
volume serial number is pretty useless as a st_dev replacement.
A hash of the native object volume/device name seems about the
only option, but getting this data cannot be done efficiently,
or deterministically. I expect I will end up implementing a
native object namespace prefix cache, with heuristics to try
to handle the issues with redirector style file systems.
Oh, and ReFS no longer guarantees it's 64 bit file ids are even
volume unique. For that you need to use the new Windows 8
info-types to query the 128 bit file id. It sure would have been
nice for them to at least provide a hash compressed 64 bit
file-id. Handling this case is going to slow things down even
I have had enough of C++, and I am not going to take it anymore.
(or, I wanna be a C-tard.)
After 20 years of using C++ (in my own peculiar way perhaps)
I am now officially a C-tard. I fought it for a while, but
there is no denying it.
I still build modular designs using virtual interfaces,
implementation hiding, polymorphism, and inheritance in
implementations. But, I do it now in C. It is not hard, it
is roughly the same amount of code and work as in C++, and
the explicity of doing it in C grows on you.
What finally drove me away from C++? Practical motivations
include the kernel environments I expect much of my code to
be compatible with, but it was also a long list of frustrations
trying to build, maintain, distribute, and/or support various
C++ based cross platform projects.
Now if MS would stop holding C99 hostage...