Coding in the Mac Kernel

June 12, 2007

Should You Program in the Kernel?

If you are thinking of writing code for the kernel environment,think carefully. Programming in the kernel can be a difficult anddangerous task. And often there is a way to accomplish what you want todo without touching the kernel.

Software that resides in the kernel tends to be expensive. Kernelcode is “wired” into physical memory and thus cannot be paged out bythe virtual memory system. As more code is put into the kernel, lessphysical memory is available to user-space processes. Consequently,paging activity will probably intensify, thereby degrading systemperformance.

Kernel code is also inherently destabilizing, much more so thanapplication code. The kernel environment is a single process, and thismeans that there is no memory protection between your extension ordriver and anything else in the kernel. Access memory in the wrongplace and–boom– the entire system can grind to a halt, a victim of akernel panic. Moreover, because kernel code usually provides servicesto numerous user-space clients, any inefficiencies in the code can bepropagated to those clients, thereby affecting the system globally.

Finally, kernel software is a real pain to write. There aresubtleties to grapple with unknown in the realm of applicationdevelopment. And because the tools for kernel development are in anearly stage of maturity, bugs in kernel code are harder to find than inuser-space software.

With all this in mind, the message is clear. It is in everyone’sbest interest to put as little code as possible into the kernel. Andany code that ends up in the kernel should be honed and rigorouslytested.

When Code Should Reside In the Kernel

A handful of situations warrant loading a driver or extension into the kernel environment:

  • The software is used by the operating system itself.
  • Multiple user-space programs will frequently use the software.
  • The software needs to respond directly to primary interrupts (those delivered by the CPU’s interrupt controller).

If the software you are writing does not match any of thesecriteria, it probably doesn’t belong in the kernel. If your software isa driver for a disk, a network controller, or a keyboard, it shouldreside in the kernel. If it is an extension to the file system, itshould live in the kernel. If, on the other hand, it is used only nowand then by a single user-space program, it should be loaded by theprogram and reside within it. Drivers for printers and scanners fallinto this latter category.

Alternatives to Kernel-Resident Code

Apple provides a number of technologies that might let you accomplish what you want to do andstay out of the kernel. First are the higher-level APIs that give yousome hardware-level access. For example, the CFNetwork and POSIXnetworking APIs are a powerful resource for many networkingcapabilities and Core Graphics (Quartz) enables you to do some fairlylow-level things with the graphics subsystem.

Second, and just as important, is the “device interface” technologyof the I/O Kit framework. Using a plug-in architecture, this technologymakes it possible for your application to interact with the kernel toaccess most hardware. In addition, you can–with a little help from theI/O Kit–use POSIX APIs to access serial, storage, or network devices.

e documents special to each particular type of driver or extension.

–>.

–>


Getting and Setting Values for Mac Generic Kernel Extension

June 9, 2007

Lets extend our generic kext created in previous post to to accept values for setting values from Kext as well as to get the values of those.

Lets start our example to implement a sysctl node with two variable : an integer and a string.We will call the new node mysysctlnode. and it will have the following properties.

  • It will be a subcategory of the existing top-level sysctl node called debug.In other words , new node’s MIB-style name would be debug.mysysctlnode
  • One of its two children will be called uint32, which will hold a 32-bit unsigned integer. The integer’s value will be readable and writable by any user.
  • The second of its children will be called string, which will hold a string up to 16 characters in length ( including the terminating NULL character).The string’s value will be read by anyone but writable only by the root user.

When our syctl kext is loaded , the kernel’s sysctl hierarchy would look like the one shown in figure , with possibly other top-level categories depending on the kernel version.

| – debug

| |

| | – mysysctlnode

| | |

| | | — uint32

| | | — string

The most general way to create a sysctl variable is to use the SYSCTL_PROC() macro , which allows a handler function to be specified for the sysctl.The handler is called when the variable is accessed for reading and writing. There exist data-type-specific macros such as SYSCTL_UINT() for unsigned integers and SYSCTL_STRING() for strings. These sysctls defined using these macros are served by predefined type-specific functions such as sysctl_handle_int() and sysctl_handle_string().We will use SYSCTL_PROC() to define our sysctl variables , with our own handler functions , although we will simply call the predefined handlers from our handlers.

Lets update the source code to include these sysctls declarations.

#include <sys/systm.h>
#include <sys/types.h>
#include <sys/sysctl.h>

static u_int32_t k_uint32 = 0; // Contents of debug.mysysctlnode.uint32
static u_int8_t k_string[16] = { 0 }; // Contents of debug.mysysctlnode.string

//Construct a node { debug.mysysctlnode } from which other sysctl objects can hang
SYSCTL_NODE(_debug, // our parent
OID_AUTO , // automatically assign us an object ID
mysysctlnode , // our name
CTLFLAG_RW, // we wil be creating children therefore , read/write
0, // Handler function ( none selected )
“demo sysctl hierarchy “) ;

//Prototypes for read/write handling functions for our sysctl nodes
static int sysctl_mysysctlnode_uint32 SYSCTL_HANDLER_ARGS;
static int sysctl_mysysctlnode_string SYSCTL_HANDLER_ARGS;

// We can directly use SYSCTL_INT() , in which sysctl_handle_int()
// will be assigned as the handling function.We use SYSCTL_PROC() and
// specify our own handler sysctl_mysysctlnode_unint32()

SYSCTL_PROC ( _debug_mysysctlnode, //our parent
OID_AUTO , // automaticall assign us an object ID
uint32, // our name
( CTLTYPE_INT | // type flag
CTLFLAG_RW | CTLFLAG_ANYBODY), //access flag ( read/write by anybody )
&k_uint32, // location of our data
0, //argument passed to our handler
sysctl_mysysctlnode_uint32, //our handler function
“IU”, // our data type ( unsigned integer )
“32-bit unsigned integer” // our description
);

// We can directly use SYSCTL_STRING() , in which sysctl_handle_string()
// will be assigned as the handling function.We use SYSCTL_PROC() and
// specify our own handler sysctl_mysysctlnode_string()

SYSCTL_PROC ( _debug_mysysctlnode, //our parent
OID_AUTO , // automaticall assign us an object ID
string, // our name
( CTLTYPE_STRING | // type flag
CTLFLAG_RW), //access flag ( read/write by root )
&k_string, // location of our data
16, //maximum allowable length of the string
sysctl_mysysctlnode_string, //our handler function
“A”, // our data type ( string )
“16-byte string” // our description
);

static int sysctl_mysysctlnode_uint32 SYSCTL_HANDLER_ARGS {

// Do some processing of our own , if neccesary
return sysctl_handle_int( oidp, oidp->oid_arg1 , oidp->oid_arg2 , req );
}

static int sysctl_mysysctlnode_string SYSCTL_HANDLER_ARGS {

// Do some processing of our own , if neccesary
return sysctl_handle_string( oidp, oidp->oid_arg1 , oidp->oid_arg2 , req );

}

kern_return_t DummySysctl_start (kmod_info_t * ki, void * d) {

printf(“DummySysctl_start \n”);

// Register Sysctl entries.
sysctl_register_oid(&sysctl__debug_mysysctlnode);
sysctl_register_oid(&sysctl__debug_mysysctlnode_uint32);
sysctl_register_oid(&sysctl__debug_mysysctlnode_string);

printf(“Registered Sysctl entries \n”);

return KERN_SUCCESS;
}

kern_return_t DummySysctl_stop (kmod_info_t * ki, void * d) {
printf(“DummySysctl_stop \n”);

// Unregister Sysctl entries.
sysctl_unregister_oid(&sysctl__debug_mysysctlnode);
sysctl_unregister_oid(&sysctl__debug_mysysctlnode_uint32);
sysctl_unregister_oid(&sysctl__debug_mysysctlnode_string);

printf(“Registered Sysctl entries \n”);

return KERN_SUCCESS;
}

Lets compile and load the kext to test it. ( Use previous blog post steps to load the built kext ).

Once its loaded use sysctl command to get and set the values of sysctl variables.

$ sysctl debug
……….

……….

debug.mysysctlnode.uint32: 0
debug.mysysctlnode.string:

$ sysctl -w debug.mysysctlnode.uint32=56
debug.mysysctlnode.uint32:  -> 56

$ sysctl debug.mysysctlnode.uint32
debug.mysysctlnode.uint32: 56

$ sysctl -w debug.mysysctlnode.string=mykernel
debug.mysysctlnode.string:
sysctl: debug.mysysctlnode.string: Operation not permitted

$ sudo sysctl -w debug.mysysctlnode.string=mykernel
debug.mysysctlnode.string:  -> mykernel

$ sysctl debug.mysysctlnode.string
debug.mysysctlnode.string: mykernel


Creating a Generic Mac Kernel Extension

June 9, 2007

In this post , we will discuss how kernel extensions are created and loaded in Mac OS X.

Note : Apple urges third-party developers to avoid the programming in the kernel unless absolutely necessary.

Lets create a trivial kext – one that implements the start and stop points. Once we can compile and load the kext we will extend it to implement a couple of sysctl entries in next post. We will call our kext DummySysctl.

We begin with XCode project instantiated from the template for generic kernel extensions. Since sysctl implementation is a BSD -only endeavor, we need to specify our kext’s dependency on BSD KPI.

We will use the kernel version of printf() function to print messages in our kext. Therefore , we need libkern,which provides the print().

Configuring KEXT

The contents of the Info.plist should look like as follows

<?xml version=”1.0″ encoding=”UTF-8″?>
<!DOCTYPE plist PUBLIC “-//Apple Computer//DTD PLIST 1.0//EN” “http://www.apple.com/DTDs/PropertyList-1.0.dtd”&gt;
<plist version=”1.0″>
<dict>
<key>CFBundleDevelopmentRegion</key>
<string>English</string>
<key>CFBundleExecutable</key>
<string>DummySysctl</string>

<key >CFBundleName</key>
<string>${PRODUCT_NAME}</string>
<key>CFBundleIconFile</key>
<string></string>
<key>CFBundleIdentifier</key>
<string>com.osxbook.kext.DummySysctl</string>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundlePackageType</key>
<string>KEXT</string>
<key>CFBundleVersion</key>
<string>1.0.0d1</string>
<key>OSBundleLibraries</key>
<dict>
    <key>com.apple.kpi.bsd</key>
    <string>8.0.0</string>

    <key>com.apple.kpi.libkern</key>
    <string>8.0.0</string>
</dict>

</dict>

</plist>

Beginning with Mac OS X 10.4 , a kext can declare dependencies either on new style kernel programing interfaces (KPI) or on compatibility interfaces. KPI dependencies are specified thru com.apple.kpi.* identifiers, where as the others are specified through com.apple.kernel.* identifiers. The Former start from version 8.0.0 ( Mac OS X 10.4 and newer ), where as latter end at version 7.9.9 ( Mac OS X 10.3 and older ) .

Lets see the source file created by XCode for us. The start function is called when the kext is loaded , and the stop function is called when its unloaded. XCode inserts the skeleton implementations of these functions in the automatically generated C file for the Generic Kernel Extension project template. We have added a printf() statement to both functions. The following shows the code of DummySysctl.c .

#include <mach/mach_types.h>

kern_return_t DummySysctl_start (kmod_info_t * ki, void * d) {

printf(“DummySysctl_start \n”);

return KERN_SUCCESS;
}

kern_return_t DummySysctl_stop (kmod_info_t * ki, void * d) {
printf(“DummySysctl_stop \n”);
return KERN_SUCCESS;
}

Lets now compile the kext. Now build the XCode project. Upon successful compilation it creates the DummySysctl.kext in the build folder of XCode.

Now Open the Terminal and move the folder where the built kext exists.

$ cd cd /Users/janakiram/KernelExtensions/DummySysctl/build/Release/

$ ls
DummySysctl.kext

Since loading a kext requires the kext bundle’s contents to have root and wheel as the owner and group , respectively, a typical compile – test – debug cycle would involve copying the kext bundle from the build directory to a temporary location – say to /tmp/ and using chown command on the copy. As we noted earlier , besides ownership, the modifiability of objects within the bundle also matters – the bundle’s contents must not be writable by any user except root.

$ cp -pr DummySysctl.kext /tmp/

$ sudo chown -R root:wheel /tmp/DummySysctl.kext

Now we can use the kextload to load the kext manually.

$ sudo kextload -v /tmp/DummySysctl.kext
kextload: extension /tmp/DummySysctl.kext appears to be valid
kextload: loading extension /tmp/DummySysctl.kext
kextload: sending 1 personality to the kernel
kextload: /tmp/DummySysctl.kext loaded successfully

If the kext fails to load , the -t option os the kextload may provide information about possible problems. For example , suppose we specified an unavailable version of the dependency – say version 7.9.9 for com.apple.kpi.libkernel , then -t option would be helpful in identifying the cause of the problem.

Besides the dependency resolution failure , other reasons for a kext’s failure to load include incorrect file permissions , a flawed bundle structure, a missing CFBundleIdentifier property in kext’s Info.plist file , and a missing or syntactically involved Info.plist file.

We can use the kextstat command to check whether our kext is currently loaded in the kernel

$ kextstat

Index Refs Address    Size       Wired      Name (Version) <Linked Against>
    1    1 0x0        0x0        0x0        com.apple.kernel (8.8.3)
    2   24 0x0        0x0        0x0        com.apple.kpi.bsd (8.8.3)
    3   41 0x0        0x0        0x0        com.apple.kpi.iokit (8.8.3)
    4   42 0x0        0x0        0x0        com.apple.kpi.libkern (8.8.3)
    5   39 0x0        0x0        0x0        com.apple.kpi.mach (8.8.3)
…..

….

 95    0 0x2f1bd000 0x2000     0x1000     com.osxbook.kext.DummySysctl (1.0.0d1) <4 2>

The value 95 in the kextstat output indicates the index at which the kext is loaded. The Kernel uses these indices for tracking the inter kext dependencies. The second value , which is 0 in this case , shows the number of references to this kext.The next value ,0x2f1bd000, is the kext ‘s load address in the kernel virtual address space. The next tow values ,memory and wired  0x2000  and 0x1000 , represent the amounts of kernel memory and wired kernel memory, respectively,used by the kext. The final value in the column is a list of indices of all other kexts that this ket references. We see that DummySysctl references two kexts : the ones loaded at indices 4(com.apple.kpi.libkern) and 2 (com.apple.kpi.bsd).

Once we are done , we can unload the kext manually using kextunload command.

$ sudo kextunload -v /tmp/DummySysctl.kext
Password:
kextunload: unload kext /tmp/DummySysctl.kext succeeded

The output from the printf() statements we inserted in our kext should appear in the /var/log/system.log

$ tail -f /var/log/system.log

Jun  9 14:38:02 administrators-computer-7 sudo: janakiram : TTY=ttyp1 ; PWD=/Users/janakiram/Kernel Extensions/DummySysctl/build/Release ; USER=root ; COMMAND=/sbin/kextload -v -t /tmp/DummySysctl.kext
Jun  9 14:38:09 administrators-computer-7 kernel[0]: DummySysctl_start
Jun  9 14:58:06 administrators-computer-7 sudo: janakiram : TTY=ttyp1 ; PWD=/Users/janakiram/Kernel Extensions/DummySysctl/build/Release ; USER=root ; COMMAND=/sbin/kextunload -v /tmp/DummySysctl.kext
Jun  9 14:58:08 administrators-computer-7 kernel[0]: DummySysctl_st


Creating a Simple Mac Kernel Extension… by hand

June 7, 2007

Basically, a KEXT is a bundle on Mac OS X (it’s also a “package”), which means it is a directory structure with some predefined form and an Info.plist file. Our sample KEXT will be named “MyFirstKernelExtension.kext”, and it will be in a directory structure that looks like this:

$ find MyFirstKernelExtension.kext
MyFirstKernelExtension.kext
MyFirstKernelExtension.kext/Contents
MyFirstKernelExtension.kext/Contents/Info.plist
MyFirstKernelExtension.kext/Contents/MacOS
MyFirstKernelExtension.kext/Contents/MacOS/MyFirstKernelExtension

To start off, we need to make the basic directory structure.
$ cd
$ mkdir -p MyFirstKernelExtension.kext/Contents/MacOS
$ cd MyFirstKernelExtension.kext/Contents/MacOS

Now we can start writing our code. Our code will contain two main routines: MyFirstKernelExtensionStart() and MyFirstKernelExtensionStop(), which are called when the KEXT is loaded and unloaded respectively. It will also contain some required bookkeeping code that’s needed in order to make our compiled binary proper. Our start and stop routines look like:
// File: MyFirstKernelExtension.c
#include <libkern/libkern.h>
#include <mach/mach_types.h>

kern_return_t MyFirstKernelExtensionStart(kmod_info_t *ki, void *d) {
printf(“Hello, World!\n”);
return KERN_SUCCESS;
}

kern_return_t MyFirstKernelExtensionStop(kmod_info_t *ki, void *d) {
printf(“Goodbye, World!\n”);
return KERN_SUCCESS;
}

… more to come in a minute

After these two methods (in the same MyFirstKernelExtension.c file) we need to put the required bookkeeping stuff.
extern kern_return_t _start(kmod_info_t *ki, void *data);
extern kern_return_t _stop(kmod_info_t *ki, void *data);

KMOD_EXPLICIT_DECL(net.macjunkie.kext.MyFirstKernelExtension, “1.0.0d1”, _start, _stop)
__private_extern__ kmod_start_func_t *_realmain = MyFirstKernelExtensionStart;
__private_extern__ kmod_stop_func_t *_antimain = MyFirstKernelExtensionStop;
__private_extern__ int _kext_apple_cc = __APPLE_CC__;

This stuff basically declares some needed structures, and it also sets up our routines (MyFirstKernelExtensionStart() and MyFirstKernelExtensionStop()) so that they’re called on load and unload, by assigning them to the _realmain and _antimain symbols respectively.

OK, now comes the tricky part: the compile. KEXTs are compiled statically, they can only use certain headers that are available in the kernel, and they can’t link with the standard C library. These requirements basically translate into a gcc command like the following:

$ gcc -static MyFirstKernelExtension.c -o MyFirstKernelExtension -fno-builtin -nostdlib -lkmod -r -mlong-branch -I/System/Library/Frameworks/Kernel.framework/Headers -Wall

If the planets are properly aligned, you won’t get any errors or warnings, and you’ll end up with a Mach-O object file in the current directory named MyFirstKernelExtension. This is your actual compiled KEXT. (At this point, it’s fun to inspect this file using otool. For example, otool -hV MyFirstKernelExtension, and otool -l MyFirstKernelExtension. Read the man page for otool(1) for more details here.)

Now, the last thing we need to do (before we actually load this thing up) is to give our KEXT an Info.plist. The easiest way to do this is to copy another KEXT’s Info.plist file, and change the names of a few things. For this example, I’m going to copy /System/Library/Extensions/webdav_fs.kext/Contents/Info.plist.
$ cd ..
$ pwd
/Users/janakiram/MyFirstKernelExtension.kext/Contents
$ cp /System/Library/Extensions/webdav_fs.kext/Contents/Info.plist .

Now, you’ll need to edit the file and change the value of the “CFBundleExecutable” key to MyFirstKernelExtension, and the value of “CFBundleIdentifier” to net.macjunkie.kext.MyFirstKernelExtension (or whatever you set that value to in your MyFirstKernelExtension.c file).

Okay, it’s show time. To load any KEXT, all files in the KEXT must be owned by root and be in group wheel. The files must also have certain permissions in order to load. Here’s the steps to load the KEXT.

$ cd /tmp
$ sudo -s
# cp -rp ~/MyFirstKernelExtension.kext .
# chown -R root:wheel MyFirstKernelExtension.kext
# chmod -R 0644 MyFirstKernelExtension.kext
# kextload -v MyFirstKernelExtension.kext
kextload: extension MyFirstKernelExtension.kext appears to be valid
kextload: loading extension MyFirstKernelExtension.kext
kextload: sending 1 personality to the kernel
kextload: MyFirstKernelExtension.kext loaded successfully
# tail -1 /var/log/system.log
Dec 15 20:15:47 janakiram-mac kernel[0]: Hello, World!

We can see that our MyFirstKernelExtensionStart() was called. Now let’s unload it and see what happens.

# kextunload -v MyFirstKernelExtension.kext
kextunload: unload kext MyFirstKernelExtension.kext succeededstKernelExtension.kext succeeded