KD extension DLLs & KDCOM protocol

WinDbg debugger allows you to debug all modern versions of Windows using a
built-in kernel debugger and either COM or IEEE1394 port. Let's see how is it
implemented. To start windows in Kernel Debugging mode, you specify additional
parameters in boot.ini
file that look like this:

Let's now see what actually happens when NTOSKRNL detects that it was started
with /DEBUG
parameter. First of all, it analyzes the /DEBUGPORT
parameter from boot.ini
and determines what packet-level plugin (KD
extension DLL
in Microsoft terminology) to load. For COM-based debugging the
plugin DLL is called KDCOM.DLL, for IEEE1394-based debugging it is called
KD1394.DLL. Fortunately, when you specify something like /DEBUGPORT=FOO
,
NTOSKRNL will try to load KDFOO.DLL and use it as a KD extension DLL. As we are
providing our own DLL making a fast interface to WinDbg, that expects a named
pipe from a virtual COM port, we need to solve two problems here:

• The problem of creating a valid KD extension DLL, i.e. providing the
  same set of exported functions working in an expected way.
• The problem of understanding and reimplementing KDCOM protocol, i.e.
  providing the same data at the end of our fast pipe, as the original
  KDCOM.DLL provides.

Note that in Windows Vista the kernel debugging flags are specified using
bcdedit.exe
utility and cannot specify a non-standard KD extension DLL. The
only way to load KDVMWare DLL to kernel is to replace a standard one, for
example, KD1394.DLL.

KD extension DLLs

Let's explore the structure of a typical KD extension DLL. A quick
analysis of KDCOM.DLL shows that it exports the following functions:

Let's analyze how this functions work.

Initialization

As it is evident from their names, two functions are used to initialize a KD
extension DLL: KdDebuggerInitialize0()
and
KdDebuggerInitialize1()
. Fortunately, Microsoft
provides PDB file for WinXP version of KDCOM.DLL. Additionally, some of the
functions are described by Ken Johnson (http://www.nynaeve.net/?p=169
). Let's use
the PDB to recover
declarations for the initialization functions:

The first function performs initial initialization of a KD extension DLL. For
example, it can read the parameters specified in BOOT.INI using the


LOADER_PARAMETER_BLOCK
::LoadOptions
field. For example, KDCOM.DLL can
get determine the COM port number to use and its baud rate. Both initialization
functions return a NTSTATUS
value with
STATUS_SUCCESS
corresponding to successful
completion. Note that if an initialization function such as
KdDebuggerInitialize0()
returns an unsuccessful
status, kernel is started without debugging support and the DLL is not actually
used any more.

Sending and receiving packets

All communication between kernel and a kernel debugger is packet-based. The
following rules describe the packet behavior:

• All KD extension DLL calls are called synchronously, i.e. should not
  leave any executing code after they return.
• A KD extension DLL guarantees successful packet delivery, i.e. retries
  sending a packet when KdSendPacket()
  was
  called until debugger acknowledges it.
• Kernel uses polling model to check whether new packets are available.
  For that purpose it calls KdReceivePacket()

  with a special parameter.

• When kernel wants to receive a packet, it knows the type of the packet
  to receive. It passes this type to KdReceivePacket()

  that drops all packets of other types (and sends corresponding resend
  requests).

• Each send/receive operation uses 2 buffers for data transfer. Typically,
  the first buffer contains some fixed-size message header and the second one
  contains variable-sized message body. However, this is a typical use case,
  not the only one. Basically, in a send operation the two buffers are simply
  sent one after another as a single data block with no additional indication
  where the first one ends and the second one starts. When a receive function
  is called with first buffer having a size of N bytes, the first N bytes of
  the message are put to it, while the rest is put to the second buffer.

Each buffer is represented by a STRING

structure defined in NTDEF.H
(on x64 systems the data
pointer is aligned at 8-byte boundary):

In KDVMWare this structure is redefined as KD_BUFFER
.

A special structure called KD_CONTEXT

maintains the global state for KD packet layer:

The RetryCount
member is set before a call to
KdSendPacket()
and specifies the number of
retries for a droppable packet to set. A droppable packet is a packet that can
be simply dropped if no acknowledgment comes from WinDbg after some number of
retries (KdSendPacket()
will just return). The
BreakInRequested
is set to
TRUE
by KdReceivePacket()
if WinDbg has requested a kernel breakpoint (ctrl+break was
pressed, or WinDbg was just started). The break-in request is not a part of a
packet and is transferred separately (see KDCOM protocol description below).

Here are the definitions for packet sending and receiving functions:

The KdReceivePacket()
return value can be
defined as a following enumeration:

The PacketType
parameter specifies the type
of the packet being sent or being received (all packets with other types should
be ignored), however there is one exception. When
PacketType
is set to 8 in a KdReceivePacket()

call, the function checks whether there is any data available (for example,
whether the COM port buffer is non-empty), and returns immediately
either KD_RECV_CODE_OK
or
KD_RECV_CODE_TIMEOUT
.

Additional support functions

A KD extension DLL exports some additional functions that are not directly
involved in packet sending/receiving and can simply return
STATUS_SUCCESS
in most of implementations:

The information about these functions was taken from
RectOS documentation
pages
. Although Microsoft implementation can be different from ReactOS one,
just returning STATUS_SUCCESS
from these
functions should work.

KDCOM protocol

Another problem to be solved in order to connect kernel and WinDbg using a
custom KD extension DLL is the protocol that KDCOM.DLL uses to transfer packets
over a COM port. As WinDbg receives and sends KDCOM packets when connected to a
kernel using a named pipe, our tool should be able to produce and to parse such
packets. In KDVMWare these packets  are processed in KDCLIENT.DLL on host
side, however, in Microsoft implementation, all packet processing logic is
implemented inside KDCOM.DLL. Let's see, how it works.

First of all, there are two kinds of packets: control packets and data
packets. Data packets directly transfer KdSendPacket()/KdReceivePacket()

data, while control packets signalize receive acknowledgment, retry requests,
resync requests, etc. Each control packet consists of a packet header, a data
block, and a terminating byte (0xAA
). A data
block contains contents of two buffers, one after another, with no indication of
where one ends and another starts. Moreover, sender and receiver can use split
the packet data in different ways:

Let's define a C structure describing the packet header:

Packet signature
is either 0x30303030
('0000'

)
for data packets, or 0x69696969
for control
packets ('iiii'

).
Packet type
specifies the exact type of the packet. Types for control and data
packets are members of the same enumeration:

As it was described before, packet type 8 is not used as a packet type.
Instead, when KdReceivePacket()
is called with
that value, it checks whether any data can be received from WinDbg and returns
immediately.

Packet ID
is used to detect if a single packet was missed, as the least
significant bit of a packet ID toggles with every new packet sent. The initial
packet ID is 0x80800800
, however, resync command
sets it to 0x80800000
. Checksum
is just
an arithmetic sum of all bytes from the data section of the packet.

As I have discovered after development of KDVMWare, a file named
windbgkd.h
was included in Windows 2000 DDK and contained information about
KDCOM protocol internals. The ReactOS version containing most of the information
from it can be found

here
. According to that file, the following packet types are actually
used:

Let's discuss the types of control packets and their roles in KDCOM protocol:

• Acknowledgment packets are sent by both Kernel and WinDbg when a data
  packet was successfully received.
• Resend packet is sent when Kernel or WinDbg has received a damaged
  packet, a packet with wrong ID, or a packet with unexpected type.
• Resync packet is sent by WinDbg when it is initially connected to
  kernel. The kernel acknowledges resync operation by sending back another
  resync packet.

To illustrate, how KDCOM packet layer works, let's check out some examples:

1. Normal operation. Kernel continiously checks for new packets. When a
   packet is found, kernel receives it (assuming it knows the type for the
   packet).



2. Normal packet sending. Kernel sends a packet to WinDbg.
   KdSendPacket()
   waits for acknowledgment
   packet from WinDbg.



3. Packet sending with retry. Kernel sends a packet to WinDbg, however the
   latter does not receive it. KDCOM then resends the packet after timeout.



4. Packet collision. Kernel sends a packet to WinDbg when WinDbg sends a
   packet to kernel. Kernel sends a resend request to WinDbg. The latter gets
   the data packet instead of acknowledgment and buffers it. Then acknowledges
   it, receives a resend request for the first packet and sends it once again.
   KDCOM receives acknowledgement and returns control to kernel, that calls
   KdReceivePacket()
   to get packet from WinDbg
   (if expects one).



5. Resynchronization. Kernel receives a packet
   and encounters a resync request from WinDbg.

Droppable packets

Some packets are "droppable". It means that
KdSendPacket()
may return control when such a packet was not acknowledged
by WinDbg after some number of retries. KDCOM.DLL treats the following packets
as droppable:

• Type 3, subtype 0x3230
• Type 7, subtype 0x3031
• Type 11, subtype 0x3430

Packet subtype (ApiNumber) is the first DWORD in the packet data block. The
types referenced here is defined in the following way in windbgkd.h
:

Resync bounce problem

There is one significant detail in original KDCOM implementation. When the
KDCOM.DLL initializes, it reinitializes the COM port and resets its buffer. A
named pipe implementation should do the same. In other case, the following
scenario is possible:

1. WinDbg connects to a named pipe with no kernel listening and sends a few
   resync packets.
2. Kernel loads our KD DLL, it receives first resync packet and replies
   with a resync.
3. WinDbg resynchronizes and sends some data.
4. Kernel receives another resync from buffer and resyncs again, replying
   with a resync.
5. WinDbg receives an unexpected resync, resynchronizes and sends another
   resync packet to acknowledge resynchronization.
6. The WinDbg/kernel couple will continue producing resync packets till the
   end of time and will never synchronize normally.

To avoid this problem, KDVMWare simply clears the named pipe receive buffer
when it receives a resync packet.

Implementation in KDVMWare

All KDCOM-related functionality is implemented in the
KdComDispatcher

class. Feel free to explore its documentation using the link above.