How Windows EDR Collects Telemetry with KAPCs (+ How Mimikatz Detects it)

Recently I had to the joy of reading “Evading EDR: The Definitive Guide to Defeating Endpoint Detection Systems.” by Matt Hand while on vacation. First off, this is an absolute must read for anybody who works with, or against ;), EDR/XDR technology. Now that I’m back home in the lab, I’m excited to share my brain dump of how Windows-based EDR software collects process-based telemetry and how malware can take a peek into these operations with some examples from open-source repos.

While monitoring for syscalls to NtCreateUserProcess works, setup and maintenance for this type of monitoring can be complicated and tedious. Thankfully, Windows provides EDR developers a much more efficient method of gathering this information via Kernel Asynchronous Procedure Calls (KAPCs). Implemented by a kernel-level Windows driver, these asynchronous (callback) routines are a method of creating a subscription event to information a driver deems interesting. While not every aspect of system activity has a callback system available (Ex:// File I/O), this is a built-in and efficient solution for process telemetry. Let’s take a look at how OpenEDR, a popular open-source EDR platform, handles this operation.

In the main driver file for OpenEDR, edrdrv.cpp, we see the following block of code residing in the DriverEntry() routine.

IFERR_RET(cfg::initialize(), "Can't initialize config\r\n");
IFERR_RET(devdisp::initialize(), "Can't initialize devices disatcher\r\n");
IFERR_RET(procmon::initialize(), "Can't initialize process monitor\r\n");
    // Base Process Telemetry
IFERR_RET(filemon::initialize(), "Can't initialize file filter\r\n");
IFERR_RET(regmon::initialize(), "Can't initialize registry filter\r\n");
IFERR_RET(objmon::initialize(), "Can't initialize process filter\r\n");
    // Pre-op/Post-op Process Filtering Telemetry
IFERR_RET(drvioctl::initialize(), "Can't initialize IOCTL device\r\n");
IFERR_RET(netmon::initialize(), "Can't initialize network monitor\r\n");
IFERR_RET(dllinj::initialize(), "Can't initialize DLL injector\r\n");

This code initializes all of OpenEDR’s KAPC subscriptions. Specifically, we’re interested in, obviously, procmon::initialize() and, not so obviously, objmon::initialize().

To take a deeper dive into the procmon::initialize() logic, we’ll move to the procmon.cpp source file.

NTSTATUS initialize()
{
    LOGINFO2("Enable Process context.\r\n");

    if (g_pCommonData->fProcCtxNotificationIsStarted) 
        return LOGERROR(STATUS_ALREADY_REGISTERED);

    IFERR_RET(PsSetCreateProcessNotifyRoutineEx(&notifyOnCreateProcess, FALSE));

    g_pCommonData->fProcCtxNotificationIsStarted = TRUE;

If you’re like me and don’t happen to be a Windows kernel developer, you’ve probably never seen the PsSetCreateProcessNotifyRoutineEx routine. Thankfully, Microsoft publishes the documentation for this function- let’s take a look.

/* The PsSetCreateProcessNotifyRoutineEx routine registers or removes a callback routine that notifies the caller when a process is created or exits. */*

NTSTATUS PsSetCreateProcessNotifyRoutineEx(
    [in] PCREATE_PROCESS_NOTIFY_ROUTINE_EX NotifyRoutine,
    [in] BOOLEAN                        Remove
);

Based on this documentation, we can see that procmon::initialize() adds the notifyOnCreateProcess() routine to the list of callback routines for process creation/deletion telemetry. OpenEDR now has efficient visibility into process telemetry!

While visibility is certainly ninety percent of the battle here, OpenEDR isn’t just Endpoint Detection software. We’re missing the ‘R’ in EDR! While the procmon module gave us visibility, the objmon module enables us to filter process-related activity (specifically, process handle acquisition in this example). Let’s open up objmon.cpp!

NTSTATUS initialize()
{
    if (g_pCommonData->fObjMonStarted)
        return STATUS_SUCCESS;

    g_pCommonData->hProcFltCallbackRegistration = nullptr;

    // Set hooks
    {
        OB_OPERATION_REGISTRATION stObOpReg[2] = {};
        OB_CALLBACK_REGISTRATION stObCbReg = {};

        USHORT OperationRegistrationCount = 0;

        // Processes callbacks
        stObOpReg[OperationRegistrationCount].ObjectType = PsProcessType;
        stObOpReg[OperationRegistrationCount].Operations = OB_OPERATION_HANDLE_CREATE | OB_OPERATION_HANDLE_DUPLICATE;
        stObOpReg[OperationRegistrationCount].PreOperation = preProcessObjectAccess;
        stObOpReg[OperationRegistrationCount].PostOperation = postProcessObjectAccess;
        OperationRegistrationCount += 1;

        stObCbReg.Version = OB_FLT_REGISTRATION_VERSION;
        stObCbReg.OperationRegistrationCount = OperationRegistrationCount;
        stObCbReg.OperationRegistration = stObOpReg;
        RtlInitUnicodeString(&stObCbReg.Altitude, edrdrv::c_sAltitudeValue);

        IFERR_RET(ObRegisterCallbacks(&stObCbReg, &g_pCommonData->hProcFltCallbackRegistration));
    }

    g_pCommonData->fObjMonStarted = TRUE;

    return STATUS_SUCCESS;
}

This KAPC setup is a little more complex than our singular call to PsSetCreateProcessNotifyRoutineEx(). Without digging into the million different WinAPI structures here, the TL;DR is:

1). Create a callback for OB_OPERATION_HANDLE_CREATE (handle creation) and OB_OPERATION_HANDLE_DUPLICATE (handle duplication) events
2). Call the preProcessObjectAccess Pre-op routine and the postProcessObjectAccess Post-op routine appropriately

Adequately named, preProcessObjectAccess is called prior to the process telemetry event happening (Ex:// before access to a handle is granted). Since this routine is being called prior to the event finalizing, this allows OpenEDR to implement what it calls, quite humorously, “selfdefense.” Let’s turn our attention to the preProcessObjectAccess routine.

OB_PREOP_CALLBACK_STATUS preProcessObjectAccess(PVOID /*RegistrationContext*/, 
    POB_PRE_OPERATION_INFORMATION preOpInfo)
{
...
    ACCESS_MASK eOrigDesiredAccess = *pDesiredAccess;
    ACCESS_MASK eExpandedDesiredAccess = expandProcessDesiredAccess(eOrigDesiredAccess);
...
    // Check selfdefense
    do 
    {
        static constexpr ACCESS_MASK c_nDeniedAccessMask = PROCESS_TERMINATE | PROCESS_CREATE_THREAD | PROCESS_SET_SESSIONID | PROCESS_VM_OPERATION | PROCESS_VM_READ | PROCESS_VM_WRITE | PROCESS_DUP_HANDLE |     PROCESS_CREATE_PROCESS | PROCESS_SET_QUOTA | PROCESS_SET_INFORMATION | PROCESS_QUERY_INFORMATION | PROCESS_SUSPEND_RESUME | /*PROCESS_QUERY_LIMITED_INFORMATION |*/ PROCESS_SET_LIMITED_INFORMATION;

        // If has parent-child relations, allow all
        if(fHasParentChildRelation)
            break;
        // If Target is not protected, allow all
        if (!pTargetCtx || !pTargetCtx->fIsProtected)
            break;

        // If Initiator is trusted, allow all
        if(procmon::isProcessTrusted(pInitiatorCtx))
            break;

        // Skip access from csrss
        if(testFlag(pInitiatorCtx->processInfo.nFlags, (UINT32)ProcessInfoFlags::CsrssProcess))
            break;

        // If no denied access
        if((eExpandedDesiredAccess & c_nDeniedAccessMask) == 0)
            break;

        // partial restrict access
        *pDesiredAccess = eExpandedDesiredAccess & ~c_nDeniedAccessMask;
...

Side Note: If you’ve never worked with access masks in Windows before, this may look confusing. Here’s a quick link to a MS documentation page before continuing ([link](Access Rights and Access Masks – Win32 apps | Microsoft Learn)).

OpenEDR looks into the Object-specific access rights process handle requests! OpenEDR first evaluates if a process has any exceptions (Ex:// if a target is not marked as protected) then actively defends processes that don’t meet those exclusions using access masks and bitwise operations. In a real-world scenario, this code could directly prevent [T1003.001, OS Credential Dumping: LSASS Memory](OS Credential Dumping: LSASS Memory, Sub-technique T1003.001 – Enterprise | MITRE ATT&CK®) exploits by denying privileged handles to LSASS.exe.

If you’re a blue-teamer reading this and thinking “Wow, this is very valuable information,” imagine how valuable this knowledge could be to a red-teamer or real-life adversary. It should come as no surprise that malware absolutely attempts to actively detect and avoid EDR technology. Let’s take a look at how a fan-favorite open-source malware variant attempts to detect KAPC-based detection.

Mimikatz comes bundled with its mimidrv.sys counterpart. While less documented than the standard Mimikatz executable, mimidrv.sys is far from less dangerous. mimidrv.sys enables Mimikatz to gain access to the same section of kernel memory that holds our precious KAPC subscriptions. After initializing mimidrv.sys with the !+ command in the Mimikatz command prompt, we can call several !notif routines to crawl through kernel memory searching for byte patterns matching the signature of KAPC subscriptions. Specifically, we’ll be analyzing the !notifProcess module. Let’s take a deeper look into the source code to see how this works!

kkll_m_notify_search(), detailed in kkll_m_notify.c, is the first function we’ll want to look into to discover the functionality of !notifProcess().

NTSTATUS kkll_m_notify_search(PKKLL_M_MEMORY_GENERIC generics, SIZE_T cbGenerics, PUCHAR * ptr, PULONG pRoutineMax, PKKLL_M_MEMORY_OFFSETS * pOffsets)
{
    NTSTATUS status = STATUS_NOT_FOUND;
    PKKLL_M_MEMORY_GENERIC pGeneric;
    UNICODE_STRING stringStart, stringEnd;
    PUCHAR start, end;

    if(pGeneric = kkll_m_memory_getGenericFromBuild(generics, cbGenerics))
    {
        RtlInitUnicodeString(&stringStart, pGeneric->start);
        RtlInitUnicodeString(&stringEnd, pGeneric->end);
        start = (PUCHAR) MmGetSystemRoutineAddress(&stringStart);
        end = (PUCHAR) MmGetSystemRoutineAddress(&stringEnd);

        if(start && end)
        {
            status = kkll_m_memory_genericPointerSearch(ptr, start, end, pGeneric->Search.Pattern, pGeneric->Search.Length, pGeneric->Offsets.off0);
            if(NT_SUCCESS(status))
            {
                if(pRoutineMax)
                    *pRoutineMax = pGeneric->Offsets.off1;
                if(pOffsets)
                    *pOffsets = &pGeneric->Offsets;
            }
        }
    }
    return status;
}

pGeneric holds the following details (assuming you’re running this on a Windows 10, 19041 build) after calling kkll_m_memory_getGenericFromBuild():

KIWI_OS_INDEX OsIndex; 
    // KiwiOsIndex_10_2004
KKLL_M_MEMORY_PATTERN Search; 
    // {sizeof(PTRN_W10_1703_Process), PTRN_W10_1703_Process},
    // {0x33, 0xff, 0x6a, 0x00, 0x8b, 0xd0, 0x8b, 0xcb, 0xe8}
PWCHAR start; 
    // MmGetSystemRoutineAddress(L"PsSetCreateThreadNotifyRoutine")
PWCHAR end;
    // MmGetSystemRoutineAddress(L"IoCreateSymbolicLink")
KKLL_M_MEMORY_OFFSETS Offsets 
    // { -4, 64}

This information comes from the “generics database” within the ProcessReferences[] array. This “generic” information is hardcoded by Delpy (the author of Mimikatz), not dynamically fetched. If you’re unfamiliar, Windows build-specific memory searching is a very tedious process that the book The Art of Memory Forensics: Detecting Malware and Threats in Windows, Linux, and Mac Memory really drives home.

After receiving the generics information (A.K.A what we’re looking for and where to find it), kkll_m_memory_genericPointerSearch() is basically just a wrapper for the kkll_m_memory_search() function inside of kkll_m_memory.c. Here’s a quick glance at the actual “working” code.

NTSTATUS kkll_m_memory_search(const PUCHAR adresseBase, const PUCHAR adresseMaxMin, const UCHAR *pattern, PUCHAR *addressePattern, SIZE_T longueur)
{
    for(*addressePattern = adresseBase; (adresseMaxMin > adresseBase) ? (*addressePattern <= adresseMaxMin) : (*addressePattern >= adresseMaxMin); *addressePattern += (adresseMaxMin > adresseBase) ? 1 : -1)
        if(RtlEqualMemory(pattern, *addressePattern, longueur))
            return STATUS_SUCCESS;
    *addressePattern = NULL;
    return STATUS_NOT_FOUND;
}

This is one of many methods malware can use to detect the presence of EDR technology on a protected asset. From this point, red teamers or adversaries can pursue easier targets without EDR software, research bypasses for a specific EDR, or continue with the knowledge that their actions are being monitored.

EDR technology is complex, like many concepts in security, but it’s important to remember that nothing is magic. With the help of proper documentation, open-source code, and a little bit of time spent sorting through it all, any concept is graspable!