A Meeting with Dr. Watson(Debugging Dead Locks)

Dr. Watson is an application debugger included with the Microsoft Windows operating system. It may be named drwatson.exe, drwtsn32.exe or dwwin.exe, depending on the version of Windows.Dr. Watson for Windows is a program error debugger that gathers information about your computer when an error (or user-mode fault) occurs with a program.Dr. Watson creates a text file that is Drwtsn32.log.

By default, the log file created by Dr. Watson is named Drwtsn32.log and is saved in the following location:
drive:\Documents and Settings\All Users.WINNT\Application Data\Microsoft\Dr Watson
Virtual Memory
On Windows NT and above the memory is divided into two parts, the user mode and the kernel mode. What differentiates user mode from kernel mode is the privilege level. The primary memory restriction placed on user-mode programs is that they cannot access any of the kernel-mode memory.

Although a user-mode program can try to directly communicate with a hardware device, the system prevents it from doing so. You probably have seen the result of such an attempt, the point where Dr. Watson pops up.
A kernel-mode component must determine whether an exception is the result of a legal or an illegal operation; when a kernel-mode component catches an illegal exception, it notifies the Dr. Watson user-mode application.

kernel-mode device drivers and subsystems can almost do anything they want. This lack of protection also emphasize the need to take care when loading a third-party device driver, because once in kernel mode the software has complete access to all OS data.
To configure Dr. Watson, follow these steps:
Click Start, and then click Run.
Type drwtsn32, and then click OK.
Types of Memory Dump Files

When Stop errors occur, the system automatically dumps the contents of its RAM to the paging file, and then writes the pagefile contents to the systemdrive root by default. Analyzing the dump file can help provide more information about the root cause of a problem.

Small memory dump
Small memory dump files contain the least information, but consume the least disk space, 64 kilobytes (KB). Small memory dump files are sometimes referred to as "mini" dump files.

Kernel memory dump
This is an intermediate size dump file that records only kernel-level memory and can occupy several megabytes (MB) of disk space. When a Stop error occurs, Windows saves a kernel memory dump file to a file named systemroot\Memory.dmp and create a small memory dump file in the systemroot\Minidump folder

Complete memory dump
A complete memory dump file contains the entire contents of physical memory when the Stop error occurred. The file size is equal to the amount of physical memory installed plus 1 MB.
When a Stop error occurs, the operating system saves a complete memory dump file to a file
named systemroot\Memory.dmp and creates a small memory dump file in the systemroot\Minidump folder.

How to generate a memory dump file using Dr.Watson:

Register Dr.Watson as the default debugger in the operating system:

Go to Start > Run > type drwtsn32 -i and press Enter.

Check that the program has been successfully registered as the default debugger and click OK.

Configure Dr.Watson:

Go to Start > Run > type drwtsn32 and press Enter to start Dr.Watson.

Select a folder for saving log file in the Log File Path field, e.g. ะก:\drwtsn.

Select the same folder in the Crash Dump field.

Select Full in the Crash Dump Type section.

Click OK.


Check that the full dump file has been successfully generated.


DeadLock Situation

A deadlock is a situation in which two computer programs sharing the same resource are effectively preventing each other from accessing the resource, resulting in both programs ceasing to function.

Program 1 requests resource A and receives it.
Program 2 requests resource B and receives it.
Program 1 requests resource B and is queued up, pending the release of B.
Program 2 requests resource A and is queued up, pending the release of A.

Now neither program can proceed until the other program releases a resource. The operating system cannot know what action to take. At this point the only alternative is to abort (stop) one of the programs.

Debugging Dead Lock Situation with Windump:

When you reach a deadlock, the PC appears to hang. With a userdump you can get the information to resolve this problem.

WinDbg !locks command will examine process critical section list and display all locked critical sections, lock count and thread id of current critical section owner.

0:000> !locks
CritSec NTDLL!LoaderLock+0 at 784B0348
LockCount          4
RecursionCount     1
OwningThread       624
EntryCount         6c3
ContentionCount    6c3
*** Locked

CritSec LOCALSPL!SpoolerSection+0 at 76AB8070
LockCount          3
RecursionCount     1
OwningThread       1c48
EntryCount         646
ContentionCount    646
*** Locked

If we look at threads #624 and #1c48 we could see them mutually waiting for each other:
If we look at threads #624 and #1c48 we could see them mutually waiting for each other:

TID#624 owns CritSec 784B0348 and is waiting for CritSec 76AB8070

TID#1c48 owns CritSec 76AB8070 and is waiting for CritSec 784B0348


0:000>~*kv

. 12 Id: bc0.624 Suspend: 1 Teb: 7ffd3000 Unfrozen
0000024c 00000000 00000000 NTDLL!ZwWaitForSingleObject+0xb
76ab8000 76a815ef 76ab8070 NTDLL!RtlpWaitForCriticalSection+0×9e
76ab8070 76a844f8 00cd1f38 NTDLL!RtlEnterCriticalSection+0×46
00cd1f38 76a8a1d7 00000000 LOCALSPL!EnterSplSem+0xb
00000000 00000000 00cd1f38 LOCALSPL!FindSpoolerByNameIncRef+0×1f
00000000 777f19bc 00000001 LOCALSPL!LocalGetPrinterDriverDirectory+0xe
00000000 777f19bc 00000001 spoolss!GetPrinterDriverDirectoryW+0×59
00000000 777f19bc 00000001 spoolsv!YGetPrinterDriverDirectory+0×27
00000000 777f19bc 00000001 WINSPOOL!GetPrinterDriverDirectoryW+0×7b
50000000 00000001 00000000 BRHLUI04+0×14ea
50002ea0 50000000 00000001 BRHLUI04!DllGetClassObject+0×1705
00000000 00000000 000cb570 NTDLL!LdrpRunInitializeRoutines+0×1df
000cc8f8 0288ea30 0288ea38 NTDLL!LdrpLoadDll+0×2e6
000cc8f8 0288ea30 0288ea38 NTDLL!LdrLoadDll+0×17)
000c1258 00000000 00000008 KERNEL32!LoadLibraryExW+0×231
000c150c 0288efd8 00000000 UNIDRVUI!PLoadCommonInfo+0×17e
000c150c 0288efd8 00000007 UNIDRVUI!DwDeviceCapabilities+0×1a
00070000 00071378 00000045 UNIDRVUI!DrvDeviceCapabilities+0×19

. 13 Id: bc0.1c48 Suspend: 1 Teb: 7ffd2000 Unfrozen
0000010c 00000000 00000000 NTDLL!ZwWaitForSingleObject+0xb
784b0301 78468d38 784b0348 NTDLL!RtlpWaitForCriticalSection+0×9e
784b0348 74fb4344 00000000 NTDLL!RtlEnterCriticalSection+0×46
74fb0000 02c0f2a8 00000000 NTDLL!LdrpGetProcedureAddress+0×122
74fb0000 02c0f2a8 00000000 NTDLL!LdrGetProcedureAddress+0×17
74fb0000 74fb4344 02c0f449 KERNEL32!GetProcAddress+0×41
017924b0 00000000 00000001 ws2_32!CheckForHookersOrChainers+0×1f
00000101 02c0f344 017924b0 ws2_32!WSAStartup+0×10f
00cdf79c 02c0f4f4 76a8c9bc LOCALSPL!GetDNSMachineName+0×1e
00000000 76a8c9bc 780276a2 LOCALSPL!GetPrinterUrl+0×2c
0176f570 ffffffff 01000000 LOCALSPL!UpdateDsSpoolerKey+0×322
0176f570 76a8c9bc 01792b90 LOCALSPL!RecreateDsKey+0×50
00000000 00000002 01792b90 LOCALSPL!SplAddPrinter+0×521
01791faa 0176a684 76a5cd34 WIN32SPL!InternalAddPrinterConnection+0×1b4
01791faa 02c0fa00 02c0fabc WIN32SPL!AddPrinterConnectionW+0×15
00076f1c 02c0fabc 01006873 spoolss!AddPrinterConnectionW+0×49
00076f1c 00000001 77107fb0 spoolsv!YAddPrinterConnection+0×17
00076f1c 02020202 00000001 spoolsv!RpcAddPrinterConnection+0xb
01006868 02c0fac0 00000001 rpcrt4!Invoke+0×30
00000000 00000000 000d22c8 rpcrt4!NdrStubCall2+0×655
000d22c8 00076fe0 000d22c8 rpcrt4!NdrServerCall2+0×17
010045fc 000d22c8 02c0fe0c rpcrt4!DispatchToStubInC+0×32
0000002b 00000000 02c0fe0c rpcrt4!RPC_INTERFACE::DispatchToStubWorker+0×100
000d22c8 00000000 02c0fe0c rpcrt4!RPC_INTERFACE::DispatchToStub+0×5e
000d3210 00076608 813b0013 rpcrt4!LRPC_SCALL::DealWithRequestMessage+0×1dd
000d21d0 02c0fe50 000d3210 rpcrt4!LRPC_ADDRESS::DealWithLRPCRequest+0×10c
770c9ad0 00076608 770cb6d8 rpcrt4!LRPC_ADDRESS::ReceiveLotsaCalls+0×229
00076608 770cb6d8 0288f9a8 rpcrt4!RecvLotsaCallsWrapper+0×9
00074a50 02c0ffec 77e7438b rpcrt4!BaseCachedThreadRoutine+0×11f
00076e68 770cb6d8 0288f9a8 rpcrt4!ThreadStartRoutine+0×18
770d1c54 00076e68 00000000 KERNEL32!BaseThreadStart+0×52

The command kv. reveals the function callback stack for the thread that was active at the time of the trip. This stack is read from bottom to top: the topmost function is the last function to be called and the bottommost function is the first function to be called for this thread.

Once you locate the thread, you see it has a call to the WaitForCriticalSection function, which means that not only does it have a lock, but it is also waiting for an object that is locked by something else. You can find out what is locking the object by looking at the first parameter of the WaitForCriticalSection call.

References:
http://www.cs.rpi.edu/academics/courses/fall04/os/c10/
http://en.wikipedia.org/wiki/Deadlock
http://windbg.info/download/doc/pdf/WinDbg_cmds.pdf
Location: Bengaluru, Karnataka, India

Comments

New comments are not allowed.