Capture Thread Info when Application is slow

When Windows Client or Server Agent is slow, use WinDbg to catch thread info:

• Download and install WinDbg (https://go.microsoft.com/fwlink/p/?LinkId=536682); during the installation, select the 'Debugging Tools for Windows' checkbox only. You can also download an old build from http://wcbuild.gladinet.com/releases/windbg/X86-Debuggers-And-Tools-x86_en-us.msi.
• After WinDbg is installed, copy SOS.dll
  • The default x86 windbg is installed under "C:\Program Files (x86)\Windows Kits\10\Debuggers\x86"
  • The x86 SOS.dll is under C:\Windows\Microsoft.NET\Framework\v4.0.30319\SOS.dll
  • Copy SOS.dll from the Framework folder to the WinDbg installation folder. You can copy&paste in Windows Explorer directly. Or open command prompt as Administrator and run
    copy c:\Windows\Microsoft.NET\Framework\v4.0.30319\SOS.dll C:\Program Files (x86)\Windows Kits\10\Debuggers\x86
• Start WinDbg:
  • for Windows Client, start WinDbg as regular user
  • for Server Agent, start WinDbg as Administrator

• Contact Support (ticket@gladinet.com) with the Desktop Client version to send you the Symbol files corresponding to the Client installed.
• Create a local folder C:\symbols, to store the symbol files downloaded from Microsoft (symbols from the Client can also be saved on this folder).
• New Windows Client uses ClientShell.exe. And new Server Agent uses GladServerAgentService.exe. Both are built from Centrestack server. For the symbols files, need windows client/server agent pdb and Centrestack server pdb.
  • Windows Client/Server Agent symbol package has name like pdbxxxx.zip. Here, xxxx is the  version. For example for version 12.3.3162.51136, the pdb file is pdb3162.zip. Unzip it. You will get the folder pdb3162\pdb, which contains the .pdb files.
  • CentreStack Server symbol file has the format of gcpdbxxxx.xxxxx. For example, for CentreStack 12.3.9434.51139, the symbol file name is gcpdb9434.51139.zip. Unzip it to get folder like gcpdb9434.51139\pdb9434. Inside will find the folder ClientShell and ServerAgentService, which contains pdb files.
  • For Windows Client, the symbol path is srv*C:\symbols*https://msdl.microsoft.com/download/symbols;C:\gcpdb9434.51139\pdb9434\ClientShell;C:\symbols\pdb3162\pdb
  • For Server Agent, the symbol path is srv*C:\symbols*https://msdl.microsoft.com/download/symbols;C:\gcpdb9434.51139\pdb9434\ServerAgentService;C:\symbols\pdb3162\pdb
  • From the WindDbg go to 'File' -> 'Symbol File Path', set symbol path 

• Windows Client:

In WinDbg, go to File, Open Executable. Go to the Windows Client installation folder and select 'ClientShell.exe'. Click 'Open', to launch the Windows Client in WinDbg.

When start Windows Client like this, do NOT start WinDbg as administrator first.

• Server Agent:

Go to Windows Services, find Cloud Server Agent Access Service, right-click and go to Properties, open the Log On tab, here enable 'Allow service to interact with desktop' (disable this after collecting the traces). Press OK.

In WinDbg, press F6, select the process 'GladServerAgentService.exe' and click 'OK'. It will attach WinDbg to the process.

• On WinDbg command input line, run 'x kernel32!TerminateP*'. It returns the exact name of the TerminateProcess function. For example: 

0:090> x kernel32!TerminateP*
776289b0 KERNEL32!TerminateProcessStub (<no parameter info>)

Here, the function name is 'KERNEL32!TerminateProcessStub'

Run 'bp kernel32!TerminateProcessStub', to set a breakpoint on the call 'kernel32!TerminateProcessStub'. This way, we can get all the thread information, instead of only one thread.

In some machines, the function returned via x is 'KERNEL32!TerminateProcess'. In that case, set the breakpoint for the function via 'bp KERNEL32!TerminateProcess'

• Type 'g' to continue. The command line field will show '*BUSY* Debuggee is running…'.

• Run the operation which causes the application to be slow.
• When the system is slow, go to Windbg and select 'Debug' -> 'Break', to break in.

• If you are running 64-bit Windbg, run '!wow64exts.sw' first
• Type 'kb' then '~*kb' to get thread info

• Type '.load SOS.dll', then '~*e!ClrStack'
• Type '!dumpheap -stat', then '!analyze -v'
• Type '.dump /ma c:\temp\crash.dmp', to create dump. The dump file will be created under c:\temp. If the folder doesn't exist, please change the path. Zip the dump file and keep it, in case we need to retrieve more info from the dump.
• Finally, save the output in WinDbg (Edit -> Write Window Text to File)
• Click 'Debug' -> 'Detach Debuggee', to detach WinDbg.

• Send the saved output and the dump file (if requested) to Gladinet support.