Skip to content
Peace Lee edited this page Aug 19, 2017 · 81 revisions

Welcome to guider wiki!

  • Contents
1. What is guider?
2. How to use
3. Options
4. Mode
- Thread mode
	Interval Info
	Graph Info
	Syscall Info
	Module Info
	Custom Event Info
	Preemption Info
	Signal Info
	User Event Info
	Kernel Event Info
	Dependency Info
- Function Mode
	CPU Info
	Memory Info
	Heap Info
	Block Info
	Kernel Event Info
	Function Graph Info
- Top Mode
	File Info
	Stack Info
	Memory Info
	Summary Info
	Report Info
	Image Info
- File Mode
	Process Info
	File Info
- System Mode
	To do
  • What is guider?
Do you struggle to improve system performance or to find root cause that makes system abnormal?   
Guider is made to measure amount of system resource usage and to trace system behavior.   
You can analyze your performance issues effectively with this tool.   

Guider pursues three characteristics as bellow.
1. easy to use: just run without installation or setting
2. measure correctly: time in ms, size in MB
3. integrate features: show as much information as possible

Document files are available.
- [Download](https://github.com/iipeace/guider/files/1149974/guider_20170715.pdf)
- [Open](http://events.linuxfoundation.org/sites/events/files/slides/guider_20170715.pdf)
  • How to use
Input command as bellow to start accurate profiling in thread mode
# guider.py record 

Input command as bellow to start realtime profiling in top mode
$ guider.py top 

Input "Ctrl + c" key to finish profiling 

Input command as bellow to see more examples
$ guider.py -h -a
  • Options
[record mode]
    top        [top]
    record     [thread]
    record -y  [system]
    record -f  [function]
    record -F  [file]
    view       [page]
[control mode]
    list
    start|stop|send [pid]
[record options]
    -e  [enable_optionsPerMode:bellowCharacters]
          [function] {m(em)|b(lock)|h(eap)|p(ipe)|g(raph)}
          [thread]   {m(em)|b(lock)|i(rq)|l(og)|n(et)|p(ipe)|r(eset)|g(raph)|f(utex)}
          [top]      {t(hread)|d(isk)|w(fc)|W(chan)|s(tack)|m(em)|I(mage)|g(raph)|r(eport)|f(ile)}
    -d  [disable_optionsPerMode:bellowCharacters]
          [thread]   {c(pu)}
          [function] {c(pu)|u(ser)}
          [top]      {r(ss)|v(ss)}
    -s  [save_traceData:dir/file]
    -S  [sort_output:c(pu)/m(em)/b(lock)/w(fc)/p(id)/n(ew)/r(untime)]
    -u  [run_inBackground]
    -W  [wait_forSignal]
    -R  [record_repeatedly:interval,count]
    -b  [set_bufferSize:kb]
    -D  [trace_threadDependency]
    -t  [trace_syscall:syscalls]
    -T  [set_fontPath]
    -H  [set_functionDepth]
    -j  [set_reportPath:dir]
    -U  [set_userEvent:name:func|addr:file]
    -K  [set_kernelEvent:name:func|addr{:%reg/argtype:rettype}]
    -C  [set_commandScriptPath:file]
    -w  [set_customRecordCommand:BEFORE|AFTER|STOP:file:value]
    -x  [set_addressForLocalServer:{ip:}port]
    -X  [set_requestToRemoteServer:{req@ip:port}]
    -N  [set_addressForReport:req@ip:port]
    -n  [set_addressForPrint:ip:port]
    -m  [set_objdumpPath:file]
[analysis options]
    -o  [save_outputData:dir]
    -P  [group_perProcessBasis]
    -p  [show_preemptInfo:tids]
    -l  [set_addr2linePath:files]
    -r  [set_targetRootPath:dir]
    -I  [set_inputValue:file|addr]
    -q  [configure_taskList]
    -L  [convert_textToImage]
[common options]
    -a  [show_allInfo]
    -Q  [print_allRows]
    -i  [set_interval:sec]
    -g  [set_filter:comms|tids{:file}]
    -A  [set_arch:arm|x86|x64]
    -c  [set_customEvent:event:filter]
    -E  [set_errorLogPath:file]
    -v  [verbose]
  • Mode

    1. Thread mode

      [Thread Info] [ Elapsed: 2.050 ] [ Start: 2849868.198 ] [ Running: 112 ] [ CtxSwc: 3357 ] [ LogSize: 4054 KB ] [ Unit: Sec/MB/NR ]
      ==========================================================================================================================================================
      __________Thread Info___________|_____________CPU Info______________|______SCHED Info______|________BLOCK Info________|_____________MEM Info_____________|
                                      |                                   |                      |                          |                                  |
                  Name(  Tid/  Pid)|LF|Usage(    %)|Delay(  Max)|Pri| IRQ |  Yld| Lose|Steal| Mig| Read( MB/  Cnt)|WCnt( MB)| Sum(Usr/Buf/Ker)|Rcl|Wst|DRcl(Nr)|
      ==========================================================================================================================================================
      # CPU: 12
      
                CORE/0(-----/-----)|--| 0.00(  0.1)| 0.00( 0.00)|  0| 0.00|    7|    -|    -|   -| 0.00(  0/    1)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/1(-----/-----)|--| 0.00(  0.1)| 0.10( 0.00)|  0| 0.00|  147|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/2(-----/-----)|--| 0.00(  0.1)| 0.16( 0.00)|  0| 0.00|  211|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/3(-----/-----)|--| 0.00(  0.1)| 0.11( 0.00)|  0| 0.00|  181|    -|    -|   -| 0.00(  0/    0)|  32(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/4(-----/-----)|--| 0.00(  0.1)| 0.11( 0.00)|  0| 0.00|  232|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/5(-----/-----)|--| 0.30( 14.8)| 0.18( 0.00)|  0| 0.00|  179|    -|    -|   -| 1.26(  6/  495)|  19(  0)|  61( 57/  0/  3)|  0|  0|0.00( 0)|
                CORE/6(-----/-----)|--| 0.00(  0.0)| 0.35( 0.00)|  0| 0.00|   57|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/7(-----/-----)|--| 0.00(  0.0)| 0.60( 0.00)|  0| 0.00|  100|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/8(-----/-----)|--| 0.00(  0.0)| 0.44( 0.00)|  0| 0.00|   59|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                CORE/9(-----/-----)|--| 0.00(  0.0)| 1.94( 0.00)|  0| 0.00|   37|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
               CORE/10(-----/-----)|--| 0.07(  3.4)| 0.00( 0.00)|  0| 0.00|    2|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
               CORE/11(-----/-----)|--| 0.00(  0.0)| 2.05( 0.00)|  0| 0.00|   39|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
      
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      # Hot: 4
      
              synergyc( 3604/ 3602)|  | 0.17(  8.5)| 0.00( 0.00)|  0| 0.00|    3|   14|    3|   0| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
       arm-starfish-li(16087/16087)|  | 0.13(  6.3)| 0.00( 0.00)|  0| 0.00|    0|   20|  157|   4| 1.26(  6/  496)|   0(  0)|  61( 57/  0/  3)|  0|  0|0.00( 0)|
                guider(16088/16088)|  | 0.07(  3.4)| 0.00( 0.00)|R90| 0.00|    2|    0|    2|   0| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
      
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      
      

      This is a main output in thread mode.
      User can analyze resource usage of threads accurately.

      [CPU Info] section includes bellow information.

      Usage: cpu time(ms) that a thread has been running
      Delay: total time(ms) that a thread has been being preempted
      Pri: highest priority that a thread has been running at
      IRQ: cpu time(ms) that a thread has been running in interupt context

      [SCHED Info] section includes bellow information.

      Yld: yield number
      Lose: preempted number
      Steal: preemption number
      Mig: migration number

      [BLOCK Info] section includes bellow information.

      Read: total time(ms) that a thread has been being blocked for disk read
      MB: total size(MB) that a thread has been reading from disk
      Cnt: total number that a thread has been reading from disk
      WCnt: total number that a thread has been writing to disk
      MB: total size(MB) that a thread has been writing to disk

      [MEM Info] section includes bellow information.

      Sum: total size(MB) that a thread has been allocating to memory(PAGE)
      Usr: user-space size(MB) that a thread has been allocating to memory(PAGE)
      Buf: page cache size(MB) that a thread has been allocating to memory(PAGE)
      Ker: kernel-space size(MB) that a thread has been allocating to memory(PAGE)
      Rcl: total size(MB) that a thread has been being reclaimed by other threads
      Wst: wasting size(MB) that a thread has been allocating to memory(PAGE) in kernel space
      Drcl(Nr): total time(ms) and number that a thread has been reclaiming directly

    +  Interval Info [-i option]    
        ``````````````````````````````````````````
        [Thread Interval Info] [ Unit: 1 Sec ]
        ==========================================================================================================================================================
                    Name(  Tid/  Pid):   1   2  2.05
        ==========================================================================================================================================================
        # Total(%/MB/Cnt)
    
                  CORE/0(    0/-----):   0   0   0
                  CORE/1(    0/-----):   0   0   0
                  CORE/2(    0/-----):   0   0   0
                  CORE/3(    0/-----):   0   0   0
                  CORE/4(    0/-----):   0   0   0
                  CORE/5(    0/-----):  20   8   5
                  CORE/6(    0/-----):   0   0   0
                  CORE/7(    0/-----):   0   0   0
                  CORE/8(    0/-----):   0   0   0
                  CORE/9(    0/-----):   0   0   0
                 CORE/10(    0/-----):   6   0   0
                 CORE/11(    0/-----):   0   0   0
    
                     MEM(    0/-----):  15  44   1
    
                  BLK_RD(    0/-----):   3   3   0
                  BLK_WR(    0/-----):   0   0   0
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        # CPU(%)
    
                synergyc( 3604/ 3602):  17   0   0
         arm-starfish-li(16087/16087):   4   8   5
                  guider(16088/16088):   6   0   0
                KnottyUI(15896/15896):   0   0   0
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        # Delay(%)
    
         arm-starfish-li(16087/16087):   0   0   0
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        # MEM(MB)
    
         arm-starfish-li(16087/16087):  15  44   1
                  guider(16088/16088):   0   0   0
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        # BLK_RD(MB)
    
         arm-starfish-li(16087/16087):   3   3   0
         Plex Media Serv( 2965/ 2956):   0   0   0
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        # BLK_WR(MB)
    
         Plex Media Serv( 2965/ 2956):   0   0   0
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is interval output about resource usage in thread mode.    
        User can analyze resource usage of threads in each specific seconds.    
    
    +  Graph Info [-i -e g option]    
        ![guider_interval](https://cloud.githubusercontent.com/assets/15862689/23349234/a6894a6a-fcf4-11e6-911f-7d2df9e1d571.png)    
        This is graphical output about resource usage in thread mode.    
        User can analyze resource usage of threads in graphical image.    
    
    +  IRQ Info  [-a -e i option]   
        ``````````````````````````````````````````
        [Thread IRQ Info]
        ==========================================================================================================================================================
              IRQ       (                             Name                             ):    Count       Usage     ProcMax    ProcMin    InterMax   InterMin
        ==========================================================================================================================================================
        # IRQ(9) / Total( 0.004) / Cnt(1226)
    
                  irq/88(                             ahci                             ):          527  0.001570   0.000013   0.000002   0.035694   0.000010
                  irq/90(                             eth0                             ):            6  0.000010   0.000002   0.000001   0.879542   0.000073
                  irq/92(                           nouveau                            ):           35  0.000535   0.000024   0.000014   0.016661   0.001491
               softirq/1(                            TIMER                             ):           85  0.000118   0.000005   0.000001   0.231687   0.000002
               softirq/3(                            NET_RX                            ):            6  0.000084   0.000033   0.000001   0.879542   0.000073
               softirq/4(                            BLOCK                             ):          527  0.001180   0.000019   0.000001   0.035696   0.000010
               softirq/7(                            SCHED                             ):           24  0.000050   0.000014   0.000001   0.231684   0.007992
               softirq/8(                           HRTIMER                            ):            1  0.000004   0.000004   0.000004   0.000000   0.000000
               softirq/9(                             RCU                              ):           15  0.000006   0.000001   0.000000   0.299594   0.003970
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about interrupt details in thread mode.    
    
    +  Syscall Info [-t option]    
        ``````````````````````````````````````````
        [Thread Syscall Info]
        ==========================================================================================================================================================
                    Name( Tid)    Syscall     SysId        Usage       Count           Min             Max             Avg
        ==========================================================================================================================================================
         Chrome_ChildIOT(4264)
                            sys_recvmsg          47        0.000           2      0.000002        0.000010        0.000006
                              sys_futex         202        0.000           1      0.000010        0.000010        0.000010
                             sys_gettid         186        0.000           1      0.000001        0.000001        0.000001
                         sys_epoll_wait         232        0.000           2      0.000000        0.000001        0.000000
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         Chrome_IOThread(3735)
                            sys_sendmsg          46        0.000           1      0.000015        0.000015        0.000015
                               sys_read           0        0.000           2      0.000001        0.000011        0.000006
                          sys_epoll_ctl         233        0.000           2      0.000001        0.000009        0.000005
                         sys_epoll_wait         232        0.000           2      0.000001        0.000001        0.000001
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         Plex DLNA Serve(23531)
                              sys_futex         202        1.100          47      0.000001        0.099949        0.023407
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
        ``````````````````````````````````````````
        [Thread Syscall History]
        ==========================================================================================================================================================
                    Name( Tid)        Time        Diff     Type            Syscall     SysId  Core      Return    Parameter
        ==========================================================================================================================================================
                  guider(13089)   0.005889    0.000000     exit          sys_write         1    11           1    -
                  guider(13089)   0.005892    0.000002     both          sys_close         3    11           0    (3, 7f4f84292000, 7f4f84293000, 4f43e0, 252e7a0, 7f4f84267740)
                  guider(13089)   0.005897    0.000007     both         sys_munmap        11    11           0    (7f4f84292000, 1000, 0, 4f43e0, 252e7a0, 7f4f84267740)
                  guider(13089)   0.005924    0.000013     both           sys_open         2    11          -2    (1f83f20, 241, 1b6, 91a870, 208d106, 240)
                  guider(13089)   0.005981    0.000011     both           sys_open         2    11           3    (1f83f20, 241, 1b6, 91a870, 208d106, 240)
                  guider(13089)   0.005994    0.000001     both          sys_fstat         5    11           0    (3, 7ffdf528fa60, 7ffdf528fa60, 1, 0, 240)
                  guider(13089)   0.005998    0.000000     both          sys_fstat         5    11           0    (3, 7ffdf528fa70, 7ffdf528fa70, d, 252e7a0, 7f4f84215414)
                  guider(13089)   0.005999    0.000004     both           sys_mmap         9    11    139979496431616 (0, 1000, 3, 22, ffffffff, 0)
                  guider(13089)   0.006009    0.000183     both          sys_write         1    11           1    (3, 7f4f84292000, 1, 4f43e0, 252e7a0, 7f4f84267740)
                  guider(13089)   0.006194    0.000001     both          sys_close         3    11           0    (3, 7f4f84292000, 7f4f84293000, 4f43e0, 252e7a0, 7f4f84267740)
                  guider(13089)   0.006198    0.000007     both         sys_munmap        11    11           0    (7f4f84292000, 1000, 0, 4f43e0, 252e7a0, 7f4f84267740)
                  guider(13089)   0.006222    0.000012     both           sys_open         2    11           3    (1f83f20, 241, 1b6, 91a870, 208d106, 240)
                  guider(13089)   0.006235    0.000001     both          sys_fstat         5    11           0    (3, 7ffdf528fa60, 7ffdf528fa60, 1, 0, 240)
                  guider(13089)   0.006239    0.000000     both          sys_fstat         5    11           0    (3, 7ffdf528fa70, 7ffdf528fa70, d, 252e7a0, 7f4f84215414)
        ``````````````````````````````````````````
    
        This is output about system call details in thread mode.    
    
    +  Module Info [-a option]    
        ``````````````````````````````````````````
        [Thread Module Info]
        ==========================================================================================================================================================
        _Type_|_Time_|_____Module_____|__Thread Name___( Tid )|Elapsed|
        ==========================================================================================================================================================
         FREE | 1.892|    psmouse     |           rmmod(13336)|
         LOAD | 3.525|    psmouse     |          insmod(13339)|  0.003|
         FREE | 5.545|    psmouse     |           rmmod(13342)|
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about kernel module details in thread mode.    
    
    +  Creation Info [-a option]    
        ``````````````````````````````````````````
        [Thread Creation Info] [Alive: +] [Die: -] [CreatedTime: //] [ChildCount: ||] [CpuUsage: <>] [WaitTimeForChilds: {}] [WaitTimeOfParent: []]
        ==========================================================================================================================================================
        + Plex DLNA Serve(3249) <0.001> |2|
             - Plex(13345) /4.641/ <0.002> |1|
                  - Plex(13346) /6.643/ <0.004>
             + Plex(13348) /6.707/ <0.000>
        + Plex DLNA Serve(3243) <0.000> |3|
             - Plex(13343) /6.138/ <0.000>
             - Plex(13344) /6.138/ <0.000>
             - Plex(13347) /6.694/ <0.001>
        + systemd-udevd(449) <0.002> |2|
             + systemd-udevd(13337) /1.892/ <0.001>
             + systemd-udevd(13338) /1.893/ <0.001>
        + bash(27395) <0.010> |4|
             - rmmod(13336) /0.392/ <0.002>
             - insmod(13339) /2.018/ <0.010>
             - rmmod(13342) /4.045/ <0.002>
             + guider(13349) /6.441/ <0.104>
        + kthreadd(2) <0.000> |2|
             - kthreadd(13340) /3.525/ <0.000>
             + kthreadd(13341) /3.534/ <0.000>
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about thread creation details in thread mode.    
        User can analyze thread tree and wait time that a parent thread waits its child threads.    
    
    +  Custom Event Info [-c option]    
        ``````````````````````````````````````````
        [Thread CUSTOM Event Info]
        ==========================================================================================================================================================
                     Event                           Comm( Tid )      Count  MaxPeriod  MinPeriod
        ==========================================================================================================================================================
                 hrtimer_cancel                     TOTAL(  -  )         52   0.111849   0.000109
                                                 synergyc( 3604)         33   0.003995   0.000373
                                                   guider(22424)          6   0.002951   0.000147
                                                   guider(14882)          2   0.111849   0.111849
                                          Chrome_IOThread( 3735)          2   0.000266   0.000266
                                                     Xorg( 1636)          2   0.000109   0.000109
                                                 synergyc( 3602)          2   0.000053   0.000053
                                                 KnottyUI(22243)          1   0.000000   0.000000
                                          Plex Media Serv(14880)          1   0.000000   0.000000
                                          chromium-browse( 4273)          1   0.000000   0.000000
                                          BrowserBlocking( 3729)          1   0.000000   0.000000
                                                    gmain( 2233)          1   0.000000   0.000000
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
        ``````````````````````````````````````````
        [Thread CUSTOM Event History]
        ==========================================================================================================================================================
                     EVENT                  TIME                COMM(  TID) ARG
        ==========================================================================================================================================================
                 hrtimer_cancel            0.033069         synergyc( 3602) hrtimer=ffff880fc45d1a70
                 hrtimer_cancel            0.033122         synergyc( 3602) hrtimer=ffff880fc45d1a70
                 hrtimer_cancel            0.065970           guider(14882) hrtimer=ffff88101fd6dce0
                 hrtimer_cancel            0.177819           guider(14882) hrtimer=ffff88101fd6dce0
                 hrtimer_cancel            0.521354           guider(22424) hrtimer=ffff88101fc4dce0
                 hrtimer_cancel            0.524193           guider(22424) hrtimer=ffff880e6b9a5de0
                 hrtimer_cancel            0.525349           guider(22424) hrtimer=ffff88101fc4dce0
                 hrtimer_cancel            0.526243           guider(22424) hrtimer=ffff8800c126bde0
                 hrtimer_cancel            0.526390           guider(22424) hrtimer=ffff8801009f3de0
                 hrtimer_cancel            0.529209            gmain( 2233) hrtimer=ffff880f8fe8fa70
                 hrtimer_cancel            0.529341           guider(22424) hrtimer=ffff88101fc4dce0
                 hrtimer_cancel            0.725987  Chrome_IOThread( 3735) hrtimer=ffff880f67b3ddf8
                 hrtimer_cancel            0.726215  BrowserBlocking( 3729) hrtimer=ffff8800c126bde0
                 hrtimer_cancel            0.726253  Chrome_IOThread( 3735) hrtimer=ffff880f67b3ddf8
                 hrtimer_cancel            0.726379  chromium-browse( 4273) hrtimer=ffff880f66e75de0
                 hrtimer_cancel            1.072608  Plex Media Serv(14880) hrtimer=ffff88101fccdce0
                 hrtimer_cancel            1.388184         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.392177         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.396172         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.400166         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.402252         KnottyUI(22243) hrtimer=ffff880f9176dde0
                 hrtimer_cancel            1.404160         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.408155         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.412149         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.415771         synergyc( 3604) hrtimer=ffff8800c127fde0
                 hrtimer_cancel            1.416144         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.420138         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.424133         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.428128         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.432122         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.436117         synergyc( 3604) hrtimer=ffff88101fcadce0
                 hrtimer_cancel            1.438559             Xorg( 1636) hrtimer=ffff880fc4abd938
        ``````````````````````````````````````````
    
        This is output about custom event in thread mode.    
        User can analyze ftrace tracing points.    
    
    +  Preemption Info [-p option]    
        ``````````````````````````````````````````
        [Thread Info] [ Elapsed: 2.742 ] [ Start: 75.657 ] [ Running: 139 ] [ CtxSwc: 20255 ] [ LogSize: 5500 KB ] [ Unit: Sec/MB/NR ]
        ==========================================================================================================================================================
        __________Thread Info___________|_____________CPU Info______________|______SCHED Info______|________BLOCK Info________|_____________MEM Info_____________|
                                        |                                   |                      |                          |                                  |
                    Name(  Tid/  Pid)|LF|Usage(    %)|Delay(  Max)|Pri| IRQ |  Yld| Lose|Steal| Mig| Read( MB/  Cnt)|WCnt( MB)| Sum(Usr/Buf/Ker)|Rcl|Wst|DRcl(Nr)|
        ==========================================================================================================================================================
        # CPU: 4
    
                  CORE/0(-----/-----)|--| 0.19(  6.8)| 0.00( 0.00)|  0| 0.00|    0|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                  CORE/1(-----/-----)|--| 0.21(  7.5)| 0.00( 0.00)|  0| 0.00|    0|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                  CORE/2(-----/-----)|--| 0.40( 14.6)| 0.00( 0.00)|  0| 0.00|    0|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                  CORE/3(-----/-----)|--| 0.17(  6.4)| 0.65( 0.00)|  0| 0.00|    1|    -|    -|   -| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        # Hot: 9
    
         surface-manager( 1615/ 1615)|  | 0.29( 10.7)| 0.02( 0.00)|  0| 0.00|  161|  314|   77|   4| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                  guider( 3474/ 3474)|  | 0.06(  2.3)| 0.01( 0.00)|  0| 0.00|    2|   40|    1|   1| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
             tWindowCore( 1442/ 1407)|  | 0.06(  2.3)| 0.01( 0.00)|  0| 0.00|  333|  200|  314|   3| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
           mali-renderer( 1681/ 1615)|  | 0.05(  1.9)| 0.00( 0.00)|  0| 0.00|  559|  101|  540|  10| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                  guider( 4875/ 4875)|  | 0.05(  1.7)| 0.00( 0.00)|  0| 0.00|    2|   10|    2|   0| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                tvideoCC( 2422/ 1613)|  | 0.04(  1.4)| 0.00( 0.00)|  0| 0.00|  285|    7|  268|   2| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
               pq_thread( 2558/ 1613)|  | 0.04(  1.3)| 0.00( 0.00)|  0| 0.00|  103|   30|   97|   0| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
                n0000028( 2450/ 1613)|  | 0.03(  1.0)| 0.00( 0.00)|  0| 0.00|  529|    3|  473|   5| 0.00(  0/    0)|   0(  0)|   0(  0/  0/  0)|  0|  0|0.00( 0)|
    
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        # PRT: Tid(1615) / Comm(surface-manager) / Total( 0.017) / Threads(45)
    
           mali-renderer( 1681/    0)|  | 0.01( 40.5)
         com.webos.app.i( 1785/    0)|  | 0.00( 10.9)
                n0000034( 2512/    0)|  | 0.00(  7.0)
                 tHDMI5V( 2458/    0)|  | 0.00(  6.7)
                tvideoCC( 2422/    0)|  | 0.00(  4.4)
                n0000028( 2450/    0)|  | 0.00(  4.1)
                     sdx( 4858/    0)|  | 0.00(  2.8)
                 tAUDATV( 2470/    0)|  | 0.00(  2.7)
                tCMState( 2490/    0)|  | 0.00(  2.5)
                    tAVD( 2454/    0)|  | 0.00(  2.0)
                tAUDHDMI( 2469/    0)|  | 0.00(  1.6)
            kworker/u9:1( 1524/    0)|  | 0.00(  1.4)
             rcu_preempt(    7/    0)|  | 0.00(  1.1)
                  sleepd( 2251/    0)|  | 0.00(  0.9)
                n0000005( 2226/    0)|  | 0.00(  0.9)
                  tHDMI1( 2459/    0)|  | 0.00(  0.8)
         hdmi20_rx_video( 2451/    0)|  | 0.00(  0.8)
              te_monitor(  940/    0)|  | 0.00(  0.8)
          mali-event-hnd( 1676/    0)|  | 0.00(  0.8)
         physical-device( 3010/    0)|  | 0.00(  0.7)
            HSM Read Map( 2548/    0)|  | 0.00(  0.6)
                  tHDMI3( 2461/    0)|  | 0.00(  0.5)
                  tHDMI4( 2462/    0)|  | 0.00(  0.4)
                  tHDMI2( 2460/    0)|  | 0.00(  0.4)
                  ACCT00( 2524/    0)|  | 0.00(  0.4)
           tSettingsDone( 2563/    0)|  | 0.00(  0.4)
               ARTIMER_T( 2473/    0)|  | 0.00(  0.4)
        ``````````````````````````````````````````
    
        This is output about preemption details in thread mode.    
        User can analyze which threads had preempted a specific thread.    
    
    +  Signal Info [-a option]    
        ``````````````````````````````````````````
        [Thread Signal Info]
        ==========================================================================================================================================================
         TYPE        TIME           SENDER(  TID)   SIGNAL           RECEIVER(  TID)
        ==========================================================================================================================================================
         RECV    0.394702                          SIGCHLD               bash(27395)
         SEND    1.892381            rmmod(13336)  SIGCHLD               bash(27395)
         RECV    2.027533                          SIGCHLD               bash(27395)
         SEND    3.525273           insmod(13339)  SIGCHLD               bash(27395)
         RECV    4.047565                          SIGCHLD               bash(27395)
         SEND    5.545286            rmmod(13342)  SIGCHLD               bash(27395)
         RECV    6.547838                          SIGCHLD               bash(27395)
         SEND    7.946579        swapper/4( 0[4])  SIGALRM             screen( 7400)
         RECV    7.946588                          SIGALRM             screen( 7400)
         SEND    8.042445           guider(13349)   SIGINT             guider(13335)
         RECV    8.042476                           SIGINT             guider(13335)
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about signal traffic in thread mode.    
    
    +  User Event Info [-U option]    
        ``````````````````````````````````````````
        [Thread USER Event Info]
        ==========================================================================================================================================================
                     Event                           Comm( Tid )      Usage      Count    ProcMax    ProcMin   InterMax   InterMin
        ==========================================================================================================================================================
                    diskFunc                        TOTAL(  -  )   0.004830          1   0.004830   0.004830   0.000000   0.000000
                                                    a.out(15158)   0.004830          1   0.004830   0.004830   0.000000   0.000000
    
                    cpuFunc                         TOTAL(  -  )   0.000000          1   0.000000   0.000000   0.000000   0.000000
                                                    a.out(15158)   0.000000          1   0.000000   0.000000   0.000000   0.000000
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
        ``````````````````````````````````````````
        [Thread USER Event History]
        ==========================================================================================================================================================
                     EVENT                TYPE     TIME                COMM(  TID)      CALLER         ELAPSED
        ==========================================================================================================================================================
                    diskFunc              ENTER   1.981194            a.out(15158)                           -
                    diskFunc               EXIT   1.986024            a.out(15158)        0x400ab3    0.004830
                    cpuFunc               ENTER   1.986029            a.out(15158)                           -
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about user event in thread mode.    
        User can analyze user-level function call events.    
    
    
    +  Kernel Event Info [-K option]    
        ``````````````````````````````````````````
        [Thread KERNEL Event Info]
        ==========================================================================================================================================================
                     Event                           Comm( Tid )      Usage      Count    ProcMax    ProcMin   InterMax   InterMin
        ==========================================================================================================================================================
                      open                          TOTAL(  -  )   0.000000          0   0.000000   0.000000   0.000000   0.000000
                                                   guider(15403)   0.000000          0   0.000000   0.000000   0.000000   0.000000
                                                   guider(22424)   0.000000          0   0.000000   0.000000   0.000000   0.000000
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
        ``````````````````````````````````````````
        [Thread KERNEL Event History]
        ==========================================================================================================================================================
                     EVENT                TYPE     TIME                COMM(  TID)         CALLER            ELAPSED ARG
        ==========================================================================================================================================================
                      open                 EXIT   0.000000           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/sched/sched_switch/filter"
                      open                 EXIT   0.000064           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/sched/sched_switch/enable"
                      open                 EXIT   0.000186           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/sched/sched_migrate_task/enable"
                      open                 EXIT   0.000414           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/sched/sched_process_exit/enable"
                      open                 EXIT   0.000644           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/sched/sched_process_wait/enable"
                      open                 EXIT   0.000877           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/filter"
                      open                 EXIT   0.000921           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/enable"
                      open                 EXIT   0.000959           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/power/suspend_resume/enable"
                      open                 EXIT   0.001026           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/module/module_load/enable"
                      open                 EXIT   0.001260           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/module/module_free/enable"
                      open                 EXIT   0.001488           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/module/module_put/enable"
                      open                 EXIT   0.002246           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/power/cpu_idle/enable"
                      open                 EXIT   0.002985           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/power/cpu_frequency/enable"
                      open                 EXIT   0.003325           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/vmscan/mm_vmscan_direct_reclaim_begin/enable"
                      open                 EXIT   0.003552           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/vmscan/mm_vmscan_direct_reclaim_end/enable"
                      open                 EXIT   0.003778           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/task/enable"
                      open                 EXIT   0.004226           guider(15403)            do_sys_open          -  1>"/sys/kernel/debug/tracing/events/signal/enable"
        ``````````````````````````````````````````
    
        This is output about kernel event in thread mode.    
        User can analyze kernel-level function call events.    
    
    +  Dependency Info [-D option]    
        ``````````````````````````````````````````
        [Thread Dependency Info]
        ==========================================================================================================================================================
          Total/Inter                 From( Tid) ->               To( Tid)    Event
        ==========================================================================================================================================================
        # Dep: 182
    
          0.002/0.000               guider(3474) ->    mali-renderer(1681)    kick
          0.002/0.000        mali-renderer(1681) ->           tHDMI4(2462)    kick
          0.003/0.000               guider(3474) ->              sdx(4858)    kick
          0.003/0.000             n0000028(2450) ->           ACCT00(2524)    kick
          0.004/0.001                                  mali-renderer(1681)    wakeup
          0.012/0.008               guider(3474) ->             qosm(3484)    kick
          0.012/0.001                                    PmLogDaemon(1417)    recv
          0.019/0.007                                  mali-renderer(1681)    wakeup
          0.137/0.118                                  mali-renderer(1681)    wakeup
          0.139/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          0.152/0.014      surface-manager(1615) ->    mali-renderer(1681)    kick
          0.154/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          0.169/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          0.171/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          0.185/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          0.187/0.002                                  mali-renderer(1681)    wakeup
          0.204/0.017       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          0.221/0.017       mali-event-hnd(1676) ->            MICOM(2380)    kick
          0.221/0.000       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          0.269/0.048      surface-manager(1615) ->    mali-renderer(1681)    kick
          0.371/0.102       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          0.654/0.283                                  mali-renderer(1681)    wakeup
          0.669/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          0.804/0.135       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.002/0.198      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.004/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.018/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.021/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.035/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.037/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.052/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.054/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.068/0.014      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.071/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.085/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.087/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.102/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.104/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
          1.118/0.015      surface-manager(1615) ->    mali-renderer(1681)    kick
          1.121/0.002       mali-event-hnd(1676) ->    mali-renderer(1681)    kick
        ``````````````````````````````````````````
    
        This is output about dependency of each threads in thread mode.    
    
    1. Function mode [-f option]

      [Function Thread Info] [ Elapsed: 3.569 ] [ Start: 2858278.707 ] [ Threads: 23 ] [ LogSize: 1541 KB ]
      ==========================================================================================================================================================
      _______________________Thread________________________|__CPU__|_________________________PAGE_________________________|__HEAP__|______BLOCK_______|_CUSTOM_|
                                                           |       |                                                      |        |                  |        |
      ______Name______|__Tid__|__Pid__|_Target_|_DIE_|_NEW_|__PER__|__ALLOC__(__USER__/__BUF___/__KERN__)|__FREE__|_UFREE_|__EXP___|__READ__|__WRITE__|___NR___|
      ==========================================================================================================================================================
                 a.out|  15939|  15939|   *    |     |  v  |  98.3%|   11416K(   3140K/      0K/   8276K)|   104K|     84K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|  23531|   3010|   *    |     |     |   0.2%|       0K(      0K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|  15944|  -----|   *    |  v  |  v  |   0.2%|       0K(      0K/      0K/      0K)|  2024K|      0K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|   9871|   3010|   *    |     |     |   0.2%|       0K(      0K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
                guider|  15936|  15936|   *    |     |     |   0.2%|       0K(      0K/      0K/      0K)|   188K|    120K|      0K|      0K|       0K|       0|
                  bash|   4011|   4011|   *    |     |     |   0.2%|      92K(     84K/      0K/      8K)|     0K|      0K|      0K|      0K|       0K|       0|
           ksoftirqd/1|     37|     37|   *    |     |     |   0.2%|       0K(      0K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
                compiz|   2571|   2571|   *    |     |     |   0.2%|      12K(     12K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|  15943|  -----|   *    |  v  |  v  |   0.0%|       0K(      0K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|   8385|   3010|   *    |     |     |   0.0%|       0K(      0K/      0K/      0K)|     4K|      0K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|  15941|  -----|   *    |  v  |  v  |   0.0%|       0K(      0K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
         kworker/u24:2|    679|    679|   *    |     |     |   0.0%|       4K(      0K/      0K/      4K)|     0K|      0K|      0K|      0K|      64K|       0|
                  Xorg|   1636|   1636|   *    |     |     |   0.0%|       8K(      0K/      0K/      8K)|    40K|      8K|      0K|      0K|       0K|       0|
       chromium-browse|   4273|   4273|   *    |     |     |   0.0%|       4K(      4K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
       chromium-browse|   4507|   4507|   *    |     |     |   0.0%|       4K(      4K/      0K/      0K)|     0K|      0K|      0K|      0K|       0K|       0|
               rcuos/3|     11|     11|   *    |     |     |   0.0%|       0K(      0K/      0K/      0K)|     8K|      0K|      0K|      0K|       0K|       0|
                screen|   3565|   3565|   *    |     |     |   0.0%|       4K(      0K/      0K/      4K)|    40K|      4K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|   3243|   3010|   *    |     |     |   0.0%|      40K(      0K/      0K/     40K)|     0K|      0K|      0K|      0K|       0K|       0|
           kworker/2:0|  13550|  13550|   *    |     |     |   0.0%|       0K(      0K/      0K/      0K)|   516K|     12K|      0K|      0K|       0K|       0|
       Plex DLNA Serve|   3246|   3010|   *    |     |     |   0.0%|       8K(      0K/      0K/      8K)|     0K|      0K|      0K|      0K|       0K|       0|
               rcuos/8|     16|     16|   *    |     |     |   0.0%|       0K(      0K/      0K/      0K)|    16K|      0K|      0K|      0K|       0K|       0|
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      

      This is output about thread list profiled in function mode.
      User can select specific threads with -g option and analyze resource usage by function.
      But symbol of threads that already died will not be shown.

    +  CPU Info    
        ``````````````````````````````````````````
        [Function CPU Info] [Cnt: 394] [Interval: 8ms] (USER)
        ==========================================================================================================================================================
        __Usage__|___________________Function____________________|_____________________________________________Binary_____________________________________________
        ==========================================================================================================================================================
           99.0% |                    cpuTest                    | /media/disk/work/test/a.out
           +  100.0% | <- startTest [/media/disk/work/test/a.out] <- main [/media/disk/work/test/a.out]
                         <- __libc_start_main [/lib/x86_64-linux-gnu/libc-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            0.5% |                    memset                     | /lib/x86_64-linux-gnu/libc-2.19.so
           +  100.0% | <- startTest [/media/disk/work/test/a.out] <- main [/media/disk/work/test/a.out]
                         <- __libc_start_main [/lib/x86_64-linux-gnu/libc-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            0.3% |                  _int_malloc                  | /lib/x86_64-linux-gnu/libc-2.19.so
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            0.3% |               00007f756e3e7ee4                | ??
           +  100.0% | <- 000000000044676f [/media/disk/work/test/a.out]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Function CPU Info] [Cnt: 394] [Interval: 8ms] (KERNEL)
        ==========================================================================================================================================================
        __Usage__|____________________________________________________________________Function____________________________________________________________________
        ==========================================================================================================================================================
          100.0% |                                                          hrtimer_interrupt
           +   99.5% | <- local_apic_timer_interrupt <- smp_apic_timer_interrupt <- apic_timer_interrupt
           +    0.3% | <- local_apic_timer_interrupt <- smp_apic_timer_interrupt <- apic_timer_interrupt <- do_page_fault <- page_fault
           +    0.3% | <- local_apic_timer_interrupt <- smp_apic_timer_interrupt <- apic_timer_interrupt <- __do_fault <- handle_mm_fault <- __do_page_fault
                         <- do_page_fault <- page_fault
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about cpu usage of specific threads by function in function mode.    
    
    +  Mem Info [-f -e m option]    
        ``````````````````````````````````````````
        [Function Page Info] [Total: 11416KB] [Alloc: 11444KB(817)] [Free: 188KB(47)] (USER)
        ==========================================================================================================================================================
         Usage ( Usr  / Buf  / Ker  )|___________________Function____________________|________________LifeTime________________|______________Binary_______________
        ==========================================================================================================================================================
         10256K(  2048/     0/  8208)|                    memset                     | AVR: 1.563 / MIN: 1.560 / MAX: 1.568   | /lib/x86_64-linux-gnu/libc-2.19.so
          +  10256K(  2048/     0/  8208)| <- startTest [/media/disk/work/test/a.out] <- main [/media/disk/work/test/a.out]
                                             <- __libc_start_main [/lib/x86_64-linux-gnu/libc-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
           960K(   956/     0/     4)|                  _int_malloc                  | AVR: 1.559 / MIN: 1.554 / MAX: 1.560   | /lib/x86_64-linux-gnu/libc-2.19.so
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            56K(    16/     0/    40)|               00007f756e3e81e7                | AVR: 1.569 / MIN: 1.568 / MAX: 1.569   | ??
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            44K(    36/     0/     8)|                   sysmalloc                   | AVR: 1.560 / MIN: 1.558 / MAX: 1.568   | /lib/x86_64-linux-gnu/libc-2.19.so
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            12K(    12/     0/     0)|           elf_machine_rela_relative           | AVR: 1.568 / MIN: 1.568 / MAX: 1.568   | /lib/x86_64-linux-gnu/ld-2.19.so
          +     12K(    12/     0/     0)| <- dl_main [/lib/x86_64-linux-gnu/ld-2.19.so] <- _dl_sysdep_start [/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             8K(     8/     0/     0)|                    realloc                    | AVR: 1.568 / MIN: 1.568 / MAX: 1.568   | /lib/x86_64-linux-gnu/ld-2.19.so
          +      4K(     4/     0/     0)| <- _dl_map_object [/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             8K(     4/     0/     4)|                    dl_main                    | AVR: 1.568 / MIN: 1.568 / MAX: 1.568   | /lib/x86_64-linux-gnu/ld-2.19.so
          +      8K(     4/     0/     4)| <- _dl_sysdep_start [/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Function Page Info] [Total: 11416KB] [Alloc: 11444KB(817)] [Free: 188KB(47)] (KERNEL)
        ==========================================================================================================================================================
         Usage ( Usr  / Buf  / Ker  )|___________________Function____________________|__________________________________LifeTime__________________________________
        ==========================================================================================================================================================
          8192K(     0/     0/  8192)|          do_huge_pmd_anonymous_page           |                    AVR: 1.563 / MIN: 1.562 / MAX: 1.564
          +   8192K(     0/     0/  8192)| <- handle_mm_fault <- __do_page_fault <- do_page_fault <- page_fault
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
          3084K(  3084/     0/     0)|                handle_mm_fault                |                    AVR: 1.563 / MIN: 1.554 / MAX: 1.569
          +   3076K(  3076/     0/     0)| <- __do_page_fault <- do_page_fault <- page_fault
          +      4K(     4/     0/     0)| <- __get_user_pages <- get_user_pages <- copy_strings.isra.17 <- copy_strings_kernel <- do_execve_common.isra.23
                                             <- SyS_execve <- stub_execve
          +      4K(     4/     0/     0)| <- __do_page_fault <- do_page_fault <- page_fault <- load_elf_binary <- search_binary_handler
                                             <- do_execve_common.isra.23 <- SyS_execve <- stub_execve
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about memory usage of specific threads by function in function mode.    
    
    +  Heap Info [-f -e h option]    
        ``````````````````````````````````````````
        [Function Heap Info] [Total: 18913KB] [Alloc: 19074KB(16)] [Free: 161KB(1)] (USER)
        ==========================================================================================================================================================
        __Usage__|___________________Function____________________|_____________________Binary______________________|____________________Source____________________
        ==========================================================================================================================================================
          10244K |                    mmap64                     | /lib/x86_64-linux-gnu/libc-2.19.so              | /build/eglibc-oGUzwX/eglibc-2.19/misc/../sysdeps/unix/syscall-template.S:81
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
           4752K |                     __brk                     | /lib/x86_64-linux-gnu/ld-2.19.so                | /build/eglibc-oGUzwX/eglibc-2.19/elf/../sysdeps/unix/sysv/linux/x86_64/brk.c:31
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
           3917K |                    realloc                    | /lib/x86_64-linux-gnu/ld-2.19.so                | /build/eglibc-oGUzwX/eglibc-2.19/misc/../sysdeps/unix/syscall-template.S:81
          +    3897K | <- _dl_map_object [/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Function Heap History] [Cnt: 15]
        ==========================================================================================================================================================
        _____________VAddr______________|____Size____|__Size(KB)__|__Size(MB)__|______COMM_______(__TID__)|__CORE__|______TIME_______|
        ==========================================================================================================================================================
                139733299458048         |       8192 |          8 |          0 |            a.out(  16284)|      5 |  2858989.603987 |
        ------------------------------------------------------------------------------------------------------------------------------
                                 [User] | realloc[/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                139733293277184         |    3949248 |       3856 |          3 |            a.out(  16284)|      5 |  2858989.604134 |
        ------------------------------------------------------------------------------------------------------------------------------
                                 [User] | realloc[/lib/x86_64-linux-gnu/ld-2.19.so] <- _dl_map_object[/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                139733297184768         |      24576 |         24 |          0 |            a.out(  16284)|      5 |  2858989.604164 |
        ------------------------------------------------------------------------------------------------------------------------------
                                 [User] | realloc[/lib/x86_64-linux-gnu/ld-2.19.so] <- _dl_map_object[/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                139733297209344         |      17088 |         16 |          0 |            a.out(  16284)|      5 |  2858989.604191 |
        ------------------------------------------------------------------------------------------------------------------------------
                                 [User] | realloc[/lib/x86_64-linux-gnu/ld-2.19.so] <- _dl_map_object[/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                139733299286016         |       4096 |          4 |          0 |            a.out(  16284)|      5 |  2858989.604241 |
        ------------------------------------------------------------------------------------------------------------------------------
                                 [User] | realloc[/lib/x86_64-linux-gnu/ld-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about heap usage of specific threads by function in function mode.    
    
    +  Block Info [-f -e b option]    
        ``````````````````````````````````````````
        [Function BLK_RD Info] [Size: 10396KB] [Cnt: 90] (USER)
        ==========================================================================================================================================================
        __Usage__|___________________Function____________________|_____________________Binary______________________|____________________Source____________________
        ==========================================================================================================================================================
          10384K |                    __read                     | /lib/x86_64-linux-gnu/libc-2.19.so              | /build/eglibc-oGUzwX/eglibc-2.19/io/../sysdeps/unix/syscall-template.S:81
          +   10384K | <- startTest [/media/disk/work/test/a.out] <- main [/media/disk/work/test/a.out]
                         <- __libc_start_main [/lib/x86_64-linux-gnu/libc-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             12K |               00007f756e3e81e7                | ??                                              |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Function BLK_RD Info] [Size: 10396KB] [Cnt: 90] (KERNEL)
        ==========================================================================================================================================================
        __Usage__|____________________________________________________________________Function____________________________________________________________________
        ==========================================================================================================================================================
          10396K |                                                                   submit_bio
          +   10016K | <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead <- page_cache_async_readahead
                         <- generic_file_aio_read <- do_sync_read <- vfs_read <- SyS_read <- tracesys
          +     128K | <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead <- page_cache_sync_readahead
                         <- generic_file_aio_read <- do_sync_read <- vfs_read <- SyS_read <- tracesys
          +      96K | <- do_mpage_readpage <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead
                         <- page_cache_async_readahead <- generic_file_aio_read <- do_sync_read <- vfs_read <- SyS_read <- tracesys
          +      80K | <- do_mpage_readpage <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead <- page_cache_sync_readahead
                         <- generic_file_aio_read <- do_sync_read <- vfs_read <- SyS_read <- tracesys
          +      48K | <- do_mpage_readpage <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead <- page_cache_sync_readahead
                         <- generic_file_aio_read <- do_sync_read <- vfs_read <- SyS_read <- tracesys
          +      12K | <- _submit_bh <- bh_submit_read <- ext4_get_branch <- ext4_ind_map_blocks <- ext4_map_blocks <- _ext4_get_block <- ext4_get_block
                         <- do_mpage_readpage <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead
                         <- page_cache_async_readahead <- generic_file_aio_read <- do_sync_read <- vfs_read <- SyS_read <- tracesys
          +      12K | <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead <- page_cache_sync_readahead
                         <- generic_file_aio_read <- do_sync_read <- vfs_read <- kernel_read <- prepare_binprm <- do_execve_common.isra.23 <- SyS_execve
                         <- stub_execve
          +       4K | <- _submit_bh <- bh_submit_read <- ext4_get_branch <- ext4_ind_map_blocks <- ext4_map_blocks <- _ext4_get_block <- ext4_get_block
                         <- do_mpage_readpage <- mpage_readpages <- ext4_readpages <- __do_page_cache_readahead <- ondemand_readahead
                         <- page_cache_sync_readahead <- generic_file_aio_read <- do_sync_read <- vfs_read <- SyS_read <- tracesys
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about disk(block) usage of specific threads by function in function mode.    
    
    +  Kernel Event Info [-f -K option]    
        ```````````````````````````````````````
        [Function segflt_exit, segflt_enter Info] [Cnt: 2] [Total: 2] (USER)
        ==========================================================================================================================================================
        __Usage__|___________________Function____________________|_____________________Binary______________________|____________________Source____________________
        ==========================================================================================================================================================
              2  |                   faultTest                   | /media/disk/work/test/a.out                     | ??:?
               +      2 | <- startTest [/media/disk/work/test/a.out] <- main [/media/disk/work/test/a.out]
                            <- __libc_start_main [/lib/x86_64-linux-gnu/libc-2.19.so]
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Function segflt_exit, segflt_enter Info] [Cnt: 2] [Total: 2] (KERNEL)
        ==========================================================================================================================================================
        __Usage__|____________________________________________________________________Function____________________________________________________________________
        ==========================================================================================================================================================
              2  |                                                            do_page_fault
               +      2 | <- page_fault
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
        ``````````````````````````````````````````
        [Function segflt_exit, segflt_enter History] [Cnt: 2] [Total: 2]
        ==========================================================================================================================================================
        _____________Event______________|______COMM_______(__TID__)|__CORE__|______TIME_______|
        ==========================================================================================================================================================
                  segflt_enter          |            a.out(  15657)|    010 |  2870585.085374 |
        ---------------------------------------------------------------------------------------
                                 [Args] | (bad_area+0x0/0x4a <ffffffff81722b0e>)
                                 [User] | faultTest[/media/disk/work/test/a.out] <- startTest[/media/disk/work/test/a.out] <- main[/media/disk/work/test/a.out]
                                          <- __libc_start_main[/lib/x86_64-linux-gnu/libc-2.19.so]
                               [Kernel] | do_page_fault <- page_fault
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  segflt_exit           |            a.out(  15657)|    010 |  2870585.085420 |
        ---------------------------------------------------------------------------------------
                                 [Args] | (__do_page_fault+0x462/0x560 <ffffffff817362a2> <- bad_area <ffffffff81722b0e>)
                                 [User] | faultTest[/media/disk/work/test/a.out] <- startTest[/media/disk/work/test/a.out] <- main[/media/disk/work/test/a.out]
                                          <- __libc_start_main[/lib/x86_64-linux-gnu/libc-2.19.so]
                               [Kernel] | do_page_fault <- page_fault
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about stacks of specific threads when a specific kernel function is called in function mode.    
    
    +  Function Graph Info [-f -e g option]    
        ``````````````````````````````````````````
        # tracer: function_graph
        #
        #     TIME        CPU  DURATION                  FUNCTION CALLS
        #      |          |     |   |                     |   |   |   |
        2868141.800659 |    2)               |  SyS_read() {
        2868141.800660 |    2)   0.090 us    |    fget_light();
        2868141.800660 |    2)               |    vfs_read() {
        2868141.800661 |    2)               |      rw_verify_area() {
        2868141.800661 |    2)               |        security_file_permission() {
        2868141.800661 |    2)               |          apparmor_file_permission() {
        2868141.800661 |    2)               |            common_file_perm() {
        2868141.800661 |    2)   0.077 us    |              aa_file_perm();
        2868141.800662 |    2)   0.373 us    |            }
        2868141.800662 |    2)   0.605 us    |          }
        2868141.800662 |    2)   0.072 us    |          __fsnotify_parent();
        2868141.800662 |    2)   0.069 us    |          fsnotify();
        2868141.800662 |    2)   1.446 us    |        }
        2868141.800662 |    2)   1.727 us    |      }
        2868141.800663 |    2)               |      do_sync_read() {
        2868141.800663 |    2)               |        pipe_read() {
        2868141.800663 |    2)               |          mutex_lock() {
        2868141.800663 |    2)   0.028 us    |            _cond_resched();
        2868141.800663 |    2)   0.366 us    |          }
        2868141.800664 |    2)   0.021 us    |          generic_pipe_buf_confirm();
        2868141.800664 |    2)   0.020 us    |          generic_pipe_buf_map();
        2868141.800664 |    2)   0.364 us    |          pipe_iov_copy_to_user();
        2868141.800665 |    2)   0.027 us    |          generic_pipe_buf_unmap();
        2868141.800665 |    2)   0.066 us    |          anon_pipe_buf_release();
        2868141.800665 |    2)   0.020 us    |          generic_pipe_buf_confirm();
        2868141.800665 |    2)   0.029 us    |          generic_pipe_buf_map();
        2868141.800666 |    2)   0.304 us    |          pipe_iov_copy_to_user();
        2868141.800666 |    2)   0.027 us    |          generic_pipe_buf_unmap();
        2868141.800666 |    2)               |          anon_pipe_buf_release() {
        2868141.800666 |    2)               |            put_page() {
        2868141.800666 |    2)               |              __put_single_page() {
        2868141.800667 |    2)               |                free_hot_cold_page() {
        2868141.800667 |    2)   0.042 us    |                  free_pages_prepare();
        2868141.800667 |    2)   0.086 us    |                  get_pageblock_flags_mask();
        2868141.800667 |    2)               |                  free_pcppages_bulk() {
        2868141.800667 |    2)   0.042 us    |                    _raw_spin_lock();
        2868141.800668 |    2)   0.042 us    |                    get_pageblock_flags_mask();
        2868141.800668 |    2)   0.019 us    |                    __mod_zone_page_state();
        2868141.800668 |    2)   0.043 us    |                    get_pageblock_flags_mask();
        2868141.800668 |    2)   0.020 us    |                    __mod_zone_page_state();
        2868141.800669 |    2)   0.042 us    |                    get_pageblock_flags_mask();
        2868141.800669 |    2)   0.020 us    |                    __mod_zone_page_state();
        2868141.800669 |    2)   0.028 us    |                    get_pageblock_flags_mask();
        2868141.800670 |    2)   0.029 us    |                    __mod_zone_page_state();
        2868141.800670 |    2)   0.029 us    |                    get_pageblock_flags_mask();
        2868141.800670 |    2)   0.028 us    |                    __mod_zone_page_state();
        2868141.800670 |    2)   0.042 us    |                    get_pageblock_flags_mask();
        2868141.800671 |    2)   0.028 us    |                    __mod_zone_page_state();
        2868141.800671 |    2)   0.027 us    |                    get_pageblock_flags_mask();
        2868141.800671 |    2)   0.020 us    |                    __mod_zone_page_state();
        2868141.800671 |    2)   0.043 us    |                    get_pageblock_flags_mask();
        2868141.800671 |    2)   0.029 us    |                    __mod_zone_page_state();
        ``````````````````````````````````````````
    
        This is output about function graph of specific threads in function mode.    
        User can analyze function calls in kernel mode.    
    
    1. Top mode [top option]

      [Top Info] [Time: 2863246.420] [Interval: 1.0] [Ctxt: 1711] [Fork: 0] [IRQ: 898] [Core: 12] [Task: 337/814] [RAM: 64374] [Swap: 65477] [Unit: %/MB/NR]
      ==========================================================================================================================================================
        ID   | CPU (Usr/Ker/Blk/IRQ)| Mem (Free/Anon/File/Slab)| Swap (Used/ InOut )| Reclaim  | BlkRW | NrFlt | NrBlk | NrSIRQ  | NrMlk | NrDrt  |   NetIO    |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      Total  |  0 %( 0 / 0 / 0 / 0 )|60942( -2 / 0  / 0  / 0  )| 481  ( 0  /  0/0  )|   0/0    |  0/0  |   0   |   0   |   125   |  13   |   9    |   3K/2K    |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      Core/0 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/1 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/2 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/3 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/4 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/5 | 17 %(17 / 0 / 0 / 0 )|#################                                                                                    | 1171 Mhz [1171-3418]
      Core/6 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 3418 Mhz [1171-3418]
      Core/7 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/8 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/9 |  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/10|  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      Core/11|  0 %( 0 / 0 / 0 / 0 )|                                                                                                     | 1171 Mhz [1171-3418]
      ==========================================================================================================================================================
          Process      ( ID  / Pid / Nr / Pri)| CPU(Usr/Ker/Dly)|  Mem(RSS/Txt/Shr/Swp)| Blk( RD / WR /NrFlt)| Yld | Prmt | FD | LifeTime|    SignalHandler    |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
              synergyc ( 3602/    1/   3/C  0)|  17( 17/  0/  -)|  285(152/  0/  1/  0)|   0(   -/   -/    0)|    -|     -|  64|795:19:42|      180000200      |
         firewire_ohci (  216/    2/   1/C-20)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    -|     -|  64|795:20:45|                     |
       chromium-browse ( 3715/ 3711/   1/C  0)|   0(  0/  0/  -)|  654(  3/ 40/  0/  7)|   0(   -/   -/    0)|    -|     -| 256|795:19:29|      1800104e8      |
                vsftpd ( 1306/    1/   1/C  0)|   0(  0/  0/  -)|   22(  0/  0/  0/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|        12001        |
              firewire (  215/    2/   1/C-20)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    -|     -|  64|795:20:45|                     |
             scsi_eh_5 (  224/    2/   1/C  0)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    -|     -|  64|795:20:45|                     |
        NetworkManager ( 1147/    1/   4/C  0)|   0(  0/  0/  -)|  335(  4/  1/  3/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:42|      180010000      |
       unity-panel-ser (11628/ 1803/   3/C  0)|   0(  0/  0/  -)|  476( 15/  0/ 10/  0)|   0(   -/   -/    0)|    -|     -|  64|  1:46:31|      180004002      |
       Plex Media Serv ( 2956/ 2931/  16/C  0)|   0(  0/  0/  -)|  436( 40/  9/  9/ 29)|   0(   -/   -/    0)|    -|     -| 128|795:20:34|      1800044ee      |
                 getty ( 1785/    1/   1/C  0)|   0(  0/  0/  -)|   17(  0/  0/  0/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|                     |
       unity-scope-loa ( 8422/ 1803/   4/C  0)|   0(  0/  0/  -)|  652( 23/  0/ 11/  0)|   0(   -/   -/    0)|    -|     -| 128| 22:14:19|      180000000      |
             scsi_eh_0 (  219/    2/   1/C  0)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    -|     -|  64|795:20:45|                     |
       chromium-browse ( 4348/ 3715/  10/C  0)|   0(  0/  0/  -)| 1105( 31/ 40/ 13/ 23)|   0(   -/   -/    0)|    -|     -| 512|795:17:56|      1c0014eed      |
                  bash ( 4027/ 3565/   1/C  0)|   0(  0/  0/  -)|   26(  3/  0/  1/  1)|   0(   -/   -/    0)|    -|     -| 256|795:19: 1|      4b817efb       |
       indicator-datet ( 2097/ 1803/   6/C  0)|   0(  0/  0/  -)| 1133(  4/  0/  3/  1)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|      180000000      |
           gvfsd-trash ( 2658/ 1803/   4/C  0)|   0(  0/  0/  -)|  421(  2/  0/  2/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:39|      180000000      |
       indicator-bluet ( 2091/ 1803/   3/C  0)|   0(  0/  0/  -)|  256(  1/  0/  1/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|      180000000      |
                 osspd ( 1610/    1/  12/C  0)|   0(  0/  0/  -)|  419(  0/  0/  0/  4)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|      180004003      |
       indicator-power ( 2093/ 1803/   3/C  0)|   0(  0/  0/  -)|  270(  2/  0/  2/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|      180000000      |
       upstart-event-b ( 1910/ 1803/   1/C  0)|   0(  0/  0/  -)|   20(  1/  0/  0/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|      180014002      |
       chromium-browse ( 3711/ 3692/   1/C  0)|   0(  0/  0/  -)|  654(  7/ 40/  4/  7)|   0(   -/   -/    0)|    -|     -| 256|795:19:30|      1800104e8      |
       chromium-browse ( 4472/ 3715/  10/C  0)|   0(  0/  0/  -)| 1092( 43/ 40/ 23/ 10)|   0(   -/   -/    0)|    -|     -| 512|795:17:50|      1c0014eed      |
          rtkit-daemon ( 2189/    1/   3/C  1)|   0(  0/  0/  -)|  164(  1/  0/  0/  0)|   0(   -/   -/    0)|    -|     -|  64|795:20:41|      180000000      |
       chromium-browse ( 4306/ 3715/  10/C  0)|   0(  0/  0/  -)| 1310(152/ 40/ 77/ 18)|   0(   -/   -/    0)|    -|     -| 512|795:17:58|      1c0014eed      |
      
      

      This is output about resource usage of processes profiled in top mode.
      User can analyze resource usage of process / thread(-e t option) in real-time.

    +  File Info [top -e f option]    
        ``````````````````````````````````````````
        [Top File Info] [Time: 2863483.030] [Proc: 337] [FD: 5979] [File: 1543] [Unit: %/MB/NR]
        ==========================================================================================================================================================
              PROC       ( ID  / Pid / Nr / Pri)| FD |                                                   PATH                                                    |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         chromium-browse ( 3790/ 3692/   3/C  0)| 285|    FILE: 253   SOCKET: 21   DEVICE: 5   PIPE: 4   EVENT: 2   PROC: 0                                        |
                                                | 383|    /run/shm/.org.chromium.Chromium.nxq6Ay (deleted)                                                         |
                                                | 336|    /run/shm/.org.chromium.Chromium.YW8qa2 (deleted)                                                         |
                                                | 335|    /run/shm/.org.chromium.Chromium.9rkJ0S (deleted)                                                         |
                                                | 332|    /run/shm/.org.chromium.Chromium.OTJc3T (deleted)                                                         |
                                                | 326|    socket:[33361129]                                                                                        |
                                                | 316|    /run/shm/.org.chromium.Chromium.L1Qv1V (deleted)                                                         |
                                                | 305|    /run/shm/.org.chromium.Chromium.rdnIXH (deleted)                                                         |
                                                | 301|    /run/shm/.org.chromium.Chromium.0iBtn5 (deleted)                                                         |
                                                | 298|    /run/shm/.org.chromium.Chromium.x6rmSQ (deleted)                                                         |
                                                | 297|    /run/shm/.org.chromium.Chromium.Lu3ppd (deleted)                                                         |
                                                | 295|    /run/shm/.org.chromium.Chromium.P715y8 (deleted)                                                         |
                                                | 290|    /run/shm/.org.chromium.Chromium.oM0SpM (deleted)                                                         |
                                                | 288|    socket:[33345211]                                                                                        |
                                                | 287|    /run/shm/.org.chromium.Chromium.1NqMr4 (deleted)                                                         |
                                                | 285|    /run/shm/.org.chromium.Chromium.O3VbrY (deleted)                                                         |
                                                | 284|    /run/shm/.org.chromium.Chromium.SDOlTq (deleted)                                                         |
                                                | 283|    /run/shm/.org.chromium.Chromium.zmSpuP (deleted)                                                         |
                                                | 281|    /run/shm/.org.chromium.Chromium.qNsy7E (deleted)                                                         |
                                                | 280|    /run/shm/.org.chromium.Chromium.dkoRM4 (deleted)                                                         |
                                                | 279|    /run/shm/.org.chromium.Chromium.SvUOXX (deleted)                                                         |
                                                | 278|    /run/shm/.org.chromium.Chromium.V3OReG (deleted)                                                         |
                                                | 277|    /run/shm/.org.chromium.Chromium.ZXafZ4 (deleted)                                                         |
                                                | 276|    /run/shm/.org.chromium.Chromium.yFXvT0 (deleted)                                                         |
                                                | 270|    /run/shm/.org.chromium.Chromium.f0pFHz (deleted)                                                         |
                                                | 269|    /run/shm/.org.chromium.Chromium.N98Qhi (deleted)                                                         |
                                                | 268|    /run/shm/.org.chromium.Chromium.Ig99pZ (deleted)                                                         |
                                                | 267|    /run/shm/.org.chromium.Chromium.4fhSXu (deleted)                                                         |
                                                | 266|    socket:[943325]                                                                                          |
        ``````````````````````````````````````````
    
        This is output about file usage of system profiled in top mode.    
        User can analyze file usage of system.    
    
    +  Stack Info [top -e s option]    
        ``````````````````````````````````````````
        [Top Info] [Time: 2870953.430] [Interval: 1.0] [Ctxt: 3121] [Fork: 5] [IRQ: 1633] [Core: 12] [Task: 336/815] [RAM: 64374] [Swap: 65477] [Unit: %/MB/NR]
        ==========================================================================================================================================================
          ID   | CPU (Usr/Ker/Blk/IRQ)| Mem (Free/Anon/File/Slab)| Swap (Used/ InOut )| Reclaim  | BlkRW | NrFlt | NrBlk | NrSIRQ  | NrMlk | NrDrt  |   NetIO    |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        Total  |  1 %( 1 / 0 / 0 / 0 )|60891( -2 / 0  / 0  / 0  )| 481  ( 0  /  0/0  )|   0/0    |  0/0  |   0   |   0   |   184   |  13   |   24   |   8K/1K    |
        ==========================================================================================================================================================
            Process      ( ID  / Pid / Nr / Pri)| CPU(Usr/Ker/Dly)|  Mem(RSS/Txt/Shr/Swp)| Blk( RD / WR /NrFlt)| Yld | Prmt | FD | LifeTime|    SignalHandler    |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                 rcuob/1 (   22/    2/   1/C  0)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64|797:29:13|                     |
                                           100% | rcu_nocb_kthread+0x2a8/0x2e0 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             kworker/3:1 (13295/    2/   1/C  0)|   0(  0/  0/  0)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    2|     0|  64| 65: 6:55|                     |
                                           100% | worker_thread+0x1d1/0x410 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             kworker/0:1 (22466/    2/   1/C  0)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64| 17: 6:53|                     |
                                           100% | worker_thread+0x1d1/0x410 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            kworker/11:0 (17707/    2/   1/C  0)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64| 39:20:14|                     |
                                           100% | worker_thread+0x1d1/0x410 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
           kworker/11:0H (   89/    2/   1/C-20)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64|797:29:13|                     |
                                           100% | worker_thread+0x1d1/0x410 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            ksoftirqd/10 (   82/    2/   1/C  0)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64|797:29:13|                     |
                                           100% | smpboot_thread_fn+0xd3/0x1a0 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             watchdog/10 (   80/    2/   1/F 99)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64|797:29:13|                     |
                                           100% | smpboot_thread_fn+0xd3/0x1a0 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            migration/10 (   81/    2/   1/F 99)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64|797:29:13|                     |
                                           100% | smpboot_thread_fn+0xd3/0x1a0 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            migration/11 (   86/    2/   1/F 99)|   0(  0/  0/  -)|    0(  0/  0/  0/  -)|   0(   -/   -/    0)|    0|     0|  64|797:29:13|                     |
                                           100% | smpboot_thread_fn+0xd3/0x1a0 <- kthread+0xc9/0xe0 <- ret_from_fork+0x58/0x90
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about stacks of threads in top mode.    
        User can analyze stacks of threads.    
    
    +  Memory Info [top -e m option]    
        ``````````````````````````````````````````
        [Top Info] [Time: 2871726.780] [Interval: 1.0] [Ctxt: 3184] [Fork: 0] [IRQ: 1136] [Core: 12] [Task: 349/830] [RAM: 64374] [Swap: 65477] [Unit: %/MB/NR]
           ==========================================================================================================================================================
          ID   | CPU (Usr/Ker/Blk/IRQ)| Mem (Free/Anon/File/Slab)| Swap (Used/ InOut )| Reclaim  | BlkRW | NrFlt | NrBlk | NrSIRQ  | NrMlk | NrDrt  |   NetIO    |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        Total  |  8 %( 8 / 0 / 0 / 0 )|46126(-11 / 0  / 10 / 0  )| 481  ( 0  /  0/0  )|   0/0    |  0/0  |   0   |   0   |   350   |  13   |  3548  |   0K/0K    |
           ==========================================================================================================================================================
            Process      ( ID  / Pid / Nr / Pri)| CPU(Usr/Ker/Dly)|  Mem(RSS/Txt/Shr/Swp)| Blk( RD / WR /NrFlt)| Yld | Prmt | FD | LifeTime|    SignalHandler    |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                    gzip (21266/21265/   1/C  0)|  98( 98/  0/  -)|   11(  0/  0/  0/  0)|   0(   -/   -/    0)|    -|     -|  64|  0: 1:36|      181805001      |
                                     (1)[STACK] | SIZE:   0M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT:  20K / NONE:   0K
                                      (6)[FILE] | SIZE:  10M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT:  68K / NONE:8184K
                                       (2)[ETC] | SIZE:   0M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT:   0K / NONE:   0K
                                      (8)[ANON] | SIZE:   0M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT: 376K / NONE:   0K
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                synergyc ( 3602/    1/   3/C  0)|  16( 16/  0/  -)|  285(152/  0/  1/  0)|   0(   -/   -/    0)|    -|     -|  64|797:41: 2|      180000200      |
                                     (3)[STACK] | SIZE:  16M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT:  28K / NONE:   0K
                                     (20)[FILE] | SIZE:  47M / RSS:   1M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT: 116K / NONE:  35M
                                       (2)[ETC] | SIZE:   0M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT:   0K / NONE:   0K
                                     (13)[ANON] | SIZE: 221M / RSS: 151M / PSS: 151M / SWAP:   0M / HUGE: 150M / LOCK:   0K / SDRT:   0K / PDRT: 151M / NONE:  51M
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                     tar (21265/21262/   1/C  0)|   2(  0/  2/  -)|   25(  1/  0/  1/  0)|   0(   -/   -/    0)|    -|     -|  64|  0: 1:36|      180000000      |
                                     (1)[STACK] | SIZE:   0M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT:  24K / NONE:   0K
                                     (11)[FILE] | SIZE:  24M / RSS:   1M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT: 104K / NONE:  15M
                                       (2)[ETC] | SIZE:   0M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT:   0K / NONE:   0K
                                      (9)[ANON] | SIZE:   0M / RSS:   0M / PSS:   0M / SWAP:   0M / HUGE:   0M / LOCK:   0K / SDRT:   0K / PDRT: 288K / NONE:   0K
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about memory details of processes in top mode.    
        User can analyze memory details of processes.    
    
    +  Summary Info [top -o option]    
        ``````````````````````````````````````````
        [Top Summary Info]
        ==========================================================================================================================================================
         IDX  |          Interval           | CPU(%) | MEM(MB)  | BlkRW(MB) | BlkWait(%) | SWAP(MB) | Rclm(MB) | NrFlt | NrCtxt | NrIRQ  |  NrTask  |   NetIO    |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
            1 |        START -  2871178.270 |      4 |    60463 |    5/0    |          0 |      481 |   0/0    |    17 |   3547 |   1658 |  340/819 |  205K/15K  |
            2 |  2871178.270 -  2871179.270 |      8 |    60451 |    0/0    |          0 |      481 |   0/0    |     0 |   6815 |   1210 |  340/818 |   0K/0K    |
            3 |  2871179.270 -  2871180.280 |      7 |    60439 |    0/0    |          0 |      481 |   0/0    |     0 |   6329 |   1204 |  340/818 |   0K/0K    |
            4 |  2871180.280 -  2871181.280 |     10 |    60429 |    0/0    |          0 |      481 |   0/0    |     0 |   6261 |   1262 |  340/818 |   0K/0K    |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Top CPU Info] [Unit: %]
        ==========================================================================================================================================================
              COMM       ( ID  / Pid / Nr / Pri)| Avg |   1    2    3    4
        ==========================================================================================================================================================
             [CPU]       (  -  /  -  / -  /   -)|   7 |   4    8    7   10
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  Cooker (19737/19733/   2/C  0)|  82 |  27   103  99   101
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                synergyc ( 3602/    1/   3/C  0)|   8 |  18    0    0   17
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                KnottyUI (19733/19732/   1/C  0)|   3 |   0    6    4    4
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  screen ( 3565/    1/   1/C  0)|   0 |   0    1    0    2
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             kworker/2:0 (13550/    2/   1/C  0)|   0 |   0    1    0    0
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  guider (19823/27395/   1/C  0)|   0 |   0    1    0    1
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         Plex DLNA Serve ( 3010/ 2956/  48/C  0)|   0 |   0    1    0    1
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Top VSS Info] [Unit: MB]
        ==========================================================================================================================================================
              COMM       ( ID  / Pid / Nr / Pri)| Max  |   1    2    3    4
        ==========================================================================================================================================================
             [FREE]      (  -  /  -  / -  /   -)| 60463 | 60463604516043960429
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                synergyc ( 3602/    1/   3/C  0)|  285 |  285   0    0   285
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  screen ( 3565/    1/   1/C  0)|   33 |   0   33    0   33
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  Cooker (19737/19733/   2/C  0)|  460 |  425  437  448  460
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             kworker/2:0 (13550/    2/   1/C  0)|    0 |   0    0    0    0
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                KnottyUI (19733/19732/   1/C  0)|  125 |   0   125  125  125
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  guider (19823/27395/   1/C  0)|   41 |   0   41    0   41
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         Plex DLNA Serve ( 3010/ 2956/  48/C  0)| 1286 |   0  1286   0  1286
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Top RSS Info] [Unit: MB]
        ==========================================================================================================================================================
              COMM       ( ID  / Pid / Nr / Pri)| Max  |   1    2    3    4
        ==========================================================================================================================================================
             [FREE]      (  -  /  -  / -  /   -)| 60463 | 60463604516043960429
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                synergyc ( 3602/    1/   3/C  0)|  152 |  152   0    0   152
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  screen ( 3565/    1/   1/C  0)|    6 |   0    6    0    6
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  Cooker (19737/19733/   2/C  0)|  320 |  286  297  309  320
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
             kworker/2:0 (13550/    2/   1/C  0)|    0 |   0    0    0    0
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                KnottyUI (19733/19732/   1/C  0)|   48 |   0   48   48   48
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  guider (19823/27395/   1/C  0)|   15 |   0   14    0   15
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         Plex DLNA Serve ( 3010/ 2956/  48/C  0)|  580 |   0   580   0   580
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Top Block Info] [Unit: %]
        ==========================================================================================================================================================
              COMM       ( ID  / Pid / Nr / Pri)| Sum |   1    2    3    4
        ==========================================================================================================================================================
                  Cooker (19737/19733/   2/C  0)|   4 |   4    0    0    0
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
    
        [Top Memory Details] [Unit: MB]
        ==========================================================================================================================================================
              COMM       ( ID  / Pid ) |   Type   |  Cnt  |  VIRT  |  RSS   |  PSS   |  SWAP  |  HUGE  |  LOCK(KB)  |   PDRT(KB)   |   SDRT(KB)   |   NONE(KB)   |
        ==========================================================================================================================================================
         Plex DLNA Serve ( 3010/ 2956) |  [TOTAL] |   273 |   1285 |    580 |    578 |     98 |    574 |          0 |       590044 |            0 |       107580 |
                                       |    STACK |    50 |    392 |     42 |     42 |      0 |     42 |          0 |        43376 |            0 |            0 |
                                       |      SHM |     2 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     FILE |    63 |    169 |      4 |      2 |      4 |      0 |          0 |          160 |            0 |       107376 |
                                       |      ETC |     2 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     ANON |   156 |    724 |    534 |    534 |     94 |    532 |          0 |       546508 |            0 |          204 |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                  Cooker (19737/19733) |  [TOTAL] |    75 |    470 |    331 |    320 |      0 |      2 |          0 |       318872 |        17840 |       124720 |
                                       |    STACK |     2 |     17 |     15 |     15 |      0 |      0 |          0 |        15448 |           12 |            0 |
                                       |      SHM |     6 |      0 |      0 |      0 |      0 |      0 |          0 |            4 |           12 |            0 |
                                       |     FILE |    32 |     77 |      4 |      1 |      0 |      0 |          0 |          240 |          772 |        59312 |
                                       |      ETC |     2 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     ANON |    33 |    376 |    312 |    304 |      0 |      2 |          0 |       303180 |        17044 |        65408 |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         chromium-browse ( 4263/ 3715) |  [TOTAL] |  1327 |   1666 |    192 |    164 |      8 |     38 |          0 |       161364 |         3460 |       989704 |
                                       |    STACK |    13 |     96 |      0 |      0 |      0 |      0 |          0 |          496 |            0 |            0 |
                                       |      SHM |    27 |     59 |      8 |      6 |      0 |      0 |          0 |         6296 |          152 |            0 |
                                       |     FILE |   204 |    617 |     33 |      7 |      5 |      0 |          0 |          148 |         3064 |       411188 |
                                       |      ETC |     2 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     ANON |  1081 |    894 |    151 |    151 |      3 |     38 |          0 |       154424 |          244 |       578516 |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         chromium-browse ( 4273/ 3715) |  [TOTAL] |   855 |   1371 |    173 |    146 |      6 |     50 |          0 |       145552 |         3456 |       715588 |
                                       |    STACK |    10 |     72 |      0 |      0 |      0 |      0 |          0 |          480 |            0 |            0 |
                                       |      SHM |    25 |     65 |      4 |      3 |      0 |      0 |          0 |         3528 |          148 |            0 |
                                       |     FILE |   204 |    617 |     31 |      5 |      5 |      0 |          0 |          148 |         3064 |       411188 |
                                       |      ETC |     2 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     ANON |   614 |    617 |    138 |    138 |      1 |     50 |          0 |       141396 |          244 |       304400 |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
                synergyc ( 3602/    1) |  [TOTAL] |    38 |    284 |    152 |    151 |      0 |    150 |          0 |       155292 |            0 |        89688 |
                                       |    STACK |     3 |     16 |      0 |      0 |      0 |      0 |          0 |           28 |            0 |            0 |
                                       |      SHM |     0 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     FILE |    20 |     47 |      1 |      0 |      0 |      0 |          0 |          116 |            0 |        36808 |
                                       |      ETC |     2 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     ANON |    13 |    221 |    151 |    151 |      0 |    150 |          0 |       155148 |            0 |        52880 |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
         chromium-browse ( 4306/ 3715) |  [TOTAL] |  1327 |   1308 |    151 |    124 |     18 |     30 |          0 |       123172 |         3460 |       741656 |
                                       |    STACK |    10 |     72 |      0 |      0 |      0 |      0 |          0 |          372 |            0 |            0 |
                                       |      SHM |    30 |     99 |     49 |     48 |      0 |      0 |          0 |        49712 |          152 |            0 |
                                       |     FILE |   204 |    617 |     31 |      5 |      6 |      0 |          0 |          160 |         3064 |       411188 |
                                       |      ETC |     2 |      0 |      0 |      0 |      0 |      0 |          0 |            0 |            0 |            0 |
                                       |     ANON |  1081 |    520 |     71 |     71 |     12 |     30 |          0 |        72928 |          244 |       330468 |
        ----------------------------------------------------------------------------------------------------------------------------------------------------------
        ``````````````````````````````````````````
    
        This is output about summarized resource usage of system profiled in top mode.    
        User can analyze summarized resource usage of system.    
    
    +  Report Info [top -e r -j option]    
        ``````````````````````````````````````````
        {
          "task": {
            "nrThread": 836,
            "nrBlocked": 3,
            "nrCtx": 7466,
            "nrProc": 355
          },
          "mem": {
            "anonDiff": 7,
            "slabDiff": 7,
            "free": 40103,
            "anon": 2843,
            "dirty": 443,
            "file": 19713,
            "freeDiff": -15,
            "total": 64374,
            "slab": 734,
            "fileDiff": 0
          },
          "system": {
            "nrSoftIrq": 603,
            "uptime": 2872587.2,
            "interval": 1.0100000002421439,
            "pid": 22424,
            "nrIrq": 1322
          },
          "event": {},
          "swap": {
            "usage": 481,
            "drReclaim": 0,
            "total": 65477,
            "usageDiff": 0,
            "bgReclaim": 0
          },
          "cpu": {
            "kernel": 2,
            "nrCore": 12,
            "total": 3,
            "user": 1,
            "irq": 0
          },
          "block": {
            "read": 0,
            "ioWait": 3,
            "write": 34,
            "nrFault": 0
          }
        }
        ``````````````````````````````````````````
    
        This is output about report file as json format in top mode.    
        User can analyze resource usage and event of system.    
    
    +  Image Info [top -I guider.out -e g option]    
        ![guider_top_graph](https://cloud.githubusercontent.com/assets/15862689/23285445/a03e0bf0-fa74-11e6-9f5a-872a3f10fe48.png)    
        ![guider_top_chart](https://cloud.githubusercontent.com/assets/15862689/24597375/67f31f22-1880-11e7-8290-64554ed2859c.png)    
        This is a graph about resource usage of system profiled in top mode.    
        User can analyze resource usage of process / thread(with -e t) by graph.    
    
    1. File mode [-F option]

      [File Process Info] [ Process : 347 ] [ RAM: 216496(KB) ][ Keys: Foward/Back/Save/Quit ] [ Capture: Ctrl+\ ]
      ==========================================================================================================================================================
      ____Process_____(_Pid_)|_RAM(KB)_|_____Thread_____(_Tid_) |
      ==========================================================================================================================================================
       chromium-browse( 3692)|  141756 |   LevelDBEnv   (4573 ) |ppapi_crash_upl (4337 ) |SimpleCacheWork (4243 ) |SimpleCacheWork (4242 ) |
                                       |SimpleCacheWork (3928 ) |extension_crash (3843 ) |renderer_crash_ (3794 ) |SimpleCacheWork (3792 ) |
                                       |SimpleCacheWork (3791 ) |gpu-process_cra (3789 ) |BrowserBlocking (3788 ) |BrowserBlocking (3787 ) |
                                       |Chrome_HistoryT (3786 ) |BrowserWatchdog (3740 ) |  AudioThread   (3739 ) |  threaded-ml   (3738 ) |
                                       |CompositorTileW (3737 ) |   IndexedDB    (3736 ) |Chrome_IOThread (3735 ) |Chrome_CacheThr (3734 ) |
                                       |Chrome_ProcessL (3733 ) |Chrome_FileUser (3732 ) |Chrome_FileThre (3731 ) |Chrome_DBThread (3730 ) |
                                       |BrowserBlocking (3729 ) | inotify_reader (3726 ) |CrShutdownDetec (3725 ) |  D-Bus thread  (3724 ) |
                                       |NetworkChangeNo (3723 ) |     gmain      (3719 ) |     gdbus      (3718 ) |  dconf worker  (3717 ) |
                                       |chromium-browse (3716 ) |sandbox_ipc_thr (3709 ) |chromium-browse (3692 ) |BatteryStatusNo (21714) |
                                       |WorkerPool/1966 (19666) |
       chromium-browse( 3892)|  131528 |chromium-browse (3892 ) |
       chromium-browse( 3790)|  131528 |Chrome_ChildIOT (3901 ) |    Watchdog    (3900 ) |chromium-browse (3790 ) |
       chromium-browse( 4472)|  130292 |ScriptStreamerT (4481 ) |HTMLParserThrea (4480 ) |CompositorTileW (4479 ) |CompositorTileW (4478 ) |
                                       |CompositorTileW (4477 ) |CompositorTileW (4476 ) |CompositorTileW (4475 ) |   Compositor   (4474 ) |
                                       |Chrome_ChildIOT (4473 ) |chromium-browse (4472 ) |
       chromium-browse( 4263)|  130284 |DedicatedWorker (8208 ) |ScriptStreamerT (4328 ) |HTMLParserThrea (4284 ) |CompositorTileW (4270 ) |
                                       |CompositorTileW (4269 ) |CompositorTileW (4268 ) |CompositorTileW (4267 ) |CompositorTileW (4266 ) |
                                       |   Compositor   (4265 ) |Chrome_ChildIOT (4264 ) |chromium-browse (4263 ) | FFmpegDemuxer  (19496) |
                                       |     Media      (12691) |
       chromium-browse( 4273)|  130260 |ScriptStreamerT (4282 ) |HTMLParserThrea (4281 ) |CompositorTileW (4280 ) |CompositorTileW (4279 ) |
                                       |CompositorTileW (4278 ) |CompositorTileW (4277 ) |CompositorTileW (4276 ) |   Compositor   (4275 ) |
                                       |Chrome_ChildIOT (4274 ) |chromium-browse (4273 ) |
       chromium-browse( 4393)|  130224 |ScriptStreamerT (4411 ) |HTMLParserThrea (4408 ) |CompositorTileW (4400 ) |CompositorTileW (4399 ) |
                                       |CompositorTileW (4398 ) |CompositorTileW (4397 ) |CompositorTileW (4396 ) |   Compositor   (4395 ) |
                                       |Chrome_ChildIOT (4394 ) |chromium-browse (4393 ) |
       chromium-browse( 4306)|  130180 |ScriptStreamerT (4336 ) |HTMLParserThrea (4322 ) |CompositorTileW (4317 ) |CompositorTileW (4315 ) |
                                       |CompositorTileW (4314 ) |CompositorTileW (4313 ) |CompositorTileW (4312 ) |   Compositor   (4310 ) |
                                       |Chrome_ChildIOT (4307 ) |chromium-browse (4306 ) |
       chromium-browse( 4348)|  129976 |ScriptStreamerT (4379 ) |HTMLParserThrea (4370 ) |CompositorTileW (4355 ) |CompositorTileW (4354 ) |
                                       |CompositorTileW (4353 ) |CompositorTileW (4352 ) |CompositorTileW (4351 ) |   Compositor   (4350 ) |
                                       |Chrome_ChildIOT (4349 ) |chromium-browse (4348 ) |
       chromium-browse( 4507)|  129868 |ScriptStreamerT (4542 ) |HTMLParserThrea (4515 ) |CompositorTileW (4514 ) |CompositorTileW (4513 ) |
                                       |CompositorTileW (4512 ) |CompositorTileW (4511 ) |CompositorTileW (4510 ) |   Compositor   (4509 ) |
                                       |Chrome_ChildIOT (4508 ) |chromium-browse (4507 ) |
       chromium-browse( 4434)|  129844 |ScriptStreamerT (4443 ) |HTMLParserThrea (4442 ) |CompositorTileW (4441 ) |CompositorTileW (4440 ) |
                                       |CompositorTileW (4439 ) |CompositorTileW (4438 ) |CompositorTileW (4437 ) |   Compositor   (4436 ) |
                                       |Chrome_ChildIOT (4435 ) |chromium-browse (4434 ) |
       chromium-browse(22287)|  129764 |HTMLParserThrea (22295) |CompositorTileW (22294) |CompositorTileW (22293) |CompositorTileW (22292) |
                                       |CompositorTileW (22291) |CompositorTileW (22290) |   Compositor   (22289) |Chrome_ChildIOT (22288) |
                                       |chromium-browse (22287) |
      
      [File Usage Info] [ File: 1345 ] [ RAM: 216496(KB) ] [ Keys: Foward/Back/Save/Quit ]
      ==========================================================================================================================================================
      __RAM(KB)___|_File(KB)_|__%___|_____________________________________________________Library & Process_____________________________________________________
      ==========================================================================================================================================================
            19392 |    43180 |   44 | /usr/lib/chromium-browser/chromium-browser [Proc: 17] [Link: 1]
                                    |  chromium-browse ( 3715) |  chromium-browse ( 4273) |  chromium-browse ( 4263) |  chromium-browse ( 3711) |
                                    |  chromium-browse ( 4306) |  chromium-browse ( 3692) |  chromium-browse ( 4289) |  chromium-browse ( 3790) |
                                    |  chromium-browse ( 4393) |  chromium-browse ( 4434) |  chromium-browse ( 4507) |  chromium-browse (22287) |
                                    |  chromium-browse (22277) |  chromium-browse ( 4348) |  chromium-browse ( 3892) |  chromium-browse ( 4472) |
                                    |  chromium-browse (22307) |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
            17360 |    23652 |   73 | /usr/lib/chromium-browser/libs/libwebcore_shared.so [Proc: 17] [Link: 1]
                                    |  chromium-browse ( 3715) |  chromium-browse ( 4273) |  chromium-browse ( 4263) |  chromium-browse ( 3711) |
                                    |  chromium-browse ( 4306) |  chromium-browse ( 3692) |  chromium-browse ( 4289) |  chromium-browse ( 3790) |
                                    |  chromium-browse ( 4393) |  chromium-browse ( 4434) |  chromium-browse ( 4507) |  chromium-browse (22287) |
                                    |  chromium-browse (22277) |  chromium-browse ( 4348) |  chromium-browse ( 3892) |  chromium-browse ( 4472) |
                                    |  chromium-browse (22307) |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
            13132 |    25284 |   51 | /usr/lib/chromium-browser/libs/libcontent.so [Proc: 17] [Link: 1]
                                    |  chromium-browse ( 3715) |  chromium-browse ( 4273) |  chromium-browse ( 4263) |  chromium-browse ( 3711) |
                                    |  chromium-browse ( 4306) |  chromium-browse ( 3692) |  chromium-browse ( 4289) |  chromium-browse ( 3790) |
                                    |  chromium-browse ( 4393) |  chromium-browse ( 4434) |  chromium-browse ( 4507) |  chromium-browse (22287) |
                                    |  chromium-browse (22277) |  chromium-browse ( 4348) |  chromium-browse ( 3892) |  chromium-browse ( 4472) |
                                    |  chromium-browse (22307) |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
             9536 |     9536 |  100 | /usr/lib/chromium-browser/libs/libv8.so [Proc: 17] [Link: 1]
                                    |  chromium-browse ( 3715) |  chromium-browse ( 4273) |  chromium-browse ( 4263) |  chromium-browse ( 3711) |
                                    |  chromium-browse ( 4306) |  chromium-browse ( 3692) |  chromium-browse ( 4289) |  chromium-browse ( 3790) |
                                    |  chromium-browse ( 4393) |  chromium-browse ( 4434) |  chromium-browse ( 4507) |  chromium-browse (22287) |
                                    |  chromium-browse (22277) |  chromium-browse ( 4348) |  chromium-browse ( 3892) |  chromium-browse ( 4472) |
                                    |  chromium-browse (22307) |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
             4172 |     4172 |  100 | /usr/lib/x86_64-linux-gnu/dri/nouveau_dri.so [Proc: 4] [Link: 1]
                                    |  chromium-browse ( 3892) |  chromium-browse ( 3790) |             Xorg ( 1636) |           compiz ( 2571) |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
             3880 |     8548 |   45 | /usr/lib/chromium-browser/libs/libmodules.so [Proc: 17] [Link: 1]
                                    |  chromium-browse ( 3715) |  chromium-browse ( 4273) |  chromium-browse ( 4263) |  chromium-browse ( 3711) |
                                    |  chromium-browse ( 4306) |  chromium-browse ( 3692) |  chromium-browse ( 4289) |  chromium-browse ( 3790) |
                                    |  chromium-browse ( 4393) |  chromium-browse ( 4434) |  chromium-browse ( 4507) |  chromium-browse (22287) |
                                    |  chromium-browse (22277) |  chromium-browse ( 4348) |  chromium-browse ( 3892) |  chromium-browse ( 4472) |
                                    |  chromium-browse (22307) |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
             3780 |     5868 |   64 | /usr/lib/chromium-browser/libs/libnet.so [Proc: 17] [Link: 1]
                                    |  chromium-browse ( 3715) |  chromium-browse ( 4273) |  chromium-browse ( 4263) |  chromium-browse ( 3711) |
                                    |  chromium-browse ( 4306) |  chromium-browse ( 3692) |  chromium-browse ( 4289) |  chromium-browse ( 3790) |
                                    |  chromium-browse ( 4393) |  chromium-browse ( 4434) |  chromium-browse ( 4507) |  chromium-browse (22287) |
                                    |  chromium-browse (22277) |  chromium-browse ( 4348) |  chromium-browse ( 3892) |  chromium-browse ( 4472) |
                                    |  chromium-browse (22307) |
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      

      This is output about file usage of processes profiled in file mode.
      User can analyze file usage (on-memory) of process.

    2. System mode [-y option]

      [System General Info]
      ==========================================================================================================================================================
              TYPE         Information
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      Launch               # ./guider.py record -F . -o . -
      Arch                 x64
      Time                 2017-08-15 04:31:23
      OS                   #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016
      Kernel               Linux 3.13.0-92-generic
      RunningTime          797 hour  35 min
      Load                 192% (1 min)    /   71% (5 min) /   28% (15 min)
      Threads              3/829 (running/total)
      LastPid              20150
      Cmdline              BOOT_IMAGE=/boot/vmlinuz-3.13.0-92-generic.efi.signed root=UUID=54329ccb-dcd9-4835-81c5-e63189905219 ro quiet splash vt.handoff=7
      ==========================================================================================================================================================
      
      [System CPU Info]
      ==========================================================================================================================================================
              TYPE         Information
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      Physical             1
      CoresPerCPU          6
      Logical              12
      Vendor               GenuineIntel
      Model                Intel(R) Core(TM) i7-3970X CPU @ 3.50GHz
      Cache(L2)            15360 KB
      Perf                 7004.23
      Address              46 bits physical, 48 bits virtual
      ==========================================================================================================================================================
      
      [System Memory Info] [ Unit: MB ]
      ==========================================================================================================================================================
      [  DESC]     Memory       Swap     Buffer      Cache     Shared     Mapped     Active   Inactive PageTables       Slab SReclaimable SUnreclaim    Mlocked
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      [ TOTAL]      64374      65477
      [  FREE]      51910      64996
      [USAGE1]      12463        481        144       8392        120        258       1950       9132         60        376        257        119          0
      [USAGE2]      12549        481        144       8392        120        258       2031       9132         59        381        257        124          0
      [  DIFF]         86          0          0         -1         -1          0         80          0         -1          5          0          5          0
      ==========================================================================================================================================================
      
      [System Disk Info] [ Unit: ms/KB ]
      ==========================================================================================================================================================
            Dev         Maj   Min  RdSize RdTime WrSize WrTime FileSystem MountPoint <Option>
      ----------------------------------------------------------------------------------------------------------------------------------------------------------
      /dev/sdb2          8    18     4     296    356   45940     ext3    /media/disk <rw,relatime,data=ordered>
      /dev/sda1          8     1     0      0      0      0       vfat    /boot/efi <rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortnam
                                                                                                                                       =mixed,errors=remount-ro>
      /dev/sda2          8     2     0      0      0      0       ext4    / <rw,relatime,errors=remount-ro,data=ordered>
      ==========================================================================================================================================================
      

      This is output about system information in system mode.
      But this information will be shown in all outputs.

Clone this wiki locally