kk Blog —— 通用基础


date [-d @int|str] [+%s|"+%F %T"]
netstat -ltunp
sar -n DEV 1

iostat 命令

http://blog.csdn.net/zhangjay/article/details/6656771

http://www.cnblogs.com/mfryf/archive/2012/03/12/2392000.html

iostat用于输出CPU和磁盘I/O相关的统计信息.

命令格式:

1
2
iostat [ -c | -d ] [ -k | -m ] [ -t ] [ -V ] [ -x ] [ device [ ... ] | ALL ] [ -p [ device | ALL ]  ]
       [ interval [ count ] ]

1)iostat的 简单使用

iostat可以显示CPU和I/O系统的负载情况及分区状态信息. 直接执行iostat可以显示下面内容:

1
2
3
4
5
6
7
8
# iostat
Linux 2.6.9-8.11.EVAL (ts3-150.ts.cn.tlan)      08/08/2007

avg-cpu:  %user   %nice    %sys %iowait   %idle
          12.01    0.00        2.15    2.30       83.54

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
hda               7.13       200.12        34.73     640119     111076

各个输出项目的含义如下:

avg-cpu段:

1
2
3
4
5
%user: 在用户级别运行所使用的CPU的百分比.
%nice: nice操作所使用的CPU的百分比.
%sys: 在系统级别(kernel)运行所使用CPU的百分比.
%iowait: CPU等待硬件I/O时,所占用CPU百分比.
%idle: CPU空闲时间的百分比.

Device段:

1
2
3
4
5
tps: 每秒钟发送到的I/O请求数.
Blk_read /s: 每秒读取的block数.
Blk_wrtn/s: 每秒写入的block数.
Blk_read:   读入的block总数.
Blk_wrtn:  写入的block总数.

2)iostat参 数说明

iostat各个参数说明:

1
2
3
4
5
6
7
8
9
10
11
-c 仅显示CPU统计信息.与-d选项互斥.
-d 仅显示磁盘统计信息.与-c选项互斥.
-k 以K为单位显示每秒的磁盘请求数,默认单位块.
-p device | ALL
 与-x选项互斥,用于显示块设备及系统分区的统计信息.也可以在-p后指定一个设备名,如:
 # iostat -p hda
 或显示所有设备
 # iostat -p ALL
-t    在输出数据时,打印搜集数据的时间.
-V    打印版本号和帮助信息.
-x    输出扩展信息.

3)iostat输 出项目说明

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
rrqm/s: 每秒进行 merge 的读操作数目。即 delta(rmerge)/s
wrqm/s: 每秒进行 merge 的写操作数目。即 delta(wmerge)/s
r/s: 每秒完成的读 I/O 设备次数。即 delta(rio)/s
w/s: 每秒完成的写 I/O 设备次数。即 delta(wio)/s
rsec/s: 每秒读扇区数。即 delta(rsect)/s
wsec/s: 每秒写扇区数。即 delta(wsect)/s
rkB/s: 每秒读K字节数。是 rsect/s 的一半,因为每扇区大小为512字节。(需要计算)
wkB/s: 每秒写K字节数。是 wsect/s 的一半。(需要计算)
avgrq-sz: 平均每次设备I/O操作的数据大小 (扇区)。delta(rsect+wsect)/delta(rio+wio)
avgqu-sz: 平均I/O队列长度。即 delta(aveq)/s/1000 (因为aveq的单位为毫秒)。
await: 平均每次设备I/O操作的等待时间 (毫秒)。即 delta(ruse+wuse)/delta(rio+wio)
svctm: 平均每次设备I/O操作的服务时间 (毫秒)。即 delta(use)/delta(rio+wio)
%util: 一秒中有百分之多少的时间用于 I/O 操作,或者说一秒中有多少时间 I/O 队列是非空的。即 delta(use)/s/1000 (因为use的单位为毫秒)
如果 %util 接近 100%,说明产生的I/O请求太多,I/O系统已经满负荷,该磁盘可能存在瓶颈。

Blk_read 读入块的当总数.
Blk_wrtn 写入块的总数.
kB_read/s 每秒从驱动器读入的数据量,单位为K.
kB_wrtn/s 每秒向驱动器写入的数据量,单位为K.
kB_read 读入的数据总量,单位为K.
kB_wrtn 写入的数据总量,单位为K.
rrqm/s 将读入请求合并后,每秒发送到设备的读入请求数.
wrqm/s 将写入请求合并后,每秒发送到设备的写入请求数.
r/s 每秒发送到设备的读入请求数.
w/s 每秒发送到设备的写入请求数.
rsec/s 每秒从设备读入的扇区数.
wsec/s 每秒向设备写入的扇区数.
rkB/s 每秒从设备读入的数据量,单位为K.
wkB/s 每秒向设备写入的数据量,单位为K.
avgrq-sz 发送到设备的请求的平均大小,单位是扇区.
avgqu-sz 发送到设备的请求的平均队列长度.
await I/O请求平均执行时间.包括发送请求和执行的时间.单位是毫秒.
svctm 发送到设备的I/O请求的平均执行时间.单位是毫秒.
%util 在I/O请求发送到设备期间,占用CPU时间的百分比.用于显示设备的带宽利用率.当这个值接近100%时,表示设备带宽已经占满.

4)iostat示 例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# iostat
显示一条统计记录,包括所有的CPU和设备.

# iostat -d 2
每隔2秒,显示一次设备统计信息.

# iostat -d 2 6
每隔2秒,显示一次设备统计信息.总共输出6次.

# iostat -x hda hdb 2 6
每隔2秒显示一次hda,hdb两个设备的扩展统计信息,共输出6次.

# iostat -p sda 2 6
每隔2秒显示一次sda及上面所有分区的统计信息,共输出6次.

taskset 命令

1
2
3
4
5
6
7
8
9
#taskset --help
taskset (util-linux 2.13-pre7)
usage: taskset [options] [mask | cpu-list] [pid | cmd [args...]]
set or get the affinity of a process

-p, --pid operate on existing given pid
-c, --cpu-list display and specify cpus in list format
-h, --help display this help
-v, --version output version information
  • 加-c用的是cpu-id,不加-c用的mask

举例:

1、开启一个只用0标记的cpu核心的新进程(job.sh是你的工作脚本)

1
#taskset -c 0 sh job.sh

2、查找现有的进程号,调整该进程cpu核心使用情况(23328举例用的进程号)

1
2
3
#taskset -pc 0 23328
pid 23328's current affinity list: 0-3  #0-3表示使用所有4核进行处理
pid 23328's new affinity list: 0 #调整后改为仅适用0标记单核处理

ixgbe两个合并包功能

http://downloadmirror.intel.com/22919/eng/README.txt

http://www.360doc.com/content/12/1101/17/9008018_245137867.shtml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
  LRO
  ---
  Large Receive Offload (LRO) is a technique for increasing inbound throughput
  of high-bandwidth network connections by reducing CPU overhead. It works by
  aggregating multiple incoming packets from a single stream into a larger 
  buffer before they are passed higher up the networking stack, thus reducing
  the number of packets that have to be processed. LRO combines multiple 
  Ethernet frames into a single receive in the stack, thereby potentially 
  decreasing CPU utilization for receives. 

  IXGBE_NO_LRO is a compile time flag. The user can enable it at compile
  time to remove support for LRO from the driver. The flag is used by adding 
  CFLAGS_EXTRA="-DIXGBE_NO_LRO" to the make file when it's being compiled. 

	 make CFLAGS_EXTRA="-DIXGBE_NO_LRO" install

  You can verify that the driver is using LRO by looking at these counters in 
  ethtool:

  lro_flushed - the total number of receives using LRO.
  lro_aggregated - counts the total number of Ethernet packets that were combined.

  NOTE: IPv6 and UDP are not supported by LRO.

  HW RSC
  ------
  82599 and X540-based adapters support HW based receive side coalescing (RSC) 
  which can merge multiple frames from the same IPv4 TCP/IP flow into a single
  structure that can span one or more descriptors. It works similarly to SW
  Large receive offload technique. By default HW RSC is enabled and SW LRO 
  cannot be used for 82599 or X540-based adapters unless HW RSC is disabled.
 
  IXGBE_NO_HW_RSC is a compile time flag. The user can enable it at compile 
  time to remove support for HW RSC from the driver. The flag is used by adding 
  CFLAGS_EXTRA="-DIXGBE_NO_HW_RSC" to the make file when it's being compiled.
  
	 make CFLAGS_EXTRA="-DIXGBE_NO_HW_RSC" install
 
  You can verify that the driver is using HW RSC by looking at the counter in 
  ethtool:
 
	 hw_rsc_count - counts the total number of Ethernet packets that were being
	 combined.

	...

max_vfs
-------
Valid Range:   1-63
Default Value: 0

  If the value is greater than 0 it will also force the VMDq parameter to be 1
  or more.

  This parameter adds support for SR-IOV.  It causes the driver to spawn up to 
  max_vfs worth of virtual function.  

  NOTE: When either SR-IOV mode or VMDq mode is enabled, hardware VLAN 
  filtering and VLAN tag stripping/insertion will remain enabled.
  Please remove the old VLAN filter before the new VLAN filter is added.
  For example, 
  
	ip link set eth0 vf 0 vlan 100     // set vlan 100 for VF 0
	ip link set eth0 vf 0 vlan 0       // Delete vlan 100 
	ip link set eth0 vf 0 vlan 200     // set a new vlan 200 for VF 0
  
The parameters for the driver are referenced by position.  So, if you have a 
dual port 82599 or X540-based adapter and you want N virtual functions per 
port, you must specify a number for each port with each parameter separated by
a comma.

For example:
  modprobe ixgbe max_vfs=63,63

NOTE: If both 82598 and 82599 or X540-based adapters are installed on the same 
machine, you must be careful in loading the driver with the parameters. 
Depending on system configuration, number of slots, etc. it's impossible to 
predict in all cases where the positions would be on the command line and the 
user will have to specify zero in those positions occupied by an 82598 port.

With kernel 3.6, the driver supports the simultaneous usage of max_vfs and DCB 
features, subject to the constraints described below. Prior to kernel 3.6, the 
driver did not support the simultaneous operation of max_vfs > 0 and the DCB 
features (multiple traffic classes utilizing Priority Flow Control and Extended 
Transmission Selection).

When DCB is enabled, network traffic is transmitted and received through multiple 
traffic classes (packet buffers in the NIC). The traffic is associated with a 
specific class based on priority, which has a value of 0 through 7 used in the 
VLAN tag. When SR-IOV is not enabled, each traffic class is associated with a set 
of RX/TX descriptor queue pairs. The number of queue pairs for a given traffic 
class depends on the hardware configuration. When SR-IOV is enabled, the descriptor 
queue pairs are grouped into pools. The Physical Function (PF) and each Virtual 
Function (VF) is allocated a pool of RX/TX descriptor queue pairs. When multiple 
traffic classes are configured (for example, DCB is enabled), each pool contains a 
queue pair from each traffic class. When a single traffic class is configured in 
the hardware, the pools contain multiple queue pairs from the single traffic class.

The number of VFs that can be allocated depends on the number of traffic classes 
that can be enabled. The configurable number of traffic classes for each enabled 
VF is as follows:

  0 - 15 VFs = Up to 8 traffic classes, depending on device support

  16 - 31 VFs = Up to 4 traffic classes

  32 - 63 = 1 traffic class 

When VFs are configured, the PF is allocated one pool as well. The PF supports 
the DCB features with the constraint that each traffic class will only use a 
single queue pair. When zero VFs are configured, the PF can support multiple 
queue pairs per traffic class.

如果编译时disable了LRO,但没有disable RSC,可以用 ethtool -C eth2 rx-usecs 0 临时解决,或用 max_vfs=1 ???

https://bugzilla.redhat.com/show_bug.cgi?id=680998

1
2
3
4
5
6
7
8
9
10
11
12

  Chris Wright has this board in hands, here the comment from him:
  > OK, disabling hw RSC with 'ethtool -C eth2 rx-usecs 0' (thanks
  > Herbert!) is bringing this back for me (something like ~1800 Mb/s).
  > This is roughly what booting with max_vfs=1 should have done, so I'm not
  > sure why that didn't work.

  Note that disabling coalescing with ethtool results in better, 
  though still poor performance as would be expected since we're disabling coalescing. 
  The "max_vfs=1" parameter disables RSC as a side-effect and 
  doesn't have the performance hit that disabling interrupt coalescing on the NIC does. 
  In internal testing, "max_vfs=1" results in ~2.5x better performance than using ethtool.