Monitoring Disk Latency

Question

We have quite a few physical machines in our environment, yet we have no trending data over a period of time for disk latency statistics. We can easily run the standard iostat commands to see what the latency is at any particular point in time. But there is no historical data that we can pull from easily.

Are there any recommended tools to monitor disk latency other than screen scraping the output of iostat in a script? We have IOPs graphs, but those are generally not overly useful unless we have latency statistics paired with them.

We have MRTG available currently for trending, and can use Nagios for alerting of a particular threshold being exceeded.

We're generally using RHEL systems on Dell platforms. So if there are any specific easy-to-use tools provided by these providers that I'm unaware of, pointing those out would be beneficial as well.

if you want historical view, you can use graphs on nagios, make plugin from iostat command — klerk, May 18 '14 at 12:25
@vladeli do you know of any pre-built plugins for this scenario? — cheesesticksricepuck, May 18 '14 at 12:59
i use custom script which takes iostat values, you can find lot of scripts for nagios check for this case, also important for grpahs is to have performance data so you can take values and import in graph — klerk, May 18 '14 at 13:35
Are you using a Nagios graphing plugin? If so, which one are you using? — cheesesticksricepuck, May 18 '14 at 13:45
our Nagios environment is custom based, but core feature is nagios, for graphing we use nagiosgraph and rrd also rewrited in perl for this, but for smaller environments you can use pnp4nagios so any plugin which have perf data buit in will autoamtoccaly give you graphs, it is much more easier to configure — klerk, May 18 '14 at 13:52

score 1 · Answer 1 · answered Sep 05 '21 at 11:10

1

sysstat (sar) will capture disk performance stats. It can also generate SVG graphs to give trending views. http://sebastien.godard.pagesperso-orange.fr/features.html#main_features

answered Sep 05 '21 at 11:10

steve

21,582
5
48
75

atop can also be set to catpure at regular intervals, but personally I prefer sar for this stuff – symcbean Jun 05 '23 at 08:49

score 0 · Answer 2 · answered Sep 05 '21 at 10:58

0

munin has nowadays default plugins that can graph block device latency. Do you have specific requirements for the latency monitoring software?

That plugin just reads /proc/diskstats and /sys/block/*/stat and makes the results human-readable.

answered Sep 05 '21 at 10:58

Mikko Rantalainen

3,899
1
25
32

Monitoring Disk Latency

2 Answers2