(Note, if you see > or < replace them with < and > respectively. WordPress messes them up)
Well, I am sitting in a NetApp Cluster Mode 8.1.x, and looking through the performance section. And we had some discussions. I figured it was time to share some of the information I have posted before with graphing with powershell and also more advanced…
In the past, I had used 3D modeling in the Stats package R, to do 3D modeling of the CPU on a san.
Here is a video of what I am talking about:
This was the customer referenced in 2 previous posts, where their DFM and alerting software was using the incorrect counters, and saying their CPU usage was 85-90% CPU utilization.
The moment you stop being a lame, and start loving a stats package like R, your life will because supercalifrigginlishious! R basically kicked matlabs ass, and its freaking free!
What you will need:
- R Statistical package
- R-Studio (the best freaking gui for R there is)
- Hexbin library for R (installed from cran mirrors)
- RGL library for R (installed from cran mirrors)
Depending on your operating system, the requirements for Hexbin and RGL may need other software packages. Possibly OpenGL, possibly other stuff
In R or R-Studio, this is how you install the packages:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
# install.packages("rgl", dependencies = TRUE) Installing package(s) into ‘/Library/Frameworks/R.framework/Versions/2.15/Resources/library’ (as ‘lib’ is unspecified) trying URL 'http://lib.stat.cmu.edu/R/CRAN/bin/macosx/leopard/contrib/2.15/rgl_0.92.892.tgz' Content type 'application/x-gzip' length 7774197 bytes (7.4 Mb) opened URL ================================================== downloaded 7.4 Mb The downloaded binary packages are in /var/folders/jy/fh6_yq116558k8p0lwtbbv180000gn/T//Rtmpl2HvSP/downloaded_packages # install.packages("hexbin", dependencies = TRUE) Installing package(s) into ‘/Library/Frameworks/R.framework/Versions/2.15/Resources/library’ (as ‘lib’ is unspecified) Warning in install.packages : dependencies ‘marray’, ‘affy’, ‘Biobase’ are not available trying URL 'http://lib.stat.cmu.edu/R/CRAN/bin/macosx/leopard/contrib/2.15/hexbin_1.26.0.tgz' Content type 'application/x-gzip' length 785521 bytes (767 Kb) opened URL ================================================== downloaded 767 Kb The downloaded binary packages are in /var/folders/jy/fh6_yq116558k8p0lwtbbv180000gn/T//Rtmpl2HvSP/downloaded_packages # |
Reference my other post on “Graphing Performance Data from NetApp”. You need to gather time and CPU data from DFM/OnCommand Core. This is how you dump out data:
|
1 |
dfm perf data retrieve -o netapp1 -C system:avg_processor_busy -d 36000 -S simple -m max -s 600 |
So, we get some data that looks like this:
|
1 2 3 4 5 6 |
Timestamp netapp1:avg_processor_busy ------------------------------------------------------------------------------- 2012-01-20 06:50:53 0.700 2012-01-20 07:00:53 1.655 2012-01-20 07:10:53 0.690 ... truncated ... |
The problem with this is the dashed lines, and also the characters in the header “netapp1:avg_processor_busy”.
Edit the text file to look like this:
|
1 2 3 4 5 |
Timestamp avg_busy 2012-01-20 06:50:53 0.700 2012-01-20 07:00:53 1.655 2012-01-20 07:10:53 0.690 ... truncated ... |
Now, we go back to R-Studio, and load up the libraries we installed:
|
1 2 3 4 5 |
# library(hexbin) Loading required package: grid Loading required package: lattice # library(rgl) # |
Great! Now the fun begins.
I have a textfile from DFM of stats (in this case 3 months worth with every 10 minutes with a 1 minute rolling max avg cpu)
Load it up!
|
1 |
# netappvf01 |
So what does this mean? Well…
- “netappvf01 <- read” means take the file we are referencing and read it (tab from DFM, but read as whitespace) delimited into the variable netappvf01
- View() shows you a table of the data
- attach(variablename) means that we take the variable above and make that the workset of data that we will be using.
- plot(column1, column2, title) does a simple scatter plot of the data to test that the data is good:
This is not very pretty or useful. We can’t carve it up. Thats why we want to goto 3D!
Because I’m insanely anal… I actually want to use my hourly data as datapoints as well! I am going to go through my text file,and actually turn it into a 3 column data. Edit the file and just add another column in.
|
1 2 3 4 5 |
Timestamp hour avg_busy 2012-01-20 06:50:53 0.700 2012-01-20 07:00:53 1.655 2012-01-20 07:10:53 0.690 ... truncated ... |
I like to use this data as a new variable… netappdr01.3col
|
1 2 3 4 5 6 7 |
# netappdr01.3col <- read.table("/u01/projects/customers/XXXXXXXXX/netappdr01.3col.txt", header=T, quote="\"") # View(netappdr01.3col) # attach(netappdr01.3col) The following object(s) are masked from 'netappvf01': Timestamp, avg_busy (ignore this) # # p <- as.POSIXlt(as.Date(Timestamp)) |
The big change here is we are making a new variable p, and forcing it into a timestamp from the other column data.
|
1 2 3 4 |
---- To 3d plot by day of month. # plot3d(x=p$mday,y=hour,z=avg_busy, xlab="day of month", ylab="hour", zlab="CPU", type="s", col="red", size=2,radius=.5, box=F) ----- To 3d plot by day of week # plot3d(x=p$wday,y=hour,z=avg_busy, xlab="day of week", ylab="hour", zlab="CPU", type="s", col="red", size=2,radius=.5, box=F) |
By looking at the data in a 3d representation, we were able to see that their backup application and SnapMirror schedules conflicting was the root cause of a CPU spike on the SAN, but within acceptable limits.
[asa]0596809158[/asa]
[asa]0387938362[/asa]
[…] never ever ever trust any CPU information because most collect the wrong counter to be useful. ok let’s look deeper. netapp2> priv set diag <—- awwww yeah gettin dirty […]