Grafana dashboard

Monitoring FreeNAS with InfluxDB and Grafana

At work I have done some monitoring projects which I’ve done many blog posts about. At home I have a small vSphere environment serving partially as a Lab but it also runs some services we use at home. Of course I do monitoring of this environment as well, and I use both InfluxDB and Grafana as we do at work.

One of my VMs runs Plex Media Server and recently I moved my media library to a separate box running FreeNAS. I’ve used FreeNAS as a part of my lab earlier as an ISCSI target and serving storage for VMs, but it’s now only serving my media files to the Plex VM.

FreeNAS monitoring

FreeNAS builtin monitor
FreeNAS builtin monitor

The FreeNAS has it’s own performance monitoring available through the Web GUI, but of course I have wanted to incorporate it in my own monitoring solution. I’m not very familiar with the FreeBSD OS which FreeNAS runs on, and I wasn’t very keen on installing any agents on it. continue reading

vSphere Performance data – Monitoring VMware vSAN performance

In my blog series on building a solution for monitoring vSphere Performance we have scripts for pulling VM and Host performance. I did some changes to those recently, mainly by adding some more metrics for instance for VDI hosts.

This post will be about how we included our VSAN environments to the performance monitoring. This has gotten a great deal easier after the Get-VSANStat cmdlet came along in recent versions of PowerCLI.

We will build with the same components as before, a PowerCLI script pulling data and pushing it to an InfluxDB time-series database and finally visualizing it in some Grafana dashboards. continue reading

Chronograf dashboard

Exploring the TICK stack

For those of you that have read my blog you probably know I’ve done a series on performance monitoring infrastructure with the help of InfluxDB.

InfluxDB is a part of the TICK stack delivered by InfluxData. All components are open-sourced and available. The TICK stack consists of, Telegraf, InfluxDB, Chronograf and Kapacitor.

This post will do a quick review and some examples on how I have started exploring them in my Performance monitoring project. continue reading

vSphere Performance data – Part 8 – Wrap-up and next steps

This is Part 8 and last part (I think…) of my series on vSphere Performance data.

Part 1 discusses the project, Part 2 is about exploring how to retrieve data, Part 3 is about using Get-Stat for the retrieval. Part 4 talked about the database used to store the retrieved data, InfluxDB. Part 5 showed how data is written to the database. Part 6 was about creating dashboards to show off the data. Part 7 added more data to the project. This part will try to wrap up and look at some future steps.

When I started my project I did it with a clear picture on how and what software I would use. Therefore I didn’t look around much for if and how others had done it. After a while I did find that (of course) several others have done similar projects with vSphere performance data, InfluxDB and Grafana. continue reading

vSphere Performance data – Part 7 – More data

This is Part 7 of my series on vSphere Performance data.

Part 1 discusses the project, Part 2 is about exploring how to retrieve data, Part 3 is about using Get-Stat for the retrieval. Part 4 talked about the database used to store the retrieved data, InfluxDB. Part 5 showed how data is written to the database. Part 6 was about creating dashboards to show off the data. This post adds even more data to the project!

One thing I’ve learned by this project is that when you gather data you are always looking out for other data sources to add! Until now we’ve seen how we have retrieved metrics from all VMs in our environment on 20 second intervals. But what about the ESXi hosts running those VMs? And what about the Storage arrays they use? continue reading

vSphere Performance data – Part 6 – The Dashboard(s)

This is Part 6 of my series on vSphere Performance data.

Part 1 discusses the project, Part 2 is about exploring how to retrieve data, Part 3 is about using Get-Stat for the retrieval. Part 4 talked about the database used to store the retrieved data, InfluxDB. Part 5 showed how data is written to the database. This post will show some results of our work!

As I talked about in Part 1 I had decided to go with Grafana as the front-end. Grafana is an open source software for time series analytics which can make use of several datasources, making it the perfect match for the data in my InfluxDB. continue reading

vSphere Performance data – Part 5 – The script

This is Part 5 of my series on vSphere Performance data.

Part 1 discusses the project, Part 2 is about exploring how to retrieve data, Part 3 is about using Get-Stat for the retrieval. Part 4 talked about the database used to store the retrieved data, InfluxDB. This one will do some actual work and will retrieve data from vCenter and write it to the database.

The last post showed how to write some data to the performance database InfluxDB through its API. As Powershell is good at interacting with APIs this is what I will use for writing the data. continue reading

vSphere Performance data – Part 4 – InfluxDB

This is Part 4 of my series on vSphere Performance data.

Part 1 discusses the project, Part 2 is about exploring how to retrieve data, Part 3 is about using Get-Stat for the retrieval. This post will be about the database used to store the retrieved data, InfluxDB.

The last post left of with the beginning of a script that had retrieved data from vCenter. Before I can finish that script I need to have somewhere to put that data. As I discussed in Part 1 I had decided to use InfluxDB for this purpose. continue reading

vSphere Performance data – Part 3 – Get-stat

This is the Part 3 in my series on vSphere performance data.

Part 1 discussed the project, Part 2 was about checking the methods of retrieving data and ended with me realizing I would use Get-Stat against all (4000) VMs to retrieve data.
Part 2 was posted over a month ago as I have been busy preparing for the VCP 6.5 DCV exam (which I passed btw) as well as upgrading/migrating our vCenter servers, but I have actually been able to do a lot of work on this project as well so there will be some updates in the next couple of days.

Previously I had done some benchmarks on retrieving data from VMs using PowerCLI and the Get-Stat cmdlet. I would land on roughly 1 second per VM to retrieve and process the metrics I wanted. As I discussed in part 2 that would result in 4000 seconds to retrieve the data I needed, and with my goal to retrieve all 20 sec metrics within 5 minutes I would need to have around 14 scripts running simultaneously to achieve this. continue reading