Some InfluxDB gotcha’s

If you’ve followed my vSphere performance data blog series you probably have noted that I used InfluxDB as the database for storing the performance data.

With over 4 months of performance data in the InfluxDB I’ve picked up some gotcha’s along the way (there’s probably more lying around which I’ve not come over yet).

In this blog post I’ll outline what I’ve learned so far

(Save) Disk space

One of them is of course, and this is an obvious one, the amount of data and the corresponding disk space needed to store it.

Although InfluxDB is very good at compressing your data and saving a lot of space doing that it depends on you to do the right choices on how you write your data.… continue reading

vSphere Performance data – Part 8 – Wrap-up and next steps

This is Part 8 and last part (I think…) of my series on vSphere Performance data.

Part 1 discusses the project, Part 2 is about exploring how to retrieve data, Part 3 is about using Get-Stat for the retrieval. Part 4 talked about the database used to store the retrieved data, InfluxDB. Part 5 showed how data is written to the database. Part 6 was about creating dashboards to show off the data. Part 7 added more data to the project. This part will try to wrap up and look at some future steps.

When I started my project I did it with a clear picture on how and what software I would use.… continue reading

vSphere Performance data – Part 7 – More data

This is Part 7 of my series on vSphere Performance data.

Part 1 discusses the project, Part 2 is about exploring how to retrieve data, Part 3 is about using Get-Stat for the retrieval. Part 4 talked about the database used to store the retrieved data, InfluxDB. Part 5 showed how data is written to the database. Part 6 was about creating dashboards to show off the data. This post adds even more data to the project!

One thing I’ve learned by this project is that when you gather data you are always looking out for other data sources to add!… continue reading

vSphere Performance data – Part 6 – The Dashboard(s)

This is Part 6 of my series on vSphere Performance data.

Part 1 discusses the project, Part 2 is about exploring how to retrieve data, Part 3 is about using Get-Stat for the retrieval. Part 4 talked about the database used to store the retrieved data, InfluxDB. Part 5 showed how data is written to the database. This post will show some results of our work!

As I talked about in Part 1 I had decided to go with Grafana as the front-end. Grafana is an open source software for time series analytics which can make use of several datasources, making it the perfect match for the data in my InfluxDB.… continue reading

vSphere Performance data – Part 5 – The script

This is Part 5 of my series on vSphere Performance data.

Part 1 discusses the project, Part 2 is about exploring how to retrieve data, Part 3 is about using Get-Stat for the retrieval. Part 4 talked about the database used to store the retrieved data, InfluxDB. This one will do some actual work and will retrieve data from vCenter and write it to the database.

The last post showed how to write some data to the performance database InfluxDB through its API. As Powershell is good at interacting with APIs this is what I will use for writing the data.… continue reading

vSphere Performance data – Part 4 – InfluxDB

This is Part 4 of my series on vSphere Performance data.

Part 1 discusses the project, Part 2 is about exploring how to retrieve data, Part 3 is about using Get-Stat for the retrieval. This post will be about the database used to store the retrieved data, InfluxDB.

The last post left of with the beginning of a script that had retrieved data from vCenter. Before I can finish that script I need to have somewhere to put that data. As I discussed in Part 1 I had decided to use InfluxDB for this purpose.

InfluxDB is a time-series database built for storing large amounts of timestamped data.… continue reading

vSphere Performance data – Part 3 – Get-stat

This is the Part 3 in my series on vSphere performance data.

Part 1 discussed the project, Part 2 was about checking the methods of retrieving data and ended with me realizing I would use Get-Stat against all (4000) VMs to retrieve data.
Part 2 was posted over a month ago as I have been busy preparing for the VCP 6.5 DCV exam (which I passed btw) as well as upgrading/migrating our vCenter servers, but I have actually been able to do a lot of work on this project as well so there will be some updates in the next couple of days.… continue reading

vSphere Performance Data – Part 2 – Retrieve data

This is part 2 of my vSphere Performance Data series. Part 1 described the project and my goals with the project.

This post will be my thoughts on retrieving performance data from our vSphere environment. As I described in part 1 our environment consist of 100+ hosts and 4000+ VMs. These are hosted in 3 different vCenters in the same SSO domain (Enhanced Linked Mode). All hosts and vCenters are at version 6. We are in the process of upgrading vCenter to 6.5 as I’ve talked about in a previous post.

Currently we are using Turbonomic in our environment which also retrieves performance data from the vCenters that it uses in it’s main purpose which is to balance the load in our environment.… continue reading

vSphere performance data – Part 1

There is lots of posts on retrieving performance data from your vSphere environment (I’ll probably use a lot of them in this series), but here’s my take on it.

My ultimate goal is to build my own database of performance data and have a nice front-end presenting this. I also want to have an API that extracts data from the performance DB which I will use in our in-house portals and dashboards.

This project can be quite big and complex so I will split it in 3 parts:

  • Retrieve performance data from vSphere
  • Push data to a database and set up a front-end
  • Build an API on top of the perf db

These will not necessarily correspond to the parts in this blog series as I suspect the posts would be quite long and the different parts will take some time to complete.… continue reading