Automating two-node vSAN cluster setup

In a previous post I described how we are setting up remote offices for a customer with two-node vSAN clusters. I meant to get this post out right after that previous one, but things happened… Anyways, here’s how we automated those two-node vSAN clusters.

Currently we have 7 of these racks ready with more to come. As these will be installed at distant locations we are extra keen on knowing that they are all configured as they should, and that the configuration is the same cross these multiple locations.

Of course, this calls for automation. And with our favorite automation tool PowerCLI we have put together a script to do the vSAN configuration for us.… continue reading

Automating disk limits in vSphere

Following up on my last post on Limiting disk i/o in vSphere we will now see how to automate this.

First off we need some way to identify what limit a specific disk should have.

You can do this in multiple ways, you could use an external source, you can use Tags or as we’ve done, Storage Policies.

All our VMs and VMDKs have a Storage Policy to control the Storage it will be compliant with. We have named our policies so we can identify what kind of limit it should have.
We can now use this to set the limit corresponding to the policy.… continue reading

Working with disk limits in PowerCLI

This will be a post following on the previous one on how to control disk i/o in vSphere

That post showed how you set IOPS limits either through the UI or with PowerCLI.

Even though you set the limit on individual disks you need to work through the VM. And when you retrieve the disk limits for a VM you’ll get back limits on disks identified by the device key and not the label.

To make this a bit easier to work with I’ve created some Powershell functions that wraps the *-VMResourceConfiguration cmdlets and one that gives you the label of the disks when you query the limits for a VM.… continue reading

Limiting disk i/o in vSphere

As a Service provider we need to have some way of limiting individual VMs from utilizing too much of our shared resources.

When it comes to CPU and Memory this is rarely an issue as we try to not over-committing these resources, at least not the Memory. For CPU we closely monitor counters like CPU Ready and Latency to ensure that our VMs will have access to the resources they need.

For storage this can be more difficult. Where we usually have 50-60 VMs on a host we will probably have hundreds on a Storage Array (SAN). Of course the SAN should be spec’ed to handle the IOPS and Throughput you need, but you also need to balance the amount of disk space available and maybe most importantly, the cost.… continue reading

Slides and scripts from VMUG sessions

I had the privilege of delivering 3 sessions at VMUG Norway this week in Oslo, Trondheim and Bergen.

With the extremely nice weather in Norway this week in mind the attendance were great and as always the discussions were valuable.

My session on vSphere Performance monitoring were the short version of the blog series I did about how we built our solution for doing performance monitoring of vSphere with InfluxDB and Grafana, and how we easily can customize with adding metrics and datasources.

The main goal of my session was to demonstrate how easy it is to get started with a project like this and get som actual value.… continue reading

Speaking at VMUG Norway

I’ll be speaking at VMUG Norway’s meetings this May.

As always there will be “three sessions in three cities”.

  • Oslo, May 29th
  • Trondheim, May 30th
  • Bergen, May 31st

The topic for my session will be how we have built our own vSphere Performance monitoring solution which I’ve also done a blog series about.

The VMUG meetings are free, for more information check out https://www.vmug.com/norway. I hope you’re able to join!… continue reading

vSphere Performance data – Monitoring VMware vSAN performance

In my blog series on building a solution for monitoring vSphere Performance we have scripts for pulling VM and Host performance. I did some changes to those recently, mainly by adding some more metrics for instance for VDI hosts.

This post will be about how we included our VSAN environments to the performance monitoring. This has gotten a great deal easier after the Get-VSANStat cmdlet came along in recent versions of PowerCLI.

We will build with the same components as before, a PowerCLI script pulling data and pushing it to an InfluxDB time-series database and finally visualizing it in some Grafana dashboards.… continue reading

Running Grafana on the Red Hat Openshift Container Platform

Last year we started building our own solution for Performance Monitoring of our Infrastructure platform with the focus on the VMware vSphere environment. The components used for this solution is PowerCLI for extracting the metrics, InfluxDB for storing the metrics, and Grafana for presenting the metrics.

I did a Blog series on this project which explains in detail what we did when building the solution.

The solution has been very well received and are used daily by many of my colleagues, and we frequently update the solution with new metrics and dashboards.

In my initial tests before deciding to build the solution with InfluxDB and Grafana I ran these components in Docker containers locally on my laptop.… continue reading

Import-SpbmStoragePolicy error – Object reference not set to an instance of an object

In a previous post I’ve talked about issues in the StoragePolicy and Tag cmdlets in PowerCLI. I found a workaround by ignoring certificate warnings and setting my date format to en-US.

Today I tried to replicate some Storage Policies from one vCenter to another and I found that I got new errors…

I can export the policies without issues, but when I try to Import the policy to the new vCenter I get the following error: “Object reference not set to an instance of an object”. Update 2018-04-06: VMware has confirmed the issue and stated it will be fixed in PowerCLI 10.1

I double-checked my workaround found in the mentioned blog post, but that was in place.… continue reading

More DRS group automation

Following up on my last post on Automating DRS Groups with PowerCLI I found that we also need to automatically remove VMs and Hosts from a given DRS Group.

Although I could have included this in the previous script which creates the groups and adds members I wanted to separate them. There could for instance be times when you would like to run such a script on a different interval than the one that adds members as well as other scenarios. I believe it’s also a good practice to build smaller scripts and functions that have more specific tasks. You could argue that the creation script also could be split up into a part that creates groups and a part that adds the members, and even maybe further splitting Hosts from VMs but that would be a future task.… continue reading