Check the overall health of VMware vSAN

Many information are available from the vSAN cluster pane. VMware has added a dedicated tab for vSAN and some performance counters. In the below screenshot, I show the overall vSAN Health. VMware has included several tests to validate the cluster health such as the hardware compatibility, the network, the physical disk, the cluster and so on.

The hardware compatibility list is downloaded from VMware to validate if vSAN is supported on your hardware. If you take a look at the below screenshot, you can see that my lab is not really supported because my HBA are not referenced by VMware. Regarding the network, several tests are also validated such as the good IP configuration, the MTU, if ping is working and so on. Thanks to this single pane, we are able to check if the cluster is healthy or not.

In the capacity section, you get information about the storage consumption and how the deduplication ratio.

In the same pane you get also a charts which give you the storage usage by object types (before deduplication and compression).

The next pane is useful when a node was down because of an outage or for updates. When you restart a node in vSAN cluster, this last must resync information from its buddy. When the node was down, lot of data were change on the storage and the node must resync these data. This pane indicates which vSAN objects must be resynced to support the chosen RAID level and the FTT (Failure To Tolerate). In case of resync, this pane indicates of many components to resync, the remaining bytes to resync and an estimated time for this process. You can also manage the resync throttling.

In Virtual Objects pane, you can get for each vSAN object the health state. You can check also if the object is compliant with the VM storage policy that you have defined (FTT, RAID Level, Cache pining etc.). Moreovoer, in the physical disk placement tab, you get also the component placement and which are active or not. In my lab, I have a two-node vSAN cluster and I have defined in my storage policy RAID 1 with FTT=1. So for each object, I have three components: two times the data and witness.

In physical disks pane, you can list the physical disks involved in vSAN for each node. You can know also which components are store on which physical disks.

In the proactive tests, you can test a VM creation to validate that everything is working. For example, this test helped me one time to troubleshoot MTU issue between hosts and switches.

vSAN performance counters

Sometime you get poor performance and you expect better. So, you need to find the performance bottleneck. The performance counters can help you to troubleshoot the issue. In performance tab you get the classical performance counters about CPU memory and so on.

VMware has also added two sections dedicated for vSAN performance counters: vSAN – Virtual Machine Consumption and vSAN – Backend. The below screenshot shows you the first section. It is useful because this section indicates you the throughput, the latency and the congestion.

The other section presents performance counters related to backend. You can get the throughput taken by resync job, the IOPS ad latency of vSAN.

The post Monitor and troubleshoot VMware vSAN performance issue appeared first on Tech-Coffee.

S2D is a storage solution and so it is a critical component. The S2D availability can also affect the virtual machines and applications. Therefore, we have to monitor S2D to ensure availability but also performance. When you enable Storage Spaces Direct, a new cluster role is also enabled: the Health Service. This cluster role gathers metrics and alerts of all cluster nodes and provide them from a single pane of glass (an API). This API is accessible from PowerShell, .Net and so on. Even if Health Service is a good idea, it is not usable for day-to-day administration because health service provides real time metrics and no historical. Moreover there is no GUI with health service.

Microsoft has written a management pack for Operations Manager which get information from health service API on a regular basis. In this way, we are able to make chart based on these information. Moreover, SCOM is able to raise alerts regarding S2D state. If you are using SCOM and S2D in your IT, I suggest you to install the Storage Spaces Direct management pack

Requirements

The below requirements come from the management pack documentation. To install the Storage Spaces Direct management pack you need:

• System Center Operations Manager 2016
• A S2D cluster based on Windows Server 2016 Datacenter with KB3216755 (Nano Server not supported)
• Enable agent proxy settings on all S2D nodes
• A working S2D cluster (hyperconverged or disaggregated)

You can download the management pack from this link.

Management pack installation

After you have downloaded and installed the management pack, you get the following files

Then, open an Operations Manager console and navigate in Administration. Right click on Management Packs and select import Management Packs. Then select Add from disk.

If you have Internet on your server, you can select Yes in the following pop-up to resolve dependencies with the online catalog.

In the next Window, Click on Add and select the Storage Spaces Direct management pack files. Then click on Install.

In monitoring pane, you should get a Storage Spaces Direct 2016 “folder” as below. You should also get the following error. It is because the management pack is not yet fully initialized and you have to wait a few minutes.

Operations Manager configuration

First, be sure that on agent proxy is enabled for each S2D nodes. Navigate in Administration | Agent Managed. Then right click on the node and select properties. In Security, be sure that Allow this agent to act as a proxy and discover managed objects on other computers is enabled.

Now I need to create a group for S2D nodes. I’d like this group is dynamic to not populate it manually. To create a group, navigate in Authoring. Right click on groups and select Create a new Group.

Provide a name for this group then select a destination management pack. I have created a dedicated management pack for overrides. I have called this management pack Storage Spaces Direct – Custom.

In the next window, I just click next because I don’t want to provide explicit group members.

Next I create a query for dynamic inclusion. The rule is simple: each server which has an Active Directory DN containing the word Hyper-V is added to the group.

As you can see in the below screen, my S2D nodes are added in a specific OU called Hyper-V. Each time I’ll add a node, the node is moved to this OU, and so my Operations Manager group is populated.

In the next screen of the group wizard, I just click on next.

Then I click again on next because I don’t want to exclude objects from this group.

Then your group is added and should be populated with S2D nodes. Now navigate to Monitoring | Storage | Storage Spaces Direct 2016 | Storage Spaces Direct 2016. Click on the “hamburger” menu on the right and select Add Group.

Then select the group you have just created.

From this point, the configuration is finished. Now you have to wait a long time (I’ve been waiting for 2 or 3 hours) before getting all information.

Monitor S2D

After a few hours, you should get information as below. You can get information about storage sub system, volume, nodes and file shares for disaggregated infrastructure. You can click on each square to get more information.

On the below screenshot, you can get information about volume. They are really valuable because you have the state, the total capacity, IOPS, throughput and so on. Active alerts on volume are also displayed.

Below screenshot shows information about Storage Sub System:

Conclusion

If you are already using Operations Manager 2016 and Storage Spaces Direct, you can easily monitor your storage solution. The management pack is free so I really suggest you to install it. If you are not using Operations Manager, you should find a solution to monitor S2D because the storage layer is a critical component.

The post Monitor S2D with Operations Manager 2016 appeared first on Tech-Coffee.

The health service aggregates monitoring information (fault and metrics) of all nodes in the cluster. These information are available from a single point and can be used by PowerShell or across API. The Health Service can raise alerts in real-time regarding event in the cluster. These alerts contain the severity, the description, the recommended action and the location information related to the fault domain. Health Service raises alerts for several faults as you can see below:

The rollup monitors can help to find a root cause of a fault. For example, in order that the server monitor is healthy, all underlying monitor must be also healthy. If an underlying monitor is not healthy, the parent monitor shows an alert.

In the above example, a drive is down in a node. So, the Health Service raises an alert for the drive and the parent node monitor is in error state.

In the next version, the Health Service will be smart. The cluster monitor will be “only” in warning state because the cluster still has enough node to run the service and after all, a single drive down is not a severe issue for the service. This feature should be called severity masking

The Health Service is also able to gather metrics about the cluster such as IOPS, capacity, CPU usage and so on.

Use Cluster Health Service

Show Metrics

To show the metrics gathered by the Health Service, run the cmdlet Get-StorageHealthReport as below:

Get-StorageSubSystem *Cluster* | Get-StorageHealthReport

As you can see, you have consolidated information as the memory available, the IOPS, the capacity, the average CPU usage and so on. We can imagine a tool that gather information from the API several times per minute to show charts or pies with these information.

Show Alerts

To get current alerts in the cluster, run the following cmdlet:

Get-StorageSubSystem *Cluster* | Debug-StorageSubSystem

To show you screenshot, I run cmdlet against my lab Storage Spaces Direct cluster which is not best practices. The following alert is raised because I have not enough reserve capacity:

Then I stop a node in my cluster:

I have several issues in my cluster! The Health Service has detected that the node is done and that some cables are disconnected. It is because my Mellanox adapters are direct attached to the other node.

SCOM Dashboard

This dashboard is not yet available at the time of writing but in the future, Microsoft should releaser the below SCOM dashboard which leverage the Cluster Health Service.

Another example: DataOn Must

DataOn is a company that provides hardware which are compliants with Storage Spaces (Direct). DataOn has also released dashboards called DataOn Must which are based on Health Service. DataOn Must is currently only available when you buy DataOn hardware. Thanks to Health Service API, we can have fancy and readable charts and pies about the health of the Storage Spaces Direct Cluster.

I would like thanks Cosmos Darwin for the topic review and to have left me the opportunity to talk about severity masking.

The post Cluster Health Service in Windows Server 2016 appeared first on Tech-Coffee.

Requirements

• An operational System Center Virtual Machine Manager up to date;
• An operational System Center Operations Manager up to date;
• (Optional) An operational SCOM reporting service (SSRS) with a SQL Server Analysis Service (SSAS) (the SSRS and SSAS must have the same instance name). The SSRS must allow report access on port 80 (HTTP).

Install the Operations Manager console on VMM servers

The first step is to install the Operations Manager console on all VMM servers. So if you have several VMM node members of a failover cluster, you have to install the Operations Manager console on each node. So mount the SCOM ISO and install the console:

Once the Operations Manager console is installed, you should update it to the same rollup update than your management server.

Import required management packs

First download these management packs:

• Management Pack for SQL Server;
• Management pack for Windows Server Operating System.

Next extract and import these management packs from SCOM console:

Add them from disk.

Once you have selected all files from both management packs, just click on import.

Repeat the above procedure but this time instead of adding management pack from disk, select add from the catalog. Search the below management packs:

• Windows Server Internet Information Services Library;
• Windows Server Internet Information Services 2003;
• Windows Server 2008 Internet Information Services 7;

Once you have found them, you can import them.

Credentials

To connect Virtual Machine Manager to SCOM, we need an account in the Administrator user role from Virtual Machine Manager. On my side I have added the account Home\sa-omg-mgtact to VMM administrator user role.

Next in Operations Manager, I add the group Home\GG-VMMADM to the Operations Manager Administrators user role. This group contains a service account used by VMM.

Connect Virtual Machine Manager to SCOM

In Virtual Machine Manager console, navigate to settings and System Center Settings. Then right click on Operation Manager Server and click on properties.

First screen details requirements to connect Virtual Machine Manager to Operations Manager. Verify that all is ok and click on next.

Next specify the server name of a management server and select the Run As account that has been added to Operations Manager Administrators user role.

Then specify an Operations Manager account that will be used to connect to Virtual Machine Manager. If you have not already added it to VMM Administrators user role, the wizard will add it for you.

Once Virtual Machine Manager and Operations Manager are connected, you can right click on Operations Manager settings and properties again. The connection status should indicate OK.

SQL Server Analysis Services

First connect to your Analysis Service and add a VMM Run As account to the Server Administrators:

Next you have to install Microsoft® SQL Server® 2012 Analysis Management Objects on each VMM server node.

Come back to the Virtual Machine Manager console and edit properties of Operation Manager Settings. Next select SQL Server Analysis Services. Specify the server that hosts the SCOM Reporting Service and the Analysis Service. Then specify the SSAS instance name, the port (0 means default port) and the Run As account with administrative privilege on SSAS instance.

Be careful, the instance name between the Analysis Service and the Reporting Service must be the same. Moreover the Reporting Service must allow report access on port 80 by HTTP. For further information you can read this TechNet topic.

Monitor Virtual Machine Manager

First thing enabled from Virtual Machine Manager is the Performances and Resources Optimization (PRO). It indicates issues on Hyper-V hosts or Virtual Machines as below.

Next if you navigate in the SCOM console you should have three new folders:

First a diagram view is available that shows what is monitored in the virtualized infrastructure.

Next many monitors are available to check the health of components managed by Virtual Machine Manager. Below you have three examples: the clouds, the hosts and the virtual machine health.

Some performance counters are also available. Below this is an example of performance counters about Hyper-V host free memory and network packets received by a virtual machine.

Import management pack for VMM host and Virtual Machine dashboards

In May 2014, new dashboards were available for Virtual Machine Manager. These dashboards look better than the others. So why not install them? J

Before importing this management pack, make sure that your management server is updated to at least the rollup update 2. Moreover this management pack requires Component Library Management Pack. You can import this management pack from this path:

<Install Dir>\Microsoft System Center 2012 R2\Operations Manager\Server\Management Packs for Update Rollups

Once you have imported this management pack, you can download the management pack for Virtual Machine Manager host and Virtual Machine dashboards here. Next extract it and import it from the SCOM console.

Once the management pack is imported, you should have two new dashboards:

Below this is the VMM Host dashboard J.

The post Monitor Virtual Machine Manager from SCOM appeared first on Tech-Coffee.

• Import DPM management packs in SCOM;
• Install central console on Management server;
• (Optional) Install central console on your console servers.

Import DPM management packs in SCOM

First of all, mount Data Protection Manager ISO on your SCOM management server. Then open your Operation Manager console and navigate to Administration tab. Right click on Management packs and select import Management Packs.

Next, click on Add and select Add from disk.

Navigate to SCDPM\ManagementPacks\<Your Language> in the DPM DVD drive. Select the two management packs and click on Open.

You will have a security warning about these management packs (MP). Because these MPs come from Microsoft, I trust them and so I click on Install.

The next step is the installation of the central console on the SCOM operation manager.

Install central console on Management server

Open the DPM DVD drive and launch the setup.exe. Next click on DPM Central Console as below. If you have not installed the Microsoft Visual C++ 2008 redistributable package you will have an error. But you are lucky because you just have to navigate to SCDPM\Redist\vcredist in
DPM DVD drive and run vcredist2008_x64.exe to solve the problem.

Next accecpt the license terms and conditions and click OK.

If you have installed a SCOM console on your management server, you have to install Central Console server-side and client-side Components. If you have not, just install Central Console server-side Components.

On the prerequisites check screen, click on Next if all is validated.

On installation settings screen click on next.

Because I update my systems with SCCM, I do not want to use Microsoft Update.

So it is ok, now DPM is monitored by SCOM.

Install central console on your console servers

If you are like me and you don’t like to manage servers from themselves, you should have a Remote Desktop server where you have installed all MMC and System Center consoles. In this case, you have to install the central console on this server also. So follow the process of the last part and on Central Console Opt-in screen, just Install Central Console client-side Components as below:

Monitor Data Protection Manager with SCOM

To view the result of the integration of DPM monitoring in SCOM, open an Operation Manager console. Open Monitoring tab. You should have the same tree as below:

So as you can see, it is possible to monitor the health of the disks, DPM servers, protected servers, protection groups etc. Moreover you can monitor the data sources state in function of the workload (SQL server, File System, Hyper-V etc.). It is very useful when you have multiple DPM servers. On the below capture, it is the disk monitoring:

And now the DPM servers status :

Protection groups health :

Unhealthy data sources :

The post Monitor Data Protection Manager with SCOM appeared first on Tech-Coffee.