Home > Articles > Operating Systems, Server

VMware vSphere Deployment and Management

In this chapter, Edward Haletky discusses how many VMware management components communicate between themselves and the virtualization hosts, as well as some of the pitfalls inherent within this communication.
This chapter is from the book

Many people do not consider the deployment and management of virtual machines to be much of a security issue, but it is. Because most, if not all, VM deployment is done over the administrative network on which the VMware ESX Service Console, VMware ESXi Management Appliance, and vCenter Management Server (VC) hosts, it is an important aspect to discuss in the context of virtualization security. There are several threats to the VMware vSphere™ and Virtual Infrastructure that can target the specific management tools, whether they are vCenter Management Server, the Virtual Infrastructure Client (VIC), Lab Manager, or even webAccess. Some vulnerabilities are easier to exploit than others, but they do exist. In addition to the straightforward vulnerabilities, issues exist with authentication, roles and permissions, and access restrictions.

In addition to the normal tools that ship with the VMware ESX, ESXi, and Server hypervisors, we will branch our discussion to include the VMware Stage, Lab, and Life Cycle Managers. At this time we are not pulling into our discussion the VMware Virtual Desktop Manager (VDM) but we will discuss this in Chapter 10, "Virtual Desktop Security."

One of the first things to understand is how data flows among all the management tools within the virtual environment. In some cases, there are settings that will change how data flows, and will discuss those as well.

Management and Deployment Data Flow

Of chief interest when discussing security and management is how the management data flows around the virtual and physical network. Several management clients and tools are in use when we attempt to manage the virtual infrastructure, and they all have their own management methodologies and constraints. We will discuss all the primary VMware management products except the Virtual Desktop Manager and VMware View Manager within this chapter. Chapter 10 discusses the ins and outs of VDI including VDM and VMware View Manager.

All traffic from management clients to either virtualization hosts or VC and back is encrypted using the secure socket layer (SSL) with the exception of the initial handshake done to establish SSL connectivity. SSL allows for end-to-end encryption as defined in Chapter 2, "Holistic View from the Bottom Up." Also, as defined in Chapter 2, SSL is susceptible to a MiTM certificate injection attack. We discuss this further within this chapter.

Most of the mechanisms discussed in the following sections use SSL over port 443, and you may wonder how it can keep everything straight. How does the system know to send data to the SDK versus webAccess versus VIC access? VMware solved this problem with extensive use of reverse proxies based on the entry point into the VC, VMware ESX, VMware ESXi, or VMware Server hosts. Reverse proxies hide the destination port from the client, which also decreases the overall attack surface of exposed ports. Everything appears to tunnel through port 443. However, this does create a series of daemons within VC, VMware ESX, and VMware ESXi that could be listening on external ports yet do not need to do so.

When VMware ESX, ESXi and VC are installed, they create a set of self-signed certificates to enable SSL to be used. These self-signed certificates are based on a root certificate not within any normal browser, so they will generally trigger requests for approvals when they are used. The exception is when ESX speaks to VC; that is approved automatically.

VIC to VC (Including Plug-Ins)

The most commonly used management tool is the Virtual Infrastructure Client (VIC), which can either connect directly to a VMware ESX or ESXi host, or to the VMware VirtualCenter (VC) server. This key management tool behaves differently when connected to VC than the host. When connected to VC, data flows from the VIC to VC and then stops or heads on to the host. Initially the VIC will talk on port 80, switch to port 443 for initial SSL setup, and then be switched to port 902 for all further communication.

Most traffic goes from VIC to VC, then either stops at VC or is sent on to the virtualization host. Those items that are global in nature do not always go on to the host. For example Datacenter, Cluster, Alarms, Tasks, and Permissions do not go on to the host. Yet, subsets of clusters, specifically VMware HA configurations, do go on to the host. Communication between VC and the host uses SSL over port 902.

Of particular interest is the case of using the remote console, because this path goes to VC to retrieve the host on which the VM resides and then goes directly to the host using SSL over port 902. Because the name of the VMware ESX or ESXi host comes from VC, name resolution related errors often occur when using the remote console from a workstation that does not have the DNS entries for the VMware ESX or ESXi host.

The VIC to VC data flow can be summarized as follows:

  • VIC to VC for Overview pages (these do not use SSL).
  • VIC to VC for Datacenter, Cluster, Alarms, Tasks, and Permissions settings using SSL
  • Remote Console to Host setup using SSL
  • VIC to VC to Host for all other actions using SSL

Figure 6.1 presents the data flow in a visual fashion.

Figure 6.1

Figure 6.1 VIC to VC data flow

The easiest way to get a handle on what goes where is to look at the list of permissions within the VIC when connected to VC. Why the permissions list? Because it is a very good breakdown of the functionality allowed for the VIC as Figure 6.2 depicts. Multiple VMware vCenters are shown within the diagram; these are all the same vCenter server. This format makes the diagram easier to understand because you can easily tell which data would be forwarded on to the virtualization hosts and the VMware Update Manager, as well as what would stay within the vCenter server.

Figure 6.2

Figure 6.2 Breakdown of dataflow per permission

The data flow is also affected by the role in use. The read-only role, for example, will not display data that is not already within the VC database. This seems counterintuitive because read-only access can also see real-time performance graphs. This is because VC is gathering the data in the background and storing this data within its own database. This happens regardless of the role currently logged in.

On the host, one important daemon is involved in all this processing, which initially listens on port 443 and then switches to port 902. Specifically, most traffic talks to the vmware-hostd daemon which is the vmware-authd service. Host Health Status information is gathered by VC via the cimserver or Pegasus using SSL port 902. The 5588 and 5589 ports are also open on the virtualization host for health status but are not used by VC or the VIC directly, yet these ports can be used by other tools.

The current VIC does not accept two-factor authentication directly; it relies entirely on the authentication of the workstation from which the VIC is running. There is a way to use this existing authentication as credentials for the VIC to VC connection and not require another challenge response platform, and that is to use the -passthroughAuth option for the VIC to provide a form of single-sign-on to VC.

VIC plug-ins either communicate with VC, go direct to a host, or go to other resources using as many communication paths as there are plug-ins. VC plug-ins bend all the rules about which ports to open and how communication happens over them. The plug-in generally runs within the context of the current credentials within VC and directly on the host. In other words, plug-ins do not necessarily require further authentication. We discuss plug-in issues later within this chapter.

VMware webAccess also does not support two-factor authentication but relies on the workstation from which the system was run to handle this aspect of authentication. VMware webAccess does not have a pass-through authentication mode, yet many single-sign-on tools are available, such as HP's ProtectTools Security Manager, which ships with most HP laptops today.

VIC to Host

The VIC to host data flow is a direct connection to the host over port 80, which is reversed proxy to port 443, which in turn talks SSL and eventually uses port 902 talking SSL for all future communication between the VIC and host. This is similar to how VC connects to the host in the VIC to VC discussion. There is no need to discuss this one in as much detail as we did in the previous discussion because everything talks to the host as shown in Figure 6.3.

Figure 6.3

Figure 6.3 VIC to host data flow

If VC is employed, and changes are made directly on the host, VC will need to query the host for updates to its databases. However, this generally happens in the background. In some rare cases, or in cases where you need the information immediately within a VIC to VC connection, a VIC to VC connection can be forced to refresh its data through a manual refresh link that is on nearly every screen of the VIC. It should be noted that the roles and permissions used for VIC to VC are ignored, and the host specific roles and permissions are now used.

Like the VIC to VC discussion, VIC to Host connectivity also does not have the capability of two-factor authentication. However, the same option discussed previously can provide a single-sign-on capability as long as the host is also participating in a directory service that the workstation is using. We discuss setting up directory services later in this chapter. This implies that single-sign-on will not work for VMware ESXi, because it is not possible to configure a directory service on this type of host. However, there is hope for VMware ESXi administrators with the Hy-Trust security appliance, which sits between your VMware ESX or ESXi hosts to apply additional granular credentials to all access whether from the VIC, VC, or other management tools.

VC webAccess

VC webAccess, shown in Figure 6.4, occurs over port 80 and then uses a reverse proxy to connect to port 8009, which speaks SSL. webAccess does not grant the same capabilities as the VIC with VC version 2.5.x, yet does provide a method to review and control individual VMs. Similar to the VIC to VC, further communication may occur from VC to the host to gather information about VMs over port 902. However, any remote console connection is direct to the host over port 902. webAccess is subject to the roles and permissions set up within VC.

Figure 6.4

Figure 6.4 VC webAccess data flow

With VC webAccess there exists no support for two-factor authentication, yet single-sign-on is possible using Web browser add-ons or hardware devices within your workstation.

ESX(i) webAccess

ESX(i) webAccess works identically to VC webAccess except that there is a direct connection to the host to get information about each VM. Figure 6.5 depicts this data flow, which is subject to all the roles and permissions configured on the ESX(i) host and are unrelated to those configured on VC. ESX(i) webAccess occurs over port 80 and then switches to port 443, where it stays and speaks SSL.

Figure 6.5

Figure 6.5 ESX webAccess data flow

With VMware ESX and ESXi webAccess, no support exists for two-factor authentication, yet single-sign-on is possible using Web browser add-ons or hardware devices within your workstation. One example of such a device is the previously mentioned HP ProtectTools Security Manager.

VI SDK to VC

Similar to the VIC to VC discussion is the VI SDK to VC data flow in that some aspects of the VI SDK will stop at the VC server and others will go directly onto the ESX server using the normal VC to ESX data flow. However, the initial connection using the VI SDK to the VC server is done using the method of VC webAccess except that the target URL is https://virtualcenterserver/sdk instead of just http://virtualcenterserver. Specifically, the SDK will attach to port 443, which talks over SSL and then reverse proxy to port 8086.

The SDK is extremely complex and supports many bindings to its Simple Object Access Protocol (SOAP) interfaces. The Web Service Definition Language (WSDL) interfaces are the key components of the VI SDK. Everything that can be done within the VIC can be duplicated using the VI SDK. Figure 6.6 depicts this data flow, which is subject to all the roles and permissions that VIC to VC must obey.

Figure 6.6

Figure 6.6 VI SDK to VC data flow

However, no mechanism exists to duplicate the remote console within the VI SDK. Yet you can create your remote console links that go directly to the host using the following PERL VI Toolkit code. You can create multiple types of links, as well. In the following code, $rcview has a value of 33, which tells MKS to display the virtual machine's console as well as details such as event logs. A value of 9 tells MKS to display the virtual machine's console as well as the inventory panel. A value of 36 tells MKS to display the virtual machine's console only. A value of 12 tells MKS to add an inventory panel to the display of just the console.1

# Create MKS link partial code
my $rcview = 33;
my $vm_view = Vim::find_entity_views( view_type => 'VirtualMachine' );

for my $vm (@$vm_view) {
      my $name = $vm->name;
      my $moref= $vm->{'mo_ref'}->value;
      my $url =
qq{wsUrl=http://localhost/sdk&vmId=VirtualMachine|${moref}&ui=${rcview}}
;
      my $rcurl = "https://${virtual_center}/ui/vmDirect.do?view=" .
encode_base64($url, q{}) . "_";
      print "$name $rcurl\n";
}

The developer of a VI SDK application should include the capability to use two-factor authentication or single-sign-on because the VI SDK does not provide this feature. Furthermore, the VI SDK does not currently verify the veracity of the SSL certificates in use, other than the typical self-signed check that occurs. Further verification of server certificates is required.

VI SDK to Host

Similar to the VI SDK to VC discussion, the VI SDK to host bypasses VC completely and can perform all the actions that the VIC connected to the host can perform. Figure 6.7 depicts this data flow, which is subject to the roles and permissions set on the host and not those set on VC.

Figure 6.7

Figure 6.7 VI SDK to Host data flow

RCLI to Host

RCLI to Host is a combination of VI SDK to Host and VIC to Host activity defined previously. Some actions use the VI SDK explicitly, whereas others talk over port 902 to complete their tasks, as does the VIC. The RCLI, like the VI SDK, does not have the capability to support two-factor authentication or single-sign-on within its functionality.

RCLI to VC

Similar to RCLI to Host, RCLI to VC makes use of the VI SDK as well as the VIC to VC communication channels. VC to Host communication happens over port 902.

SSH to Host

SSH to Host is specifically for VMware ESX and not generally for VMware ESXi. By default, VMware ESX denies direct root access using SSH, which a very good setting to keep because it maintains the defense in depth available to the GNU/Linux service console environment. To enable SSH on VMware ESXi, you must first break the only security shield VMware ESXi really has, because there is no defense-in-depth capability within VMware ESXi. SSH talks over port 22 by default, but that can be changed as necessary.

SSH has the capability to create preshared keys for communication to enable single-sign-on. In addition, it is possible to implement two-factor authentication using SSH.

Console Access

Console access can be granted while at the keyboard in front of the host or via a remote access card such as the HP Integrated Lights Out (ILO) or Dell Remote Access Card (DRAC) adapters. These provide network access directly to the console as if you were physically at the console. The tools generally talk over port 80 and switch to port 443 for SSL communication via a Web browser. Because a web browser is in use, single-sign-on is supported as described in the VC webAccess section. In addition, most remote access devices also support their own two-factor authentication.

For VMware ESX console, but not for VMware ESXi, it is possible to enable two-factor authentication using PAM modules provided by the vendors of these products like RSA.

Lab Manager

Lab Manager presents to the user its own interface, yet behind the scenes it communicates with VC using the VI SDK to VC path. Lab Manager can have its own authentication methods outside the ones normally used for VC or even ESX.

Site Manager

Site Manager, like Lab Manager, presents to the user its own interface, yet behind the scenes it communicates with VC using the VI SDK to VC path. Site Manager can have its own authentication methods outside the ones normally used for VC or even ESX.

LifeCycle Manager

LifeCycle Manager, like Lab Manager, presents to the user its own interface, yet behind the scenes it communicates with VC using the VI SDK to VC path. LifeCycle Manager can have its own authentication methods outside the ones normally used for VC or even ESX. VC to ESX communication happens over port 902.

AppSpeed

AppSpeed, like Lab Manager, presents to the user its own interface, yet behind the scenes it communicates with VC using the VI SDK to VC path. AppSpeed can have its own authentication methods outside the ones normally used for VC or even ESX.

CapacityIQ

CapacityIQ, like Lab Manager, presents to the user its own interface, yet behind the scenes it communicates with VC using the VI SDK to VC path. CapacityIQ can have its own authentication methods outside the ones normally used for VC or even ESX.

VMware Update Manager

VMware Update Manager (VUM) runs as a service to communicate to the VMware Patch repository on the vmware.com Web site. The update manager will download the patches and provide a local repository to use when patching your VMware ESX and ESXi hosts. VMware Update Manager uses the esxupdate function on each ESX server to connect to port 80, which then reverse proxies to the update manager server on port 8084. On the update manager host, esxupdate pulls down each patch in the necessary order and updates the host, rebooting as necessary. It is also possible to use an Internet facing host to download the VMware Update patches so that your VUM host does not need direct access to the Internet.

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020