Home > Articles > Operating Systems, Server

The Xen Hypervisor

This chapter is from the book

Xen Boot Options

The Xen hypervisor and Domain0 actually gain their privileged position in the system during boot. At system boot time, a bootloader such as GRUB—the GNU GRand Unified Bootloader, is used to choose the software that will control the machine. Because it is the first software "introduced" to the hardware, the Xen hypervisor now has the power to grant control to or withhold control from any other software.

The Xen hypervisor is inserted in the GRUB menu and boots first. The hypervisor then starts Domain0. If other guest domains have been previously created, Domain0 can automatically start them by consulting configuration files that specify the proper settings. These configuration files are discussed in more detail in Chapter 6, "Managing Unprivileged Domains."

Listing 3.2 shows some excerpts from the GRUB configuration file on the LiveCD used in Chapter 2. On the Xen LiveCD, it is /boot/grub/menu.lst, but on some systems it may be in other locations such as /boot/grub/grub.conf or /etc/grub.conf. For the Debian, CentOS, and OpenSUSE Domain0 sections, notice that the same kernel and modules are loaded (/boot/xen-3.0.3.0.gz, /boot/vmlinuz-2.6.16.29-xen, and /boot/initrd-2.6.16-xen.img). This is the equivalent of booting the Xen hypervisor. However, the image passed in the first module line differs. For the Debian Domain0, it is rootfs.img; for the CentOS Domain0, it is centos.img; and for the OpenSUSE Domain0, it is opensuse.img. Also notice that in the last section for the Native Non-Xen kernel, the Xen hypervisor is not booted. Instead, the non-Xen kernel and modules, /boot/vmlinuz-2.6.16-2-686-smp and /boot/initrd-2.6.16-2-686-smp.img, are specified. Of course, in all cases, the kernel and modules are coming from the CD as the root rather than the hard drive.

Listing 3.2. Excerpt from the GRUB Configuration File on the Xen LiveCD

terminal console
timeout  10
default  0

title  Debian-based Dom0 (from testing)
root   (cd)
kernel /boot/xen-3.0.3.0.gz watchdog
module /boot/vmlinuz-2.6.16.29-xen  ro selinux=0
       ramdisk_size=32758 image=rootfs.img boot=cow quiet
module /boot/initrd-2.6.16-xen.img

title  CentOS-4.1-based Dom0
root   (cd)
kernel /boot/xen-3.0.3.0.gz watchdog
module /boot/vmlinuz-2.6.16.29-xen  ro selinux=0
       ramdisk_size=32758 image=centos.img boot=cow
       hotplug=/sbin/hotplug quiet
module /boot/initrd-2.6.16-xen.img

title  OpenSUSE-10.0-based Dom0
root   (cd)
kernel /boot/xen-3.0.3.0.gz watchdog
module /boot/vmlinuz-2.6.16.29-xen  ro selinux=0
       ramdisk_size=32758 image=opensuse.img boot=cow mkdevfd
       quiet
module /boot/initrd-2.6.16-xen.img

title  Debian on Native Kernel 2.6.16 (from testing)
       (text mode only)
root   (cd)
kernel /boot/vmlinuz-2.6.16-2-686-smp ro 2 selinux=0
       ramdisk_size=32758 image=rootfs.img boot=cow
initrd /boot/initrd-2.6.16-2-686-smp.img

Table 3.2 describes a number of additional options that can be used to configure Xen's behavior at boot. They are added directly to the Xen command line in the GRUB configuration file. One of the most important options is how much memory to allocate to Domain0. Domain0 specifies to the hypervisor how much memory is allocated to all other guest domains, but the Xen hypervisor can be told the amount of memory to allocate to Domain0 directly. Because Domain0 should be simple (running the Xen daemon and not much else), it shouldn't need as much memory as a normal guest domain would. Chapter 4, "Hardware Requirements and Installation of Xen Domain0," explains the installation and configuration of Domain0 in detail.

Table 3.2. Xen Boot Options

Option

Category

Description

dom0_mem=X

Standard Configuration

Amount of memory to be allocated to Domain0.

Example: dom0_mem=64M

Units can be specified as B for bytes, K for kilobytes, M for megabytes, or G for gigabytes.

If no units are specified, kilobytes is assumed.

lapic

Advanced Configuration

Forces use of the local APIC even if disabled by the uniprocessor BIOS.

nolapic

Advanced Configuration

Do not use the local APIC in a uniprocessor system even if enabled by the BIOS.

apic=default

Advanced Configuration

Specifies the APIC platform. This is usually determined automatically.

apic=bigsmp
apic=es7000
apic=summit
boot option"
apic=es7000
apic=summit"
apic=es7000
apic=summit
console=vga
console=com1
console=com2H
console=com2L

Advanced Configuration

Specify where Xen console I/O should be sent. More than one destination may be chosen.

For vga, use regular console for output and keyboard for input.

Default is console=com1,vga

For com1, use serial port com1. com2H and com2L allow a single port to be used for both console and debugging. For com2H, characters must have most significant bit (MSB) set. For com2L, characters must have MSB cleared.

conswitch=Cx
conswitch=C

Advanced Configuration

Used to specify a different character sequence to indicate that serial-console input should be switched between Xen and Domain0. C specifies the "switch character." To actually switch, press Ctrl+C three times for the specified C.

Example: conswitch=a

The default switch character is a.

conswitch=ax

This also controls whether input is automatically switched to Domain0 when Domain0 boots. If the switch character is followed by an x, autoswitching is disabled. Any other value or omitting the x entirely enables autoswitching.

nosmp

Advanced Configuration

Disable SMP support.

sched=credit
sched=sedf

Advanced Configuration

Choose a CPU scheduler for Xen to use. Because Xen offers an API for new schedulers to be added, more scheduling choices may appear in the future. The credit scheduler is currently recommended.

badpage=X,Y,...

Advanced Configuration

You can specify a list of physical memory pages not to use because they contain bad bytes. Of course, you could also replace the bad memory!

Example: badpage= 0x111, 0x222,0x333

noirqbalance

Advanced Configuration

Disable software IRQ balancing and affinity. Can be used for systems that have hardware for IRQ-routing issues—why do the job twice in hardware and software?

com1=<BAUD>,<DPS>,<IOBASE>,
<IRQ>
com2=<BAUD>,<DPS>,<IOBASE>,
<IRQ>

Advanced Configuration/Debugging

Xen output can be sent to the serial Typically ports, com1 and com2. this is done for debugging purposes.

Example:

com1=auto, 8n1, 0x408, 5
com2=9600, 8n1

BAUD = the baud rate.

DPS = Data, parity, and stop bits.

IOBASE = I/0 port base.

IRQ = Interrupt Request Line.

If some configuration options are standard, the configuration string can be abbreviated.

mem=X

Debugging/Testing

Limit the amount of physical memory used in the system.

Other than debugging/testing, you generally want to use all the memory you have in your system!

Units can be specified as B for bytes, K for kilobytes, M for megabytes, or G for gigabytes.

If no units are specified, kilobytes is assumed.

tbuf_size=X

Debugging

Request X pages of per-cpu trace buffer space. Traces will be generated in debug builds only.

noreboot

Debugging

Requests that the machine not be automatically rebooted in the case of fatal errors.

sync_console

Debugging

Requests that all console output be sent immediately rather than buffered. Useful for reliably catching the last output generated by a fatal error.

watchdog

Debugging

Enable the Non-Maskable Interrupt (NMI) watchdog timer, which can be helpful in reporting certain failures.

nmi=fatal

Debugging

Specify what to do with an Non-Maskable Interrupt (NMI), parity, or I/O error.

nmi=dom0
nmi=ignore

For fatal, prints a diagnostic message and hangs.

For Dom0, informs Domain0 of the error.

For ignore, ignores the error.

apic_verbosity=debug

apic_verbosity=verbose

Debugging

Requests that information about the local APIC and IOAPIC be printed.

Many of the hypervisor's configuration options control logging, such as the kind of output produced. You can request that the hypervisor produce a substantial amount of debugging information in the form of log messages or even traces of its activity. This output can be sent directly to the screen and/or to one of the serial ports in the system. This is most useful in diagnosing problems at boot. See the Debugging category of options in Table 3.2.

Another configuration option of note specifies what scheduling policy the hypervisor uses. Xen makes changing the scheduling policy easy. A number of different scheduling policies are currently available, including CREDIT and SEDF. SEDF provides weighted CPU sharing and includes real-time algorithms to ensure time guarantees. The CREDIT scheduler is a proportional fair share scheduler.

The CREDIT scheduler was implemented last but is generally recommended. It load balances virtual CPUs of the guest domains across all available physical CPUs. It is a work-conserving scheduler, meaning that physical devices will be idle only when there is no work to do. In contrast, a non-work-conserving scheduler may allow physical devices to be idle even when there is work to do if the idleness helps make the system more predictable.

The CREDIT scheduler allows each domain to be assigned both a weight and a cap. Domains with higher weights get more physical CPU time. Caps are used to limit the amount of CPU a domain can consume.

Other advanced configuration options include whether Symmetric Multiprocessing (nosmp) is supported, whether software IRQ balancing should be done (noirqbalance), and how the Advanced Programmable Interrupt Controller should be handled (apic). There are also some options to override the APIC settings in your BIOS settings. As you might imagine, not all of these options are applicable to every machine. Table 3.2 summarizes the available configuration options.

Table 3.3 contains options that, when specified on the Xen command line in the grub.conf file, will also be propagated to Domain0's command line when it boots. These options have the same syntax as a regular Linux kernel command line. Domain0's kernel is also modified to support one additional nonstandard boot option (xencons or Xen consoles), which is also described in Table 3.3.

Table 3.3. Xen-Related Linux Kernel Command Line Options

Linux Option

Description

acpi=off,force,strict,ht,noirq,...

Specify how the Xen hypervisor and Domain0 should parse the BIOC ACPI tables.

acpi_skip_timer_override

Specify that Xen and Domain0 should ignore the timer-interrupt override instructions specified in the BIOS ACPI tables.

noapic

Specify that Xen and Domain0 should use the legacy PIC and ignore any IOAPICs present in the system.

xencons=off
xencons=tty
xencons=ttyS

Enable or disable the Xen virtual console. If enabled, specify the device node to which it is attached.

For tty, attach to /dev/tty1; tty0 at boot time.

For ttyS, attach to /dev/ttyS0.

The default is ttyS for Domain0 and tty for DomUs.

The GRUB configuration file from the Xen LiveCD did not use many of options described in Tables 3.2 and 3.3 (only the watchdog option). In Listing 3.3, we show how the Debian-based Domain0 section of the GRUB configuration file shown in Listing 3.2 could be modified to use the dom0_mem and xencons options.

Listing 3.3. GRUB Configuration File Illustrating Use of Additional Boot Options

title Debian-based Dom0 (from testing)
root (cd)
kernel /boot/xen-3.0.3.0.gz dom0_mem=65536 watchdog
module /boot/vmlinuz-2.6.16.29-xen ro selinux=0
     ramdisk_size=32758 xencons=off image=rootfs.img
     boot=cow quiet
module /boot/initrd-2.6.16-xen.img

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020