Home > Articles > Operating Systems, Server > Microsoft Servers

Windows 2000 Memory Management

The Windows 2000 kernel makes heavy use of the protected-mode virtual memory management mechanisms of the Intel i386 CPU class. To get a better understanding of how Windows 2000 manages its main memory, it is important to be at least basically familiar with some architectural issues of the i386 CPU. This excerpt from Undocumented Windows 2000 Secrets: A Programmer's Cookbook gives you all the information you need about i386 memory management structures.

The Windows 2000 kernel makes heavy use of the protected-mode virtual memory management mechanisms of the Intel i386 CPU class. To get a better understanding of how Windows 2000 manages its main memory, it is important to be at least basically familiar with some architectural issues of the i386 CPU. The term i386 might look somewhat anachronistic because the 80386 CPU dates back to the Stone Age of Windows computing. Windows 2000 is designed for Pentium CPUs and better. However, even these newer processors rely on the memory management model originally designed for the 80386 CPU—with some important enhancements, of course. Therefore, Microsoft usually labels the Windows NT and 2000 versions built for Intel processors "i386" or even "x86". Don't be confused about that—whenever you read the numbers 86 or 386 in this book, keep in mind that the corresponding information refers to a specific CPU architecture, not a specific processor release.

i386 Memory Management Data Structures

Some portions of the sample code following are concerned with low-level memory management and peek inside the mechanisms outlined above. For convenience, I have defined several C data structures that make this task easier. Because many data items inside the i386 CPU are concatenations of single bits or bit groups, C bit-fields come in handy. Bit-fields are an efficient way to access individual bits of or extract contiguous bit groups from larger data words. Microsoft Visual C/C++ generates quite clever code for bit-field operations. Listing 1 is part one of a series of CPU data type definitions, containing the following items:

  • X86_REGISTER is a basic unsigned 32-bit integral type that can represent various CPU registers. This comprises all general-purpose, index, pointer, control, debug, and test registers.

  • X86_SELECTOR represents a 16-bit segment selector, as stored in the segment registers CS, DS, ES, FS, GS, and SS. In Figure 1, selectors are depicted as the upper-third of a logical 48-bit address, serving as an index into a descriptor table. For computational convenience, the 16-bit selector value is extended to 32-bits, with the upper half marked "reserved". Note that the X86_SELECTOR structure is a union of two structures. The first one specifies the selector value as a packed 16-bit WORD named wValue, while the second breaks it up into bit-fields. The RPL field specifies the Requested Privilege Level, which is either 0 (kernel-mode) or 3 (user-mode) on Windows 2000. The TI bit switches between the Global and Local Descriptor Tables (GDT/LDT).

  • X86_DESCRIPTOR defines the format of a table entry pointed to by a selector. It is a 64-bit quantity with a very convoluted structure due to its historic evolution. The linear base address defining the start location of the associated segment is scattered among three bit fields named Base1, Base2, and Base3; with Base1 being the least significant part. The segment limit specifying the segment size minus one is divided into the pair Limit1 and Limit2, with the former representing the least-significant half. The remaining bit-fields store various segment properties. For instance, the G bit defines the segment granularity. If zero, the segment limit is specified in bytes; otherwise, the limit value has to be multiplied by 4KB. Like X86_SELECTOR, the X86_DESCRIPTOR structure is made up of a union to allow different interpretations of its value. The dValueLow and dValueHigh members are helpful if you have to copy descriptors without regard to their internal structure.

  • X86_GATE looks somewhat similar to X86_DESCRIPTOR. In fact, both structures are related—while X86_DESCRIPTOR is a GDT entry and describes the memory properties of a segment, X86_GATE is an entry inside the Interrupt Descriptor Table (IDT) and describes the memory properties of an interrupt handler. The IDT can contain Task, Interrupt, and Trap Gates. (No, Bill Gates is not stored in the IDT!) The X86_GATE structure matches all three types, with the Type bit-field determining the identity. Type 5 identifies a Task Gate, Types 6 and 14 Interrupt Gates, and Types 7 and 15 Trap Gates. The most significant type bit specifies the size of the gate: 16-bit gates have this bit set to zero; otherwise, it is a 32-bit gate.

  • X86_TABLE is a tricky structure that is used to read the values of the GDTR or IDTR by means of the assembly language instructions SGDT (store GDT register) and SIDT (store IDT register), respectively. Both instructions require a 48-bit memory operand, where the limit and base address values will be stored. To maintain DWORD alignment for the 32-bit base address, X86_TABLE starts out with the 16-bit dummy member wReserved. Depending on whether the SGDT or SIDT instruction is applied, the base address must be interpreted as a descriptor or gate pointer, as suggested by the union of PX86_DESCRIPTOR and PX86_GATE types. The wLimit member is the same for both table types.

Listing 1 i386 Registers, Selectors, Descriptors, Gates, and Tables.

// =================================================================
// INTEL X86 STRUCTURES, PART 1 OF 3
// =================================================================

typedef DWORD X86_REGISTER, *PX86_REGISTER, **PPX86_REGISTER;

// -----------------------------------------------------------------

typedef struct _X86_SELECTOR
  {
  union
    {
    struct
      {
      WORD wValue;      // packed value
      WORD wReserved;
      };
    struct
      {
      unsigned RPL   : 2; // requested privilege level
      unsigned TI    : 1; // table indicator: 0=gdt, 1=ldt
      unsigned Index  : 13; // index into descriptor table
      unsigned Reserved : 16;
      };
    };
  }
  X86_SELECTOR, *PX86_SELECTOR, **PPX86_SELECTOR;

#define X86_SELECTOR_ sizeof (X86_SELECTOR)

// -----------------------------------------------------------------

typedef struct _X86_DESCRIPTOR
  {
  union
    {
    struct
      {
      DWORD dValueLow;    // packed value
      DWORD dValueHigh;
      };
    struct
      {
      unsigned Limit1  : 16; // bits 15..00
      unsigned Base1  : 16; // bits 15..00
      unsigned Base2  : 8; // bits 23..16
      unsigned Type   : 4; // segment type
      unsigned S    : 1; // type (0=system, 1=code/data)
      unsigned DPL   : 2; // descriptor privilege level
      unsigned P    : 1; // segment present
      unsigned Limit2  : 4; // bits 19..16
      unsigned AVL   : 1; // available to programmer 
      unsigned Reserved : 1;
      unsigned DB    : 1; // 0=16-bit, 1=32-bit
      unsigned G    : 1; // granularity (1=4KB)
      unsigned Base3  : 8; // bits 31..24
      };
    };
  }
  X86_DESCRIPTOR, *PX86_DESCRIPTOR, **PPX86_DESCRIPTOR;

#define X86_DESCRIPTOR_ sizeof (X86_DESCRIPTOR)

// -----------------------------------------------------------------

typedef struct _X86_GATE
  {
  union
    {
    struct
      {
      DWORD dValueLow;     // packed value
      DWORD dValueHigh;
      };
    struct
      {
      unsigned Offset1  : 16; // bits 15..00
      unsigned Selector  : 16; // segment selector
      unsigned Parameters : 5; // parameters
      unsigned Reserved  : 3;
      unsigned Type    : 4; // gate type and size
      unsigned S     : 1; // always 0
      unsigned DPL    : 2; // descriptor privilege level
      unsigned P     : 1; // segment present
      unsigned Offset2  : 16; // bits 31..16
      };
    };
  }
  X86_GATE, *PX86_GATE, **PPX86_GATE;

#define X86_GATE_ sizeof (X86_GATE)

// -----------------------------------------------------------------

typedef struct _X86_TABLE
  {
  WORD wReserved;          // force 32-bit alignment
  WORD wLimit;           // table limit
  union
    {
    PX86_DESCRIPTOR pDescriptors; // used by sgdt instruction
    PX86_GATE    pGates;    // used by sidt instruction
    };
  }
  X86_TABLE, *PX86_TABLE, **PPX86_TABLE;

#define X86_TABLE_ sizeof (X86_TABLE)

// =================================================================

Figure 1 Flat 4GB memory segmentation.

The next set of i386 memory management structures collected in Listing 2 relates to demand paging and contains several items illustrated in Figures 2 and 3:

  • X86_PDBR is, of course, a structural representation of the CPU's CR3 register, also known as the page-directory base register (PDBR). The upper 20 bits contain the page-frame number (PFN), which is an index into the array of physical 4KB pages. PFN=0 corresponds to physical address, 0x00000000, PFN=1 to 0x00001000, and so forth. 20 bits are just enough to cover the entire 4GB address space. The PFN in the PDBR is the index of the physical page that holds the page-directory. Most of the remaining bits are reserved, except for bit #3, controlling page-level write-through (PWT), and bit #4, disabling page-level caching if set.

  • X86_PDE_4M and X86_PDE_4K are alternative incarnations of page-directory entries (PDEs) for 4MB and 4KB pages, respectively. A page-directory contains a maximum of 1,024 PDEs. Again, PFN is the page-frame number, pointing to the subordinate page. For a 4MB PDE, the PFN bit-field is only 10 bits wide, addressing a 4MB data page. The 20-bit PFN of 4KB PDE points to a page-table that ultimately selects the physical data pages. The remaining bits define various properties. The most interesting ones are the "Page Size" bit PS, controlling the page size (0 = 4KB, 1 = 4MB), and the "Present" bit P, indicating whether the subordinate data page (4MB mode) or page-table (4KB mode) is present in physical memory.

  • X86_PTE_4K defines the internal structure of a page-table entry (PTE) contained in a page-table. Like a page-directory, a page-table can contain up to 1,024 entries. The only difference between X86_PTE_4K and X86_PDE_4K is that the former lacks the PS bit, which is not required because the page size must be 4KB, as determined by the PDE's PS bit. Note that there is no such thing like a 4MB PTE, since the 4MB memory model doesn't require an intermediate page-table layer.

  • X86_PNPE represents a "page-not-present entry" (PNPE); that is, a PDE or PTE where the P bit is zero. According to the Intel manuals, the remaining 31 bits are "Available to Operating System or Executive." If a linear address maps to a PNPE, this means that this address is either unused, or it points to a page that is currently swapped out to one of the pagefiles. Windows 2000 uses the 31 unassigned bits of the PNPE to store status information of the page. The structure of this information is undocumented, but it seems that bit #10, named PageFile in Listing 2, is set if the page is swapped out. In this case, the Reserved1 and Reserved2 bit-fields contain values that enable the system to locate the page in the pagefiles, so it can be swapped in as soon as one of its linear addresses is touched by a memory read/write instruction.

  • X86_PE is included for convenience. It is merely a union of all possible forms a page entry can take, comprising the PDBR contents, 4MB and 4KB PDEs, PTEs, and PNPEs.

Listing 2 i386 PDBR, PDE, PTE, and Page-Not-Present Entry Values.

// =================================================================
// INTEL X86 STRUCTURES, PART 2 OF 3
// =================================================================

typedef struct _X86_PDBR // page-directory base register (cr3)
  {
  union
    {
    struct
      {
      DWORD dValue;      // packed value
      };
    struct
      {
      unsigned Reserved1 : 3;
      unsigned PWT    : 1; // page-level write-through
      unsigned PCD    : 1; // page-level cache disabled
      unsigned Reserved2 : 7;
      unsigned PFN    : 20; // page-frame number
      };
    };
  }
  X86_PDBR, *PX86_PDBR, **PPX86_PDBR;

#define X86_PDBR_ sizeof (X86_PDBR)

// -----------------------------------------------------------------

typedef struct _X86_PDE_4M // page-directory entry (4-MB page)
  {
  union
    {
    struct
      {
      DWORD dValue;      // packed value
      };
    struct
      {
      unsigned P     : 1; // present (1 = present)
      unsigned RW    : 1; // read/write
      unsigned US    : 1; // user/supervisor
      unsigned PWT    : 1; // page-level write-through
      unsigned PCD    : 1; // page-level cache disabled
      unsigned A     : 1; // accessed
      unsigned D     : 1; // dirty
      unsigned PS    : 1; // page size (1 = 4-MB page)
      unsigned G     : 1; // global page
      unsigned Available : 3; // available to programmer
      unsigned Reserved : 10;
      unsigned PFN    : 10; // page-frame number
      };
    };
  }
  X86_PDE_4M, *PX86_PDE_4M, **PPX86_PDE_4M;

#define X86_PDE_4M_ sizeof (X86_PDE_4M)

// -----------------------------------------------------------------

typedef struct _X86_PDE_4K // page-directory entry (4-KB page)
  {
  union
    {
    struct
      {
      DWORD dValue;      // packed value
      };
    struct
      {
      unsigned P     : 1; // present (1 = present)
      unsigned RW    : 1; // read/write
      unsigned US    : 1; // user/supervisor
      unsigned PWT    : 1; // page-level write-through
      unsigned PCD    : 1; // page-level cache disabled
      unsigned A     : 1; // accessed
      unsigned Reserved : 1; // dirty
      unsigned PS    : 1; // page size (0 = 4-KB page)
      unsigned G     : 1; // global page
      unsigned Available : 3; // available to programmer
      unsigned PFN    : 20; // page-frame number
      };
    };
  }
  X86_PDE_4K, *PX86_PDE_4K, **PPX86_PDE_4K;

#define X86_PDE_4K_ sizeof (X86_PDE_4K)

// -----------------------------------------------------------------

typedef struct _X86_PTE_4K // page-table entry (4-KB page)
  {
  union
    {
    struct
      {
      DWORD dValue;      // packed value
      };
    struct
      {
      unsigned P     : 1; // present (1 = present)
      unsigned RW    : 1; // read/write
      unsigned US    : 1; // user/supervisor
      unsigned PWT    : 1; // page-level write-through
      unsigned PCD    : 1; // page-level cache disabled
      unsigned A     : 1; // accessed
      unsigned D     : 1; // dirty
      unsigned Reserved : 1;
      unsigned G     : 1; // global page
      unsigned Available : 3; // available to programmer
      unsigned PFN    : 20; // page-frame number
      };
    };
  }
  X86_PTE_4K, *PX86_PTE_4K, **PPX86_PTE_4K;

#define X86_PTE_4K_ sizeof (X86_PTE_4K)

// -----------------------------------------------------------------

typedef struct _X86_PNPE // page not present entry
  {
  union
    {
    struct
      {
      DWORD dValue;      // packed value
      };
    struct
      {
      unsigned P     : 1; // present (0 = not present)
      unsigned Reserved1 : 9;
      unsigned PageFile : 1; // page swapped to pagefile
      unsigned Reserved2 : 21;
      };
    };
  }
  X86_PNPE, *PX86_PNPE, **PPX86_PNPE;

#define X86_PNPE_ sizeof (X86_PNPE)

// -----------------------------------------------------------------

typedef struct _X86_PE // general page entry
  {
  union
    {
    DWORD   dValue; // packed value
    X86_PDBR  pdbr;  // page-directory Base Register
    X86_PDE_4M pde4M; // page-directory entry (4-MB page)
    X86_PDE_4K pde4K; // page-directory entry (4-KB page)
    X86_PTE_4K pte4K; // page-table entry (4-KB page)
    X86_PNPE  pnpe;  // page not present entry
    };
  }
  X86_PE, *PX86_PE, **PPX86_PE;

#define X86_PE_ sizeof (X86_PE)

// =================================================================

Figure 2 Double-layered paging with 4KB pages.

Figure 3 Single-layered paging with 4MB pages.

In Listing 3, I have added structural representations of linear addresses. These structures are formal definitions of the "Linear Address" boxes in Figures 2 and 3:

  • X86_LINEAR_4M is the format of linear addresses that point into a 4MB data page, as shown in Figure 3. The page-directory index PDI is an index into the page-directory currently addressed by the PDBR, selecting one of its PDEs. The 22-bit Offset member points to the target address within the corresponding 4MB physical page.

  • X86_LINEAR_4K is the 4KB variant of a linear address. As outlined in Figure 2, it is composed of three bit-fields: As in a 4MB address, the upper 10 PDI bits select a PDE. The page-table index PTI has a similar duty, pointing to a PTE inside the page-table addressed by this PDE. The remaining 12 bits are the offset into the resulting 4KB physical page.

  • X86_LINEAR is another convenience structure that simply unites X86_LINEAR_4M and X86_LINEAR_4K in a single data type.

Listing 3 i386 Linear Addresses.

// =================================================================
// INTEL X86 STRUCTURES, PART 3 OF 3
// =================================================================

typedef struct _X86_LINEAR_4M // linear address (4-MB page)
  {
  union
    {
    struct
      {
      PVOID pAddress;    // packed address
      };
    struct
      {
      unsigned Offset : 22; // offset into page
      unsigned PDI  : 10; // page-directory index
      };
    };
  }
  X86_LINEAR_4M, *PX86_LINEAR_4M, **PPX86_LINEAR_4M;

#define X86_LINEAR_4M_ sizeof (X86_LINEAR_4M)

// -----------------------------------------------------------------

typedef struct _X86_LINEAR_4K // linear address (4-KB page)
  {
  union
    {
    struct
      {
      PVOID pAddress;    // packed address
      };
    struct
      {
      unsigned Offset : 12; // offset into page
      unsigned PTI  : 10; // page-table index
      unsigned PDI  : 10; // page-directory index
      };
    };
  }
  X86_LINEAR_4K, *PX86_LINEAR_4K, **PPX86_LINEAR_4K;

#define X86_LINEAR_4K_ sizeof (X86_LINEAR_4K)

// -----------------------------------------------------------------

typedef struct _X86_LINEAR // general linear address
  {
  union
    {
    PVOID     pAddress; // packed address
    X86_LINEAR_4M linear4M; // linear address (4-MB page)
    X86_LINEAR_4K linear4K; // linear address (4-KB page)
    };
  }
  X86_LINEAR, *PX86_LINEAR, **PPX86_LINEAR;

#define X86_LINEAR_ sizeof (X86_LINEAR)

// =================================================================

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020