Home > Articles > Programming > Windows Programming

This chapter is from the book

5.6 -Windows XP and Windows Server 2003 Volume Shadow Copy Service

With Windows XP and Windows Server 2003, Microsoft has implemented the volume shadow copy service to provide a framework that allows a coordinated and consistent point-in-time copy of disk volumes. For legal reasons, Microsoft has chosen to refer to the functionality as a volume shadow copy, which is really no different from the more popularly known term, snapshots. Volume shadow copy service has been implemented via a filter driver (called volsnap.sys) beneath the file system.

Microsoft makes a volume shadow copy software development kit available on a nondisclosure basis. This SDK appears to be aimed at three broad categories:

  1. ISVs who might wish to develop volume shadow copy writers, including Microsoft Exchange, SQL Server, Oracle, SAP, Sybase, and others.

  2. ISVs developing backup and storage management applications. These vendors would develop requestors for the volume shadow copy service.

  3. Independent hardware and software vendors (IHVs and ISVs) developing hardware and software for backup, fault tolerance, and data integrity. Examples include VERITAS, EMC, and their competitors in this area. These vendors would develop snapshot providers.

When an application does not have code to support the snapshot service, the application data will still be backed up in a state that will be consistent to the same degree as if the system had failed to shut down gracefully. When an application has code to support snapshot services, the application is expected to provide those services in a restore operation as well. This is logical because the application is now expected to furnish some data (such as what files need to be backed up, and a backup and restore methodology) when a backup is requested, and it is expected to interpret and act on this same data when presented with the data upon a restore operation.

The major advantage is that backup or restore, as it exists in Windows operating systems, is a tricky business that may not work reliably 100 percent of the time. The new snapshot service will facilitate total reliability while also enabling more complicated scenarios that until now were not possible in Windows.

The volume shadow copy architecture shipping with Windows XP and Windows Server 2003 consists of four types of modules, as shown in Figure 5.8:

Figure 8Figure 5.8 Volume Shadow Copy Architecture

  1. Writers

  2. Requestors

  3. Volume shadow copy service

  4. Providers

These modules are described in detail in Sections 5.6.1 through 5.6.4.

As far as the different modules that constitute the snapshot functionality are concerned, snapshot provider 2 has a kernel mode component, and snapshot provider 1 does not. The snapshot provider service is shown in gray in Figure 5.8 to signify that Microsoft supplies this functionality and that vendors do not have to bother writing such a service. Microsoft also provides some of the other components shown, but their functionality is limited either in terms of working with a specific application or in terms of the range of features provided; hence vendors are expected and encouraged to write these components using the snapshot SDK.

Writers and providers need to implement a Component Object Model (COM) out-of-proc provider as described in the volume shadow copy SDK.1 A provider is typically implemented as a finite state machine. The state machine transitions from one state to another upon receipt of an event that is generated by the shadow copy service. A provider will receive an event (generated by the shadow copy service), but exactly which event depends on the current state of the provider and whether an error has occurred. To state this differently, a provider has a preferred event that it expects to arrive, allowing it to transition to the next normally expected state. However, an error may occur and the provider may receive a different (from the expected) event, and the provider should be able to handle that as well.

The Microsoft shadow copy framework in Windows Server 2003 and Windows XP provides some core functionality needed for storage management, including the following:

  • Defining the point in time at which the snapshot is triggered

  • Providing synchronization services among applications, databases, operating systems, and file systems to flush cached data, suspend writers, create the snapshot, and provide notification to applications, databases, and operating system elements that they can resume normal mode of operation

  • Providing a single API that can be used by backup/restore applications

  • Providing a platform for managing the snapshot

The Microsoft shadow copy framework can handle a set of volumes for which a snapshot must be done as a single set. The only behavior supported is "all or none"; that is, if any one operation fails, all of them fail. The Microsoft shadow copy framework also issues a request (to the snapshot provider) to delete the snapshot when the requestor application is done with the snapshot. If the snapshot needs to be available later, the snapshot provider or requestor application must provide the necessary functionality. ISVs can build applications that can create, catalog, and manage multiple snapshots using the shadow copy architecture; however, such applications are not part of the Windows Server 2003 product available from Microsoft.

5.6.1 Writers

Snapshot writers are simply the applications that write data (applications simply reading data are not much of a problem). Examples of snapshot writers include Microsoft Exchange, Microsoft SQL Server 2000, SAP, and Oracle. Microsoft and third-party ISVs are both expected to develop snapshot writer–aware applications. Snapshot writers should be developed with the snapshot SDK. In particular, snapshot writers will receive two events from the snapshot service indicating that they should freeze all write activity, and a different thaw event indicating that write activity can now happen (hopefully signifying successful creation of the snapshot). There are other events that the writers can get from the snapshot service, and details can be obtained from the snapshot SDK. Because the applications can define the consistency of data they expect to achieve, they should be able to perform the quiesce operation fairly quickly.

Further, it is worthwhile understanding one big advantage of the volume shadow copy service architecture over traditional hardware-based snapshot mechanisms used with Windows 2000 and previous Windows versions. With these classic versions of creating a snapshot, the hardware-based mechanism had no means of determining the state of the application and of the operating system software and cache in particular. This meant that an appreciable percentage of the snapshots were inconsistent. Further, the only way of determining the health of the snapshot created required running an application data consistency checker, which could run for hours.

By comparison, the volume shadow copy–based architecture not only attempts to flush caches and hold writes, but it can also determine in a matter of a minute or so, whether the created snapshot is consistent or not. Once the thaw event has been signaled, the volume shadow copy service simply interrogates the writers as to whether they successfully managed to hold their write operations between the freeze and thaw events. If any of the writers failed, the snapshot is deemed to have failed.

Snapshot writers are also expected to provide data required for backup and restore—for example, what files need to be copied, what files need to be excluded, what collection of objects needs to be treated collectively as a single set. This data is stored by the snapshot service in a writer metadata document that is in XML format. Writers can also use this document to store data that is of interest to them. To restore data, the application simply hands the collection of data to the writer application to accomplish the restore operation.

Microsoft has announced that it will ship a SQL Server 2000 and Exchange writer, as well as writers for various other components of the Windows Server operating system. Microsoft is cooperating with ISVs to develop writers for other applications, including Active Directory.

5.6.2 Requestors

Requestors are typically backup applications that request the creation of a snapshot by making the appropriate API call to the volume shadow copy service provided by Microsoft. What's interesting here is the fact that the model remarkably simplifies some issues for the backup application writer. The backup application no longer has to solve the difficult issue of where the data to be backed up is, or what part of the data consists of application log files and what special treatment those files merit.

The appropriate writer (e.g., Microsoft SQL Server) is responsible for specifying the files and directories that should be included in the backup. Restore operations also become a lot simpler because the restore application again does not have to locate the data and figure out what files to pass to what API of the application (such as Exchange or SQL). The restore application simply hands the collection of data to the writer (application) and lets it accomplish the restore operation.

5.6.3 Volume Shadow Copy Service

The Windows NT volume shadow copy service, written by Microsoft, coordinates the activities for all the snapshot components. In particular, it provides the following:

  • A single interface for backup applications or snapshot requestors to deal with, rather than multiple APIs from multiple applications.

  • A single interface to produce, manipulate, and delete a crash-consistent volume snapshot or ghost volume.

  • A single interface to allow different applications and snapshot providers to register and deregister as snapshot writers or snapshot providers.

  • Synchronization and coordination among the various components to accomplish creation, deletion, or movement of snapshots, as well as backup and restore operations. The service prioritizes snapshot providers like this: Hardware providers have the highest priority, software providers are next, and the default Microsoft-provided snapshot provider has the lowest priority.

ISVs do not need to worry about writing a volume shadow copy service. Think of the snapshot service provided by Microsoft as being akin to a print spooler. You need only one print spooler. Some vendors (such as the provider vendors) simply need to write the equivalent of a printer driver. Other vendors need to write the equivalent of a printing application.

5.6.4 Providers

Snapshot providers are expected to be written by ISVs and IHVs to create, delete, and manipulate snapshots. As described earlier in this chapter, snapshot providers need to be written as a COM out-of-proc provider, via the snapshot SDK.

The provider may also have a kernel mode component—for example, a filter driver that is located between the file system and the Logical Disk Manager (LDM). This kernel mode functionality may also be optionally implemented in hardware instead. Note that even a hardware-based provider will still leverage the rest of the functionality provided by the framework—for example, definition of the point in time, I/O synchronization, and platform for building storage management applications, including backup, restore, and snapshot management applications.

One prime example of a snapshot provider is the volsnap.sys driver that ships with Windows XP and is also expected to ship with Windows Server 2003. This provider uses copy-on-write technology to create the necessary minimal data in a side store to be able to re-create the volume at a given point in time. The big assumption is that the required amount of free disk space is indeed available. This provider can handle NTFS, FAT32, and raw volumes on Windows Server 2003. However, this snapshot provider can provide only read-only snapshots and handle only one snapshot per volume. This limitation is in the provider itself and not the infrastructure. ISVs and IHVs can build richer functionality in their providers and writers if they so desire.

A complete description of all the events that are signaled to the snapshot provider via its COM provider is available in the snapshot SDK. A couple of important events are discussed here:

  • PreCommitSnapshot. When the snapshot provider receives a PreCommitSnapshot event, it should start all I/O operations that take a long time—for example, synchronizing of the mirror.

  • CommitSnapshot. When the provider receives a CommitSnapshot event, it should recognize that the snapshot service will time out the operation within 10 seconds. Hence the functionality here should be extremely fast. Further, until the snapshot is completed, Windows NT will wait to send any write operations to the volumes for which snapshots are being made. This means the provider should not do any I/O on this volume, and if it does, it should not expect the I/O to complete until the snapshot is either complete or is aborted.

Conceptual Windows NT NDMP Implementation

Snapshot providers must provide certain mandatory functionality and may provide functionality that exceeds the mandatory functionality. The mandatory functionality is as follows:

  • Providers are responsible for locating the storage needed to create the snapshot. The framework provided by Microsoft does not provide any such functionality.

  • The provider must mount the snapshot in a different namespace and not have the snapshot mounted as a separate volume. An inspection of Windows XP shows that the Microsoft snapshot provider mounts the snapshot at \Device\HarddiskSnapshotX.

5.6.5 Windows NT I/O Subsystem Modifications

Though the modifications to the Windows NT I/O subsystem are not explicitly part of the snapshot environment, it is worthwhile noting that a fair amount of work was needed in the file systems, I/O stack, and file system filter drivers to accomplish a consistent and reliable point-in-time snapshot. In particular, Microsoft added two IOCTLs that all file systems and file system filter drivers need to implement:

  1. IOCTL_VOLSNAP_FLUSH_AND_HOLD_WRITES, which should be both chained on and acted upon. The chaining is to allow drivers farther down the stack chain to also act on the IOCTL. The action consists of flushing and holding all file system metadata. Once all data has been flushed, no further writes should be issued until the outstanding IRPs issued to flush all data and metadata have completed.

  2. IOCTL_VOLSNAP_RELEASE_WRITES also needs to be chained on and acted upon. This IOCTL indicates either successful completion of the snapshot or abandonment of the snapshot operation.

Some relevant portions of the Windows NT operating system have also been modified to trigger these IOCTLs at the appropriate time. Although Microsoft has already modified the file system and file system filter drivers that it ships to provide this functionality, ISV-shipped filter drivers need to do the same.

Section 5.8 describes an industry standard called Network Data Management Protocol (NDMP). But before that topic is discussed, it is worthwhile noting the relationship between volume shadow copy architecture in Windows XP/Windows Server 2003 and NDMP. The shadow copy architecture is a means of creating a clone of the data that needs to be backed up; NDMP can be used to move the data from the clone to tape or other backup media.

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020