.NET Reference Guide

A Better File.Copy Replacement

Last updated Jun 17, 2011.

As useful as it is, the .NET File.Copy method has its drawbacks. Perhaps most important from my perspective is that using it to copy a very large file from a server can cause the server to exhibit the pathological caching behavior that I described in More Large File Problems.

Another drawback is that File.Copy doesn't expose the full functionality of the CopyFileEx API function. CopyFileEx lets you monitor progress and also lets you cancel the copy operation--things that File.Copy doesn't let you do because it calls the older CopyFile API function, which lacks those features.

In A Faster File Copy, I presented a method that outperforms File.Copy, but it too lacks the advanced features of CopyFileEx, and also can trigger the bad caching behavior.

In this article, I'll look more closely at CopyFileEx and show how to expose its full functionality to .NET programs.

The CopyFileEx API function

The C prototype for CopyFileEx looks like this:

BOOL WINAPI CopyFileEx(
  __in      LPCTSTR lpExistingFileName,
  __in      LPCTSTR lpNewFileName,
  __in_opt  LPPROGRESS_ROUTINE lpProgressRoutine,
  __in_opt  LPVOID lpData,
  __in_opt  LPBOOL pbCancel,
  __in      DWORD dwCopyFlags
);

The first two parameters are simply the source and destination file names. The last parameter is used to specify flags that control how the file is to be copied. The documentation has full descriptions of how those flags affect the copy.

The third parameter is the address of a function that is called by CopyFileEx to report progress. In the API, it's called CopyProgressRoutine:

DWORD CALLBACK CopyProgressRoutine(
  __in      LARGE_INTEGER TotalFileSize,
  __in      LARGE_INTEGER TotalBytesTransferred,
  __in      LARGE_INTEGER StreamSize,
  __in      LARGE_INTEGER StreamBytesTransferred,
  __in      DWORD dwStreamNumber,
  __in      DWORD dwCallbackReason,
  __in      HANDLE hSourceFile,
  __in      HANDLE hDestinationFile,
  __in_opt  LPVOID lpData
);

Most of those parameters are self explanatory. dwCallbackReason will be one of these two values:

• CALLBACK_CHUNK_FINISHED (0) - Another part of the file was copied.
• CALLBACK_STREAM_SWITCH (1) - Another stream was created and is about to be copied. This is the value passed when the callback is first invoked.

The return value of the function should be one of four values:

• PROGRESS_CANCEL (1) - Cancel the copy operation and close the destination file.
• PROGRESS_CONTINUE (0) - Continue the copy operation
• PROGRESS_QUIET (3) - Continue the copy operation, but stop calling the progress routine.
• PROGRESS_STOP (4) - Stop the copy operation. It can be restarted at a later time.

The lpData parameter passed to the CopyProgressRoutine is the same as the lpData parameter that the caller passes to CopyFileEx. It is user-specified data that the API function doesn't use, but instead passes to the progress routine.

The pbCancel parameter to CopyFileEx is a pointer to a BOOL (a 32-bit integer value) that, if set at any time during the copy progress, will cancel the operation.

You'll note that there are two ways to cancel a copy: you can set the value referred to by pbCancel to 1, or you can return PROGRESS_CANCEL or PROGRESS_STOP from the progress function. Having two ways to cancel means that you don't have to implement the progress function if you want the ability to cancel.

The lpProgressRoutine, pbCancel, and lpData, parameters to CopyFileEx are optional. If set to NULL, the corresponding functionality is not available.

Calling CopyFileEx from C#

Ultimately, I want to create a method called CopyFile that has this signature:

static public void CopyFile(
    string sourceFilename,
    string destinationFilename,
    CopyProgressDelegate progressHandler,
    Object userData,
    CopyFileOptions copyOptions,
    CancellationToken cancelToken)

And, of course, simplified versions that let you dispense with the unnecessary things if you want to. For example, you should be able to replace calls to File.Copy with this:

FileUtil.CopyFile(srcFilename, destFilename);

But I'm getting a little ahead of myself. First we have to create a managed wrapper for the Windows API.

In order to call CopyFileEx from C#, we need to create a managed prototype for the function, and also provide definitions for the progress routine. I showed a very simplified version in More Large File Problems, but that didn't provide the progress or cancel functionality.

CopyFileEx makes use of several sets of constants. When converting to .NET, we typically gather related constants and create an enum. So there are three separate enumerations:

    [Flags]
    public enum CopyFileOptions
    {
        None = 0,
        AllowDecryptedDestination = 0x00000008,
        CopySumlink = 0x00000800,
        FailIfExists = 0x00000001,
        NoBuffering = 0x00001000,
        OpenSourceForWrite = 0x00000004,
        Restartable = 0x00000002
    }
    public enum CopyCallbackReason
    {
        ChunkFinished = 0,
        StreamSwitch = 1
    }
    public enum ProgressCallbackResult
    {
        Continue = 0,
        Cancel = 1,
        Stop = 2,
        Quiet = 3
    }

The CopyProgressRoutine has a lot of parameters. A literal translation to C# would look like this:

    public delegate Int32 APICopyProgressRoutine(
        Int64 TotalFileSize,
        Int64 TotalBytesTransferred,
        Int64 StreamSize,
        Int64 StreamBytesTransferred,
        Int32 StreamNumber,
        Int32 CallbackReason,
        IntPtr SourceHandle,
        IntPtr DestinationHandle,
        Object UserData);

Given those definitions, the managed prototype for CopyFileEx becomes:

[DllImport("kernel32.dll", SetLastError = true)]
public static extern bool CopyFileEx(
    string lpExistingFileName,
    string lpNewFileName,
    APICopyProgressRoutine lpProgressRoutine,
    Object lpData,
    ref Int32 lpCancel,
    CopyFileOptions dwCopyFlags);

Whereas it's possible to call that directly, there are some decidedly unfriendly things lurking in the call. First, I think it's too much to ask a client of my new file copy method to create a callback function that takes nine parameters. So I've created a class, CopyProgressArgs to contain all of that data.

    public class CopyProgressArgs
    {
        public readonly Int64 TotalFileSize;
        public readonly Int64 TotalBytesTransferred;
        public readonly Int64 StreamSize;
        public readonly Int64 StreamBytesTransferred;
        public readonly Int32 StreamNumber;
        public readonly Int32 CallbackReason;
        public readonly IntPtr SourceHandle;
        public readonly IntPtr DestinationHandle;
        public readonly Object UserData;
        public ProgressCallbackResult Result { get; set; }
        public CopyProgressArgs(Int64 fsize, Int64 xferBytes, Int64 strmSize, Int64 strmXferBytes,
            Int32 strmNum, Int32 reason, IntPtr srcHandle, IntPtr destHandle, Object uData)
        {
            TotalFileSize = fsize;
            TotalBytesTransferred = xferBytes;
            StreamSize = strmSize;
            StreamBytesTransferred = strmXferBytes;
            StreamNumber = strmNum;
            CallbackReason = reason;
            SourceHandle = srcHandle;
            DestinationHandle = destHandle;
            UserData = uData;
        }
    }

The progress delegate, then, is a simple method that takes that one parameter:

    public delegate void CopyProgressDelegate(CopyProgressArgs e);

Forcing the user to pass a reference to a memory location in order to support cancellation is not a particularly .NET-friendly way to do things, either. .NET 4.0 introduced a new unified model for cancellation, which is more flexible and easier to use. The new file copy method will make use that, instead.

Internally, the code has to translate calls to the .NET-friendly method:

static public void CopyFile(
    string sourceFilename,
    string destinationFilename,
    CopyProgressDelegate progressHandler,
    Object userData,
    CopyFileOptions copyOptions,
    CancellationToken cancelToken)

To the Windows API function:

[DllImport("kernel32.dll", SetLastError = true)]
public static extern bool CopyFileEx(
    string lpExistingFileName,
    string lpNewFileName,
    APICopyProgressRoutine lpProgressRoutine,
    Object lpData,
    ref Int32 lpCancel,
    CopyFileOptions dwCopyFlags);

That's all done by a private method called CopyFileInternal:

struct CopyFileResult
{
    public readonly bool ReturnValue;
    public readonly int LastError;
    public CopyFileResult(bool rslt, int err)
    {
        ReturnValue = rslt;
        LastError = err;
    }
}
static private CopyFileResult CopyFileInternal(
    string sourceFilename,
    string destinationFilename,
    CopyProgressDelegate progressHandler,
    Object userData,
    CopyFileOptions copyOptions,
    CancellationToken cancelToken)
{
    // On error, throw IOException with the value from Marshal.GetLastWin32Error
    CopyProgressDelegate handler = progressHandler;
    int cancelFlag = 0;
    APICopyProgressRoutine callback;
    if (handler == null)
    {
        callback = null;
    }
    else
    {
        callback = new APICopyProgressRoutine((tfSize, xferBytes, strmSize, strmXferBytes,
          strmNum, cbReason, srcHandle, dstHandle, udata) =>
        {
            var args = new CopyProgressArgs(tfSize, xferBytes, strmSize, strmXferBytes,
              strmNum, cbReason, srcHandle, dstHandle, udata);
            handler(args);
            return (Int32)args.Result;
        });
    }
    if (cancelToken.CanBeCanceled)
    {
        cancelToken.Register(() => { cancelFlag = 1; });
    }
    bool rslt = CopyFileEx(
        sourceFilename,
        destinationFilename,
        callback,
        userData,
        ref cancelFlag,
        copyOptions);
    int err = 0;
    if (!rslt)
    {
        err = Marshal.GetLastWin32Error();
    }
    return new CopyFileResult(rslt, err);
}

CopyFileInternal returns a CopyFileResult, which contains a return value as well as a LastError value. The reason for this is so that the code can properly handle asynchronous requests, as you'll see later.

CopyFileInternal takes care of translating parameters, for the call to CopyFileEx, creating the Windows-friendly callback method, and marshaling those callbacks to the .NET-friendly callback delegate. It also handles setting up the pbCancel flag and making it work with the CancellationToken passed by the client.

When the copy operation is done, CopyFileInternal gets the return value and, if necessary, the Win32 error code, and returns them to the caller.

With CopyFileInternal doing all the heavy lifting, the public CopyFile method is quite simple:

static public void CopyFile(
    string sourceFilename,
    string destinationFilename,
    CopyProgressDelegate progressHandler,
    Object userData,
    CopyFileOptions copyOptions,
    CancellationToken cancelToken)
{
    var rslt = CopyFileInternal(
        sourceFilename,
        destinationFilename,
        progressHandler,
        userData,
        copyOptions,
        cancelToken);
    if (!rslt.ReturnValue)
    {
        throw new IOException(string.Format("Error copying file. GetLastError returns {0}.",
          rslt.LastError));
    }
}

Error handling here could be better. As it is, CopyFile will throw IOException on error, and just report the last Win32 error code. More robust error handling would give a better description of the particular error. For example, the destination drive is full, cannot access the source file, etc.

In the complete class shown below, you'll see that I've added some convenience methods like CopyFileNoBuffering, which takes just the source and destination file names and automatically supplies the CopyFileOptions.NoBuffering flag.

I also added BeginCopyFile and EndCopyFile methods so that clients can easily start an asynchronous copy using the typical .NET asynchronous call semantics.

Here's the full source for the FileUtil class that contains the various CopyFile overloads and the associated definitions.

using System;
using System.IO;
using System.Runtime.InteropServices;
using System.Runtime.Remoting.Messaging;
using System.Threading;
namespace Mischel.IO
{
    [Flags]
    public enum CopyFileOptions
    {
        None = 0,
        AllowDecryptedDestination = 0x00000008,
        CopySumlink = 0x00000800,
        FailIfExists = 0x00000001,
        NoBuffering = 0x00001000,
        OpenSourceForWrite = 0x00000004,
        Restartable = 0x00000002
    }
    public enum CopyCallbackReason
    {
        ChunkFinished = 0,
        StreamSwitch = 1
    }
    public enum ProgressCallbackResult
    {
        Continue = 0,
        Cancel = 1,
        Stop = 2,
        Quiet = 3
    }
    public class CopyProgressArgs
    {
        public readonly Int64 TotalFileSize;
        public readonly Int64 TotalBytesTransferred;
        public readonly Int64 StreamSize;
        public readonly Int64 StreamBytesTransferred;
        public readonly Int32 StreamNumber;
        public readonly Int32 CallbackReason;
        public readonly IntPtr SourceHandle;
        public readonly IntPtr DestinationHandle;
        public readonly Object UserData;
        public ProgressCallbackResult Result { get; set; }
        public CopyProgressArgs(Int64 fsize, Int64 xferBytes, Int64 strmSize, Int64 strmXferBytes,
            Int32 strmNum, Int32 reason, IntPtr srcHandle, IntPtr destHandle, Object uData)
        {
            TotalFileSize = fsize;
            TotalBytesTransferred = xferBytes;
            StreamSize = strmSize;
            StreamBytesTransferred = strmXferBytes;
            StreamNumber = strmNum;
            CallbackReason = reason;
            SourceHandle = srcHandle;
            DestinationHandle = destHandle;
            UserData = uData;
        }
    }
    public delegate void CopyProgressDelegate(CopyProgressArgs e);
    public class FileUtil
    {
        private delegate Int32 APICopyProgressRoutine(
            Int64 TotalFileSize,
            Int64 TotalBytesTransferred,
            Int64 StreamSize,
            Int64 StreamBytesTransferred,
            Int32 StreamNumber,
            Int32 CallbackReason,
            IntPtr SourceHandle,
            IntPtr DestinationHandle,
            Object UserData);
        [DllImport("kernel32.dll", SetLastError = true)]
        public static extern bool CopyFileEx(
            string lpExistingFileName,
            string lpNewFileName,
            APICopyProgressRoutine lpProgressRoutine,
            Object lpData,
            ref Int32 lpCancel,
            CopyFileOptions dwCopyFlags);
        struct CopyFileResult
        {
            public readonly bool ReturnValue;
            public readonly int LastError;
            public CopyFileResult(bool rslt, int err)
            {
                ReturnValue = rslt;
                LastError = err;
            }
        }
        static private CopyFileResult CopyFileInternal(
            string sourceFilename,
            string destinationFilename,
            CopyProgressDelegate progressHandler,
            Object userData,
            CopyFileOptions copyOptions,
            CancellationToken cancelToken)
        {
            // On error, throw IOException with the value from Marshal.GetLastWin32Error
            CopyProgressDelegate handler = progressHandler;
            int cancelFlag = 0;
            APICopyProgressRoutine callback;
            if (handler == null)
            {
                callback = null;
            }
            else
            {
                callback = new APICopyProgressRoutine((tfSize, xferBytes, strmSize, strmXferBytes,
                  strmNum, cbReason, srcHandle, dstHandle, udata) =>
                {
                    var args = new CopyProgressArgs(tfSize, xferBytes, strmSize, strmXferBytes,
                      strmNum, cbReason, srcHandle, dstHandle, udata);
                    handler(args);
                    return (Int32)args.Result;
                });
            }
            if (cancelToken.CanBeCanceled)
            {
                cancelToken.Register(() => { cancelFlag = 1; });
            }
            bool rslt = CopyFileEx(
                sourceFilename,
                destinationFilename,
                callback,
                userData,
                ref cancelFlag,
                copyOptions);
            int err = 0;
            if (!rslt)
            {
                err = Marshal.GetLastWin32Error();
            }
            return new CopyFileResult(rslt, err);
        }
        static public void CopyFile(
            string sourceFilename,
            string destinationFilename,
            CopyProgressDelegate progressHandler,
            Object userData,
            CopyFileOptions copyOptions,
            CancellationToken cancelToken)
        {
            var rslt = CopyFileInternal(
                sourceFilename,
                destinationFilename,
                progressHandler,
                userData,
                copyOptions,
                cancelToken);
            if (!rslt.ReturnValue)
            {
                throw new IOException(string.Format("Error copying file. GetLastError returns {0}.",
                  rslt.LastError));
            }
        }
        static public void CopyFile(
            string sourceFilename,
            string destinationFilename,
            CopyProgressDelegate progressHandler,
            Object userData,
            CopyFileOptions copyOptions)
        {
            CopyFile(sourceFilename, destinationFilename, progressHandler, userData, copyOptions,
              CancellationToken.None);
        }
        static public void CopyFile(
            string sourceFilename,
            string destinationFilename)
        {
            CopyFile(sourceFilename, destinationFilename, null, null, CopyFileOptions.None);
        }
        static public void CopyFileNoBuffering(
            string sourceFilename,
            string destinationFilename)
        {
            CopyFile(sourceFilename, destinationFilename, null, null, CopyFileOptions.NoBuffering);
        }
        private delegate CopyFileResult CopyFileInvoker(
            string sourceFilename,
            string destinationFilename,
            CopyProgressDelegate progressHandler,
            Object userData,
            CopyFileOptions copyOptions,
            CancellationToken cancelToken);
        static public IAsyncResult BeginCopyFile(
            string sourceFilename,
            string destinationFilename,
            CopyProgressDelegate progressHandler,
            Object userData,
            CopyFileOptions copyOptions,
            CancellationToken cancelToken,
            AsyncCallback callback = null)
        {
            var caller = new CopyFileInvoker(CopyFileInternal);
            return caller.BeginInvoke(
                sourceFilename,
                destinationFilename,
                progressHandler,
                userData,
                copyOptions,
                cancelToken,
                callback,
                null);
        }
        static public void EndCopyFile(IAsyncResult ar)
        {
            AsyncResult rslt = (AsyncResult)ar;
            var caller = (CopyFileInvoker)rslt.AsyncDelegate;
            var copyResult = caller.EndInvoke(ar);
            if (!copyResult.ReturnValue)
            {
                throw new IOException(string.Format("Error copying file. GetLastError returns {0}.",
                  copyResult.LastError));
            }
        }
    }
}

Because EndCopyFile will be called at some unknown time after CopyFileEx returns, there's no guarantee that a call to Marshal.GetLastWin32Error would return the correct value. That's the reason why CopyFileInternal gets the error code and returns it in the CopyFileResult structure. Otherwise, EndCopyFile wouldn't return the correct error value.

Using the new CopyFile method

In the simplest case, calling the new CopyFile method is just like calling File.Copy. That is, to copy from srcFilename to destFilename, you can write:

FileUtil.CopyFile(srcFilename, destFilename);

Or, if you want to avoid locking up a server that contains a very large file:

FileUtil.CopyFileNoBuffering(srcFilename, destFilename);

If you want to monitor progress and support cancellation, you have to do just slightly more work. The code below starts an asynchronous copy and reports progress as each chunk is copied. At any point, you can press Enter to cancel the copy operation.

static int Main(string[] args)
{
    string srcFilename = args[0];
    string destFilename = args[1];
    Console.WriteLine("Copying {0} to {1}", srcFilename, destFilename);
    Console.WriteLine("Press Enter to cancel the copy operation.");
    var cts = new CancellationTokenSource();
    try
    {
	var ar = FileUtil.BeginCopyFile(
	    srcFilename,
	    destFilename,
	    MyCopyProgress,
	    "Hello, world",
	    CopyFileOptions.NoBuffering,
	    cts.Token,
	    CopyDoneCallback);
	// Wait for user to press Enter.
	Console.ReadLine();
	// set the cancel flag
	cts.Cancel();
	// wait for the async call to finish
	ar.AsyncWaitHandle.WaitOne();
    }
    finally
    {
	cts.Dispose();
    }
    return 0;
}
static void CopyDoneCallback(IAsyncResult ar)
{
    Console.WriteLine();
    Console.WriteLine("Copy done.");
    try
    {
	FileUtil.EndCopyFile(ar);
    }
    catch (IOException ex)
    {
	Console.WriteLine(ex.Message);
    }
}
static void MyCopyProgress(CopyProgressArgs e)
{
    Console.Write("\r{0:N0}/{1:N0} ({2:P2})", e.TotalBytesTransferred, e.TotalFileSize,
        (double)e.TotalBytesTransferred / e.TotalFileSize);
    e.Result = ProgressCallbackResult.Continue;
}

It's a little more work, but not a lot more work. The benefits are huge. You can start a file copy in the background, monitor its progress, and cancel it at any time. In addition (and most important from my perspective), the ability to specify no buffering means that my programs won't inadvertently crash a server when it tries to copy large files.