Microsoft Sync Framework - A primer to the file sync provider

Introduction

Currently I am working on a project that requires some files/folders to be synchronized at regular intervals across networks ranging in size fromaninexpensive LAN to and expensive WAN. The problem was to search for a technology that would be effective and at the same time gel well with the .Net environment, as the rest of the application(s) were to be written in C#. In this process I came across the Microsoft Sync Framework which is still in the Community Preview phase but thought it would be a good idea to sharemy researchwith the community.

What is the Sync Framework?

Sync Framework is a Microsoft framework for developing systems that can synchronize any type of data between any type of device and across any type of network. This is the objective behind the framework. In reality it is an extensible framework which can be used by developers to write sync providers that can sync data between devices/data sources or just use existing sync providers to create applications with data synchronization capabilities. When we talk about synchronization, the things that come to our mind is efficiency, reliability, conflict resolution, etc and the Sync Framework handles all of these parameters gracefully with features like: updatingincremental data changes instead ofthe entire dataset, defining conflict resolution policies, graceful cancellation of synchronizations when required without data inconsistencies, support for recycle bin, etc.

Microsoft currently offers 3 types of providers that can be used by developers to write applications that can leverage the functionality in these providers. The sync providers that are currently available from Microsoft are.

• Provider for ADO.Net:For synchronizing data between ADO.Net supported data sources.
• Provider for File Systems: For synchronizing Files and Folders on an NTFS or FAT file system. Provider for Feeds like RSS: For synchronizing data from Feeds.

This article does not focus on writing a sync provider and just focuses on creating a small application to demonstrate the sync provider for File Systems. Probably sometime soon I will write another article explaining how to utilize additional sync providers.

The File Sync Provider in C#

As mentioned above, the File Sync provider is for synchronizing files and directories on the FAT or NTFS file systems. The way the File Sync works is by comparing the metadata for the replica and detecting changes in files from the last time the replica was synchronized. The metadata is persisted in a file named "filesync.metadata" both in the source and destination replicas. If you are wondering what is a Replica, it is the locations between which the synchronization would happen. For example if your application uses File Sync to synchronize data between folder C:\Sync\Folder1 and C:\Sync\Folder2, then these two folders can be termed as replicas.

Now I will write a simple application to demonstrate the File Sync Provider. There are numerous ways of implementing this technology. The following program is just to give you a brief understanding of the technology.

Let's say I want to synchronize the contents of the folder C:\Sync\Destination with the contents of the folder C:\Sync\Source.

1. Create a new windows application in Visual studio.
2. Add reference to the assemblies "Microsoft.Synchronization" and Microsoft.Synchronization.Files".
3. Place a button named "btnSynchronize" on a form.

Write the following code in the Click handler of the button

private void btnSynchronize_Click(object sender, EventArgs e)
{
    // Generate a unique Id for the source and store it in a file for future reference.
    SyncId sourceId = GetSyncID(@"C:\Sync\Source\File.ID");

    // Generate a unique Id for the destination and store it in a file for future reference.
    SyncId destId = GetSyncID(@"C:\Sync\Destination\File.ID");

    // Create a FileSyncProvider object by passing the SyncID and the source location.
    FileSyncProvider sourceReplica = new FileSyncProvider(sourceId, @"C:\Sync\Source\");

    // Create a FileSyncProvider object by passing the SyncID and the destination location.
    FileSyncProvider destReplica = new FileSyncProvider(destId, @"C:\Sync\Destination\");

    // Initialize the agent which actually performs the synchronization.
    SyncAgent agent = new SyncAgent();

    // Assign the source replica as the Local Provider and the destination replica as the Remote provider
    // so that the agent knows which is the source and which one is the destination.
    agent.LocalProvider = sourceReplica;
    agent.RemoteProvider = destReplica;

    // Set the direction of synchronization from Source to destination as this is a one-way synchronization.
    // You may use SyncDirection.Download if you want the Local replica to be treated as Destination
    // and the Remote replica to be the source; use SyncDirection.DownloadAndUpload or
    // SyncDirection.UploadAndDownload for two-way synchronization.
    agent.Direction = SyncDirection.Upload;

    // Make a call to the Synchronize method for starting the synchronization process.
    agent.Synchronize();
}

// This is a private function I have created to generate the SyncID if
// it is the first time a Sync is happening on the source and destination replicas
// or else retrieve the SyncID stored in a pre-defined file (File.ID) on the source and destination replicas.
// I have used the GUID for generating the SyncID, whereas you may also
// use a unique string, byte array, etc. for generating the SyncID.
// The objective here is to have a unique SyncID.
private static SyncId GetSyncID(string syncFilePath)
{
    Guid guid;
    SyncId replicaID = null;

    if (!File.Exists(syncFilePath)) // The ID file doesn't exist.
    {
        // Create the file and store the guid which is used to instantiate the instance of the SyncId.
        guid = Guid.NewGuid();
        replicaID = new SyncId(guid);
        using (FileStream fs = File.Open(syncFilePath, FileMode.Create))
        using (StreamWriter sw = new StreamWriter(fs))
        {
            sw.WriteLine(guid.ToString());
        }
    }
    else
    {
        // Read the SyncID from the existing file.
        using (FileStream fs = File.Open(syncFilePath, FileMode.Open))
        using (StreamReader sr = new StreamReader(fs))
        {
            string guidString = sr.ReadLine();
            guid = new Guid(guidString);
            replicaID = new SyncId(guid);
        }
    }

    return replicaID;
}

The source code above is just a simple example of synchronizing contents in one folder with that of another. There can be more advanced requirements like excluding files from synchronization, recovering files which are deleted in synchronization, etc. For these you have to use the FileSyncScopeFilter and the FileSyncOptions while creating the FileSyncProvider object for source and destination replicas.

For excluding files from synchronization, you have to create an object of the FileSyncScopeFilter, add the filenames to be excluded from the synchronization and then pass the FileSyncScopeFilter object to the FileSyncProvider constructor while creating an object for the source replica.

For example if I was to exclude the file named "File.ID" from the synchronization in the above source code, I would have to do the following.

Add the following code just before creating the object named sourceReplica.

FileSyncScopeFilter scopeFilter = new FileSyncScopeFilter();
scopeFilter.FileNameExcludes.Add("File.ID");

// Modify the code for creating the sourceReplica object by passing the scopeFilter object to the constructor.
FileSyncProvider sourceReplica = new FileSyncProvider(sourceId, @"C:\Sync\Source\", scopeFilter, FileSyncOptions.None);

// That's all you do to exclude a file from synchronization. 
// You would be wondering what the FileSyncOptions.None does in the code above. 
// Well, FileSyncOptions is an enumeration that you use to specify additional information 
// on how the synchronization handles deletes, overwrites, etc. 

// For example, the synchronization process may lead to the deletion of file(s) 
// from the destination replica if the same no longer exists in the source replica. 
// In such cases, you may want to send the deleted file to the recycle bin for future reference. 
// The FileSyncProvider does not send the deleted file to the recycle bin by default, 
// but you can set the FileSyncOptions.RecycleDeletes option while creating the destReplica object 
// to send all deleted files to the recycle bin as follows:

FileSyncProvider destReplica = new FileSyncProvider(destId, @"C:\Sync\Destination\", scopeFilter, FileSyncOptions.RecycleDeletes);

Other FileSyncOptions that you may use are.

FileSyncOptions.RecycleOverwrites
FileSyncOptions.CompareFileStreams FileSyncOptions.ExplicitDetectChanges

Resources

You will need the Microsoft Sync Framework to create sync programs in .Net. You can download the same and other related resources from http://msdn2.microsoft.com/en-us/sync/default.aspx.

Conclusion

As the intention of this article is just to be a primer on the Microsoft Sync Framework, there are numerous options apart from the ones in this article that you can explore to make powerful synchronization applications.