Resolving Conflicts in your Data Sync app

Introduction

Applications that allow users to modify data while offline must deal with conflicts. A conflict occurs when two or more clients try to modify the same data in between synchronizations.

Example: A user modifies a record while offline. When back online, they discover that the record has been deleted by another user.

Discussions about conflicts can be broken down into two parts.

  • Conflict Detection is the ability of an application to detect the presence of outdated data which cannot be stored without possible data loss occuring.

  • Conflict Resolution is how the application handles the conflict and ensures the correct data is stored.

In most cases, the way conflicts are detected and resolved are specific to an application and the underlying data storage. You implement conflict detection exclusively in the code associated with mutations.

Voyager Server provides conflict detection on the server side while Voyager Client provides conflict resolution on the client side.

Detecting Conflicts on the Server

At a high level, there is a typical flow to detecting conflicts.

  1. A Mutation Occurs - A client tries to modify or delete an object on the server using a GraphQL mutation

  2. Read the Object - The server reads the current object that the client is trying to modify from the data source

  3. Conflict Detection - The server compares the current object with the data sent by the client to see if there is a conflict. The developer can choose how the comparison is performed.

Pluggable Conflict Detection

The aerogear/voyager-conflicts module uses the concept of pluggable conflict detection to help developers with the Conflict Detection steps regardless of their storage.

The conflict detection is enabled by Voyager Server, while the fetching and storing of data is the responsibility of the developer.

Pluggable conflict detection supports the following implementations:

  • VersionedObjectState - depends on the version field supplied in objects (the version field is used by default when importing conflictHandler). For details, please see: Version Based Conflict Detection

  • HashObjectState - depends on a hash calculated from the entire object. For details, please see: Hash Based Conflict Detection

These implementations are based on the ObjectState interface that can be extended to provide custom implementations for conflict detection.

Prerequisites
  • GraphQL server with resolvers.

  • Database or any other form of data storage that can cause data conflicts. AeroGear recommends that you store data in a secure location. If you use a database, it is your responsibility to administer, maintain and backup that database. If you use any other form of data storage, you are responsible for backing up the data.

Version Based Conflict Detection

Version based conflict resolution is the recommended and simplest approach for conflict detection and resolution. The core idea is that every object has a version property with an integer value. A conflict occurs when the version number sent by the client does not match the version stored in the server. This means a different client has already updated the object.

Procedure
  1. Import the @aerogear/voyager-conflicts package.

    const { conflictHandler } = require('@aerogear/voyager-conflicts')
  2. Add a version field to the GraphQL type that should support conflict resolution. The version should also be stored in the data storage.

    type Task {
      title: String
      version: Int
    }
  3. Add an example mutation.

    type Mutation {
      updateTask(title: String!, version: Int!): Task
    }
  4. Implement the resolver. Every conflict can be handled using a set of predefined steps, for example:

    // 1. Read data from data source
    const serverData = db.find(clientData.id)
    // 2. Check for conflicts
    const conflict = conflictHandler.checkForConflicts(serverData, clientData)
    // 3. If there is a conflict, return the details to the client
    if(conflict) {
        throw conflict;
    }
    // 4. Save object to data source
    db.save(clientData.id, clientData)

In the example above, the throw statement ensures that the client receives all necessary data to resolve the conflict client-side. For more information about this data, please see Structure of the Conflict Error.

Since the conflict will be resolved on the client, it is not required to persist the data. However, if there is no conflict, the data sent by the client should be persisted. For more information on resolving the conflict client-side, please see: Resolving Conflicts on the Client.

Hash Based Conflict Detection

Hash based conflict detection is a mechanism to detect conflicts based on the total object being updated by the client. It does this by hashing each object and comparing the hashes. This tells the server whether or not the objects are equivalent and can be considered conflict free.

Procedure
  1. Import the @aerogear/voyager-conflicts package.

    const { HashObjectState } = require('@aerogear/voyager-conflicts')
  2. When using the HashObjectState implementation, a hashing function must be provided. The function signature should be as follows:

    const hashFunction = (object) {
      // Using the Hash library of your choice
      const hashedObject = Hash(object)
      // return the hashedObject in string form
      return hashedObject;
    }
  3. Provide this function when instantiating the HashObjectState:

    const conflictHandler = new HashObjectState(hashFunction)
  4. Implement the resolver. Every conflict can be handled using a set of predefined steps, for example:

    // 1. Read data from data source
    const serverData = db.find(clientData.id)
    // 2. Check for conflicts
    const conflict = conflictHandler.checkForConflicts(serverData, clientData)
    // 3. If there is a conflict, return the details to the client
    if(conflict) {
        throw conflict;
    }
    // 4. Save object to data source
    db.save(clientData.id, clientData)

In the example above, the throw statement ensures the client receives all necessary data to resolve the conflict client-side. For more information about this data please see Structure of the Conflict Error.

Since the conflict will be resolved on the client, it is not required to persist the data. However, if there is no conflict, the data sent by the client should be persisted. For more information on resolving the conflict client-side, please see: Resolving Conflicts on the Client.

Structure of the Conflict Error

The server needs to return a specific error when a conflict is detected containing both the server and client states. This allows the client to resolve the conflict.

 "extensions": {
        "code": "INTERNAL_SERVER_ERROR",
        "exception": {
          "conflictInfo": {
            "serverState": {
                 //..
            },
            "clientState": {
              //..
            }
          },
        }
 }

Resolving Conflicts on the Client

At a high level, there is a typical flow to resolving conflicts on the client.

  1. A Mutation Occurs - A client tries to modify or delete an object on the server using a GraphQL mutation.

  2. Read the Object - The server reads the current object the client is trying to modify from the data source (usually a database).

  3. Conflict Detection - The server compares the current object with the data sent by the client to see if there was a conflict. If there is a conflict, the server returns a response to the client containing information outlined in Structure of the Conflict Error

  4. Conflict Resolution - The client attempts to resolve this conflict and makes a new request to the server in the hope that this data is no longer conflicted.

The conflict resolution implementation requires the following additions to your application:

Developers can either use the default conflict resolution implementations, or implement their own conflict resolution implementations using the conflict resolution mechanism.

By default, when no changes are made on the same fields, the implementation will attempt to resend the modified payload back to the server. When changes on the server and on the client cover the same fields, one of the specified conflict resolution strategies can be used. The default strategy will apply client changes on top of the server side data. Developers can modify strategies to suit their needs.

Working with Conflict Resolution on the Client

To enable conflict resolution, the server side resolvers must be configured to perform conflict detection. Detection can rely on many different implementations and return the conflict error back to the client. For more information about how to do this, please see Server Side Conflict Detection.

Next, we must provide mutations with one extra piece of information that will be used to resolve any conflicts, by using the context. This piece of information is the returnType parameter. This parameter defines the Object type being operated on.

If using Data Sync’s offlineMutation you can provide this parameter directly as follows:

client.offlineMutation({
  ...
  returnType: 'Task'
  ...
})

or if using Apollo’s mutate function:

client.mutate({
  ...
  context: {
    returnType: 'Task'
  }
  ...
})

The client will then automatically resolve them based on the current strategy and notify listeners if the developer supplied any.

Conflict resolution will work with the recommended defaults and does not require any specific handling on the client.

For advanced use cases, the conflict implementation may be customised by supplying a custom conflictProvider in the application config. See Conflict Resolution Strategies below.

Default Conflict Implementation

By default, conflict resolution is configured to rely on a version field on each GraphQL type. A version field will also need to be saved to the database in order to detect changes on the server.

For example:

type User {
  id: ID!
  version: String!
  name: String!
}

The version field is controlled on the server and will map the last version that was sent from the server. All operations on the version field happen automatically. However, users need to make sure that the version field is always passed to the server for mutations that supports conflict resolution:

type Mutation {
  updateUser(id: ID!, version: String!): User
}

Conflict Resolution Strategies

Data Sync allows developers to define custom conflict resolution strategies. You can provide custom conflict resolution strategies to the client in the config by using the provided ConflictResolutionStrategies type. By default developers do not need to pass any strategy as UseClient is the default. Custom strategies can also be used to provide different resolution strategies for certain operations:

let customStrategy = {
  resolve = (base, server, client, operationName) => {
    let resolvedData;
    switch (operationName) {
      case "updateUser":
        delete client.socialKey
        resolvedData = Object.assign(base, server, client)
        break
      case "updateRole":
        client.role = "none"
        resolvedData = Object.assign(base, server, client)
        break
      default:
        resolvedData = Object.assign(base, server, client)
    }
    return resolvedData
  }
}

This custom strategy provides two distinct strategies to be used when a conflict occurs. They are based on the name of the operation to give developers granular control. To use this custom strategy, pass it as an argument to conflictStrategy in your config object:

let config = {
...
  conflictStrategy: customStrategy
...
}

Listening to Conflicts

Data Sync allows developers to receive information about the data conflict that has occurred between the client and the server.

When a conflict occurs, Data Sync attempts to perform a field level resolution of data - it will check all fields of its type to see if both the client or server has changed the same field. The client can be notified in one of two scenarios.

If both client and server have changed any of the same fields, the conflictOccurred method of the ConflictListener will be triggered.

If the client and server have not changed any of the same fields, and the data can be easily merged, the mergeOccurred method of your ConflictListener will be triggered.

Developers can supply their own conflictListener implementation

class ConflictLogger implements ConflictListener {
  conflictOccurred(operationName, resolvedData, server, client) {
    console.log("Conflict occurred with the following:")
    console.log(`data: ${JSON.stringify(resolvedData)}, server: ${JSON.stringify(server)}, client: ${JSON.stringify(client)}, operation:  ${JSON.stringify(operationName)}`);
  }
  mergeOccurred(operationName, resolvedData, server, client) {
    console.log("Merge occurred with the following:")
    console.log(`data: ${JSON.stringify(resolvedData)}, server: ${JSON.stringify(server)}, client: ${JSON.stringify(client)}, operation:  ${JSON.stringify(operationName)}`);
  }
}

let config = {
...
  conflictListener: new ConflictLogger()
...
}

Pre-Conflict Errors

Data Sync provides a mechanism for developers to check for a 'pre-conflict' before a mutation occurs. It does this out of the box by checking whether or not the data being sent conflicts locally. This happens when a mutation (or the act of creating a mutation) is initiated and before being sent new data arrives via subscriptions.

An example of when this is useful could be when a user performs the following actions:

  1. Open a form on their device

  2. Begin working on the pre-populated data on this form

  3. While working, the client receives new data from the server from subscriptions

  4. The client is now conflicted but the user is unaware.

  5. When the user presses submit Data Sync notices that their data is conflicted and provides the developer with a way to warn the user.

To use this feature, and therefore potentially save unecessary round-trips to the server with data which is definitely conflicted, developers can make use of the error returned by Data Sync. An example of how developers can use this error can be seen below.

return client.offlineMutation({
  ...
}).then(result => {
  // handle the result
}).catch(error => {
  if (error.networkError && error.networkError.localConflict) {
    // handle pre-conflict here by potentially
    // providing an alert with a chance to update data before pressing send again
  }
})