Summary: This section describes how to configure the replication granularity and what are the trade-offs of the various options.

Replication Granularity

By default, all data and operations are replicated to all target spaces that belong to the same replication group (symmetrical replication). You can define replication granularity to start at the space level and end up at the space object level. Below are the different replication granularity options:

  • Space level replication - using the replication transmission matrix you can set up asymmetrical replication relationships, where only specific operations are replicated from a source space to a specific target space(s).
  • Class level replication - using the partial replication option and the @SpaceClass (replicate=true) decoration, these replicate only specific class instances.
  • Space object level replication - using configuration, you can block specific space objects from being replicated.
Deploying a data-grid to use the partial replication mode can be done using the following command:
gs.sh deploy-space -cluster schema=partitioned-sync2backup total_members=1,1 
  -properties embed://cluster-config.groups.group.repl-policy.policy-type=partial-replication mySpace

Replication Groups and Replication Policy

There can be several replication groups in a cluster. However, each space can belong to one group only. The replication group determines the replication policy, which has the following main attributes:

  • Full or partial replication - in full replication mode, every space object written to the space is replicated to all other space members in the replication group. In partial replication mode, classes decorated with @SpaceClass (replicate=true) are replicated to other spaces in the replication group. This can improve the performance.
  • Replication to all or some spaces - all-to-all replication (fully symmetric replication) might be irrelevant in large clusters, and is often not necessary. You can define selective replication using the replication transmission matrix.
  • Notifications - defines whether or not notification templates are propagated, and whether or not replicated space objects should trigger notifications on the target space.
  • Replication mode - synchronous or asynchronous.

Since all of these attributes have usable defaults, the replication mechanism is extremely easy to use, yet flexible and powerful.

Replication Transmission Matrix

When a replication group is defined, by default, each member of the group replicates all changes to all other members. For example, consider a replication group of four members: SP1, SP2, SP3 and SP4. Below is the replication diagram:

Sometimes a finer replication definition is needed - some spaces do not need to replicate all operations and data to other spaces. For example, assume that SP1 is an agency, connected to the agent computers SP2, SP3, and SP4. SP1 replicates and receives replications from SP2, SP3 and SP4, but SP2, SP3 and SP4 do not replicate data and operations between them.

To provide this functionality, define a transmission matrix over the replication group. For each pair (source and target space) define the following:

  • Whether or not the pair replicates with each other.
  • Which operations are allowed to replicate - write (write a new space object/extend lease of existing space object), update, take (take/delete a space object/lease cancellation), and notify (replicate a notify template). By default, all operations are replicated.
  • Replication mode - asynchronous or synchronous. If using synchronous mode, you can choose to use multicast or unicast as the communication protocol. If using asynchronous mode, you can choose to use the sync_on_commit option.
  • Recovery mode - if enabled, the space to use for recovery when started.
  • Replication filter - you can define for each source/target pair the replication filter implementation class.

By using the transmission matrix, you can improve performance and reduce communication overheads.
Following is an example of a replication matrix definition with two spaces, where the replication from container1:sp1 node to container2:sp2 node occurs in synchronous mode with multicast, and the replication from container2:sp2 node to container1:sp1 node occurs in asynchronous unicast mode.

Controlling Replication at the Operation Level

When using a replicated topology, you can instruct the primary space (source) to replicate only specific operations into the backup (replica) space. This means that you can actually "switch off" replication activity for certain operations.
This can be done via:

  • Replication Filter.
  • cluster-config.groups.group.repl-policy.permitted-operations configuration.

GigaSpaces allows you to control replication at the operation level without implementing any replication filter. This is done by adding the <cluster-config.groups.group.repl-policy.permitted-operations> tag to your space cluster configuration. When using the cluster-config.groups.group.repl-policy.permitted-operations, specify a comma-separated list of the operations you want to be replicated to all backup (replica) spaces.

Here are the available operations (in the form they should appear with the permitted-operations parameter):

  • write
  • take
  • extend_lease
  • update
  • lease_expiration
  • notify
  • discard

With GigaSpaces replicated topology, the take and clear operations are identical. Therefore, referrals to the take operation in this section are also relevant for the clear operation.

For example:

<os-core:space id="space" url="/./mySpace" >
    <os-core:properties>
        <props>
             <prop key="cluster-config.groups.group.repl-policy.permitted-operations">write, extend_lease, lease_expiration, notify</prop>
        </props>
    </os-core:properties>
</os-core:space>

In the example above, the take and update operations are not replicated to all spaces, while the write, extend_lease, expire_lease, and notify_registration operations are replicated to all spaces.

An empty value means no operations to be replicated.

You can also specify that a certain operation be replicated in a distributed manner. See the following section.

Distributed Replication

In GigaSpaces, replicated operations are usually performed first on the primary space, and only then passed to the rest of the members - this ensures consistency. Distributed replication allows you to define that an operation is performed on all replicated spaces at once.

A typical use-case of this feature is with the take (clear) operation. Usually, the take operation is performed first on the primary space, then a list of UIDs is passed to the replica spaces, and the objects are taken from these. When working with large amounts of objects (hundreds of thousands, or millions), there might be a need for the replication to be faster. With distributed replication, the objects are taken from all the replicated spaces at once.

If you choose to use distributed replication, make sure that no other operations are performed on the cluster while a distributed operation (in this case, take) is being performed. Otherwise, you might encounter inconsistent data in some of the members.

To enable distributed replication, exclude the operation you want to perform from the permitted-operations list, and enable broadcast mode (using <broadcast-condition>unconditional</broadcast-condition>) - see below. For example, if you want to perform the take operation in a distributed manner, configure the following:

<os-core:space id="space" url="/./mySpace" >
    <os-core:properties>
        <props>
             <prop key="cluster-config.groups.group.repl-policy.permitted-operations">write, notify</prop>
             <prop key="cluster-config.groups.group.load-bal-policy.take.policy-type">hash-based</prop>
             <prop key="cluster-config.groups.group.load-bal-policy.take.broadcast-condition">unconditional</prop>
        </props>
    </os-core:properties>
</os-core:space>

As you can see in the configuration above, take is not specified as part of the permitted-operations tag. The listed operations (write, notify) are for the purpose of the example and aren't relevant in this case.

Replication Filters

You can call your own business logic whenever the data is replicated. For example, you can modify the space objects data, compress/decompress, or block specific operations and space objects from being replicated to other spaces. Your business logic is called whenever the replication packet leaves the source space (output event), and arrives at the target space (input event).
This is described under the Cluster Replication Filters section.

Replicating the Original State

Whenever you run in asynchronous replication mode and perform write and update operations, the target space might get the latest state of the space object, both for the write and the update operations. In order to replicate the space object's original state with every operation, not just the latest one, turn on the cluster-config.groups.group.repl-policy.repl-original-state option.

When this option is set to true, the replication mechanism maintains for every space object, its full state, together with the operation. For example, if a space object has been written to the space and updated several times within a specific replication batch, then the replication mechanism replicates the operations in their correct order, together with the space object's original state for each operation.

Take operation only replicate the object ID and does not hold any other information on the object (such as the object full state).

When this option is set to false, the replication mechanism replicates the latest state of the space object to the target space. For example, if a space object has been written to the space and updated several times within a specific replication batch, the replication mechanism replicates the operations in their correct order. However, the replication mechanism still uses the space object with its latest state for all operations.

The repl-original-state impacts FIFO
In order to perform FIFO-based operations when having a replicated space, make sure that the cluster-config.groups.group.repl-policy.repl-original-state value is set to true.
When using a sync replication
The repl-original-state is always true (setting it to false has no effect), even when the cluster uses mirror.
Only when cluster replication is async, repl-original-state can be set to false.
GigaSpaces.com - Legal Notice - 3rd Party Licenses - Site Map - API Docs - Forum - Downloads - Blog - White Papers - Contact Tech Writing - Gen. by Atlassian Confluence