Summary: An example of a custom elastic scale handler implementation
The Elastic Middleware Services functionality is provided as a technology preview in XAP 7.1. As such, it is subject to API and behavior changes in the next XAP releases without going the usual deprecation process of the XAP API.



Our custom Elastic Scale Handler will startup agents on our lab machines when there is a need to scale out, and kill the agents when there is a need to scale in. We would like to limit the machines used by our implementation declaratively.

Deployment

Our example implementation class is GigaSpacesLabElasticScaleHandler. At deploy time, we will specify the class name and properties to be passed to the implementation.

Discover the ESM

Admin admin = new AdminFactory().createAdmin();
ElasticServiceManager esm = admin.getElasticServiceManagers().waitForAtLeastOne();

Deploy "myGrid"

ProcessingUnit pu = esm.deploy(
    new ElasticDataGridDeployment("mygrid")
            .elasticScaleHandler(
                    .new ElasticScaleHandlerConfig(GigaSpacesLabElasticScaleHandler.class.getName())
                            .addProperty("machines", "lab-12,lab-13,lab-36,lab-37") ));

Wait for deployment

Space space = pu.getSpace();
space.waitFor(pu.getSpace().getTotalNumberOfInstances());
GigaSpace gigaSpace = space.getGigaSpace();

Setup

We are going to use SSH to connect to the machines to startup the agents. For that we can use an external library (JSCH) that simplifies the use of SSH.
To run, you will need Elastic Handler implementation class and dependent jars in the ESM classpath.

Place these under the following library folder to be loaded by the ESM when it starts up:

<GigaSpace>/lib/platform/esm/my-ElasticScaleHandlerImpl.jar
<GigaSpace>/lib/platform/esm/jsch-0.1.42.jar
<GigaSpace>/lib/platform/esm/ant.jar
<GigaSpace>/lib/platform/esm/ant-jsch.jar

The last two jars, ant.jar & ant-jsch.jar, are part of the standard ant distribution (for more information - see Using SSH).

Implementation

The ElasticScaleHandler interface consists of a very simple API.

public void init(ElasticScaleHandlerConfig config);
public boolean accept(Machine machine);
public void scaleOut(ElasticScaleHandlerContext context(;
public void scaleIn(Machine machine);

Initialization

The init method is passed a configuration constructed as part of the deployment.
From it we can extract the properties - in this case the machines to be used.

public void init(ElasticScaleHandlerConfig config) {
    String machinesProp = config.getProperty("machines");
    String[] machines = machinesProp.split(",");
    for (String machine : machines) {
        freeMachines.addMachine(machine);        
    }
}

Filtering

The accept method receives a machine which is a candidate to start a GSC on. Machines are discovered by the lookup service and are automatically considered as candidates for a new GSC. This method acts as a filter for machines we would like to exclude (by returning false).
A scale out request will be called only if none of the discovered machines can be used.

For example, we can check that the discovered machine was from our pool.

public boolean accept(Machine machine) {
    if (freeMachines.containsMachine(machine) ||
        allocatedMachines.containsMachine(machine)) {
            return true;
    } else {
        return false;  
    }
}

Scaling Out

The scaleOut method parameter is a context from which we can extract information available to the ESM. This method will be called requesting a new machine. Implementation may choose to block the call until the machine is loaded, but this will block further progress of the ESM to other processing units. It is recommended to perform the call asynchronously, since starting a machine may take up to 15 minutes on some environments. As long as the machine has not been discovered, the ESM will repeatedly call this method until a new machine has been allocated. It is up to the implementation to maintain the state and not allocate new resources before pending allocations have completed.

public void scaleOut(ElasticScaleHandlerContext context) {
    if (freeMachines.isEmpty()) {
        logger.fine("Can't scale-up - no more machines");
        return;
    }
   
    if (!singleThreadPool.hasPendingRequest()) {
        logger.fine("Rejecting request, until previous machine is up");
        return;
    }
        
        
    final String freeMachine = freeMachines.removeFirst();
    logger.info("Request to scale out - using: " + freeMachine);

    singleThreadPool.execute(new Runnable() {
        public void run() {
            try {
                runSSHCommand(freeMachine, "./gs-agent.sh gsa.global.gsm 0 gsa.gsc 0");
                allocMachines.addMachine(freeMachine);
            } catch (Exception e) {
                logger.log(Level.SEVERE, "Failed to scale up on machine: " + freeMachine, e);
                freeMachines.addMachine(freeMachine);
            }
        }
    });
}

Scaling In

The scaleIn method parameter denotes a machine we would like to remove. Implementations may choose to ignore the request, or decide to remove the machine based on cost/benefit factors. Some environments have a pay-per-use by the hour, and it would be more cost effective to keep the machine up.
Implementation may choose to block until it has scaled in or handle the request asynchronously.

public void scaleIn(Machine machine) {
    if (!allocMachines.containsMachine(machine)) {
        return;
    }
        
    logger.info("Request to scale in - using: " + hostName);
        
    singleThreadPool.execute(new Runnable() {
        public void run() {
            try {
                runSSHCommand(hostName, "killall -9 java");
                freeMachines.addMachine(hostName);
            } catch (Exception e) {
                logger.log(Level.SEVERE, "Failed to scale in " + hostName, e);
            }
        }
    });
}

Using SSH

You can use SSH directly with the JSCH API, but it is simpler to use the Ant JSCH extension.
For this, you will need the ant.jar and ant-jsch.jar jars. Otherwise, you can exclude them.

public void runSSHCommand(String host, String command) {
    SSHExec task = new SSHExec();

    task.setCommand(command);
    task.setHost(host);
    task.setTrust(true);
    task.setUsername("<Username>");
    task.setPassword("<Password>");
    task.setTimeout(5000);
    
    task.execute();
}

If the timeout expires, execute() will throw a BuildException.

GigaSpaces.com - Legal Notice - 3rd Party Licenses - Site Map - API Docs - Forum - Downloads - Blog - White Papers - Contact Tech Writing - Gen. by Atlassian Confluence