Summary: Explains how to deploy your processing unit onto the GigaSpaces
Service Grid to get automated SLA management and self-healing capabilities
OverviewDeploying your processing unit to the service grid is the preferred way to run in your production environment. The service grid provides the following main benefits to every processing unit deployed onto it:
The Deployment ProcessOnce built according to the processing unit directory structure, the processing unit can be deployed via the various deployment tools available in GigaSpaces XAP (UI, CLI, Ant, Maven or the Admin API). After you package the processing unit and deploy it via one of the deployment tools, the deployment tool uploads it to all the running GSMs, where it is extracted and provisioned to the GSCs.
Distribution of Processing Unit Binaries to the Running GSCsBy default, when a processing unit instance is provisioned to run on a certain GSC, the GSC downloads the processing unit archive from the GSM into the <GigaSpaces Root>/work/deployed-processing-units directory (The location of this directory can be overridden via the com.gs.work system property). Downloading the processing unit archive to the GSC is the recommended option, but it can be disabled. In order to disable it, the pu.download deployment property should be set to false. This will not download the entire archive to the GSC, but will force the GSC to load the processing unit classes one at a time from the GSM via a URLClassLoader. Processing Unit Deployment using various Deployment ToolsGigaSpaces provides several options to deploy a processing unit onto the Service Grid. Below you can find a simple deployment example with the various deployment tools for deploying a processing unit archive called myPU.jar located in the /opt/gigaspaces directory:
Admin API
Deploying via code is done using the GigaSpaces Admin API. The following example shows how to deploy the myPU.jar processing unit using one of the available GSMs. For more details please consult the documentation and javadoc of the Admin API. Admin admin = new AdminFactory().addGroup("myGroup").create(); File puArchive = new File("/opt/gigaspaces/myPU.jar"); ProcessingUnit pu = admin.getGridServiceManagers().deploy( new ProcessingUnitDeployment(puArchive)); Ant Deploying with Ant is based on the org.openspaces.pu.container.servicegrid.deploy.Deploy class (in fact, all of the deployment tools use this class although it is not exposed directly to the end user). In the below example we create an Ant macro using this class and use it to deploy our processing unit. The deploy class is executable via its main() method, and can accept various parameters to control the deployment process. These parameters are identical to these of the deploy CLI command, for a complete list of the available parameters please consult the deploy CLI reference documentation.. <deploy file="/opt/gigaspaces/myPU.jar" /> <macrodef name="deploy"> <attribute name="file"/> <sequential> <java classname="org.openspaces.pu.container.servicegrid.deploy.Deploy" fork="false"> <classpath refid="all-libs"/> <arg value="-groups" /> <arg value="mygroup" /> <arg value="@{file}"/> </java> </sequential> </macrodef> GigaSpaces CLI Deploying via the CLI is based on the deploy command. This command accepts various parameters to control the deployment process. These parameters are documented in full in the deploy CLI reference documentation.. > <gigaspaces root>/bin/gs.sh(bat) deploy myPU.jar GigaSpaces UI
Elastic Processing Unit Deployment using the Admin APIWhen deploying a partitioned Processing Unit or a paritioned Space it is recommended to use the new Elastic Processing Unit. This can be done via the Admin API. The following example shows how to deploy a processing unit as an Elastic PU. Step 1 - Start a GigaSpaces agent on each machine:You should have one GigaSpaces agent running the ESM. No GSCs should be started.
Windows
rem Agent deployment that potentially can start management processes
set LOOKUPGROUPS=myGroup
set JSHOMEDIR=d:\gigaspaces
start cmd /c "%JSHOMEDIR%\bin\gs-agent.bat gsa.global.esm 1 gsa.gsc 0 gsa.global.gsm 2 gsa.global.lus 2"
Linux # Agent deployment that potentially can start management processes
export LOOKUPGROUPS=myGroup
export JSHOMEDIR=~/gigaspaces
nohup ${JSHOMEDIR}/bin/gs-agent.sh gsa.global.esm 1 gsa.gsc 0 gsa.global.gsm 2 gsa.global.lus 2 > /dev/null 2>&1 &
Step 2 - Run the deployment code:// Wait for the discovery of the managers and at least one agent Admin admin = new AdminFactory().addGroup("myGroup").create(); admin.getGridServiceAgents().waitForAtLeastOne(); admin.getElasticServiceManagers().waitForAtLeastOne(); GridServiceManager gsm = admin.getGridServiceManagers().waitForAtLeastOne(); // Deploy the Elastic Processing Unit. // Set the maximum memory and CPU capacity and initial capacity File puArchive = new File("/opt/gigaspaces/myPU.jar"); ProcessingUnit pu = gsm.deploy( new ElasticStatefulProcessingUnitDeployment(puArchive) .memoryCapacityPerContainer(16,MemoryUnit.GIGABYTES) .maxMemoryCapacity(512,MemoryUnit.GIGABYTES) .maxNumberOfCpuCores(32) // uncomment when working with a single machine agent //.singleMachineDeployment() // set the initial memory and CPU capacity .scale(new ManualCapacityScaleConfigurer() .memoryCapacity(128,MemoryUnit.GIGABYTES) .numberOfCpuCores(8) .create()) ); // Wait until the deployment is complete. pu.waitForSpace().waitFor(pu.getTotalNumberOfInstances()); Step 3 - Scale the PU by increasing the memory and CPU capacity:// Scale in runtime to the desired memory and CPU capacity pu.scale( new ManualCapacityScaleConfigurer() .memoryCapacity(256,MemoryUnit.GIGABYTES) .numberOfCpuCores(16) .create() ); Step 4 - To undeploy the Processing Unit run the following:// Wait up to 10 seconds for the processing unit to be discovered ProcessingUnit pu = admin.getProcessingUnits() .waitFor("myPU",10,TimeUnit.SECONDS); if (pu != null) { // Wait up to 3 minutes for the processing unit // (including all instances) to be gracefully undeployed // For versions prior to 8.0.5 (7.x - 8.0.4) use pu.undeploy() pu.undeployAndWait(3, TimeUnit.MINUTES); } Since we are undeploying an Elastic Processing Unit, this will also terminate all the GSCs hosting the PU. Running the deployment code from the command line is very convinient. Rename the java file extension to groovy, and run it using the following command:
Windows
%JSHOMEDIR%\tools\groovy\bin\groovy deploy.groovy Linux ${JSHOMEDIR}tools/groovy/bin/groovy deploy.groovy Hot DeployTo enable business continuity in a better manner, having system upgrade without any downtime, here is a simple procedure you should follow when you would like to perform a hot deploy, upgrading a PU that includes both a business logic and a collocated embedded space: You can script the above procedure via the Administration and Monitoring API, allowing you to perform system upgrade without downtime. Restart a running PU via the GS-UITo restart a running PU (all instances) via the GS-UI you should: Restart a running PU via the Admin APIThe ProcessingUnitInstance includes few restart methods you may use to restart a PU instance: restart()
restartAndWait()
restartAndWait(long timeout, TimeUnit timeUnit)
Here is an example code that is using the ProcessingUnitInstance.restart to restart the entire PU instances in an automatic manner: import java.util.concurrent.TimeUnit; import java.util.logging.Logger; import org.openspaces.admin.Admin; import org.openspaces.admin.AdminFactory; import org.openspaces.admin.pu.ProcessingUnit; import org.openspaces.admin.pu.ProcessingUnitInstance; import com.gigaspaces.cluster.activeelection.SpaceMode; public class PUReatartMain { static Logger logger = Logger.getLogger("PUReatart"); public static void main(String[] args) { String puToRestart = "myPU"; Admin admin = new AdminFactory().createAdmin(); ProcessingUnit processingUnit = admin.getProcessingUnits().waitFor( puToRestart, 10, TimeUnit.SECONDS); if (processingUnit == null) { logger.info("can't get PU instances for "+puToRestart ); admin.close(); System.exit(0); } // Wait for all the members to be discovered processingUnit.waitFor(processingUnit.getTotalNumberOfInstances()); ProcessingUnitInstance[] puInstances = processingUnit.getInstances(); // restart all backups for (int i = 0; i < puInstances.length; i++) { if (puInstances[i].getSpaceInstance().getMode() == SpaceMode.BACKUP) { restartPUInstance(puInstances[i]); } } // restart all primaries for (int i = 0; i < puInstances.length; i++) { if (puInstances[i].getSpaceInstance().getMode() == SpaceMode.PRIMARY) { restartPUInstance(puInstances[i]); } } admin.close(); System.exit(0); } private static void restartPUInstance( ProcessingUnitInstance pi) { final String instStr = pi.getSpaceInstance().getMode() != SpaceMode.PRIMARY?"backup" : "primary"; logger.info("restarting instance " + pi.getInstanceId() + " on " + pi.getMachine().getHostName() + "[" + pi.getMachine().getHostAddress() + "] GSC PID:" + pi.getVirtualMachine().getDetails().getPid() + " mode:" + instStr + "..."); pi = pi.restartAndWait(); logger.info("done"); } } Application Deployment and Processing Unit Dependencies
Web User InterfaceEach application gets its own page in the web user interface which includes a symbol for each processing unit, and a directed arrow showing the dependencies between the deployed processing unites. See the example below where the feeder PU depends on the Space PU (feeder->space). The feeder instance is deployed only after the space is deployed Deployment DependenciesThe sample code below deploys an application named "data-app" which consists of a space and a feeder processing unit. The feeder processing unit instances are deployed only after the space deployment is complete (each partition has both a primary and a backup space instance).
Admin API
Admin admin = new AdminFactory().addGroup("myGroup").create(); File feederArchive = new File("/opt/gigaspaces/myfeeder.jar"); Application dataApp = admin.getGridServiceManagers().deploy( new ApplicationDeployment("data-app") .addProcessingUnitDeployment( new ProcessingUnitDeployment(feederArchive) .name("feeder") .addDependency("space")) .addProcessingUnitDeployment( new SpaceDeployment("space")) ); for (ProcessingUnit pu : dataApp.getProcessingUnits()) { pu.waitFor(pu.getTotalNumberOfInstances()); } Admin API + XML Since XAP v9.0.1 the processing unit dependencies can be described using an XML file. Admin admin = new AdminFactory().addGroup("myGroup").create(); //The dist zip file includes feeder.jar and application.xml file File application = new File("/opt/gigaspaces/examples/data/dist.zip"); //Application folders are supported as well //File application = new File("/opt/gigaspaces/examples/data/dist"); Application dataApp = admin.getGridServiceManagers().deploy( new ApplicationFileDeployment(application) .create() ); for (ProcessingUnit pu : dataApp.getProcessingUnits()) { pu.waitFor(pu.getTotalNumberOfInstances()); } Here is the content of the application.xml file (that resides alongside feeder.jar in dist.zip): <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xmlns:os-admin="http://www.openspaces.org/schema/admin" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd http://www.openspaces.org/schema/admin http://www.openspaces.org/schema/9.0/admin/openspaces-admin.xsd"> <context:annotation-config /> <os-admin:application name="data-app"> <os-admin:space name="space" /> <os-admin:pu processing-unit="feeder.jar"> <os-admin:depends-on name="space"/> </os-admin:pu> </os-admin:application> </beans> GigaSpaces CLI + XML Since XAP v9.0.1 the processing unit dependencies can be described using an XML file. > <gigaspaces root>/bin/gs.sh(bat) deploy-application examples/data/dist.zip Here is the content of the application.xml file (that resides alongside feeder.jar in dist.zip): <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xmlns:os-admin="http://www.openspaces.org/schema/admin" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd http://www.openspaces.org/schema/admin http://www.openspaces.org/schema/9.0/admin/openspaces-admin.xsd"> <context:annotation-config /> <os-admin:application name="data-app"> <os-admin:space name="space" /> <os-admin:pu processing-unit="feeder.jar"> <os-admin:depends-on name="space"/> </os-admin:pu> </os-admin:application> </beans> The reason for imposing this dependency is that the space proxy bean in the feeder processing unit would fail to initialize if the space is not available. However, this restriction could be too severe since the feeder is a singleton processing unit. For example, if a container with the feeder and a space instance fails, the space is still available (the backup is elected to primary). However the feeder is not re-deployed until the space has all instances running, which will not happen unless a container is (re)started. Adaptive SLAThe feeder can relax this restriction, by specifying a dependency of at least one instance per partition. Now the feeder is redeployed as long as the space has a minimum of one instance per partition. The downside of this approach is that during initial deployment there is a small time gap in which the feeder writes data to the space while there is only one copy of the data (one instance per partition).
Admin API
Admin admin = new AdminFactory().addGroup("myGroup").create(); File feederArchive = new File("/opt/gigaspaces/myfeeder.jar"); // The ProcessingUnitDependenciesConfigurer supports dependencies on a minimum number of instances, // on a minimum number of instances per partition, or waiting for a deployment of another processing unit to complete. Application dataApp = admin.getGridServiceManagers().deploy( new ApplicationDeployment("data-app") .addProcessingUnitDeployment( new ProcessingUnitDeployment(feederArchive) .name("feeder") .addDependencies(new ProcessingUnitDeploymentDependenciesConfigurer() .dependsOnMinimumNumberOfDeployedInstancesPerPartition("space",1) .create()) .addProcessingUnitDeployment( new SpaceDeployment("space")) ); for (ProcessingUnit pu : dataApp.getProcessingUnits()) { pu.waitFor(pu.getTotalNumberOfInstances()); } Admin API + XML Since XAP v9.0.1 the processing unit dependencies can be described using an XML file. Admin admin = new AdminFactory().addGroup("myGroup").create(); //The dist zip file includes feeder.jar and application.xml file File application = new File("/opt/gigaspaces/examples/data/dist.zip"); //Application folders are supported as well //File application = new File("/opt/gigaspaces/examples/data/dist"); Application dataApp = admin.getGridServiceManagers().deploy( new ApplicationFileDeployment(application) .create() ); for (ProcessingUnit pu : dataApp.getProcessingUnits()) { pu.waitFor(pu.getTotalNumberOfInstances()); } Here is the content of the application.xml file (that resides alongside feeder.jar in dist.zip): <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xmlns:os-admin="http://www.openspaces.org/schema/admin" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd http://www.openspaces.org/schema/admin http://www.openspaces.org/schema/9.0/admin/openspaces-admin.xsd"> <context:annotation-config /> <os-admin:application name="data-app"> <os-admin:space name="space" /> <os-admin:pu processing-unit="feeder.jar"> <os-admin:depends-on name="space" min-instances-per-partition="1"/> </os-admin:pu> </os-admin:application> </beans> GigaSpaces CLI + XML Since XAP v9.0.1 the processing unit dependencies can be described using an XML file. gigaspaces/bin/gs.sh deploy-application gigaspaces/examples/data/dist.zip Here is the content of the application.xml file (that resides alongside feeder.jar in dist.zip): <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xmlns:os-admin="http://www.openspaces.org/schema/admin" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd http://www.openspaces.org/schema/admin http://www.openspaces.org/schema/9.0/admin/openspaces-admin.xsd"> <context:annotation-config /> <os-admin:application name="data-app"> <os-admin:space name="space" /> <os-admin:pu processing-unit="feeder.jar"> <os-admin:depends-on name="space" min-instances-per-partition="1"/> </os-admin:pu> </os-admin:application> </beans> Undeployment OrderWhen it is time to undeploy dataApp, we would like the feeder to undeploy first, and only then the space. This would reduce the number of false warnings in the feeder complaining that the space is not available. The command dataApp.undeployAndWait(3, TimeUnit.MINUTES) undeploys the application in reverse dependency order and waits for at most 3 minutes for all processing unit instances to gracefully shutdown. In the example above, the feeder instance will complete the teardown of all the spring beans, and only then the space would undeploy. Monitoring Deployment Progress
Admin admin = new AdminFactory().create(); admin.getProcessingUnits().getProcessingUnitInstanceProvisionStatusChanged().add( listener ) //or admin.getProcessingUnits().getProcessingUnit("xyz").getProcessingUnitInstanceProvisionStatusChanged( listener ) //or admin.addEventListener(new ProcessingUnitInstanceProvisionStatusChangedEventListener() { @Override public void processingUnitInstanceProvisionStatusChanged( ProcessingUnitInstanceProvisionStatusChangedEvent event) { ProvisionStatus newStatus = event.getNewStatus(); ... } }); A compound listener (implements several interfaces) can be registered using the Admin.addEventListener(...). Monitoring Processing Unit instance Fault-detectionUsing the member-alive-indicator (see Monitoring the Liveness of Processing Unit Instances ) the Grid Service Manager (GSM) actively monitors each processing unit instance. When an "is alive" check fails, it suspects that the processing unit instance is no longer alive, and retries to contact it (using the configured retries and timeouts in pu.xml under os-sla:member-alive-indicator). When all retries fail, the GSM reports that it detected a failure and tries to re-deploy it on an available Grid Service Container (GSC). These member-alive-indicator transitions are reflected using the Admin API MemberAliveIndicatorStatus. Use the API to register for status changed events, and better visibility of GSM decisions based on the fault-detection mechanism. An alert is fired upon a fault- detection trigger, also visible in Web User Interface. The MemberAliveIndicatorStatus has three states: ALIVE, SUSPECTING and FAILURE. The transition from SUSPECTING to FAILURE is final. From this state the processing unit instance is considered not alive. The transition from SUSPECTING back to ALIVE can occur if one of the retries succeeded in contacting the processing unit instance. Admin admin = new AdminFactory().create(); admin.getProcessingUnits().getProcessingUnitInstanceMemberAliveIndicatorStatusChanged().add( listener ) //or admin.getProcessingUnits().getProcessingUnit("xyz").getProcessingUnitInstanceMemberAliveIndicatorStatusChanged( listener ) //or admin.addEventListener(new ProcessingUnitInstanceMemberAliveIndicatorStatusChangedEventListener() { @Override public void processingUnitInstanceMemberAliveIndicatorStatusChanged ( ProcessingUnitInstanceMemberAliveIndicatorStatusChangedEvent event) { MemberAliveIndicatorStatus newStatus = event.getNewStatus(); ... } }); A compound listener (implements several interfaces) can be registered using the Admin.addEventListener(...). |
![]() |
GigaSpaces.com - Legal Notice - 3rd Party Licenses - Site Map - API Docs - Forum - Downloads - Blog - White Papers - Contact Tech Writing - Gen. by Atlassian Confluence |