Summary: Explains how to deploy and manage an Elastic Processing Unit (EPU)
IntroductionAn Elastic Processing Unit (EPU) is a Processing Unit with additional capabilities that simplify its deployment across multiple machines. Containers and machine resources such as Memory and CPU are automatically provisioned based on Memory and CPU requirements. When a machine failure occurs, or when scale requirements change, new machines are provisioned and the Processing Unit deployment distribution is balanced automatically. The PU scale is triggered by modifying the requirements through an API call. From that point in time the EPU is continuously maintained to satisfy the specified capacity (indefinitely, or until the next scale trigger). When a processing unit is undeployed, machines and resources which were provisioned and are no longer needed will scale-in. PrerequisitesThe Elastic Processing Unit is managed by a separate service called an Elastic Service Manager (ESM). This manager exists to enforce the elastic requirements. In future versions, the Elastic Service Manager will be merged with the Grid Service Manager (GSM). In current release (including 8.0.1), the Grid Service Agent should maintain one global instance of an ESM. OverviewAn Elastic Processing Unit is currently deployed using the Admin API. The deployment is done similar to deployment of a Processing Unit, but with additional capabilities exposed. The API defines elasticity from a resource perspective and not from Processing Unit Instance perspective. The resources - Memory and CPU - describe the scaling requirements. ElasticityElasticity is supported for Space Cannot resolve external resource into attachment. data-grid deployments, Stateful Cannot resolve external resource into attachment. and Stateless Cannot resolve external resource into attachment. Processing Unit deployments. Elasticity is achieved by calling a scale trigger for scaling up/down (adding or removing containers) or out/in (adding or removing machines) based on the Memory/CPU scaling requirements. Machine ProvisioningAn Elastic Machine Provisioning Cannot resolve external resource into attachment. implementation can be plugged-in to enable starting, stopping and discovering of virtual machines. One such open-source plugin is the Citrix Xen Server Cannot resolve external resource into attachment. - a virtualization software technology that enables multiple virtual machines to run on a single physical server. We have been using it internally for testing. Another alternative, is RackSpace Cannot resolve external resource into attachment. - a cloud provider offering a set of pooled computing resources, enabling an application to scale dynamically and increase its share of resources on-the-fly. To get you started on a simple "pool" of machines, the default plugin Cannot resolve external resource into attachment. scales based on machines discovered by the Lookup Service. A machine can be added to the "pool" by simply starting a Grid Service Agent on it. ScalingScaling is either done manually or eagerly. A manual scale is achieved by calling the scale API on a Processing Unit, and supplying the Memory/CPU requirements to satisfy. For example scaling from 2GB to 4GB, or scaling from 2 cores to 8 cores, or both. The scaling is enforced by first scaling up/down and only if requirements can't be met with the current set of resources, scaling out/in by requesting more/less machine resources. On the other hand, eager scaling will automatically use up any discovered resource - scaling the processing unit to consume as many resources as it can. For example, by starting up a new machine (with a Grid Service Agent), the processing unit will eagerly scale by starting containers occupying relocated/incremented instances. RebalancingBalancing a system is done in respect to the number of Memory and CPU cores used per machine. An unbalanced system can be caused by new scaling requirements or even machine failure. The Processing Unit instances would need to be relocated, incremented or decremented to fit the new system capacity. For Stateful and Space data-grid deployments, balancing is more complex - and is done with minimal or no disruption to the client application. Life-cycle managementA Processing Unit's instances are maintained until it is undeployed. Throughout the life-cycle of the processing unit, containers and machines can come and go, but the instances are always maintained. If a container suddenly terminates, a new container will be started. If a Machine is suddenly shutdown, a new machine will be provisioned. 1-2-3 Getting StartedTo get you started on a single machine, the following example shows how to deploy an Elastic Space Processing Unit and how it scales. 1 - Setting up the environmentThe Grid Service Agent should start and manage a global instance of an ESM, GSM and LUS. There should be only one global ESM instance.
Windows
set LOOKUPGROUPS=myGroup
set JSHOMEDIR=d:\gigaspaces
start cmd /c "%JSHOMEDIR%\bin\gs-agent.bat gsa.global.esm 1 gsa.gsc 0 gsa.global.gsm 1 gsa.global.lus 1"
Linux export LOOKUPGROUPS=myGroup
export JSHOMEDIR=~/gigaspaces
nohup ${JSHOMEDIR}/bin/gs-agent.sh gsa.global.esm 1 gsa.gsc 0 gsa.global.gsm 1 gsa.global.lus 1 > /dev/null 2>&1 &
2 - Deploying an Elastic Space data-gridUsing the Admin API we can discover the running services. Since discovery is asynchronous, we need to wait for at least one instance to be discovered.
// Wait for the discovery of the managers and at least one agent Admin admin = new AdminFactory().addGroup("myGroup").create(); admin.getGridServiceAgents().waitForAtLeastOne(); admin.getElasticServiceManagers().waitForAtLeastOne(); GridServiceManager gsm = admin.getGridServiceManagers().waitForAtLeastOne(); //deploy a Space - total memory 1GB, each container 128MB //initially start with a total of 512MB (i.e. 4 containers = 128MB x4 = 512MB) ProcessingUnit pu = gsm.deploy( new ElasticSpaceDeployment("mySpace") .maxMemoryCapacity(1,MemoryUnit.GIGABYTES) .memoryCapacityPerContainer(128,MemoryUnit.MEGABYTES) //initial scale .scale( new ManualCapacityScaleConfigurer() .memoryCapacity(512,MemoryUnit.MEGABYTES) .create()) //deploy on a single machine .singleMachineDeployment() ); //wait for all instances to be discovered pu.waitForSpace().waitFor(pu.getTotalNumberOfInstances()); ... 3 - Scale up by memory capacityThe initial scale is optional, and can be also invoked after the processing unit has been deployed. Containers won't be started until it is invoked.
... //manually trigger a scaling requirement //scale to a total of 768MB (i.e. 6 containers = 128MB x6 = 768MB) pu.scale( new ManualCapacityScaleConfigurer() .memoryCapacity(768,MemoryUnit.MEGABYTES) .create()); The same API can be used as a request to scale down/in - by specifying less memory. pu.scale(
new ManualCapacityScaleConfigurer()
.memoryCapacity(256,MemoryUnit.MEGABYTES)
.create());
Tutorial SlidesElastic Processing Unit tutorial: Section Contents |
![]() |
GigaSpaces.com - Legal Notice - 3rd Party Licenses - Site Map - API Docs - Forum - Downloads - Blog - White Papers - Contact Tech Writing - Gen. by Atlassian Confluence |