Summary: External Data Source API. This page explains how to implement custom external data source.
Overview
The External Data Source API provide three major functionalities, each is defined by a specific interface:
External Data Source ComponentsBased on your application persistency requirements - you may need to implement all EDS interfaces or only few of them. Data RetrievalThere are two options to implement Data retrieval: SQL-based APIThe SQLDataProvider allows you to implement a full-featured EDS with databases that support SQL queries. Object-based APIThe DataProvider should be used for databases that don't support SQL queries.
Data PersistencyData Persistency can be implemented as Synchronous or Asynchronous. In Synchronous approach, DataPersister logic is running in the same space instance, in Asynchronous approach DataPersister logic runs in a Mirror Service (see Asynchronous Persistency with the Mirror) In either approach, DataPersistor interface is implemented and configured appropriately. There are two options to implement Data Persistency:
Persistency Transactional ModeIf the EDS implements the BulkDataPersister interface, data persistency operations that fall under a transaction are translated to a list of BulkItem(operation+data), and passed to executeBulk(List<BulkItem> items). Persistency Non-Transactional ModeIn the EDS does not implement BulkDataPersister interface, or space operations are not wrapped under a transaction, persistence operations are invoked on a per space operation basis. The DataPersister interface has three methods for each space operation:
Initialization and ShutdownInitialization and Shutdown done via the ManagedDataSource interface:
Multi Space class EDS implementationYou should have a single EDS implementation per space (or space cluster). This implementation should handle all the different space classes the space handling. In such a case, your EDS classes (DataProvider, etc) should not use generics. This how you could load/persist any type of object. As part of the executeBulk, initialLoad , etc methods you should check the data type (instance of) and have the specific code to handle the specific class object (if needed). API MatrixEDS Implements:SQLDataProvider, DataPersister
EDS Implements:DataProvider, DataPersister
EDS Implements:DataProvider,SQLDataProvider,DataPersister
Custom EDS implementations looking for transactional behavior on persisting data should implement BulkDataPersister Interface and invoke the space operations with appropriate transaction boundaries. All update operations (write, writeMultiple, take, takeMultiple) that fall under a space transaction will be invoked as one executeBulk call. If a Custom EDS implements BulkDataPersister but the space operations are not invoked under transaction, above mapping still holds true.
All External Data Source APIs
ExampleSee below a simple EDS implementation example. The space with this example running in a partitioned mode (2 nodes) using LRU Caching policy mode.
The EDS implementation class
This class includes a simple main method that will allow you to test it when running within your IDE debugger: package com.test.eds; import java.util.ArrayList; import java.util.List; import java.util.Properties; import java.util.logging.Logger; import org.openspaces.core.GigaSpace; import org.openspaces.core.GigaSpaceConfigurer; import org.openspaces.core.space.UrlSpaceConfigurer; import com.gigaspaces.datasource.DataIterator; import com.gigaspaces.datasource.DataProvider; import com.gigaspaces.datasource.DataSourceException; import com.gigaspaces.datasource.ManagedDataSource; public class MyEDS implements DataProvider<MyData> { static Logger logger = Logger.getLogger("MyEDS"); public static void main(String[] args) { try { GigaSpace spaceNode1 = startClusterNode(1); GigaSpace spaceNode2 = startClusterNode(2); logger.info("Partition 1 has " + spaceNode1.count(null) + " objects"); logger.info("Partition 2 has " + spaceNode2.count(null) + " objects"); GigaSpace gigaspace = new GigaSpaceConfigurer( new UrlSpaceConfigurer("jini://*/*/space")).gigaSpace(); MyData obj = new MyData(); obj.setId("a"); obj.setData("aaaaa"); obj.setRouting(1); logger.info("APP-write"); gigaspace.write(obj); MyData templ1 = new MyData(); templ1.setId("1"); templ1.setRouting(1); MyData templ2 = new MyData(); templ2.setId("a"); templ2.setRouting(1); MyData templ3 = new MyData(); templ3.setId("b"); templ3.setRouting(1); logger.info("APP-read Loaded object from EDS"); MyData ret = gigaspace.read(templ1); if (ret == null) logger.info("Could not read loaded object"); else logger.info("read loaded object - OK!"); logger.info("APP-read"); ret = gigaspace.read(templ2); if (ret == null) logger.info("Could not read object"); else logger.info("read object - OK!"); logger.info("APP-read unloaded object"); ret = gigaspace.read(templ3); if (ret == null) logger.info("Can't find the object - OK"); else logger.info("read loaded object - OK!"); logger.info("APP-readMultiple"); MyData retArray[] = gigaspace.readMultiple(new MyData(), Integer.MAX_VALUE); } catch (Exception e) { e.printStackTrace(); } } public DataIterator<MyData> iterator(MyData query) throws DataSourceException { logger.info(">> Partition ID:" + partitionID + " EDS-iterator"); return new MyDataIterator(query); } public MyData read(MyData query) throws DataSourceException { logger.info(">> Partition ID:" + partitionID + " EDS-read ID:" + query.getId()); return null; } int partitionID; public void init(Properties props) throws DataSourceException { logger.info(props.toString()); partitionID = Integer .valueOf( props.get(ManagedDataSource.STATIC_PARTITION_NUMBER) .toString()).intValue(); logger.info(">> Partition ID:" + partitionID + " EDS-init"); } public DataIterator<MyData> initialLoad() throws DataSourceException { logger.info(">> Partition ID:" + partitionID + " EDS-initialLoad"); List<MyData> initData = new ArrayList<MyData>(); // load the space with some data MyData obj = new MyData(); obj.setId("1"); obj.setData("1111111"); obj.setRouting(1); initData.add(obj); logger.info(">> Partition ID:" + partitionID + " trying to load " + initData.size() + " objects into the space"); return new MyDataIterator(initData); } public void shutdown() throws DataSourceException { } static GigaSpace startClusterNode(int id) { GigaSpace space = new GigaSpaceConfigurer(new UrlSpaceConfigurer( "/./space?cluster_schema=partitioned-sync2backup&total_members=2,0&id=" + id) .addProperty("space-config.persistent.enabled", "true") .addProperty("space-config.engine.cache_policy", "0") // LRU .addProperty("space-config.external-data-source.usage", "read-only") .addProperty("space-config.persistent.StorageAdapterClass", "com.j_spaces.sadapter.datasource.DataAdapter") .addProperty( "space-config.external-data-source.data-source-class", MyEDS.class.getName())).gigaSpace(); return space; } } The DataIterator package com.test.eds; import java.util.Iterator; import java.util.List; import com.gigaspaces.datasource.DataIterator; import com.j_spaces.core.client.SQLQuery; public class MyDataIterator implements DataIterator<MyData>{ List<MyData> data; Iterator<MyData> dataIter; public MyDataIterator(List<MyData> initData) { data=initData; dataIter = data.iterator(); } public MyDataIterator(SQLQuery<MyData> query) { } public MyDataIterator(MyData query) { } public void close() { } public boolean hasNext() { if (dataIter!=null ) return dataIter.hasNext(); return false; } public MyData next() { if (dataIter!=null ) return dataIter.next(); else return null; } public void remove() { if (dataIter!=null ) dataIter.remove(); } } The Space Class package com.test.eds; import com.gigaspaces.annotation.pojo.*; import com.gigaspaces.metadata.index.SpaceIndexType; @SpaceClass (persist=true) public class MyData { String id; String data; Integer routing; @SpaceId (autoGenerate=true) public String getId() { return id; } public void setId(String id) { this.id = id; } @SpaceIndex(type=SpaceIndexType.BASIC) public String getData() { return data; } public void setData(String data) { this.data = data; } @SpaceRouting public Integer getRouting() { return routing; } public void setRouting(Integer routing) { this.routing = routing; } } |
![]() |
GigaSpaces.com - Legal Notice - 3rd Party Licenses - Site Map - API Docs - Forum - Downloads - Blog - White Papers - Contact Tech Writing - Gen. by Atlassian Confluence |