HomeSolutionsData Synchronization

Data Synchronization

The goal of data synchronization is to establish consistency between sets of data across systems, databases, files or devices. It is fundamental to a wide variety of applications that require data to be shared, copied or collaboratively processed.  Such data may be managed separately by multiple applications or databases, yet needs to be kept consistent between these systems.

The need for data synchronization may be continuous.  After initial data synchronization occurs it is often desirable that the synchronization process continue so that participant systems remain in a consistent (synchronized) state over time.  Alternatively, synchronization may occur as a one-time, bulk operation which is frequently the case during migration projects or an initial data load.

data-sync   

For example, let’s say your organization has been tasked with creating and maintaining an enterprise-wide system of record for customer data, a product list, or vendor information; or perhaps you are responsible for generating an audit trail to comply with industry regulations; or may need to collaborate with partners or offshore developer teams, sharing source code and project related documents.

Such data may be produced or captured only by a single system, yet may need to be made available to all other applications.  You will need to make sure that such data are consistent across multiple applications and systems, providing either the relevant portions   
of needed information or a super-set of all information across participant systems.  As such, data synchronization can be either one-way or bi-directional, allowing changes to be captured at various sources and shared across several targets.

Data Synchronization Challenges

There are numerous challenges to implementing an efficient and reliable data synchronization processes.

  • Participant systems are often heterogeneous and may include legacy systems, packaged applications, RDBMS, mainframes, files, etc. 
  • Data structures and format may vary widely across the participant systems.  These differences may require complex mappings between source and target formats, as well as aggregations, calculations, etc.
  • Data synchronization may require a low latency, near real-time change capture process in order to reduce processing time or may restrict data volume or band-width.
  • When data conflicts occur, they must be managed and resolved taking into account update precedence, ordering and ownership.

 

Key Platform Features

The StreamScape data synchronization solution enables your organization to synchronize content across a wide variety of systems, applications and databases.  Integration Service Packs allow the application fabric to connect to a broad range of external systems or be embedded into applications or systems that require synchronization, making them part of the fabric's infrastructure.  The solution facilitates reliable data synchronization in batch or real-time fashion and fully supports transactional semantics and native security.  Our infrastructure is built to be fast, simple and scalable providing a flexible foundation for data exchange in a secure, cost-effective and consistent fashion.  

Key features include:

  • Graphical environment for management, auditing and repair (conflict resolution) of data synchronization processes using a web-based Operations  Console. 
  • Integrated user registration and profile management allows the system to dynamically provision users, groups, and manage their profile information making it easy to synchronize with external LDAP servers, such as Oracle Internet Directory, Microsoft Active Directory or Lotus Domino.
  • Fast, scalable and easy to use data grid that enables processing, query and aggregation of in-flight data using SQL or a user-defined Domain Specific Language.
  • Broad connectivity to packaged applications, BPM and integration tools, LDAP, files and databases.
  • Access to a variety of structured and semi-structured (unstructured) data formats, including Excel spreadsheets, XML documents, and binary files.
  • Data virtualization capabilities, allowing users to easily access relational data sources as scheduled, on demand or in real time.
  • Flexible real-time change capture for a variety of data sources and applications including transactional access to databases, messaging systems and files.
  • Semantic Mapping and Transformation of in-flight data.
  • Rule-based filtering and routing of data between source and target systems using SQL-like syntax. 
  • Collaborative development and deployment capabilities, allowing artifacts and entire processes to be shared and reused without complex configuration.