StreamScapeTechnologies - Welcome to Streamscape Technologies

Customer Case Study: Simplifying Large-Scale System Integration

Developing a reliable, high performance infrastructure for the capital markets industry has never been more challenging. Increasing system complexities brought on by regulatory compliance and growing data volumes require a more sophisticated set of tools and developer skill sets. Addressing these challenges requires a fresh perspective on the problems facing the industry today.  Essential to success in today’s rapidly changing environment is a new distributed application platform for building collaborative data processing systems.

The StreamScape Service Application Engine™ delivers compelling long-term value due to it's light-weight, scalable design and ability to manage large amounts of in-flight (transient) data that is often critical to back-office operations. Regardless of the size of the IT staff, organizations can take advantage of a comprehensive data-centric integration solution that can evolve with the needs of the business.

This technical breif is a customer case study that illustrates how one of the industry's leading private equity firms used the StreamScape application platform as a force multiplier to implement a large-scale, back-office application integration system.

Problem Overview

An enterprise architecture often consists of disparate applications and systems, all contributing to the overall business process of an organization. Back-office systems frequently include Customer Relationship Management (CRM), Accounting and General Ledger Applications, Payment, Trade Settlement and Enterprise Resource Planning (ERP) Systems. Coherent flow of information between such systems ensures that a business is able to function in an optimal and reliable fashion.

Although enterprise systems are often fragmented and isolated, they typically work on the same data in a collaborative fashion and in many cases must have access to a common set of information that is available to other applications.  For example, CRM and Payment systems may need to share entity information to ensure that customers are defined as payees prior to affecting payments.  As such, enterprise systems often need to solve two critical problems in order to function efficiently:

• Data Synchronization: Allows multiple systems to keep identical copies of critical reference data necessary for isolated function. Data synchronization is often asymmetric, meaning that source and target formats are different. As such, data mapping is often required between systems.  For reliability, data synchronization systems may need to implement certain rules, such as (i) enforcing strict order in which changes are applied (ii) declaring dependency between target systems to ensure all targets receive the same changes (iii) providing a recourse mechanism for suspending synchronization or retrying the operations, thereby performing manual conflict resolution if needed.

• Data Distribution: Facilitates a way of moving transactional data between systems in a reliable fashion.  Similar to synchronization the solution should allow for data mapping, ordering of data stream elements and the ability to manage and retry data distribution operations. Unlike synchronization, data distribution typically involves several steps, comprising a more complex business process that may require error handling, error escalations and correlation of multiple process steps.

In addition to the core integration problems described, there are several logistical challenges that need to be addressed by a potential solution. Typically, after a baseline set of processes for synchronization and distribution are created, other systems are enrolled as participants in order to present a unified version of critical reference data across the enterprise; or to subscribe to transactional feeds.  Hence, after the enterprise architecture has proven successful, broader adoption presents the following challenges:

• Time-to-Market for new implementations is critical to the success of a solution's adoption across an enterprise.  An implementation that cannot be easily duplicated will not evolve into a sustainable solution regardless of success.

• Change Management for existing processes that require altering of distribution rules, escalation policies, data mapping or other performance-related configuration settings should take a small fraction of the time and not require significant effort.  Efficient change management is essential to long term viability of a solution as systems and their data formats tend to evolve over time.

The team in our case study encountered all of the problems described.  They were further challenged by a limited budget, extremely tight time frames and limited resources for developing and managing the resulting solution.

Environment Overview

The overall environment consisted of the following components:

General statistics for the overall installation present a composite view of all the environments (excluding any Dissaster Recovery systems).

Unique and Cost-Effective Solution

StreamScape's technology allowed our customer to stay ahead of the curve by providing a cost-effective solution to the integration problem, allowing a small team of technologists to deploy and manage several complex and disparate integration environments.  The solution leveraged a number of unique software capabilities to get the job done:

• Does not rely on external technologies (ie. Database, Application Server, Web Server)
• Allows for direct visibility and query of Process Queues and Resources via Web and SQL
• Provides Auditing of all Process Flow steps and actions
• Provides Auditing of Process Content
• Allows external applications (ie. JD Edwards, Oracle) to submit Process Queue Work
• Supports Large Data (Messages) in excess of 20MB

The application engine takes a unique approach to solving the integration problem by cutting across a number enterprise architecture disciplines.  By combining agile development techniques found in Service Oriented Architecture with filtering and aggregation mechanisms prevalent in Event Stream Processing technologies the system delivers a scalable and adaptive solution.  

Key Benefits

• Faster time-to-market.  New processes take 2-3 days to develop and test, instead of 3 weeks
• Simplicity. No external dependencies on Database, Messaging Systems, Application Servers or Web Servers
• Improved Change Management.  Changes in data mapping and logic flow take hours instead of days 
• A reliable distributed architecture with built-in fault isolation
• System Fault Tolerance and Redundancy
• Operational Visibility into System Components and Data Flow
• SLA Management for External Systems including Audits and SQL Query Capture
• Integrated E-Mail Distribution and Escalation to Operations Staff
• Significant reduction in cost