Business continuity with real-time data integration

Business continuity with real-time data integration

Enterprises want to protect their data. As the appetite for data volumes grows, storage technology becomes a critical business asset on which business continuity relies. My recent survey in the medium-size enterprise segment shows the five dominant investment directions at the level of data management architecture: disaster recovery (DR), high availability (HA), backup, data processing performance and migration to more advanced databases.

This suggests that corporations generally have sufficiently structured data collections but are concerned with business continuity and continuous availability of data. What infrastructures can provide these assurances?

The most basic approach to provide data safety is hardware or software replication that automatically maintains a secondary copy of critical data. Home-made backup methods, based on open source software are also not unheard of.

Oracle customers have more options: Oracle Active Data Guard can be used if the source and copies are based on identical Oracle versions and data models, while Oracle RAC promises transparent application failover if the copies are in close proximity. Finally, as the Infrastructure-as-a-Service (IaaS) model wins the market for its price and elasticity, one can take advantage of a remote Cloud to host a secondary copy of the business-critical data.

In this post I want to focus on yet another option, and that is the Real-Time Data Integration model. As an example I am going to discuss Oracle GoldenGate, a solution that permits you to manage the data critical to your business in safety, ensuring business continuity without disruption even if the data is distributed among multiple, heterogeneous business applications and architectures.

Key facts about GoldenGate
By 2009, GoldenGate, a San Francisco-based vendor of real-time data integration software was still a modest size (by Silicon Valley standards), but already had a considerable market presence: 500 global customers including top commercial banks, telcos and retail companies.

What led Oracle to the acquisition? With its product portfolio, Oracle could already build advanced HA architectures for its customers, based on their own products.  However, Oracle could not directly compete compete in the domain of real time data replication.

How does it work? The secret is simple: GoldenGate can synchronize two heterogeneous databases, by reading the real-time history of all transactional changes at the source database. The delta is then  piped to the other database(s), again in real-time. In technical language, this is called Change Data Capture (CDC).

Simplicity is the feature of killer apps; GoldenGate guarantees transactional integrity when copying between source and target databases. With the simple deployment of  GoldenGate between databases, one can build a highly available architecture with each database holding accurate copies of each other's data.

Most importantly, GoldenGate permits active-active database replication. Imagine two separate branches of a corporation with separate IT systems (typical internal legacy of an acquisition process and a nightmare for sysadmins). With active-active, even though the two offices are not fully integrated, users can write to both locations at the same time, and the data will remain consistent.

Here we touch one of the core problems of distributed data storage systems, which is the issue of consistency between geographically separated data replicas. Oracle GoldenGate guarantees that even in an unstable environment where networks and host servers occasionally drop out, transactions will never be missed or skipped. Active-active is not without it’s challenges – collision detection must be developed, but Oracle GoldenGate provides a foundation upon which business logic can be easily built.

When to use real-time data integration
In the real-world, corporations use mixed IT architectures from multiple vendors (often a legacy of corporate history). When selecting a technology for a real-time data integration platform, the ease of integration across the spectrum of database and application platforms from multiple vendors is key. GoldenGate is no exception here; its Oracle license permits use with OracleDB and competitive products alike, knowing that this is the only way to maintain the competitive edge. Here are a few scenarios when real-time data integration can be considered:

High-availability: Oracle GoldenGate will automatically maintain a live remote copy of your application’s interim data, so your business application can fail over  to the secondary storage during a disaster recovery scenario, with minimal down-time.
Live migration: upgrade, migration or maintenance of a production system typically involves downtime. Oracle GoldenGate enables zero-downtime migration, as the new system can be populated with old system data without operational interruption of the former.
Integration of heterogeneous systems: your applications rely on Oracle, MS SQL, Sybase, DB2 and what-not? A real-time data integration platform  can make them all operate on the same, shared data with minimal integration effort.
Mergers, acquisitions and IT consolidation in a growing enterprise: before your final, uniform architecture will be devised, a CDC technique can quickly consolidate data from branches and departments. (By the way, we both know that there is no such thing as “final architecture”!)
Query offloading: an interesting side-effect of sharing replicated data among multiple data marts is the improvement of OLTP performance and availability. Queries processed simultaneously by multiple servers execute faster and your reports are ready sooner.
When to use something else?
Savvy managers will carefully consider their options when building a real-time data integration architecture. For instance, care needs to be taken when planning for real-time data consolidation of multiple production systems, when at the same time thinking of it as a HA architecture. Can your business survive if one of them fails? This often depends on short and long-term investment management strategies.

Another issue addressed specifically to existing Oracle customers is how does GoldenGate coexist with other technologies, be it Oracle or non-Oracle that partially fulfill the same or similar functionality? You obviously don’t want to duplicate your investment. It is wise to learn and correctly differentiate between scenarios whereGoldenGate can complement RAC , Active Data Guard or other high-availability paradigms, and those where it is easier to use one of these products standalone. With Oracle licensing, these are not trivial questions and substantial cost savings can be made through informed decisions.