Fernando Labastida’s Integration Blog

By Nena Marin, Sr. Solutions Architect, Pervasive Software

Real-time data warehousing is at the core of enterprise intelligence initiatives.  Forecasting, trending and what to do next require freshest knowledge of what is happening now.  

Pervasive Data Integrator (DI) provides workflow based ETL for real-time concurrent warehouse loading. As an example, the following integration process implements a Data Warehouse load using Pervasive DI.

In this dimensional model, the data warehouse is comprised of dimension and fact tables.  The fact tables contain measures (like $ and units) and relationship to every dimension or attributes via foreign keys.  The dimensional model provides a framework that is scalable yet standard across the enterprise. 

Pervasive DI process designer provides a canvas to create workflows for data warehouse loads.  Each map/transformation node in the workflow loads a specific table; dimensions and facts.  At load level 1, dimension tables without dependencies can be loaded concurrently.  At load level 2, dimensions with downstream dependencies can be loaded concurrently. In this example, the Product_Dimension includes a foreign key to (or is downstream dependent on) the Product_Class_Dimension.  Since fact tables contain foreign keys or dependencies to all dimensions, fact tables are loaded last in the workflow process. 

Pervasive DI workflow offers seamless concurrency, native connectivity and SQL querying to join data from multiple sources. The pervasive ETL process designer also provides handy features like decision steps, email notification and a scripting language for custom source to target transformations.  There are built in options to profile the ETL engine performance during your load processes. 

Finally, processes are scheduled as tasks for automation. 

Pervasive Software on Tuesday announced the launch of Pervasive DataCloud, a hosted, on-demand integration offering that allows integration to be delivered as a service. For full disclosure, Pervasive software is my employer, but I can’t help get excited  by the fact that this is something that I think everbody can get behind!  Now companies that use a strictly Saas model for all their applications, such as Netsuite for their ERP & CRM, Salesforce.com for their CRM, Google Apps for corporate email and productivity tools, and a host of other Saas applications, can also deploy integration between all these apps “in the clouds,” so-to-speak. They’ll have nothing on-premise but a bunch of wireless laptops!

It will also, of course, be useful for those who are slowly migrating to a Saas computing model for their corporate apps, but still run traditional on-premise applications that they need to connect to their Saas applications.

 This story has been picked up my many sources, such as eWeek, Data Monitor, and Curt Monash, to name a few, and is generating lots of buzz.

QuickArrow, a SaaS services automation vendor out of Austin, TX recently announced an embedded integration solution called QuickConnectIT.  Powered by Pervasive’s Data Integrator V9, QuickConnectIT enables companies to connect their QuickArrow hosted professional services automation application with other SaaS or on-premise applications, such as CRM, ERP, and accounting applications.  QuickArrow is one of many SaaS vendors who are embedding integration tools into their SaaS applications.  Why are these software-as-a-service companies embedding integration?  As elucidated by QuickArrow’s press release,

“The partnership (with Pervasive)…enables QuickArrow to better focus on it’s core competency, Professional Services Automation (PSA).”

They’d prefer to do what they do best instead of getting bogged down on custom-coding one-off integrations everytime they implement a solution at a new customer site. 

As a follow-on to my late Thanksgiving announcement of the release of Version 9.0 of Pervasive’s Data Integrator, Mike Hoskins, Pervasive CTO and General Manager of the Integration Division, was interviewed in IT Week out of the UK. The FIRST thing they asked him was about the SDK that allows partner companies or corporate clients to build their own adapters. It seems as though the opening up of the Pervasive platform is getting some attention in the media.

Hoskins also commented on the integration market in general. Despite the fact that integration software has existed for a long time (Pervasive’s integration platform has been around for more than 20 years, having gotten it’s start as Data Junction), Hoskins noted that the vast majority of companies out there are still using custom code to integrate their disparate applications. That’s changing, he said, as upper management recognizes the inefficiencies of this model, and ISVs continue to do battle with the integration problem.

Finally, IT Week mentions software-as-a-service (saas) applications and their integration challenges, to which Hoskins replies:

“The Achilles heel for the SaaS model is that people assume that because software is easy to discover, install and understand, integration will be easy as well.”

Read the article at: http://www.itweek.co.uk/itweek/analysis/2204237/integration-platforms-steal

Prominent blogger on data and application integration issues, as well as SOA issues, Loraine Lawson at her blog Mergers and Integrations, picked up on the IT Week UK article, asking if automated integration for saas was a Holiday Miracle.

She added additional street cred to Hoskins’ comment about the growth of demand for integration by saas vendors, pointing to two other blog posts that comment on the increasing role of business executives in acquiring software, as well as the increasing challenges for IT of integrating saas applications.

Read Loraine’s blog post here: http://www.itbusinessedge.com/blogs/mia/?p=260

First of all, Happy Thanksgiving! It was yesterday, and today is supposedly a non-work day for everybody too, although you get a few way-too-serious people (me) that can’t seem to stay away from their computer.

Version V.9 Announced

Ok, now for the late announcements. Pervasive Software, my employer, has recently released it’s newest version, V. 9.0. It’s taken a revolutionary step in a sort of open-source-ish-like direction, separating the connectors and adapters from it’s core mapping and data transformation platform. It has included:

“A new Data Services platform with a SDO (Service Data Objects) standards-based Data Mediation Services (DMS) software developer kit that allows Pervasive and subsequently partners and customers to rapidly create new adapters”

(To download a trial version click here, and email me to turn on the license key).

As if 150 connectors to different file formats, technologies, databases, and applications weren’t enough, Pervasive is enabling third parties now to create adapters and connectors to specific applications for which none exist at the moment.

Traditionally, if there wasn’t an adapter for, say, a proprietary ERP system, there were other ways to connect to it, such as with an ODBC connector or via flat files. However, consultants or technical people trained on the integration toolset had to write complex business rules in order to replicate the functionality of the ERP system from which data was being extracted or to which data is being written.

With the new Data Mediation Services (DMS), instead of having to wait for Pervasive to include a particular request for a proprietary adapter in its normal product development lilfe-cycle (which could take up to a year), Pervasive consultants, system integrators, or even client companies, can write their own adapters and drop them right into the Pervasive platform.

Saas Fixed-Price Integration

Another recent development is the announcement of the FastTrack integration solution. Up till now, data integration companies have sold tools that either developers or business analysts use to configure integrations between applications.

“If you want to buy a shirt, we’d sell you a sewing machine,” says our CTO, Mike Hoskins.

Now Pervasive offers a fully configured integration, with software and services included in a fully delivered fixed-price subscription model. This is great for several reasons:

• Companies with no IT departments can buy an integration right now
• All the risk is on Pervasive: fixed price delivery model regardless of end-points
• Pricing model that maps to the current software-as-a-service (saas) paradigm

Ok, my son is about to play in his Thanksgiving soccer tournament, so I am out of here…