Fernando Labastida’s Integration Blog

Ok, I promise this will be my last post on the outlook for 2008!  David Lithicum of Intelligent Insight has listed his 5 predictions for Saas in 2008. Lo and behold he mentioned integration as # 2 on the list!  He said that integration between SaaS CRM solutions and back-end systems will be an afterthought for most companies and they will “…end up figuring things out on the fly.”

However, Peter Coffee over at Salesforce.com was kind enough to refer back to one of my original blog posts where I argued SaaS CRM applications are becoming de-facto data or SOA hubs (with a little help from data integration vendors).

Read on Salesforcetimes.com, CNN Money posted an article, originally in Investor’s Business Daily, where Mark Benioff, Salesforce.com’s illustrious CEO, in his typically modest fashion, claimed that 2008 will be the year for SaaS.

In the article they mention the “end of software” mantra that Salesforce.com has used since it’s inception; I’ve been hearing that from the Salesforce.com camp since I was working for a new defunct internet start-up back in 1999.  However, now I really believe it’s coming true. A large percentage of my integration business is from customers wanting to connect SaaS applications to traditional on-premise software.

 Read the Benioff article here.

CIO Insight, in a recent posting in their “Future of IT” series, announced that 60 percent of those surveyed for their “Future of IT” survey have said they’ve adopted SOA and web services, and three quarters say it will be “the bedrock of their IT architecture.”  However, they are concerned that this last figure is 5%  below the survey results of January of last year.  One of the potential reasons they cite is the failure to meet the expectation that SOA will save companies money, among other things.

I believe that, to a certain extent, SOA has been over-hyped.  As a sales person I recognize that what I sell is not for everyone; SOA is not for everyone either.  Architecting an overarching SOA architecture, with an Enterprise Service Bus (ESB), and all enterprise applications connected to the ESB so that ad-hoc applications can be built “on-the-fly” from the functionality of these enterprise apps, is for large enterprises. And it’s not for every large enterprise either.  There are large companies with dozens of different geographical locations or departments where it doesn’t make sense for all applications to be connected under a SOA umbrella.

And gone are the days when large, 12-18 month enterprise projects are the way to go; the success of Salesforce.com vis-a-vis Siebel is ample evidence of that.  In my world, companies large and small contact me for tactical point-to-point integrations.  If their project works out well with my product, then they’ll roll it out as part of a larger product and buy more of my product.  And it works.  They didn’t have to marshal dozens of people and put together new teams and come up with new strategic IT and Business initiatives and get everybody aligned with the business and IT goals and disrupt what they were currently doing so that their companies could embark on these huge monolithic projects. A small team of go-getters just “got it done.”

So I’ll get off my soap box now, but I think that companies are starting to see that SOA is not for everyone.

It’s been too long since I’ve posted to this blog; end of year craziness is my excuse. And why, you ask, am I posting on Christmas eve? You’re probably thinking: what a bad family man, he’s working when he should be spending time with his family in harmonious communion. Well, I’ll have you know I was up real early this morning baking and I’m taking my rest time to write a little relaxing blog entry!

On to the top IT priorities for 2008. According to CIO Insight magazine, the number 1 and number 3 top priorities in terms of technology that will improve a company’s business strategy are business intelligence and data and application integration, at 44% and 29% respectively. Since business intelligence requires data integration tools to build the data warehouses that BI tools sit on top of, then two of the top five priorities regarding technology for business improvement for 2008 will require application integration tools.

According to SearchCIO.com, the top two priorities are enterprise software and business intelligence (again), respectively. Enterprise software implementations achieve minimal value if not integrated seamlessly with other enterprise application. For example, in order to provide a 360 degree view of the customer and to encourage usage of new Saas or on-demand CRM applications, these have to be integrated with order entry and accounting apps, again, requiring a data and application integration tools.

Ok, I’ll stop tooting the data integration horn and get back to baking and preparing brisket. And no, there’s not enough food for everybody!

“Get on the bus!”

I wanted to follow-up on my post about legacy migration from Nov. 30th to comment on some blog posts about ESBs (Enterprise Service Bus) and SOA (Service Oriented Architecture).

For a definition of SOA, I saw a great one on a recent post “SOA Essentials, Part 1,” by Jeff Davis,

“A Service Oriented Architecture, or SOA, represents an approach towards software development that emphasizes the creation of reusable software services that are based upon discrete units of business functionality.”

In other words, certain functions from applications like a CRM or ERP application are “componentized,” and can be put together to make distinct services that serve a particular function such as “create account,” and these services can then be put together to form an “orchestrated business process.”  These new business processes are essentially composite applications that can be created on-the-fly with the new web services technology available today. 

An ESB is supposed to the be messaging layer in a SOA environment.  Messages from different applications are communicated, using WSDL and SOAP protocols, and transported in the ESB, or service bus.  However, Blogger Loraine Lawson from IT Business Edge, today references an article by fellow blogger Robin Bloor which essentially expands the mission of an ESB from that of being a SOA-enabler to being an “integration-on-demand” enabler.

ESBs, according to Bloor, after interviewing executives at ESB provider CapeClear Software, “…actually herald from the Enterprise Application Integration days. So, he contends, it’s really more than “just” a messaging software for SOA – it’s an integration engine or mediation engine…”

That still does not address the issue of making the functionality from legacy applications available as a service in a SOA environment. ESB vendors essentially need other integration software, such as Pervasive’s Data Integrator, to provide the “on-ramp to the bus,” or “last-mile connectivity,” on to the ESB. Our experience at Pervasive, with ESB provider Sonic Software, has been that older versions of ERP, accounting or CRM packages, or home-grown applications, don’t have web services APIs.  They need a tool that can connect natively to these applications or to the underlying databases and expose their data or functionality as a service first, before they can actually connect to the ESB.

So in the beautiful brave new world of web services and SOA, where every application uses XML and can communicate with every other application out there, legacy applications do not fit into this perfect picture.  However, it will rear it’s ugly head again and again, and as in my previous post, the majority of data in companies out there is still locked away in legacy applications.

With all the buzz and sexiness of SOA and web services, vendors have created an environment where customers feel inferior if they don’t have the latest modern application.  However, these applications are nothing without data. Data is the one constant at a company, whereas software has become a disposable item. At our company, we have gone through about four different CRM applications, but we have kept the data on our customers.

The majority of data, at companies large and small, exists in old hard-to-get-to file formats or technologies. If a company has been around for at least 20 years, they may have applications on mainframes, or built on older databases that don’t adhere to modern DBMS standards, or data that exists essentially as flat files, such as in COBOL formats.  But getting that data out of older systems and into new, modern systems that utilize these latest technologies is a very hard problem which software vendors typically gloss over.

If an ERP or CRM vendor tells you “migrating your data to our new app. is a cinch, no problem at all,” then you know somebody’s lying.

Most applications these days are built on modern databases that are self-contained wonders. They have things like tables, fields, relationships between these tables and fields, passwords, programming, indexes, metadata, all wrapped in one nice package. 

Most legacy systems use older data storage technologies such as VSAM, ISAM or COBOL which have no indexes, metadata, nor relationships, just long rows of data, with rudimentary attempts to indicate what each section of data means, when they were written, etc.

Converting legacy data to data that is neatly placed in rows and columns and in a format that can be utilized by one of the new applications is not something one of the modern SOA vendors can do, and requires a traditional data integration or ETL (Extract Transform and Load) tool to perform.  It’s not sexy or modern, but it’s something countless organizations face every day.

So before going out and buying the latest SOA-compliant whiz-bang application, you need to really consider how you’re going to get all your valuable customer and product data from those old hard-to-get-to systems into your beautiful, shiny new application.

By Fernando Labastida

DM Review is a great source for articles on data and application integration. However, I have to take issue with a recent article on Application Integration for Midmarket Companies. The article starts out on the right foot, comparing enterprise-level ETL and EAI tools and custom coding, and poking holes in both. Enterprise-level ETL and EAI tools are like “using a chainsaw to open a letter,” and custom code is a very risky proposition: it monopolizes the time of the most skilled developers, lacks scalability and flexibility, and is not reusable, among other things.

However, the prescribed solution is all wrong. The article recommends an “integration appliance” as the answer to the midmarket integration challenge, asserting positive attributes such as low-TCO, no need for programming skills, fast delivery, and simpler operations. An integration appliance is a self-contained hardware/software combination that delivers “integration-in-a-box.”

However, customers and system integrators who have had experience with integration appliances point out the following drawbacks:

1. Simple integrations, such as direct mapping of data from one source to another, can be done without programming; however, when complex business rules have to be implemented, a more common scenario, appliances still require programming, often taking more time to implement than customers expect.
2. Appliances tend to be very costly. Up front costs are high and are not justified when the cost of their attendant software and hardware components are factored in; a proven integration application, coupled with a state-of-the-art server, properly configured by a system administrator, is more economical and provides more flexibility. Connecting additional end-points to the appliance, when complex business rules have to be implemented, cannot be done simply through drag-and-drop configuration, but requires programming, increasing the ongoing costs of ownership.
3. Meeting increasing performance demands with an appliance is complicated. Appliances are delivered with a fixed hardware configuration; optimizing the hardware for increasing data volume requires additional intervention by the appliance vendor. The latest application integration software, on the other hand, comes already equipped to utilize additional CPUs or cores if and when the client company decides to add horsepower, which it can do very cost effectively.

The best solution for midmarket application integration is a robust, agile, lightweight, low-TCO, integration software platform, that can either be delivered as a toolset with an easy-to-use yet powerful visual integration interface and extensive connectivity, or as a fully delivered, fixed-price integration solution.

Click here for a free trial of Pervasive’s new Data Integrator V.9, with the revolutionary new Data Mediation Services for creating your own connectors and adapters. After downloading the software, contact me so I can issue you a license key.

I’ve worked in the Latin American markets for many years now, and I have to say it is a difficult market to penetrate for IT vendors. The biggest competition for software vendors is not other vendors, but custom code. Salaries for developers and engineers are very low compared to U.S. salaries, sort of akin to the salaries of Indian engineers which has enabled the explosive growth of outsourced development. So it’s been a pleasant surprise to see that companies in Mexico and Brazil have really taken to the concept of an integration application.

However, there are peculiarities of selling into the Latin American marketplace that I’ve had to learn in order to be successful. I’ve been advised by my system integrator in Mexico that you can’t say your integration platform helps companies “save on development costs,” because development costs are so low anyway. You’ve got to sell them on the time aspect, how an integration platform will enable a company to configure a new integration on-the-fly, or how an integration platform can enable a company to create repeatable and documentable processes that won’t walk out the door with developers who quit.

Another aspect of selling in Latin America, which actually makes an integration platform a more compelling value proposition south of the border than in the U.S., is the IT and application environment of mid-sized companies. Case-in-point, in working with a Mexican Business Intelligence vendor (there are a whole crop of home-grown ISVs in Mexico and Brazil that I’ve discovered), I’ve learned that in the Mexican manufacturing sector, in areas outside of Mexico City, many companies have older versions of Oracle or SAP and have not kept up with their maintenance and support. Despite the fact that both Oracle and SAP have their own ETL tools that they practically give away, these older versions don’t, presenting a quagmire to the Business Intelligence company: their product works on top of a data mart or data warehouse, but these companies have no way, save custom code, to build data marts or data warehouses. An integration tool will enable them to increase their potential market size and go after the mid-market, which in Mexico is ripe for the picking!

As a follow-on to my late Thanksgiving announcement of the release of Version 9.0 of Pervasive’s Data Integrator, Mike Hoskins, Pervasive CTO and General Manager of the Integration Division, was interviewed in IT Week out of the UK. The FIRST thing they asked him was about the SDK that allows partner companies or corporate clients to build their own adapters. It seems as though the opening up of the Pervasive platform is getting some attention in the media.

Hoskins also commented on the integration market in general. Despite the fact that integration software has existed for a long time (Pervasive’s integration platform has been around for more than 20 years, having gotten it’s start as Data Junction), Hoskins noted that the vast majority of companies out there are still using custom code to integrate their disparate applications. That’s changing, he said, as upper management recognizes the inefficiencies of this model, and ISVs continue to do battle with the integration problem.

Finally, IT Week mentions software-as-a-service (saas) applications and their integration challenges, to which Hoskins replies:

“The Achilles heel for the SaaS model is that people assume that because software is easy to discover, install and understand, integration will be easy as well.”

Read the article at: http://www.itweek.co.uk/itweek/analysis/2204237/integration-platforms-steal

Prominent blogger on data and application integration issues, as well as SOA issues, Loraine Lawson at her blog Mergers and Integrations, picked up on the IT Week UK article, asking if automated integration for saas was a Holiday Miracle.

She added additional street cred to Hoskins’ comment about the growth of demand for integration by saas vendors, pointing to two other blog posts that comment on the increasing role of business executives in acquiring software, as well as the increasing challenges for IT of integrating saas applications.

Read Loraine’s blog post here: http://www.itbusinessedge.com/blogs/mia/?p=260

First of all, Happy Thanksgiving! It was yesterday, and today is supposedly a non-work day for everybody too, although you get a few way-too-serious people (me) that can’t seem to stay away from their computer.

Version V.9 Announced

Ok, now for the late announcements. Pervasive Software, my employer, has recently released it’s newest version, V. 9.0. It’s taken a revolutionary step in a sort of open-source-ish-like direction, separating the connectors and adapters from it’s core mapping and data transformation platform. It has included:

“A new Data Services platform with a SDO (Service Data Objects) standards-based Data Mediation Services (DMS) software developer kit that allows Pervasive and subsequently partners and customers to rapidly create new adapters”

(To download a trial version click here, and email me to turn on the license key).

As if 150 connectors to different file formats, technologies, databases, and applications weren’t enough, Pervasive is enabling third parties now to create adapters and connectors to specific applications for which none exist at the moment.

Traditionally, if there wasn’t an adapter for, say, a proprietary ERP system, there were other ways to connect to it, such as with an ODBC connector or via flat files. However, consultants or technical people trained on the integration toolset had to write complex business rules in order to replicate the functionality of the ERP system from which data was being extracted or to which data is being written.

With the new Data Mediation Services (DMS), instead of having to wait for Pervasive to include a particular request for a proprietary adapter in its normal product development lilfe-cycle (which could take up to a year), Pervasive consultants, system integrators, or even client companies, can write their own adapters and drop them right into the Pervasive platform.

Saas Fixed-Price Integration

Another recent development is the announcement of the FastTrack integration solution. Up till now, data integration companies have sold tools that either developers or business analysts use to configure integrations between applications.

“If you want to buy a shirt, we’d sell you a sewing machine,” says our CTO, Mike Hoskins.

Now Pervasive offers a fully configured integration, with software and services included in a fully delivered fixed-price subscription model. This is great for several reasons:

• Companies with no IT departments can buy an integration right now
• All the risk is on Pervasive: fixed price delivery model regardless of end-points
• Pricing model that maps to the current software-as-a-service (saas) paradigm

Ok, my son is about to play in his Thanksgiving soccer tournament, so I am out of here…