Fernando Labastida’s Integration Blog

By Fernando Labastida

So, there’s a recession on the way, or already here. Some accounts say it’s going to get pretty bad, and others are not so pessimistic. All I know is that the job losses have been worse than expected, and despite yessterday’s upswing in the stock market due to the Fed’s offer to lend banks $200 billion in exchange for risky mortgage debt, others are not so sanguine about the move.

What are corporate IT departments to do in the midst of such uncertain economic times? Is this the time to retrench and not invest in software? Not on your life. Most companies are probably in the middle of one integration project or another, such as building a SOA infrastructure, building a data warehouse for business intelligence purposes, migrating old data from legacy applications to a new SaaS application, or connecting their CRM, accounting, ERP, project management, inventory management or manufacturing operations in order to make their business more efficient.

These projects are all oriented towards one thing: decreasing costs, and increasing revenue. This is no longer the late 90s / early 2000s. Corporations no longer purchase software to acquire “cool technology.” Software projects are now purely utilitarian, with CIOs focusing on bottom line as well as top line revenues only.

What corporate IT departments need to do is stop messing around and get going with these projects.

If projects are being delayed because of the use of custom code, companies need to acquire a tool to cut down development time. This is not the time to “do-it-yourself.”

Here are the reasons for acquiring a tool, such as (surprise) an integration tool, rather than relying on labor-intensive manual processes, during a recession:

• Low total cost of ownership
• Faster time to market
• Flexible, scalable implementations
• Higher level of integration with third-party technology
• Integrated, cross-functional processes
• Automated, standardized design processes
• Optimization of development resources
• High reliability through proven performance
• Self-documenting

This is no time to use your valuable development resources on the mundane task of integrating applications. They need to be put on tasks designed to optimize efficiency in order to weather the hard times. This is the moment you’ve been waiting for, the moment to acquire labor-saving software tools.

Fernando A. Labastida

By Dennis Hall, Director of Business Development, Pervasive Software 

Over 100 business people attended the the Softletter SaaS (Software as a Service) conference in Altanta, GA on January 30-31st.  Softletter, headed by Merrill R. (Rick) Chapman, produced a stellar event filled with valuable topics from start-up financing to delivering managed services and knowledgeable speakers, including Zach Nelson (NetSuite), Peter Lee (Salesforce.com) and Mike Hoskins (Pervasive Software). 

Almost everyone at the SaaS conference had integration pain.  This fact was interesting to me because 1) IT and traditional vendors have been calling data integration the “Achilles heel” of SaaS since 2003, 2) Executives from Zach Nelson (SMB XML) to Tim Minahan (SupplyExcellence blog master) have been saying integration for SaaS was a minimal issue or solved since 2004 and 2007 respectively and 3) Solving integration pain pays my bills!

So let me share a few of the most common pain points from the SaaS providers I spoke with…extended Sales cycles, implementation delays and tapping into traditional systems behind the corporate firewall.

Extended Sales Cycles

After years and years of software salespeople telling customers that integration was “no problem” and the customer finding after the deal closed that integration was going to cost an additional 30-50% of the entire deployment budget*, customers are increasingly keen to understand (in detail) how SaaS vendors can tap into key systems, such as CRM, Inventory or A/R.  (side note: even today almost all of the software salespeople I’ve spoken with hope and pray that “integration stuff” doesn’t come up until after the deal is sealed!) 

In addition, our partners tell us that their customers are also asking more frequently to see the SaaS application working with their internal data in the proof of concept phase.  If your integration solution has not been pre-planned, then you should plan to wait in line behind the 10 other IT projects in front of you – good luck!  My favorite quote came from Louise Allen, VP Products at QuickArrow, who said “the relationship with Pervasive has reduced integration discussions from weeks to less than 15 minutes.”

Implementation delays

Again, the length of the line in IT (the dreaded IT Bandwidth objection!) often dictates how long the business drivers must wait to realize the benefits of the SaaS solution. The easier the SaaS vendor can make implementation or consumption, the faster they implement and start recurring revenue streams.  We see many vendors thrust the responsibility to standardize the existing (mostly legacy) data from multiple sources into the vendors “pre-built template”.  Why?  Because it’s hard work that requires technical and subject matter expertise.

Unfortunately, many end users underestimate the scope of work required to make this happen.  I’ve seen such implementations delayed by over 6 months with costs in excess of over $100,000 to implement.  After the “pre-migration” work is done, the vendor writes scripts and lays eyeballs on the incoming data streams for information that does not fit or comply with the business rules.   Most times, bad data does get through resulting in unhappy customers (buyer’s remorse anyone?) and increased support costs (fix it now or pay later!).

Integrating with “Stone-Age”, “Brick and Mortar”, or “Traditional” applications

Pick your vernacular! Zach Nelson either coined or borrowed the phrase “Stone Age applications” to describe the SAPs of the world– the audience loved it of course…how three years of SaaS proliferation has emboldened these folks!  Best of Breed companies understand the importance and value of aligning the SaaS application with their existing business systems and processes.  The challenge is that the majority of source or target systems do not have modern, web-enabled hooks or interfaces so you just can’t connect “in the cloud”.  A multi-tenant environment adds additional complexity because VPN and custom-code won’t work…they just don’t scale!  So what are you going to do?  Dedicate a server to every client – good luck if you want to grow to salesforce.com proportions.  Just call your company an ASP and pack it up! 

SaaS vendors (like all others) understand the implications of getting integration right – it entrenches them in the customer’s environment thereby providing the “long tail” - increasing the probability of renewals year over year.  At our company, we’ve integrated our SaaS CRM system with four plus systems, including accounting and project management apps.  Sure, we eat our own dog food with Data Integrator, but the effort is still time consuming and costly so it will take major discomfort for us to change.

Whether you’re a SaaS provider, Systems Integrator or IT Professional, data integration will rare it’s ugly head at some point in your next deployment.  It’s up to you to make a difference for the market and your organization by making sure your project is “integration-ready” well before the launch.

Dennis Hall, Director of Business Development, Pervasive Software

*Gartner 2006

I found an “oldie but goodie” blog post about our own Pervasive Business Integrator from the blog of a “…Multi-Disciplined Healthcare IT Contractor.”  One of the key quotes in the blog posting:

This product has a large amount of features and goes way behind the impression that I was given that it was a mapping tool. In my mind it’s more like SQL DTS on sterioids (sic).

Read it and enjoy!

For those of you who can read Spanish, I found a cool new blog about CRM in Latin America called, believe it or not, CRM en Latinoamérica.  It’s got a good mix of postings about the practices of CRM, such as “Customer Service is the New Marketing.” about social networking, such as Facebook opening up a portal for Hispanics, and of course topics near and dear to my heart such as CRM technology, “10 Things  your CRM System Must Have.” Don’t forget number 11, integration with your back-end systems!!!

I received a very nice email from Michael Kuhbock, the Founder and Chairman Emeritus of the Integration Consortium. He provided me with another definition of integration that he said allowed end-users at corpoorations to provide a better understanding, from a business-point-of-view,  to their business counterparts:

“Being a business man rather than a technical professional or engineer,” Kuhbock said,  ”I have found that one of the main reasons the industry  has alignment problems is due to innefective communication.”

Here’s the definition of integration from the Integration Consortium, courtesy of Michael Kuhbock:

Integration (in-te-gra-tion) - a combination of parts or objects that work well together.
Communication (com-mu-ni-ca-tion) - the exchange of information. Integration is the successful communication between data, applications, processes, people and enterprises.       

I really like this definition because it’s functional as opposed to technical.  Yes, technology has to be involved to make parts of  this happen, but at the end of the day integration is more about a way of operating a company or an organization rather than a cool technology that everybody has to have. A company can have excellent integration between applications, but if the people are not communicating  with each other or using the data effectively, then there’s no business value there. 

 Thanks, Michael!

The word “integration” is bandied about so frequently and in such a cavalier manner, it is assumed that everybody knows what integration is.  However, integration means a lot of things to a lot of people.

 The Merriam-Webster online dictionary definition of integration says:

1: the act or process or an instance of integrating: as a: incorporation as equals into society or an organization of individuals of different groups (as races) b: coordination of mental processes into a normal effective personality or with the individual’s environment2 a: the operation of finding a function whose differential is known b: the operation of solving a differential equation

The first definition describes one of the great socio-political accomplishments of the latter part of the twentieth century, and the latter definition refers to a complex mathematical equation.  However, this doesn’t really work for our purposes.  A more useful definition from TechDictionary.com:

Putting diverse hardware and/or software components together to work as a system.

That’s a little bit better.  However, when we narrow the term down to data integration, and we get my favorite definition, one provided by Georgetown University on a glossary page on Data Warehousing:

Data Integration The movement of data between two co-existing systems. The interfacing of this data may occur once every hour, once a day, etc.

That’s it, it’s as simple as moving data between two co-existing systems. This can be done in four different ways:

Migration: moving data from a legacy application, with data stored in a legacy format such as Cobol or Isam, to a modern CRM or ERP application. This is usually a one-time movement of massive amounts of historical data.

ETL: Extract, Transform and Load of data. Extracting data from operational applications, such as CRM, ERP, accounting, manufacturing, or other systems into a database used for the sole purpose of manipulating the data and reporting on the data, usually for business analysis purposes. This is usally the movement of large amounts of data in a batch process, durng certain intervals such as hourly, nightly, weekly, etc.

Application Integration: The movement of data in small chunks between two business systems, such as from a CRM application to an ERP system, or from a manufacturing system to an accounting system.  This is usually done at an API level, as opposed to through the database or text interface as is more typical of the first two scenarios, and is usually event-based and real-time, or near real-time.

B2B: Integration of data between trading partners, such as wholesalers, manufacturers, retailers, transportation companies, etc.  This also is usually real-time or near real-time, and event based, and entails transforming EDI documents or documents in any other format, transported over the internet, ftp, EDI VANS, etc.

There are many tools out there that do one or the other of the above scenarios, and some even do all of them.  The best integration tools for a company usually depends on what their primary focus is, how much data is being moved, the required frequency and speed of data integration, among others. Integration tools are usually not a one-size fits all solution; however, there are some tools that are very flexible, or rather agile, that can be used for almost all data integration scenarios. Most companies, if they’ve been around for any length of time, usually have diverse integration needs, and so sometimes it makes more sense for them to acquire a tool that can span the breadth of integration scenarios so as to capitalize on a one-time investment in software as well as knowledge.

Ok, I promise this will be my last post on the outlook for 2008!  David Lithicum of Intelligent Insight has listed his 5 predictions for Saas in 2008. Lo and behold he mentioned integration as # 2 on the list!  He said that integration between SaaS CRM solutions and back-end systems will be an afterthought for most companies and they will “…end up figuring things out on the fly.”

However, Peter Coffee over at Salesforce.com was kind enough to refer back to one of my original blog posts where I argued SaaS CRM applications are becoming de-facto data or SOA hubs (with a little help from data integration vendors).

Read on Salesforcetimes.com, CNN Money posted an article, originally in Investor’s Business Daily, where Mark Benioff, Salesforce.com’s illustrious CEO, in his typically modest fashion, claimed that 2008 will be the year for SaaS.

In the article they mention the “end of software” mantra that Salesforce.com has used since it’s inception; I’ve been hearing that from the Salesforce.com camp since I was working for a new defunct internet start-up back in 1999.  However, now I really believe it’s coming true. A large percentage of my integration business is from customers wanting to connect SaaS applications to traditional on-premise software.

 Read the Benioff article here.

It’s been too long since I’ve posted to this blog; end of year craziness is my excuse. And why, you ask, am I posting on Christmas eve? You’re probably thinking: what a bad family man, he’s working when he should be spending time with his family in harmonious communion. Well, I’ll have you know I was up real early this morning baking and I’m taking my rest time to write a little relaxing blog entry!

On to the top IT priorities for 2008. According to CIO Insight magazine, the number 1 and number 3 top priorities in terms of technology that will improve a company’s business strategy are business intelligence and data and application integration, at 44% and 29% respectively. Since business intelligence requires data integration tools to build the data warehouses that BI tools sit on top of, then two of the top five priorities regarding technology for business improvement for 2008 will require application integration tools.

According to SearchCIO.com, the top two priorities are enterprise software and business intelligence (again), respectively. Enterprise software implementations achieve minimal value if not integrated seamlessly with other enterprise application. For example, in order to provide a 360 degree view of the customer and to encourage usage of new Saas or on-demand CRM applications, these have to be integrated with order entry and accounting apps, again, requiring a data and application integration tools.

Ok, I’ll stop tooting the data integration horn and get back to baking and preparing brisket. And no, there’s not enough food for everybody!

“Get on the bus!”

I wanted to follow-up on my post about legacy migration from Nov. 30th to comment on some blog posts about ESBs (Enterprise Service Bus) and SOA (Service Oriented Architecture).

For a definition of SOA, I saw a great one on a recent post “SOA Essentials, Part 1,” by Jeff Davis,

“A Service Oriented Architecture, or SOA, represents an approach towards software development that emphasizes the creation of reusable software services that are based upon discrete units of business functionality.”

In other words, certain functions from applications like a CRM or ERP application are “componentized,” and can be put together to make distinct services that serve a particular function such as “create account,” and these services can then be put together to form an “orchestrated business process.”  These new business processes are essentially composite applications that can be created on-the-fly with the new web services technology available today. 

An ESB is supposed to the be messaging layer in a SOA environment.  Messages from different applications are communicated, using WSDL and SOAP protocols, and transported in the ESB, or service bus.  However, Blogger Loraine Lawson from IT Business Edge, today references an article by fellow blogger Robin Bloor which essentially expands the mission of an ESB from that of being a SOA-enabler to being an “integration-on-demand” enabler.

ESBs, according to Bloor, after interviewing executives at ESB provider CapeClear Software, “…actually herald from the Enterprise Application Integration days. So, he contends, it’s really more than “just” a messaging software for SOA – it’s an integration engine or mediation engine…”

That still does not address the issue of making the functionality from legacy applications available as a service in a SOA environment. ESB vendors essentially need other integration software, such as Pervasive’s Data Integrator, to provide the “on-ramp to the bus,” or “last-mile connectivity,” on to the ESB. Our experience at Pervasive, with ESB provider Sonic Software, has been that older versions of ERP, accounting or CRM packages, or home-grown applications, don’t have web services APIs.  They need a tool that can connect natively to these applications or to the underlying databases and expose their data or functionality as a service first, before they can actually connect to the ESB.

So in the beautiful brave new world of web services and SOA, where every application uses XML and can communicate with every other application out there, legacy applications do not fit into this perfect picture.  However, it will rear it’s ugly head again and again, and as in my previous post, the majority of data in companies out there is still locked away in legacy applications.