Easily Amused for No Apparent Reason

Jason Hiner, Executive Editor at  TechRepublic.com, in his Sanity Check blog, has published a very useful article in two parts on the Alignment between IT and Business.  The first one, a summary of a Wall Street Journal article on the topic (Hiner has had problems with the WSJ’s take on IT issues in the past), points out the divisions between IT departments and a company’s busines goals, a division which can be debilitating to company growth.  Hiner sums up the article thus:

The general thrust of the article can be summed up by this line: “Success in the digital economy of the 21st century demands a strategic role for IT. And for that to happen, the glass wall between IT and the rest of a company has to be shattered.”

He cites analysts’ estimation of hundreds of billion of wasted dollars on failed IT projects, CIOs who come from technology backgrounds and don’t know how to integrate IT with business goals, and business leaders who look down on IT personnel as ‘nerds.’  Although he largely agrees with this particular WSJ article because they relied on IT industry experts Dr. Amit Basu and Professor Chip Jarnagin instead of internal writers, his own prescription simplifies the articles’ own prescription:

1. Hire a CIO who has business savvy but can also gain the respect of the techies in the IT department
2. Improve IT awareness/training among executives and team leaders throughout the business
3. Improve business awareness/training among the company’s IT managers

The second article in the series is a summary of an MIT Sloan School of Management paper on avoiding the IT Alignment Trap.  Apparently, it’s not enough to just align IT with business goals.  Companies that do that have to also make their IT processes more efficient; if not then they actually spend more on IT than the average company, and the company’s sales goals fall short of the average.

The key is not only to align IT with business goals, or rather, integrate IT with the business, but to simplify IT department processes:

1. Reducing complexity by establishing standards and getting rid of redundancy
2. Rightsource the job, by making strategic decisions as to what to keep in-house, what to outsource, and what to hande with packaged applications, and
3. Creating end-to-end accountability.

Hiner generally agrees with the MIT Sloan paper, except for it’s prescription for centralized IT management.  He advocates a certain amount of decentralized decision-making, especially as it relates to alignment with the goals of separate business units or departments.  He also disagrees with the “alignment” word, citing “integration” (my favorite word; Martin Luther King Jr. would be proud), as better because IT should always be subordinate to the business.

This article series I believe supports an argument I’ve been making throughout this blog, that IT projects should be considered from a purely cost-benefit point-of-view.  Relating this to integration, the purpose of this blog, Loraine Lawson cites an interview with Philip Russom with the Data Warehousing Institute where he states the benefits of “rightsourcing,” in this case using a packaged application:

Data integration requires a senior-level programmer typically, he said, which means you’ll be paying at least one programmer six figures to spend months coding data integration from scratch. By comparison, you could build a comparable solution in two to three months with a vendor tool for less money, he said.

In my experience, the time can be considerably less than two to three months.

Integration’s role in a company where IT and Business are aligned or integrated should be to get out of the way as quickly as possible, not be a bottleneck, stay simple, repeatable and in the background so IT can properly support business goals.

Fantastic article in The New York Times online’s Techology section on Saturday, “Why Old Technologies Are Still Kicking.” by Steve Lohr.

The two most important implications about this article for this blog, because it speaks to the heart of the existence of this blog in the first place:

1. Legacy technologies are not going way, meaning integration between disparate systems is a perpetual need for corporations, and

2. Strategic technology decisions are business decisions, pure and simple.

The gist of the article is that the mainframe is not only here to stay, but IBM has invested huge sums to retool the mainframe and modernize it because of the demand within key sectors of the economy, such as the financial industry, for it’s mission critical capabilities.

“I.B.M.’s most recent model, the z10, represents an investment of $1.5 billion and the work of 5,000 technical professionals,” says Lohr in the article.

This despite the prediction in 1991 by Stewart Alsop, cites Lohr, that mainframe technology would disappear by 1996. Why is this? Here’s the money paragraph of the article:

The unfulfilled predictions of demise, experts say, tend to overestimate the importance of pure technical innovation and underestimate the role of business judgment. “The rise and fall of technologies is mainly about business and not technological determinism,” said Richard S. Tedlow, a business historian at the Harvard Business School.

So for all those who say they don’t need to worry about integrating legacy technologies or invest in integration skills or technologies because “all applications are now web services enabled,” think again. The mainframe, and hence legacy technology, are alive and well, and getting stronger!

Many companies who have never embarked on an integration project are now plunging head first into their first ever integration projects; at the same time, many departments within larger corporations are also tackling their first integration projects. There seems to be various reasons why companies and departments are starting to look at integration now, such as CRM implementations, Business Intelligence, and mergers and acquisitions. I won’t get into that here. The important thing is more and more IT departments are starting some form of data or application integration project, and are looking for ways to get started.

The goal of this post is to list the top five things IT directors need to ensure a successful integration project.

1. Determine the goal of your integration project. An integration project should be pursued from a pure cost/benefit standpoint. How will it advance your business goals? Are you trying to build an executive dashboard so the CEO and CFO can graphically see marketshare, profitability, and sales trends? Then you’re going to need to build a data warehouse with data from various sources, including ERP, CRM, text files and your website, from which your executive reporting tool can make pretty business pictures. Do you want to enable your sales executives to know everything that is going on with your customers, so they can avoid accepting a purchase order from an account that is on credit hold? That would require synchronization between your CRM system and your ERP or accounting system. Are you hoping to make it easier for your suppliers to provide you with shipping information and invoices so they just automatically show up in your wholesale or retail management system? Then you need to EDI-enable these systems.

2. Determine what kind of integration you’re pursuing. Is it a migration, an ETL project, an application integration or B2B integration? Here’s a nice little diagram that can help you with this:

If you’re trying to synchronize data between your CRM system and your ERP or Accounting System (often the ERP/Accounting system is the “system of record,” meaning that’s where the customer master data is stored), then it’s an interface integration. This requires real-time, or near real-time movement of bits and pieces of data back and forth between both systems at a business logic level. On the other hand, if you’re trying to extract data from operational systems, such as your ERP or CRM system and dump them into a repository in order to slice and dice the data for more accurate reporting on your business, then you’re looking at ETL. The requirements here are usually for nightly, weekly or even monthly batch loads from your operational systems, usually late at night or on weekends when these systems that run your business won’t take too big of a performance hit.

3. Determine the sources and destinations of your data. The best way to break down an integration project into easily understandable steps, and to calculate the time and effort it will take, is to determine where the data is coming from and where it’s going to. I’m not just talking about what applications you’re moving data to and from, but also what tables or data objects, and how many. So, for example, if your goal is for your sales people to close a sale in your CRM system so it will kick-off a sales order in your ERP system, this would involve:

• The Accounts or Company object in your CRM system
• The Contact object in your CRM system
• The Opportunity object in your CRM System
• The Product object in your CRM system

That’s four objects in your CRM system. You also have to determine object name and count in your ERP system, as well as determine how data from your CRM system will change, combine or interact for it to make sense to your ERP system and to successfully create a sales order.

4. Determine your resources. Many times an integration project is so easy that it can be done in-house with minimal effort expended. Sometimes what you thought might be very easy turns out to be a very complicated project that drags on for eight months with the current manpower at your disposal. Knowing who you have available and what his or her skills are is crucial. Some integration projects require just one business analyst. These could be simple migrations such as exporting data into a flat file from one system and importing that same flat file. Sometimes Microsoft Excel is all you need to do this. However, most integrations are not as simple. Interface type integrations require lots of heavy programming. You would need somebody familiar with java, C++, and web services programming skills, and they don’t come cheaply! These issues are largely mitigated by commercially available integration tools, which are typically designed for use by a business analyst. If your company has no technical resources (rare), or if they are all allotted to other projects and not available for your project, then it might make sense to hire a consultant for a time to do the integration for you.

5. Choose your approach: Build vs. Buy. This is largely determined by number 4 above. It doesn’t make sense to invest in a $100,000 ETL tool if all you’re doing is loading data from a mailing list into your CRM system. It can also be the death of a project if you decide to use in-house resources, and it ends up taking up to 6-9 months, or requires a highly-paid java programmer to update your custom code every time you want to add a field to your CRM to ERP synchronization piece. It’s up to you. It might not be too much of a headache to do it in-house if it’s a fairly straightforward integration with few to any changes in business logic, and if data structure and field names are the same. If, however, you, have to transform the data in some way, or you’re integrating between two completely different data sources or data types (which covers the majority of integrations), then you should opt for a data integration tool. Most of the time your urgent integration project will not be your last. Because people come and people go, and high-valued technical resources are constantly being poached by other companies, an integration tool that is easy to learn and use will enable you to tackle present and future integrations without having to rely on the knowledge locked away in the brain of your top developer.

As an add-on to my March 12th posting about Integration and The Recession, I neglected one of the most important reasons to acquire integration: Business Intelligence. In a recession it is important to be ruthless about measuring every aspect of your business, from profits, sales, marketing effectiveness, all the way to your web marketing campaigns. By the way, in the aforementioned article, ThePPCGuru.com mentions web marketing as the most cost effective, as well as the most fully measureable marketing method out there, ideal for the recession.

Business Intelligence requires a functional reporting tool, from industry stalwarts Business Objects and Cognos, to up-and-coming value business intelligence company Bitam out of Mexico. It also, (surprise surprise) requires a data integration tool, or ETL tool (extract, transform and load), in order to bring data from various sources such as back-end business applications, websites, web analytics applications, etc. into a data warehouse or data mart on which the business intelligence tool works.

A great little article by Sean McGrath from from ITworld.com appeared yesterday which should serve sales people as well as data integration project managers equally. Essentially, to summarize, if an executive says she wants to connect A to B, you need to dig deeper to see what that really means.  Is it connecting A to B to “advance some process that B is involved in”? Or is it to connect A with information from B in order to enable a report from a reporting module in B.

In reality, McGrath says, A to B in the first scenario is just a simple application integration scenario, where B needs data from A in order to effect a business process.  The second scenario is an ETL scenario, whereby you need to actually take data from A and B and put it into a third repository, C, in order to run a report on this.

I think McGrath does a great job of simplifying the message that a seemingly straightforward integration request is not as straightforward as it seems.  You need to ask the right questions, dig deeper, and find out what the end-goal is.

Gartner’s  report in Tekrati about 9 cost-cutting measures for the tough economic times has a chock full ‘o’ tips for corporations on how to cut costs in their data management initiatives. You can click on the above link for the full report, but here are my comments on a few of the points:

Optimize Data Integration Tools Licensing: Gartner notes typical investments in data integration tools of $200,000 - $500,000, and $50,000 to $100,000 for annual maintenance.  If companies are spending this much on data integration then they seriously need to consolidate.  If you want to cut costs for the hard times, cut out the expensive, kludgy, inflexible data integration tools. And if you think you’re going to lose functionality then you’ve fallen victim to the slick marketing campaign of the super-expensive data integration vendor.

Leverage Established Data Structures and Data Integration Processes: Gartner here touts the benefits of re-use of data related assets built in the past.  In my opinion this is one of the principle driving factors in standardizing on a data integration tool, the ability to re-use data integration processes.  If data integration processes are not easily re-usable then there are no efficiencies to be gained by acquiring a data integration tool.  Easy re-usability should be at the top of anybody’s list of deciding factors when looking at consolidating (see above) on integration vendors.

Defer Replacement of Custom-Coded Architectures: Gartner, I believe, is talking about the opportunity cost of directing labor to the non-productive activity of learning a new data integration tool and replacing custom code that works, as well as the expense involved (again, they mention the exorbitant amount of $100,000 to $500,000 for data integration tools) in acquiring data integration software. Any integration tool worth it’s salt should be architected to “integrate” already existent custom-coded processes that work without having to replace everything, and without forcing a company to invest that much in licensing and maintenance costs.  Again, “big iron” integration vendors have done such a great marketing job that it is generally believed current custom-coded processes necessarily have to be replaced by a data integration tool.  Not so.  A data integration tool should incorporate what has already been done, serve the purpose of making subsequent integrations easy to implement, and require less than $100,000 in licensing.

Explore Open Source Licensing: Open source data integration tools, again, in my humble opinion, would end up costing corporations more, not less.  They typically offer a very reduced set of adapters and connectors, forcing corporatons to custom code connections to the APIs of those applications they want to connect to.

By Fernando Labastida

So, there’s a recession on the way, or already here. Some accounts say it’s going to get pretty bad, and others are not so pessimistic. All I know is that the job losses have been worse than expected, and despite yessterday’s upswing in the stock market due to the Fed’s offer to lend banks $200 billion in exchange for risky mortgage debt, others are not so sanguine about the move.

What are corporate IT departments to do in the midst of such uncertain economic times? Is this the time to retrench and not invest in software? Not on your life. Most companies are probably in the middle of one integration project or another, such as building a SOA infrastructure, building a data warehouse for business intelligence purposes, migrating old data from legacy applications to a new SaaS application, or connecting their CRM, accounting, ERP, project management, inventory management or manufacturing operations in order to make their business more efficient.

These projects are all oriented towards one thing: decreasing costs, and increasing revenue. This is no longer the late 90s / early 2000s. Corporations no longer purchase software to acquire “cool technology.” Software projects are now purely utilitarian, with CIOs focusing on bottom line as well as top line revenues only.

What corporate IT departments need to do is stop messing around and get going with these projects.

If projects are being delayed because of the use of custom code, companies need to acquire a tool to cut down development time. This is not the time to “do-it-yourself.”

Here are the reasons for acquiring a tool, such as (surprise) an integration tool, rather than relying on labor-intensive manual processes, during a recession:

• Low total cost of ownership
• Faster time to market
• Flexible, scalable implementations
• Higher level of integration with third-party technology
• Integrated, cross-functional processes
• Automated, standardized design processes
• Optimization of development resources
• High reliability through proven performance
• Self-documenting

This is no time to use your valuable development resources on the mundane task of integrating applications. They need to be put on tasks designed to optimize efficiency in order to weather the hard times. This is the moment you’ve been waiting for, the moment to acquire labor-saving software tools.

Fernando A. Labastida

By Dennis Hall, Director of Business Development, Pervasive Software 

Over 100 business people attended the the Softletter SaaS (Software as a Service) conference in Altanta, GA on January 30-31st.  Softletter, headed by Merrill R. (Rick) Chapman, produced a stellar event filled with valuable topics from start-up financing to delivering managed services and knowledgeable speakers, including Zach Nelson (NetSuite), Peter Lee (Salesforce.com) and Mike Hoskins (Pervasive Software). 

Almost everyone at the SaaS conference had integration pain.  This fact was interesting to me because 1) IT and traditional vendors have been calling data integration the “Achilles heel” of SaaS since 2003, 2) Executives from Zach Nelson (SMB XML) to Tim Minahan (SupplyExcellence blog master) have been saying integration for SaaS was a minimal issue or solved since 2004 and 2007 respectively and 3) Solving integration pain pays my bills!

So let me share a few of the most common pain points from the SaaS providers I spoke with…extended Sales cycles, implementation delays and tapping into traditional systems behind the corporate firewall.

Extended Sales Cycles

After years and years of software salespeople telling customers that integration was “no problem” and the customer finding after the deal closed that integration was going to cost an additional 30-50% of the entire deployment budget*, customers are increasingly keen to understand (in detail) how SaaS vendors can tap into key systems, such as CRM, Inventory or A/R.  (side note: even today almost all of the software salespeople I’ve spoken with hope and pray that “integration stuff” doesn’t come up until after the deal is sealed!) 

In addition, our partners tell us that their customers are also asking more frequently to see the SaaS application working with their internal data in the proof of concept phase.  If your integration solution has not been pre-planned, then you should plan to wait in line behind the 10 other IT projects in front of you – good luck!  My favorite quote came from Louise Allen, VP Products at QuickArrow, who said “the relationship with Pervasive has reduced integration discussions from weeks to less than 15 minutes.”

Implementation delays

Again, the length of the line in IT (the dreaded IT Bandwidth objection!) often dictates how long the business drivers must wait to realize the benefits of the SaaS solution. The easier the SaaS vendor can make implementation or consumption, the faster they implement and start recurring revenue streams.  We see many vendors thrust the responsibility to standardize the existing (mostly legacy) data from multiple sources into the vendors “pre-built template”.  Why?  Because it’s hard work that requires technical and subject matter expertise.

Unfortunately, many end users underestimate the scope of work required to make this happen.  I’ve seen such implementations delayed by over 6 months with costs in excess of over $100,000 to implement.  After the “pre-migration” work is done, the vendor writes scripts and lays eyeballs on the incoming data streams for information that does not fit or comply with the business rules.   Most times, bad data does get through resulting in unhappy customers (buyer’s remorse anyone?) and increased support costs (fix it now or pay later!).

Integrating with “Stone-Age”, “Brick and Mortar”, or “Traditional” applications

Pick your vernacular! Zach Nelson either coined or borrowed the phrase “Stone Age applications” to describe the SAPs of the world– the audience loved it of course…how three years of SaaS proliferation has emboldened these folks!  Best of Breed companies understand the importance and value of aligning the SaaS application with their existing business systems and processes.  The challenge is that the majority of source or target systems do not have modern, web-enabled hooks or interfaces so you just can’t connect “in the cloud”.  A multi-tenant environment adds additional complexity because VPN and custom-code won’t work…they just don’t scale!  So what are you going to do?  Dedicate a server to every client – good luck if you want to grow to salesforce.com proportions.  Just call your company an ASP and pack it up! 

SaaS vendors (like all others) understand the implications of getting integration right – it entrenches them in the customer’s environment thereby providing the “long tail” - increasing the probability of renewals year over year.  At our company, we’ve integrated our SaaS CRM system with four plus systems, including accounting and project management apps.  Sure, we eat our own dog food with Data Integrator, but the effort is still time consuming and costly so it will take major discomfort for us to change.

Whether you’re a SaaS provider, Systems Integrator or IT Professional, data integration will rare it’s ugly head at some point in your next deployment.  It’s up to you to make a difference for the market and your organization by making sure your project is “integration-ready” well before the launch.

Dennis Hall, Director of Business Development, Pervasive Software

*Gartner 2006

I found an “oldie but goodie” blog post about our own Pervasive Business Integrator from the blog of a “…Multi-Disciplined Healthcare IT Contractor.”  One of the key quotes in the blog posting:

This product has a large amount of features and goes way behind the impression that I was given that it was a mapping tool. In my mind it’s more like SQL DTS on sterioids (sic).

Read it and enjoy!

For those of you who can read Spanish, I found a cool new blog about CRM in Latin America called, believe it or not, CRM en Latinoamérica.  It’s got a good mix of postings about the practices of CRM, such as “Customer Service is the New Marketing.” about social networking, such as Facebook opening up a portal for Hispanics, and of course topics near and dear to my heart such as CRM technology, “10 Things  your CRM System Must Have.” Don’t forget number 11, integration with your back-end systems!!!