Fernando Labastida’s Integration Blog

In keeping a blog on “Integration for Business,” I sometimes feel I have to walk a fine line between keeping neutral and writing an objective commentary on integration issues, and wanting to promote the fact that I work for an integration tools company that sells really cool integration products.  In this post I’m going to cross that line in a major way and talk about the products I sell, so please bear with me.

As I go about my daily job of trying to convince companies to purchase Pervasive’s integration products vs. doing integration in-house or buying a competitors’, I often rely on the knee-jerk claim of the “ease-of-use” of our sofware.  I also get sick and tired of hearing potential customers tell me that some of my competitors’ products are apparently easier to use than my product, even though sometimes they forget that some of those products come bundled with 80 hours of free implementation services (paid for by the high-cost of their product), which of course makes it easier by default.

I got so sick of my competitors’, as well as my own rhetoric, that I decided to try out the product I sell for myself. I wanted to see how easy it actually was to use my product to load leads (I am a sales person) into a test ”developers’” account in Salesforce.com.

I had a handy flat file, ASCII Delimited (tab separated) with names of systems integrators on the West Coast that I had downloaded from Hoovers. I opened the Map Designer tool of Pervasive’s Data Integrator, and went to the purple “Source Connection” tab…

…where I chose ASCII (delimited) as the connector, out of the list of 160+ connectors, in order to connect to the file.

I then chose the file I was going to connect to using the “Source/File” dialog box, and because I had seen my Systems Engineer do demos several times, I knew I had to choose “True” for header in order to make the very first row, the one that has “Account, First Name, Last Name,” etc. become the column header row.

Then I went into the green “Target Connection” tab and from the drop-down list of 160+ connectors chose Salesforce.com v. 10 as my target connector, entered my username, password, and security token (now that Salesforce.com has enforced more enhanced security, you either have to approve the particular browser on the particular machine you’re using to be used to log-in, or you need to add a security token after your password).  Then I hit connect!

What happened?  Nothing….which was a good thing.  If I had hit “connect,” and I had gotten an error, then it would have been bad news, wouldn’t it?  Well, my next step was to choose which table I was going to write to within the “Table” dialog box, still within the green “Target Connector” tab.  I chose the “Leads” table from within the list of all the Salesforce.com tables.

Oh, by the way, not only can you view standard Salesforce.com tables within the Multimode Table Selector dialog box within Pervasive Data Integrator’s Map Designer, you can also view custom tables as well!

Finally, I went to the yellow “Map Fields” tab, where I saw the source fields on the left in purple (or are they blue?), and the target fields in green on the right. 

However, I didn’t see the standard Salesforce.com Lead fields, so I was confused.  Then I spied something on the upper right of the screen, a little R1. I noticed it was a drop-down menu, and sure enough, the one drop-down choice was “Leads,” the table I had chosen in the previous screen.  I chose that and was presented with all the “Leads” fields standard to Salesforce.com.

Pervasive’s Map Designer allows you to use cool functions such as “Match by name,” so that if all the field names in the source as well as the target are exactly the same, you can save a lot of time and map all the fields at once.  There’s also a “Match by position” function, assuming the fields are in the same order.  Finally, you can just drag and drop all the fields at once, transferring en masse the source fields to the target. 

None of those will work in this case, since Salesforce.com has specific field names and positions, so each field has to be mapped individudally.  That’s not such a hassle, especially if you don’t really want to map every single field over.  All I did was individually drag and drop the fields I wanted mapped over from the Source side to the corresponding field in the Target side (First Name to FirstName, Line of Business to Industry, etc.).  I had done my mapping!

The fields I had mapped over appeared as bold in my source fields.  Then I decided to verify that the map was valid, so I pressed the little check-mark on the upper right-hand side of the tool. Since there were some fields I was not mapping to it prompted me to remove those from the schema, which I did. Then I got a “Map is Valid” dialog box.

Now, for the integration.  Pervasive’s Map Designer allows you to execute a transformation from the tool itself, or to save your map as an XML file to deploy it to a run-time engine for automation purposes.  Since I’m a sales guy I don’t know how to do any of that stuff, and all I need to do is upload a few leads anyway that I was going to call the next day, so I’m going to execute the map from the design tool. I would do that by clicking the litttle arrow on the upper left side of the tool, right next to the check-mark.

Before that, however, I needed to make sure these leads weren’t already in my Salesforce.com. Ok, the only leads there were what came with my sample “developer’s version” of Salesforce.com. 

Now I hit the execute button, the little arrow on the upper left.  After a log file appears, and a dialog box with a progress bar, I went back to the “Leads” tab in Salesforce.com, refreshed the browser, and voila!  There are my new leads updated 4/22/2008!

So easy, a salesman can do it!

Many companies who have never embarked on an integration project are now plunging head first into their first ever integration projects; at the same time, many departments within larger corporations are also tackling their first integration projects. There seems to be various reasons why companies and departments are starting to look at integration now, such as CRM implementations, Business Intelligence, and mergers and acquisitions. I won’t get into that here. The important thing is more and more IT departments are starting some form of data or application integration project, and are looking for ways to get started.

The goal of this post is to list the top five things IT directors need to ensure a successful integration project.

1. Determine the goal of your integration project. An integration project should be pursued from a pure cost/benefit standpoint. How will it advance your business goals? Are you trying to build an executive dashboard so the CEO and CFO can graphically see marketshare, profitability, and sales trends? Then you’re going to need to build a data warehouse with data from various sources, including ERP, CRM, text files and your website, from which your executive reporting tool can make pretty business pictures. Do you want to enable your sales executives to know everything that is going on with your customers, so they can avoid accepting a purchase order from an account that is on credit hold? That would require synchronization between your CRM system and your ERP or accounting system. Are you hoping to make it easier for your suppliers to provide you with shipping information and invoices so they just automatically show up in your wholesale or retail management system? Then you need to EDI-enable these systems.

2. Determine what kind of integration you’re pursuing. Is it a migration, an ETL project, an application integration or B2B integration? Here’s a nice little diagram that can help you with this:

If you’re trying to synchronize data between your CRM system and your ERP or Accounting System (often the ERP/Accounting system is the “system of record,” meaning that’s where the customer master data is stored), then it’s an interface integration. This requires real-time, or near real-time movement of bits and pieces of data back and forth between both systems at a business logic level. On the other hand, if you’re trying to extract data from operational systems, such as your ERP or CRM system and dump them into a repository in order to slice and dice the data for more accurate reporting on your business, then you’re looking at ETL. The requirements here are usually for nightly, weekly or even monthly batch loads from your operational systems, usually late at night or on weekends when these systems that run your business won’t take too big of a performance hit.

3. Determine the sources and destinations of your data. The best way to break down an integration project into easily understandable steps, and to calculate the time and effort it will take, is to determine where the data is coming from and where it’s going to. I’m not just talking about what applications you’re moving data to and from, but also what tables or data objects, and how many. So, for example, if your goal is for your sales people to close a sale in your CRM system so it will kick-off a sales order in your ERP system, this would involve:

• The Accounts or Company object in your CRM system
• The Contact object in your CRM system
• The Opportunity object in your CRM System
• The Product object in your CRM system

That’s four objects in your CRM system. You also have to determine object name and count in your ERP system, as well as determine how data from your CRM system will change, combine or interact for it to make sense to your ERP system and to successfully create a sales order.

4. Determine your resources. Many times an integration project is so easy that it can be done in-house with minimal effort expended. Sometimes what you thought might be very easy turns out to be a very complicated project that drags on for eight months with the current manpower at your disposal. Knowing who you have available and what his or her skills are is crucial. Some integration projects require just one business analyst. These could be simple migrations such as exporting data into a flat file from one system and importing that same flat file. Sometimes Microsoft Excel is all you need to do this. However, most integrations are not as simple. Interface type integrations require lots of heavy programming. You would need somebody familiar with java, C++, and web services programming skills, and they don’t come cheaply! These issues are largely mitigated by commercially available integration tools, which are typically designed for use by a business analyst. If your company has no technical resources (rare), or if they are all allotted to other projects and not available for your project, then it might make sense to hire a consultant for a time to do the integration for you.

5. Choose your approach: Build vs. Buy. This is largely determined by number 4 above. It doesn’t make sense to invest in a $100,000 ETL tool if all you’re doing is loading data from a mailing list into your CRM system. It can also be the death of a project if you decide to use in-house resources, and it ends up taking up to 6-9 months, or requires a highly-paid java programmer to update your custom code every time you want to add a field to your CRM to ERP synchronization piece. It’s up to you. It might not be too much of a headache to do it in-house if it’s a fairly straightforward integration with few to any changes in business logic, and if data structure and field names are the same. If, however, you, have to transform the data in some way, or you’re integrating between two completely different data sources or data types (which covers the majority of integrations), then you should opt for a data integration tool. Most of the time your urgent integration project will not be your last. Because people come and people go, and high-valued technical resources are constantly being poached by other companies, an integration tool that is easy to learn and use will enable you to tackle present and future integrations without having to rely on the knowledge locked away in the brain of your top developer.

By Dennis Hall, Director of Business Development, Pervasive Software 

Over 100 business people attended the the Softletter SaaS (Software as a Service) conference in Altanta, GA on January 30-31st.  Softletter, headed by Merrill R. (Rick) Chapman, produced a stellar event filled with valuable topics from start-up financing to delivering managed services and knowledgeable speakers, including Zach Nelson (NetSuite), Peter Lee (Salesforce.com) and Mike Hoskins (Pervasive Software). 

Almost everyone at the SaaS conference had integration pain.  This fact was interesting to me because 1) IT and traditional vendors have been calling data integration the “Achilles heel” of SaaS since 2003, 2) Executives from Zach Nelson (SMB XML) to Tim Minahan (SupplyExcellence blog master) have been saying integration for SaaS was a minimal issue or solved since 2004 and 2007 respectively and 3) Solving integration pain pays my bills!

So let me share a few of the most common pain points from the SaaS providers I spoke with…extended Sales cycles, implementation delays and tapping into traditional systems behind the corporate firewall.

Extended Sales Cycles

After years and years of software salespeople telling customers that integration was “no problem” and the customer finding after the deal closed that integration was going to cost an additional 30-50% of the entire deployment budget*, customers are increasingly keen to understand (in detail) how SaaS vendors can tap into key systems, such as CRM, Inventory or A/R.  (side note: even today almost all of the software salespeople I’ve spoken with hope and pray that “integration stuff” doesn’t come up until after the deal is sealed!) 

In addition, our partners tell us that their customers are also asking more frequently to see the SaaS application working with their internal data in the proof of concept phase.  If your integration solution has not been pre-planned, then you should plan to wait in line behind the 10 other IT projects in front of you – good luck!  My favorite quote came from Louise Allen, VP Products at QuickArrow, who said “the relationship with Pervasive has reduced integration discussions from weeks to less than 15 minutes.”

Implementation delays

Again, the length of the line in IT (the dreaded IT Bandwidth objection!) often dictates how long the business drivers must wait to realize the benefits of the SaaS solution. The easier the SaaS vendor can make implementation or consumption, the faster they implement and start recurring revenue streams.  We see many vendors thrust the responsibility to standardize the existing (mostly legacy) data from multiple sources into the vendors “pre-built template”.  Why?  Because it’s hard work that requires technical and subject matter expertise.

Unfortunately, many end users underestimate the scope of work required to make this happen.  I’ve seen such implementations delayed by over 6 months with costs in excess of over $100,000 to implement.  After the “pre-migration” work is done, the vendor writes scripts and lays eyeballs on the incoming data streams for information that does not fit or comply with the business rules.   Most times, bad data does get through resulting in unhappy customers (buyer’s remorse anyone?) and increased support costs (fix it now or pay later!).

Integrating with “Stone-Age”, “Brick and Mortar”, or “Traditional” applications

Pick your vernacular! Zach Nelson either coined or borrowed the phrase “Stone Age applications” to describe the SAPs of the world– the audience loved it of course…how three years of SaaS proliferation has emboldened these folks!  Best of Breed companies understand the importance and value of aligning the SaaS application with their existing business systems and processes.  The challenge is that the majority of source or target systems do not have modern, web-enabled hooks or interfaces so you just can’t connect “in the cloud”.  A multi-tenant environment adds additional complexity because VPN and custom-code won’t work…they just don’t scale!  So what are you going to do?  Dedicate a server to every client – good luck if you want to grow to salesforce.com proportions.  Just call your company an ASP and pack it up! 

SaaS vendors (like all others) understand the implications of getting integration right – it entrenches them in the customer’s environment thereby providing the “long tail” - increasing the probability of renewals year over year.  At our company, we’ve integrated our SaaS CRM system with four plus systems, including accounting and project management apps.  Sure, we eat our own dog food with Data Integrator, but the effort is still time consuming and costly so it will take major discomfort for us to change.

Whether you’re a SaaS provider, Systems Integrator or IT Professional, data integration will rare it’s ugly head at some point in your next deployment.  It’s up to you to make a difference for the market and your organization by making sure your project is “integration-ready” well before the launch.

Dennis Hall, Director of Business Development, Pervasive Software

*Gartner 2006

For those of you who can read Spanish, I found a cool new blog about CRM in Latin America called, believe it or not, CRM en Latinoamérica.  It’s got a good mix of postings about the practices of CRM, such as “Customer Service is the New Marketing.” about social networking, such as Facebook opening up a portal for Hispanics, and of course topics near and dear to my heart such as CRM technology, “10 Things  your CRM System Must Have.” Don’t forget number 11, integration with your back-end systems!!!

Ok, I promise this will be my last post on the outlook for 2008!  David Lithicum of Intelligent Insight has listed his 5 predictions for Saas in 2008. Lo and behold he mentioned integration as # 2 on the list!  He said that integration between SaaS CRM solutions and back-end systems will be an afterthought for most companies and they will “…end up figuring things out on the fly.”

However, Peter Coffee over at Salesforce.com was kind enough to refer back to one of my original blog posts where I argued SaaS CRM applications are becoming de-facto data or SOA hubs (with a little help from data integration vendors).

Read on Salesforcetimes.com, CNN Money posted an article, originally in Investor’s Business Daily, where Mark Benioff, Salesforce.com’s illustrious CEO, in his typically modest fashion, claimed that 2008 will be the year for SaaS.

In the article they mention the “end of software” mantra that Salesforce.com has used since it’s inception; I’ve been hearing that from the Salesforce.com camp since I was working for a new defunct internet start-up back in 1999.  However, now I really believe it’s coming true. A large percentage of my integration business is from customers wanting to connect SaaS applications to traditional on-premise software.

 Read the Benioff article here.

I’ve worked in the Latin American markets for many years now, and I have to say it is a difficult market to penetrate for IT vendors. The biggest competition for software vendors is not other vendors, but custom code. Salaries for developers and engineers are very low compared to U.S. salaries, sort of akin to the salaries of Indian engineers which has enabled the explosive growth of outsourced development. So it’s been a pleasant surprise to see that companies in Mexico and Brazil have really taken to the concept of an integration application.

However, there are peculiarities of selling into the Latin American marketplace that I’ve had to learn in order to be successful. I’ve been advised by my system integrator in Mexico that you can’t say your integration platform helps companies “save on development costs,” because development costs are so low anyway. You’ve got to sell them on the time aspect, how an integration platform will enable a company to configure a new integration on-the-fly, or how an integration platform can enable a company to create repeatable and documentable processes that won’t walk out the door with developers who quit.

Another aspect of selling in Latin America, which actually makes an integration platform a more compelling value proposition south of the border than in the U.S., is the IT and application environment of mid-sized companies. Case-in-point, in working with a Mexican Business Intelligence vendor (there are a whole crop of home-grown ISVs in Mexico and Brazil that I’ve discovered), I’ve learned that in the Mexican manufacturing sector, in areas outside of Mexico City, many companies have older versions of Oracle or SAP and have not kept up with their maintenance and support. Despite the fact that both Oracle and SAP have their own ETL tools that they practically give away, these older versions don’t, presenting a quagmire to the Business Intelligence company: their product works on top of a data mart or data warehouse, but these companies have no way, save custom code, to build data marts or data warehouses. An integration tool will enable them to increase their potential market size and go after the mid-market, which in Mexico is ripe for the picking!

By Fernando Labastida

There is an interesting phenomenon happening in the small and medium business segment. The widespread adoption of on-demand or software as a service (saas) CRM, led by Salesforce.com, and followed by companies such as NetSuite and RightNow Technologies.

Well, that’s not really new.

What is new is the expanded use of saas CRM software within these mini-enterprises, whether independent businesses or smaller divisions or departments of larger corporations, as their principal business platform. Since saas CRM manages the lifeblood of the business, sales and customers, and is increasingly more user friendly and flexible, it is becoming the preferred method for companies to manage their business.

As a result, it is also becoming the de facto integration hub, or SOA enabler, for the smaller enterprise.

A case in point is the experience of a well-known educational products sales company. It’s parent company sells educational toys through retailers. However, it launched a division that sells education-oriented items to schools and school districts, such as a handheld screen-based interactive tool that uses story narratives to teach English proficiency to non-native English speakers. This newer division established a territory sales model, with geographically-based sales executives selling to school districts in their area.

The main corporate entity has only a handful of account managers who sell to large retailers such as Wal-Mart and Toys’r'Us. Whereas it is geared towards a retail sales model and related B2B IT infrastructure, the newer division had the infrastructure needs of a territory-based direct sales model. They required a CRM application to track leads, opportunities, and closed sales, and because of the reduced bandwidth of this smaller business unit, they required the efficiency gains of an automated commission calculating application.

With no dedicated IT resources (IT resources are tied to corporate and are available “on-loan” to the new division), and a need to ramp-up quickly, the division chose to bring the CRM and commission calculation functionality of the on-demand model. They chose Salesforce.com and Xactly Corporation, respectively, to fulfill these functions. The one on-premise application they had access to was Oracle Financials for accounting.

The missing piece was to integrate these applications together. They chose to go with a packaged integration platform, adopting their subscription-based pricing model and on-premise software.

In addition to being the CRM platform for the new division, Salesforce.com is also serving as the de facto “enterprise service bus” to incorporate the accounting functionality of Oracle Financials, and to trigger Xactly to do it’s job of calculating sales commissions.

This use of Salesforce.com as a de facto on-demand ESB platform was noted in an August 2007 white paper entitled “Busting Myths of On-Demand Integration,” by Peter Coffee, Director of Platform Research.

“On-demand platforms exhibit the growing capability to provide a foundation for integration,” he said, citing a May 2007 announcement of the Salesforce.com SOA technology that enables the exposure and consumption of web services.

In the same paragraph he notes:

“This is not to say, however, that a move to a Web services protocol strategy (such as that of using a saas application such as Salesforce.com) is a prerequisite for on-demand integration…there are options available for use with the salesforce.com platform” such as custom coding or a third party integration platform.

In other words, on-demand applications, Salesforce.com being the most prominent, are quickly establishing themselves as integration hubs the way ESB providers such as Sonic Software, IBM’s Websphere, and BEA’s Weblogic were formulated to be.

These SOA solutions, however, are cost-prohibitive for smaller companies, divisions or departments, and are often managed by enterprise IT staffs who are unresponsive to the needs of the department. These smaller enterprises have to fend for themselves, and are adopting on-demand applications that require little to no IT involvement.

IT typically has to get involved when it comes to integration, according to Coffee. Such was the case with the educational products company. Their IT department provided the input that the newer division needed to give the technical “thumbs-up” to the integration solution. But due to human bandwidth issues they decided to go with a fully delivered integration solution as opposed to the traditional toolset that is typically sold to IT departments.

Tying together Salesforce.com, Oracle Financials and Xactly Corporation was done in the span of four months and cost less than $50,000. Why did it take that long? Because they had to take a breather between deciding on an integration vendor and a commission calculation vendor.

Compare that with enterprise application integration projects which typically take nine months or more and cost hundreds of thousands of dollars, and you can see why Salesforce.com, together with fully configured integration solutions, are quickly becoming the “integration hubs” or systems of record for the smaller enterprise.

Fernando Labastida is an account executive with Pervasive Software, and serves the Northern California, Pacific Northwest, Southwest Canada, Minnesota, and all of Latin America. He can be reached at http://www.labastida.com or 512-945-9273.