53 Percent of Reference Customers Fail to Receive Positive Return, According to Nucleus Research Study

Customers may experience initial benefits, but many deployments hindered by inexperienced consultants and poorly defined business objectives

Nucleus Research (NucleusResearch.com), the leading provider of bottom-line focused technology research, today announced the results of an independent study of Art Technology Group (ATG, Nasdaq: ARTG). Fifty-three percent of ATG customers interviewed did not believe that they had achieved a positive return on investment (ROI) from their deployment of ATG's e-commerce platform. Despite some early benefits gained from implementing the platform, overwhelming consulting costs and challenges, coupled with a lack of clear business objectives for using the platform, prevented many companies from achieving payback from their investment.

The Real ROI from ATG

Over the past several years, companies that have invested in ATG's e-commerce Web sites and platforms have derived some benefits from their deployments, but overall have not achieved the ROI they expected. While most hoped for improvements in revenue through online sales, reduced costs of supporting customers and partners, and scalability for supporting ongoing e-commerce objectives, obstacles such as inexperienced consultants and unclear e-business objectives often prevented full utilization and return from the technology. Concerns about deployment challenges and the technology's support for future returns were so great that nine of the companies listed on ATG's Web site had discontinued using the platform.

The full report is available for download from Nucleus Research at NucleusResearch.com

"ATG has demonstrated it has the ability to initially provide personalization and an out-of-box commerce solution, but fails to follow through and deliver a complete and cost effective product," says Rebecca Wettemann, Nucleus Research's vice president of research. "The majority of referenced customers are still waiting to achieve ROI on their ATG deployments, and many have discontinued service as a result. Companies that can complete a pre-deployment ROI analysis taking into account costs, returns, and risks associated with the e-commerce project stand to attain the greatest return on their investment."

The report is the latest in Nucleus Research's "Real ROI" series, which most recently has included reports on SAP (NYSE: SAP), Manugistics (NASDAQ: MANU), i2 (NASDAQ: ITWO), Siebel (NASDAQ: SEBL) and Cognos (NASDAQ: COGN). The "Real ROI" reports provide Nucleus Research's subscription customers with fair, unbiased data about the costs, benefits and financial best-practices associated with deploying a specific vendor's enterprise technology deployment. Nucleus' analysts use a standard methodology built around a series of in-depth interviews with independently researched customers who have implemented a specific technology. Nucleus analysts independently contact these vendors' reference customers-without the vendor's knowledge-in order to take a broad look at the returns and costs associated with various technology deployments.

Consulting Costs, Lack of Clear Goals Challenge ROI

Fixed costs including software, hardware, and license maintenance of ATG deployments were often within budgeted expectations. However, consulting engagements with ATG partners did not deliver the expected value—most respondents felt that the consultants lacked the requisite skills and experience needed to deploy the ATG platform. Many ATG consulting partners disappointed companies with their inability to perform at the levels required for this type of implementation. These challenges ultimately impacted project costs and return on investment. Companies ended up spending more than they had initially budgeted because of extended deployment times due to a lack of understanding on the part of the consultants, and underestimated levels of customization.

Many companies based decisions to invest in an application solely on system design considerations like scalability, fault tolerance, and load balancing rather than a combination of design criteria and business objectives. But many projects that were justified by the projected scalability benefit were not accompanied by any assessment of the projected growth rate of the Web site or the number of additional development projects that could reasonably be undertaken within a certain timeframe. Ultimately, the scalability was unnecessary.

The cost factors of ATG:

• Software: Nucleus found that the average ATG software license investment was $1,266,154 and companies paid an average of $207,583 annually in license maintenance.
• Hardware: For the ATG deployments studied, the average investment in hardware was $1,643,333, with a median hardware cost at $150,000.
• Consulting: The majority of companies invested in consulting with ATG services staff or consulting partners. The average consulting spend for ATG deployments was $2,625,000. Some companies spent less by leveraging internal developers and other technical resources.
• Personnel: On average, companies deployed 7.1 full-time employees (FTEs) to manage their deploymentsand 5.5 FTEs to support the solution on an ongoing basis.
• • Training: The average needed for ATG training was 53 hours and in cases where familiarity with the technology was low, training time was as high as 160 hours. Levels of training were often dependent on existing Java-related skills of developers.

Potential ROI Benefits

Nucleus found companies achieving benefits from ATG in three areas:

• Improved technology management. In general, the degree to which companies benefited from improved technology management was determined by a company's original IT environment prior to deploying ATG. Many companies had been supporting e-commerce initiatives through the use of homegrown platforms, EDI solutions or other non-scalable systems. The more unreliable and less scalable the system, the greater the productivity and returns from migrating to a more robust system.
• Improved revenues from online customer interactions. Few companies provided an exact percentage increase in revenues as a result of ATG, and most companies only reported anecdotal evidence of this benefit. The impact on online revenues varied company to company. The transition to more marketing-oriented sites built allowed for expansion of sales operations, and could introduce personalization to the end user experience. Because of this some found revenues that came in through the online channel would rise.
• Improved customer support operations. Companies using ATG were able to reduce routine customer calls and transaction costs, and improve quality of customer interactions. This occurred because of the ability to divert customers to the Web for completing certain transactions, and allowed them to search for product-related information. Instead of answering basic questions, customer service staff could focus on motivating callers to purchase.

Study Methodology

Nucleus analysts independently contacted customers listed as references on the ATG website without ATG's knowledge to gather data for this report. Of the 92 companies listed on the site and contacted, Nucleus received responses from 19 willing to participate, 9 companies that indicated they were no longer customers of ATG and retired the platform, 37 did not respond and 27 declined to participate for various reasons. The information was gathered on behalf of Nucleus clients looking for accurate and unbiased data and analysis about the costs and returns associated with a technology deployment

Nucleus asked companies about various aspects of their ATG deployments that would impact ROI, including the following:

• Why and when did you select ATG?
• Which SAP solutions have you implemented?
• How long did your ATG deployment take?
• Did you stay within your deployment budget?
• What are the most significant returns and benefits from your ATG applications?
• How much did you spend on the project; in particular, how much did you spend on software, hardware, consulting, training, and personnel?
• What were the key deployment challenges?
• Do you think that the costs of your ATG deployment have been outweighed by the returns?

The full report is available at NucleusResearch.com. Nucleus Research clients who are interested in more detailed information concerning the costs and returns of specific ATG applications should contact Nucleus Research client services.

About Nucleus Research

Nucleus Research is an independent global research and advisory firm that provides CFOs and CIOs with the financial and technology insight they need to make clear, accurate assessments of the returns from their technology investment. The company is the first firm of its kind to blend financial analysis with comprehensive technology expertise to deliver 100% impartial return-on-investment (ROI) information to organizations worldwide. Nucleus Research uses an uncompromising set of processes and tools to evaluate the financial return on IT assets throughout a technology's life cycle, from selection and deployment to upgrade and retirement. Nucleus Research's ROI assessment methodology can be applied to virtually any technology investment. For more information, please visit NucleusResearch.com.