In our experiences covering more than 75 litigation matters involving software and project failure, the same themes, allegations and defenses appear time and time again (see Figure 1).  Depending on the specific facts, some arguments are persuasive, others are red herrings.  Our experience suggests some high-yield practices for not only preventing project problems, but for winning in court -- should it come to that. 

With all of the "panaceas" introduced over the decades to help ensure successful system solutions, such as structured methodologies, JAD, object-oriented programming, test coverage analyzers, COCOMO models, function point metrics, CMMi standards, automated tools, agile methods, TQM, inspections, and more, successful systems still elude many otherwise successful businesses.

Figure 1.

Example Cases

The following two example cases illustrate several major themes that recur in litigation in systems development projects.  In reviewing such highly summarized cases, it is easy for the reader to underestimate the amount of know-how, sleuthing and preparation required. Once you know the answer, it's so simple -- but in some cases we have analyzed 500,000 -1,000,000 pages or electronic images of documents to arrive at and support our conclusions. With two or more sides to every story and even more contradictions, it takes hard work and tenacity, along with simple, independent, credible testimony to the triers of fact to win the day.

This first example case began as a copyright and trade secret infringement case but it ended up turning on testing objectives. Sometimes you never know where the cases may take you.

In this case, one party, a small systems applications developer (the "Developer") very early on built, licensed and maintained vertical software for the wholesale distribution business when no appropriate solutions were available. Eventually he enhanced and globalized it until it became the very popular and robust industry leader. Now in his seventies, he wanted to sell the software company in a way that kept his employees gainfully employed within the acquiring company.

The opposing party (the "Acquirer") was a billion dollar systems developer and integrator that already enjoyed success with its own products - but in the manufacturing arena. The Acquirer believed it was presently losing customers who wanted integrated manufacturing and distribution package and thus wanted an outstanding complementary distribution product to complete its product line offering.

The Complaint

After looking at many available systems, the Acquirer gave the Developer a multi-million dollar letter of intent to purchase the assets and software of the latter's company, with the right, "in its sole judgment", to cancel the deal if after intense examination the software did not meet the needs of the Acquirer.

From preliminary discussions and visits the Acquirer knew about the organization and the software, its history, its pricing and costs, its competitors, its functionality and capabilities. The Acquirer claimed it now needed to review the Developer's software code, technical architecture and structure in detail; approve its design; interview the customers, programmers and testers, trainers, implementers, maintenance staff, hot-line group, marketing staff, and sales staff; review standards and documentation; review and analyze the testing suite.  The Acquirer would then apply its own system tests, to be sure that the system worked as promised and could be maintained by the Acquiring company.

This all sounded reasonable at the time, given applicable law required the Acquirer act in good faith -- even though it could cancel the deal in its sole judgment. The letter of intent also provided that the intense review would be completed within 45 days.  Well, Bang! On the 45th day exactly, the Developer received a one-page letter that indicated that the Acquirer would not complete the acquisition because the system did not meet the performance requirements promised by the Developer, or the profitability requirements needed by the Acquirer. The Acquirer in a very short time came out with a similarly configured system.

The Developer at first charged the Acquirer with copyright infringement.  Following our initial involvement in the case as expert witnesses, and based upon our opinions, the attorney charged "bad faith" and "trade secret misappropriation"  -- very difficult to prove!

The Analyses, Strategies and Opinions

During our research we discovered that the Acquirer was targeting the $2-$20 million dollar wholesaler distributor customer with particular focus on SIC codes 50 and 51. Tracking back to the actual business plan of the Acquirer, we also discovered in their mission statement that they said, "We are to get our product to market in a record 11 months -- and not the 23-26 months that we estimate to develop this system in-house from scratch." 

Through detailed research regarding the average order dollar values and number of line items per order for relevant SIC code orders in that time frame, we determined that the highly "over-transactioned" performance tests performed by the Acquirer supported distribution companies with sales between $350 and $400 million in annual sales, 10 times  larger than  the Acquirer's targeted customers.   We concluded that the tests were, in fact, intended to make the system fail.  Fortunately, the actual speed and timing of the transactions were logged automatically on the system during these "load tests".  This enabled us be very accurate in our analysis instead of having to estimate the transaction rate and response times.

With our analysis and opinion as their support, the Developer's attorneys argued that the tests were performed in "bad faith".  This legal attack would not allow the Acquirer to avoid the letter of intent because of the results of the erroneous tests. We went on to prove that if appropriate loads were used, the system would have performed within the promised sub-second response time 98% of the time and never over a 2.5 second response time (vs. the unacceptable 20-45 seconds resulting from the Acquirer's loads).   .

Bad Faith

Next, we established motives for bad faith to show the jury why the Acquirer went through all this trouble and effort if it never intended to accept/acquire the software. We discovered that of the 6 people who reviewed the system in excruciating detail, 4 of them went on to develop the Acquirer's own system from scratch. And while their original estimates indicated it would take over 2 years to develop, in fact the system was running and available in slightly more than half that time.

We pointed out that through its intensive review and analysis of the system, afforded them under the letter of intent, the Acquirer was able to:

• Acquire the ability to bring their product to market more quickly by understanding how certain complex algorithms worked in the Developer's system

• Learn how the best available system was designed and implemented, including the use of standards, technical architecture and design considerations, screen navigation, levels of reporting, and module interfaces

• Adopt a suite of functionality that was studied and evolved by the Developer in over 17 years in the marketplace -- thus avoiding a long trial and error process in determining what current customers really needed and wanted (i.e., the Trade Secrets)

• Learn the future enhancements that were slated, and their estimated costs and benefits

Trade Secret Misappropriation

While it is very difficult to prove infringement of copyrights based upon similar functionality (because copyrights only protect the expression of the function, not the idea of the function), we were successful by employing unusual tactics.  We argued that the particular set of available and future features and functions was indeed a trade secret of the Developer.

Furthermore, by copying this suite of functionality, even though with a different "look and feel", the Acquirer would be able to (1) bring its product to market more quickly, and (2) win away customers and prospects from the Developer -- two things that make trade secrets so valuable -- and that could only be accomplished by having access to other developers and documents (under false pretenses).

We were also able to bring into question curious similarities regarding interfaces between functions in both the Developer's and Acquirer's systems. While there were several "public domain" ways to implement any of the "allegedly copied" critical functions and algorithms, the Acquirer almost always did it similarly to the Developer -- an astronomically low probability for 12 complex functions if no copying occurred.

Lastly, we developed an analysis that showed that the Acquirer would be able to achieve the profit goals and margins targeted in its business plan for the new system because of the synergies provided by the Acquirer's manufacturing applications and the market position already enjoyed. We developed several launch, pricing, and maintenance scenarios to prove this point.

The Verdict

After four years of arguing this case, it settled almost literally on the courthouse steps, the day I was to testify, in favor of a happy Developer and his attorneys.

"Shoes, Shirts, In-Stock, Out-of-Stock"

This example deals with the implementation of a world-class "ERP" system for one of the world's largest retail clothing operations ("Customer")  -- who needed the system before fast approaching Y2K.  The vertical retail version of the product was essentially "brand new" (i.e., like a beta) and would require, more than ever, dramatically increased customer-site installations, volumes of data, interfaces to third party systems, and challenging distribution networks.  Former systems developers and consultants jumped on the bandwagon of the leading ERP companies to become "leading" integrators of those ERPs - and they had to trail-blaze many "firsts".

The Customer decided to license and modify the "best" (most expensive) integrated ERP solution, rather than buy smaller but still "best practices" leading application packages from different vendors,  that they would then integrate. To avoid stock-outs while maintaining a reasonably low level of inventory, the ERP had to keep track of thousands upon thousands of products, sizes, fabrics and colors, so that the customer could forecast, know how much was sold, how much to order, how to price in relation to competitors in each market, when to mark down seasonal or slow moving products, and to distribute them efficiently and on-time to all of its one thousand plus stores.  All of this required number crunching on a scale not previously tamed by an ERP. 

Several other factors made this implementation a real challenge: 1) the project faced the hard and fast Y2K deadline 2) the customer wanted the "Cadillac" of ERPs, even though they faced severe budget restrictions, and 3) the customer wanted to do much of the work with its own staff to save money, even though that staff lacked the experience and qualifications for the job.  With no time to run a proper competition for an integrator, the customer turned for guidance and advice to the consulting arm of its auditor ("Consultant") and proceeded without properly formalizing who was responsible for what.

After several schedule slips, the project squeaked by the deadline and went live.  However, the customary period of bumpiness following any ERP implementation took longer than usual, and the customer was unhappy about the consequent loss of profits and consumer goodwill and subsequent stock price drop.

Prelude

The customer had already successfully sued the ERP developer with whom the customer had its own contract, not involving the Consultant.  The system was missing several critical functions at go-live, and many aspects of the new system could not be adequately tested in the resulting compressed time frame. 

The customer was not satisfied with the recovery in this settlement, and so it turned its sights on the Consultant that it asked to guide the project.

The Complaint

In the complaint, the Customer alleged that the Consultant was hired as a general contractor-type integrator, responsible for all aspects of the project and responsible for delivering a working system on time. The customer alleged that the Consultant, abusing its audit relationship of long standing, did not have the qualifications or staff needed to successfully manage the project and deliver the working system on time. Further, the Customer alleged that the Consultant failed to give proper advice about hardware requirements, functionality, and how to manage the developer. The Customer alleged that the Consultant failed to properly test the system before going live. Ultimately, the Customer blamed the problems experienced after go-live on bad design and configuration, which it blamed on the Consultant.

The Analyses, Strategies, and Opinions

Hardware: It was evident from the outset that the implementation of the ERP would needed lots of processing power to handle all of the number crunching required within the limited nightly batch window. There was a deep concern about what kind of hardware would be required to handle the very data-driven US retail market. No configuration this large had ever been developed and used.  A preliminary stress test and performance test was prepared by the developer and reviewed with the Consultant and Customer.  The test, which by necessity was based on a smaller operation, raised serious concerns about how much hardware and computer horse power would be required to assure that the 1,200 stores could be polled and processed daily, analyzed, and turned into inventory and transportation requests that would replenish goods at each of the stores before opening time the following morning.  Failure to meet the daily turnaround schedule would be catastrophic, not just inconvenient.  Accordingly, both the consultant and the developer made strong recommendations from the outset that the customer should purchase the most powerful servers then available on the market at that time.  Despite those recommendations, the customer opted to buy servers from its current hardware provider, based on that provider's unsubstantiated promises about throughput performance, and based on the fact that this option was cheaper than the purchase of more powerful servers. We were able to determine that the hardware selected caused a number of problems on the project. First, it caused delays, as it took much longer than expected for the hardware vendor to pass the preliminary performance stress test, even to make crucial decisions and trade-offs about how to design and configure the ERP functionality.

Authority and Control: A major thrust of the customer's case was that the Consultant was a general contractor-type integrator, and much of the opinion of their expert witness was premised upon that allegation. Among other things, we opined that a general contractor-type integrator typically has authority to: determine what tasks need to be completed; define a schedule of deliverables and deadlines; determine how many personnel and who to assign to the various tasks; hire and fire subcontractors on whose deliverables the integrator depended; and sanction or make changes to address poor quality or tardiness of deliverables.  We showed through contract and meeting documentation that the Consultant did not have such control or authority.  The plaintiff's wish to hold the Consultant responsible without giving it authority to manage to succeed was not a defensible position.

Project Management: The Consultant played a major role in the planning the work. Senior managers of the Consultant were assigned as project managers, and these managers created project plans for the various teams. We were able to show that authority over those plans however was retained by the customer. We were also able to show through meeting and planning documents that the customer retained control over the work, deciding which resources would be assigned to do what work and deliverables, and in what sequence. We explained the importance of the fact that the customer established direct relationships with the other system and project vendors, and how that prevented the Consultant from exercising any degree of influence over work that was affected by those other vendors.

Qualifications: The customer alleged that the Consultant concealed its lack of personnel after promising highly qualified people. We were able to demonstrate that the customer understood the risks in being one of the first implementations of the retail ERP in North American, and that the fallout was that fewer resources were available with prior experience in implementing that solution. We showed those conditions are inherent in any pioneer project.  We opined that the Consultants assigned personnel had sufficient qualifications for the work, and adequate transferable experience and expertise from predecessor versions of the system and implementations.

Gaps:  Initially, the customer engaged the Consultant to identify a few hundred functionality gaps in the developer's solution.  Thereafter, the customer excluded the Consultant from the process of managing and resolving those gaps.  The customer established its own contract with the developer, and itself managed the developer's work on the gap solutions.  We were able to demonstrate that the Consultant bore no responsibility for the poor quality of the gap solutions and the delays in their completion.  We were also able to explain how the delays in the gap solution caused other delays in the project, including the extension of the design phase, and the consequential compression of the testing phase.

Scope:  As in most projects, the customer's business personnel played a crucial role in the design, configuration and testing of the system.  Documents that we researched and analyzed revealed the customer's business experts were stretched too thinly, often unavailable to make timely decisions, often disagreed with one another, and revisited and changed decisions they had previously made.  We were able to explain the impact that this has on the design, and the wasteful need for more time and effort on re-working and re-testing modules on the system.

Testing:  From the outset, the Consultant insisted that one of the Customer's experienced staff be appointed as "Test Integration Manager" (TIM) to oversee the testing of all components and integrating them into a cohesive whole. Only very late in the project did the customer accept this recommendation.  We were able to show that the Consultant's TIM did the best under the circumstances to get the most important testing done within the very limited time frame left. We explained that the Consultant could not be blamed for overlapped and incomplete testing, where that incompleteness resulted from delay caused by the customer's conduct and choices. We also opined that, under the circumstances, the Consultant did an admirable job in adapting its methodology to cover the crunched testing schedule which, in fact, contributed to the success of this project.[1]

Poor Communication:  The Customer lambasted the Consultant in the litigation over the Customer's perception that the Consultant did not adequately communicate its recommendations and observations about project risks. However, Consultant managers testified that they communicated all of their concerns and observations about project risks verbally to the Customer's managers on an on-going basis, and to customer top executives (President and CFO) as needed.  We were able to explain that the optimal mode of communication often depends upon the particular circumstances of a project, and the personalities involved.

Post Go-Live Problems:  The ERP system did go live with limited functionality, due to a number of pending crucial gap solutions.  The system did experience some stress and bumpiness with the customer blaming the problems on design decisions made by the Consultant, and inadequate testing of system components.  We were able to show that the problems emanated in part from the underpowered hardware, and that once the hardware was upgraded most performance problems vanished. 

We opined that several other factors hampered the functionality, including the late implementation of a strong archiving procedure to offload the more than 10 gigabytes of daily detail data to a summarized data warehouse, originally and repeatedly recommended by the Consultants but refused by the customer until well after go-live.

Moreover, we were able to demonstrate that some problems could have been avoided if the customer had followed the performance tuning recommendations that the Consultant made prior to go-live.  We demonstrated that many problems related to the customer's own choices, such as lack of proper training of users, poor qualifications of customer technical staff, poor configuration of the customer's data center and a poor conversion process conducted by the customer that led to initial data and reported stock-out problems.

The Verdict

The Customer's expert issued a rebuttal expert report, which went noticeably on the defensive.  Several months later, the court made a ruling on the Consultant firm's motion for summary judgment that devastated the Customer's case, and a  few months after that, the customer walked away from the litigation.

What Helps Projects Succeed, Also Helps Success in Court!

There are no guarantees for winning any case - but consider using the following maxims for systems development/implementation projects to help you prevent the above scenarios and increase your chances of project or courtroom SUCCESS.

Stay real - avoid over-optimism! Create: realistic and shared expectations and contract language; realistic estimates and schedules with contingency plans; realistic progress reports and root-cause analyses for missed targets and getting back on track.  "Realistic commitments are attainable and sustainable."[2] And measurable!

Commit to, and make available, the "appropriate" staff, users, vendors and management throughout the life of the project.  Experience counts!

Create the criteria for defining "acceptance" and project "success" in writing before the project starts. The harder this is to do, the more important it is!

Employ a good "standard" systems development methodology from the start,  and ensure project staff are trained in it. Employ automated and collaborative tools. Ensure the appropriate project communications infrastructure and repositories are available and used.

Ensure that effective executive escalation processes are in place to address problems and issues conclusively.

Deliver top quality deliverables -as if a jury will review them 3-5 years from then - because it can happen!  Know what you are signing (off) and what it means.

[1] Note: we opined that this system/project was a success. Additionally, one year after go-live and extended bumpiness, the customer began achieving the bargained for benefits and told its investors and analysts so.

[2] Radical QA: A Guide to Managing Expectations and Results. http://www.wsrcg.com/Articles/Radical%20QA%20Article%20-%20Feb2.pdf

Comments (0) >>

Write comment

You must be logged in to post a comment. Please register if you do not have an account yet.