This blog post presents a first-hand experience from Sam Klaidman (Principal Adviser, Middlesex Consulting), who served as Vice President of Service in the late 1980s. In this narrative, Klaidman shares his account of a significant product recall involving some of the company’s most demanding customers, including major financial institutions and the U.S. military. He details how a proactive and transparent approach to addressing a component issue not only averted a crisis but also strengthened customer trust and led to unexpected future opportunities

Setting the Stage: High Stakes and High Expectations

In the late 1980s, I took on the role of Vice President of Service at my company. While it sounded like a prestigious position, the reality was our customers were some of the most demanding you could imagine. We served industry giants such as the largest stock exchange in the U.S., the two biggest credit card providers, numerous international airlines, major banks and brokerage firms, and even the missile defense organization within the U.S. military. It’s worth noting that in the event of a major disaster in New York City, the stock exchange’s communication networks were designed to be reactivated before hospitals and emergency services.

Our products were built for reliability, featuring fault-tolerant and redundant systems designed to be nearly impossible to fail. Our largest system, a common offering, handled a massive 4096 in-and-out cables. The only single point of failure was our input/output PC boards, each managing eight cables, with 32 of these boards residing in each of the 16 cabinets. Interestingly, our manufacturing team had the foresight to meticulously track every component used, recording the vendor, part number, and lot number – a highly unusual practice at the time. This level of detail would prove invaluable.

The Discovery: A Potential Crisis Brews

Typically, individual circuit board failures were considered routine maintenance, much like refueling a car – not a cause for major concern. However, one day, our two technicians responsible for testing and repairing all circuit boards returned from the field and came to my office with alarming data. They had identified a component lot that appeared to have unacceptable reliability issues. After reviewing their findings, I agreed with their assessment and immediately involved our VPs of Manufacturing and Quality. We quickly concurred: we faced a significant problem that required immediate action – replacing all circuit boards built with parts from this “bad” lot.

Next, I contacted our CEO, and together with the VPs of Sales and Marketing, we held a crucial meeting. The decision was swift and unanimous: a proactive recall was necessary. Our marketing team then developed a clear and concise script for our salespeople to explain the situation to our valued customers. Fortunately, the issue did not affect all our installations or even all the boards within a single system.

Taking Decisive Action: The Recall Process

The recall process took approximately six months to complete. Our team, along with our international dealers, worked in close collaboration with our customers’ network technicians to execute the exchange efficiently.

Here’s a breakdown of the process:

• Identification: Our field engineers identified the specific affected boards within the cabinets
• Removal: The faulty boards were carefully removed
• Transfer: The customer’s network technicians meticulously moved the eight cables to the exact locations on the replacement boards
• Installation: Our field engineers reinstalled the new boards

Return: The removed components were carefully packaged and returned to our U.S. facility at our expense.

The Unexpected Outcome: Building Unshakeable Trust

The response from our affected customers was overwhelmingly positive. They were genuinely impressed by our ability to pinpoint the low-reliability components and our proactive approach to addressing the issue before it caused any disruption to their operations. Our sales team reported that our customers viewed us as their most trusted partners as a direct result of our handling of the situation.

The positive impact of our proactive recall extended beyond immediate customer satisfaction. Within six months, our service group was entrusted to oversee and participate in a major project: relocating all the equipment in one of our key customer’s data centers. When our project leader, our regional service manager, arrived at the initial meeting, he found representatives from industry giants IBM and Pacific Bell also on the team. At the time, our annual sales were around $60 million. The customer’s Network VP explicitly stated to the group that we were selected to lead this critical project precisely because of the way we had proactively organized and executed the recall.

A Sustainable Practice: Giving New Life to Returned Components

You might wonder about the fate of all those returned circuit boards. Fortunately, we had a well-established and publicly communicated “Equivalent to New” policy for all new systems and all warranty and failure replacement electronics. We diligently updated all the returned boards, bringing them up to our stringent standards, and then returned them to our inventory to fulfill new orders.

Key Takeaway: The Power of Proactive Service Recovery

This experience taught me a powerful lesson: proactive and transparent service recovery can transform a potential crisis into an opportunity to strengthen customer relationships and build lasting trust. By taking ownership of the problem and acting decisively, we not only mitigated risk but also solidified our reputation as a reliable and dedicated partner. This experience aligns with the principles of effective service recovery, where addressing issues proactively and exceeding customer expectations can lead to increased loyalty and stronger partnerships