Basketball season always reminds me of the power of teamwork. The best basketball teams function as a cohesive unit for the good of the organization and the desired outcome. While bad teams can win games, accomplish some of the organization’s goals, and excel in some aspects, their success is usually unreliable. They typically fall short of the ultimate goal, and produce a bad product for the customer along the way.
Recently I was a part of a Customer Troubleshooting team that demonstrated the best principles of teamwork while working to debug a customer problem.
The customer problem presented itself as a component failure resulting in an error in finding expected files and output. As one of the senior engineers pointed out, we had seen this problem before, most often due to security software interoperability or misconfiguration. In rare cases, it was due to an isolated file system or system disk access problem.
Normally, a senior engineer would quickly work on the problem from their vantage point and years of experience, but in this case the SIOS team dynamically adapted to the changing information, applied what they knew about the software, OS, and found a solution that met the identified goals quickly. In fact, the resolution was much faster than if any one team member had demanded that the call follow a certain flow and logic.
Seven lessons from a SIOS Customer Experience Troubleshooting call
1. Remember the goal is to stay customer focused
At times it can be easy to believe that the goals are as follows:
● Identify the root cause of the component failure
● Resolve a customer issue on their test system as quickly as possible with no code changes
● Provide the customer with a series of steps and information to prevent the issue from recurring
However, the only goal that really matters for the team of Customer Experience engineers is to “Satisfy the Customer.” All of the other goals, outcomes, and details are secondary. Remember the goal of the organization and team.
2. Make a game plan
Our Team’s first approach to solving the problem was to check the obvious issues from previous customer feedback and support cases. Explaining the game plan was essential to our ability to orient the customer and validate that their goal was our top priority. It was also essential for aligning the team on the planned approach and the way the call would evolve. This alignment allowed each team member to understand where things were headed so that they could provide input and suggestions. Our game plan was to start by eliminating obvious issues and typical misconfigurations of the solution.
3. Communicate
Communication is essential. Not only between the team members, but also with the customer. Communicate proactively to eliminate misunderstandings, close gaps in understanding, and produce alignment. Communicate upfront to make sure that the problem is clearly understood, and helpful information is gleaned directly from the customer and also from among the team members. Providing clear and concise communication up front helps in several respects including:
a. Ensuring a common understanding
b. Identifying possible causes and issues
c. Keeping the goals of the customer in mind
d. Reminding one another of the current strategy
e. Updating, improving or collaborating on the output from checks, commands, or information being provided
f. Making the customer feel like an active – and valued – participant in the process
During our customer case, as the process and session evolved, our team communicated to the customer what we were hoping to rule out, what the results were with respect to initial findings, and next steps. We also communicated with each other with respect to our understanding of the output from each check or command, and what the possible options would be for continuing towards a resolution.
4. Revise the plan
“Everyone has a plan until they get punched in the mouth.” ~ Mike Tyson
Every plan sounds good until the results come in. When our team’s initial checks failed to provide an immediate root cause, we needed to revise the plan. Ideally, as in this case, the revised plan should be based on what was learned in the initial phase. Three ways to revise the plan include:
a. Looking for what worked.
What assumptions proved valid? Analyze the output and results from the initial plan. What worked? Where did you validate the initial assumptions? What information does this validation provide for the next steps? In our case, our initial validation proved that the system root disk was not the issue and that the failing component was isolated.
b. Look for what didn’t work.
What assumptions proved invalid? Where did your testing prove that your assumptions and instincts were wrong? What does this information tell you about the problem? In our case, it identified the problem residing in a particular call within the overall solution
c. Look for what was inconclusive.
What assumptions proved inconclusive? This is the area where your new plan has the most freedom to grow and provide results. What tests neither ruled out or validated an assumption? What was missing? What steps could be added or taken away from the strategy to provide more conclusive information? Inconclusive results often provide you with new avenues and theories to test. In our customer’s situation, the inconclusive results created a new set of tests that would later narrow the problem down to one of two components.
Be willing to adjust your approach as new information arrives.
5. Communicate the new plan
Communication is essential. Not only the original plan, but ongoing revisions and updates. Communicating the new plan is also important and this communication should be between all stakeholders and participants. Share what has worked, and what didn’t work. Be willing to slow down and explain how the next set of actions will contribute to the goal. In addition, it is also a good idea to keep the customer informed on the expected time it would take to continue towards the goal and resolution. Even though the resolution was within the time planned, this is one area that our team could have done better with regards to the issue.
6. Trust your teammates
Most successful teams are based on trust. They trust that as situations evolve, each person can adapt and step up. A good team learns that if their path is blocked, they need to look for a teammate who can continue towards the goal or score the basket. Before the pass they have to trust that this teammate will follow their training and make the next right play.
Of course, working together for several years enables our team to have a high level of trust, but in any team trusting the team looks like:
a. Accepting the feedback from one team member
b. Agreeing to suggestions. For example agreeing to review the differences between working systems and non-working systems.
c. Repeating commands and comments
d. Retrying commands from early analysis so that members could have a second look at error output and information
e. Waiting on the team to provide next steps
7. Celebrate success together
Great teams also celebrate success. After narrowing in on two errors, the team’s suggested comparison of the permissions of working systems with those of the problematic cluster worked. A quick review of the differences confirmed the problem-an accidental one character permission change during the Customer update process.
“Teamwork is great. It makes the dream work.” Individually, a great player can win a few games, score a lot of points, solve issues and troubleshoot problems, but ultimately they are unlikely to reach the goal. Individually, any of the team members on our call could have resolved the issue in time. Perhaps, the resolution would have taken longer, multiple hours instead of a single hour. However, collectively the issue was resolved in a much shorter window. How? Each team member focused on the goal, helped devise a plan, applied what they knew and had learned through experience, revised the plan, communicated well, trusted each other, and finished off with success.
