"One of the key aspects of deploying neural networks in a post-COVID world and even a pre-COVID world is an abundance of sanitized data, which most companies just don't have."
The Poirier Group is a data management consulting company. In the below interview, Peter Murphy, Consultant at The Poirier Group shares the importance of integrating high-visibility, real-time and sanitized data when developing sustainable data-based solutions such as chatbots and other neural networks like AI. He pulls from real examples in both call center and manufacturing settings to discuss data representation strategies and pitfalls, as well as the importance of measurement considerations in solution design.
At a very high level, without visibility into what’s happening, the success and failure of solutions can’t be measured.
Data plays this amazing role throughout organizations, from the top to the bottom. However, understanding what’s going on at a deep level is essential in providing solutions for design improvements.
High visibility, real-time data is a necessary tool for managers so they’re not flying blind. Without accessible data, an enormous communication effort is required on behalf of the whole company to keep everyone up to date. Whereas if the systems are set up for direct, real-time data to be analyzed live, it creates a more accurate representation of progress and success across the organization: how many units are being manufactured in a week? Has that gone up or down? Why? What happened? How can this be continuously improved upon?
To have a sustainable solution, it needs to be created and put in place. However, a baseline needs to be understood (i.e. how well you were doing before) and the success of the solution needs to be frequently measured against that baseline. Because it doesn’t matter if the best solution in the world is implemented if people stop using it in two weeks and management doesn’t know about it.
So, highly visible data systems keep people accountable, keep people informed and enable people to take action to better their company, and better themselves.
There’s a variety of reasons why people don’t count on this data. Some don’t know they need to; some think it’s not important. A couple of main barriers include a false assumption that data will trickle up to the appropriate levels of management; and the upfront cost to implement the infrastructure.
1. Assumption that Data will Trickle Up: A lot of people, especially in newer management roles, have an enormous amount of faith that data and information will trickle up to them, but it doesn’t.
There’s the famous story of the fan belt solution. A box-filling machine at a company was not working properly. Sometimes the boxes were left empty when they weren’t supposed to be and the company didn’t want to waste the time figuring out which ones were empty. So they spent several million dollars and installed a weighting scale system so that when a box went along the conveyor belt, and the scale felt that the box was underweight, an alarm would go off and make everybody come over and take the box off and fix it.
It was successful for a while but about two weeks into it, the data showed that the alarm wasn’t going off anymore. There were no failures, but they hadn’t changed anything. They didn’t know what happened. So, they go down to the floor, and there’s a standing rotating fan next to the belt.
One of the workers on the floor had set it up, and as a box went by on the belt, if it was empty, the box was blown off the belt by the fan. When they asked him “Why did you do that?” he said, “the alarm was really annoying.”
The point is to trust people who are actually working on the shop floor and at various levels in the organization, because they know the problems and they likely have the solutions.
A lot of people have faith that those solutions and information from various levels of an organization will trickle up to management but they often don’t. The data just shows it afterwards. In this example, they wasted their money, but they wouldn’t have known they had wasted their money if they hadn’t had that data of the “unexplained” remedy of the problem.
2. Upfront infrastructure costs, particularly when implementing processes to connect data between different systems. It can be difficult to get two systems to talk to each other but is essential to answer core questions about a business. However, these upfront costs end up saving more time, money and headaches in the long run.
CASE STUDY:
At a call center, there was a system to measure the reason codes for why a call happened that was manually entered every time. There was a separate system, which measures other call metrics (i.e., how long a customer was on hold, how long they waited in a queue, etc.). However, these systems didn’t talk to each other. There was no way to answer the question “How long does a certain type of call take on average?”
They didn’t account for that in upfront infrastructure costs because they didn’t think they needed it, even though this information is valuable when improving the customer experience. It’s a lot more expensive and more complicated to do after the fact than to do it initially.
There are several different strategies. The one we often recommend is a balanced scorecard (BSC). A Balanced Scorecard is an excellent system for measuring the current state of a variety of different aspects of a business. It’s very multi-purpose and can measure metrics, keep track of OKRs and bring clarity around whether a company is doing the right things to move towards a strategic goal vs. just doing good work, or “busy work”. It’s a robust and insightful tool to measure what’s important and figure out exactly what needs to be paid attention to. A balanced scorecard is also a tool that can be easily integrated into a live dashboard (vs. a static document) if there is the right data to track real-time metrics across departments.
1. Not linking up the data between two systems
2. Lack of data representation
3. Improper use of math and statistics: Whether something actually implies causation rather than just correlation, or how good a correlation is. A trendline can be put on anything, but it doesn’t mean something valuable.
a) Hold times in a call center might be correlated with the volume of calls. But presenting that data as hold time causes more calls would be inaccurate.
b) If there is an average call time of six minutes on Spanish calls, and three minutes on English calls, it does not mean the average time spent on a call is four and a half minutes because there isn’t the same number of Spanish calls as English calls.
4. Improperly Measuring Data: How something is measured can alter what it says so it’s important to know what the data means and how to appropriately collect it.
CASE STUDY:
A customer calls into a call center and waits in a queue, then speaks to an agent and gets passed to a manager. The manager then realizes the customer is in the wrong place and sends them to another queue, where they talk to an agent who solves the problem. Is that classified as a single call? Two calls by queue? Or as three touchpoints with an agent?
All of them are correct, but they all say very different things. Is the data for this call aggregated by customer or by agent? Is the agent workload being measured or the customer journey? It’s important to be very conscious when presenting data and to understand what the data is actually saying.
Knowing how well a solution performed relies on high visibility data.
To refer back to the fan blowing away boxes as mentioned earlier, that was an important outcome of their measurement. They were measuring it, they were tracking it, and they learned that they had a better solution.
If solutions aren’t measured, it’s impossible to quantify how successful they were and if the time and effort were worth it. It will also be more troublesome to justify future endeavours because it can’t be proved that the actions took were valuable.
So, proper measurement is essential. We at TPG, do a ton of work with Lean Six Sigma, (i.e. DMAIC process – design, measure, analyze, improve, control). The ‘M’ representing Measure means knowing that what is being done is impactful and making sure the correct thing is being measured. Meaning, setting a baseline to understand that the problem being solved is actually a problem, or is the most important problem. The focus is not measuring on the other side to see how well a solution worked.
A lot of wasted money can be spent to solve something that wasn’t a problem in the first place. And people do that all the time.
With regards to applying best practices, the Lean Six Sigma methodology has stood the test of time, and it is exceptionally strong. However, more fundamentally, someone needs to be in charge of the data.
There are a lot of companies that don’t explicitly have these roles. It doesn’t necessarily need to be a data scientist, but it needs to be someone who can manage and control the data and make sure everyone up and down the line interacting with that data knows what that data means, where it comes from what it represents. Because if there is a breakdown in communication, that data is worthless.
Without someone managing all the data, it may result in unreliable systems based on people reporting different things in different ways. Especially for larger companies, it’s important to organize data and figure out who is responsible for it, and how it’s going to be reported.
Data will play an enormous role in measuring efficiency, determining how to improve and determining how to manage remote workplaces going forward.
As systems like AI and deep learning become more prevalent, it’s important to recognize that these are neural networks. One of the key aspects of deploying neural networks in a post-COVID world and even a pre-COVID world is an abundance of sanitized data, which most companies just don’t have.
Moving into a post-COVID world where work-from-home is a common practice, there will be more digital work, an enormous amount of digital interaction, and more data at our fingertips. With this increased amount of data on how people interact, how they work and what they do, it’s even more important to make sure those fundamental data practices are in place because otherwise, it will just produce an enormous amount of junk. And that’s not helpful to anyone.
Chatbots are predominantly based on neural networks. The goal is basically to make a Frequently Asked Questions page more interactive. And many customers are not particularly tech-savvy, which still make up a large proportion of online purchases. So, building out a chatbot system to have that conversation can help people who aren’t comfortable navigating websites by themselves. When building out these chatbot systems, it’s important to keep in mind what people are frequently going to say and how they’re going to say it. Bots are trained based on examples of conversations. Since there are many different ways people can ask questions, they need to be trained on all of them.
One of the important guidelines for a chatbot is to make it very clear that it’s a robot so a customer doesn’t think they are talking to a human and get frustrated.
Chatbots are another example of building proper systems. They provide an enormous amount of data but if they are not carefully set up, it can cause more harm than good. So it’s very important before we start the process to make sure the correct systems are in place.
The key trend in innovation is applying AI and neural networks and using data to derive solutions that people wouldn’t necessarily see. But, the key opportunity is that 90% of companies are four or five levels back from where they can use it. They don’t yet have the foundational infrastructure and data to implement those solutions. While AI is a shiny Buzzword, unless a solid data foundation is built and there is a data warehouse to keep all records sanitized and up to date, there’s no point in forcing a solution like this.
The opportunity is not in applying neural networks, it’s in enabling companies to apply them in the future by implementing the fundamental processes of good data practice.
Tell me a little about your career path. What made you get into consulting?
“Well, my career path has been an interesting experience, with a lot of learning involved. I mean that in a good way. I’ve worked in my life as a sound engineer, a camp Counselor, I worked as a technical analyst, doing research, a master’s student, I worked on smart city projects, and I’ve done sales.
I love trying new things. I love doing different things all the time and learning new fields as often as I can. And my basic skill sets are speaking and math, which has proven to be a fairly versatile skill set, which is good because I did my degree in chemical engineering and it has been fantastic to apply that everywhere.
I got into consulting because I love learning, I love jumping around to different things, trying new things. I wanted to find a career field where I can do things that I love everyday; where can I learn new things; speak to new people; and apply my math skills. Consulting ticked every one of those boxes. When I was given the opportunity to apply for TPG, I jumped on it and have been happy ever since.”
Is there anything that you’re passionate about outside of your job role?
“I love teaching, and I love mentoring. I work and I have worked on a bunch of summer camps, I take on judging and mentoring roles mostly at UofT where I have connections to do so, but also anywhere I can.
I like forms of public speaking, mentoring, teaching where you can help people figure out what they want to do figure out who they are. And I find it to be a really rewarding experience.”
What drew you to TPG? And what do you like about working here?
“A friend who worked here previously recommended TPG to me. He highly recommended the organization because I had a series of complaints about larger consulting firms in the way they operated. I didn’t like the idea of coming in, handing a solution and leaving. And when I told him that, he went, ‘Oh, you have to come to TPG, that’s perfect, they all share that same philosophy.’
So I was recommended it because it fit my values. It was somewhere I would be a good cultural fit and something I would enjoy doing. And honestly, it has been true. It’s wonderful to work in a place with so many people who match your values and embody integrity, honesty, and intelligence. We have smart people who do good jobs, and we make sure our clients are happy at the end of the day, and that’s something I value. So it’s been wonderful to work here.”
