HDI - Implementing Problem Management
7.1 Overview
7.1.1 Description
A problem may be the result of one or more incidents; the Problem Management process seeks to identify the root cause of those problems. The goals of the process are to minimize severity and any adverse impacts caused by problems and ultimately to eliminate them where it makes business sense to do so.
7.1.2 Relationships to other processes
|
Each ITIL process relates with the others to some degree. Some of the processes are completely dependent on one another; this is the case with Problem and Incident Management. It is sometimes difficult to differentiate between these two processes. While the role of Incident Management is to find a quick resolution or a `fix' so that the customer can get back to work as quickly as possible, Problem Management's purpose is to identify the root cause and eliminate it. When the symptoms are addressed and not the problem itself, the problem will almost certainly reappear.
|
The escalation of an issue from problem to a Request For Change (RFC) securely links the processes of Change and Problem Management as well. Knowing that a `problem' can be the result of one or more incidents, these relationships can be illustrated as shown in Figure 7.2 below.
While Change and Incident Management are the two processes that most directly affect or are affected by Problem Management, other relationships can have an impact as well. The failure of any of the other ITIL processes can cause an incident and/ or a problem. For example, poor planning for load volume (Capacity Management) or lack of redundancy in critical systems (Continuity Management) can affect the availability of hardware, software, data or the network (Availability Management). It is easy to see the potential for these situations either to be, or to become, a problem for an organization. Similarly, a failed change that has been deployed (Release Management) can also generate a problem for the support team and core business users.
7.1.3 Key inputs and outputs to the process
Problem Management can be broken down into two components, reactive and proactive.
In reactive mode, Problem Management is 'fire-fighting', similar to the mode that Incident Management typically engages in. All service events, whatever the cause, are (or should be) logged through the Support Center. Incident Management is a key source of information in isolating and identifying the fact that a problem exists. When Incident Management recognizes the possibility chat an incident or group of incidents may be a problem, it then becomes Problem Management's responsibility to find the cause and eliminate it. Through this cycle disruptions for the customer are reduced by providing a workaround, or by solving the problem entirely.
|
These temporary fixes or permanent solutions are recorded in Configuration Management and become part of the knowledgebase. By reducing the number of issues that need to be escalated to second or third level support staff, the first level resolution rate will increase.
Additionally, Problem Management has responsibilities that are proactive in nature. Most significantly of these are root cause and trend analysis. In these, it becomes apparent why the has collected in Incident Management is so important. The quality of information gathered a trouble tickets directly affects the simplicity with which root cause and trend analysis can be performed.
|
| Description | Source | Importance
|
| INPUTS
|
| Incident details | Incident Management
| High
|
| Configuration details |
Configuration Management | High
|
| dowtime details |
Availability Management | High
|
| failed change details |
Release Management | High
|
| Defined workarounds |
Incident Management | Medium
|
| potential problem reported |
Incident Management
Capacity Management
Availability Management
| Medium
|
| Trend reported |
Incident Management | Medium
|
| Event survey results |
Support Center | Medium
|
| Annual survey results |
Customer Service Management | Low
|
| anecdotal evidence |
all | Low
|
| OUTPUTS
|
| Known error |
Problem Management | High
|
| Fix/workaround |
Problem Management | High
|
| Management/metric reporting |
Problem Management | High
|
| Major problem review |
Problem Management | Medium
|
| Request for Change |
Problem Management | Medium
|
| Updated and closed problem and known error records |
Problem Management | medium
|
| Suggested improvements for procedures, documentation, training needs |
Problem Management | Medium
|
| Anecdotal evidence |
Problem Management | Low
|
7.1.4 Possible problems and issues
Not separating Incident Management from Problem Management: these processes are closely related but distinct in their goals. The two processes have clearly different definitions and a built in conflict between their focuses. Organizations are often tempted to combine these two and this is often the case in smaller IT departments where resources are scarce. The firefighting inherent to Incident Management, and the need to get the customers up and running as quickly as possible often does not leave time to perform Problem Management and examine issues for root causes. On the other hand, time spent on more in-depth investigation of problems takes away from the quick resolution of incidents that customers want and need. Even where resources are not scarce, the same individual or group should not manage both processes unless the conflicts are acknowledged and time can be set aside to work exclusively on one process versus another. This is very difficult to balance and is not recommended.
No shared understanding of what a problem is: ITIL provides the definition of a problem as the unknown, underlying cause of multiple incidents or a single significant incident. Yet in some organizations this has either not been accepted or it is not internalized and understood by staff. Handing out guidebooks is not enough! Everyone in the Support Center should be speaking the same language and the distinction between an incident and a problem must be made clear to all. The processes surrounding them must also be clearly defined, documented and understood.
Lack of recognition of conflicting goals of Incident and Problem Management: it makes sense that a reduction in problems should result in a reduction of incidents and an increase in the Support Center's first level resolution rate at first. As the Support Center becomes more proactive, it will start solving problems before incidents can occur. Eliminating and preventing problems will then decrease the number of calls received and reduce the number of `easy' tickets. This in turn makes the overall closure rates at first level decrease. This is one of the inherent conflicts between Problem and Incident Management that sometimes puts the processes at cross-purposes. The targets for first level resolution should be adjusted and measured accordingly.
Problem Management as a discipline: Problem Management should be adopted as more than just another mechanistic process. Common techniques must exist and be understood (e.g. how to perform root cause analysis in your organization). Managers must look for and reward the desired behaviors and react accordingly to unwanted outcomes with training and coaching where necessary. For Problem Management to become a discipline, it must become part of the culture. Team members should be clear about their responsibilities, know how to accomplish them and know what to expect if they do not.
Problem Management without a distinct process owner: Problem Management often fails at this level because either no one group or no one individual has been assigned ownership of the overall process. Without central responsibility for monitoring all existing problems, their status, business impact and resolutions, the speed and quality with which problems are resolved is extremely inconsistent. Some problems may be forgotten and never solved at all.
Problem Management seen as bureaucracy: when staff members see Problem Management as additional layers of regulations that keep them from getting the job done, they will resist it. They may also see Problem Management as a process designed to apportion blame. It is imperative that everyone understands the process is not about finding fault; it is about finding the answer to a problem and resolving it so that it does not occur again.
No Problem Management: often Incident Management is implemented without its complementary support process of Problem Management. While Incident Management is not enough, it is preferable to the customer than waiting until a permanent solution is found. But if Incident Management staff starts to focus on resolving problems instead of getting the customer up and running, customer satisfaction will suffer. Ultimately the customer wants both; to get back to work (Incident Management) and to never experience the problem again (Problem Management).
Problem Management versus the `hero complex': incidents that are easy to resolve with quick fixes or workarounds provide constant positive rewards for a technician. By staying reactive, they can provide quick action on these issues, allowing them to `rescue' customers and get them back to work. When staff members are rewarded for fixing rather than preventing, they have no incentive to proactively seek the elimination of recurring incidents. Establishing Problem Management can help break this cycle by rewarding people for preventing incidents.
![[To top of Page]](../images/up.gif)
7.2 - Implementation
7.2.1 The implementation process
Begin by assigning functional and process ownership to a designated Problem Manager with the reputation, respect and resources to get the job done. It is most effective when the same person(s) does not handle Incident and Problem Management. Instead these responsibilities should be clearly delineated to avoid conflicts with competing objectives.
Separation of these functions provides balance of effort to both processes. While the Incident Management team, or Support Center, focuses on getting the user back to work (either through a temporary fix or a workaround procedure) the Problem Management team begins to work on a permanent solution. As the Single Point of Contact (SPOC) for the IT department, however, the Support Center and its Manager cannot abdicate responsibility for participating in the implementation of Problem Management in the organization. Incident Management will often be the first to realize the need for a Problem Management process and begin the drive for its implementation within the organization.
7.2.2 Support Center Manager's role
Responsibilities and activities
Because Problem Management cannot succeed without a firmly established Incident Management process, the Support Center Manager's key focus needs to be on the development and optimization of Incident Management. During implementation of Problem Management, the Support Center Manager should be working closely with the designated Problem Manager or Management team to design how the processes will work together. Notification, escalation and feedback loops should be of primary concern.
In a smaller organization, where responsibility for the Incident and Problem Management processes resides with the Support Center Manager, concentration should be on defining how much time can be spent exclusively on each. A lot of thought needs to be given to how to keep proactive Problem Management activities from being neglected during periods where there are high volumes of incidents or whether the organization should implement proactive activities at all. When responsibility resides within the same group, a lead should be assigned to each process to ensure coverage.
In an organization attempting to implement both components of Problem Management at the same time, or where resources are limited, Problem Management may have to be limited initially to reactive problem and error control. After enough historical data has been accumulated to be useful, the focus can increase to include proactive activities: root cause analysis, trend identification, problem prevention, instigation of RFCs, and providing management information. In any case, it is important to review time and resources periodically for both reactive and proactive processes to ensure balance.
Deliverables
- Notification Process Flowcharts
- Escalation Process Flowcharts
- Feedback Process Flowcharts
- Population of Knowledgebase: Process and Procedure
Competencies
- People skills
- Business knowledge
- Technical knowledge
- Management skills (including measurement, metrics, reporting and analysis)
- Conflict resolution skills
- Root cause analysis skills
- Facility with Problem Management techniques, such as Root Cause Analysis
- Sound knowledge of the Incident Management process and Support Center function
- Understanding of Knowledge Management and learning organization concepts (especially as they pertain to problems, known errors, and workarounds)
Key performance indicators (KPIs)
- Percentage increase in first level resolution
- Percentage increase in knowledgebase entries
- Percentage decrease in escalations to second and third level support
7.2.3 Support Center Function's role
Responsibilities and activities
- Reviewing resolved incidents and identifying potential problems from incident records
- Associating incidents with existing problem records or creating new problem records if necessary
- Ensuring problems are dealt with according to agreed timescales and priorities
- Ensuring that accurate and complete information is entered in all problem records
- Ensuring that problems are assigned to the appropriate groups for investigation
- Ensuring that resolutions and workarounds to problems are verified
- Deciding whether Known Error records should be created and whether, if a permanent resolution has been found but not yet implemented, investigation work should continue in order to produce a workaround
- Assessing Known Errors to determine whether the permanent resolutions could or should be applied
- Closing problem records and Known Error records when appropriate and informing Incident Management so that resolutions or workarounds can be applied to associated incidents
Problem Management should be separated from Incident Management if at all possible. The Support Center staff should work with their manager to enable the Problem Management team to get their process started. As the SPOC to the customer, the Support Center is in a position to offer a great deal of input regarding the problems experienced and how they affect the business customers. They are also more familiar than anyone else in the department with the data accumulated within the trouble ticket application because they receive and record all calls and deal directly with users on a regular basis.
In a small IT department, where Incident and Problem Management cannot be separated, a Support Center Manager might identify separate individuals in the Support Center to act as leads for the reactive and proactive aspects of Problem Management as a `specialty'. Alternatively, staff could perform multi-week rotations on some periodic basis to focus on Problem Management, analyze data and initiate RFCs. It is important to periodically review the time and resources spent on each to ensure that enough attention is being given to both. It is especially important to pay attention to the proactive activities, as these quickly may be ignored when incident levels are high.
7.2.4 Planning for implementation
Steps to take
Establish a formal Incident Management system.
There should be some level of Incident Management already in place such as formal problem categorizations, descriptions, prioritizations, impact assessments, duration estimations and resolution processes. It is best practice, and wise planning, to segregate Problem Management responsibility from Incident Management. An individual or group must be made accountable and responsible for the Problem Management process in your organization. This need not be a full time role.
Use Incident Management data and information to drive the Problem Management system
.
Assuming that the Incident Management process is well enough established to capture reliable and compliant data, begin to focus on using that data within Problem Management. If an effective process is not yet in place, the ticket tracking system is unlikely to be fully utilized to record it. Adjust existing methods or create new ways to capture the information needed to perform good, detailed analysis.
Define a process for gathering issues to review for further analysis.
The Problem Manager, or Problem Management team, will define a process for gathering issues from many sources (including incident records, faulty changes, discussions with Service Level Management regarding services and customer perceptions, client satisfaction surveys and industry benchmarking) to review for further analysis. They should plan two-way communication paths so that everyone in the department knows and understands the process.
Provide sufficient resources to staff the Problem Management system.
The Problem Manager does not need to have the technical experience to resolve the problem hinself/herself. However, there should be someone in the team with the technical expertise to help guide Problem Management in your organization. They will be able to help pinpoint possible causes and suggest potential solutions and the correct people to implement them. If you cannot have a group dedicated to Problem Management, the Problem Manager should have access to the technical resources necessary to assist them in the analysis and resolution.
Track the problems from identification to resolution.
Determine the method to employ for tracking problems. Choose or develop tools that will allow you to record problems through the entire life cycle, including its resolution. If your ticket tracking tool supports mapping incidents to a `master ticket' or `case' (or better still to a problem ticket) consider using this functionality to track problems and to associate related incident tickets to problem tickets (either manually or automatically).
Groups to contact
- Customers - conduct focus groups, workshops or surveys to determine the areas of most concern to the customer base.
- Incident Management - the front line support staff is the best source for reporting potential problems to Problem Management. Processes for ongoing communication need to be established between the two groups. This starts with mapping incidents within the ticket tracking tool to problems, if your tool supports that functionality. At a minimum, the Support Center Manager must analyze tickets and produce reports to share with the Problem Manager as a basis for isolating potential or active problems based on ticket trends.
- Service Level Management - understanding the core business and what is most important to your customers will have already been identified and documented in the Service Level Agreements established by this group. This is an invaluable source of information to Problem Management in identifying priorities in problem resolution.
- Change Management - once the resolution to a problem is identified, an RFC may be required to implement the solution. Consult with Change Management to establish processes for submitting the RFCs, and a feedback loop for monitoring the results of the proposed resolution once implemented. Some ticket tracking tools have integrated change management systems. If yours does not, the ticket-tracking tool should be interfaced with your Change Management system. For example, in break/fix situations, when it is not a case of swapping like for like, an incident ticket should be generated and linked to a change ticket; this ensures a complete picture of `what changed?' on the Change Management side.
- Configuration Management - when the root cause of a problem has been identified and a workaround has been developed, the problem is then defined as a Known Error. Known error data should be stored in the Configuration Management database (CMDB), so a process needs to be developed for providing this information. (Note that the CMDB is a logical database that could be in multiple physical locations, so a `bug tracking' knowledgebase containing problem, known error, and workaround information that stands separate from the incident system is within best practice). Not all Known Errors will be pursued to a complete resolution so other ways need to be found for tracking that information.
Necessary information and data
- Ticket tracking - including solid descriptions of the issue and work done (and by whom) to assist in researching problems, trend analysis and root cause analysis. Good documentation in the trouble tickets cannot be stressed enough.
- Change tracking - data should include the source of the RFC, when the change was implemented and whether the problem was resolved.
- Configuration Database - tracking of the Known Errors in a knowledgebase that is a logical part of the CMDB is done here, in addition to all the hardware, software, and license information which can be invaluable in researching a problem.
Measurements that should be in place
- Number of tickets per week/month/quarter
- Percentage First Call Resolution
- Top Ten Issues/Incidents Reported
- Number of tickets due to failed Release Management
- Number of Requests for Change due to failed Release Management
- Number of repeat incidents/problems
- Average problem diagnosis and resolve time
- Number backlogged of `open' problems and errors
- Number of proactive changes raised by Problem Management, particularly from major incident and problem reviews
7.2.5 Implementing key process activities: hints and tips What to implement first
Incident Management: you will need access to reliable and compliant data for root cause analysis and trend analysis. Robust Incident Management will ensure a good starting point for you to collect, review and analyze this information.
Things that always work
The Pareto Principle : if you examine the top 80% of your tickets, you will probably find that they cover about 20% of the top issues in your department. You may want to investigate and consider purchasing a software application that can automatically apply this principle to your data for you.
Little things that deliver big returns
If the tickets have been entered and categorized correctly you will be able to look for tickets recorded on similar issues. A good ticket tracking system will allow you to `relate' your tickets to make this even easier.
Little things that always get forgotten
Your customer: do not forget to verify the priority and business impact to your customer. It makes no sense to spend resources (time, people or money) working on problems that are not important to your customer.
7.2.6 Key process activities
Implement Incident Management: if the Incident Management process is not in place in your organization this will need to be implemented before implementing Problem Management or, less preferably, in tandem. Problem Management depends on Incident Management to identify issues that may be problems, and also on the data collected through Incident Management to perform root cause analysis.
- Identify resources: determine whether you will be able to segregate Incident and Problem Management. Identify who will have overall responsibility for the Problem Management process. Establish a multifunctional support system for Problem Management, especially the process for obtaining technical resources when it becomes necessary.
- Define roles and responsibilities: everyone involved with Problem Management needs to be clear about their roles and responsibilities. There should be no doubt on how to interact with one another, or how Problem Management interfaces with the other ITIL processes.
Meet with appropriate groups: in the implementation of Problem Management you will need to meet with the groups previously outlined. The most important of these are Incident Management and your customers. The relationship with Incident Management will be ongoing, and communication paths should be established as early as possible. While information obtained from Service Level Management may provide insight and input into the customers' perspective, you should not limit Problem Management by not connecting with your customers directly.
- Review the Incident Management Tool: verify that the tool is capturing all the data that you need. Determine how you will be able to obtain data from the tool and how you will be able to produce reports.
- Develop a method to track problems through resolution: if your ticket tracking application can be adapted, plan and design the changes that need to be made. Simple spreadsheets can be used for tracking temporarily, or may work adequately for your purposes. A sample is included in Annex A7.6.
- Establish a notification process: it is best practice for potential problems to be passed to Problem Management through only one source, Incident Management. This provides consistency in categorization and understanding of the business impacts. A process for raising issues to Incident Management needs to be developed as well as how Incident Management will notify Problem Management.
- Establish a submission/identification process: pursuing problems or Known Errors may or may not be required. Assess each to determine whether the time, cost or impact is justified. Is it more important to investigate a large number of insignificant issues or a smaller number of issues that have a high business impact?
- Establish the prioritization process: define criteria for assessing a problem's priority such as business impact, business urgency and/or technical severity. Those with the highest priority should be resolved first.
- Establish a feedback process: the status of problems and Known Errors should be continually communicated to Incident Management, which is responsible for following up with the affected customers. Determine the method and frequency for providing this feedback.
- Establish an escalation process: establish a process for escalating those problems that will be tracked through resolution to the appropriate technical resources.
- Define a method for performing root cause analysis: it is estimated that there are over 40 investigation methods. Select a method that makes sense for your organization and adopt it as a standard process. Provide training if necessary so that everyone can understand and follow the logic of the chosen method. A training guide is provided in Annex A7.7.
Define a method for processing Known Errors: once a problem has been identified and a fix has been determined, two primary processes need to happen. The fix needs to be communicated to Incident Management and the Known Error needs to be reported to Configuration Management to update the CMDB.
- Define Resolution Process: Proposed permanent resolutions for problems or known errors will work their way through the Change Management and Release Management processes. Problem Management is responsible for tracking them through these phases until they are resolved. Your plan should define Problem Management's involvement in the other processes and how information will be passed between them. Additionally, the resolution process should include a plan for communicating the status to Incident and Configuration Management. Results of the attempted solutions should be monitored to verify that the problem has been permanently resolved.
7.2.7 Methods and techniques
Root cause analysis: the root cause of a problem can be found by examining all the data you can possibly gather surrounding an issue. You might start by mining the data in your ticket tracking system. Examine the data from multiple perspectives:
- who are the most frequent callers to your Support Center?
- which departments have the most problems?
- what are the most common issues being reported?
Popular techniques cited in ITIL that can be used in Problem Management include:
- Kepner and Tregoe Root Cause Analysis - for determining the root cause of a problem while it is active
- Ishikawa (cause-effect) diagrams and brainstorming - also useful for tracing root cause while a problem is active
- Service Outage Analysis - for determining, during post-implementation review of an event, factors contributing to the outage so proactive measures can be taken to prevent such failures in the future
- Failure Mode Effects Analysis and Fault Tree Analysis - for proactively identifying the causes of and downstream effects of failure to take measures to pre-empt such failures.
The most important thing about techniques is to ensure they are shared within your organization.
7.3 - Ongoing Operations
7.3.1 The implementation process
Problem Management can be described in its most fundamental way as call avoidance. While Problem Management has a reactive component, it can and should be more focused on proactive activities. Removing root causes, preventing the occurrence (or recurrence) of Incidents and minimizing the consequences of those that cannot be prevented are the key activities involved. When your systems are stable, and your service processes are well defined and working as designed, incidents and problems become easier to manage.
The scope of this process is to count recurring tickets, analyze trends and identify problems. Incident Management can exist without Problem Management and it is far easier to manage each Incident as it arises than to invest the time necessary to perform root cause analysis. But by itself, Incident Management is ineffective. Many problems can have the same root cause but display different symptoms. This can make it difficult to recognize a problem or even to realize that a series of incidents may be related. Similarly, the organization could be experiencing a number of problems that share common symptoms but are not related at all.
7.3.2 Support Center Manager's rote
Responsibilities and activities
The Support Center Manager should work very closely with the Problem Manager to ensure a continued feedback loop between Incident and Problem Management. The Support Center owns the Incident Management process, and functions as the SPOC, interfacing with customers. With the implementation of Problem Management, the Support Center will continue to be the main source of potential issues to be examined on the reactive side of Problem Management and instrumental in defining business impact. The Support Center will also act as the SPOC for Problem Management to receive requests to review incidents as potential problems. In this way, there can be consistency of definitions and interpretations.
On the proactive side, Problem Management will be investigating trends and root cause on data provided in trouble tickets by the Incident Management staff. There should be continuous dialog regarding the collection and categorization of this data and the Support Center Manager is ultimately responsible for the quality of this data. Ideally, Problem Management could automate parameter triggers to flag a specific set of conditions as needing Root Cause analysis.
As problems are categorized as Known Errors, Problem Management should be providing the workarounds back to the Service Manager and Incident Management team. For permanent solutions, it is expected that the Support Center Manager is actively participating in the Change Management process and therefore aware of the resolution before it is implemented and fully aware of Release Management's implementation schedule. It is the Support Center Manager's responsibility to keep the Support Center aware of information about the changes coming out of the Release and Change Management processes so that they will be informed and prepared when these changes occur.
In the case of a small center where the Support Center Manager is overseeing both Incident and Problem Management, it is important that the appropriate staffing is identified and kept informed as if they were separate groups. It is a good idea to have a proactive problem specialist on this team to participate with the Support Center Manager in the Change Advisory Board so neither process gets overlooked.
Deliverables
- Management reports, and associated analysis, required for the management of the Problem Management process
- Communication to other processes and functional managers, as well as to suppliers, users and customers
- A `top 10' list of problems, including what has been done so far to deal with them, and what will be done next, is highly recommended
7.3.3 Support Center Function's role
Responsibilities and activities
Participating in the Problem Management process, staff should be seeking feedback regarding the potential problems they have reported. Either it is not a problem, or there should be some kind of workaround provided. The Support Center by nature is extremely reactive and will not necessarily have time to constantly ask for this information. However if these reports are not being provided, it is their responsibility to report that to their Manager to follow up. The Manager should ensure that the Support Center obtains the information that it needs to provide effective service and provide status updates to the customers. Other responsibilities would include analyzing data, investigating problems, submitting RFCs, monitoring Known Errors and monitoring the status of problem resolutions.
In a small center, the proactive Problem lead would be responsible for providing this information to the Manager and the rest of their team. This lead person should also attend Change Advisory Board meetings and work with the Support Center Manager to understand and communicate the impacts from the Problem Management perspective.
In either case, producing a daily report of the top ten most common issues each day is an effective way to provide a minimum level of ongoing reactive Problem Management. Workarounds or permanent solutions should be added to the knowledgebase and Support Center staff should be allocating the time needed to maintain this database regularly. The top ten lists can also be used proactively to monitor what the problems are, track what has been done to resolve them and plan what will be done next.
Deliverables
- Daily Top Ten Report of Problems
KPIs
- Percentage reduction in resolution time
- Percentage reduction in time to implement workarounds
- Percentage reduction of problem diagnosis time
- Percentage reduction of average number of undiagnosed problems
- Percentage reduction in repeat incidents/problems
- Percentage reduction in average time to diagnose and resolve problems
- Percentage reduction of the time to implement fixes to Known Errors
- Percentage reduction of the average backlog of `open' problems and errors
- Percentage increase of proactive RFCs raised by Problem Management, particularly from major incident and problem reviews
7.4 - Optimization
7.4.1 - The optimization process
The more control over incidents an IT department has, the better availability and reliability it can provide to its customers. As more and more problems are eliminated from the environment, the Support Center can begin to focus on improvements in efficiency; isolating application errors, developing training plans and providing better service. With proper communication and management reporting, the benefits will be clearly visible and IT will be better positioned with the core business.
7.4.2 Support Center Manager's role Responsibilities and activities
Incident reduction leads to Incident prevention. To optimize the process, the Support Center Manager can now begin focusing on improving efficiencies internally. Working with the Service Level Manager, efforts to redesign Operational Level Agreements (OLAs), the internal `service level agreements' between the Support Center and the second and third level support teams, can yield better turnaround time for ticket resolution. Review of historical data may reveal areas within the customer SLAs that should be revised as well. Developing training plans that involve the Support Center staff's interaction with the other teams provide value at the Support Center and open new career path opportunities. Successful companies train their staff to anticipate customer needs and to solve Problems before the customer even knows they exist.
Deliverables
- Updated OLA (internal)
- Updated SLA (external)
- Processes
- Training plans
KPIs
- Increase in proactive changes implemented as a result of Problem Management
- Number of service disruptions caused by incidents or problems
- Percentage of reduction in recurring incidents or problems
- Percentage of reduction in Known Errors
- Decrease in First Level Resolution rate
- Decrease in problems as result of failed Change/Release Management
7.4.3 Support Center Function's role Responsibilities and activities
Good Problem Management will lower the number of tickets that are easily resolved. As problems and some of the routine incidents disappear, Support Center staff can begin to focus on the proactive goals of the Support Center. This includes more time to spend on difficult issues and in follow-up with customers. A Support Center that used to be too busy handling password resets can now find out why some tickets are still unresolved at second level. They can identify areas where training would be appropriate in giving them the skills they need to resolve more of the complex issues.
7.4.4 Future impact of this process on the Support Center
|
Proactive problem solving guarantees that your problems will stay fixed. The Support Center will be able to demonstrate savings to the organization and embark on a continuing program of process and service improvement. Above all, the purpose of implementing Problem Management is to improve service and provide quality support to your customers.
|
7.5 Measurement, costing and management reporting
7.5.1 Implementing Benefits and costs
Benefits and Costs
| Most of your customers are not concerned about why a problem occurred, nor do they want to spend time on resolving it. They just want their computer system to work! |
The cost of problems is proportional to the volume of incidents and number of customers that are affected by them. Duration of an incident and the resources needed to resolve them also accumulate costs to the business due to lost productivity. Implementation of Problem Management will help offset these costs, reducing the Incidents that interfere with the ability of the organization to conduct its business. Ultimately, this will provide better system availability and reliability and a better overall customer experience.
Cost elements for implementation
Typically there are no costs for tools required to implement Problem Management because the tools should already be in place for other processes (i.e. ticket tracking for Incident Management). If you are not staffed to handle Problem Management, you may consider acquiring additional staff. In most cases, the roles and responsibilities for this process are distributed among existing employees.
Making the business case to implement Problem Management
If you have previously made the business case for the implementation of ITIL in your organization, Problem Management will already make sense. Discuss the drawbacks of remaining reactive as an IT organization. You should be able to articulate how recurring tickets affect productivity. A key point to make to customers is that a problem resolved through Problem Management stays resolved. You will be able to promise better availability and reliability of systems and improved service quality. But be careful to set your customers' expectations so that they expect continuous ongoing improvement rather than immediate results.
Management reporting
Top Ten reporting of incidents, broken out by individual and by department give a useful perspective of the issues facing the organization and allows the IT department to demonstrate its commitment to eliminating problems.
Management reports should explain how metrics and key performance indicators relate to the strategic objectives of the organization. For instance, if a strategic goal of IT is to provide systems and tools that maximize staff productivity, make sure that management reports show actual data and also compare that data to organizational goals. For example: in support of the goal to maximize productivity, the overall system availability goal is set at 98.5% during 24 hours of operation. The management reports should show the availability of individual systems and the overall average. Any major deviations from the goal should be explained along with a statement about what needs to be or is being done to address the cause(s). If the deviation was a one-time situation and is not expected to recur, your report should include a statement to that effect.
7.5.2 Ongoing operations
Cost elements for ongoing operations
To move beyond reactive Problem Management, it may be necessary to look at staffing levels. If feasible and justifiable in your organization, consider acquiring additional staff. If you have been managing Incident and Problem Management through one individual or group, now may be the time to consider breaking these processes out. Promoting one of your specialists to focus on Problem Management may be in order at this time.
If you do not have good reporting tools in place, it may also be the time to consider purchasing some. These products vary in price and functionality. Look for one that will interface easily with your ticket tracking system.
Metrics and Key Performance Indicators
- Number of RFCs generated through Problem Management
- Amount of time spent on problems (broken out by problem type)
- Comparison of number of tickets received for a specific problem before and after root cause and/or known error is established
- Comparison of reactive vs. proactive problem responses
- Problem's impact on availability
Management reporting
Because so much time is spent investigating incidents, finding workarounds, diagnosing root causes, and in the identification and implementation of problem resolutions, reports to management should reflect the time and effort spent on each. To give that information context, this should be measured against the impact of the problem to see the value in the resource expenditure.
7.5.3 Optimization
Benefits and costs
Ineffectiveness has a high cost for the organization in downtime and in employee turnover. If employees do not have adequate tools, they cannot work effectively (loss of productivity = cost to organization) and they may become dissatisfied (potential for increase in turnover).
Metrics and Key Performance Indicators
- Increase in proactive changes implemented as a result of Problem Management
- Decrease in number of service disruptions caused by incidents or problems
- Percentage reduction in recurring incidents or problems
- Percentage reduction in known errors
- Decrease in First Level Resolution rate
Management reporting
Problem Management reports should be geared towards monitoring the quality of the process.
7.5.4 Tools Implementation
- Ticket tracking
- Change tracking
Ongoing operations
- Configuration Management Database (CMDB)
- Knowledgebase
- `Canned' knowledge for common problems
- Report writing tool that interfaces with your ticket tracking application
- Word processing
- Spreadsheet with statistical 'plug-in'
Optimizing
- Automating aspects of the Problem Management process (event correlation to auto-problem ticketing, etc.)
- Driving shared techniques down to the organization (e.g. Kepner-Tregoe problem analysis) and continuously improving the use of these techniques
- Implementing problem management tools to support techniques (software that supports Failure Mode Effects Analysis or group brainstorming)
- Implementing continuous improvement of Problem Management though review and audit
- Launching improvement programs to improve the alignment of Problem Management with the business and its drivers
Annex Documents
Overview
Annex A7.1 - Problem Management Implementation Checklist
- Implement Incident Management
- Identify resources
- Define roles and responsibilities
- Meet with appropriate groups
- Identify and acquire tools
- Develop method to track problems through resolution
- Establish a notification process
- Define problem submission/identification process
- Establish a feedback process
- Establish an escalation process
- Develop a problem classification scheme
- Get training on and adopt shared problem management techniques, e.g., Kepner and Tregoe problem analysis, Ishikawa (fishbone) diagrams, brainstorming and flowcharting techniques, trend analysis
- Adopt a method for performing root cause analysis
- Develop a method for processing known errors
- Define a resolution process
- Establish a shared practice for problem post-implementation reviews
- Determine problem management metrics
- Establish content, target audience, and frequency of reports, review and audit procedures
- Define integration of problem management with incident management, change management, configuration management, availability and capacity management
- Define interface between problem management and application and infrastructure capability development, especially with regard to the transfer of bug tracking information to problem management during production acceptance
Annex A7.2 - Problem Management Ongoing Process Checklist
- Identify problem
- Record problem
- Determine root cause
- Record known errors in CMDB
- Evaluate justification for resolution
- Pass resolutions to Change Management
- Pass known error, problem, and workaround information to incident management
- Conduct post-implementation reviews where appropriate
Annex A7.3 - Problem Management Optimization Checklist
- Work with Service Level Management to redesign SLAs and OLAs
- Work with Availability and Capacity Management on availability and performance issues
- Identify process issues surrounding turnaround time
- Identify potential training issues
- Identify potential technology issues
- Adjust measurements and reports
- Continuous process improvement
- Continuous service improvement
Annex A7.4 - Sample Problem Management Process Workflow
Annex A7.5 - Problem Management Evaluation Checklist
Questions to ask in determining justification for problem resolution.
- What is the impact of the incident(s) reported?
- What services arc affected?
- What customers are affected?
- Is it a "problem"?
- Is there a quick fix?
- How will work be done?
- Who will do the work?
- What is the priority for resolving it?
- Does it require more technical expertise to resolve?
- How long will it take?
- How much will it cost to implement a fix?
- How much will the fix cost on an ongoing basis?
- How does the cost of the fix compare to the cost of doing nothing?
- Is the workaround an adequate solution?
Annex A7.6 - Sample Problem Tracking Spreadsheet
