Outsourcing is increasingly under review by many publishers for all aspects of their business, including ad operations. If you do decide to go with an outsourcer, one of the major challenges is designing and implementing the service level agreement, or SLA. The SLA is meant to define success metrics, help govern the relationship in terms of what is expected, and also provide guidelines for unacceptable performance. Evaluating your SLA helps safeguard your process and contractual relationship but it also provides an opportunity for you to plan your management of the process and have fruitful discussions with your potential outsourcing partners.
Broadly speaking measurements for a SLA should be in line with the important outcomes of your process. You want to measure outcomes which you consider important success measures related to your overall goals. For example, if you are outsourcing creative testing, you would probably not want to measure success by the number of tests done on a creative, but whether a creative was accurately tested overall.
To set a baseline of an acceptable error rate, it is helpful if you have some stats of how your on-site team performs on key tasks before you outsource. This can help evaluate reasonable levels of errors for your outsourcing partner. If you do not have records on your team performance that match SLA goals, you’ll need to set a baseline in another way with your outsource partner. For example, you can set levels which are considered acceptable to run the business with an agreement to review after a set period of time.
It is important that your measurements can be objectively tracked – quality is often subjective. Measurements should be tracked in a reasonably easy way. Be careful of the assumption that an outsourcer’s workflow solution can track all key measurements. If not managed properly, tracking SLA stats can take a lot of time and as the goal is to reduce work for the on-site team, its important to have the outsourcer pick up as much responsibility for this as possible. For some measurements (such as management time) you may not want to track it all the time, but only do so if it seems like there are problems in that area.
Measurements to consider:
Turn around time (TAT) – This is something that the outsourced team should most likely track otherwise it will be a resource drain on your in-house staff.
Errors – Errors for the process you are outsourcing are probably pretty straightforward. If someone is testing for errors you might want to also consider what type of errors. Are these routine or general errors such as typos or are they ‘process errors’ which demonstrate that the overall process isn’t understood or being followed? You may want to allow a certain number of permissible ‘general errors’ such as 3 per week and another permissible amount for ‘process errors’ such as 1 per month. The in-house team will most likely need to determine what is an error and there should be a high level understanding that the in-house team (as opposed to the outsourced team) will have the final say in what is to be considered an error.
Time spent – Outsourcing can come with hidden costs in terms of management and training time that team members or management become involved with. You may want to set an acceptable level of team or management time needed to manage the processes on a weekly or monthly basis and track over time.
Amount of Communication – Another key measurement might be the time or number of communications needed between your team and the outsourced team. Errors and training time can be hidden in the outsourced team requesting clarification from the on-site team over and over again. If the number of clarifications on a particular error is being measured, it will help make sure the outsourced team is attentive to writing their questions clearly and cut down on unnecessary or repeat communication and hidden training.
Learning over time – You may want to build in the expectation that errors reduce over time due to learning by setting different standards for 6 months, or a year down the road.
Auditing a SLA is in the eye of the beholder. If the outsourcer is tracking errors, they cannot necessarily catch all of the errors that your team would catch, and you can run into the ‘fox guarding the hen house’ problem. For that reason, you will most likely want to track some SLA measurements and not rely fully on the outsourcer to ‘grade themselves’. Tracking needs to be planned and managed carefully so as not to create more work for your in-house team. You probably will not be able to check all of the work, and if the relationship is working well, you shouldn’t need to. Generally, you will want to pick an acceptable percent of the work to double-check and have your team track errors. This percentage may change over time. For example in the beginning you may track 80% of the work and then gradually move down to 5%. In planning for the outsourcing engagement make sure you have the resources on your team to track and monitor what is needed to keep the engagement on course. Because of the need for tracking and management of the outsourced work, it is not necessarily the case that an extra head at the outsourcer means you can reduce headcount.
In case the outsourcing arrangement doesn’t work out as planned, it’s useful to know in advance what will happen if the outsourcer does not meet the SLA. It is typical in the industry to build in fees the outsourcer will pay if they do not meet SLA requirements for a certain period of time, but fees may not protect the business in all cases. There should also be a mediating framework and procedure for reviewing whether a process should be outsourced. This is hard to consider at the beginning of an engagement but having a vague idea of such a situation is useful to resolve problems as you confront them en route. I recommend continuing to evaluate the wins and challenges the same way you evaluate a new employee. If after a year, an outsourcer is making persistent mistakes which you would not accept from an employee, you should have a performance review.
In conclusion, if you’ve been tasked with evaluating a service level agreement, make sure you know how outsourcing will impact your process, how you will measure the successes and failures of your partner and at a fundamental level that the SLA helps accomplish what an outsource partner should do – outsource some of the work your department is responsible for in a reliable way.