Measuring the Impact of Development

Are you getting the best results (and support) for your training? Measuring can enhance your success

Overview

Measurement should take place at three levels – individual, team and business. What is measured at each of these three levels will be different, and like any measurement system, there is a trade-off between quality and cost.

Individual level

The most important aspect to measure at this level is the individual’s skills level. The best way to do this is through an individualised competence development plan that describes the core competencies required in any job and enables measurement of the individual against this. The measurement process combines the individual’s perceptions as well as their managers and, if appropriate, those of peers and subordinates. Quarterly assessments can be undertaken to measure development. Alternatively, the system could be used to measure the ‘before’ and ‘after’ skills levels relative to a specific course of learning. Individual development can easily be converted into percentage improvements.

Team level

As a collection of individuals, the suggested measurement process here has to do with team-based project work undertaken in the context of a specific learning intervention. For example, measuring a team’s development is difficult unless the team is participating on the same initiative or course. Where there is a requirement for a team-based project, measurement takes the form of an assessment of the financial and non-monetary impact of the project undertaken. The project sponsor who is external to the team would be tasked with quantifying the benefits of the improvement project upon completion. If done well, this provides ROI justifications for the development spend.

Business level

The further away from the business, the more tenuous the links. Retention of key staff is influenced by a myriad of factors, however exit interviews and other information will indicate whether training is a help or hindrance. In our view, the best approach is to take training spend as a proportion of wage bill and link this with the development of individual competences. An organization may be investing an amount equivalent to 2% of its wage bill on training and development and achieving an average of 30% competence improvement per annum. The challenge for the HRD team may then be to reach 40% competence development with the same spend.

According to many senior executives, people are viewed as ‘the organisation’s greatest asset’. When, then, are they are not valued in the balance sheet? Part of the answer lies in the prudence of accounting conventions, wary of valuing intangible assets (such as people or brands) that are prone to significant variations in strength and weakness, and may not maintain their value. This difficulty is compounded by other challenges associated with valuing intangible assets, such as the absence of a single ratio or measure that is in widespread use.

In our experience, two closely related performance measurement techniques are particularly valuable. They are Beneficiary-owned measurement (BOM) and Random Sample Measurement.

Beneficiary-owned measurement (BOM)

This idea is based on the fact that measurement is best carried out by those who have the most to gain from the changes. Each piece of development is always preceded with the creation of a measurement alliance. They buy into the need for change in the individual, team and business and become supporting sponsors.

For example:

- Ideally, what would you (team member) like to see change in team A?

- In reality, where are you/they now?

- What does this mean for you?

- So, will you sponsor the change with a monthly development report?

If the report shows change, then investment continues. If it doesn’t, we stop. The objectives are to move away from the task requirement of measurement towards outcomes and fulfilment. If there is sufficient senior-level commitment, BOM delegates substantial power to grow and develop the program to the ‘measurement alliance’.

Random Sample Measurement

This is best used where some catch up measurement is required before BOM-based systems are put in place. It is ideally used where significant learning and development interventions are already in place. The tools used include:

- Objectives questionnaires.

- Observed change questionnaires.

- BOM interviews.

- Statistical analysis.

- Extrapolation.

This technique is based on a diverse range of samples and the need to measure short-term cost/benefit and long-term return on investment.

In addition, several prominent studies have highlighted how HR measures can be applied to improve organisational performance.

Common problems with performance measurement techniques

Senior executives in general and HR professionals in particular are faced with two vital issues underpinning the success and performance of their organisations. First, what is the relationship between learning and development activities, and financial performance? Second, how should measures of performance be applied and integrated into the organisation?

Executives typically encounter one or more of problems with performance measurement systems.

• Too many measures obscuring the most significant issues and diverting attention from other issues, such as understanding cause and effect between business variables.
• Measures are disconnected, unrelated to the firm’s strategy and business priorities.
• Results are overly emphasised without necessarily providing an adequate explanation of how they were achieved.
• Reward systems are not aligned with performance measures; consequently, the desired behaviours are not encouraged.
• Measurement systems are divisive, failing to support team-based working and collaboration.
• Short-termism is encouraged as measurement leads to an intense focus on improving the next quarter’s results.

Given the sums invested in human capital activities – notably training and development – and the clear link, proven in a range of studies, between investments in employees and organisational effectiveness, the need for systems to measure successful performance is clearly vital. According to General Electric’s former CEO, Jack Welch: "The three most important things you need to measure in a business are customer satisfaction, employee satisfaction and cash flow". Although Welch later modified the last measurement item from cash flow to shareholder value, the importance of the other two – and their connection – remains strong.

Performance measurement techniques

1. Watson Wyatt’s Human Capital Index

Several studies have linked investments in people with performance. These include Watson and Wyatt, whose 2002 Employee Trust Levels Survey (involving 13,000 employees) found that the rate of three year total returns is three times higher at companies with high trust levels than at companies with low trust. Their survey highlighted the impact of people management practices, with five issues (highlighted in the Human Capital Index) directly affecting profits:

- Total rewards and accountability.

- Collegial, flexible working place.

- Recruiting and retention excellence.

- Communications integrity.

- Focused HR service technologies.

2. B&Q’s Employee Engagement Programme

One approach to measuring the link between investments in people and the firm’s resulting performance is provided by B&Q’s Employee Engagement Programme. This puts employee engagement and customer loyalty at the top of the agenda.

With the B&Q approach, every manager has a regular, one-page report summarising their performance in two areas: managing human capital and managing traditional finance measures. The results of the B&Q programme highlight the business benefits of employee engagement.

As a result of B&Q’s Employee Engagement Programme, labour turnover reduced from 35% to 28% (each percentage point of attrition cost the company at least £1m) and profits increased, with turnover per employee rising from £87,000 in 1998 to £106,000 in 2002. The role of the finance team is central to the success of the process: designing, funding and managing the performance of the programme. Other features of the programme underpinning B&Q’s success include close liaison between HR and retail operations, objective measurements directly focusing actions on enhancing performance, and staff commitment to the programme.

3. University of Bath Model

Research by John Purcell from the University of Bath highlights the link between employee investments and performance. (For further details see The HRM-Performance Link: Why, How and When does People Management Impact on Organisational Performance? 2004 John Lovett Memorial Lecture, University of Limerick, delivered by John Purcell, School of Management, University of Bath.) This study emphasises the significance of Ability, Motivation and Opportunity to participate (AMO) in delivering desired outcomes and performance improvements.

Three elements are essential for AMO to succeed:

- Top-level commitment.

- Active support for front-line managers (for example, with training and development).

- Recognition of the importance of discretionary behaviour.

For example, the importance of people routinely doing what is needed, without being told, to improve effectiveness and achieve their organisation’s aims.

4. The Service Profit Chain and Sears

The starting point for the Service Profit Chain is the view that, traditionally, managers focus on results. These are historical, giving information about what has been achieved in the past, whereas what is really required for market leadership is an emphasis on managing value drivers. Of these value drivers, employee retention, employee satisfaction and employee productivity determine customer satisfaction, revenue growth and profitability.

Market leadership is about actively managing value drivers. These are more difficult to define, but they directly influence your future.

Often, difficulties arise when different departments take responsibility separately for each aspect of the chain. So, for example, HR focus intently on employee retention, satisfaction and productivity, largely in isolation from marketing professionals who are focusing on issues of customer satisfaction and loyalty. Neither HR nor marketing are then linked closely enough with the finance team, who themselves focus almost exclusively on ‘their’ part of the chain – revenue and profit. The SPC highlights how closely each issue drives performance. It is essential, therefore, that senior managers look across the whole business, effectively sharing information and communicating.

Sears provides one of the best examples of employee practices connecting directly with organisational performance. Senior executives at the US-based retailer realised in the early-1990s that future performance was not going to be enhanced simply by developing a different strategy or adjusting their marketing plans. The firm was running significant losses – what was required was a thorough understanding of three issues:

- How employees felt about working at Sears.

- How employee behaviour affected customers’ shopping experience.

- How customers’ shopping experience affected profits.

On a single day, Sears asked 10% of their workforce – 30,000 employees – how much profit they thought the firm made for each dollar sold. The average answer was 46 cents, whereas in reality the answer was 1 cent. This highlighted the need for employees, and especially those at the front line, to better understand the issues determining profitability and success. Sears’ approach was to develop the employee-customer-profit model (ECPM), making explicit the chain of cause and effect. Because employees were better able to see the implications of their actions it changed the way they thought and acted – and this, in turn, was reflected in bottom line performance.

The Sears approach to creating an employee-customer-profit model (which is a specific version of the service profit chain) started by devising a set of measures based on objectives in three areas: making Sears a compelling place to work, a compelling place to shop and a compelling place to invest.

For the top 200 managers at Sears, incentives are based on Total Performance Indicators (TPI) – which include non-financial as well as financial measures.

- 1/3 on employee measures – attitude about the job and company.

- 1/3 on customer measures – customer impression and retention.

- 1/3 on financial measures – return on assets, operating margin and revenue growth.

As a result of the employee-customer-profit chain, managers at Sears are recruited, promoted and appraised on the basis of 12 criteria: customer service orientation, initiative and sense of urgency, business knowledge and literacy, problem solving, developing associates and valuing their ideas, teamworking skills, two way communication skills, valuing diversity, empowerment skills, interpersonal skills, change leadership and integrity. These are grouped into three areas – the three P’s: passion for the customer, performance leadership and people adding value.

The Sears experience highlights the need for intuition and common sense to be blended with sound statistical analysis. This results in a way of working specific to each firm’s situation.

5. Measuring intellectual capital: Skandia

The Swedish financial services firm Skandia provides an interesting example of a firm that may be showing the way to the future, by regularly valuing and reporting on their intellectual capital – the knowledge and expertise within the organisation.

One of the first people to quantify and value intellectual capital was Leif Edvinsson. Appointed in 1991 as the world’s first Director of Intellectual Capital at Skandia, Sweden’s largest financial services corporation, Edvinsson divides intellectual capital into three types. Human capital, which is in the heads of employees; structural capital which remains in the organisation, and customer capital, deriving from the relationships the company enjoys with its customers. Customer capital is often seen as a sub-set of structural capital.

The aim of Skandia’s measures is to track whether intellectual capital is increasing or decreasing, focusing the organisation’s culture and thinking on increasing its intangible assets. In Edvinsson’s view: "Intellectual capital is a combination of human capital – the brains, skills, insights and potential of those in an organisation – and structural capital – things like the processes wrapped up in customers, processes, databases, brands and systems. It is the ability to transform knowledge and intangible assets into wealth-creating resources, by multiplying human capital with structural capital. This is the intellectual capital multiplier effect."

At Skandia, human capital is further divided into several elements: customer focus, process focus and renewal and development focus. Edvinsson has designed a process for each business unit to report on all of these areas of intellectual capital. The importance placed on Edvinsson’s work was highlighted by the inclusion in Skandia’s annual report of the value of its intangible intellectual capital assets, which was estimated at more than $15 billion. However, for Edvinsson the real benefit has been even greater: managing intellectual capital has nurtured innovation and new thinking, and has helped create a mind-set enabling Skandia to compete in the future.

The rise of knowledge and intellectual capital suggests that to be successful, organisations will need to focus on the reconfiguration of existing systems (including the organisation culture) to support knowledge workers. Also significant is the creation of a learning organisation – one that is constantly sensing, valuing and sharing information, using it in a flexible way to improve efficiency, generate profitable new ideas and, overall, add value for customers. Finally, those organisations that value intellectual capital invest in training and coaching employees at all levels, and they free managers to manage people. The result is improved productivity.

Integrating HR measures

Using scorecards

Scorecards are widely used to benchmark performance in three non-financial areas, augmenting financial measures. These are customer relationships, its key internal perspectives and its learning and growth.

When performance measures for these areas are combined with financial metrics, they provide a broader perspective on the company’s strengths and activities. They also offer a powerful method of organising and co-ordinating a company’s operations, ensuring that all activities are aligned with the strategy.

However, for scorecards to work, it is essential that there is buy-in at all levels, with the value of the process clearly recognised. This is because scorecards need to influence behaviour throughout the organisation. Furthermore, good scorecards highlight the links between cause and effect, they sit easily with the budgeting process and they are not used as a basis for incentives, where they may be viewed too narrowly or negatively, undermining their effectiveness. As if these hurdles were not enough, people perceive a problem with the term ‘Balanced Scorecard’, as it fails to inspire or communicate the practical, dynamic nature of what is involved.

Several organisations have developed their own scorecards, including Swedish financial services corporation Skandia (figure 5). Other measures emphasised include operational excellence, with businesses such as Tesco consistently delivering to their customers. Product leadership is another measure displayed by innovative firms such as Dyson who stipulate that ‘x% of revenue must come from products developed within the last x years’, and so is customer intimacy – a combination of customer loyalty and value.

The HR Scorecard

With traditional scorecards, cost-based measures fail to take account of the value of people, focusing instead on issues such as cost per hire, staff turnover, error rates and complaints. Scorecards can also be seen as static, bureaucratic and unrelated to the needs of the business, failing to engage people. HR metrics need to be workable, reflecting business realities, rather than being too internally-focused or disconnected from business priorities. The result is a new approach, the HR scorecard, developed by Mark Huselid, Brian Becker and David Ulrich.

The HR Scorecard is based on an emerging view of the role of HR. According to business academic and writer Jeffrey Pfeffer, the role of HR is threefold:

- Maintaining the organisation’s culture and values against short-term pressures.

- Reminding managers of the studies highlighting the importance of people management as a source of competitive advantage.

- Ensuring that values such as concern for staff are implemented.

Pfeffer’s view results from a concern that performance targets – often resulting from short-term financial pressures – are driving HR’s work. His counter to this is that HR should instead take the lead in driving the attainment of business targets, not simply follow where the rest of the business leads. Emphasising the link between people and performance is central to HR’s task of leading the progress towards business objectives.

Conclusion: measuring the relationship between people and profit

Assessing the correlation between individual actions and attitudes and profitability is inherently complex, but understanding, measuring and managing this link is an essential role for HR professionals. In general, are six stages involved.

1. Develop causal models based on the firm’s strategic plan.
2. Assemble and organise the data.
3. Turn data into information, using correlation techniques.
4. Continually assess, update and refine the model.
5. Base actions on findings.
6. Assess the outcomes.

It is clear that discretionary behaviour affects employee engagement; this, in turn, affects customers and profits. Relevant issues to consider include:

• What is affecting employee engagement – what are the key drivers?
• Is your understanding of employee engagement reflected in your training and development budget? Are you investing the right amounts in the right places? Are you measuring the results on employee engagement of your activities?
• Are you getting value for money?
• Do people across the business understand the current situation, their priorities and how this affects their actions?

Key points

Including ‘people’ measures in an overall corporate scorecard raises the profile of human capital and ensures management focus. Examples, such as B&Q’s Employee Engagement Programme, reveal the connection between strong people practices, increased customer satisfaction and financial results.

Choosing the right HR measures means finding the link between motivating staff and achieving vital business outcomes – including issues as diverse as product innovation, safety and customer satisfaction.

There are opportunities for HR and marketing professionals to share customer data. HR has data about what motivates and engages internal customers, whilst marketing has insights about external customers’ needs and wants. Together, they provide a 360-degree view of customer behaviour.

Understanding the elements of workforce success – what makes employees productive, effective and willing to use their initiative to achieve the organisation’s goals is an imperative for HR professionals. Awareness of these issues and how they can be applied provides HR with another essential role in ensuring the firm’s commercial success.

Finally, HR measures should not be over-complicated or excessively elaborate, seeking scientific validity, when what is needed is a practical, robust and informed approach.