why do we need to measure skills better better indicators for better policies

Why do we need to measure skills better? Better indicators for better policies!

By Fabio Manca.

While there is a lot of discussion around the need to strengthen skills, reduce skills mismatch and promote the creation of the right skill set to boost economic growth, defining what ‘skills’ truly are remains an extremely difficult task. Translating the whole idea of ‘skills’ into an univocal indicator is even more arduous and it entails several empirical and theoretical challenges, many of which are still unresolved today.

The term ‘skills’ encompasses a wide range of attributes. It can refer to both generic skills and job/occupation/sector specific skills. Generic skills may include cognitive skills such as information-processing skills (e.g. numeracy, literacy and problem-solving) as well as non-cognitive skills (such as perseverance, self-organisation, presentation, team-work and other such soft skills). Job-specific skills, by contrast, are attributes that are not generally transferrable from one job/occupation/sector to another and refer, for example, to firm-specific knowledge about the functioning and culture of the organisation, technical knowledge, or practical competencies that are specific to a particular sector or occupation (e.g. manipulation of light pulses in the context of quantum optics, hairdressing, etc.).

Being able to directly measure all the above aspects would be extremely useful but economists and analysts usually face severe data limitations (e.g. small sample size, data comparability, measurement error etc.) and are, in many instances, forced to use second-best proxies to describe skills and build indicators.

OECD “Getting Skills Right” (forthcoming) shows, for instance, that qualification titles/levels or field of studies are the most common variables used to monitor the skills of national workforce across a countries. While these variables have the advantage of being readily available from National Statistical Offices, they do not necessarily map to the true skills required on the job and, similarly, to variables that can be used to design meaningful skills policy (Quintini (2011) shows, for instance, the existence of substantial variability in the skills and readiness to perform a job amongst individuals with the same education credentials).

Reducing the ‘noise’ and the error in the way we measure skills is imperative and it can help to robustly pin down the effects of education and employment policies on a wide range of economic and social dimensions (i.e. economic growth, inequality but even happiness or job-quality).

Apart from the much needed accuracy in measuring skills, another aspect is fundamental in the choice of skills proxies but sometimes overlooked: skills indicators (and analyses) are only useful as long as they promote the development of initiatives that can tangibly help economies and individuals reach higher levels of prosperity through robust evidence-based policies.

The implementation aspects related to the translation of skills information into effective policy are, therefore, fundamental here. Skills indicators need to be designed to effectively inform policy decisions and, for that purpose, they need to be appropriate to the concept being measured (i.e. years or education, skill proficiency or field of study may all be appropriate depending on what is being studied) and understood by policy makers in the first place (i.e. they need to have the adequate level of aggregation/complexity for policy makers to use them to design concrete policies). Unfortunately, most of the existing national and regional exercises that monitor skills within (and across) countries are not always useful for policy making.

Recent evidence from a OECD survey to Ministries and Social Partners (OECD “Getting Skills Right”, forthcoming) shows that these stakeholders struggle when trying to decipher the available information on skills (Table 1) and, even more importantly, when trying to use it to develop skills policies.

Table 1: Barriers limiting the translation of skills assessment and anticipation information to effective policies

Blog20.1Notes: Ministries of Labour / Education take into account the 26 responses from either ministry reporting at least one barrier (Austria (2), Belgium (3), Canada, Chile (2), Denmark, Estonia, Finland, France, Germany (2), Hungary, Ireland, Korea, the Netherlands, Norway (2), Poland, Portugal, Slovenia, Switzerland, Turkey and the United States).

Source: OECD Questionnaire on Anticipating and Responding to Changing Skill Needs: Ministry of Labour, Ministry of Education.

While in some cases the output of skills monitoring is too technical (pointing, among other things, to the urgent need of modernising Ministries’ analytical capabilities!), in many other cases results are either not sufficiently disaggregated for policy purposes (e.g. results and proxies are too broad and vague) or skills proxies just do not map to useful variables that can shape the design of concrete policies.

Strengthening the quality of skills proxies while, at the same time, designing them in such a way that they can be concretely used for policy action is a fundamental challenge for all countries.

Among the many policy uses that can be given to skills information, active labour market policies and skills-matching activities can greatly benefit from the development of more refined and ‘policy-friendly’ skills proxies.

Some countries have already well-developed statistical infrastructures built around large skills taxonomies. These large databases aim to categorise and measure occupations’ skills requirements to help job-seekers and Public Employment Services to match job-seekers skills to those required by the labour market in a more effective manner.

The O*NET database (https://www.onetonline.org/) sponsored by the  U.S. Department of Labor, Employment and Training Administration (USDOL/ETA) provides, for instance, hundreds of standardized and occupation-specific descriptors with detailed information on knowledge, skills, abilities, interests work activities and context of more than 800 occupations. This database, available to the public at no cost, is continually updated by surveying a broad range of workers from each occupation and asking them about the actual skills needed to carry out their daily job-tasks rather than focusing only on their formal qualifications or titles (the collected skills information informs the O*NET Career Exploration Tool with the objective to provide information to workers and students looking to find or change careers as well as to Public Employment Service (PES) caseworkers in charge of the implementation of active labour market policies and of matching the skills of the workforce to the available jobs).

A similar exercise is run by the Swedish PES through the development of a ‘Digital Matching Tool’. The Swedish Digital Matching Tool goes way beyond the use of old-fashioned occupations, fields of study or qualifications proxies as it allows job-seekers and employers to search for each other through a system of skill-tags inputted directly by the final users. An IT engineer, for instance, can promote her/his CV (listed in the PES’ website) by adding skills like C++, Javascript, HTML or PHP. Employers, in turn, can look for those specific competences and skills instead of being constrained to search through (sometimes meaningless) job titles or qualifications, these latter unable to capture the depth and specificity of the skills required for the job or owned by the worker.

The exploitation of Big Data such as real-time vacancies linked to skills-tags and other relevant skills and labour market information is still in its infancy but it bears the promise of being extremely helpful in addressing skills mismatch and in providing high-quality and flexible information to policy makers. This will contribute to find smart and effective solutions to ensure that the available (true) skills, in many cases underutilised, are effectively put at use at work. The OECD is at the frontline in the production of skills indicators (WISE dataset) as well as in the interpretation and analysis of the impact of skills on various economic and social dimensions. Better indicators are going to help deliver better policies.