3.1 The importance of the assessment stage
The first part of the Keynet Project, outlined in Chapter 2 above, addressed the three issues of the nature of the skills, the ways in which they can be delivered in a range of contexts, and the extent to which it is possible to isolate examples of 'good practice'. The next part of the Project focused upon 'assessment' and tried to answer four questions:

who is the assessment guidance for? what sort of information do they require? what are the principles of assessment most appropriate to key skills? is there a 'most appropriate' method of assessing key skills?

 

These questions were posed and answered in an 'Assessment Guide' that targeted teachers, trainers and employers. In summary, the following points were made.

Who is the 'Guide' for?
It was pointed out that the 'Assessment Guide' had been produced in order to help two groups of people develop and deliver more effective teaching and training programmes in which the key skills of communication, application of number, IT, problem solving, working with others and improving own learning and performance play a significant part. These two groups are

Teachers and trainers; Trainers and employers.

The fact that the two groups are not mutually exclusive is highlighted by the presence of the 'trainers' in both categories. These two groups require accurate information about their students, trainees or employees. But there is a third group in the equation - the student, trainee, employee or the unemployed individual - and the situation or context in which they find themselves. They require equally important information about their achievement, strengths and weaknesses, in key skills. This information is usually expressed in the short hand term - a 'qualification'.

What sort of information do they require?
Teachers and trainers need to know the strengths and weaknesses of their students or trainees, especially as a means of planning the next stage of a course or programme, and to be able to 'plug the gaps' in a student's knowledge, understanding and skills. These are largely diagnostic and formative assessment.

Trainers are a little more specific in their demands of assessment. They, too, require information of a general nature, but are also often dealing with trainees who have job-specific needs, e.g. their employer is particularly interested in their developing IT skills or the ability to work with others. In such cases, the assessments must be firmly rooted in specific contexts. They also have to have systems in place that feed back to a wide range of employers.

These two groups of users are especially concerned that the assessments throw up results and information that tell the student what her/his strengths and weaknesses are.

Employers also want to now if the claimed skills really have been achieved - and to have the evidence that this is true. So, the employer is interested in summative assessment supported by evidence.

The student, trainee, employee and the unemployed want to be certain that any programme or training course allows her/him to achieve the stated objectives and to have reliable evidence that it has been achieved. In other words, that a qualification is valid. They also need to know what they require to progress and, for example, to be eligible for promotion. Also important are those who are not currently working but who want to return to work or enter employment for the first time. In many cases they may have been outside the main stream of education or training for many years. This group will be especially keen to discover information about their current level of attainment, their strengths and weaknesses - that is, diagnostic assessment.

So, bearing in mind this very wide range of interest, are there any general principles of assessment that can be applied to educational and training programmes containing key skills?

ISome basic assessment principles? It was emphasised that a possible list of principles that should underpin any key skills assessment programme would include those issues shown below:

Determine at the outset what information is required It is important that only the required information should be provided. So, careful attention needs to be paid to the form of the assessment. If the employer requires information about an employee's ability to use transferable key skills under pressure then the assessment must reflect this, e.g. a timed assessment. If on the other hand the ability to work co-operatively is the requirement, then group-based assessments must be developed and used. Initial assessment should result in each student/trainee receiving an individual action plan. Assessment must be a valid experience for the student/trainee/employee As far as possible, assessments must combine contextualised assessments (e.g. the task based upon a course or work activity with which the student/trainee is familiar) and those that provide the opportunity for the assessment of the transferable skills elements of key skills (e.g. encountering a problem in a novel situation). The assessment should provide information that is reliable The methods of assessment used by the teacher or trainer must produce results and information that can be depended upon by all involved in the process. That means that no matter who the person being assessed, or on when the assessment occurs, or by whom assessed, the result is the same. The results of the assessment should mean something to the users This has two meanings. The results should convey information that is readily understandable to the user, but they should also stand as a symbol for acceptability. Clearly, the UK government hopes to achieve both these aims through its new free-standing Key Skills Qualification. · Certification should be manageable and cost effective It is no use developing an assessment programme that is too complicated for people to understand and too costly to use. Accordingly, the assessment techniques used should be capable of being used by trainers and understood by trainee and employer - and be cost effective.

The 'Assessment Guide' looked in detail at the important first stage of assessment - that which should take place at the start of a training programme, i.e. diagnostic assessment.

Assessment - a method of working?
Most of the 'Guide' contains examples of different types of assessment. Sometimes a technique is used in one context and then, on another occasion, in another. One of the main purposes of the 'Guide' is to show how assessment may be used to give the trainee, the unemployed, the trainer and employer information about the individual's achievement, weaknesses and progress in the acquisition and the ability to use key skills. It is up to assessors to decide which assessment strategy and technique is most appropriate to their needs. The diagram below suggests a possible approach to and method of working in a key skills assessment programme.

A starting point This is the teaching or training programme. Decide how the 'key skills' are to be delivered: · by special key skills programmes, e.g. an IT course, an Application of Number or Communication course. Through a student/trainee's main courses or programme. Assessment strategy This largely depends upon the approach adopted to training: If special key skills programmes are used then assessment can focus on one or more key skill, e.g. on assessing communication. If the approach to training is based on 'embedding' the key skills within the main training programmes then each assessment programme must consider the issue of the 'context' of the assessment, e.g. a test designed to assess application of number skills is presented within an exercise on, for example, health and safety, or welding techniques. Assessment techniques The type of tasks used largely depends on your assessment strategy. For example, key skills focused programmes allow the use of techniques that target aspects of a key skill directly. Contextualised key skills require the use of techniques that assess the key skills indirectly.

 

3.2 Evaluation of the 'Assessment Guide'?
In many respects, the 'Assessment Guide' is a central document of the project since it works at two levels. It links the teaching and learning process to certifcation and accreditation (the 'public' face of competence), and learner with trainer and employer. As a result, the 'Guide' was itself assessed and evaluated by a range of users. This was done through two questionnaires, each targeting something different but producing results that were complementary

The first Questionnaire targeted a range of trainers and employers in the North Wales TEP and asked general questions about key skills as well as the 'Assessment Guide'. ·
The second Questionnaire was more specific and was circulated to a more limited audience. Almost all the questions focused on the 'Assessment Guide'

Questionnaire 1: Key skills as a strategy for combating exclusion from the labour market: the views of employers and training providers
The results of the Questionnaire analysis were as follows:

VIEWS OF EMPLOYERS
Question	Response
How important are 'key skills' in the following: (a) Your initial decision to employ someone (b) the further development of their level of key skills competence	Very important (100% ) Important (100%). All respondents commented that they aim to improve their employees' skills, knowledge and competence in all areas of their 'job roles'.
Which of the key skills are the most important/least important in your organisation? Please rank them 1-6, with 1 being the most important	1. Communication (93%) 2. Working With Others (93%) 3. Problem Solving (84%) 4. Improving Own Learning (85%) 5. Application of Number (80%) 6. Information Technology (78%)
How do you determine/assess the key skills of new entrants/employees?	1. Individual diagnostic tests/tasks. (90%) 2. Group interviews and discussions.(85%) 3. 1-to-1 interviews. (74%) 4. 'Having-a-go' at the job. (69%)
To what extent are employees encouraged to improve their skills? (a) Formally (by attending courses or training) (b) Informally, through their own initiative	95% said 'Yes'.Most provision (80%) was by 'in-house' training. Methods are mainly briefings and workshops. A number of employers (mainly the largest) use 'key colleague development 'strategies'. 90% supplement this in-house training with external, largely college-based training. 76% said 'Yes', but it was 'encouraged' rather than 'expected'. In 60% of work places, the individual employee found their own external key skills courses, mainly in IT.
What skills/qualities do you look for in the more mature entrant/employee (40+)?	Exactly the same as other employees.
What kind of guidance on key skills do you think would be useful to your organisation?	Support on raising the key skills levels of all employees (75%). At the lowest cost (70%). In a manageable form (68%). No jargon 63%)
Any other comments, including response to the 'Assessment Guide'	A number of discrete comments were made, but that which attracted the greatest number was a general criticism of the lack of 'preparedness' of entrants straight from school or college. This was particularly evident in the lack of a positive attitude to work. One employer summed it up with the statement - 'Teaching new skills to colleagues is easier than changing or improving attitudes.' There was a general welcome for the 'Assessment Guide'. All the respondents stated that their 'in-house trainers' would find it useful. There was particular praise for the exemplars. 65% were pleased with examples from other members of the EU.

 

VIEWS OF TRAINING PROVIDERS
Question	Response
Key skills programmes provided	FTFE and TEC-funded courses 16-25 year-olds as part of NT framework GNVQ and GCE Key Skills Awards BTEC/HNC/HND Common Skills Full time, part time and day release programmes
How important are 'key skills' in the following: (a) the 'employability' of young work seekers? (b) the further development of their job-specific and general competence?	Very important (100%). Reasons given include the following: The evidence in 'Future Skills Wales'.· Develops the flexibility required in modern industry and commerce. Important but, the standards of the new Key Skills Qualification need to be established and accepted by employers. Vitally important (100%). Most respondents commented that, even though they aim to develop their trainees' skills, knowledge and competence in all areas of their 'job roles', they are especially concerned to improve their attitude to learning and improvement.
Which of the key skills are the most important/least important in your organisation? Please rank them 1-6, with 1 being the most important	1. Communication (100%) 2. Working With Others (85%) 3. Information Technology (84%) 4. Application of Number (80%) 5. Problem Solving (63%) 6. Improving Own Learning (57%)
How do you determine/assess the key skills of new entrants/employees?	1. Individual diagnostic tests/tasks. (90%) 2. Basic Skills Assessment Pack (80%) 3. College-devised assessments (75%) 4. A screening process (64%) 5. Prior qualifications and discussion (50%)
To what extent do you think most employers in your area encourage their employees to improve their key skills? (a) Formally (by attending courses or training)   (b) Informally, through their own initiative	Uniformally negative responses. The training depends largely on the size of the firm. Most employers, including some of the larger ones, are reluctant to release employees. As a result, colleges and training organisations mainly see MAs and NTs, and even then attempts are made to block release, because of 'pressure of work'. Again, a generally negative response, with virtually all the providers pointing out that it is difficult to quantify. It was suggested that most 'on-the-job' training involving key skills would rarely be called as such. The specific work place context would dominate. A small number of providers claim that the informal, work place approach to key skills development cannot and does not work. Again, the comment was made by a significant number of providers that employers are keen to improve their employee's key skills but are not so keen to release them from work.
What skills/qualities do you think they look for in the more mature entrant/employee (40+)? How do they help them acquire these skills?	Again the providers state that it is difficult to make a judgement on this. However, they suggest that employers expect them to have 'acquired' Communication, Problem Solving and Teamwork Skills through 'experience'. There was a general feeling that many employers make an effort to provide IT training for this group of entrants/ employees.
What kind of guidance on key skills do you think would be useful to your organisation and employers?	For Training Providers A range of exemplar materials (93%) 'Off-the-shelf' learning/resource packs with appropriate guidance (86%) Help with portfolio building (76%) The benefits are to the student/trainee (60%) Models of 'off-the-job' delivery strategies(57%) More Welsh Language materials (13%) For employers The benefits of key skills (to them and employees) (90%) Importance of progression in key skills (76%) Exemplar materials (66%)
Any other comments	A general plea that due acknowledgement must be made to a paradox that exists. While most employers value the competencies identified under 'key skills' they are reluctant to give time or money for staff to acquire them. It is suggested that this is largely because they believe that as they are 'generic' most employees should already possess them. As a result, they do not believe they should be doing the work of the education and training system! It is claimed that another difficulty centres on the student/trainee/ employee. They often think that because they have GCSEs and GCEs in Mathematics, English and IT they do not need to develop their key skills. The 'Assessment Guide' was welcomed by all respondents. Training providers were especially pleased to see the range of guidance provided, and use of 'well chosen and relevant exemplars'.

In summary the following points may be made:

1. The Questionnaire was circulated as part of the preparatory work for the production of the Assessment Guide. It was sent to a number of organisations within the North Wales TEP, including 30 employers in a range of occupations from small to large, and 20 training providers. Completed Questionnaires were received from half of the employers and two-thirds of the training providers. The attached collations summarise the responses of both groups. In addition to the information requested, these responses provide some useful further information that could be of use in the overall KeyNet programme.

2. The following observations may be made about the responses - other colleagues may draw other or different conclusions:

Key skills are held to be important both by employers and training providers and, by extension, to employees and those seeking employment.

However, there is a clear and sometimes worrying difference between the attitudes of employers and trainers to the provision of key skills training. A majority of the training providers emphasise the fact that many employers are still unconvinced about the value, to them, of key skills.

The ranking of the six key skills highlights a number of interesting similarities and differences between the two groups. Communication and Working With Others are clearly seen as the most important skills. Interestingly, Application of Number and IT occupy different positions in the ranking depending upon their relative importance and status in the occupation/firm, or in the eyes of the respondent.

There is real demand on the part of employers and providers for the provision of relevant and cost-effective training and assessment resources and guidance materials. There is a significant demand for exemplars in diagnostic assessment and support materials in IT, especially some that would target the older entrant or employee. However, there is a strong plea from employers that materials should be jargon-free.

Implicit in the returns from employers and training providers is the need to address the issue of changing the attitudes of the different groups, including the trainee, employee or the unemployed, to the importance of key skills.

 

Questionnaire 2: evaluation of the 'Assessment Guide'

General

1. The second Questionnaire was more specific and targeted training providers from both within the North Wales TEP and from other parts of Wales. Even though approaches were made to the TEP policy makers and managers few positive responses were forthcoming, the most common being that we should approach those organisations and individuals within the TEP who deal with matters relating to training and assessment. It is these responses that are summarised below; those received from areas outside the TEP were used a 'control' group. There was a very high correlation between the two sets of responses.

2. The Questionnaire was sent to 20 individuals or individual representatives of training providers, 14 to people within the TEP and 6 outside. 11 responses were received, 6 from the North Wales TEP and 5 from other parts of Wales.

3. Those responding were currently involved in training apprentices, Modern Apprenticeships, National Traineeships, individual 're-employment programmes, and the assessment of students as part of the UK Key Skills Qualification.

4. The Questionnaire consisted of a limited number of open-ended questions. They were:

In what context do you 'come into contact with key skills?
Give your general opinion of the 'Assessment Guide'.
List its three main strengths.
Give any suggestions for improvement.

The table that follows provides a summary of the responses received from all the correspondents.

Questions	Summary of responses
1. In what context do you come into contact with key skills?	Modern Apprenticeships National Traineeships Teaching Re-employment training Assessing/moderating the key Skills Qualification
2. Give your general opinion of the 'Guide'.	A well presented document· Good range of exemplar material Useful description and explanation of 'why' we are assessing Simply and clearly presented
3. List its three main strengths.	The range of assessment techniques - as shown in the Annexes The analytical approach to the assessment of individuals during programmes of education and training The examples of tasks and practical activities trainers/assessors need to provide for the effective appraisal of earlier training programmes
4. Suggestions for improvement.	Clearer and more guidance on the relationships between trainee competence and 'level' (1-5) of achievement Information/guidance could be provided on 'basic skills' and their links or calibration with key skills. At the start of a course or training programme, many trainees function at the lower levels of achievement Provide more guidance on the nature of competence

 

The general conclusions that may be drawn from these two sets of Questionnaire are:

there is clearly a need for more guidance on both the delivery and assessment of key skills. This guidance should be designed for all 'players' in the process, i.e. trainees, trainers and employers.

the 'Assessment Guide' clearly meets the needs of those targeted by the Questionnaires.

once amended, as required, the 'Guide' should be made available in as many modes as possible (electronic as well as printed format) in order to reach the greatest possible audience.

 

back | top | next