Academic preparedness of students – an exploratory study

The high level of student failure, accompanied by an increased drop-out rate, is problematic in higher education. It is especially a concern in programmes with the subjects of Mathematics, Accounting and Science. Over many years, models of student admission and selection have been widely researched both internationally and in South Africa. Research indicates that in the academic domain, underpreparedness results from a combination of a lack of English proficiency, mathematical ability and effective study skills. In view of the above, and government policy directives to broaden access in the scarce skills areas to increase student throughput, foundation provision was introduced for students of Commerce, Information Technology, Business, Mathematics and Informatics courses at the Vaal Triangle Campus (VTC) of North-West University (NWU) in 2010. The question at that time then arose as to what criteria should be used for placing students in the extended programme. The placement of first-year students in appropriate programmes should be done with sensitivity to enhance academic success but, at the same time, should not ‘label’ students as underprepared. This paper provides perspectives on the selection criteria available for predicting academic success/preparedness, and then reports on students’ own experiences. An action research study was conducted on the academic achievement of two cohorts of first-year students at the VTC of NWU. The quantitative results of the performance of first-year students in their core modules are compared to the results of predictive tests written after admission. The results provide valuable insight into the placement of students.


Introduction
The student success rate in the public higher education (HE) system illustrated the dysfunctionality of the education system. Success rate is used in the context of course rather than programme performance, and refers to the percentage of passes in relation to the total course registrations. Only seven out of a total of twenty-three institutions met the success rate norm of 80% in HE (Malele, 2011). Less than one-third of the students completed a qualification within the required minimum time. In view of this and government policy directives to broaden access in the scarce skills areas and to increase student throughput, 1 .
foundation provision was introduced for students of Commerce, Information Technology, Business, Mathematics and Informatics programmes at the Vaal Triangle Campus (VTC) of North-West University (NWU) in 2010.
The South African Higher Education Act No. 101 of 1997 places responsibility and accountability for the selection of students for higher education at institutional level. It also states that admission policies must provide for appropriate measures for redress of past inequalities and may not unfairly discriminate in any way (Department of Education, 1997). Concerns were raised after the 2008 and 2009 National Senior Certificate (NSC) mathematics results showed that an increased number of students across the country achieved more than 80% in the final Mathematics examination. It was found that the NSC Mathematics examination did not discriminate sufficiently between candidates, particularly among high achievers (Department of Education, 2009).
Admission is the process through which students enter tertiary education. Systems vary widely from country to country and from institution to institution. Different criteria for different programmes are set. With Mathematics being a prerequisite for many university programmes, the question that arose was what the best process and testing would be to channel the students into the appropriate programme (normal versus extended).
Public concern has been expressed about NSC results, but little documented research could be found of cohort studies conducted to measure the accuracy of NSC results as a predictor of students' performance at universities (Nel & Kistner, 2009;Ramphele, 2009).
The National Benchmark Tests (NBT) were commissioned by Higher Education South Africa (HESA) and reflect almost ten years of research and collaboration among leading content specialists and higher education researchers. The NBTs, launched in 2009, were intended to provide a service to HE institutions by providing them with information additional and complementary to school-leaving results that would assist them in the admission and placement of students in appropriate curricular routes. The assessment was designed to measure a writer's levels of proficiency in Academic Literacy, Quantitative Literacy and Mathematics as related to the demands of tertiary study. Mathekga (2010) indicated that national data were needed on the usefulness of the NBT as a placement and academic development instrument.
The questions that framed the research were the following: • Is the NBT a better indicator of academic preparedness than NSC results?
• What can institutions do to support underprepared students in Mathematics when entering the HE system?
The outcome of this project will be used to revise admission criteria as well as to indicate which tests are recommended for future first-year intakes.

Access to higher education
The fields of Information Technology, the Sciences and Accounting are characterised by the need for structured thinking, and mathematical and problem-solving skills during both the time of study and practice. Mathematics is therefore a prerequisite for enrolment in these courses. As also noted by researchers, learners are very good at remembering and repeating material from lecture notes and text books, but are unable to solve problems which they have not previously seen or discussed. This may be the result of the fact that the teaching arena is still dominated by the pedagogy-based old teaching methodologies, which are mostly characterised by the deductive teaching approach (Naudé & Westhuizen, 1996;Miller, 1998;Sheppard et al., 2009).

Exclusionary practices
The challenge of grappling with barriers that exclude learners from higher education seems to be a worldwide phenomenon (Gayle, Verridge & Davies, 2002). Johnson (2007) argued that current practices disguise the ongoing effective exclusion of some groups from higher education, even where such systems are publicly committed to equitable access for all people.

Socio-economic circumstances
In virtually every country of the world, participation in higher education -rates of entry and completion, as well as type and prestige of institution attended -is closely associated with the socio-economic status of the individuals (Gladieux & Swail, 2000). Efforts to raise the quality of education of poor children have largely failed. Apart from a small minority of black children attending former white schools, and a small minority of schools performing well in largely black areas, the quality of public education remains poor. Literacy and numeracy test scores are low by African and global standards (NPC, 2010). Low literacy levels among parents, poor nutrition, violence and social fragmentation are factors that explain why the performance of school children from poor communities remains low relative to that of their wealthier peers.
Poor school system Hall (2001) acknowledges that the South African Department of Education is correct in attributing social exclusion from higher education to poor schooling [in-school factor] and the lack of financial aid [out-of-school factor] in the face of poverty. Research evidence highlights the significance of factors or problems within the education system itself. These include the ongoing changes and amendments to curricula, the type of teacher training, inadequate support to teachers, teaching time compared to other activities and the availability of learning and teaching materials such as text books. Several other complex issues play a role in the quality of education. Curriculum design; language issues; the use of technology; the efficacy of the bureaucracy; the balance of power between parents, schools and the bureaucracy; and high levels of violence against women and children are all relevant factors (Bitzer, 2005;Boinamo, 2007). Without dismissing any of these factors, the conclusion of the National Planning Commission (NPC, 2010), led by Minister Trevor Manual, was that the main problems lie in teacher performance and the quality of school leadership.
The results of a study conducted by Nel and Kistner (2009) showed that grade inflation of the NSC results occurred particularly in the results of the lower performance group. According to a 1995 paper entitled 'Indicators of Grade Inflation,' presented by Robert Mullen at the 35th Annual Forum of the Association for Institutional Research, grade inflation is defined as "...when a grade is viewed as being less rigorous than it ought to be" (Mullen, 1995). Generally, grade inflation can be described as a practice whereby the actual, real value of an A is deflated, so that it becomes an average grade among learners. Foxcroft (2006) confirms that grade inflation ('the upward creep') has possibly been the cause of the increase in high-risk students in higher education. Grade inflation could also be explained in terms of curriculum stagnation or changes in curriculum, a lack of experience in assessment and the manipulation of the memorandums according to which the matriculation examination papers are marked (Yeld, 2005).

Language
Another challenge is the language that most admission tests are administered in. The predominant trend in admissions testing in South Africa is to administer tests in the language of instruction, which in many cases is English. In the general field of testing and assessment in South Africa there is currently much debate around the language used for conducting the assessment. It is questionable whether the administering of tests in the common language used in the business world or the language of instruction as opposed to the language in which the test taker is most proficient, constitutes fair and ethical assessment practices (Foxcroft, 2004). An Academic Literacy test has a very strong language component and there is a very real danger that the level of proficiency of the test taker in the language in which the test is administered could contaminate the test results. Furthermore, there is strong evidence of qualitative differences in the reading and processing skills used by first-and second-language English speakers (Enright et al., 2000).
Any measurement of academic competency will be influenced by the above factors. It is therefore predictable how learners from a certain school are likely to perform, simply because of the school's reputation. Hoover (2008) states that admission tests anticipate a future when these tests only reflect what students learned in high school. The authors agree that tests for academic preparedness tend to reinforce poor schooling while giving insufficient information about the potential to succeed in an environment where effective support, active learning, well-designed study material and qualified and dedicated lecturers are provided.
In this paper the authors provide a successful sample case in which we describe the cohort study that we used. We also provide a comparative analysis of students' performance between the NSC, the NBT and their performance in their core first-year modules. Lastly we show how the intervention during orientation benefited the students.

National benchmark test (NBT)
The NBT intends to provide a service to higher education institutions by assisting in the interpretation of school-leaving results, such as those of the National Senior Certificate (NSC). The NBT was designed to: • provide institutions with information that is additional and complementary to schoolleaving results that will assist them in the admission and placement of students in appropriate HE curricula and in programme provision; • assess the entry-level academic and quantitative literacy and mathematics proficiency of students; • assess the relationship between entry-level proficiency and school-level exit outcomes; • provide information to assist in the placement of students in appropriate curricular routes (e.g. regular, augmented, extended, bridging or foundation programmes); and to • inform the development of curriculum for higher education programmes.
The core (domains) areas are: • Academic Literacy (AL): Students' capacity to cope successfully with the demands of academic study in the medium of instruction.
• Quantitative Literacy (QL): Students' ability to manage situations or solve problems of a quantitative nature in real contexts relevant to higher education.
• Mathematics (Maths): Students' manifest ability to grasp mathematical concepts, which form part of the NSC Mathematics curriculum.

Proficient
Performance in domain areas suggests that academic performance will not be adversely affected.

Intermediate
Challenges in domain areas identified such that it is predicted that academic progress will be affected. If admitted, students' educational needs should be met in a way deemed appropriate by the institution (e.g. extended or augmented programmes, special skills provision). AL (42%), QL (38%) and Maths (34%)

0%
Serious learning challenges identified: it is predicted that students will not cope with degree-level study without extensive and long-term support, perhaps best provided through bridging programmes or FET colleges. Institutions that register students performing at this level would need to provide such support.

Extended programmes
At the beginning of 2010 the extended programmes were introduced to try to support the underprepared students. Some students struggle with Mathematics for different reasons, such as a low matric mark for Mathematics, lack of confidence, a poor school system and the mindset that Mathematics is difficult.
The students enrolling for Mathematics in the extended programmes do not comply with the minimum prerequisites for the normal programmes, since their matric results for Mathematics are below the minimum requirements as set out in the programme admission criteria as specified by each institution. However, any other student may decide to study through the extended programmes. Three extended programmes were introduced in the Sciences and Commerce and Information Technology programmes, which include foundation provision in Problem Solving, Mathematics, Programming, Accounting and Statistics. The duration of the extended programme is four years, instead of the normal three years for a Bachelor's degree. The challenge is to advise students on the most suitable programme based on academic preparedness and to ensure that equity of access is matched by equity of outcome.

Research methodology
Action research aims simultaneously to contribute to the practical concerns of people in an immediate problematic situation and to further the goals of social science. Thus there is a dual commitment in action research to study a system and at the same time to collaborate with members of the system in changing it in what is jointly regarded as a desirable direction. Accomplishing this twin goal requires the active collaboration of researcher and client, and it thus stresses the importance of co-learning as a primary aspect of the research process (O'Brien, 1998). Action research is an intellectually demanding form of self-reflective enquiry undertaken by participants in social situations in order to improve the rationality and justification of their own practices, their understanding of these practices, and the situations in which the practices are carried out (Carr & Kemmis, 1986).
What separates this type of research from general professional practices, consulting or daily problem solving is the emphasis on scientific study, which is to say the researcher studies the problem systematically and ensures that the intervention is informed by theoretical considerations. It has been noted that participants in action learning teams are encouraged to participate in team projects that have "strategic value to the client organisation" (Raelin & Raelin, 2006).
The research was conducted as described by Susman (1983), who distinguishes five phases: diagnosing, action planning, taking action, evaluating and specification of learning.
Cycle 1: conducting the NBT test

Phase 1: Diagnosing
As the purpose of this study was to establish what data can be used to assist in the placement of students, its success depended heavily on the timing of the tests, and subsequently a day was scheduled during the orientation week for all first-year students in the targeted programmes to write both papers of the NBT.

Phase 2: Action planning
The main role players, on which the success of the research project heavily relied, were staff members responsible for the orientation week as well as academic advisors. Senior students were trained as invigilators. Academic advisors were invited to provide input into the scheduling of the orientation week. They requested the NBT to be written on the first day of the orientation programme to ensure that students were not tired and to provide time for marking so that results could be used for curricular advice before class commencement.

Phase 3: Taking action
A total of 893 students wrote the Academic Literacy paper and 518 students wrote the Mathematics paper. Classes commenced and the students remained in their original programmes (as determined by the admission requirements). The NBT feedback was received one month after classes had commenced, and individual feedback sessions with students were scheduled in which to discuss the NBT results.

Phase 4: Evaluating
Tables 2-4 provide a summary of the achievement measurements for the NBT. When analysing these results based on the defining criteria in Table 1, it is evident that the majority of the students will experience challenges with academic performance in higher education.
Performance in domain areas suggests that academic performance will not be adversely affected. If admitted, students should be placed in regular programmes of study. For Academic Literacy, only 16% of the students had a performance level of 'basic', while the percentage increased to 51% for Quantitative Literacy and 53% for Mathematics. According to the criteria in Table1, these students will not cope with degree-level study without extensive and long-term support, perhaps best provided through foundation provision.

Phase 5: Findings from Cycle 1
The overall performances of students were very low. Several factors hampered the conducting of the NBT, most of which were related to the administration of the NBT. The administration and organisation of the learners took longer than anticipated and almost nine hours were used to complete the process of writing two three-hour NBT papers. The feedback was received one month after the test was written.
The other problem identified was the students' perception of the importance of the test. The students had already been admitted so they did not view a low score in the test as a high risk.

Cycle 2: Correlation of the NBT and NSC with students' performance Phase 1: Diagnosing
After Cycle 1, the researchers had both the NSC and NBT results, but the question that still remained was whether these tests were a good indicator of academic preparedness.
Due to the time delay between the writing of the NBT and the receiving of the results, limited opportunity existed for academic advisors to advise students to enrol in the extended rather than the normal programme.

Phase 2: Action planning
The cohort of students was tracked and their performance was monitored throughout the semester. The core modules in their respective programmes were identified so that the performance on those modules could be compared with the results from Cycle 1.

Phase 3: Taking action
After the mid-year exam, we compared the results obtained in the NBT subtests with the students' scholastic performance and their first-year academic performance in their core modules.
Correlations were drawn between the students' performance in the core modules and the results of the tests they wrote during admission.
All the statistical analyses were done by the researcher using Microsoft Office Excel 2003 and SAS 9.1. All numerical variables were presented as a mean with standard deviation. Categorical variables were summarised using frequencies and percentages. Table 5 below gives a comprehensive summary of the correlation between the NBT, scholastic performance and the performance of the students in selected first-year modules. All correlations higher than 0.1 are shown in the summary. An 'x' in Table 5 indicates that no significant correlation was found.

Phase 4: Evaluating
The NBT Mathematics scores correlate strongly (0.52) with the performance of the students in Mathematics 1, but even in this case the NSC Mathematics mark was a better predictor and showed the highest correlation (0.83). The Accounting results of the students showed a strong correlation with their NSC performance in Mathematics (0.32) and Accounting (0.41). Their was no significant correlation with the NBT results.
The Financial Accounting results showed a correlation with the Academic Literacy and Quantitative Literacy results of the NBT, but the most significant correlation was with NSC Mathematics (0.41).
The only case where the NBT scores showed better correlation with the actual performance of the students was between the BMI Mathematics module and NBT Mathematics. These findings correspond to those of a previous investigation done by the Faculty of Natural Sciences and Engineering at the Potchefstroom Campus of NWU (NWU, 2010) and others that matric performance correlates best with first-year academic performance and that it is still the best single predictor of future academic success.
The highest correlations were between the BCom Mathematics and NSC Accounting (0.75) and the NSC Mathematics (0.83) results.  Table 6 contains a summary of the actual Mathematics results of the students at the end of the first semester of the first year. In all three programmes, the pass rate of the students in Mathematics was far below the norm of 80% set by the Department of Education. This low pass rate will have a negative effect on the future graduation rate of the cohort of students, but more importantly, it disheartens the students. Although these students were admitted to these courses, these results indicate they would have benefited from foundation provision. The fact that the matric performance in Mathematics shows better correlations than the NBT results can most likely be ascribed to the fact that the students were better prepared for the matric examination and that they viewed the NSC in a much more serious light than the NBT. The relatively small difference between the results obtained in the NBT Mathematics marks and the matric Mathematics marks can be ascribed to the fact that the Mathematics subtest of the NBT is based on the matric Mathematics syllabus. The two variables therefore actually measure the same outcomes, but at different times.

Phase 1: Diagnosing
What became evident during the previous cycle was that although the students met the minimum requirements for the normal programme, some of them would still have been better off being placed in the extended programme from the start. Results indicated that many borderline cases, although they complied with the minimum requirements, also struggled with Mathematics. As can be seen from Table 6, testing seems to confirm that there is a problem with bridging the gap between school and higher education, and interventions will have to be considered to address this problem.

Phase 2: Action planning
In an attempt to bridge the gap, a short learning programme was designed and offered during the orientation week. The rationale for this intervention was the belief that a student who could master these learning outcomes in an encouraging study environment, with academic support, has the potential to successfully complete the normal degree programme. Students who failed this test were strongly advised to enrol in the extended programmes. The target group was all first-year students enrolled for a Mathematics module.
As will be shown in the next section, this intervention resulted in a higher success rate in the normal programme as well as in the extended programme, since students received the right support at the right time.

Phase 3: Taking action
In 2011, six hours (three contact sessions of two hours each) with the students were scheduled. The following learning outcomes on NQF level 4 were covered, which the student should have mastered at the end of this section: § Define the concepts of the slope of a line, parallel lines and perpendicular lines. The focus was on both theory and application. Important aspects of writing mathematical proofs were discussed as well as common mistakes made in writing proofs. The teaching strategy focused on constructive learning with active student participation as well as dedicated time for group work and self-assessment. Definitions and theorems were discussed as well as the 'why' and 'how' of fundamental theory to deepen the students' understanding. Every example was followed up with an exercise. The students were allowed to work in groups of a maximum of three. The lecturer and assistants (senior students) were available for support. A more multifaceted example was then explained to students. They were given the opportunity to complete additional exercises.
Specific interventions were used to try to maximise student involvement and create a sense of accomplishment, e.g. a prize for the first student with a correct answer. The students were given the opportunity to explain solutions to their fellow students. The short learning programme concluded with a formal assessment.

Phase 4: Evaluating
Students who failed the above test were advised to enrol in the extended programmes.
The positive outcome of this intervention can be seen from the results in Table 7.
• A total of 289 students were enrolled in the extended programmes in 2011, compared to 211 in 2010.
• The pass rate for Mathematics in the three-year programmes improved from 36% to 68%.
• The pass rate for Mathematics in the extended programmes improved from 62% to 76%. Phase 5: Findings from Cycle 3 The availability of the test results at an early stage assisted students in choosing between enrolling in the normal or extended programme. The intervention, followed by a test, was a good indication of the academic potential of the students in Mathematics. The percentage of students who passed the first test and passed the module at the end of the semester was 88%.

Conclusion and recommendations
The NBT can be used for classifying students as proficient to less proficient for further studies, but the boundaries for the different categories have not yet been determined satisfactorily and require further research. The matric performance in different subjects can just as well be used for this purpose.
Another big limitation of both the NSC and NBT is that they focus on current competencies, rather than potential. Potential for success depends on many social and cultural factors not included in cognitive measures. It might be an oversimplification to reduce performance to a single meaningful number such as an NSC or NBT score. Further cohort studies can assist in identifying measures that affect academic success.
The findings of this research led to the formalisation of a structured learning programme to be offered during orientation week. The rationale for this intervention is the belief that a student who is able to master a predetermined set of learning outcomes in a conducive study environment with academic support, should have the potential to enrol in the normal degree programme: this has been confirmed by the research project. A further advantage foreseen is that the institution can direct the outcomes, timing and turnaround time of the conducting and marking of the tests. Learners who fail to achieve these outcomes will be enrolled in the extended programmes.
When comparing the students' performance in the NBT and the NSC, the relationship between the NSC and first-year results is stronger than that of any other results. The authors are of the view that instead of developing a parallel system of assessment, the NSC should be further understood, developed and strengthened as an assessment instrument for predicting success in higher education studies. This can be done by creating structures and forums in which academics and instructional designers involved in Mathematics and Mathematics school teachers can engage in discussions, share experiences and collectively address the gaps in students' understanding of Mathematics.