Student enrollment in college computer science (CS) and information technology (IT) programs are in a downward trend in most developed countries.1,4,7,9 At the same time, the U.S. Bureau of Labor and their counterparts in other countries forecast an increased need for skilled CS/IT graduates.7
To reverse the downwards trend in college level CS/IT enrollment, ACM launched the Computer Science Teachers Association (csta.acm.org), whose mission is to “directly and passionately” advocate for computer science education in the high school.
One major initiative is to develop a comprehensive computer science curriculum that educates high school students and informs them about the profession because a major factor in the enrollment decline is a lack of knowledge about possible CS/IT careers.4,9 In this article, we examine one CS/IT program (the Cisco Networking Academy that has been offered in high schools since 1978) to understand what lessons we can draw for other CS/IT high school initiatives.
The Cisco Networking Academy
The Networking Academy offers several CS/IT programs, but the most popular is the Cisco Certified Network Associate program (CCNA). The four course CCNA program has been adopted by over 10,000 high school, college and non-traditional education institutions in more than 150 countries.
There are four key components to the program. First, a centralized curriculum designed by the Academy and regularly updated based on task analyses of network engineers is distributed over the Internet. It also is linked to high school curriculum standards across the states in science, mathematics, and language arts. The curriculum includes interactive learning materials for key concepts, lab exercises for hands-on skills, and a network simulation package that enables students to use virtual environments which contain equipment that their schools do not have.
Second, local high school teachers teach the courses as part of the normal curriculum in their school. Teachers can customize the curriculum materials to their environment.
Third, the curriculum includes standards-based testing designed using the same psychometrically sound methods as the Cisco professional certification exams. Local teachers can use the tests for formative and/or summative purposes and can weight them as part of student grades as is most appropriate for their environment.
Fourth, a program of ongoing training, technical support, and instructor certification for participating schools is conducted by other high schools and community colleges, not by Cisco. Thus teachers from “lead” schools train and support teachers in course content and how to teach. This approach has enabled the Academy to grow to over 10,000 schools worldwide in less than 10 years.
The Study
We examined how the characteristics of the students, their school, and the instructional practices contributed to student achievement and confidence, as these factors influence the choice to enter college programs. We studied U.S. high school students entering the first course in the CCNA program using a multilevel statistical technique called Hierarchical Linear Modeling; 27% of eligible U.S. students participated (5,392 students at 764 high schools).
Individual Factors
Demographic data (age, gender) were collected by the Academy during course registration. Males often have greater interest in computers than do females.3 We tracked age because the program is taken by students as young as 14, and age may be a factor in achievement.
We examined students’ prior academic success and computer technical skills as these are often important predictors of achievement.6 Many studies have linked short term and long term motivation to achievement,11 so we examined student’s motivation in this class, their desire for lifelong learning, and their career goal (CS/IT or not). Students who have selected a career closely related to an educational program tend to perform better than students who are indecisive about their goals.11 These variables were assessed using a survey administered as students entered the program. We used self-reported Grade Point Average (GPA) as the measure of prior academic success, as self-reported GPA is a reliable measure.6 Technology skills, short-term motivation and long-term motivation (desire for life-long learning) were measured using 3-6 questions for each variable (alpha values from .86 to .92). Career goal was a single item question.
School Factors
One of the most influential factors in student achievement is the amount of money spent in the schools;10 students in economically disadvantaged regions generally perform lower on standardized tests. We used the per capita income (in thousands) in the student’s ZIP code from the U.S. Census database as the measure of income.
Geographic location of the school is important because urban and rural locations are often found to have lower achievement and confidence.5 We used the NCES database to classify schools as urban, suburban, town, or rural.
We expected that students at lead schools that trained other schools (and were selected for their capabilities) would have higher achievement because of better physical, educational, and human resources. We used the Academy database to classify schools as being a lead school or not.
Instruction Factors
Instruction quality, the ability of an instructor to provide instructional activities in the classroom that engage students in meaningful learning activities, significantly affects student learning.2 Instruction quality (alpha=.92) was collected via a course evaluation at the end of the course.
Class size may affect student achievement, as it influences the amount of attention each student gets from the instructor, especially for younger students. We used the Networking Academy enrollment database to determine class sizes.
Achievement and Confidence
The individual, school, and instructional factors were used to predict student achievement and confidence with technology. Achievement was measured using the grade from the online, final exam administered by Networking Academy testing system. Confidence focused on confidence with networking as this was the subject of the course. Confidence is an important factor influencing the choice of a CS/IT major.1 It was collected via a course evaluation at the end of the course (alpha=.92).
Results
Table 1 shows the results from the statistical analysis, including the relative importance of each factor (the “effect size” as measured as a percent of the variance explained). A factor with a relative importance below 10% has little practical impact, so while it may be significant, it is not particularly useful in developing programs.
Student Achievement
Student ability (GPA and technology skills), short term motivation, and gender affected student achievement. Females scored lower than males by about 3.6 percentage points, even after controlling for other factors. However, age, the desire for lifelong learning, and whether or not one had a CS/IT career goal had no impact on achievement.
At the instructional level, quality of instruction, but not class size affected achievement. Surprisingly however, no school level factors had a significant impact on achievement. Geographic location, per capita income, and whether the school was a lead school had no impact on achievement.
The most important factor influencing student achievement in high school CS/IT classes is the individual student’s GPA. After this, four factors play a moderately important role: the student’s motivation, the quality of instruction, the student’s technology skills, and, to a lesser extent, the student’s gender (males achieve higher).
Student Confidence
Student ability (GPA, and technology skills), motivation (short term motivation, career goal, and desire for lifelong learning) and gender all affected students’ confidence in their use of technology. However, age did not affect confidence. At the instructional level, quality of instruction mattered, but class size did not. At the school level, per capita income and an urban location increased confidence but being a lead school did not.
The most important factor influencing student confidence in high school CS/IT classes is the quality of instruction. After this, three factors play a moderately important role: the student’s motivation, the student’s technology skills, and, to a lesser extent, the student’s gender (males are more confident).
Recommendations
The most important factors influencing student achievement and confidence are individual student factors (ability, motivation, technology skills, and gender) and the quality of instruction they receive. One common sense recommendation is therefore to select able students who want to succeed; CS/IT courses should be elective courses, not required courses so that motivated students can self-select into courses and teachers can screen for abilities.
Having a CS/IT career goal did not have an important impact on achievement or confidence. About 50% of the high school students in the Networking Academy program had no intention of pursuing a career related to CS/IT, aspiring instead to business, trade, or professional careers. Thus we recommend that entry-level CS/IT courses in high schools should be designed for CS/IT majors and non-majors alike. By attracting those who do not initially intend to pursue CS/IT education, we have the opportunity to win over students to CS/IT careers. These courses should not focus on skills needed by CS/IT professionals, but rather on general CS/IT skills of use to all. For example, programming is a key skill for some CS/IT professionals, but is not a key CS/IT skill for all. In contrast, analysis, design, and effective use of IT is a critical CS/IT skill for all.
Male students had higher achievement and were more confident than female students, even though the average GPA of females in our sample was higher than that of males. One important recommendation is to pay particular attention to the recruitment, education, and retention of female students, who, without such attention, are likely to under perform and feel less confident than their male counterparts.
Age had no impact; 15-year-olds performed as well as 18-year-olds. Thus CS/IT programs can be geared to all high school levels; they should not be targeted only to juniors or seniors who may have already formed career plans and opinions of CS/IT.
The quality of instruction was especially important for confidence. The Networking Academy devotes considerable resources to teacher training, requiring teachers to pass a standardized exam before teaching a course. In addition to initial training, teachers must complete 16 hours of annual training and regular re-testing to ensure their skills remain up to date in the ever changing world. Given the importance of instruction quality, we believe that other CS/IT initiatives should also provide considerable initial and ongoing teacher training and certification.
Perhaps most surprisingly, school factors such as geographic location, and income had little impact. This is in sharp contrast to research showing that students from wealthier suburbs outperform students from poorer urban and rural locations.5,10 This program overcame the traditional handicaps faced by low income, urban schools, and enabled students to rise to the level of their own ability and motivation as influenced by the quality of instruction they received. We hypothesize that this leveling of the playing field is due to the balancing of centralized support (training, 24/7 help, curriculum, and assessment materials) with local control permitting instructors to adapt the curriculum to their students’ particular circumstances.
The Networking Academy program is a prime example of corporate strategic philanthropy8 in which Cisco Systems, Inc. has built on its core strength in networking to develop a program to improve CS/IT education worldwide (in both developed and developing nations). By partnering with high schools (and colleges, universities, and non-governmental organizations), Cisco has leveraged its unique skills and the unique skills of these institutions to affect the lives of over 1 million students in more than 150 countries around the world. We believe this type of partnership is a useful model for enhancing CS/IT education in the US and around the world.
The Cisco model assures a common curriculum and assessment strategy in schools across the U.S. As such, the Cisco program, in addition to providing a public/private partnership model, provides an ideal opportunity to identify success factors in the design of any program. We suspect the training and technical support for instructors is a critical factor in student success as reflected in the importance of instructional quality in student learning and confidence. As importantly, the results suggest programs should appeal broadly to students: to majors and non-majors and to students at all level of high school.
We conclude that the blend of centralized curriculum and testing, combined with local instruction and a strong instructor support program, enables the best of both worlds: clear standards-based national curriculum and assessment, and local control and customization of instruction with well-trained teachers.
Join the Discussion (0)
Become a Member or Sign In to Post a Comment