In traditional recitation arrangements (e.g. manually graded homework and recitation sessions that are aligned with the lecture), students receive their results with some time delay while the lecture venue progresses. An inherent danger is that potential deficits are neither detected nor corrected. For traditional recitations to be effective (foremost to have small sessions) they require personnel, which, particularly for German universities of applied sciences, is not available. A second problem, which in part occurs due to lack of practice, is, that many students do not review the lectures. Instead, many students attempt to cram the lecture material close to the exams, which, if successful at all, does not lead to long-term mastery of the content. Therefore, base knowledge is missing for subsequent curricular venues, which should have been established in earlier venues.

Taking a closer look, students today are missing a “pacemaker”, which keeps them on track in the learning process. To eliminate or at least mitigate the described deficits, in 2006, the VITA – Virtual Teaching Assistant project was started at the University of Applied Sciences Braunschweig/Wolfenbüttel. Within the framework of the ELAN III project eÜbungen, the concept was expanded to the universities of applied sciences at Oldenburg/Ostfriesland/Wilhelmshaven and Hannover. Computer-graded practice problems fulfill several functions within this project:

The spectrum of possible problem types includes, besides classical multiple-choice problems, in particular problems with open-ended answers. In math and science, such answers can take the form of numerical values (in the sciences including units) or mathematical expressions. For numerical answers, it is checked if the answer provided by the student agrees within tolerances with the correct answers. If the answer is a mathematical expression, it is checked if that expression is mathematically equivalent to the correct answer. Parameters of the problems, such as due dates, maximum number of tries, number of achievable points, etc., can be freely adjusted. Beyond that, different students usually get different versions of the same problem. For example, these versions may differ in that they have different numerical values, different parameters in mathematical functions, etc. This makes it more difficult for students to blindly copy answers, but at the same time does not hinder collaboration on solution strategies. The figure illustrates this with an example where the student has to calculate the derivative of the given function. The input in the right panel is indicative of a common mistake, which in this case triggers a hint. A selection of such practice problems can be found at www.fh-wf.de/vita/Demoaufgaben (mostly in German; some are multilingual where the language will be determined from the browser settings).

Figure 1 - Screenshots of two random versions of the same problem.
Right: The student's input exhibits a common mistake which triggers an appropriate hint.

As technological platform for the computer-graded practice problems, LON-CAPA was chosen, which is a member of the CampusSource exchange. A particular selection criterion for us was the availability of symbolic algebra functionality, however, CampusSource exchange offers other systems with homework functionality: metacoon Edition wislearn, ILIAS, Moodle, and WebAssign all offer automatically graded homework functionality that could be used in similar didactical scenarios. The varying implementations in some of these systems were added later to existing course management functionality, or, as in the case of LON-CAPA and WebAssign, are part of the initial design. The Canadian site edutools.info, though somewhat outdated, provides useful system comparisons.

LON-CAPA is open-source software, which runs under Linux. From the get-go, the system was designed as a network, so all servers at the over 130 institutions that currently use LON-CAPA can exchange learning content with each other. The network currently offers instructors access to more than 310,000 online learning components in various disciplines, of which more than 120,000 are practice problems. The LON-CAPA Academic Consortium is governing the network. The Consortium ensures that only bona fide-institutions can join this network, and that locally standards of data security are observed. In spite of the networked character of the system, personal data is only permanently stored at the home institution of the user, thus fulfilling data privacy legislature.

So far, the practice problems generated in the framework of the VITA project have been used in 43 courses across the participating universities in Lower Saxony. In general, every single problem has been used in more than one course. About 20% of these problems have also been used by other universities participating in the LON-CAPA network worldwide.

The pacemaker function was confirmed. Participating instructors report that students are noticeably better prepared if such problems are made available for preparation and review of lectures. Students also see this positive influence: in a survey of more than 130 students at the universities of applied sciences Braunschweig/Wolfenbüttel and Oldenburg/Ostfriesland/Wilhelmshaven half-way into the eÜbungen project, computer-graded practice problems were welcomed by 60% and resented by 10%. Interestingly enough, the very same aspect can play a role both ways. Supporters evaluate the pace maker function positively, while skeptics view it as turning college into high school.

From the instructor perspective, another positive effect is that the students are asking more questions about the problems and thus about the learning content. On the negative side, occasionally the higher initial and administrative effort is mentioned, compared to traditional teaching venues. Some instructors see the danger of misinterpretation of results, e.g., if students arrive at the correct result through systematic trial and error, or by a wrong mathematical algorithm. How significant these effects are cannot be evaluated at this time. These effects can be reduced by using concept-oriented questions and teaching methods and can be compensated by combining computer based and traditional exercises.

Regardless of the disadvantages overall the vast majority of instructors and learners evaluated the usage of computer-based practice problems as positive and helpful. The mentioned problems are starting points for subsequent work and research projects, such as the development of tools and the introduction of innovative teaching concepts.

Weblinks