Home / Archive / 5. Issue / Evaluating ourselves to death. An alternative way to compare on- and offline learning
Document Actions

 

Evaluating ourselves to death.
An alternative way to compare on- and offline learning

Author: Preußler, Annabell
annabell.preussler@uni-due.de

Lehrstuhl für Mediendidaktik und Wissensmanagement, Universität Duisburg-Essen
Forsthausweg 2
47057 Duisburg

Supervisor: Prof. Dr. Peter Baumgartner
peter.baumgartner@donau-uni.ac.at

Department of Interactive Media and Educational Technology, Danube-University Krems
Dr.-Karl-Dorrek-Straße 30
A-3500 Krems

urn:nbn:de:0009-5-20753

Hochschulschrift (Dissertation)

FernUniversität in Hagen, Lehrgebiet Medientheorie und Medienpädagogik, 2008

URL: http://deposit.fernuni-hagen.de/505/1/Wir_evaluieren_uns_zu_Tode.pdf

Recently, media based learning and e-learning are up to date. Current trends focus on cooperative and collaborative learing, regarding the importance of web 2.0 applications and media literacy. Institutions of higher education seem to accepted the dare during the last years. Online- and blended learning arrangements become important parts of the curriculum - with great potentials: Interaction and activity are high in the ratings and formal learning becomes more and more informal. As this is an antagonism dicactical issues and practical benefits seem to be ambiguous. It is still relevant to evaluate and measure the additional values of learning scenarios for being able to develop concepts for the use of the new media. For measuring the success of e-learing, comparision studies are used in most cases - one setting is compared to another. But are such comparisions useful?

We are "evaluating ourselves to death" regarding the surges of evaluation studies that are conducted with various forms, various claims and definitions and various topics. One part of these studies report of no significant difference between the settings, another part says that e-learning is more successful while the third part found out that the best outcome can be reached with traditional learning. Thus, reliable conclusions are not possible - even worse: in the end it is not even clear what learning outcome really is, as the definitions change in the plurality of the studies (cp. Annabell Preussler & Peter Baumgartner 2006).

So, what makes learning successful and how can this be detected? How can a high quality in learning be reached?In the area of education this is evaluated by measuring the learning outcome. It becomes an indicator for the construct of learning quality, but actually, learning outcome is a construct as well. Observable characteristics are deducted from a particular theoretical construct and must be questioned on the basis of the underlying theoretical assumptions. If, for example, the ability of remembering is used as the key indicator of learning, then many other possible indicators (such as apply, evaluate, create, etc.) are excluded or ignored.

Secondly, there is yet another problem: For comparing the learning outcome of two (or more) different settings, mostly the same assessement for both groups is used. Then there is the risk of regarding only one specific dimension of learning. In most cases this is the dimension that one setting can achieve anyway. Advantages of the other setting are not taken into account.

In this work a meta-evaluation was conducted in order to find out how the problem is being dealt with in practice (Annabell Preussler 2008). Evaluation studies regarding the coherence of e-learning and learning outcome and and the influence of e-learning respectively were analysed and compared. One hypothesis was that learning outcome can not be operationalized clearly and that nonspecific comparisons of e-learning and traditional learning can not be conducted unlimitedly meaningfully.

In order to analyse and evaluate existent studies a meta-evaluation was conducted. This was done as a review (cp. Cook & Gruder, 1978:17), which did not explicitly focus on the representativeness of the single results but documented indifferent perceptions of learning outcome and their operationalisation and measurement. After the specification of the relevant criteria, 11 primar studies were involved in the meta-evaluation. The coding was done using the methods of deductive content analysis (cp. Mayring, 1993, cp. Widmer, 1996:64).

The meta-evaluation revealed the following: While reasonable and practical procedures reason to good feasibility of the single studies, the studies regarding accuracy (validity and reliability) and especially the adequacy of research decisions score far worse. Different conditions were either measured with the same tests or - if the learning objectives are properly tested at the same level of cognitive processes - the design of the assessement was not related to the intended learning objectives.

For future studies comparing the outcome of e-learning to traditional learning these recommendations are given:

  1. The intended learning objectives of both groups should be in the same level of cognitive process dimensions (Lorin W. Anderson & David R. Krathwohl 2001).

  2. The assessment should be adequate to these objectives

  3. The design of the assessement should correspond to the learning arrangement in the test groups.

License

Any party may pass on this Work by electronic means and make it available for download under the terms and conditions of the free Digital Peer Publishing License. The text of the license may be accessed and retrieved via Internet at http://www.dipp.nrw.de/lizenzen/dppl/fdppl/f-DPPL_v1_de_11-2004.html

Language
  1. Deutsch
  2. English
Contents
Navigation
Tag Cloud
3D MUVES academic research accreditation of prior learning activity tracking adaptivity administrative authorities adult education agent ambient computing apl artifical neural network assessment Aufsatzsammlung Ausstattung Australien authoring Authoring Tool avatar barrier-free e-learning Behörden berufliche Bildung Berufliche Fortbildung betriebliche Fortbildung Bildung Bildungseinrichtung Bildungsplattform blended learning Business Plan CampusSource Engine central actors CMS Cognitive learning collaboration collaborative filtering communication communication task competence development computer aided assessment computer aided grading computer literacy computer networks computer science education Computer security computer simulation Computer-assisted instruction computer-based-training Computersimulationen computerunterstützes Bewerten computerunterstütztes Üben congress content management content sharing context controlling CopperCore cost accounting cost management courseware critical thinking cross-media publishing darwikinism data interchange data mining Data processing decision making design design research desktop publishing development and exploitation of software components Didaktik didaktische Methodik didaktischer Entwurf digital age distance education distance learning document server Dozent Drehbuch e-assessment e-journal e-language learning e-learning e-Learning 2.0 e-learning application e-learning design e-learning technology E-Learning-Integration e-learning-Plattformen e-learning-systems ecology education Educational Broadcasting Educational content Educational technology Educational Television Edukalibre project EDV-Infrastruktur Effizienzmessung ELAN III eLearning community portals Electronic Documents eleed eLite project emergence eModeration Empirische Studie Employees engineering education Enterprise Java Open Source Architecture (EJOSA) Entwurfsprozess Lernen Erwachsenenpädagogik Erziehungswissenschaften ESOL EU-US cooperation in higher education eupace european market place European program european projects evaluation examples Fernerkundung Flow-Analyse folksonomy Fortbildung Four Level Frame Work Geoinformatik Geschäftsmodell gestures grounded theory Handlungsorientiertes Lernen Hannover 2004 higher education higher education administration HIS-Software Hochschule hybrid concepts ideal types identity management immediacy IMMEDIATE immersive impact of technology implementation Implementierung IMS Learning Design incidental learning informal learning information appliance information literacy information management information quality information retrieval information systems information technology information visualization Informationsmanagement informelles Lernen Ingenieurausbildung innovation instructional design instructor support integration integration of blended learning intelligente Tutorensysteme interaction interactive technology Interaktiver Software-Entwurf internet Internet in education invisible computing IT Infrastructure Library ITIL Java juristische Argumentation knowledge creation knowledge management Kollaboration Kongress Kursplattform Kursverwaltung künstliche neuronale Netze language learning theories large course design Latent Semantic Analysis latent semantic analysis LCMS learn at work learner learning learning by teaching learning content management system (LCMS) learning design learning fields learning management platform learning management system learning networks learning object repository learning objects learning platform learning processes learning resources learning situation learning technology standards learning theory Learning-Content-System Learnmanagement Lehrer Lehrerfortbildung Lernen Lernen durch Lehren Lernerfolg Lernfeld Lernmanagement-Plattfom Lernplattform Lernpsychologie Lernsituation library lifelong learning LMS logfile analysis LOM lsa Management Management von verteilten Lehrinhalten market orientation mash-up mathematical application mathematical problem solving mathematische Anwendung Medienautor Mediendidaktik Medienpädagogik medium sized enterprise metadata middle east MOA-model Mobile Learning Model Driven Architecture (MDA) Modellversuch Moodle motivation multilinguality multimedia Multimedia-Autor multivariate Methoden multivariate methods Nachhaltigkeit von Inhaltsentwicklungen navigation support negotiation of meaning net-based pedagogy new media in technical education nursing education nursing skills and competences online course online survey Onlinestudie open content open course open source Open University Support System (OpenUSS) OpenFUXML OpenUSS organisation model Organisationsforschung Organisationsmodell Organizational learning PACE paradigms peer-support performance of blended learning Personal Competence Manager personalization personalized recommendation phenomenography pilot project placement positioning prior knowledge Problem-Based Learning process and chemical engineering professional development programming programming interface project management Projektmanagement Prozessmanagement Psychologie public administration Pädagogik quality Qualitätsmanagement R-Statistical computing recommender system Redaktionssystem reflection repository networks reusability and interoperability reusable learning materials ROI reusable object and instruction role expectations Rollenverständnis satisfactions scientific use of web SCORM Second Life second life Security measures selbstorganisiertes Lernen self directed learning environment self-directed learning self-organized community self-organized learning seminarbegleitend Seminardesign service service integration serviceorientierte Gesamtkosten situated learning small and medium sized companies small sized enterprise social behaviour social constructivist Social Learning social network analysis social networks social software socio-cultural perspective software design tool software development software engineering software for science and Mathematics software tools Software-Management Soziale Lernplattform soziale Netze statistics Statistik Statistikprogramm R strategisches Controlling strategy student student mobility Studienseminar swemof system architecture Systematische Instruktionsdesign tags taxonomy TCO-Tools TCS teacher community teachers' in-service training teaching book technology enhanced learning Tele Learning teleteaching television Television and Learning Television supported Learning TENCompetence text mining text vector space model thesis Total Cost of Ownership (TCO) Total Cost of Services Training of Computer-assisted instruction Training of Employees tutor workload tutoring systems UML Umsetzungsbeispiele United States of America university university teaching Universität Zürich USA-Deutschland user generated content user interaction user interface variation video conference Videokonferenz virtual environment virtual learning environments Virtuelle Fachhochschule (VFH) Virtuelle Lehrerfortbildung virtuelle Lernumgebung virtuelle Universität virtueller Campus vocational education and training web web 2.0 web based training web portal web-based information systems web-based instruction web-based learning webservice Weiterbildung Wiki wissenschaftliches Arbeiten Wissensmanagement Worldwide Web WWW XML XML-Editor Ökologie öffentliche Verwaltung