Who Is Going to Inspire Tutoring’s Possible Two Sigma Results?

Benjamin Bloom wrote a seminal 1984 paper commonly referred to as the “two sigma problem” for the outcome differences between one-on-one tutoring and group instruction, in which he identified the problem of any resource constrained environment:

Can researchers and teachers devise teaching-learning conditions that will enable the majority of students under group instruction to attain levels of achievement that can, at present, be reached only under good tutoring conditions?

The piece classified three types of instruction: classroom, mastery learning, and tutoring. The former two types are 30:1 classroom settings with mastery learning incorporating the pedagogical principle that you don’t move forward until mastery has been achieved. Of course, the distribution of results creates a two sigma outcome difference for tutoring over general classroom instruction, with mastery learning in between. The article from 1984 concluded with four recommendations, the top two being most salient: improve using the mastery learning feedback-corrective process and choose instructional material proven to be effective.

Can technology close this gap, identified from an article written in the age of the PCjr?

The New York Times Magazine cited Bloom in an unfortunately named piece called “The Machines Are Taking Over” in the Sunday Magazine. It assessed the potential for machine-based tutoring given results from programs like ASSISTments and Cognitive Tutor. Unlike pre-programmed educational software, these two are intended to improve student learning by using the resources of a massive data of correct or incorrect answers, then the tutoring function should help steer students to the correct answer.

ASSISTments (developed by Neil Heffernan, who is the featured professor of the article) can be previewed online:

And of course like most computers it can be finicky.  The article mentions, “It’s no accident that ASSISTments and other computerized tutoring systems have focused primarily on math, a subject suited to computers’ binary language.” And so it is finicky on how it wants answers:

For readers paying very close attention, I have entered the incorrect answer (well, ahem, I believe: 5 5/14 being correct.) Unfortunately, at the end the test did not generate what it thought my answer was — the test just ended! What is the class of error that would generate me writing 6/14 instead? Other errors would be easier; for example, had I written 10/28, that seems simpler to solve the error in my expression.  What would it mean if I wrote 100/280 and inflated instead of reduced? Standardized testing companies — with even larger data sets — are very good at constructing these questions.

Cognitive Tutor, acquired by Apollo Education last year, isn’t much better when it comes to user interface, but Cognitive Tutor didn’t have an online session to try before I buy:

Interactive Example

There is a question about whether there are improved student outcomes from the use of such software identified by the Department of Education. A review of the literature at their “What Works Clearinghouse” shows some pro and con studies, but nothing decisive. The smell of the DOS prompt from the PCjr days still lurks on most of these programs. The lack of flair and inspiration smokes out a larger point.

Needless to say the technology is surely advancing very fast, and thanks to the big data of learning, we will be able to classify errors and reaffirm pathways of learning much in the same way as brain myelination. Companies such as Grockit and Knewton are usually well regarded for their ability to provide “apative” testing to known problem sets (in their case standardized tests), and were surprisingly omitted from the article. The current state of the software looks much longer on potential than results right now, but the exponential technological effects of Moore’s Law will kick in soon enough.

The role of the tutor is far beyond error correction — instead, it lies in inspiration. If the machines can “take over,” a lot of the labor-intense functionality of observation and correction — Bloom’s feedback mechanisms — in order to help optimize the tutor-tutored time, then technology will have possibly enabled even 30:1 classroom settings to be optimized like a 1:1 tutoring function.  One could assume a Platonic piece of software that was perfect in error correction and knew the optimal path for all learning (or hard sciences at least).  But would people use it?

The tutor looks into the life and needs and goals of the student.  For that, ASSISTments and its heirs will still be in need of personalized learning the likes of which one still might be able to get online from the INSTAedus of the world.