At work, my title is the Data Warehouse Manager. Mostly, I work with numbers. Lots of test scores, checklist data, compositve variables and the like. Numbers are cool and they can often yield plenty of insight. However, numbers are not the only story that can be told! So much of business intelligence right now focuses on natural-language processing (NLP). I once heard someone say that getting to 80% in NLP was almost comically easy but getting to 90% has proven to be amazingly difficult! Obviously, I’m not content to wait around until the singularity (though I do believe it is near) to be able to utilize textual data for informing educational decisions. So then where is the merit in utilizing textual data if we can be only 90% sure of something? Numeric metrics like test scores are perfectly clear and definite, right?
Wrong! Any good educator knows that test scores (especially from a single test) are not the only way to gain additional insight. While the strengths and weaknesses of the standardized-testing model are a different discussion entirely, suffice it to say that the community needs to remember that, when it comes to education, all numbers are fuzzy. The learning process is not discrete, definite or clean-cut so why should we expect a signle number to show singular accuracy with any great fidelity? In fact, we cannot and it is for this reason that an automatic evaluation of textual data might help educators make better decisions. Perhaps an example is in order.
Let us say that every so often in a math class, teachers write a brief one-paragraph essay about each student summing up their competency and general performance. Then let us also say that students are able to write a sefl-evaluation each time their teacher writes something. So after, a period of time, each student has a bit of text in addition to their test scores and general grades. I propose using some sort of automated classification mechanism to evaluate the textual data and produce best guesses as to the nature of the content. Afterall, if there are 25 students in a class and there are 10 classes taught each day, then that’s a lot of evaluation data to be read! So what about using something like a naive bayesian classifier to sort through things?
But wait, doesn’t this just give us another mechanism to boil down the real-world intuition of the educational system? Well, no. If we let go of the idea that these numbers are perfectly discrete (which we know they aren’t), then we can use them to inform our descisions rather than letting the data decide for us. Probabilities are ok so long as we use them correctly. Few people would wear shorts in the winter just because the weatherman said there’s a chance of sun. So how might such a system be used?
Well, the point of a naive classifier is that it needs to be trained. First, teachers would need to assmble a training data set of some sort. In younger grades, this might be quite easy because students have a more limited vocabulary and might not construct sentences with as much nuance as they learn to do in later grades. As for teacher evaluations, there is a somewhat-finite set of words in the vocabulary of an educator to usefully describe educational progress. Relying on the consistency of such a set nomeclature might very well yield consistent results. Though, all of this would need to be tested anyway, right? This is just a thought experiment…
So the system would be able to chew through these evaluations and produce a guess about whether they reflect positively on student learning or not. Perhaps they could even be used to demonstrate growth. Getting this right would take work but it might lead to a useful way to model evaluative-data. Obviously, relying too much on this might very well to a negation in the positive nature of narrative evaluations but it’s worth a try!
