Can machine learning ace the toughest tests?

$257
Raised of $9,000 Goal
3%
Ended on 5/30/13
Campaign Ended
  • $257
    pledged
  • 3%
    funded
  • Finished
    on 5/30/13

About This Project

A talk about how involving teachers, students and administrators into dialogues and incorporating alternative technologies can bring change to the education system. Proving that machine learning can ace the toughest tests will help convince deciders that the time has come to rethink assessments in education.

Ask the Scientists

Join The Discussion

What is the context of this research?

Machine learning is a field of computer science that deals with training a system over a corpus of documents hoping that it can then be useful at the task of recognizing certain features in new unknown documents. The interesting thing with school exams is that usually you have a large history of graded and even annotated documents which makes a great corpus for a machine learning task.

What is the significance of this project?

This project matters to me because I feel that we are collectively focusing on skills at school that might not be the most relevant today. Before Gutemberg, people would memorize the contents of whole books. I feel that computing and the internet have also introduced a huge change in paradigms but that the education system is slow to catch up. Applying machine learning to exams can help evaluate how much an exam assesses qualities that are still intrinsically human (such as intuition and creativity) as opposed to those that computers get better and better at every day. I hope that this will lead to a rebalancing of what skills are taught at school.

What are the goals of the project?

My goal for this project is to build a computer system that will be able to read an exam description and that will be able to achieve a grade that is above the human average at it. The system will be open source in order for everyone to be able to adapt it to their own exam structure. I will let the backers decide on which corpus of exams I should use as a basis for this research. Here are a few ideas: • The French National competitive exam that young people take at the end of "Classes Préparatoires". A huge corpus of exams from previous years with their corrections is available and maintained online. • Coursera.org tests all follow the same format and are widely available. It is a great opportunity to build a system that is not domain specific. • The GMAT which is also standardizes and for which a lot of answers and practice exams are available online. It offers a writing component which could require some more advanced natural language generation techniques.

Budget

Please wait...

This endeavor will require storage, computing power and a team of people annotating documents and formatting them so that they can be read by computers. These tasks are costly but will will rely on the cheapest provider in that market: Amazon Web Services. Also any help from the backers on the annotation and formatting tasks would be greatly appreciated and would help reduce our dependency on Mechanical Turk.

Meet the Team

Thibaut Labarre
Thibaut Labarre
Computational Linguist

Affiliates

I am a student in computational linguistics at the University of Washington. Before that I was a student in an engineering school in France and I went through a process called "classes préparatoires" which basically involves 2 years of cramming for a national competitive exam by studying intensely mostly math and physics. I thought that this system was really outdated since the emphasis was on achieving good grades rather than understanding concepts. And the pedagogy could use a lot of improvement in order to foster such qualities as innovation and systemic approaches.
View Profile

Team Bio

During his childhood, Thibaut went through very different educational systems: a Japanese Hoikuen, an international Montessori school, distance learning in elementary school, a French engineering grande école and finally an American university. Building on this breadth of experiences, he co-founded the New Students, an open movement that aims at involving teachers, students, administrators and citizens in rethinking the way we teach and learn in the age of the Internet.

Thibaut Labarre

During his childhood, Thibaut went through very different educational systems: a Japanese Hoikuen, an international Montessori school, distance learning in elementary school, a French engineering grande école and finally an American university. Building on this breadth of experiences, he co-founded the New Students, an open movement that aims at involving teachers, students, administrators and citizens in rethinking the way we teach and learn in the age of the Internet.

Lab Notes

Nothing posted yet.

Additional Information

I plan to make everything open source. The end result could be an automatic exam taking machine which can always be useful :) Throughout the journey I think it should be possible to post the progressively improving results of the machine learning system and the grades they receive.

It does challenge the convention that current exams are good for assessing the skills they are supposed to assess.

The next steps would be to apply this exam learning system to other types of exams. The legacy could be that we have discovered a good way of assessing whether an exam tests skills that computers are better at than humans. At the very least it should yield some insights about what is hard and what isn't in those exams and help build a better pedagogy around them.

We all have a role to play in this great shift in the way we share and create knowledge. Microryza helps by breaking the dependence of research on red tape. This project would help break the assumption that exams assess the skills that they are built to assess.

Project Backers

  • 12Backers
  • 3%Funded
  • $257Total Donations
  • $21.42Average Donation
Please wait...