Methods
Summary
Amazons Mechanical Turk (m-turk) is a crowd-sourcing website whereby users from around the world can complete small on-line tasks for cash. We ask if these m-turkers can help us detect sleep features, and if their detection quality matches that of qualified sleep technicians.
In phase one of this project 760 hrs of EEG data from 100 sleeping subjects will be analyzed for sleep spindles by experts (expert sleep technicians) and non-experts (m-turkers) . We have developed an on-line sleep scoring interface which displays sections of EEG data (artefacts removed) to users. After a reading detailed instructions on how to detect sleep spindles (the patterns we are looking for in the brainwaves), users draw boxes around potential spindles and mark how confident they are that the box contains a spindle. M-turkers are paid a small amount for their time. Each segment of EEG data is viewed multiple times (20+ times for non-experts), at least 5 times for experts. Applying a general consensus rule to all the expert and non-expert data separately, a golden standard for the locations of sleep spindles can be generated. Performance of the experts and non-experts can then be compared. The gold standard from the expert sleep technicians will then be released publicly,
This released data will be hugely beneficial in the development of automated spindle detection algorithms.
Protocols
This project has not yet shared any protocols.