About This Project

Ask the Scientists

What is the context of this research?

Traditional Chinese medicine (TCM) is not only widely practiced in Asia, but also considered as the most effective alternative medicine in Western countries. In TCM, the features of tongue, including shape, color, coating, and pattern, are used as a basis for diagnosis of disease and classification of health status. Recent scientific studies have also confirmed some relationships between tongue features and diseases such as cancers and AIDS.

However, there are a number of issues in the traditional tongue diagnosis. In particular, the inspection of the tongue by eye may be subjective depending on doctor’s condition and experience. The traditional method also lacks standards for quantification. There are no tongue images recorded as part of electronic medical records.

Therefore, an electronic system that automatically records and systematically interprets tongue images is highly desirable. In recent years, mobile devices are explored as a platform for medical purposes. We believe that mobile devices are ideally suited for tongue diagnosis.

What is the significance of this project?

In this project we will develop a tongue image analysis App—ITongue. ITongue will allow the user to take a photo of the tongue with a smartphone or tablet, such as an Android phone or an iPad. Then the App will automatically analyze the image to interpret the person’s health status and provide advices accordingly.

Although the App is not intended for professional diagnosis, this household product can provide warning, especially in physical and mental weaknesses, and the users can follow up with hospital visits or lifestyle changes. Any person can use the App anywhere. In particular, the App can record the tongue images of the same person over time so that abnormal changes can be detected sensitively and timely. Hence, the proposed App may potentially benefit a broad population and reduce healthcare costs by preventive medicine.

What are the goals of the project?

Our long-term goal is a smartphone App that can be downloaded from an App store, and used by millions of people. The objective of this proposed project is to develop a fully functional tongue image analysis App—ITongue. This App will automatically analyze the tongue image to annotate person’s health status and provide advices through (1) image segmentation of the photo to isolate the tongue image, (2) feature retrieval of the tongue image, (3) comparison of the retrieved features with the annotated features in the database and medical annotations of health status, and (4) suggestion of hospital visits and lifestyle changes based on an expert system. Items (1) and (2) have been achieved so far. The funds to be raised will be used to support the development of items (3) and (4). For this purpose, we will conduct the following work:

(a) We will develop a knowledge base of annotated tongue images. We have a collection of tongue images and will collect more. We will collaborate with a professional TCM doctor, who is a visiting scholar at our university for the annotations. The annotations will include various TCM features, such as color, coating, shape, etc., and the health status associated with the person.

(b) We will develop an annotation system for health status based on the comparison between a new tongue photo and each of the annotated tongue images in the knowledge base. We will first develop a computational algorithm to compare tongue images from the TCM perspective. We will perform feature selections for the comparison and explore a distance measure that best reflects the medical relevance of the tongue image comparison. We will then develop a consensus-based method for assessing the health status based on the top hits of tongue image comparison.

(c) We will develop an expert system to advise the users based on the tongue image analysis and additional questionnaires. The advices include changing of life styles, avoiding or taking certain types of food, seeing a doctor (for severe health problems), etc.

(d) We will develop an application programming interface (API). The input the API is a tongue photo and the output is the annotation and medical advice. This API will be implemented as a cloud service, which can reduce the computing time into sub-second level.

As a token of appreciation, each donor will receive a beta version of the App for free.