A real-time and automatic gesture classification method from dynamic image of upper half of the body is proposed, as a basic study to recognize naturally expressed gesture during conversation. The method consists of (1)feature extraction, (2)feature analysis and (3)classification, and a prototype system has been developed based on the method, followed by the system evaluation by subject experiment. Comparing the classification by the system with that by human subjects, most of gestures were classified appropriately in real time, however, there were a few gestures which could not be segmented properly, especially when the gesture has only small head movement such as light nodding.