A teleoperation system based on virtual environment (VE) is an emergent technology for operating a robot in remote or hazardous environment. We have developed a VE-based teleoperation system for robot-arm manipulation in a simplified real world. The VE for manipulating the robot arm is constructed by measuring the 3D positions of the objects around the robot arm by motion-stereo method. The 3D position is estimated by using two-(calibration) planes method based on images captured by the CCD camera on the robot-arm, since the two-planes method does not need pin-hole-model assumption to the camera system. The precision of this 3D-measurement is evaluated through experiments and the derived is the theoretical model to the error in the measurement. This measurement system is applied to VE-based teleoperation experiment for Peg-in-hole practice by the robot arm.