Free viewpoint video technology, which provides a high sense of presence and immersion by allowing users to view live video from any viewpoint, is increasingly demanded in the entertainment field, as an application that provides new experiences for users. In this study, for the purpose of developing a system which enables immersion into a moving miniature world, a method to generate high-definition free viewpoint video for small-scale spaces in real time was realized. It can generate high-definition free viewpoint images by combining free viewpoint depth images applying noise reduction processing and the corresponding color images considering occlusion and the normal direction of the object’s surface. Consideration of occlusion and the normal direction of the object’s surface improved the quality of the generated images. In addition, the realized methods were able to generate clear free viewpoint images even when the free viewpoint approached the object.