This paper proposes a framework that supports the photorealistic superimposition of virtual objects onto real scenes obtained by free-viewpoint image generation, which enables users to freely change their viewpoints in virtualized real world constructed using preliminarily recorded images. The framework combines the offline rendering of virtual objects and the free-viewpoint image generation to take advantage of the higher quality of offline rendering without the additional computational cost of online processing; i.e., it incurs only the cost of the online free-viewpoint image generation, which is simplified by pre-generating structured viewpoints. Based on the proposed framework, we develop a practical application that superimposes lost buildings of a historical relic into a virtualized environment using omnidirectional images captured from the sky, thereby, allowing the users to change of their viewpoint on a two-dimensional 400m x 400m plane using viewpoints of 20m x 20m grid structure.