The SGMSD-Net system organizes the creation of immersive scenes in three successive stages and surpasses previous models in visual quality and semantic coherence
An international team of researchers developed an artificial intelligence model capable of creating immersive panoramic images for virtual tourism from simple text descriptions. The system, called SGMSD-Net, produces complete high-resolution scenes with greater semantic accuracy and visual detail than previous approaches.
The research, led by Lun Wei, a professor at Ma’anshan University in the Chinese province of Anhui, was published in the International Journal of Information Quality. The work aims to solve a common problem in current generative models: the tendency to generate visually similar images without accurately capturing the meaning of the requested scene.
To overcome this limitation, the model combines explicit semantic guidance with a diffusion process structured across multiple resolution scales. This achieves greater coherence between the input text and the final image.
The operation is organized into three successive phases that range from global design to texture detail. In the first stage, the system defines the overall spatial distribution and colors of the scene. In the second, it refines the edges of objects and main structures. In the third, it restores textures, lighting patterns, and high-frequency details.
How the semantic guide works
This scheme allows each level of the process to focus on a specific type of visual information, rather than trying to optimize all aspects of a high-resolution image in a single pass. The semantic component operates through an encoding module that converts textual labels —such as scene category, type of monument, or weather condition— into numerical vectors.
These vectors generate modulation parameters that act directly on the intermediate layers of the neural network through a mechanism called FiLM. This way, the model adjusts the scale and offset of visual features according to the meaning of the received description.
Tests were conducted with two datasets: SUN360, which includes over 26,000 panoramic images of interiors and exteriors, and Matterport3D, with over 10,000 viewpoints of 90 real architectural environments. In both cases, SGMSD-Net outperformed four benchmark models —Pix2PixHD, SPADE, DDPM, and Stable Diffusion— in the FID, LPIPS, and CLIP Score indicators.
These indicators measure the distribution similarity with real images, perceptual consistency, and the degree of alignment between image and description. The model achieved an average score of 9 out of 10 in the subjective evaluation conducted by ten volunteers, who rated clarity, realism, and semantic coherence.
Results and possible applications
In the Semantic Attribute Accuracy indicator, the system achieved an average of 92.9 percent, surpassing the second-best evaluated model by 5.9 percentage points. An ablation analysis confirmed that the three-stage architecture is superior to simpler variants: it reduces the FID by about 21 percent compared to the single-stage model and improves the CLIP Score by between 2.4 and 2.8 percent.
Regarding computational cost, the system requires 356.2 million parameters and takes 10.3 seconds per image. The GPU memory consumption is 13.9 gigabytes, lower than the 15.8 required by Stable Diffusion. The authors note applications in cultural heritage, museums, and urban planning.
Potential uses include producing virtual tours of tourist destinations before travel, educational visualization of hard-to-access historical sites, and generating content for exhibitions. The possibility of integrating images with virtual and augmented reality systems is also mentioned.
The team acknowledges limitations in the geographical and cultural coverage of the datasets used, as well as performance under extreme lighting conditions or very complex spatial structures. The model also relies on predefined semantic labels, which reduces its ability to interpret free or ambiguous descriptions. As next steps, the researchers propose integrating larger-scale multimodal models and mechanisms that combine attribute guidance with natural language interaction.
