Generating 3D indoor environments with consistent styles

• Κύριος συγγραφέας: Πασχαλίδης, Γεώργιος
• Άλλοι συγγραφείς: Paschalidis, Georgios
• Γλώσσα: English
• Έκδοση: 2022
• Θέματα: Scene generation; Transformers; Autoregressive style

In the current Master thesis, we address the scene generation task, with our main focus being on indoor scenes synthesis. Existing approaches pose the scene generation task as a layout creation problem. Namely the task is to populate a scene with a set of labelled bounding boxes that correspond to a set of furniture pieces. In particular, these methods typically seek to learn a probability distribution over a set of attributes that define each object such as their shape, category, orientation and position in the scene. During generation, the generated bounding boxes are replaced with 3D objects by retrieving them from a library of assets based on various criteria such as size, object category etc. Naturally, since the object retrieval process is independent from the layout generation there are no guarantees that the generated objects will be coherent in terms of style and appearance. To this end, in this work, we propose a novel generative model for indoor scenes that takes into consideration the per-object style during the generation process. We believe that this is a crucial step towards generating realistic environments. In particular, we build on top of ATISS [18], which is the state-of-the-art indoor scene generation pipeline. Specifically, we extend its capabilities by also incorporating a style prediction module. Further- more, we also propose a novel retrieval procedure that instead of simply relying on the size to replace bounding boxes with 3D models, takes into account the per-object style. Our experimental evaluation showcases that our model consistently generates stylistically meaningful scenes, (i.e. the nightstands next two a bed or the chairs around the table should have similar appearance), while performing on par with ATISS wrt. the scene generation quality. Finally, we also introduce various metrics that can be used for evaluating the generated scenes in terms of the style coherence.