Interactive Sketching of Urban Procedural Models – SIGGRAPH 2016
We present an interactive multi-stage stroke-based system that combines sketching and procedural modeling. Our system allows untrained users to create realistic buildings using a tablet or a mouse. Using a deep learning approach, we can find the grammar and the parameters in real time. This allows the user to sketch and edit the model interactively, and, since it is a grammar, we can create
a wide variability of 3D models from a single sketch. Normally, in procedural modeling, a grammar and a set of parameters are used to generate a 3D model. It has the advantage of the detail amplification, few lines of code can generate numerous examples. However, the parameters are hard to control and the complete grammars are tedious and complex to write. Our methods tries to hide all this complexity and manual tuning. The user can sketch few strokes and obtain a complete 3D model. We first run an offline training. Then, in
runtime, the user sketches the building interactively Instead of create training examples manually,
we use our procedural engine to generate images using a custom renderer. For each grammar snippet, we create hundreds
of samples varying their parameters. With the generated samples, we train two types
Convolutional Neural Networks: one for recognition of snippets and the second one for parameter
estimation. Then, the user sketches a building and our
system finds the grammar snippet and the parameters within milliseconds. Iteratively, the user
sketches the different components of the building, and our system continuously updates the grammar that generates the 3D model. We present two sketching sessions. Initially, the user wants to create an observatory.
First creates a dome giving details to the walls and cover
Adds an annex building with specific windows, Then sketches the entrance, with a big gate.
And concludes the observatory using copy. Then, the user decides he prefers to sketch
the Capitol. Instead of start from scratch, removes the
side domes and modifies the main entrance Sketching the proper windows and roofs.
Finally, add the main tower with few strokes and copying and resizing operations. As second sketch session, the user wants to
create a building inspired in the Willis Towers. First sketches the main block and sketches
the windows. Then sketches a tower, then the user can copy,
resize adding floors, and move it. This is possible since we are simply manipulating
a grammar. Then the user decides to modify it such to
create the Empire State. Removes the towers
Resize and center And repeats the process Here are some of our results. Since our system is based on grammars, we
can use one sketch to generate a wide range of variations that are inspired in the original
model. This is ideal to design complete cities where buildings have similar styles but present
different configurations. The user sketches a 3D shape and we compare
our system against the well-known MCMC algorithm commonly used in inverse procedural modeling. Our method finds a solution within few milliseconds,
in contrast, MCMC results has less accuracy and need significantly more time. We have a group of 8 participants use our
system as evaluation. We ask them to create two buildings that look similar to two target buildings. They successfully created them in a very short time. We also asked them to create some original buildings. Here some extra sequences The user can change the parameters directly in the grammar The user can modify the object type by drawing it again.