2020
Cité des Sciences et de l’Industrie – Universcience
Temporal Exhibition consisting of
multiple interactive exhibits
The Ai Software, we developed with Unity, is is running on multiple iPads aranged around a physical model of Montparnasse Tower and it’s environment. The tablets were hanging from the ceiling, but freely movable, so the viewer can focus on different points of interest.
We used the iPad cameras to track the physical model and determin the position of the user’s iPad relative to the real model by matching the video to a point cloud we had taken of our model. We then used that life video as first layer of our AR-application.
For tracking the model we needed a high contrast, but when using the app, we wanted to dull the backround a bit to lead the view to the important foreground. So we rendered a virtual 3d copy of the same model life as a semitransparent glowy blueish layer on top of the video as a virtual backdrop and to says – the magic begins here.
The focus of the app was showing, why the Montparnasse tower was a technological masterpiece when it was constructed in 1973. Yet the current state is a dark and dull tower and it is planned to be overhauled to become a more pleasent view in the future. Our main augmentation layer was build up by life rendered virtual 3d models that show the construction in the 1970s, the present state and planned the planned reconstructio of the tower. These highly complex and animated models were rendered in realtime on top of the background – or rather into the backround for performance.
To give detailed information on what the user is shown, interface elements hover in the virtual 3d space close to the sections of the tower they relate to. So users can pick their information of interest. These interface elements appeared in accordance to what state or section of the tower was currently shown, to not overwhelm the user.
On top of everything else there was a 2d interface, that stayed fixed. It contained the most iportant (main category) interaction elements, used to toggle between the different states and sections of the tower: Construction under ground, construction of the tower, current state and planned renovation. This way we wanted to show the users what is inportant, and what they are just looking at.
When it comes to Virtual Reality a major problem is, that a single Computer needs to render 3d objekts at 240fps (frames per second) 120fps for each eye in high resolution. This high frame rate is neccesairy to prevent motion-sickness.
Augmented Reality apps can easily run at 30 fps, but they often need to render on the teeny tiny internal graphics chip of a tablet or phone.
In comparisson, some tripple-A video games run at 30fps for a single screen, on huge PC graphics card.
Todays motion design departments use so called render farms – multiple high-end graphics cards, working together – which is not aplicable for single device aplications like AR or VR – and still sometimes rendering a single frame (image) may take minutes, because they are rendered into videos. So time is not so much an issue.
And motion designers think in Cinema-4d’s complex rendering options and multiple layers with blend-modes in After Effects or even pixel perfect text fields in Illustrator.
Unreal Engine or Unity 3d however needs you to think in polygons inside a moving 3d space and effects achieved by clever shader programming, which is a slightly different language and needs a much higher precision in preparation of 3d models and 2d assets.
I do know Cinema-4d and all of Adobe‘s main products, so one of my main tasks, when developing a VR or AR application in cooperation with motion designers is to translate their usually high expectations into doable Unity tasks and (often repeatedly) ask for precise 2d and 3d assets to lower the gigantic resource overhead for Unity 3d or Unreal.
Ignore it, put the sun at noon on the day of a soltice – 90 degrees down, which would look dul or put the sun into a different ‘sweet-spot’ for each of the four tablets, depending on their quadrant.
Finally we agreed on having it rotate, and by that making the scene more interseting and illustrating a big problem of the Montparnasse tower: It’s taking the sun for a vast area around.
