reckless glitch design – benjamin kiesewetter

experimenta - abstract your mind

Mixed Feelings

Intro

For the new experimenta Science Center in Heilbronn Germany, TRIAD filled 800 sqare meters with about 40 interactive exhibits.

 

I worked on quite a few of their basic concepts, designs and scientific background and about three of them became mainly my babies from concept over graphics to programming.

 

Two of them have some basic simularities, Luftparcours, the paraglider simulator and Smartphone Zombie, a simulation of a walk through a city while using a smartphone. Both use a similiar comic style and are programmed with the Unity 3d gaming editor. While working for TRIAD I prepared the graphical assets to be used by an at that time unknown programmer, since the programming was to be done by Kurt Huettinger, a company for planing and manufacturing exhibitions and interactive exhibits. Some months later I happened to become that programmer and got my own work back on the table.

Year

2017-2019

Customer
TRIAD Berlin,
Kurt Hüttinger GmbH & Co. KG,
experimenta gGmbH
What
  • visualize emotions by using abstract sliders
  • Unity 3d game dev environment
  • raymarching shaders
My Part
  • Research | first Storyboard
  • Game mechanics | Interaction
  • Software Development in Unity 3d
  • interpreting sliders into dynamically generated and altered shaders
screen captures of different parameter settings

slide your feelings

The name comes from the German idiomatic expression gemischte Gefühle haben – to have ambivalent feelings and thus to be not sure or even concerned about something by intuition.

 

The idea was to take random words like autumn, birthday or homework and have the users make abstract visualisation of their feelings towards it by pulling some sliders on a board as known from audio mixing tables.

 

In the end there were 14 sliders agreed on. Brightness, saturation contrast and colour for each, background and foreground. The trick was to have them also influence other parameters. For example contrast will also influence crispness and blur, while brightness can overshoot and make certain areas burn out without totally loosing contrast, etc.

The last seven sliders – fragmentation, splitting, pattern, movement, agregate state, variance and transparency – will do the main magic of morphing the shapes of the foreground objects.

exp_gemischte_gefuehle_slidersketch_Asset 22
exp_gemischte_gefuehle_slidersketch_Asset 23
exp_gemischte_gefuehle_slidersketch_Asset 24
exp_gemischte_gefuehle_slidersketch_Asset 25
exp_gemischte_gefuehle_slidersketch_Asset 26
fragmentation
  • Change of basic shapes
  • smooth | potato shaped | cratered | porous | scattered
exp_gemischte_gefuehle_slidersketch_Asset 28
exp_gemischte_gefuehle_slidersketch_Asset 29
exp_gemischte_gefuehle_slidersketch_Asset 30
exp_gemischte_gefuehle_slidersketch_Asset 31
splitting
  • splitting of foregrund
  • single conneted continuum | cluster of objects | multiple separate Objects | multiple clusters
exp_gemischte_gefuehle_slidersketch_Asset 7
exp_gemischte_gefuehle_slidersketch_Asset 9
pattern
  • from chaos to regular different patterns
  • also changes regularity of movement
exp_gemischte_gefuehle_slidersketch_Asset 10
exp_gemischte_gefuehle_slidersketch_Asset 11
movement
  • blending of different patterns of movements
  • areas of zero movement in between
exp_gemischte_gefuehle_slidersketch_Asset 13
exp_gemischte_gefuehle_slidersketch_Asset 14
exp_gemischte_gefuehle_slidersketch_Asset 15
aggregate state
  • blending of different patterns of movements
  • areas of zero movement in between
  • also changes patterns an speed of movement
exp_gemischte_gefuehle_slidersketch_Asset 18
variance
  • changes all other parameters
  • random and different in for different areas
transparency
  • blending foreground into background
  • density of foreground
life demo of different silders-setting

'nobody' understands it

A dear college at TRIAD, Claudia Baumgartner, made the concept for Gemischte Gefühle. That was shortly before she packed some stuff on a bicycle and left to cycle around the world. How cool is that? In her absence I got to make the storyboard and do-ability research. I liked the concept a lot and tried to defend her ideas as good as possible against all attempts of simplifying the exhibit.

One important part was not be too clear about what each slider will do while still maintaining reproducibility. That way the user would be forced to use intuition instead of his conscious mind. While Claudia was somewhere on her bike in Middle East, I had to write some storyboard actually naming these abstract sliders and explaining what mumbo-jumbo they were meant to do on the screen although I was not totally sure yet, what would be possible, or how. I made some shader experiments.  My main conclusion was: Give the software developers freedom and trust and powerful hardware. And finally I merely crossed my fingers writing it down. 

Part of the concept was to compare the resulting impressions anonymously to other’s visualized feelings. The comparison gallery was given up at some point while I was difficult to reach – riding on a 100$ scooter through South East Asia for some weeks. (You may get the influence here.)

Additionally, one slider, Variance, had been changed to Transparency. “Benjamin, nobody understood it.” I would be told later but it simply looked like only a few people in the chain of approving between the designers, chefs, and the customer’s employees simply hadn’t understood that slider at first glance which made some ripples in the chain of communication back that lead into the designers being asked to change it. At that time I was busy collecting fallen off motorcycle parts somewhere in Myanmar. So I wasn’t there to explain that not understanding the sliders immediately was the whole point.

A few months later – I was freelancing by then – I took over the software development of this project for Hüttinger and got back my own storyboard. Thanks to a tip by Johanna Noltmann, the currently responsible designer back at TRIAD, I tried out a technique called ray-marching. I used a framework to build complex shader codes to morph  shapes dynamically and made these codes more efficient by hand afterwards. (Let’s not talk about the hardware decisions) Finally I got it working  much faster than I had anticipated. But it was only after I had finished that I got told about the new Transparency slider. It  already had been CNC made at that point. (carved into stone you could say)

I did not like it – the concept was abstraction and Transparency simply was not. Not looking at the extra work this was in my eyes swapping out something good and clever against something totally trivial. So there was an intense and fruitful conversation on how to map what I deemed necessary onto what had been approved, a workaround but a good one I hope.

By now what I understand of the usage of the exhibit, it did not work out as well as others. Many kids again are said not to understand it.  I know kids to be smart and fast with computer stuff but maybe this is a bit too abstract to get the idea fast enough. On the other hand there have been many bargains during the process of production and the project was stripped down quite a bit and maybe lost too much. I still love Claudia’s idea and I am fond of the shaderwork.

... technically - not even I totally do

Shaders did the magic trick in the end.

Now shaders are code snippets running on the graphics chips (GPU) and tend to be short but sometimes complex in structure. Shaders are a bit more abstract than the main code of most games and I tend to joke about shader programmers to be of a different species, such as theoretical physicist (I studied physics, I know what I am joking about)

Well this shader’s programmer actually was – or rather wasn’t of any species at all: The shader was coded procedural, meaning by a program. (No I wouldn’t call it artificial intelligence just to impress the search engines.) It was merely somebody else’s piece of code that I integrated into my code which I programmed to program the shader-programming code to code the shader and after recode that coded shader (my program) because while brilliant in most parts the implemented shader programmer produced some small but persistent bug in the shader and the shader needed to be more efficient to compensate for the under powered GPU anyway. Does that make sense?

No? Welcome to this project. And actually it does! 😉

Want to see what I am talking about? Here is the little monster:


I hope you're impressed, cause I am.

Somehow this device feels dumped and forgotten - hen or egg?