Aesthetic Attributes of Sky for Human Emotions
 
Yang Shen

Motivated by an observation that sky has fundamental emotional impact on human beings, I started this project to explore aesthetic attributes of sky that are connected to human emotions. In this project, aesthetic attributes of sky are inspired from real-life experiences, photographies, paintings, and animation movies, which I believe provide authentic connections with human emotions. They are simulated with computational attributes of real-time algorithms of sky rendering. An emotion exploration mechanism is devised to use aesthetic attributes to find sky styles of static outdoor 3D scenes for an emotion. A user study is proposed to evaluate aesthetic attributes with the emotion-exploration mechanism (are the aesthetic attributes authenticate to find a desired sky style for an emotion/feeling/mood?). The emotion-exploration mechanism could be used to enable the use of interactive configuration of aesthetic attributes of sky to create a more compelling emotion-exploration experience through sky.

Uses: Nonphotorealistic rendering is capable of broadening our ability to communicate thoughts, emotions, and feelings through computers. Artists have learned that by making images look less photorealistic they enable audiences to feel more immersed in a story (reference: [McCl93] Scott McCloud. Understanding Comics. Harper Collins Publishers, New York. 1993). The primary goal of this poster is to enable the use of interactive configuration of aesthetic attributes of sky to create a more compelling emotion-exploration experience through sky.

Assumed Aesthetic Attributes

  • Sky color - can be changed to unrealistic color, as inspired from animations

  • Daylight brightness - can be tuned to express mood, as it's natural that we feel happy in a sunny day, and feel blue in a gloomy day

  • Sun direction - can be tuned to express mood, as it's natural that we have different feelings when the sun is in different positions during the day

  • Sky layers - TBD (thought: mix of a couple layers of colors look more interesting than a single layer of color. TODO: check out the beauty or theory behind layering in painting.)

  • Cloud pattern - can be used to create infinite patterns, as it's nature is changing, which create space for creating more patterns

  • Granularity - brings a sense of liveness for film because liveness is a kind of beauty in film. TODO: check for the reason artists love granularity or noise

Evidence of assuming sky color, daylight brightness, and sun direction: we feel happier in a sunny day, and feel blue in a gloomy rainy day. The intuitive factors related to our feeling are sky color, and daylight brightness. Since sky color is majorly controlled by sun direction, I take sun direction as a candidate of aesthetic attribute of sky. In addition, inspired by animation movies (Your Name), games (Journey https://www.amazon.com/Art-Journey-Matthew-Nava/dp/0985902213), we take sky color itself as an aesthetic attribute to explore its emotional impact when sky color is changed to unrealistic.

Evidence of using Perlin Noise for cloud pattern: the continuity and uniformity of the random distribution of Perlin Noise make it perfect for simulating organic look in nature, such as clouds, fires, etc (TODO: reference "An Image Synthesizer" from Ken Perlin. it's the original paper proposed Perlin Noise.).. Specifically, Perlin Noise captures the beauty of randomness in nature. Essentially, it reproduces randomness that are neither too random nor too regular, which preoccupies humanity. Humanity is preoccupied by such randomness because "Studies indicate that people prefer recurring patterns that are neither too random nor too regular. Of particular interest is the possibility that humanity's preoccupation with fractals may be linked to survival more than aesthetics. On an African savannah, by tuning into fractal dimensions, people could tell if the tall grass was being ruffled simply by the wind or by a predator" (reference: http://www.jackson-pollock.com/jackson-pollock-drip.html)

Evidence of using Granularity as part of the style: granularity gives the impression of imperfection, which brings liveness for film. Liveness in film is a kind of beauty. TODO: find reference to prove that artists find beauty from granularity because it brings liveness.

Simulate Aesthetic Attributes

Reason the motivation of picking Rayleigh and Mie scattering - for its interactive performance and physically-based. Rayleigh and Mie Scatterings satisfy our demonstration on sky color, sun direction, daylight brightness, and brings interactive performance to allow user configure the scene interactively. We select it for the physical characteristics because human imagination arises from reality. We change Rayleigh Scatterings to maintain certain physical characteristics of sky to make sure it's recognizable when other aspects of sky are changed to unreal. (TODO: reason why picking Rayleigh Scattering to simulate aesthetic attributes. two points: a) interactive performance. b) physically-based characteristics meet two criteria: b.1) maintain partial physical characteristics to make it recognizable; b.2) change other characteristics to be potentially unreal to explore emotion) Didn't use other sky rendering techniques because it's time-consuming yet a waste of computational power for unnecessary quality of sky besides of our demonstrations of assumed aesthetic attributes (TODO: verify if there're other high-quality yet low-performance sky rendering techniques. If yes, list and reference them). (EXAMPLE FOR WRITING: Section 2.1 Fluid Simulator of Sangwon Lee's Simulating and Analyzing Jackson Pollock's Paintings published in Journal of Mathematics and Arts at June 2007)

Reason the motivation of picking Improved Perlin Noise - TODO

Reason the motivation of picking pseudorandom number generator to turbulent view direction to produce granularity:

A) a lot unreal effect from games add blurry features. we think it's because when our conscious is ambiguous, our visual perception doesn't accurately perceive reality in corresponding to the emotion of ambiguity. Specifically, our view directions turbulent. So we simulate that face of tabulated view with Pseudorandom number generator. (TODO: reason why it makes sense to use pseudorandom number generator to simulate human perception turbulent) That's the essential connection to granularity, the ambiguity within which produces curiosity compares with clear and accurate perception, which produces less sense of curiosity. Example of games and animations include: a) Alex Evans's 2015 Siggraph course: Learning from Failure: a Survey of Promising, Unconventional and 
Mostly Abandoned Renderers for ‘Dreams PS4’, a Geometrically Dense, Painterly UGC Game’. url: http://advances.realtimerendering.com/s2015/mmalex_siggraph2015_hires_final.pdf b) TODO: find concept art to demonstrate the connection of emotion and granularity/sand effect).

B) easy computation compared with extra memory for noise texture or lookup table.

  • Rayleigh and Mie scattering - simulating physically-based sky, configuring sun direction, configuring sky color, daylight brightness, and adding up sky layers

  • Improved Perlin Noise - configuring sky pattern

  • Pseudorandom number generator - configuring granularity

Emotion-Exploration Mechanism

Figure 2 shows the mechanism of emotion exploration with aesthetic attributes of sky. Each aesthetic attribute provides a spectrum of sky styles corresponding to an emotion. By combining the identified aesthetic attributes, the system provides a large spectrum of sky styles to serve as rich resource of emotion exploration.

[DISCARDED] User Study: Identify Aesthetic Attributes Connected To Human Emotions

If a scene satisfies all metric from user study, aesthetic attributes used to render the scene is identified as connected to human emotions.

Intuitive metric:

  • If the sky is recognizable

  • If an emotional difference is felt

 

Methodology:

  • For individual aesthetic attribute, render scenes with different values of the aesthetic attribute, then ask users

  • For a group of aesthetic attributes, render results with different values of each aesthetic attributes, then ask users

User Study 1: Evaluate Aesthetic Attributes with Emotion-Exploration Mechanism

Set up user experience to allow user explore emotions by configuring identified aesthetic attributes of sky. User can do:

  • Replace the 3D scene

  • Configure values of aesthetic attributes of sky

  • [Default] change camera view

 

Gather user feedback with the following metric:

  • Are the aesthetic attributes sufficient to produce desired sky styles for a feeling/mood/emotion?

NOTE: reference for writing this section:

  • Maria Shugrina, Margrit Betke, John Collomoss. Empathic Painting: Interactive Stylization Through Observed Emotional State

User Study 2: Evaluate Emotion-Exploration Mechanism

Set up user experience to allow user evaluate the performance of the emotion-exploration mechanism. User can do:

  • Configure aesthetic attribute

  • [Default] change camera view

Gather user feedback with the following metric:

  • Is the mechanism fast enough to respond to changes of aesthetic attributes?

Conclusion

Assumption: the emotional impact grew bigger with more aesthetic attributes evolved, and when the value of each aesthetic attribute is way different from its value in reality.

Improve

What could be improved at this project?

Note

Writing

  • be as specific as possible for explanations

  • be as clear as possible to explain details of the technology

  • reason why picking the technologies to simulate sky effect: why Rayleigh Scattering? why Improved Perlin Noise? why pseudorandom number generator?

  • throughly investigate the features of any technologies used in the project (for example, The nature of Perlin Noise is simulating organic look in nature, such as clouds)

Reference

  • validate the correctness and standard of each reference. i.e. if it's an url to video, what's the valid format of making video

  • take the feedback of last year's Expressive (2017) submission

Figure and Example

  • as clear as possible

  • as beautiful as possible

Compare with Related Work

  • [Figure 1] Maria Shugrina, Margrit Betke, John Collomoss. Empathic Painting: Interactive Stylization Through Observed Emotional State

Figure List
[Figure 1] Compare Dopamine with Empathic Painting: Interactive Stylization Through Observed Emotional State from Maria Shugrina, etc
[Figure 2] Mechanism of Emotion Exploration with Aesthetic Attributes of Sky