Reverie Studio
What does photo metadata mean to us? A mind-wandering journey that let people explore new possibilities by questioning the world around them.
Reverie Studio is a way of discovering and documenting one's creative thinking process visually. Reverie Studio encourages people to think out of the box and be curious about their surroundings through the lens of artificial intelligence. It strengthens the connection between people and the surroundings by placing the content in physical environment using augmented reality technology. Reverie Studio is a social platform where people can share their thinking process as well as viewing others'.
There is no existing online image platforms has utilised artificial intelligence from the cultural aspect, which means retrieving information based on the concept behind a picture rather than the literal object recognition, colour and composition alone. Even though it is still a long way until a computer can have the same performance as a human brain, research has shown that the computer can sometimes be better than humans at spotting patterns from metadata. It is then up to people to interpret those patterns.
The role of AI is to be provocative and stimulate people to think differently. It is like conducting a brainstorm session with a machine. In the future, a search engine will not wait for a keyword, it will suggest people with its own thought. The industry is working on algorithms that promote expressivity in a machine. The evolution of AI has gone from being intelligent to be as natural as a person and now move on to expressivity, which allows the computer to express its interpretation of an object, an event.
There is no existing online image platforms has utilised artificial intelligence from the cultural aspect, which means retrieving information based on the concept behind a picture rather than the literal object recognition, colour and composition alone. Even though it is still a long way until a computer can have the same performance as a human brain, research has shown that the computer can sometimes be better than humans at spotting patterns from metadata. It is then up to people to interpret those patterns.
The role of AI is to be provocative and stimulate people to think differently. It is like conducting a brainstorm session with a machine. In the future, a search engine will not wait for a keyword, it will suggest people with its own thought. The industry is working on algorithms that promote expressivity in a machine. The evolution of AI has gone from being intelligent to be as natural as a person and now move on to expressivity, which allows the computer to express its interpretation of an object, an event.
An Exploration of Machine Creativity
Before we can make a creative machine, we need to understand how creativity works in humans. A research conducted by Dr. Naama Mayseless, along with Prof. Simone Shamay-Tsoory from the Department of Psychology and Dr. Ayelet Earn from the Rambam Medical Center shows that in order to come up with a novel idea, we need both random uninhibited association as well as the ability to monitor our own ideas rationally based on the context or rules around them. The results showed two very different regions of the brain working together when subjects gave highly creative answers. The two regions are the “associative” region and the administrative control region (left-occipital temporal region), a more “conservative” region related to social norms and rules. The associative region includes the anterior medial brain areas (medial prefrontal cortex, posterior cingulate cortex, and ventral anterior cingulate cortex), which works in the background when a person is not actively concentrating, such as daydreaming.
Inspired by this research, the theory behind Reverie Studio's algorithm is to allow the machine to find associations based on multiple inputs, colour, hashtag, caption, and objects identified in the picture. It is to compare both pixel and frequency of a picture. Pixel means the colour, shapes, and content of an image which are the substance in the picture, whereas frequency is looking at the full visual, finding out the probability of two images show up together - adding hashtag and caption serve this purpose. It is like building context and memory for the machine, thus it can daydream around these keywords and concepts.
Inspired by this research, the theory behind Reverie Studio's algorithm is to allow the machine to find associations based on multiple inputs, colour, hashtag, caption, and objects identified in the picture. It is to compare both pixel and frequency of a picture. Pixel means the colour, shapes, and content of an image which are the substance in the picture, whereas frequency is looking at the full visual, finding out the probability of two images show up together - adding hashtag and caption serve this purpose. It is like building context and memory for the machine, thus it can daydream around these keywords and concepts.
System scheme
Bridging The Digital and Physical World
In order to be creative it is crucial to have inspirational experience. It is an out-of-comfort-zone experience with positive affects. As we get a lot more stimulation through different senses while interacting with the physical world it is important that Reverie Studio does not only provide a flat experience through a screen. It motivates us to go out and feel our surroundings. The question here is how to present the image search results.
Storytelling is a compelling way to inspire people, and collage is a common and efficient way for artists and designers to show their ideas. The way Reverie Studio presents the image search result is to create a 3D collage. Users can move the items in the collage to create a scene. Using pre-trained fully convolutional networks (FCNs), computer can crop out the target items in an image, and put those items in a 3D environment that can be viewed by an AR device. To identify the coordination of an item in the 3D space, a supervised learning model is trained in Reverie Studio. Each item is classified to either universe, sky, above the ground, ground level, or underground.
Storytelling is a compelling way to inspire people, and collage is a common and efficient way for artists and designers to show their ideas. The way Reverie Studio presents the image search result is to create a 3D collage. Users can move the items in the collage to create a scene. Using pre-trained fully convolutional networks (FCNs), computer can crop out the target items in an image, and put those items in a 3D environment that can be viewed by an AR device. To identify the coordination of an item in the 3D space, a supervised learning model is trained in Reverie Studio. Each item is classified to either universe, sky, above the ground, ground level, or underground.
|
|
|
Demo
|
|
|
Prototype
Discussion
The digitised world coexists with the analogue world enabling an understanding of everyday life with a viewpoint that goes beyond human perception alone. The technology is always advancing; however, the images are always the result of the way that each person, be it artist, photographer or social media phone camera using teenagers use it. Photographs will always be objects of an anthropological survey. There will be more ways for people to analyse an image and leave more traces for the laters to understand us. The future vision of the project is digital humanities. We shape the culture; all our data contributes to the collective identity of our culture. Imagining in the future when the artificial intelligence algorithm is matured and fully trained, can it be able to predict the lost knowledge, the information that people fail to preserve?