Creating collages can help an artist connect with nostalgia and memories; a generative collage art program can make this act accessible to anyone.
🌲 Dartmouth College. 🤖 Artificial Intelligence 🖌 Digital Art.
For my Master's in Computer Science and Digital Arts, I combined my interests in artificial intelligence, fine art, and social impact to create a generative program that produced collage paintings based on user input.
Introduction
The power of collages come from their assets – such as photographs and paintings – and the manipulation of those assets on a canvas. The barriers to entry in creating collages, however, are significant; artists are bound by their skillset, resources, and time.
For my thesis, I created two semi-automated collage-making programs that will help users overcome these barriers to making complex and compelling collages. The collage painting program uses a genetic algorithm to generate collage paintings.
This program uses a genetic algorithm to give users agency and allow them to influence their final collage to their preferences. Over time, these generated collages become more rich, complex, and potentially more visually and emotionally compelling. My program helped people create art, have fun, and even learn and reminisce about time periods and places.
Why?
This project was inspired by a survey I conducted titled “What’s stopping us from doing what you love?”. This survey was launched right during the initial Coronavirus wave in America, so results were especially sensitive to activities that people could not do easily from their homes. The survey gathered 116 responses, mostly from students and young adults. Of those responses, 55% missed art-creation (encompassing fine arts, music, dancing, and other creative outlets) the most. Out of that subset, 41% could not participate in their art form because of lack of free time. The results of this survey made it clear that time constraints are a significant obstacle in people pursuing what they loved doing.
I had also recently finished my own art project, where I created collage paintings of as many childhood memories as I could. I immigrated to New Zealand when I was four, and in the chaos of not knowing English for a long time and not having much stability, I don't recall much of my past. I found, however, that studying niche pop culture and news events in the same area as where I grew up helped dozens of memories come back to me for the first time in a decade and a half. I knew I wanted to see if I could achieve the same for anyone else, minus the inaccessibility of creating intensive art.
So, a project was born: could I create a program that could help people create art? Could the process of creating that art help them connect with others? Spoiler alert: yes!
The Program
The user experience itself is very streamlined; a user first enters a bit of information about themselves. Once they tell us a decade and country, a search algorithm paired with a list of keywords I selected collected around 15 images for us to put in our asset toolbox. I also allowed users to bypass the decade/country system and upload their own images if they wanted to further customize their collage-making experience.
First images: Seed initialized!
Once a user completes the first two steps, we celebrate the birth of a new family of collages. They can pick their favorite collage of the generation and create another generation filled with four offspring from that selected parent collage. These child collages will have little mutations that make them similar to, yet different from, the style of their parent collage. This process can go on forever (or, more realistically, until I shut down the program this year 😔 ).
The Algorithm
I based my program on how I perceive and understand collage art. I first started by studying 18 collage artists, including Picasso, Rauschenberg, and Höch, and identified a number of effects their collages had in common, such as tinting images, creating a stamp-like effect with images, rotating assets, and changing asset transparency. Each collage type also has several common characteristics, although the values to those characteristics vary: focal point, number of images, the concentration of images, rotation, color palette, image style (photographs, paintings, or text), and whitespace. All traditional collages also have inherent characteristics, such as canvas texture, imperfections, and dimensionality that are difficult to replicate digitally. These archetypes inspired collage genes for the genetic algorithm: asset presence, asset location, primary color, palette scheme, edits matrix (masking, rotation, stamping, pointillism), textures, and brushes.
What's an asset?
I call any object or image on my digital canvas an asset. Asset presence represents whether an asset appears on a collage, and asset location places them on the collage. The primary color and color scheme together make up the color palette for a collage. The edits matrix determines how a collage is masked, rotated, stamped, and given the pointillism effect. Textures and brushes determine the canvas overlay, background, and brushstrokes.
Asset Locations
To mimic the different ways that collages could be organized, NetworkX, a complex network package in Python, was used to produce locations for image assets. Noting the asset locations on the collages I had studied (A), I found that their locations looked similar to random graphs (B).
Evolutionary Algorithm
I wrote the program mostly in PIL and OpenCV because of their handy headless server adaptability. Every time a parent collage created an offspring, I mutated the collage genes (which assets are present, where they're located, how they're colored and filtered, where they're placed, etc.) a little bit. Here's an example of an early generation creating the next.
Results
A total of 63 participants between the ages of 18 and 68 engaged in this survey from across the world; the project was shared publicly on social media and was passed around through word of mouth. People enjoyed the program, liked the aesthetics of their final collages, and significantly felt that the program could create better art than they could have. If a participant considered nostalgia as a key tenant in creating their collages, they are 1.33 times more likely to feel nostalgic when looking at their collages and felt a stronger connection to their generated image set, although this was not statistically significant. Finally, although people who didn't consider themselves to be artists were more likely to rate the aesthetics of their final collages higher than artists, a high p-value indicates that both artists and non-artists have similar expected values to how satisfied they are with their collages’ aesthetics (translation: everyone seemed to think that these collages looked good & considered them to be art).
When some users were interviewed after their collage-making experiences, some children of immigrants shared that they tried out this program with their parents. One user’s parents joyfully exclaimed when she recognized a historical area from her home country in a collage. The user and her parent then shared a conversation about that historical area. Small moments like these are authentic examples of how generative art and collage-making can bridge generational gaps and indicate that immigrant-specific instructions can encourage similar connections. Users also reported satisfaction in creating collages representing relationships, trips, art history, and personal experiences.
If you'd like to know more (I cut a ton out to hopefully make this project easier to read about), please reach out! I'd love to share my data, studies, and algorithms.
