Artificial Intelligence

Weekly update Monday, February 4

February 20, 2019 by rememberlenny

Weekly update February 4, 2019

I will be doing my best to send a bi-weekly update on the progress around my efforts to build out a street art genealogy online, and create a tool for preserving otherwise undocumented street art.

So far in 2019, I have many exciting updates:

* Open-sourced tools for detecting street art in images using machine learning[¹]
* Published a 4000+ word article on how to train a convolutional neural network (CNN) to recognize street art in location tagged photos online.[²]
* Released a dataset of 6000 street art images and non-street art New York City images for CNN training.[³]
* Met with authors of three street art discovery/preservation apps [⁴]
* Presented the Public Art project and model training process at BetaWorks

I have been focusing on three major parts of this project: image collection, data analysis, and presentation.

For the image collection, I have continued to use Instagram scraping as the primary source for new images. Currently, this method has been effective for quickly gathering data to train deep learning models, but does not offer a long term solution for image aggregation. I have noticed a few times already that the primary methods for image collection have been shut down. Although I know this to be the case, I am able to gather hundreds of thousands of newly uploaded images a week, which is incomparable to any alternative user generated methods.

For data analysis, I have been analyzing images with associated location metadata by training deep learning models around artists and street art types (stencils, murals, letterform). I have also spent a lot of time with python notebooks, trying to find trends in certain periods of scraped images. I have been experimenting with “hot spot” detection based on images photographed in an specific area in a small amount of time. For example, detecting when new images are found from multiple people within a smaller frequency than previously found.

Finally, for presentation, I have been working on two methods: website and email newsletter. For the website, I have fortunately been able to quickly build out a web interface for loading and navigating images, but do not feel that the current methods fulfill my original intention of the project. As a result, I have not publically released any updates on this front.

For the newsletter, I have created a set of tools to determine if “new” street art is discovered in a place. Currently, I manually run 4 of python scripts to based on monitoring new images found from certain locations. I am working on establishing a steady stream of images I can monitor, to generate a weekly newsletter of “the best local street art” for respective interested subscribers.

I have been recently encouraged to consider the larger vision around the Public Art project. I am building out a steady infrastructure for housing and collecting street art, but do not had a plan for attracting an active audience. With the analogy of building a city, I am building a beautiful city with few inhabitants, but could develop a blossoming city as populous as Tokyo. Based on this, I will be consolidating my efforts.

I would appreciate thoughts around whether or not to build a healthy business around the audience interested in street art, or to follow a non-profit route. When considering the business route, I can clearly see a productization of the collected images with a high margin art, such as printed posters. The sales model around street art products offers the opportunity for driving paid traffic to the website, which would also generate traffic that would lead to user-generated image contributions. If I pursue the non-profit route, I will not have the luxury of buying growth.

Please send your thoughts to [email protected]<mailto:[email protected]>

[1]: https://github.com/rememberlenny/streetart-notstreetart

[2]: https://blog.floydhub.com/instagram-street-art/

[3]: https://www.floydhub.com/rememberlenny/datasets/streetart-notstreetart/3

[4]: https://www.canvsart.com/ & https://artpigeon.nyc/

[5]: https://betaworks-studios.com

How I Used Machine Learning to Inspire Physical Paintings

July 11, 2017 by rememberlenny

Since I was 15 years old, I have been painting graffiti under bridges and in abandoned buildings. I grew up in San Francisco when street art was booming, and inspired by the colors and aesthetic, I looked for ways to create art and taught myself to paint. As I got older, I discovered the graffiti communities on Flickr, and began making an effort to meet artists where I lived and share photos of my work online. As Tumblr grew in popularity, the community moved. Then Instagram emerged, and the community moved again.

“Gift”, Photo collection from 2010–2012. All photos taken and painted by author.

In recent years, I haven’t had the same leeway to paint in public. There was a greater cultural acceptance of street art when I lived abroad. Painting on walls was seen as beautification in areas where there was much demolition. When I moved back to the US, I started painting on larger canvases, and eventually moved toward spray cans and paint brushes.

Kawan’s “Sunset Running” project. Courtesy of Kawandeep Virdee.

Inspired by a project by Kawandeep Virdee, I photoshopped the paintings with motion blur filters, and modified the lighting effects. The result was a creative jumping-off point, enabling me to create a digitally inspired physical painting.

Last year, I started experimenting even more with digitally manipulated images, and their role in inspiring physical paintings. I began creating aesthetically beautiful images by taking classic paintings from the 18th and 19th century and running various photoshop filters over them. I found the color and contrast from these old paintings to be unmatched and beautiful.

Process for turning classic paintings into beautiful color muses.

I took the digital pieces I created and used them as the inspiration for painting new pieces by the classical paintings on a computer and then physically painting the remixed image.

The Ninth Wave hanging on my wall. Photo by author.

I continued my interest in graffiti, again using the digital space as a canvas, and spent a few months building out various software tools that I thought would be useful for graffiti artists. After creating such a large library of literally millions of paintings, I realized I wanted to do something more than just browse the images, so I started exploring different techniques around machine learning.

Painting based on Ray Collin’s Seascape series painting after digitally manipulating the photo. Photo by author via RememberLenny

I started teaching myself about the application of neural networks to do something called “style transfer,” which refers to the process of analyzing two images for the qualities that make the picture recognizable, then applying those qualities to another picture. This meant that I could replicate an image’s color, shapes, contrast, and various other features onto another. The most commonly recognized style transfer application is from Van Gogh’s “Starry Night” to any photograph.

Example from a GitHub repository that implements the Artistic Style Transfer algorithm using Torch. Credit: jcjohnson

Similar to my previous project of painting the digital sunset images, I processed pictures using the artistic style transfer algorithm and then painted them. Referring to the plethora of graffiti images I’d already collected, I used images of nature and processed them in the style of street art I thought looked interesting. The end result was an aesthetically interesting image I couldn’t imagine creating from scratch.

Process of creating the Artistic Style Transfer images.

It’s been a few months since I’ve done anything with this technique of mixing images and painting them. I hope the process depicted above can be a source of inspiration for other programmer-painters who enjoy mixing both practices.

Final version of the digitally inspired painting. Photo by author.

Below are a few examples of what an artist can create by combining street art images with photographs.

Thanks to Edwin Morris for the grammatical review and Lam Thuy Vo for the ideas.