• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Remember Lenny

Writing online

  • Portfolio
  • Email
  • Twitter
  • LinkedIn
  • Github
You are here: Home / Archives for Spatial Analysis

Spatial Analysis

Two months exploring deep learning and computer vision

December 28, 2017 by rememberlenny

Repost from Medium

I decided to develop familiarity with computer vision and machine learning techniques. As a web developer, I found this growing sphere exciting, but did not have any contextual experience working with these technologies. I am embarking on a two year journey to explore this field. If you haven’t read it already, you can see Part 1 here: From webdev to computer vision and geo.

I️ ended up getting myself moving by exploring any opportunity I️ had to excite myself with learning. I wasn’t initially stuck on studying about machine learning, but I wanted to get back in the groove of being excited about a subject. I️ kicked off my search by attending a day-long academic conference on cryptocurrencies, and by the time the afternoon sessions began, I realized machine learning and computer vision was much more interesting to me.

Getting started

I️ kick-started my explorations right around the time a great book on the cross section of deep learning and computer vision was published. The author, Adrian Rosebrock from PyImageSearch.com, compiled a three volume masterpiece on the high level ideas and low level applications of computer vision and deep learning. While exploring deep learning, I️ encountered numerous explanations of linear regression, Naive Bayesian applications (I️ realize now that I️ have heard this name pronounced so many different ways), random forest/decision tree learning, and all the other things I’m butchering.

I️ spent a few weeks reading the book and came away feeling like I️ could connect all the disparate blog posts I have read up to now to the the array of mathematical concepts, abstract ideas, and practical programming applications. I read through the book quickly, and came away with a better sense of how to approach the field as a whole. My biggest takeaway was coming to the conclusion that I️ wanted to solidify my own tools and hardware for building computer vision software.

Hardware implementation

I️ was inspired to get a Raspberry Pi and RPI camera that I️ would be able to use to analyze streams of video. Little did I know that setting up the Raspberry Pi would take painfully long. Initially, I️ expected to simply get up and running with a video stream and process the video on my computer. I️ struggled with getting the Raspberry Pi operating system to work. Then, once I️ realized what was wrong, I️ accidentally installed the wrong image drivers and unexpectedly installed conflicting software. The process that I️ initially thought would be filled with processing camera images, ended up becoming a multi hour debugging nightmare.

So far, I️ have realized that this is a huge part getting started with machine learning and computer vision “stuff” is about debugging.

  • Step 1.Get an idea. 
  • Step 2. Start looking for the tools to do the thing. 
  • Step 3. Install the software needed. 
  • Step 4. Drown in conflicts and unexpected package version issues.

My original inspiration behind the Raspberry Pi was the idea of setting up a simple device that has a camera and GPS signal. The idea was based around thinking about how many vehicles in the future, autonomous or fleet vehicles, will need many cameras for navigation. Whether for insurance purposes or basic functionality, I️ imagine that a ton of video footage will be created and used. In that process, there will be huge repositories of media that will go unused and become a rich data source for understanding the world.

I️ ended up exploring the Raspberry Pi’s computer vision abilities, but never successfully got anything interesting working as I’d hoped. I️ discovered that there are numerous cheaper Raspberry Pi-like devices, that had both the interconnectivity and the camera functionality in a smaller PCB board than a full size Raspberry Pi. Then I️ realized that rather than going the hardware route, I️ might as well have used an old iPhone and developed some software.

My brief attempt at exploring a hardware component of deep learning made me realize I should stick to software where possible. Including a new variable when the software part isn’t solved just adds to the complexity.

Open source tools

In the first month of looking around for machine learning resources, I found many open source tools that make getting up and running very easy. I knew that there were many proprietary services provided by the FANG tech companies, but I wasn’t sure how they competed with the open source alternatives. The image recognition and OCR tools that can be used as SAAS tools from IBM, Google, Amazon, and Microsoft are very easy to use. To my surprise, there are great open source alternatives that are worth configuring to avoid unnecessary service dependence.

For example, a few years ago, I launched an iOS application to collect and share graffiti photos. I was indexing images from publicly available API’s with geotagged images, such as Instagram and Flickr. Using these sources, I used basic features, such as hashtags and location data, to distinguish if images were actually graffiti. Initially, I began pulling thousands of photos a week, and soon scaled to hundreds of thousands a month. I quickly noticed that many of the images I indexed were not graffiti and instead were images that would be destructive to the community I was trying to foster. I couldn’t prevent low-quality photos of people taking selfies or poorly tagged images that were not safe for work from loading in people’s feeds. As a result, I decided to shut down the overall project.

Now, with the machine learning services and open source implementations for object detection and nudity detection, I can roll my own service that easily checks each of the photos that get indexed. Previously, if I paid a service to do that quality checking, I would have been racking up hundreds of dollars if not thousands of dollars in API charges. Instead, I can now download an AMI from some “data science” AWS box and create my own API for checking for undesired image content. This was out of reach for me, even just two years ago.

On a high level, before undergoing this process, I felt like I theoretically understood most of the object recognition and machine learning processes. After beginning the process of connecting the dots between all the machine learning content I had been consuming, I feel like I am much more clear on what concepts I need to learn. For example, rather than just knowing that linear algebra is important for machine learning, I now understand how problems are broken into multidimensional array/matrices and are processed in mass quantities to look for patterns that are only theoretically representable. Before, I knew that there was some abstraction between features and how they were represented as numbers that could be compared across a range of evaluated items. Now I understand more clearly how dimensions, in the context of machine learning, are represented by the sheer fact that there are many factors that are directly and indirectly correlated to one another. The matrix math that the multidimensional aspects of feature detection and evaluation is still a mystery to me, but I am able to understand the high level concepts.

Concretely, the reading of Adrian Rosebrock’s book gave me the insight to decode the box-line diagrams of machine learning algorithms. The breakdown of a deep learning network architecture is now somewhat understandable. I am also familiar with the datasets (MNIST, CIFAR-10, and ImageNet) that are commonly used to benchmark various image recognition models, as well as the differences between image recognition models (such as VGG-16, Inception, etc).

Timing — Public Funding

One reason I decided machine learning and computer vision are important to learn now is related to a concept I learned from the book: Areas with heavy government investment in research are on track to have huge innovation. Currently, there are hundreds of millions of dollars being spent on research programs in the form of grants and scholarships, in addition to the specific funding being allocated to programs for specific machine learning related projects.

In addition to government spending, publicly accessible research from private institutions seems to be growing. The forms of research that currently exist, coming out of big tech companies and public foundations, are pushing forward the entire field of machine learning. I personally have never seen the same concentration of public projects funded by private institutions in the form of publications like distill.pub and collectives like the OpenAI foundation. The work they are putting out is unmatched.

Actionable tasks

Reviewing the materials I have been reading, I realize my memory is already failing me. I’m going to do more action-oriented reading from this point forward. I have a box with GPUs to work with now, so I don’t feel any limitations around training models and working on datasets.

Most recently, I attended a great conference on Spatial Data Science, hosted by Carto. There, I became very aware of how much I don’t know in the field of spatial data science. Before the conference, I was just calling the entire field “map location data stuff”.

I’ll continue making efforts to meet up with different people I find online with similar interests. I’ve already been able to do this with folks I find who live in New York and have written Medium posts relevant to my current search. Most recently, when exploring how to build a GPU box, I was able to meet a fellow machine learning explorer for breakfast.

By the middle of January, I’d like to be familiar with technical frameworks for training a model around graffiti images. I think at the very least, I want to have a set of images to work with, labels to associate the images to, and a process for cross-checking an unindexed image against the trained labels.

Filed Under: Uncategorized Tagged With: Computer Vision, Machine Learning, Programming, Spatial Analysis, Towards Data Science

Two months exploring deep learning and computer vision

December 20, 2017 by rememberlenny

I’ve been reading/note taking is using an iPad Pro and LiquidText

I decided to develop familiarity with computer vision and machine learning techniques. As a web developer, I found this growing sphere exciting, but did not have any contextual experience working with these technologies. I am embarking on a two year journey to explore this field. If you haven’t read it already, you can see Part 1 here: From webdev to computer vision and geo.

—

I️ ended up getting myself moving by exploring any opportunity I️ had to excite myself with learning. I wasn’t initially stuck on studying about machine learning, but I wanted to get back in the groove of being excited about a subject. I️ kicked off my search by attending a day-long academic conference on cryptocurrencies, and by the time the afternoon sessions began, I realized machine learning and computer vision was much more interesting to me.

Getting started

I️ kick-started my explorations right around the time a great book on the cross section of deep learning and computer vision was published. The author, Adrian Rosebrock from PyImageSearch.com, compiled a three volume masterpiece on the high level ideas and low level applications of computer vision and deep learning. While exploring deep learning, I️ encountered numerous explanations of linear regression, Naive Bayesian applications (I️ realize now that I️ have heard this name pronounced so many different ways), random forest/decision tree learning, and all the other things I’m butchering.

My new book, #DeepLearning for Computer Vision with #Python, has been OFFICIALLY released! Grab your copy here: https://t.co/rQgpAflp52 pic.twitter.com/fKEyf8i2fR

— Adrian Rosebrock (@PyImageSearch) October 17, 2017

I️ spent a few weeks reading the book and came away feeling like I️ could connect all the disparate blog posts I have read up to now to the the array of mathematical concepts, abstract ideas, and practical programming applications. I read through the book quickly, and came away with a better sense of how to approach the field as a whole. My biggest takeaway was coming to the conclusion that I️ wanted to solidify my own tools and hardware for building computer vision software.

Hardware implementation

I️ was inspired to get a Raspberry Pi and RPI camera that I️ would be able to use to analyze streams of video. Little did I know that setting up the Raspberry Pi would take painfully long. Initially, I️ expected to simply get up and running with a video stream and process the video on my computer. I️ struggled with getting the Raspberry Pi operating system to work. Then, once I️ realized what was wrong, I️ accidentally installed the wrong image drivers and unexpectedly installed conflicting software. The process that I️ initially thought would be filled with processing camera images, ended up becoming a multi hour debugging nightmare.

So far, I️ have realized that this is a huge part getting started with machine learning and computer vision “stuff” is about debugging.

Step 1.Get an idea. 
Step 2. Start looking for the tools to do the thing. 
Step 3. Install the software needed. 
Step 4. Drown in conflicts and unexpected package version issues.

https://aiyprojects.withgoogle.com/vision#list-of-materials

My original inspiration behind the Raspberry Pi was the idea of setting up a simple device that has a camera and GPS signal. The idea was based around thinking about how many vehicles in the future, autonomous or fleet vehicles, will need many cameras for navigation. Whether for insurance purposes or basic functionality, I️ imagine that a ton of video footage will be created and used. In that process, there will be huge repositories of media that will go unused and become a rich data source for understanding the world.

I️ ended up exploring the Raspberry Pi’s computer vision abilities, but never successfully got anything interesting working as I’d hoped. I️ discovered that there are numerous cheaper Raspberry Pi-like devices, that had both the interconnectivity and the camera functionality in a smaller PCB board than a full size Raspberry Pi. Then I️ realized that rather than going the hardware route, I️ might as well have used an old iPhone and developed some software.

My brief attempt at exploring a hardware component of deep learning made me realize I should stick to software where possible. Including a new variable when the software part isn’t solved just adds to the complexity.

Open source tools

In the first month of looking around for machine learning resources, I found many open source tools that make getting up and running very easy. I knew that there were many proprietary services provided by the FANG tech companies, but I wasn’t sure how they competed with the open source alternatives. The image recognition and OCR tools that can be used as SAAS tools from IBM, Google, Amazon, and Microsoft are very easy to use. To my surprise, there are great open source alternatives that are worth configuring to avoid unnecessary service dependence.


For example, a few years ago, I launched an iOS application to collect and share graffiti photos. I was indexing images from publicly available API’s with geotagged images, such as Instagram and Flickr. Using these sources, I used basic features, such as hashtags and location data, to distinguish if images were actually graffiti. Initially, I began pulling thousands of photos a week, and soon scaled to hundreds of thousands a month. I quickly noticed that many of the images I indexed were not graffiti and instead were images that would be destructive to the community I was trying to foster. I couldn’t prevent low-quality photos of people taking selfies or poorly tagged images that were not safe for work from loading in people’s feeds. As a result, I decided to shut down the overall project.

#graffiti results on instagram

Now, with the machine learning services and open source implementations for object detection and nudity detection, I can roll my own service that easily checks each of the photos that get indexed. Previously, if I paid a service to do that quality checking, I would have been racking up hundreds of dollars if not thousands of dollars in API charges. Instead, I can now download an AMI from some “data science” AWS box and create my own API for checking for undesired image content. This was out of reach for me, even just two years ago.

Overview

On a high level, before undergoing this process, I felt like I theoretically understood most of the object recognition and machine learning processes. After beginning the process of connecting the dots between all the machine learning content I had been consuming, I feel like I am much more clear on what concepts I need to learn. For example, rather than just knowing that linear algebra is important for machine learning, I now understand how problems are broken into multidimensional array/matrices and are processed in mass quantities to look for patterns that are only theoretically representable. Before, I knew that there was some abstraction between features and how they were represented as numbers that could be compared across a range of evaluated items. Now I understand more clearly how dimensions, in the context of machine learning, are represented by the sheer fact that there are many factors that are directly and indirectly correlated to one another. The matrix math that the multidimensional aspects of feature detection and evaluation is still a mystery to me, but I am able to understand the high level concepts.

The previously illegible network architecture graphs are now seemingly approachable.

Concretely, the reading of Adrian Rosebrock’s book gave me the insight to decode the box-line diagrams of machine learning algorithms. The breakdown of a deep learning network architecture is now somewhat understandable. I am also familiar with the datasets (MNIST, CIFAR-10, and ImageNet) that are commonly used to benchmark various image recognition models, as well as the differences between image recognition models (such as VGG-16, Inception, etc).

Timing — Public Funding

One reason I decided machine learning and computer vision are important to learn now is related to a concept I learned from the book: Areas with heavy government investment in research are on track to have huge innovation. Currently, there are hundreds of millions of dollars being spent on research programs in the form of grants and scholarships, in addition to the specific funding being allocated to programs for specific machine learning related projects.

Example of pix2pix algorithm applied to “cat-ness”. https://distill.pub/2017/aia/

In addition to government spending, publicly accessible research from private institutions seems to be growing. The forms of research that currently exist, coming out of big tech companies and public foundations, are pushing forward the entire field of machine learning. I personally have never seen the same concentration of public projects funded by private institutions in the form of publications like distill.pub and collectives like the OpenAI foundation. The work they are putting out is unmatched.

Actionable tasks

Reviewing the materials I have been reading, I realize my memory is already failing me. I’m going to do more action-oriented reading from this point forward. I have a box with GPUs to work with now, so I don’t feel any limitations around training models and working on datasets.

Most recently, I attended a great conference on Spatial Data Science, hosted by Carto. There, I became very aware of how much I don’t know in the field of spatial data science. Before the conference, I was just calling the entire field “map location data stuff”.

I’ll continue making efforts to meet up with different people I find online with similar interests. I’ve already been able to do this with folks I find who live in New York and have written Medium posts relevant to my current search. Most recently, when exploring how to build a GPU box, I was able to meet a fellow machine learning explorer for breakfast.

By the middle of January, I’d like to be familiar with technical frameworks for training a model around graffiti images. I think at the very least, I want to have a set of images to work with, labels to associate the images to, and a process for cross-checking an unindexed image against the trained labels.


Thanks to Jihii Jolly for correcting my grammar.

Filed Under: Uncategorized Tagged With: Computer Vision, Machine Learning, Programming, Spatial Analysis, Towards Data Science

Primary Sidebar

Recent Posts

  • Thoughts on my 33rd birthday
  • Second order effects of companies as content creators
  • Text rendering stuff most people might not know
  • Why is video editing so horrible today?
  • Making the variable fonts Figma plugin (part 1 – what is variable fonts [simple])

Archives

  • August 2022
  • February 2021
  • October 2020
  • September 2020
  • August 2020
  • December 2019
  • March 2019
  • February 2019
  • November 2018
  • October 2018
  • April 2018
  • January 2018
  • December 2017
  • October 2017
  • July 2017
  • February 2017
  • January 2017
  • November 2016
  • October 2016
  • August 2016
  • May 2016
  • March 2016
  • November 2015
  • October 2015
  • September 2015
  • July 2015
  • June 2015
  • May 2015
  • March 2015
  • February 2015
  • January 2015
  • December 2014
  • November 2014
  • October 2014
  • September 2014
  • August 2014
  • July 2014
  • June 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • January 2014
  • December 2013
  • October 2013
  • June 2013
  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • January 2013
  • December 2012

Tags

  • 10 year reflection (1)
  • 100 posts (2)
  • 2013 (1)
  • academia (2)
  • Advertising (3)
  • aging (1)
  • Agriculture (1)
  • analytics (3)
  • anarchy (1)
  • anonymous (1)
  • api (1)
  • arizona (1)
  • Art (2)
  • art history (1)
  • artfound (1)
  • Artificial Intelligence (2)
  • balance (1)
  • banksy (1)
  • beacon (1)
  • Beacons (1)
  • beast mode crew (2)
  • becausewilliamshatner (1)
  • Big Data (1)
  • Birthday (1)
  • browsers (1)
  • buddhism (1)
  • bundling and unbundling (1)
  • china (1)
  • coding (1)
  • coffeeshoptalk (1)
  • colonialism (1)
  • Communication (1)
  • community development (1)
  • Computer Science (1)
  • Computer Vision (6)
  • crowdsourcing (1)
  • cyber security (1)
  • data migration (1)
  • Deep Learning (1)
  • design (1)
  • designreflection (1)
  • Developer (1)
  • Digital Humanities (2)
  • disruption theory (1)
  • Distributed Teams (1)
  • drawingwhiletalking (16)
  • education (3)
  • Email Marketing (3)
  • email newsletter (1)
  • Employee Engagement (1)
  • employment (2)
  • Engineering (1)
  • Enterprise Technology (1)
  • essay (1)
  • Ethics (1)
  • experiement (1)
  • fidgetio (38)
  • figma (2)
  • film (1)
  • film industry (1)
  • fingerpainting (8)
  • first 1000 users (1)
  • fonts (1)
  • forms of communication (1)
  • frontend framework (1)
  • fundraising (1)
  • Future Of Journalism (3)
  • future of media (1)
  • Future Of Technology (2)
  • Future Technology (1)
  • game development (2)
  • Geospatial (1)
  • ghostio (1)
  • github (2)
  • global collaboration (1)
  • god damn (1)
  • google analytics (1)
  • google docs (1)
  • Graffiti (23)
  • graffitifound (1)
  • graffpass (1)
  • growth hacking (1)
  • h1b visa (1)
  • hackathon (1)
  • hacking (1)
  • hacking reddit (2)
  • Hardware (1)
  • hiroshima (1)
  • homework (1)
  • human api (1)
  • I hate the term growth hacking (1)
  • ie6 (1)
  • ifttt (4)
  • Image Recognition (1)
  • immigration (1)
  • instagram (1)
  • Instagram Marketing (1)
  • internet media (1)
  • internet of things (1)
  • intimacy (1)
  • IoT (1)
  • iteration (1)
  • jason shen (1)
  • jobs (2)
  • jrart (1)
  • kickstart (1)
  • king robbo (1)
  • labor market (1)
  • Leonard Bogdonoff (1)
  • Literacy (1)
  • location (1)
  • Longform (2)
  • looking back (1)
  • los angeles (1)
  • Machine Learning (13)
  • MadeWithPaper (106)
  • making games (1)
  • management (1)
  • maps (2)
  • marketing (4)
  • Marketing Strategies (1)
  • Media (3)
  • medium (1)
  • mentor (1)
  • message (1)
  • mindmeld games (1)
  • Mobile (1)
  • Music (2)
  • Music Discovery (1)
  • neuroscience (2)
  • new yorker (1)
  • Newspapers (3)
  • nomad (1)
  • notfootball (2)
  • npaf (1)
  • odesk (1)
  • orbital (14)
  • orbital 2014 (14)
  • orbital class 1 (9)
  • orbitalnyc (1)
  • paf (2)
  • paid retweets (1)
  • painting (1)
  • physical web (1)
  • pitching (2)
  • popular (1)
  • post production (1)
  • Privacy (1)
  • process (1)
  • product (1)
  • Product Development (2)
  • product market fit (2)
  • Programming (6)
  • project reflection (1)
  • promotion (1)
  • prototype (17)
  • prototyping (1)
  • Public Art (1)
  • Public Speaking (1)
  • PublicArtFound (15)
  • Publishing (3)
  • Python (1)
  • quora (1)
  • Rails (1)
  • React (1)
  • React Native (1)
  • real design (1)
  • recent projects (1)
  • reddit (3)
  • redesign (1)
  • reflection (2)
  • rememberlenny (1)
  • Remote work (1)
  • replatform (1)
  • Responsive Emails (1)
  • retweet (1)
  • revenue model (1)
  • rick webb (1)
  • robert putnam (1)
  • ror (1)
  • rubyonrails (1)
  • segmenting audience (1)
  • Semanticweb (2)
  • Senior meets junior (1)
  • SGI (1)
  • Side Project (1)
  • sketching (22)
  • social capital (1)
  • social media followers (2)
  • social media manipulation (1)
  • social media marketing (1)
  • social reach (5)
  • software (3)
  • Soka Education (1)
  • Spatial Analysis (2)
  • spotify (1)
  • stanford (2)
  • Startup (21)
  • startups (7)
  • stree (1)
  • Street Art (4)
  • streetart (5)
  • stylometrics (1)
  • Technology (1)
  • thoughts (1)
  • Time as an asset in mobile development (1)
  • Towards Data Science (4)
  • TrainIdeation (42)
  • travel (1)
  • traveling (1)
  • tumblr milestone (2)
  • twitter (1)
  • twitter account (2)
  • typography (2)
  • unreal engine (1)
  • user behavior (1)
  • user experience (3)
  • user research (1)
  • user testing (1)
  • variable fonts (1)
  • video editing (2)
  • visual effects (1)
  • warishell (1)
  • Web Development (8)
  • webdec (1)
  • webdev (13)
  • windowed launch (1)
  • wordpress (1)
  • Work Culture (1)
  • workinprogress (1)
  • zoom (1)