Selling a Home with Deep Learning

Work flow used to build a model which can estimate a home's value based on an image of the home.

This can be used as a base project to launch off custom projects which use deep learning to build complex models used in estimating.

The work flow is designed with the following goals in mind.

• Keep it affordable.
• Quickly iterate.
• Provable assertions.

The process which is used in building these models follows this pattern:

• Download a bunch of images.
• Get or create meta-data about the images.
• Train a Convolutional Neural Network using these images.
• Visualize the results.
• Tweak parameters.
• Do it all again.

Setup Requirements

This is a Python project which works best on Python 2.7 but should work on Python 3.0 as well.

Mostly you'll need requirements in the main project but sub-projects will have separate requirements.

pip install -r requirements.txt

Now startup the notebooks and look around.

ipython notebook

Running anything using your EC2 account requires the proper AWS credentials file and to update the profile setting from cedarstreet to your profile's name.

It will also require security policies in place with enough access to your profile, an example is placed below.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt000",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInstances",
                "ec2:DescribeSpotInstanceRequests",
                "ec2:RequestSpotInstances",
                "ec2:DescribeSpotPriceHistory"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

When using Ansible, it will require SSH access to the VPC's Security Group where those instances are running.

Process

General Operations

Manage the system or systems which are doing the model training.

• Check list of prices for EC2 Spot instances g2.2xlarge.
• Start a spot instance in a certain region for x hours or days.
• Use Ansible to connect to system and be able to transfer files.
  • Get logs from system.
  • Run python script to get filters for a snapshot.
• Get filter and logs from system.
• Visualize results in iPython notebook.
• Shutdown instance.

Training Data Setup

Create data which will be used to create a model.

• Generate test, train and cross validation data.
  • Connect to Zillow and download pictures.
  • Categorize images using categorization Django application.
  • Output in a format for Caffe to use.

References

• Excellent Introduction to CNN
• Deeper Tutorial on CNN
• Caffe Layers
• Caffe MNIST Tutorial
• Applications of Convolutions
• Complexities of Convolutions
• MXNet Distributed Deep Learning
• RNN Introduction
• Chainer Deep Learning Framework
• NVIDIA introduction to CNN