Useful Stuff.

If I find something useful, somebody else probably does too.

Braintree Payment Solutions Review: It’s Amazing!

Braintree Payments is absolutely amazing.

They <3 developers, and it's true. Not only that, they have a neat transparent redirect system that allows your clients to POST form data directly to braintree, bypassing your servers entirely. You can get PCI compliant quite easily.

Transparent redirect secures card data completely

This is a huge benefit.

Braintree has a standard Server to Server (S2S) API with client libraries in major application environments (Python, Ruby, PHP, Java, Perl, .NET, even Node!) which stand out from ease of use already, but they offer even more: the transparent redirect system.

With TR, your HTML form elements literally point to Braintree. Clients send absolutely no data to your servers, and your PCI Compliance requirements are drastically reduced. You can qualify for a self assessment questionnaire (if you don’t enter any CC data over the phone for example) rather painlessly.

How does TR work?

There are two components to TR. One is data encrypted from the server in a hidden tr_data field. The other is the user-fillable input boxes.

When you initialize the tr_data generating function, you’ll give it a callback url for braintree to redirect to. It not only serves to direct users back to your site, but the query string contains data that you can pass to your client library to get a Transaction object exactly as if you’d done the S2S implementation. Seamless!

Step 1: post the form to braintree

Fill in the tr_data, fill in the minimum required CC fields, and post away to your merchant TR URL.

If you’re worried about your users manually submitting data, you can enforce data submission by the server by using the tr_data field.

For example, you might force the billing country to be from the US.

Step 2: handle the query string at your callback url

At the callback url you supplied the client library to generate your tr_data, pass the query string into the braintree client library. It will return a Transaction object just as if you’d done it through S2S.

Here, I check to make sure it’s not an Error response (at which point I have to put in some special logic to display the error at the submitting form view) and redirect to a success view if the transaction is successful.

Step 3: do what you will with a securely created Transaction object

The amazing thing is that the result is the same object you’d deal with if you had used the S2S API.

Clear pricing

I don’t even want to look into the 40 or 60 different rates we have for our credit cards. It’s utterly complicated and confusing. Hidden fees seem everywhere.

Not so here.

What is that, 3 numbers? All of which amount to less right off the bat than what I’ve been quoted.

I think this transparency is a trend among emerging tech companies. There’s no reason why payments have to be difficult.

Clean and clear monthly reports

Finally, a company that gives us great monthly reports. The payments industry is amazingly behind the times (everything is paper based) but at the very least this is the best experience I’ve had with clearly itemized monthly reports.

Even chargebacks are on this report. With several other payment company combinations I’ve used, all we’d get is a piece of paper in the mail and no other record.

Amazing support.

Helpful people pick up the phone in 15 seconds.

I got great support the first time I called about a decently technical question.

Amazing amazing support

This support is unreal. Our shopping cart: Shopify had some interesting error messages for cards declined due to AVS. It was unhelpful and confusing customers.

When I brought the issue up with Braintree rather casually (3 sentence email?) their support staff responded telling me it wasn’t their fault – it was Shopify’s ActiveMerchant.

Guess what they did without any further contact? They decided to submit a patch to ActiveMerchant on Github to fix detection of AVS mismatches with better error messages.

Let’s see, what happened for 3 sentences?

  1. they read it.
  2. they responded / it wasn’t their fault.
  3. they looked into ActiveMerchant and found the problem with the braintree code.
  4. they actually fixed it and submitted the patch.

I think that’s pretty amazing.

Conclusion: amazing.

Installing Solr and django-haystack on Ubuntu with OpenJDK

Install django-haystack

This part is easy – django-haystack is just a python module.

pip install django-haystack

Install OpenJDK and Solr

Solr is a Java program – we need the Java Runtime Environment to run our Solr server.

I’ll be using OpenJDK and solr-jetty on Ubuntu.

apt-get install openjdk-6-jre jetty solr-jetty 

Find and configure solr

$ find / -name "solr" 

Now that we know our solr config files live in /etc/solr/conf, have django-haystack generate the solr schema.xml file.

python build_solr_schema > /etc/solr/conf/schema.xml

Tweak solrconfig.xml

I’ll need to enable the MoreLikeThis handler by adding “” to the solrconfig.xml file.

Modify Jetty to run on port 8983 – default solr port

My jetty defaulted to port 8080 – a port I commonly use for other projects (nginx -> apache). Also, the default solr port is 8983 which my development environment uses. We’ll need to update jetty to use a different port.

I found the config file by searching for the jetty folder (/etc/jetty/jetty.xml) – modify the “port” line to whatever port you have set in your django settings HAYSTACK_SOLR_URL

<Set name="port"><SystemProperty name="jetty.port" default="8983"/></Set> 

Start jetty

Navigate to your jetty folder, and run.

$ java -jar /usr/share/jetty/start.jar

Build your solr indexes

Now we need some data to search against.
Assuming you’ve set up django-haystack, run the rebuild_index management command.

$ python rebuild_index

Set up cron job to rebuild indexes however often you need

$ crontab -e
$  0 0 * * * python update_index --age=24 --remove

Django / Python – Fabric Deployment Script and Example


I’d heard about it for at least a year+ but I never got around to it.

I was a little intimidated by it.

The examples seemed so simple, but that was the problem – they seemed too simple, like the examples you run into which look completely uninformative, just a quick blot of code and no explanation.

Well, it turns out fabric really is just that magical.

This is straight life changing!

This is as useful as when I first discovered how much easier it was (for me) to use Git for branching / merging than SVN, or deciding to switch to a linux-like environment for web development.

I’m writing about this today because I was always intimidated by it / thought it would take some time to learn, when there was no reason to be. It’s extremely simple.

Hopefully, it helps somebody out there with the same reservations.

First, let me point you to the official fabric docs where you can find the tutorial and more.

Install fabric.

First, pip install fabric on each machine you need to use fabric on.

 pip install fabric 

Create the fab file that contains functions

Create a blank file called in the directory you’d like to use the fabric commands from.

For my django project, that’s the directory that contains and My “home base”, or where you’ll find me in an average terminal window.

Create your first fabric command

In just a few lines of code, let’s set up a fabric script that asks the server to pull from the git repository and restart apache.

    # import fabrics API functions - self-explanatory once you see 
    from fabric.api import * 
    def pushpull():
        local('git push') # runs the command on the local environment
        run('cd /path/to/project/; git pull') # runs the command on the remote environment 

Let’s try running that in bash.

 bash$ fab pushpull 

It will ask you for a server to connect to that’s running fabric. You will want to enter in your username@host:port here.

Obviously, typing the full URL every time defeats the purpose of an automation script – let’s add it into our fabric file.

Add remote server commands

Let’s add the remote server information to our fabric file. We need to add our server information to our fabric script via the env.hosts parameter.

 from fabric.api import * 
 env.hosts = ['']
 def pushpull(): 
    local('git push') # runs the command on the local environment 
    run('git pull') # runs the command on the remote environment 

env is a dictionary containing various settings, one of which is ‘hosts’ – a list of hosts that all commands will run through.

Now, if we run fab pushpull, we won’t need to type in the server name. If you have SSH keys set up, you won’t need a password.

Add more server setup commands

Next, I want support for working on a development server or production.
Adding this information via a separate function ensures our commands are not tied to specific environments. For example, I use the same functions chained with “dev” or “prod” to determine which environment I am applying the command towards.

Okay, our first roadblock; it turns out that you can not dynamically set env.hosts within a function – it will not register.

 def pushpull(host=None):
    env.hosts = [host] # this will fail
 bash$ fab # fails 

You need to define it in a separate function. Here’s what I’ve done.

I’ve set up a “dev” and “prod” function that sets up the environment prior to running another fab command. For good measure, I also created an “all” command.

 dev_sever = ''
 prod_server = ''
 def dev(): 
    """ Use development server settings """
    env.hosts = [dev_server] # this is the place to add other setup, such as if the django project is in a different directory 
    # reference this variable later 
    env['dir'] = '/path/' 
 def prod():
    """ Use production server settings """ 
    env.hosts = [prod_server] 
 def all(): 
    """ Use all serves """
    env.hosts = [dev_server, prod_server]

Now, we can run the command.

 bash$ fab dev pushpull bash$ fab prod pushpull bash$ fab all pushpull 

Wow, amazing. The possibilities already are limitless!

Some tips – reuse functions.

I’ve found that writing many small functions helps since I’m not always using fabric.

For example, I have a large, full deployment function that…

  • commits local changes
  • pushes the changes
  • pulls the changes on the remote machine
  • syncdbs the remote machine
  • migrates the remote machine
  • restarts apache and other servers

Now, each one of these functions is actually another separate function, since who knows, I might have committed my changes myself and even pushed changes and now I only need a server restart – for me that’s “fab prod deploy”

Passing arguments to fabric functions.

Functions can accept arguments, and are passed to the python functions from the shell via a colon

 bash$ fab dev some_func:arg1,arg2,keyword=kwarg 

Preventing errors from blowing up

Any non success exit code will stop your function. Use the settings function from the api to allow them to silently pass.

For example, git commit -a will fail if there’s nothing to add.

    def foo():
        with settings(warn_only=True):
            local('git commit -a') # fabric will no longer abort 

Keep functions open-ended via *args

There are tons of special case scenarios with django-south and its commands, so I’ve made a migrate fabric command that accepts an arbitrary number of arguments, to support say migrate myapp 0001 —fake

    def migrate(*args):
        with cd(env['mypath']): # every command will now be run in the path specified 
            run('python migrate ' + ' '.join(args)) 
 bash$ fab dev migrate:--all,--fake,0001 

Define a way to revert a mistake

Make sure you have a way out of an operation that kills your site.

In my case, I have set up a git revert command as “revert”

    def revert():
        """ Revert git via reset --hard @{1} """
        with cd(env['dir']):
            run('git reset --hard @{1}')
            arestart() # restarts server 

This command would undo the last pull to the working state the server should have been in before we forced a pull.

Use docstrings

The first line of your docstring appears when you list your fab functions via fab –list

One final note:

Check the docs. There are functions such as sudo(‘myfunc’) that will run as root. Very useful.

Laser – Structural Beam Bending Calculations

This is the first post in the Laser category. I wrote “Structural Beam Bending” because I am not referring to a laser beam, but an aluminum beam in my laser.

It took me a while to get back on the engineering concepts train. The last time I did any unit conversions at all was at least 5 years ago.

The first major hiccup I experienced while attempting to do beam bending calculations with a program called BeamBoy was that I wanted to determine my beam lengths in inches. All of my material data is in metric, so converting mm4 to in4 took me a moment to realize you simply divide by (mm/inch)^4, which is 25.4^4, or 416,231.4256

Unlike programming, physics information is not as readily available in easily digestible formats. I will outline my experiences to help anybody else who stumbles across this blog.

To calculate beam deflection:

Find your material’s Moment of Inertia, Modulus of Elasticity, and the Distance to Farthest Fiber.

My parts come from Misumi, so their data sheet looks like this:

The Moment of Inertia for the two axes are clearly shown. The Modulus of Elasticity is a property of the material, and I couldn’t actually find the exact grade of aluminum Misumi is using, so I looked online for average values and it ranged from 9,000,000 PSI to 11,000,000 PSI.

Distance to farthest fiber is the farthest distance from the beam’s neutral axis, so I interpret that to be 1/2 the width. This may be wrong.

First, start up BeamBoy and type in the length of your beam.

Enter in the properties of your beam as discussed above: moment of inertia, modulus of elasticity, and distance to farthest fiber.

Next, add two supports to your structure, I simply added a support at the start and end of my beam.

Add a distributed load over the length of your beam equal to the weight per distance (in my case, lb/ft) to simulate the weight of the beam itself.

Optionally, add a distributed or point load to the beam. I wanted maximum deflection for a given weight so I used a point load in the “worst” possible spot: the center, away from my 2 support structures.

Click calculate and see your results!

PS: I tried doing this in Solidworks beam modeler but the results don’t seem to be accurate.. Perhaps it’s because I was using a distributed load?

The Solidworks beam simulation is potentially a huge time saver as you the length, material properties and thickness is auto calculated.

Just can’t trust it yet.

Python — Basics of Python Dictionary: Looping & Sorting

Here some bits of info about python dictionaries & looping through them.

Extra special beginner stuff.

What is a dictionary?

A python dictionary is an extremely useful data storage construct for storing and retreiving key:value pairs.

Many languages implement simple arrays (lists in python) that are keyed by a integer. For example, if you made a list [1, 2] – the first value would be retrieved via [1, 2][0].

my_list = [1, 2, 3]
# Out: 1
# Out: 2

A dictionary is a little more advanced in that the keys can be many other things than integers. Often times the key will be a string merely for the fact that it’s easy for a human to recall.

Will I remember that my_list[3] is my phone number? Not nearly as well as if I have a key named “phone_number”.

my_dict = {
	'key1': 'value1',
	'key2': 'value2',
	'key3': 'value3'
# Out: 'value1'

Major differences vs lists

- Keys are any hashable object (say strings for simplicity)
- Are NOT ordered (a list is by definition ordered)

One way I like to think about them are as little variable containers. The fact that they are wrapped in a container makes them quite useful and versatile since you can easily move the “container” around.

In fact, variables are very much related to dictionaries! Whenever you declare a variable (x=3), that variable is accessible by its string name (‘x’) via the dictionary returned by the builtin function ‘locals()’ or ‘vars()’.

Watch and see:

var1 = 'val1' # local variable definition
var2 = 'val2'

locals_dict = locals()
print locals_dict['var1']
# Out: val1

container = {}
container['var1'] = 'val1'

print container['var1']
# Out: val1

Looping through dictionaries

Now, if you did a little experimenting, you would see that if you loop through a dictionary you loop through its keys.

>>> my_dict = {
...     'key1': 'value1',
...     'key2': 'value2',
...     'key3': 'value3'
...     }
>>> for item in my_dict:
...     print item

Note that this is the equivalent of looping through “my_dict.keys()”

What about getting the values?

Based on what we’ve learned, you could always use the keys you are iterating through to pull the value from the dictionary.

for item in my_dict:
    print my_dict[item]

But there are better ways to get the values. Enter the “items” function.

We can ask the dictionary to return key, value pairs via the “items()” function.

for key, value in my_dict.items(): # returns the dictionary as a list of value pairs -- a tuple.
    print key, value

More efficient dictionary loops

Calling “my_dict.items()” requires generating the entire list of key-value pairs in-memory. Sometimes, if your dictionary is too large, this can be a severe performance bottleneck. To get around this problem we can create a generator via the “iteritems()” method.

A generator allows you to iterate one item at a time. Only the key and value are pulled into memory for every iteration and immediately discarded. There are methods for returning a key generator and value generator as well.

for key, value in my_dict.iteritems():
    print key, value

for key in my_dict.iterkeys():
    print key
for value in my_dict.itervalues():
    print value

Note that one thing you can’t do with a generator is to delete a key during the generator loop.

for x, y in mydictionary:
    del mydictionary[x] 
    # Out: RuntimeError: dictionary changed size during iteration

Sorting a Python Dictionary

Actually, python dictionaries can’t be sorted. We will have to turn them into lists to sort them.

# to my dictionary...
sorted_list = [x for x in my_dictionary.iteritems()] 

sorted_list.sort(key=lambda x: x[0]) # sort by key
sorted_list.sort(key=lambda x: x[1]) # sort by value

# to reverse the sort

Useful dictionary tips

Accessing a dictionary key that might not exist

Sometimes we know that a dictionary key might not exist. This happens a lot in loops where we don’t want to use a try/except block just to capture the exception.

For these situations we have the “get” method. Pass in the key as the first argument, and it will either return the value or None.

Pass in a second argument, and if the key doesn’t exist, it will return the second argument.

dict_ = {'key1':'value1'}
# out: 'value1'

# out: None

dict_.get('key2', "Key doesn't exist")
# out: Key doesn't exist

Get or insert into a dictionary if key doesn’t exist

Sometimes we need to insert a value if the key doesn’t exist in a dictionary. The previous “get” method only returns a value – the dictionary is unchanged.

dict_ = {}
dict_.setdefault('key1', 'value1')
# Out: value1

print dict_
# Out: { 'key1': 'value1' } 

This can be extremely useful if you need to append to a list if it exists or otherwise create a blank list.

key_value_pairs = [
    ('key1', 'value'),
    ('key1', 'value2'),
    ('key1', 'value3'),
dict_ = {}

for key, value in key_value_pairs:
    dict_.setdefault(key, []).append(value)

print dict_
# Out: { 'key1': ['value','value2','value3'] }

Generate a dictionary from tuples

It’s often useful to generate a dictionary from a list of tuples. For example, you could use a list comprehension to create a dictionary!

key_value_pairs = [
    ('key1', 'value'),
    ('key2', 'value2'),
    ('key3', 'value3'),

# now let's generate the same list of tuples via list comprehension 
key_value_pairs = [('key{0}'.format(x), 'value{0}'.format(x)) for x in range(1, 4)]

dict_ = dict(key_value_pairs)
print dict_
# Out: {'key3': 'value3', 'key2': 'value2', 'key1': 'value1'}


I’m a little surprised that this is one of my most visited blog posts.
It’s not written well, and it was written years ago!

Let me know if I can improve it… ;)

Assigning dynamic/unique variable names in a for loop in AS3

Assigning dynamic variable names to objects created in a for loop in AS3.

You have to get very creative to search these questions on google because words like “for” and “loop” are so utterly common.

If you have a loop in AS3 creating multiple variables but need them to have unique names (for later modification) you need to create something like an array to store the references to the variables.

For example, anything you create in a for loop is a temporary reference. It disappears after the loop is complete, and whatever objects were created stick around in the displayObject.

In my situation I had a loop creating my navigation elements, but needed to position each X value separately.

Here is the method I used to keep the references to these variables:

private var mcArray:Array = new Array();  // array that holds references
private function myfunction():void{
	for (var i:int=0; i&lt;2; i++) { 
		var movieClip:MovieClip = new MovieClip(); // the temporary variable
		mcArray[i] = movieClip; // store reference to temporary variable in array

Now, to use those references:

// or
// --
for (var i:int=0; i&lt;2; i++) {
	mcArray[i].foo = bar;

To be absolutely clear, the above example creates two MovieClips according to the loop (while i < 2), and places each MovieClip in the mcArray array at index i (which we have defined as the integer counting the loop).

Now if I wanted to access those individual MovieClips I could use the mcArray to do so.


I could also loop through the array to apply an animation to each item. The possibilities are endless when you use this basic concept!

Generate 1000 triangles. Animate each one to a random color and rotation.

Write HTML newsletters easier by letting tools inline the CSS for you

HTML newsletters are a *huge* pain, but mostly due to all of the inline CSS that’s hard to manage. Your code becomes a mess. Every task is like finding needles in a haystack.

Instead… just use normal CSS and use tools to inline the CSS for you, like this one by MailChimp:

npm install pad-stdio error (no versions found)

Read npm install error output very carefully, my node version was outdated.

Unpredictable mobile font size issues

If you have mobile font size issues you can’t articulate, it’s probably something like this:

Set -webkit-text-size-adjust: 100% on elements in question and see if it fixes your problem.

Export Chrome History as CSV (Spreadsheet)

Chrome stores history as an SQLITE3 database.

First, find it.
Close Chrome.

Open up the terminal, and navigate to the directory that contains the History file.

cd ~/Library/Application\ Support/Google/Chrome/Default/

Open the database with SQLITE3:

sqlite3 History

Enable Sqlite3 output mode as CSV by typing into sqlite shell:

sqlite> .headers on
sqlite> .mode csv
sqlite> .output my-chrome-output.csv

Now, execute the SQL statement (copy and paste). Note that by default, the time stamp is not “human readable” so we’ll insert some logic to convert times into our local time zone / months, years, and that good stuff.

SELECT datetime(last_visit_time/1000000-11644473600,'unixepoch','localtime'),
                 FROM urls
                 ORDER BY last_visit_time DESC

Done! Check your directory for my-chrome-output.csv.

Why this isn’t documented anywhere except in small pieces? Don’t know.

South Migration App Has No Migrations

If you get this error for an app that truly doesn’t have any migrations, and perhaps isn’t even related to your project, it most likely means the south_migrationhistory table has row that matches one of your INSTALLED_APPS.

This can happen if at one point the app had migrations, but the latest version doesn’t (happens to third party apps), or you once built an app with migrations in the early days of your project.

Remove it via SQL to fix the problem.

DELETE FROM south_migrationhistory WHERE app_name = ‘offending_appname’;

SQLAlchemy Order By Foreign Key Table Column

To order by a foreign key value, make sure you do a proper `.join` of the table you want to reference in addition to  `.options` and `joinedload` or `joinedload_all`, which is purely to retrieve data from each related row.

SQLAlchemy never ceases to amaze.. I’m glad I made the switch, as it is making me a better coder by not shielding me from the intricacies  of SQL.

How joinedload() in particular achieves this result of not impacting entity rows returned in any way is that it creates an anonymous alias of the joins it adds to your query, so that they can’t be referenced by other parts of the query. For example, the query below uses joinedload() to create a LEFT OUTER JOIN from users to addresses, however the ORDER BY added against Address.email_address is not valid – the Address entity is not named in the query:

>>> jack = session.query(User).\
... options(joinedload(User.addresses)).\
... filter('jack').\
... order_by(Address.email_address).all()
SELECT AS addresses_1_id, addresses_1.email_address AS addresses_1_email_address, addresses_1.user_id AS addresses_1_user_id, AS users_id, AS users_name, users.fullname AS users_fullname, users.password AS users_password FROM users LEFT OUTER JOIN addresses AS addresses_1 ON = addresses_1.user_id WHERE = ? ORDER BY addresses.email_address <– this part is wrong ! ['jack']

Above, ORDER BY addresses.email_address is not valid since addresses is not in the FROM list. The correct way to load the User records and order by email address is to use Query.join():

>>> jack = session.query(User).\
... join(User.addresses).\
... filter('jack').\
... order_by(Address.email_address).all()
SELECT AS users_id, AS users_name, users.fullname AS users_fullname, users.password AS users_password FROM users JOIN addresses ON = addresses.user_id WHERE = ? ORDER BY addresses.email_address ['jack']

Nginx 400 Error “client intended to send too large body”

If you get this error, try raising your large_client_header_buffers setting from the default 4 or 8k to 16k.

Added this to my nginx "server" directive:

large_client_header_buffers 4 16k;

Flask Cache Buster based on Git Revision

This is cool.

Here’s a simple flask cache buster that appends a version number to all static assets based on the current git revision short hash.

Flask is built so amazingly. It has hooks to allow modification of all urls generated by the system.

app = make_app()

def get_git_revision_hash():
Get current revision short hash and use as cache buster key.
import subprocess
return subprocess.check_output(['git', 'rev-parse', '--short', 'HEAD'])

git_revision_hash = get_git_revision_hash()
STATIC_ENDPOINTS = [</div><div class="" id="file-gistfile1-txt-LC14">    'static',</div>

<div class="" id="file-gistfile1-txt-LC15">    'shop.static',</div><div class="" id="file-gistfile1-txt-LC16">]
def static_cache_buster(endpoint, values):
if endpoint in STATIC_ENDPOINTS:
values['_v'] = git_revision_hash

Update: actually, git revision number is not what I’m looking for

This only works for developer driven content. It wouldn’t work for user uploads of course, therefore I modified this to read mtimes.

        def static_cache_buster(endpoint, values):
            if endpoint in STATIC_ENDPOINTS:
                # append timestamps of files
                modules = endpoint.split('.')
                if len(modules) == 2:
                    blueprint, _ = modules
                    static_folder =[blueprint].static_folder
                    static_folder =
                file_path = os.path.join(static_folder, values['filename'])
                mtime = os.path.getmtime(file_path)
                values['_v'] = mtime

ProtocolError: error 8 from memcached_set: PROTOCOL ERROR

Yeah you’re probably trying to use a non string cache key. In my case, a function instead of calling said function.

Python Path the Easy Way. Add .pth Python Configuration Files

So python path is always a little daunting when coming to deployment. Especially since it’s typically done outside of your neatly wrapped software package.

The easiest way to do this so far is using python configuration files. Drop dead easy: put a .pth extension file in your site-packages directory, where each line corresponds to an entry you want put into sys.path.


Get every new post delivered to your Inbox.

Join 75 other followers