Wherein I recap the things I’ve done and learned this year as a software developer (mainly at work). I’ve fixed many bugs and uttered many curses. Through it all I’ve constantly strived to make the code a better place.

Amazon Web Services

• Video encoding instances Launch/Start when there’s a backlog, and Stop/Terminate when there’s not.
• Said goodbye to Amazon SQS for job queues, and hello to Redis. Hello consistency and speed.
• Learned that one must always specify timeouts for AWS calls. Especially for SimpleDB.

Node.js

Continued to make our video upload handler more robust. The aws2js library, which we use to transfer uploaded videos to Amazon S3, essentially does an emit bomb for multi-part transfers to S3. It splits the file in chunks (without enforcing an upper limit) and attempts to upload all the parts at the same time. This causes node to crash when processing large (>800MB) files, which we encounter quite frequently. To combat this I extracted the multi-part upload portion of aws2js and created a version that uploads file parts sequentially, with retries.

I also updated nodeftpd to use pipe() for file copying across streams, rather than copying buffers explicitly in the code. It was a simple change, but I enjoy cleanups.

Sphinx

We no longer have to daily re-build our rankings index thanks to Sphinx’s real-time indexes. Took a while to import, but the ability to insert/update/delete documents via SQL is super handy. Sphinx is a great tool.

JavaScript and UI

Experienced first-hand what it means to implement a separation concerns on the client-side. A few take-aways:

• One should very rarely use DOM nodes to store data. jQuery data() is super handy, but resist the temptation and use a JavaScript data structure instead.
• Became even more entrenched in my preference for jQuery delegate(). If you’ve got more than 10 DOM elements that may trigger an event, don’t attach handlers to each one. Let the event bubble up to a parent.

PHP

• Persistent connections are your friend. Don’t use Memcache or Redis without them. For the latter, you’ll likely have to compile the redis extension (here)
• When operating outside of function or class scope, always initialize your variables. You never know what values they may hold from further up the include stack.
• Beware type conversions surround inequalities. “‘0’ < 0” will surprise you.
• Use multi-byte string functions. Always. Start now.
• Never ever build JSON by hand. You’re wasting your time and you’ll run into escaping issues.

Redis

• I love you.

Memcache

• Ugh.
• Beware the 250 character limit on memcache keys.

MySQL

• Triggers are a handy method for gradually migrating data from a MyISAM table to InnoDB.

ffmpeg / libav

Sometimes videos will fail to convert for strange reasons. If so:

• Try only 1 output format at a time
• Try “-threads 1” instead of 2+

Sysadmin

• Don’t log a whole bunch of extra stuff when a critical error occurs. Especially don’t run lsof, netcat, etc. The additional forking will surely bite your server in the ass.

NON WORK

Android

• Created my own android keyboard for small screens. Meets my needs, though not ready for public consumption.
• Launched my own GPS tracking app for personal data collection. No social shit here.
• Successfully built my own ROM. Something I hope to do much more of.

Other

• Found a special case of XSS vulnerability (here).
• Got my “code janitor” certification.