• Thanks to Pathogen, plugins can remain separate from your main configuration, which great decreases the friction of trying new plugins and keeping your existing plugins organized.
• Thanks to git, plugins have revision history and stable versions (tags).
• Thanks to github, plugins have easy and stable endpoints to install from, to get updates from, and to complain about for bugs. Additionally, forking makes it easy to fix bugs yourself or make any tweaks you want, while still keeping everything in git.
• Thanks to git submodules, you can store your list of installed vim plugins, update them when necessary, and share them between multiple machines.
• With vundle or vim-addon-manager, you can search and install new plugins, all from within vim.

The Theory

My dotfiles and vimrc have been in source control for years. I've always kept a selection of vim plugins in my dotfiles repo, specific versions of plugins that I know work together. Pathogen was an easy switch and made it much easier to try new plugins and figure out if a new plugin played well with my current configuration.

With git submodules and keeping everything in git, I've come to realize, you get all of that and more. Want to try a new plugin? Download it to your vim bundle directory and give it a try. Git status will show you that it's an untracked directory and you haven't added it your dotfiles yet. When you decide you like it, you can add it as a submodule and checkout version X, which you know works correctly with your system.

Want to try out the latest dev version? Is there even a newer version to try? Pull latest in that plugin's git repo and see if there are updates. If you like those updates, commit the latest version in the git submodules, otherwise reset everything to the last known working state.

Want to fix something in the plugin? Like some new features, but want to keep some of the old features? Fork the plugin, delete the official submodule from your dotfiles, and checkout your fork as a submodule. Everything can remain in git still.

The Practice

One of the best examples I've seen recently was a Stack Overflow question a few months back. Someone asked how to add highlighting to paired html tags. The answer was a simple plugin that added that exact functionality. Not source code or a link to a vim wiki, but a full plugin, that with the setup I just described, is a one-step process to try out.

I feel plugins have traditionally been complex things in vim, but by having this easy and module process available and understood, more cool plugins will be made. Looks like it's time to Learn Vimscript the Hard Way and start writing my own plugins now!

It's not all gloom and doom: having a local backup of your data is great for searching and viewing in ways that the original provider didn't think about. I think it's cool to be able see how emails I get year over year. A benefit of being in this day and age is that storage is cheap: storing everything you've ever created and done online is entirely possible.

What To Backup

I started out with emails and figuring out to organize things. I then moved onto email contacts, chat logs, Delicious bookmarks, and social site data. As of late, I've been working on receipts and finances, trying to move to toward existing entirely paper free! Here are some ideas, feel free to add suggestions in the comments.

• Email
• Contacts (from GMail, Facebook, or LinkedIn)
• Chatlogs (Adium)
• Bookmarks (Delicious, Pinboard)
• Social site data (Twitter, FB, LinkedIn, Google+, Reddit)
• Pictures (Flickr, Picasa)

How To Backup

Having a simple way to backup your data and having easy access to your data from a website is also another important social topic that has come up the last couple years. Having open APIs and an easy way to export your data is something you should look for and if a website doesn't have one, you should ask why not. Facebook didn't even have an export ability until the middle of 2010. Here some sites and tools that I've been using to help me:

• Google Mail: http://www.gmail-backup.com/ is my recommended tool for backing up GMail. Unfortunately it's closed source but the site has been around for years and the software has seemed reputable enough. It has a simple GUI or CLI interface that allows you to select date ranges to backup; all email from 2010, and it works for me.

  Once you download the 0.107 version, you can backup all 2011 email with a command like this: (you might need an app password if you have 2-step auth enabled)

  $ ./gmail-backup.sh backup email-2011 <EMAIL> <PASSWORD> 20110101 20111231

• Gmail Contacts: While logged into GMail, you can go to the Contacts window and export your contacts in multiple formats.

• All your Google data: Google is quite good about being open, but has a wide range of services, which don't all have the same ease of exportability. While logged in, go to https://plus.google.com/u/0/settings/exportdata and you should be able to backup your G+, Picasa, and most other data Google has on you.

• Adium is an open source OSX chat client which keeps XML logs of all your conversations, which includes your FB and GChat accounts if you have them enabled. All logs should be kept at ~/Library/Application Support/Adium 2.0/Users/Default/Logs. Since it's hard to pull out specific date ranged logs, I just back the whole log folder each year.

• Pinboard: I switched from Delicious this year and have been quite happy with Pinboard for my online bookmarking needs. To export, go to http://pinboard.in/export/

For most social sites, there is a lot of money in owning your data and every site has a different obtuse backup policy. My friend Jeremy has created a one stop shop for easily and safely backing up your data from FB, Twitter, LinkedIn, Gowalla, and Google+ which I would recommend: http://export.synack.me/

Another interesting idea for controlling your online data is what Greplin provides: simple instant search of all your online data. You can easily look across all of your data, although I don't think exporting that data is something they offer.

Final Thoughts

Once you have all this data, don't skimp on your own physical backups. Storing all of your financial and personal data unencrypted in one place is a hacker's dream. This is something I am still exploring: I use OSX's built-in DMG encryption, but that is not too portable.

Also look into Dropbox for having access to your data across all your machines or Backblaze for having a simple external backup of everything on your laptop.

What’s the coolest Python application, framework or library you have discovered in 2011?

Fabric. For a good six months of this year, I was managing everything server-wise at my last job through Fabric and learned a good deal about it, hacking in some features and giving a presentation on it at the local Pittsburgh users group. It's great to be able to automate things and to have that ability in Python.

What new programming technique did you learn in 2011?

I can't think of a specific area I focused on, but overall: I've broaden my knowledge of Python unit testing, deployment, and project packaging. I've also ben diving deep into background processing with Celery lately, which really interests me.

What’s the name of the open source project you contributed the most in 2011? What did you do?

Unfortunately, I don't think I have contributed back too much. Journal, a Python script for keeping a CLI journal, was probably my most complete project for the year, but I haven't been hacking on it recently. Reaching out more via Github and interacting more with other developers is my goal for next year.

What was the Python blog or website you read the most in 2011?

HackerNews. After moving to San Francisco, getting a job via the monthly job threads, and meeting up with people from the site, I've really come to enjoy the dialogue available on programming and Python.

What are the three top things you want to learn in 2012?

• Logging, metrics, and monitoring. Or applying the latest techniques and practices in production: using JSON for logged messages, realtime monitoring and alerting, longterm storage, indexing, and searching.
• A new language. I know a great deal about Python, but it's time to start fresh. Haskell has always been high on my list, but I've never set specific goals and forced myself to start anything. Time to change that.
• More Python? I've been diving into packaging once again, but maybe I should try and figure out how to help fix what I don't like about Python. I'm looking forward to attending PyCon US 2012 this year, hoping to get some great ideas.

What are the top software, app or lib you wish someone would write in 2012?

Unify/fix Python packaging? More Python3 releases? I still need to convert all my scripts to Python3. I have a bunch of Github improvements I'd love see happen, but maybe with the v3 API I can hack them in myself.

Want to do your own list? Here’s how:

• copy-paste the questions and answer to them in your blog
• tweet it with the #2012pythonmeme hashtag

Design For Hackers
David Kadavy
ISBN: 1119998956

What’s The Point?

Design, layout, fonts, and color theory for websites and web applications. Kadavy covers all the main points of design, how design affects how we use things, and through understanding your design decisions, how you can make better decisions.

How Was It?

A useful and interesting read! The hacker in me likes well organized and straightforward things. Each of the main elements of design were given a chapter and covered thoroughly: what they are, how they work and why they exist in a historical context. There were plenty of visual examples that were current and useful.

Typography and fonts were given special preference, given the importance and history, and a practical guide for using them in most situations was given in an appendix.

Who Should Read It?

Programmers and hackers new to the topic. As an overall guide to design, it's a great starting point and covers the bases, but is not a detailed guide on any topic.

Notes:

I met the author through a GrubWithUs group meal. He was using group meals as a kind of book tour and it was an interesting experience to eat and discuss the book together! I also got my copy signed!

What's new in 1.0?

• Android Notifications of your usage via Notify My Android
• The ability to easily run the script on Heroku instead of your own server
• Now using standard config.ini configuration for your Comcast username/password

What's already there; what is this script?

• A simple way to scrape your current bandwidth usage from Comcast's website with the ability to specific warn you if are coming close to the 250GB soft limit

Download 1.0 now (zip).

Upgrading

If you have already been running script, it might be easiest to download the 1.0 and re-enter your Comcast login into the config.ini. The previous method of editing the script itself was pretty bad and left a lot of room for errors with source control.

Android Notifications

With commits from Adalan, you can now add your API key from Notify My Android and will be able to get your usage notifications via a push notification to your phone.

Attach the -a flag when running the script to enable these notifications.

Running On Heroku

This is a unique feature of the script, but was an idea I wanted to try: not everyone has a server running 24x7 or a place where they can easily run Python scripts like this daily.

With Heroku's new Cedar stack, while oriented for running web applications, you can run any Python script on a daily schedule for free. It took a bit of hacking detailed here, but wasn't impossible.

See the HEROKU_USAGE.markdown document and try running the script on Heroku!

Until bandwidth limits disappear, enjoy the script!

This repo is my attempt that.

Overall, I definitely accomplished what I set out to do! I've been very impressed with Heroku's Cedar stack and how flexible it is. It's awesome being able to script the creation of an enviroment and instantly see if my script is working or not.

With that repo, you can basically use Heroku as a free cron running system you control. The whole stack is free and if you just have to run something once a day, that's still free. If you don't have a VPS or a server running 24x7, you can now push stuff onto Heroku and not have to worry about keeping it online, it will just work.

I found an article through HackerNews recently that, while quite old, still seemed quite relevent and impressed me greatly: "You and Your Research," a transcription of a talk given by Richard Hamming in 1986, the year I was born.

Of the many great quotes and ideas in his talk, this stood out to me:

"You should do your job in such a fashion that others can build on top of it"

That is the best way to describe my goal for this blog, along with the code in my github repos. It's time to accomplish great things.

Main changes include:

• Support for viewing entries on a specific date and over a specific time period
• The ability to create multiple entries at once from the command line
• Python 2.5 compatibility (thanks to Tox)

Read on for more details.

Viewing Existing Entries

I'm hesitant to add a full human-date parsing library as a dependency, since I don't need specific time based or future based dates, so I rolled my own. The date formats currently supported include:

1. previous day increments: "1 day ago", "a day ago", "5 days ago", "yesterday", or "today"
2. specific American date formats: "8/15", "8/15/2011", "5-6-2011", or "August 15"

So you can now view specific entries for a day by using the flag --view. Short tags work with these dates.

$ journal --view "yesterday"
$ journal --view "12 days ago"
$ journal --view "8/14"
# Easiest way to view today
$ journal -vt

And you can view all entries since a specific previous date, using the flag --since. This is sparse output, dates without entries will be skipped quietly.

$ journal --since "yesterday"
$ journal --since "7 days ago"

I am still thinking over adding week and month dates to the mix, along with any other reasonable human time formats. The --view and --since flags are modeled after Git's log command, I'm thinking about adding in a --until flag also, for doing specific ranges. Although, adding a week date range might cover that need better.

Viewing entries over multiple days might provide lots of output, so I would recommend using less or more to paginate.

$ journal --since "100 days ago" | less

Adding Multiple Entries

You can add multiple entries at a time from the command line, which will get added together in the journal.

$ journal "Foo was compeleted" "Bar was completed, after foo"
$ tail ~/.journal/2011.08.18.txt
Thu 10:52:01 2011-08-18
-Foo was compeleted
-Bar was completed, after foo

I felt simplifying multiple entry was useful, but a friend on Twitter remarked that using quotes felt limiting. I'm starting to agree with him and may change the main entry mode back to "single entry" mode, but capturing all args and not require quotes. It adds issues with semi-colons or single quotes getting caught by your shell, but maybe it's worth it? This process is also leading me toward a config file, with sensible defaults, but configurable options for those interested.

The Future

I think for the next version, better listing options of existing journal entries would be useful. Having a config file to control entry options and date formatting preferences perhaps. Have thoughts on what you like or don't like? Let me know in the comments!

The idea from Lyons' article is to keep a chronological log of everything you do at your job. This mirrors my own experiences with working so far: keeping meticulously notes of what I'm doing is always helpful and not too much extra work, since I always carry a notebook around with me for this purpose. Now I can keep things digital!

Install

The script is located on Github here: https://github.com/askedrelic/journal

You can install with:

$ pip install journal

The journal command should now be available on your command line and you can journal like so:

$ journal "Task foo completed"

and a ~/.journal/[CURRENT DATE].txt file will be created using the current date, with a timestamp and your entry. Just continue to enter your tasks through the days and weeks and build a record of your work.

Thought Process

For interest, here was my progression of work in creating this tool:

1. Creating a journal using Bash isn't really that hard:

   (bsd style for OSX)

   $ echo -e `date`"\n-work completed\n" >> ~/`date "+%Y-%m-%d.txt"`

2. I looked for existing projects that may have already solved this problem.

   I found work_log, a Ruby script more oriented for tracking time. I got some good ideas for how the CLI interface should work, but this program didn't quite solve my problem.

   There are existing blogging solutions, but a blog is heavy-weight and mostly something you manage through the web. I prefer KISS and keeping things on the console.

3. Then I rolled my own!

Several highlights for me included:

• The talk about cmd2, an interactive command line. Expanding the command line and making things easier for myself and others has always interested me. It had some interesting ideas on how I would almost want Fabric to work. View the talk here.

• Speaking of Fabric, there was a talk given on that also. Fabric seems to be popping up everywhere, I've even given a talk myself, viewable here! It was great to be able to compare to my talk: it seems best to give a quick rundown of Fabric and then dive into examples. Fabric's power seems hard to explain and easier to show.

• The talk on switching from PHP to Python from Emma was great, I love high level "lessons learned" style talks. View it here.

• One talk which I missed in person, but am glad for the great recorded videos, is the talk SQLAlchemy. It's a useful tool but seems to have such a steep learning curve. More help climbing that curve is great. View it here.

• I won a book on algorithms in Python! Algorithms are something I've been meaning to brush up on and I love Python, so must be serendipity? This is the book, it's been great reading so far.

I'm still flipping through some of the talks I missed, definitely some good material produced from this talk and definitely worth attending next year!