In Ubuntu, offlineimap installs like this:

sudo apt-get install offlineimap

Then we'll create a local Maildir dir where we want offlineimap to store our local copy of our emails:

mkdir ~/Maildir

We need to configure offlineimap. We'll create ~/.offlineimaprc, and set correct permissions on it:

touch ~/.offlineimaprc && chmod 600 ~/.offlineimaprc

Add some configuration:

[general]
accounts = gmail
ui = quiet

[Account gmail]
localrepository = gmail-local
remoterepository = gmail-remote
status_backend = sqlite

[Repository gmail-local]
type = Maildir
localfolders = ~/Maildir/Gmail

[Repository gmail-remote]
type = Gmail
remoteuser = YOUR_GMAIL_ADDRESS
remotepass = YOUR_GMAIL_PASSWORD

# Only retrieve the INBOX, no other folders for now
folderfilter = lambda folder: folder in ['INBOX']

# For now, don't delete remotely when local emails are deleted
realdelete = no

Now test your config: Run the offlineimap command in your terminal. If your config is correct, your Gmail inbox will be synched down to ~/Maildir. Be patient: this can take a while if you have a lot of unarchived email in the root inbox.

In Ubuntu, install msmtp like this:

sudo apt-get install msmtp

We'll need config for this as well: create the ~/.msmtprc file with correct permissions:

touch ~/.msmtprc && chmod 600 ~/.msmtprc

Add config (tweak to match your own credentials):

defaults
auth on
tls on

account         gmail
host            smtp.gmail.com
port            587
from            YOUR_GMAIL_ADDRESS
user            YOUR_GMAIL_ADDRESS
password        YOUR_GMAIL_PASSWORD
tls_trust_file  /etc/ssl/certs/ca-certificates.crt

Test msmtp by running the following command in the terminal (replace the email address with one of your own):

echo 'testing msmtp...' | msmtp -a gmail <RECIPIENT_EMAIL>

Did everything work? Great, let's continue.

Since our mailbox is represented by standard directories, files and strings, we'll be able to use simple Unix tools to read and manipulate emails from the command-line. We're ready to start playing with our toolbox!

This is the line we ran to verify that email sending worked after installing msmtp above:

echo 'Sent from the terminal' | msmtp -a gmail TO_ADDRESS

echo dumps the following text to STDERR. On its own, this will print the text to the terminal. We instead pipe it to msmtp, which receives the mail body on STDIN.

This one-liner counts unread emails:

find ~/Maildir/Gmail/INBOX/new -type f | wc -l

We find all files in the new folder (only files, not directories), and use wc to count how many hits we got. I think just dumping the number is a bit terse, so let's add a human readable label:

echo "Unread emails: $(find ~/Maildir/Gmail/INBOX/new -type f | wc -l)"

At this point we've got all we need to create a simple CLI-based widget to display our unread email count. We can do it by opening a terminal, and entering this command:

watch -n10 'offlineimap && echo "Unread emails: $(find ~/Maildir/Gmail/INBOX/new -type f | wc -l)"'

watch will execute the subsequent script every tenth second — synch our email, then dump out the current unread count.

We get constantly updated CLI "widget" that looks like this:

Every 10.0s: offlineimap && echo "Unread mail: $(find ~/Maildir/Gmail/INBOX/new -type f | wc -l)"          Fri Nov 30 21:14:11 2012

Unread emails: 1

Here's a simple way to check on the contents of our inbox at a glance:

grep -Rh ^Subject: ~/Maildir/Gmail/INBOX

We search recursively in our inbox for occurrences of <Line starts> Subject:. The h param makes grep only output the line, not the name of the email file.

This should match once in every email. The matching lines are spit out into the terminal, and look like this:

➜  ~  grep -Rh ^Subject:  ~/Maildir/Gmail/INBOX
Subject: Get Gmail on your mobile phone
Subject: Import your contacts and old email
Subject: Customize Gmail with colors and themes

Now we can check our inbox. Simple.

We should also be able to read a specific email. The following one-liner lets you dump out the contents of mail no. N from the top of the list above.

This one is a bit more complicated:

find ~/Maildir/Gmail/INBOX -type f | sed -n 2p | xargs cat

We find all files recursively in our inbox. We pluck out the nth line in that list (in this case number two), and pass that single filepath along to cat, which dumps out the contents of the file.

These commands work just fine, but aren't super-readable or easy to recollect. It's time to reach for shell-scripting to simplify and reuse things.

I have trouble remembering useful snippets the first time I use them. The following tricks help, though:

• You can search backwards in your terminal history by entering Ctrl-r in your terminal. Subsequent typing will display the first matching entry in your history. Push arrow up to cycle backwards through other candidates in your history. Note that this works best if you set your terminal to preserve a lot, or all, of your history between sessions.
• Personal "cheat-sheets". I've got an orgmode-file where I store handy one-liners, tools, snippets etc that I encounter, either during work, from articles and books as well as colleagues. I'm not great at retaining stuff the first time, so I like to come back and refresh or rediscover stuff later on.
• Define aliases in your shell environment. If you use Bash, create or update ~/.bashrc with lines like this:

alias helloworld="echo 'hello world'"

When you reload your environment you can use this alias like any other command. For instance, we could simplify one of our one-liners above:

alias inbox="grep -Rh ^Subject: ~/Maildir/Gmail/INBOX"

This makes checking our inbox somewhat simpler:

➜  ~  inbox
Subject: Get Gmail on your mobile phone
Subject: Import your contacts and old email
Subject: Customize Gmail with colors and themes

We'll create a script called send-email, which takes the recipient email and mail body as parameters.

#!/bin/sh

RECIPIENT=$1
TEXT=$2
echo $TEXT | msmtp -a gmail $RECIPIENT

The very first line is a shebang which tells the system how to execute the script (in this, run it as a shellscript). $1, $2 etc are variables bound to the inbound parameters. To be extra clear, we assign them to explicit variable names before executing the same command as above to send the email.

If you put this script file in your PATH you can run it from anywhere like this:

send-email EMAIL_ADDRESS "Sent from a tiny shellscript"

A bit more user friendly than the original one-liner, don't you think?

We'll port our "unread count widget" to a script called watch-unread-emails, which looks like this:

#!/bin/sh

POLLING_INTERVAL=$1
watch -n$POLLING_INTERVAL 'offlineimap && echo "Unread emails: $(find ~/Maildir/Gmail/INBOX/new -type f | wc -l)"'

The script takes one param: the number of seconds between each update. It's started like this:

watch-unread-emails 10

We'll create a script called display-inbox to peruse our emails:

#!/bin/sh

grep -Rh ^Subject: ~/Maildir/Gmail/INBOX

This is the exact same pipeline that we wrote above — only a bit more accessible since we don't have to remember that grep expression. We can simply run it like this now:

display-inbox

Our original one-liner to read a certain email was fairly complex, so let's hide that complexity as well:

#!/bin/sh

MAIL_NUMBER=$1
SED_COMMAND=$(printf "sed -n %sp" $MAIL_NUMBER)
find ~/Maildir/Gmail/INBOX -type f | $SED_COMMAND | xargs cat

The script takes "mail no. N from the top of your inbox" as an argument. We construct the sed command separately to make it more readable.

Now we can read an email like this:

read-email 2

Better, yes?

Make the threshold for writing new scripts as low as possible, and you'll end up writing more of them. That way you can't help but mold and improve your personal workflow/environment over time.

Here's two steps that will help with that:

• Create a dir in your HOMEDIR, something like ~/bin or ~/scripts. Put this dir in your PATH, making your scripts available throughout your environment. Bonus points: create a git repo of the script directory to give you version control of your scripts. Also, if you work across several machines, synch your scripts between them using Dropbox or a scheduled rsync operation.
• Create a program that makes it super simple to create new scripts. Below you'll find my ~/script/generatescript bash script. It'll take the name of the new scripts as its argument, create it in the script directory (with executable permission set), and fire up my standard editor to let me start working on it right away.

#!/bin/sh

SCRIPTPATH=~/scripts/$1
echo '#!/bin/sh
# Generated, add code here
' >> $SCRIPTPATH

touch $SCRIPTPATH
chmod a+x $SCRIPTPATH
$EDITOR $SCRIPTPATH

We'll rewrite our send-email bash-script to Ruby:

#!/usr/bin/env ruby

if ARGV.length != 2
  puts "Usage: send-email TO_ADDRESS EMAIL_BODY"
  exit 1
end

recipient = ARGV[0]
text = ARGV[1]
puts `echo #{text} | msmtp -a gmail #{recipient}`

We now start with a different shebang to make the system run the file using the Ruby interpreter.

We also add a validation of the number of parameters. CLI arguments to a Ruby program are placed in a constant, global array called ARGV. If the script is called with the wrong number of arguments we dump out a usage text and immediately exit with an error signal.

The actual execution of the msmtp we just shell out to the underlying system. This is the charm of using Ruby and other such languages for scripting: we can choose to call out to the underlying system at any time. This way, we can choose how much to lean on standard Unix tools versus the libraries and frameworks of the programming language.

Next up: watch-unread-emails.

#!/usr/bin/env ruby

if ARGV.length != 1
  puts "Usage: watch-unread-emails POLLING_INTERVAL_SECONDS"
  exit 1
end

polling_interval = ARGV[0].to_i

while true
  new_mail_dir = File.expand_path("~/Maildir/Gmail/INBOX/new/*")
  unread_count = Dir[new_mail_dir].count { |file| File.file?(file) }
  puts `clear && offlineimap`
  puts "Unread emails: #{unread_count}"
  sleep polling_interval
end

Instead of leaning on watch, we implement the same logic directly in Ruby: output unread count each nth second.

On each loop we clear our terminal of content and synch our email, then wait N seconds before we do it again. The actual unread count we find by using the Ruby File and Dir apis.

This is a bit longer than our original shellscript. However it still feels a bit more extendable and readable than the original one-liner and shellscript.

We only port display-inbox to Ruby to stay consistent here: the Ruby version of the script simple shells out the same one-liner. I find a single line of grep perfectly readable, and it makes a point: Ruby can at times be a very thin wrapper around regular shell-scripting.

#!/usr/bin/env ruby

puts `grep -Rh ^Subject: ~/Maildir/Gmail/INBOX`

Finally we port read-email to Ruby.

#!/usr/bin/env ruby

if ARGV.length != 1
  puts "Usage: read-email EMAIL_NO"
  exit 1
end

#depends on the 'mail' gem, install like this: gem install mail
require 'mail'

maildir = File.expand_path("~/Maildir/Gmail/INBOX")
all_email_filepaths = Dir["#{maildir}/**/*"].select { |f| File.file?(f) }
mail_number = (ARGV[0].to_i)-1
mail_path = all_email_filepaths[mail_number]
mail = Mail.read(mail_path)
puts mail.text_part

We use Ruby file apis to find the path with the nth mail. Then we lean on an external Ruby library (a so-called gem) called Mail to parse the email. Finally we dump the email to html to STDOUT, which leaves us with this:

<html>
<font face="Arial, Helvetica, sans-serif">
<p>You can import your contacts and mail from Yahoo!, Hotmail, AOL, and many
other web mail or POP accounts. If you want, we'll even keep importing your
mail for the next 30 days.</p>

<table cellpadding="0" cellspacing="0">
<col style="width: 1px" /><col /><col style="width: 1px" />
<tr>
  <td></td>
  <td height="1px" style="background-color: #ddd"></td>
  <td></td>
</tr>
<tr>
  <td style="background-color: #ddd"></td>
  <td background="https://mail.google.com/mail/images/welcome-button-background.png"
      style="background-color: #ddd; background-repeat: repeat-x"
    ><a href="https://mail.google.com/mail/#settings/accounts"
        style="font-weight: bold; color: #000; text-decoration: none; display: block; padding: 0.5em 1em"
      >Import contacts and mail &#187;</a></td>
  <td style="background-color: #ddd"></td>
</tr>
<tr>
  <td></td>
  <td height="1px" style="background-color: #ddd"></td>
  <td></td>
</tr>
</table>

<p>We know it can be a pain to switch email accounts, and we hope this makes
the transition to Gmail a bit easier.</p>

<p>- The Gmail Team</p>

<p><font size="-2" color="#999">Please note that importing is not available if
you're using Internet Explorer 6.0. To take advantage of the latest Gmail
features, please
<a href="http://support.google.com/mail/bin/answer.py?answer=6557&hl=en&utm_source=wel-eml&utm_medium=eml&utm_campaign=en"><font color="#999">
upgrade to a fully supported browser</font></a>.</font></p>

</font>
</html>

Raw html code isn't super readable, but perhaps we can read it in Firefox?

read-email 2 > email.html && firefox email.html

Should you stick to the simplest tools possible, or should you always jump straight to the highest available abstraction level?

While you can build anything given a Turing-complete language — see this this implementation of Tetris in sed — it's nice to step up to more expressive languages as needed.

The advantage of modern scripting-languages like Ruby and Python is less arcane syntax, and tons of useful libraries and DSLs. Modern scripting languages are also more portable than raw shell-scripting — enabling you to support Windows as well. For example: by using the Ruby File API you'll abstract away the difference between path separators, filesystem commands etc between Linux and Windows.

An downside of modern scripting languages is that they introduce additional dependencies: if you stick to standard shell-scripting and basic Unix tools, your script can function in very minimal systems without installing external packages.

I often start on small tools with a simple shellscript automation in the terminal. As soon as the script becomes a unwieldy I switch to Ruby instead.