I had some problems with Gmail, particularly emails landing in spam, that definitely weren't spam. I love the functionality of Gmail, albeit still mourning Inbox. When I tweeted about it, I got the suggestion to try a few different services, including Hey.com by Basecamp. I both remembered people clamouring for invites to Hey.com back in the day. I also remember vaguely that something was up with Basecamp.

I signed up for Hey and after two important emails directly ended up in spam and I read up on what happened at Basecamp, I decided it was time to give Gmail another shot, but make it work for me. The Hey email service promised to revolutionise email. It's opinionated approach is both a blessing and its downfall.

What I liked about Hey.com

The idea of auto-sorting email after touching them once seemed intriguing. Hey sorts emails into three buckets, namely, your inbox, a feed for newsletters and a papertrail for receipts you need searchable but never need your attention. The revolutionary idea is that hey does not use an algorithm or machine learning. Any email from a new sender does not land in your inbox but on a sorting table. There you decide where this email goes in the future.

Hey has a few other features that are neat, but that is the main feature that I liked. Touch an email once and have all subsequent email be dealt with. Without learning algorithms.

Can you auto-sort email in Gmail?

Gmail is strong on filters and labels. I love email filters.

The problem here, is that I will not go into the settings and update a filter every time I get a new email. This is in no way similar to Hey obviously, where it's one click.

I went ahead and scoured the web for some good filters and found one for the Papertrail.

subject:("Order Receipt"|"your order"|"order confirmaton"|"your payment"|"your etsy purchase"|"my account payment processed"|"your purchase"|"your gift purchase"|"thinkgeek order"|"Your threadless order"|"order from Woot"|"your amazon order"|"Return Confirmation"|"your amazon order"|"Your package"|"Order Has Been Updated"|"UPS My Choice"|"UPS Ship Notification"|"Netrition order"|"Your payment"|"Your contribution"|"order has shipped"|"Your download"|"Your Amazon.com order"|"Payment scheduled"|"bill is ready"|"order confirmation"|"thanks for becoming a backer"|"you have authorized a payment"|"changed your pledge"|"successfully funded"|"you sent a payment"|"amazon.com order"|"payment received"|"Purchase Confirmation"|"pledge receipt"|"Your TopatoCo Order"|"Humble Bundle order"|"your transaction"|"Package From TopatoCo"|"Your best offer has been submitted"|"Offer Retraction Notice"|"You've received a gift copy"|"Your Etsy Order"|"Thanks for contributing to"|"Shipping Confirmation"|"Purchase Receipt")

That was a great start but not really what I liked about Hey, where you could make a decision based on sender. So let's try and do that!

Google Apps Scripts to the rescue!

Google Scripts have a special place in the Google Cinematic Universe. It's easier to get access to some APIs than it may be with conventional tools. Additionally, you can have them run on a repeating basis.

So I had the idea of auto-updating certain filters. This is where the meat starts.

Google Apps scripts have access to the Gmail service, where you can easily retrieve filters using Gmail.Users.Settings.Filters.list('me'). The me keyword is a special access token to your personal Gmail, making it infinitely easier to work with (rather than setting up API keys).

Technically, there's no update functionality for filters. I decided to use the remove and create functionality instead. So I just have to make sure to not accidentally delete the filter and not create a new one during the auto-update.

Setting up Gmail

Gmail uses labels rather than folders, which is why I originally started using it.

I set up the Wanted, Feed, Papertrail labels to sort emails into and set up the multiple Inboxes setting as the main inbox. I'm still working on improving the system and have also added labels for social media, unwanted email and a few others that fit into my system better.

Wanted, Feed, and Papertrail are the main labels though and the script I will show you can work with any label.

I want this system to work with other labels that aren't just based on the email address of the sender. Therefore, I added another label To Sort to explicitly set a flag for emails to be used by the auto-sorting script.

Then, when I get a new email newsletter that is supposed to end up in Feed I assign the Feed and To Sort to that email. Then the script picks up this email address and it to the filter.

Setting up Filters

A long time ago, I figured that you can label Gmail filters by just including a small "text label OR actual filter". I'd normally obscure that text label, so it's never accidentally true in an actual email. Then a filter would look like {_Text_La_bel_} OR from:([email protected]). Then I never have to remember what a filter does and can simply look it up.

I kept the filter above for orders. There are a lot of emails coming from different places you order online these days. It would be redundant to add that address or addresses from other filters to the auto-update.

This is one reason, I decided to add the To Sort label.

The script is supposed to only pick up specific filters, hence I decided that the Google Apps Script below will strictly pick up filters that begin with _Auto_. To make it easier to associate auto-sort filters with a label, I then add the label name to the text label.

I wanted to make sure it doesn't accidentally pick up the word, for example _Auto_Feed (as if that would happen), so all underscores are removed from the label, like so:

{_Auto_F_e_ed_} OR ...

The Script to Sort Them All

The following script can be run. Initialize() sets up the Gmail service. Install() sets up the recurring trigger to run every 10 minutes. updateFilter() is where the meat is.

Let's walk through relevant parts in updateFilter():

1. Get All Filters in Gmail
2. Iterate through All Filters and if the filter contains _Auto_ process the following
3. Extract Gmail label name from text label in filter called autolabel
4. Search Gmail for anything that has both autolabel and To Sort
5. If we find no new emails, go to next label, otherwise extract email address and add to filter, if the filter doesn't exist yet, to try and keep filter small with unique emails
6. Remove To Sort label from email conversation
7. Gmail filters cannot be longer than 1500 characters. When the filter gets too long, split out old filters into a new label e.g. _Backup_F_e_ed_ instead. These are not updated after.
8. Update Filter with new email addresses

function Intialize() {
  return;
}

function Install() {
 
  ScriptApp.newTrigger("updateFilter")
           .timeBased().everyMinutes(10).create();

}

function updateFilter (toAddress, labelName) {

  // Get all Filters
  var my_filters = Gmail.Users.Settings.Filters.list('me').filter;
  // Get Labelname
  var my_label = GmailApp.getUserLabelByName('To Sort');
  // Iterate through filters
  for (var i=0; i", "");
        } catch(err) {
          var sender = threads[ii].getMessages()[0].getFrom();
          console.log(err + sender + ". Email was raw email.");
          
        }
        // Add Sender to Array if not exist
        filter_senders.indexOf(sender) === -1 ? filter_senders.push(sender) : console.log(sender + " already exists");
        // Remove "To Sort" label
        threads[ii].removeLabel(my_label);
      }
      // Generate the string of senders for filter
      var filter_string = filter_senders.join(" OR ");

      // If the filter is too long GMail gets upsetty
      while (filter_string.length > 1450) {
        console.log("Filter for " + auto_label + " was a bit long. Splitting into two.")
        var backup_string = filter_string.slice(0, 1400);
        var next_string = filter_string.slice(1400);
        var filter_string_index = next_string.indexOf('OR');
        var backup_string = backup_string + next_string.slice(0, filter_string_index-1);
        var filter_string = next_string.slice(filter_string_index+3);

        var backup_query = my_query.replace("_Auto_", "_Backup_").split("from:(")[0] + "from:(" + backup_string + ")";
        my_filters[i].criteria.query = backup_query;
        Gmail.Users.Settings.Filters.create(my_filters[i], 'me');
      }
      // console.log(filter_string);
      var new_query = my_query.split("from:(")[0] + "from:(" + filter_string + ")";
      // Remove old filter
      try {
        Gmail.Users.Settings.Filters.remove("me", my_filters[i].id);
      } catch(err) {
        console.log("Could not delete filter. " + err.message);
      }
      my_filters[i].criteria.query = new_query;

      
      try {
        Utilities.sleep(500);
        Gmail.Users.Settings.Filters.create(my_filters[i], 'me');
      } catch(err) {
        Utilities.sleep(500);
        my_filters[i].criteria.query = new_query + " OR {E_r_r__o_r " + auto_label + err.message.replace(" ", "") + "}";
        Gmail.Users.Settings.Filters.create(my_filters[i], 'me');
      }
      console.log("Updated " + auto_label);
    }
  }
}

Conclusion

Google Apps Scripts are surprisingly useful in the pursuit of auto-sorting emails.

I'm sure there are smarter ways to program the things above, however, for now it works well for me and has already saved me a lot of time processing email.

Now I just have to figure out what to do with all that time I used to waste on email.