0x7b2.net

My personal blog about tech and stuff.

Exploring email with emacs

Thursday, 11 February 2021

Recently I stumbled upon https://sourcehut.org/ as an alternative software forge. Seeing how that site heavily focuses on collaboration through mail-lists I wanted to get started with a proper email client.

As most of my mailing would be focused on programming, and I’m started to get quite fond of emacs for writing the obvious direction was to set it up for email as well. This post will go through my configuration, hopefully provide some useful information for others wanting to try a similair setup. I intend for this post to be something of a compilation of what I’ve found when I built my setup. Check out these three posts where I’ve gotten some, in my opinion, very good tips for my setup. 1, 2, 3

Starting of you first need to install some prerequisites on your system. If you don’t already have Emacs installed, you should probably install that now as well. The packages you need to install are mu, called maildir-utils in some package managers, and isync which contains mbsync. If you’re on arch you can install goimapnotify from the aur, otherwise you’ll have to install it manually which fortunately shouldn’t be that hard since it’s contained in a single binary.

Also this post has been sitting as todo for way to long now, so here it goes.

mbsync

Start with installing the package:

Fedora (and other dnf-based distros):
dnf install isync
Debian / Ubuntu based distros:
apt-get install isync
macOS (with Homebrew):
brew install isync

Once installed you need to configure mbsync with your email account, in my case that is using Fastmails servers with my personal domain. The configuration is stored in ~/.mbsyncrc, and you can find available options by running

man mbsync

in your terminal.

My current configuration looks like this:

# First section: remote IMAP account
# Replace this with your email provider if you're not using fastmail
IMAPAccount fastmail
Host imap.fastmail.com
Port 993
# Replace this with your email address.
User myemailaddress@mydomain.com
# The password is encrypted using gpg2.
PassCmd "gpg2 -q --for-your-eyes-only --no-tty -d ~/.mailpass/fastmail.gpg"
SSLType IMAPS
SSLVersions TLSv1.2

IMAPStore fastmail-remote
Account fastmail

# This section describes the local storage
MaildirStore fastmail-local
Path ~/Maildir/fastmail/
Inbox ~/Maildir/fastmail/INBOX
# The SubFolders option allows to represent all
# IMAP subfolders as local subfolders
SubFolders Verbatim

# This section a "channel", a connection between remote and local
Channel fastmail
Master :fastmail-remote:
Slave :fastmail-local:
Patterns * !queue
Expunge None
CopyArrivalDate yes
Sync All
Create Slave
SyncState *

Before you can run mbsync with this config you need to encrypt your password, for this I will be using gpg2 which should be pre-installed to most desktop Linux distributions.

From the mbsync man-page we can find the requirements for using a encrypted password:

PassCmd [+]command

Specify a shell command to obtain a password rather than specifying a password directly. This allows you to use password files and agents. The command must produce exactly one line on stdout; the trailing newline is optional. Prepend + to the command to indicate that it produces TTY output (e.g., a decryption password prompt); failure to do so will merely produce messier output.

To produce a file matching these requirements we will first create a plain-text file containing the password, do this using your preferred way, just take care to not leak it to your terminal history.

Begin with creating your plain-text file containing your password

vi ~/.mailpass/fastmail

after you’ve created your password file encrypt it using

gpg2 -c ~/.mailpass/fastmail

now that you have your encrypted password file you should remove the plain one

rm ~/.mailpass/fastmail.

Now, make sure that your account specific directories are present in ~/Maildir

mkdir -p ~/Maildir/fastmail

now mbsync should be configured to fetch your mail. Try running

mbsync -a

to make sure that it works. If everything worked as it should mbsync should now have downloaded your mail to the specified folder. You can now continue on to setting up mu and mu4e.

msmtp

While emacs has a built in smtp client I’ve chosen not to use it since emacs lacking multi threading support means that it will freeze completely while waiting for the mail to be sent. Solving this problem is as easy as sending mail through an external MTA. The downside for this being that you need to write one more configuration file with your smtp account and server information.

For this I’ve chosen to use msmtp which is easily installed through your package manager of choice. Configuration of msmtp is done through the ~/.msmtprc file. Here is my configuration for fastmail:

defaults
auth        on
tls         on

# Fastmail
account         fastmail
host            smtp.fastmail.com
port            465
from            name@example.com
user            name@example.com
tls_starttls    off
passwordeval    gpg2 --no-tty -q -d ~/.mailpass/fastmail.gpg

Since I’m reusing the same encrypted password as mbsync, I don’t need to do anything else to complete my setup of msmtp.

Next we’ll setup emacs to automatically choose this account depending on the mu4e context we’re currently in.

Mail client

When I started writing this post, I mainly intended to use mu and mu4e as my email client in emacs. However, sometime during writing I chose to switch to notmuch instead of mu for no better reason than that I enjoyed the workflow provided by notmuch a bit more.

Due to this change I will cover usage of both mu and notmuch together with emacs. All other configurations should be pretty much the same regardless of what client you chose.

mu and mu4e

Start with installing the required packages.

This can be by either installing the mu4e or the maildir-utils package. I’m running Fedora, so in my case the installation command is:

dnf install maildir-utils.

Since the package names differ slightly between distros you’ll have to find out what the package is called yourself. More info can be found here https://github.com/djcb/mu. If mu is not in your package manager you’ll have to build it yourself. I’m not going to describe the process in detail, but if you want to read more you can do so on the official mu info pages.

After installing mu and mu4e it is a good idea to init your Maildir and run a first index before you start configuring emacs. To init mu and doing your first index you can run

$ mu init --maildir ~/Maildir --my-address=address@example.com
$ mu index

your e-mails should now be indexed by mu and you’re ready to start configuring mu4e in emacs.

My configuration for mu4e looks something like this:

(require 'mu4e)

; get mail every 3 minutes
(setq mu4e-get-mail-command "mbsync  -a"
  mu4e-view-prefer-html t
  mu4e-update-interval 180
  mu4e-headers-auto-update t
  mu4e-compose-signature-auto-include nil
  mu4e-compose-format-flowed t)

;; to view selected message in the browser, no signin, just html mail
(add-to-list 'mu4e-view-actions
  '("ViewInBrowser" . mu4e-action-view-in-browser) t)

;; enable inline images
(setq mu4e-view-show-images t)
;; use imagemagick, if available
(when (fboundp 'imagemagick-register-types)
  (imagemagick-register-types))

;; every new email composition gets its own frame!
(setq mu4e-compose-in-new-frame t)

;; don't save message to Sent Messages, IMAP takes care of this
(setq mu4e-sent-messages-behavior 'delete)

(add-hook 'mu4e-view-mode-hook #'visual-line-mode)

;; <tab> to navigate to links, <RET> to open them in browser
(add-hook 'mu4e-view-mode-hook
  (lambda()
;; try to emulate some of the eww key-bindings
(local-set-key (kbd "<RET>") 'mu4e~view-browse-url-from-binding)
(local-set-key (kbd "<tab>") 'shr-next-link)
(local-set-key (kbd "<backtab>") 'shr-previous-link)))

;; spell check
(add-hook 'mu4e-compose-mode-hook
    (defun my-do-compose-stuff ()
       "My settings for message composition."
       (visual-line-mode)
       (org-mu4e-compose-org-mode)
           (use-hard-newlines -1)
       (flyspell-mode)))

(require 'smtpmail)

;; Configure sendmail to use msmtp.
(setq sendmail-program "/usr/bin/msmtp"
      send-mail-function 'smtpmail-send-it
      message-sendmail-f-is-evil t
      ;; This allows msmtp to automatically choose the correct account
      ;; based on from header.
      message-sendmail-extra-arguments '("--read-envelope-from")
      message-send-mail-function 'message-send-mail-with-sendmail)

;;rename files when moving
;;NEEDED FOR MBSYNC
(setq mu4e-change-filenames-when-moving t)

;;set up queue for offline email
;;use mu mkdir  ~/Maildir/acc/queue to set up first
(setq smtpmail-queue-mail nil)  ;; start in normal mode

;;from the info manual
(setq mu4e-attachment-dir  "~/Downloads")

;; Setup mu4e contexts. This is to enable adding multiple email contexts if needed in the future.
;; I will initially only enable my fastmail context but adding a new one shouldn't be harder than copying
;; the existing context and modifying the settings.
(setq mu4e-context-policy 'pick-first)
(setq mu4e-compose-context-policy 'always-ask)
(setq mu4e-contexts
  (list
   (make-mu4e-context
    :name "fastmail" ;;for acc1-gmail
    :enter-func (lambda () (mu4e-message "Entering context fastmail"))
    :leave-func (lambda () (mu4e-message "Leaving context fastmail"))
    :match-func (lambda (msg)
		  (when msg
		(mu4e-message-contact-field-matches
		 msg '(:from :to :cc :bcc) "name@example.com")))
    :vars '((user-mail-address . "name@example.com")
	    (user-full-name . "User Account1")
	    (mu4e-sent-folder . "/fastmail/Sent")
	    (mu4e-drafts-folder . "/fastmail/Drafts")
	    (mu4e-trash-folder . "/fastmail/Trash")
	    (mu4e-compose-signature . (concat "Formal Signature\n" "Emacs 25, org-mode 9, mu4e 1.0\n"))
	    (mu4e-compose-format-flowed . t)
	    (smtpmail-queue-dir . "~/Maildir/fastmail/queue/cur")
	    (smtpmail-debug-info . t)
	    (smtpmail-debug-verbose . t)))))

Now you can try running mu4e and sending an email to make sure that everything is working as it should in Emacs as well.

notmuch

Alternatively you can use notmuch instead of mu with the main change being that notmuch has it’s focus on using tags instead of regular folders like mu.

Start with installing notmuch, and notmuch-emacs if it’s not installed automatically with notmuch.

dnf install notmuch

Next you’ll need to configure notmuch. For a basic setup just open ~/.notmuch-config in your favorite editor and fill in the fields with your email-information and maildir folder. More in depth information on the notmuch configuration file can be found in their documentation.

Once you’re satisfied with your notmuch-config you can try it out by running

notmuch new

which will read all mail in the directory you pointed it at and store them in the notmuch database. If you have configured any special tags to be set for new email they will also be applied in this step.

If notmuch ran without any errors you can now begin configuring notmuch for emacs. Here’s my current config for notmuch in emacs:

;; Load Notmuch
(autoload 'notmuch "notmuch" "notmuch mail" t)

(require 'smtpmail)

(setq notmuch-fcc-dirs
      '(("user@example.com" . "fastmail/Sent -new -inbox +sent -unread +fastmail")))

(setq sendmail-program "/usr/bin/msmtp"
      user-full-mail-address "user@example.com"
      user-full-name "Firstname Lastname"
      mail-host-address "user@computer.example.com"
      send-mail-function 'smtpmail-send-it
      message-sendmail-f-is-evil t
      message-sendmail-extra-arguments '("--read-envelope-from")
      message-send-mail-function 'message-send-mail-with-sendmail)

This configuration will provide you with a simple setup of notmuch. Note that this configuration is only suitable for single account use. With this configuration done you can try launching notmuch by running M-x notmuch in emacs and open some emails. Now might also be a good time to try sending a test email to make sure sending works as well.

One caveat with using notmuch is that emails are only tagged in the notmuch database, and such any applied tags will not be visible on your other devices if you use IMAP. To remedy this you can install some third party solution for moving emails according to notmuch tags. Currently the only such software that I know of is afew, or simply writing scripts using the notmuch cli. Personally I don’t really like any of those solutions. Instead I have started work on my own solution. More on that below.

Now you should have email working in emacs, and if you’re satisfied with your setup you can consider yourself done here. There is however one more step that I want to cover in this guide. The next section will go over how to install goimapnotify for automatic fetching of emails as they’re sent to you.

notmuch-mvrs

Originally when I wrote this post I hade not started work on this project, that changed however and I’ve managed to get a MVP out during the holidays. Please do notice that this program is far from done, and you should not use it to move your emails outside of a testing environment. Enjoy: https://sr.ht/~llndqvst/notmuch-mvrs/.

goimapnotify

Lastly since we want to recieve emails immediately as they’re sent to us we need to setup goimapnotify. This program will listen to IMAP IDLE from your server, and automatically trigger an update of mbsync, as well as telling emacs and mu4e to re-run the index. This section takes most inforamtion from the Arch Linux wiki4

First, we need to install goimapnotify. If you’re on arch this is done can be done through the aur. If not you will need to build it yourself.

If you already have go installed all you need to do is to run go get -u gitlab.com/shackra/goimapnotify.

If you’re not on arch, and you don’t already have go installed, but you have docker or podman installed you can install just the goimapnotify binary by running either:

podman run --rm -it -v (pwd):/usr/src/myapp:Z -w /usr/src/myapp docker.io/library/golang /bin/sh -c "go get -u gitlab.com/shackra/goimapnotify && cp -r /go/bin/goimapnotify ." for podman, or

docker run --rm -it -v (pwd):/usr/src/myapp:Z -w /usr/src/myapp docker.io/library/golang /bin/sh -c "go get -u gitlab.com/shackra/goimapnotify && cp -r /go/bin/goimapnotify ." for docker.

After running either of those commands you should now have the goimapnotify binary in your current folder. You can now move this binary to any directory on your system you want, I’ve chosen ~/.local/bin/goimapnotify.

Now we need to configure goimapnotify. I’m going to put my configuration in ~/.config/goimapnotify/fastmail.json.

{
  "host": "imap.fastmail.com",
  "port": 994,
  "tls": true,
  "tlsOptions": {
    "rejectUnauthorized": true
  },
  "username": "name@example.com",
  "passwordCmd": "gpg2 --no-tty -q -d ~/.mailpass/fastmail.gpg",
  "onNewMail": "mbsync -a",
  "onNewMailPost": "",
  "boxes": [
    "INBOX"
    ]
}

Now you can try running goimapnotify -conf ~/.config/goimapnotify/fastmail.json to see if it’s working.

If goimapnotify starts and displays no errors we can now set it up as a service on our system. Since I only plan on ever checking my mail with emacs I have no need for a system wide service. Instead I’m going to use the prodigy package for emacs which will keep goimapnotify running in the background as long as emacs is running. That will however have to wait for a later time, since this post is now long overdue.


  1. https://rakhim.org/fastmail-setup-with-emacs-mu4e-and-mbsync-on-macos/ ↩︎

  2. https://www.reddit.com/r/emacs/comments/bfsck6/mu4e%5Ffor%5Fdummies/ ↩︎

  3. https://martinralbrecht.wordpress.com/2016/05/30/handling-email-with-emacs/ ↩︎

  4. https://wiki.archlinux.org/index.php/Isync#With%5Fimapnotify ↩︎