Mail.app wouldn't stop asking about the certificates.
So:
Bingo! Well - works4me at least.
I was using courier-imap and courier-pop until today. I've just installed the -ssl versions of both.
At install they both create a certificate for localhost based in New York. This doesn't work unless it really is localhost.
In /etc/courier there are imapd.pem, imapd.cnf, pop3d.pem and pop3d.cnf
I left the non-ssl IMAP running but only for 127.0.0.1 for squirrel
An article popped up on debian-administration.org. I'm going to have to take a closer look at one of the comments:
http://www.debian-administration.org/articles/302#comment_6
If this can work well with courier (which I guess it should) then this will solve an issue I've had a while (and I don't want to go the route of a database for this).
Edit: Various things that have come up under discussion
courier assumes $HOME - to change this you need to change the authenticator so that it changes the value of $HOME (at least according to various google searches)