Like thousands of people online… when google decided that my domain’s email is no longer hosted for free, I spent a good year or two paying for the email accounts I have, and if like me, you only need gmail for the sake of email, paying this much for email, a technology from 1971 is a crazy !

If you are the domain admin on google workspaces, you probably know that google will allow you to “Take Out” all the data from all accounts on that domain, I will assume you already did that and you have the file (All mail Including Spam and Trash.mbox).

In a different post, I showed you how to do this with dovecot’s own tools to simply copy the files, but that assumes you run your own server ! this post is for the average user, this is the for dummies of moving email from google gmail (Takout for example) to ANY email account on any platform using a nail clipper and some old socks, just kidding, you only need the socks

So, I decided to move all accounts to a server with cpanel, which is a solid solution.

So i started with my email account, it was very straight forward with mutt (apt install mutt), configure my account, then mount the mbox file and push it all to the server

mutt -f ~/takeout.mbox (T to select all messages)

But you are probably on Windows, so for your sake, I did the second account with software that works on windows, so start by downloading and installing thunderbird, once installed, setup your account, with cpanel and most other providers, all you need to know is your name, email address, and password, thunderbird should be able to find the config for your server automatically 😉 (If you are wondering, Your server publishes it’s own config and thunderbird finds it through the email’s domain)

So, now you can see all the new emails that arrived to the new server, once that happens, close thunderbird, then navigate to your thunderbird profile folder, in my case it is at

~/.thunderbird/RANDOMLETTERS.default-default/Mail/Local Folders

But since you are on windows, it would probably be somewhere inside “roaming/thunderbird/profiles”, shouldn’t be hard to find, once inside your profile, go to /Mail/Local Folders and paste the mbox file you have, start thunderbird and it should display the MBOX file on the left panel under “Local Folders”

Once that happens, you can select the messages you want to move (If you are moving thousands of messages, i would recommend you move them in batches, not all at once), right click, then go to the moveto menu that appears, and select the account/mailbox you want them moved to, and watch them move…

Done ?

Now, you can use FTP to see your mail which should be in the devicot folders, (They are not called devicot folders, it is called maildir, but because you are running devicot, I don’t like to add jargon too quickly)

new, cur, and tmp

other files reference devicot, so we know what we have

Notes

If you need to create a new thundirbird profile, under linux you need to start thunderbird with the following command

thunderbird -ProfileManager

Under windows, the profiles page is on the Help -> Troubleshooting information menu !

google mail has the following sending limits

GMAIL accounts…

google apps account limits

The links above come down to…

1- GMAIL accounts

500 Emails per day, 500 on the receiving end, or simply, 500 recipients of 500 messages

2- Google apps accounts

500 recipients.. 100 a time when using SMTP

But, what is not stated is that a large number of bounced messages, or email messages that look like spam will also get your account disabled, i found that out on a forum on the internet,

In general, you can send email with PHP through Gmail’s servers for notification and the like, but it is best if you triple check the data before it goes to your account !

Click here to see how to send email from PHP through your Google mail “Gmail” account.

Signing mail with DKIM

This tutorial will demonstrate how you can sign your outgoing emails with DKIM (DKIM is based on Yahoo’s Domain keys and Cisco’s Identified Internet mail, It is defined in RFC 4871.)

The goal of this tutorial is to have postfix sign all mails for multiple domain names, including mail that originates from PHP’s mail function, or any other that passes the Postfix or “Postfix / Sendmail” MTA.

Debian Lenny is my Operating System, Debian’s dkim-filter’s package will sign the mail, and Debian’s Postfix package will be our MTA
1- About DKIM / How it works.

Just like SSL, DKIM uses a private key and a public key to encrypt messages.

in SSL (and DKIM) messages encrypted with the private key are only decrypted by the public key and the messages encrypted by the public key can only be decrypted by the private key.

The private key is (as in SSL) stored on the server ONLY, this is why it is private, and disclosing it to anyone else defeats the purpose, and the public key, in DKIM is stored IN DNS for the receiving server to obtain with a simple DNS request to the nameservers…

A text record in DNS will have the selector as the record name, and the key as record value.

Anyone can decrypt the message by looking up the DNS record, then using that to decrypt the message, the idea is based on the following logic.

The Logic: the sender managed to encrypt this message, so the sender must be the authentic domain owner to be able to encrypt with the private key.

2- Installing DKIM

On my Debian Lenny machine with Postfix installed

apt-get install postfix dkim-filter

3- generating Private and Public keys

Now, with our dkim-filter package, dkim-genkey is automatically installed

So our domains are example1.com example2.com example3.com and example4.com, we want to sign all emails for all mentioned domain names, In our example, multiple domains on our server will use the same public and private keys !

dkim-genkey -d example1.com -t -s dkmail

dkmail that you see at the end is the selector, you can have many selectors for more than one mail server where you do not want to share the private key between servers.

in the above, we are generating a public and a private key into 2 files in the current directory, although the command includes a domain name, the keys can be used for any set of domain names really, so not to worry about that, Now, let us create a directory to store the Private Key, and move the private key to it

mkdir /var/dkim_keys
mv dkmail.private /var/dkim_keys/dkim.private

The public key is stored in the file dkmail.txt, the contents of that file are 1 line that is as follows

dkmail._domainkey IN TXT "v=DKIM1; g=*; k=rsa; t=y; p=MIGfMA0GC...AQAB" ; ----- DKIM dkmail for anyoneofyourdomains.com

While this is a typical TXT record in BIND for example, adding this to your DNS will depend on your provider, for example when using godaddy, you can click add TXT record in your DNS manager, and enter the following into the 3 provided fields

the TXT name should be

dkmail._domainkey

The TXT value should be (The dots denote a longer string, this is just an example)

v=DKIM1; g=*; k=rsa; t=y; p=MIGfMA0GC...AQAB

And the TTL should be left at

1 hour

Now, once you hit OK, you can use any linux machine that has the DIG command to make sure your changes are already visible on the internet by issuing

dig dkmail._domainkey.anyoneofyourdomains.com TXT

You should see your public key as a result, once you do, you can move on to the next steps.

Now, Edit the file /etc/default/dkim-filter and add the following line at the bottom

SOCKET="inet:8891@localhost"

this will make the application listen on port 8891 which we will use with postfix

The other config file is /etc/dkim-filter.conf

in that file i have the following settings, you can change that if you like, my file uses the DKIM-FILTER to sign multiple domain names with the same private key, the public key is applied to all domain name DNS of all 4 domains

# Log to syslog
Syslog			yes
# Required to use local socket with MTAs that access the socket as a non-
# privileged user (e.g. Postfix)
UMask			002

# Sign for example.com with key in /etc/mail/dkim.key using
# selector '2007' (e.g. 2007._domainkey.example.com)
Domain			example1.com,example2.com,example3.com,example4.com
KeyFile		/var/dkim_keys/dkim.private
Selector		dkmail


# Common settings. See dkim-filter.conf(5) for more information.
AutoRestart		yes
Background		yes
Canonicalization	simple
DNSTimeout		5
Mode			sv
SignatureAlgorithm	rsa-sha256
SubDomains		no
#ASPDiscard		no
#Version		rfc4871
X-Header		no

###############################################
# Other (less-standard) configuration options #
###############################################
# 
# If enabled, log verification stats here
#Statistics		/var/run/dkim-filter/dkim-stats
#
# KeyList is a file containing tuples of key information. Requires
# KeyFile to be unset. Each line of the file should be of the format:
#    sender glob:signing domain:signing key file
# Blank lines and lines beginning with # are ignored. Selector will be
# derived from the key's filename.
#KeyList		/etc/dkim-keys.conf
#
# If enabled, will generate verification failure reports for any messages
# that fail signature verification. These will be sent to the r= address
# in the policy record, if any.
#SendReports		yes
#
# If enabled, will issue a Sendmail QUARANTINE for any messages that fail
# signature verification, allowing them to be inspected later.
#Quarantine		yes
#
# If enabled, will check for required headers when processing messages.
# At a minimum, that means From: and Date: will be required. Messages not
# containing the required headers will not be signed or verified, but will
# be passed through
#RequiredHeaders	yes

Now, in your postfix installation, add the following lines at the bottom of the /etc/postfix/main.cf file

# DKIM
milter_default_action = accept
milter_protocol = 2
smtpd_milters = inet:localhost:8891
non_smtpd_milters = inet:localhost:8891

Now, you should restart both postfix and dkim-filter

/etc/init.d/dkim-filter restart  
/etc/init.d/postfix restart  

Now, if you send an email to Yahoo mail, or Gmail, you can view the headers and make sure that your DKIM state is pass, if not you should recheck.

Note that SPF goes very well with DKIM and should also be implemented

1- Where are log files for help…

Links for this inp progress article

http://packages.debian.org/source/lenny/dkim-milter

http://packages.debian.org/lenny/dkim-filter

http://www.topdog.za.net/postfix_dkim_milter

http://www.elandsys.com/resources/sendmail/dkim.html

https://help.ubuntu.com/community/Postfix/DKIM

https://help.ubuntu.com/community/Postfix/DomainKeys

https://help.ubuntu.com/community/Postfix

http://www.howtoforge.com/postfix-dkim-with-dkim-milter-centos5.1

http://www.sendmail.com/sm/wp/dkim/

http://www.howtoforge.com/forums/showthread.php?p=234174

http://www.howtoforge.com/set-up-dkim-for-multiple-domains-on-postfix-with-dkim-milter-2.8.x-centos-5.3

http://www.howtoforge.com/forums/showthread.php?p=231268

http://howtoforge.org/set-up-dkim-on-postfix-with-dkim-milter-centos-5.2

http://howtoforge.org/forums/showthread.php?t=26219

Inspecting Postfix’s email queue.

This post explains how to view messages in the postfix queue, another post on this blog explains how to delete or selectively delete from the postfix queue

1- Postfix maintains two queues, the pending mails queue, and the deferred mail queue,
the differed mail queue has the mail that has soft-fail and should be retried (Temporary failure),
Postfix retries the deferred queue on set intervals (configurable, and by default 5 minutes)

In any case, the following commands should be useful

1- Display a list of queued mail, deferred and pending

mailq

or

postqueue -p

To save the output to a text file you can run

mailq > myfile.txt

or

postqueue -p > myfile.txt

the above commands display all queued messages (Not the message itself but the sender and recipients and ID), The ID is particularly useful if you want to inspect the message itself.

2- View message (contents, header and body) in Postfix queue

Assuming the message has the ID XXXXXXX (you can see the ID form the QUEUE)

postcat -vq XXXXXXXXXX

Or to save it in a file

postcat -vq XXXXXXXXXX > themessage.txt

3- Tell Postfix to process the Queue now

postqueue -f

OR

postfix flush

4- Delete queued mail

Delete all queued mail

postsuper -d ALL

Delete differed mail queue messages

(The ones the system intends to retry later)

postsuper -d ALL deferred

Delete from queue selectively

To delete from the queue all emails that have a certain address in them, we can use this program (perl script)…

NOTE: This perl script seems to be free, and is all over the internet, i could not find out where it originates or who wrote it.

1- Download this file, unzip, and upload the file to your server, then from your bash command line, Change Directory to wherever you uploaded this file, for example cd /root (Just an example, You can upload it wherever you wish)

NOTE: A second script here works differently, i have not yet tested it, download it here

Now, from within that directory, execute…

./postfix-queue-delete.pl anyaddress@example.com

Any mail that has this email address in it’s IN or OUT list will be deleted

The script uses the postqueue -p then looks for your string, once found, it deletes the email by ID, this means that this script can delete messages using any text that appears when you run mailq (or postqueue -p), so if you run it with the parameter joe all mail with addresses such as joefriend@example.com and

Other moethods exist, like executing directly

mailq | tail +2 | grep -v '^ *(' | awk  'BEGIN { RS = "" } { if ($8 == "email@address.com" && $9 == "") print $1 } ' | tr -d '*!' | postsuper -d -

——————————–

Sample Messages in a differed mail queue

——————————–

SOME282672ID 63974 Mon Nov 29 05:12:30 someaddresss@yahoo.com
(temporary failure. Command output: maildrop: maildir over quota.)
localuser@exmple.com

———————————-

SOME282672ID 9440 Wed Jun 30 05:30:11 MAILER-DAEMON
(SomeHostName [xxx.xxx.xxx.xxx] said: 452  Mailbox size limit exceeded (in reply to RCPT TO command))
username@example.org

———————————-

SOME282672ID 4171 Thu Nov 25 13:22:03 MAILER-DAEMON
(host inbound.somedomain.net [yyy.yyy.yyy.yyy] refused to talk to me: 550 Rejected: 188.xx.179.46, listed at http://csi.cloudmark.com/reset-request for remediation.)
someuser@example.com

———————————

SOME282672ID 37031 Thu Nov 25 08:53:36 someuser@example.net
(Host or domain name not found. Name service error for name=example.com type=MX: Host not found, try again)
someuser@example.com 

When i get delivery problems like the message i got today from a microsoft exchange server

I'm sorry to have to inform you that your message could not be delivered to one or more recipients. It's attached below.

For further assistance, please send mail to postmaster.

If you do so, please include this problem report. You can delete your own text from the attached returned message.

host mail1.example.com[xxx.xxx.xxx.xxx] said: 550 5.7.1 Message rejected due to content restrictions (in reply to end of DATA command)

I usually run the test on the following locations

1- the road runner website allows you to check cloudmark… run a scan with
http://postmaster.rr.com/amIBlockedByRR

tells me about
Return Path Reputation Network Blacklist :
Spamhaus (ZEN) :
Road Runner Blocklist :
FBL Enrollment :
Road Runner Internal Name-Based Block :
Cloudmark Sender Intelligence :
Return Path Sender Score Reputation Score :
historical Road Runner Internal IP Address-Based block records

Now, MXtoolbox

http://www.mxtoolbox.com/blacklists.aspx

for

AHBL
BACKSCATTERER
BARRACUDA
BURNT-TECH
CASA-CBL
CASA-CBLPLUS
CBL
IMP-SPAM
INPS_DE
ivmSIP
ivmSIP24
LASHBACK
MAILSPIKE-BL
MAILSPIKE-Z
NIXSPAM
NOMOREFUNN
PSBL
RATS-Dyna
RATS-NoPtr
RATS-Spam
REDHAWK
SEM-BACKSCATTER
SEM-BLACK
SORBS-DUHL
SORBS-SPAM
SORBS-WEB
SPAMCANNIBAL
SPAMCOP
SWINOG
TRUNCATE
UCEPROTECTL1
UCEPROTECTL2
UCEPROTECTL3
WPBL

If i can not see anything in the blacklists / blocklists above, i will generally contact the server administrator