Are you having e-mail synchronization problems?
Is your e-mail experience less than satisfactory? Recently my Daughter asked for help in this area , and so did one of my friends. They were both experiencing mailbox synchronization issues. My Daughter said, “I want my e-mail to work like Mom’s does, with all my devices synchronized. (MacBook Pro, iPad, iPhone and Mac desktop at her studio). Anything that I do on one should show up on all the other devices.” Her complaints include mail read at her Mac at her studio is sometimes no longer available when she turns on her MacBook at home. And something that she deletes on one device may still there on all the other devices, and requires deleting once per device. Another complaint that she and many others make is that sometimes an incoming e-mail is lost. She has her own web domain, and her e-mail service is supplied by her website hosting company, but this also happens to people that don't get e-mail through a private domain.
My friend’s concern is that he is using only Microsoft’s Hotmail as a webmail service (viewed with a web browser). He has a new Android cell phone, replacing a feature phone. He would like a coordinated view of mail on both his webmail and his cell phone. He would like to have a copy of his e-mail stored in a PC laptop client as well. This is so he can still re-read old e-mails when he is away from home and not connected to the Internet. He has experienced problems several times when creating a long e-mail using a webmail service. If the connection drops while you are typing it in, before you hit the send button, you can loose all your typing. (Using any PC or Mac based client does not have the same risk of loosing your work while typing a big e-mail.) I recommend Thunderbird as a good free e-mail client for Windows computers, and it is available for Macs as well.
Why is my Wife’s setup working so well, and my Daughter is not satisfied? The difference is due to the protocol to connect their portable devices to their e-mail service’s servers. The unhappy people are using POP3 , and people like my wife are using IMAP4 protocol without any problems. IMAP4 works with a mix of Apple portable devices and Windows computers and Android cell phone devices.
This article is for individual users, and small business e-mail services. These use either of the two dominant protocols, POP3 or IMAP4. It does not apply to people who have corporate e-mail hosted by a Microsoft Exchange server, or similar products. These can provide synchronization between several clients using the same e-mail account.
There is a large pool of about 400,000 users of Microsoft’s e-mail services who had to use POP3 with their e-mail clients. This changed after September 2013 when Microsoft added a IMAP protocol option. Many of these users could have more than one e-mail client device; and wish they had mailbox synchronization. Some of these people could have used Exchange ActiveSync (EAS) to achieve mailbox synchronization. However unlike IMAP4, EAS is not supported on many Mac and Windows e-mail clients. Now they can switch to IMAP4 and get the benefits of having all their e-mail clients synchronized. IMAP4 has been an open standard for about 20 years. It is stable and supported by nearly all Windows and Mac clients and by IOS and Android portable devices.
What is the difference between these protocols? POP3 dates from the early 1980s, when the normal situation for email was one desktop computer used with an e-mail account. It uses one way communication, and works with only one mailbox. It can allow for mail to be left on the server, or deleted when delivered to a PC (or Mac) client. It does not have any mechanism to track more than one device reading e-mail from the server. It become less satisfactory as multiple devices become more common. In this era, most e-mail hosts only permitted a small amount of e-mail to remain on a server. At that time storage costs were much higher than they are today. POP3 can only talk to one client at a time. It does not support several clients simultaneously accessing the same e-mail account. A change made by one client can't propagate to the other clients.
The IMAP4 protocol to deliver mail from a server to an e-mail client was introduced after POP3 in the early 1990s. This protocol includes the ability to keep track of mail read and or stored on multiple devices. It has two way communication to keep all clients and the server in synchronization. An e-mail read on one device is marked read on all the other devices. It supports folder style multiple mailbox organization and synchronization of mailboxes between all devices. A central characteristic is the mail remains on the server, until the user takes some action. This action is either deletion on the local device, or moving it to a private local mailbox. Mail moved from the inbox to another IMAP mailbox updates both the server and all other clients. There is another benefit, all your mail in IMAP folders is stored on the server, as well as on your laptop. This is a backup in case of your Windows or Mac client loss, or failure.
Several significant changes occurred since the introduction of POP3. Online or cloud storage for e-mail went from an expensive resource, to almost free, mostly thanks to Google. Then came a proliferation of laptop computers, and much later came along smart phones with e-mail clients. All these encouraged the trend to keep mail on servers, and to access that e-mail from more than one device.
In the 1980’s, the use of POP3 was dominant. Gradually it become common to have configurations such as both a laptop and a desktop at home, and a desktop at work. This produced a desire to have all these automatically synchronized. This is something that POP3 was not designed to handle. There are partial solutions, such as set all clients to “leave mail on server”. Set up one client so that when it deletes mail items locally, that same item is also deleted on the server the next time the client connects. There is one function that this could not help, enabling a user to organize mail into different mailboxes. This type of organization is not synchronized between clients, and so is repeated on each client. The mail items stored on each client, and the mailboxes in use and the specific items in each mailbox tend to be different.
E-mail providers liked POP3 since it was a simple protocol, and presented a minimal load for their servers. Also important on dial up lines, less data was transferred between the host and the client than with IMAP4.
The use of the newer IMAP4 protocol provided an elegant solution for this desire for synchronization. There was a long delay before this become supported on popular free e-mail services. In 2007 Google added IMAP4 protocol to its Gmail servers. This enabled people using free mail services to enjoy mailbox synchronization across multiple devices. Microsoft’s Hotmail, Windows Live Mail and Outlook mail only added this capability in September 2013! Thus there are many Hotmail clients who started using Hotmail before Sept 2013, and had no choice but to use POP3 when they started. Many of these are not aware of the new capability, or did not understand how it would benefit them, and have still not changed to using IMAP. This is a vast pool of e-mail users who still have the frustrations of using POP3 with multiple e-mail clients.
Another interesting fact, the mail client built into Windows 8 does not support POP3. They support IMAP and Exchange Active Sync . That is due to the desire to have live tiles with real time e-mail notifications for the e-mail client on the Modern interface.
Now this history lesson is over. How does an unhappy e-mail user achieve this mailbox synchronization? If you are using Gmail, Outlook, Windows Live Hotmail, Windows Mail, or AOL Mail, change the protocol enabled at the server to IMAP, and disable POP3. Look up the client IMAP settings for your e-mail provider’s servers. Enter these settings in each of your smart phone, PC client, etc.. Now you are nearly done, wait a while as each device synchronizes, then you are all set.
If you store all your e-mails locally in Apple Mail, Outlook Express, Outlook client or other mail client on your computer, you can transfer all your existing e-mails to Gmail's server. You need a specific directions for each one of these which is beyond the scope of this article, but is available on the Internet. The general approach to do this is as follows:
If you have locally stored mail you wish to keep that is not on your current server, these are my recommended steps. This also applies if you forward your existing mail to Gmail, which will be mentioned later on. Choose the first device you switch to IMAP as your temporary master device, usually a desktop or laptop. It should be the device with the most local mail stored on it. First backup you mailboxes, tidy them up, and then switch to IMAP. Only the inbox is synchronized. Do this on a WiFi connection, as a lot of data will be up and downloaded as initial synchronization occurs. If you have other local mailboxes you would like synchronized, follow these steps for each extra mailbox:
• Create a new IMAP mailbox for each one.
• Copy all the mail from a local mailbox into the new IMAP mailbox.
• Wait while the mail uploads to the server, or advance to the next mailbox to synchronize.
When using Gmail, new IMAP mailboxes are sub-folders of [Gmail]. Once the master device is fully synchronized, then add a new e-mail account on another device. Let it synchronize to match your master device. Repeat this on all your other devices. If you don't use a new account on each device, then you will get duplicate e-mails for each item in your 2nd device's inbox that is not unique. Now there is no longer a master client device. The server has all the mail and is your new master. Depending on your settings, some Windows and Mac clients may have a copy of all of that mail. Other portable devices may only have a selectable number of days of most recent mail stored locally.
What do you do if you are on a less popular e-mail service that does not support IMAP, or on Yahoo! Mail? There is a great solution that does not need any change your e-mail address! First create a Gmail account and enable the IMAP protocol (POP3 is the default). Next go to your original e-mail server’s settings and turn on forwarding all your e-mails to your new Gmail account. Program the original server to not store a copy of the e-mail after forwarding it. Now change all your e-mail reading clients to point to your Gmail account using IMAP and you are done. On an iPhone or Android phone, all you have to do is tell it to setup a new e-mail account, pick Google as the account type, and type in your Gmail address and password. All your new mail will start arriving via Gmail, and you no longer read your old service with POP3. Everyone can still use your current e-mail address, which might even be a private domain, while you receive it from Google. Now you enjoy the mailbox synchronization due to the use of IMAP protocol. You can still access your mail via a webmail service. The only change is you point a web browser to Gmail when you are away from all your own devices, and are using someone else’s computer or iPad.
In the previous paragraph, Yahoo mail is mentioned. They support IMAP for mobile devices, but not for desktop or laptop computers. Yahoo’s official statement from Oct 2013 is:
“NOTE: These settings are only meant to be used on a mobile device and are not supported on a desktop email client. If you decide to use IMAP on a desktop email client, we will be unable to provide support. Some email clients do not allow the ability to disable ActiveSync, which can cause emails to be deleted. If emails are deleted from the desktop email client while using IMAP, they cannot be recovered.”
This shows that Yahoo does not have a full standards compliant implementation of IMAP4. There are work arounds for this. Forwarding it to Gmail which has had a working implementation of IMAP for 6 years is probably more robust.
If you forward your mail to Gmail, you can setup Gmail to transfer a copy of all the existing mail on your Hotmail server, or similar account into your new Gmail account. Then you can access your older mails on your smartphone, laptop, etc.
If you forward your existing incoming e-mail to Gmail, you can still send your mail from e-mail clients through your existing server's SMPT service. No change is required. But these are the disadvantages of not changing:
- Only the sending device has a copy of mail sent from that device. (No sent mail synchronization)
- In a public hotspot, it is possible for others to eavesdrop on your sent mail, if you use port 25.
- If you send using SMTP on port 25, you can’t send e-mail when away from home and using another ISP's (Internet Service Provider) connection. (Receiving mail and web surfing are fine. Sending using webmail works.)
However I highly recommend you change to sending through Google's SMTP server. Gmail can be set to provide the reply to and from address of your original e-mail address. You have to go through a few simple steps to prove to Google that you own that e-mail address. The benefits of sending through Gmail are:
- Gmail stores and backs up a copy of your sent mails, these can be searched by you.
- You can keep a copy of your sent mail in a "Sent Mail" IMAP folder on all your devices. This gives you the same benefits of synchronization as you get for received mail.
- Your sent mail is always encrypted and private, even when using a public hotspot.
- You can send mail using any ISP when away from home.
This last point is a great solution to a subtle issue many people experience. You may wish to skip the details, but for those who are curious here is the explanation. When you are away from home and using another ISP, you can surf the internet, receive e-mails using an e-mail client but can't send any. This occurs if your current ISP’s e-mail server requires using SMTP on port 25. Most ISPs block SMTP traffic on port 25 , unless it comes from an authorized client of that ISP, and it uses that ISP's SMPT server. (Bell, Rogers, Telus, Shaw, Cogeco all do this as an anti-SPAM sending blocking measure.) Gmail does not use port 25, and only sends from an authenticated client. Google has many measures to prevent its SMTP servers from sending SPAM. Other ISPs will not detect the mail sent via Gmail as a SMTP relay attempt, and won't try to block it.
Apple Mail users of Gmail also need to map Gmail's IMAP mailboxes to the names of mailboxes used by Apple mail. This takes about 2 minutes, with details available on Google's help.