Archiving your Mail

Coupled with our quota policies, we also have an archiving policy for Faculty & Staff to help manage users data and provide a consistent and stable email platform.  The archiving policy dictates that for all staff and faculty all email older than 1 year is removed from your current mail file on the primary mail server and placed into an archive mail file located on the Lotus archive server.

This policy has been reviewed and approved by the Dean's Office and the Executive Committee.

Why archive? Since storage costs are so low, why can't I let my mail file just keep growing in size?

Yes storage costs are low so providing additional storage space is not the reason for why we have an archiving policy. Indeed the archive is happening on the server. This means that any data that is archived from your current mail file residing on a primary email server is moved to your archive mail file which resides on an archive email server.  So your total disk space usage for email storage does not change. (Note that we are leaving out the matter of the cost of backing up this data)

Archiving is performed for different reasons. The first is performance, not just for you but for all users. Mail files are in active use all the time so the mail server is steadily processing tasks (it is never idle).  Archive files are used much less and therefore can handle larger files with more data.  The more emails (and large emails) in a mailbox, the longer it takes the server to process them. The server is continually managing mail files even if you're not using them, updating views, rebuilding indexes, compacting them, archiving messages and so on. The responsiveness of your mail file is directly related to how many emails are in your mail file and how many are in the current view.  For example, someone who has only a thousand emails in their Inbox will have a quicker and more responsive experience than someone who has 40,000 emails in their Inbox. For those who access their email in ways other than RHSmith Webmail, large numbers of emails can cause the retrieval and setup of those mail clients to crawl as the server tries to process tens of thousands of emails and convert them at once.

However the performance is not just related to you. If the server is spending a disproportionate amount of time working on your mail file, then that leaves fewer resources for the other users on the server, who also have a right to expect reasonable levels of performance.  Although we have had an archive policy for quite some time, it is only now that it is being applied strictly.  Just prior to this archive initiative, a mere 5% of our users were consuming 50% of the resources of our email servers.  As the server-side archiving is applied consistently across our user base the ratio becomes much more reasonable and the performance for all rises.

 And let's not forget, there are quotas in place to help encourage people to manage their mail files more responsibly.  By archiving older email we are doing our part to be pro-active while previously people had to keep deleting important messages to ensure they stayed in quota. Now you can archive those emails so you have them for reference at a later date. (But please do keep deleting unnecessary emails)

Are there any other reasons to archive?

Yes! Data corruption, it can and does happen.  The email servers do a pretty good job of maintaining the mail files and keeping them in good shape.  However the larger the mail file with the larger number of messages, the more vulnerable the mail file is to corrupt.  Although we at Smith IT will attempt to repair your mail file as quickly as possible, there is the possibility of data loss.  Regular maintenance is run on the mailboxes throughout the week to help combat this, but overly large mailboxes are still prone to it.  While you as an individual user might be willing to risk some data loss by not archiving, we cannot allow this as we do not want corruption in one mailbox to spread or cause instability to server tasks and possibly cause the server to crash and be offline to all users.  Although theoretically there are very large limits on the size of a mail file, practically we find that at 60,000+ messages in a view (folder) that issues can start to occur and at 120,000+ messages replication can begin to fail.  This would mean that replicas of your mail file on different servers would get out of sync and if the one that is incorrect is being backed up in the event of a DR or data loss situations, we would not be able to fully retrieve your mailbox.

Why is my mail file so large and what can I do to help manage it?

Unfortunately mail file sizes seem to grow more quickly each year.  Initially this used to be due to the increasing size of attachments.  A few years ago an email would have one or two document attachments and be about 50kb in size. Now people send large PDF files, multimedia files etc and can be many Mb in size.

This requires users to manage their mail files more proactively with some simple tasks such as:

l        When replying to emails, remove attachments. Both you and the recipient already have the attachment in the original email so why store another copy in your Sent folder?

l        Deleting emails that you dont need. This could even be as simple as someone sending you a 'funny video' email. Once you've seen the video, you can delete the email. Do you really need to keep it? What are the chances that you'll ever watch it again?

l        Delete or 'detach & link' attachments on your emails (more on this below).

Your email is exactly that email!  It is not a document storage system.  Email's primary function is to allow you to communicate with other people, but it is not, and was not, designed as a place to store documents.  If there are documents you need to keep, email is not the answer.  You can use the 'Detach & Link' option (under Extras > Detach & Link in your Inbox) that will save the files on an email anywhere you want and then put a link to them in the email.  This way if you have important documents or research that you need to keep, you can place them in your My Documents folder or on your K: drive for example.  Storing them in your mail file is not the best solution.

How can I access my archive? 

You can access your archive directly from the link http://archives.rhsmith.umd.edu. You can also login to RHSmith Webmail and click the Widgets menu item at the bottom below your Inbox and choose Archive from there.

Alternatively if you still have access to the legacy Lotus Notes client you can access from under your Inbox where you can find Tools. Double-click Tools to expand it and you'll see the Archive link. Clicking on this opens your Archive file (which also puts an icon on your Workspace).

April 2009 Update: Changes will be made to the archiving policy shortly that may change which users have archives. 

What does archiving do and what happensto my messages?

Archiving is where email messages that meet a certain criteria are moved from your current mail file to the archive mail file located on another mail server.  Our policy is that all email older than or which has not been modified for more than 1-2 years is archived. The folder structure of your archive will remain the same as it was in your regular mail, so email will be easy to find. e.g. If the server archives 10 messages from a folder called "Test" in your regular mail file, when you open your archive you would find those 10 messages in a folder called "Test" there too.

We will run a process to automatically archive all mail that meets this criteria at least once per week.  The process may run more frequently.  Currently the schedule is three times a week.  During the week the archiving process does not automatically compact your mail file unless a significant amount of email is archived, however the weekend archiving process will also compact your mail file.  This should not concern you unless you are near your quota.  Compaction will help with keep your mail file under quota.  If you are getting quota warnings during the week, and need your mailfile compacted before the weekend, please call the helpdesk at 301-405-2269 and they will compact it for you.

How much email can be stored in my archive?

We will store a maximum of 5Gb of emails in your archive. Including your active mailbox, this means we will store a total of up to 6Gb of mail on our servers.

What happens when I reach my limit of 5Gb in my archive?

We will transfer any email that is more than 4 years old from your archive to a DVD and give it to you in either Notes format or Outlook format (your choice). The email that is more than four years old will then be removed from your archive. You will be able to access those emails from the DVD (although we would recommend that you copy the data from the DVD to your computer's hard drive before trying to use it as this would perform much faster than reading the data directly from the DVD).

Can I archive important messages myself before the one-year old archiving happens?


Absolutely!  If you have emails that you have finished working with but still need to keep as they have important information you can archive them yourself whenever you wish. All you need to do is drag and drop them onto the archive link (or click Actions > Archive > Archive Selected Documents).   You can see a short video (35 sec) demonstrating the drag and drop method at 
http://bmgt5-notes.rhsmith.umd.edu/video/archivingmanually.wmv

Anything elsethat can help me work with my archive?

Yes. We are actively working through the mail files and archives and indexing them to make searching much easier. Instead of sorting emails by name or date etc, you will be able to search by word(s)

We are also currently investigating and plan to roll out in the next few months a technology that would allow you to be able to search your archive and mail file at the same time. The search results would contain links to documents in both files.

I use an email client other than RHSmith Webmail or Notes, how does archiving impact me?

Archiving removes the messages from your Inbox, so the type of mail client you are using will affect how it shows up in your client.

If you are accessing your email using POP, then you should be downloading the emails to your chosen email client and removing them from the server. This would keep 'emptying' your server-side mailbox. If you are not removing the messages from the server then a) you aren't managing your mail file on the server as messages will keep accumulating and b) you should see no impact on your mail client unless you set up a new POP client at which time emails older than one year would no longer be available in your client for re-download.

If you are using IMAP then this impacts you in a larger way. IMAP 'mirrors' your mail folders on the server with those on your client (similar to a Notes replica). If you delete a message from a folder on your IMAP client, it is deleted from the folder in Notes and vice versa. To an IMAP client when an email is archived it is the equivalent of the email being 'deleted' as it no longer appears in your server-side folder. The workaround for this is to regularly export your IMAP folders in your mail client to a separate data store. (e.g. in Outlook create a new PST file and export your IMAP folders from time to time (with the replace duplicate items option turned on).

Note that whether you use IMAP, POP or RHSmith Webmail, you can still access the archive through either the web or through the web at http://archives.rhsmith.umd.edu.