Migrating Generic IMAP to Gmail Hell

Oct/29/2014

I have a friend who is an attorney who keeps tons of email. He's been having issues with reliablilty on the shared hosting account he is on so I suggested we move him to Google Apps. Then I discovered he has nearly 15,000 email messages stored. A bit of a shocker, but it's Google. They have made migrating to Google Apps a breeze, right?

Wrong.

It has been a complete nightmare. We tried just dragging and dropping folders using his Apple Mail client. It was moving one message every three to five minutes. At that rate it would have taken just about one month to migrate all 15,000 messages. So we tried Google's tools. Big mistake. The migration tool available on the apps dashboard pretends like it's working, but it doesn't. After 2.5 days, it reported having moved all his email except it didn't. It moved two out of eight folders and only about 8,000 of the 15,000 messages. Nice job, Google.

They have a Windows desktop tool that is supposed to be better, but I couldn't make it work. It refused to accept his secret key and would bail without doing anything. Once, apparently randomly, it actually got past the secret key check, but then couldn't create any of the necessary labels and quit. I read and re-read all the support docs I could find, but could find anything that would explain why the secret key was being rejected.

Next I tried talking to these guys on the advice of the one of the IT guys at work. Apparently we're using them for a big domain migration we're undergoing right now. Supposedly they are the best 3rd-party migration service out there. There's just one catch. Their software is licensed on a per person basis and you have to buy a minimum of 10 licenses. I contacted them and asked if their free trial would work to get one person migrated. They responded that I could buy a license for one person. Except that you can't. The web site won't let you buy fewer than 10. I tried chatting with someone on their web site to see if I was missing something. He told me the minimum was 25. Do they not even know how their own site operates? I contacted them again saying I couldn't see how to order just one license. I explained that I was happy to pay for one person, but I didn't want to pay for 10 if I only needed one. Their response? Nothing. So, if you're small potatoes, don't waste your time.

I had some PHP code lying around from a time in the past when I needed to migrate email from one server to another. I didn't use it then, but decided that was my next best option. Problem: it was apparently some seat-of-the-pants coding. It was mostly broken and got some things flat out wrong. So after several hours and much trial and error, I finally got it working. Seemed to work like a charm. The folder I was testing with had about 1200 messages in it and moved to Gmail in just a few minutes. New problem: in the message list view, all the dates were being displayed based on the date the messages were moved, not the date of the last message in the conversation. Even though you can specify an "internal date" in the PHP funciton call—which some seemed to think should do the trick—it wasn't working. Even with the date parameter set and properly formatted, Gmail kept right on displaying the date of the copy, not the date of the last message. To be clear, the actual dates on the message were not affected. You could open the message and see the correct date, but that's just annoying.

This morning, I started Googling for a solution to the date problem...ironic isn't it? Apparently this has been an issue for years. This thread was started in 2009. TL;DR: use Thunderbird. So after weeks of fussing with Google's migration tools, several support calls to Google, and a failed attempt to engage with a third party, I fired up Thunderbird, pointed it at both accounts and in less than a day I have all 15,000 messages moved to Gmail. Apparently not all IMAP clients are created equal. For whatever reason Thunderbird plays nicely with Gmail.

What really bugs me about this whole thing was the long, painful process. It wasn't until I tried to write my own code, ran into an issue with my code and started looking for a solution to that specific problem that I finally found a workable solution. Why, when I spoke to the first support rep at Google about migrating this enormous quanity of email, didn't he just say, "Hey, our tools suck. Just use Thunderbird. You'll be much happier."

Hopefully, some day in the future, someone else will be trying to migrate to Gmail and will find this post sooner rather than later in his or her process.

Blast From the Past

Sep/12/2014

LiveScript, Silicon Graphics, IRIX, Solaris, NCSA Mosaic, IE3… It was kinda fun listening to Brendan Eich talk about the evolution of JavaScript. Lots of names I hadn't heard in a long time. I found it quite interesting to hear all the political machinations that surrounded the creation and development of JavaScript. It's worth a listen: for historical edification if you're a young developer, or for a bit of nostalgia if you've been around for a while.

JSJ The Origin of JavaScript with Brendan Eich

CSS Pet Peeve #17 Redux

Sep/03/2014

Found this on another site.

.dropcap, blockquote, .team-member .member-mask .mask-text fieldset legend,
.button, button, .coupon .button, input[type=submit], .font2,
.shopping-cart-widget .totals, .main-nav .menu > li > a,
.menu-wrapper .menu .nav-sublist-dropdown .menu-parent-item > a,
.fixed-header .menu .nav-sublist-dropdown .menu-parent-item > a,
.fixed-header .menu > li > a, .side-block .close-block, .side-area .widget-title,
.et-mobile-menu li > a, .page-heading .row-fluid .span12 > .back-to,
.breadcrumbs .back-to, .recent-post-mini a,
.etheme_widget_recent_comments ul li .post-title, .product_list_widget a,
.widget_price_filter .widget-title, .widget_layered_nav .widget-title,
.widget_price_filter h4, .widget_layered_nav h4, .products-list .product .product-name,
.table.products-table th, .table.products-table .product-name a,
.table.products-table .product-name dl dt, .table.products-table .product-name dl dd,
.cart_totals .table .total th strong, .cart_totals .table .total td strong .amount,
.pricing-table table .plan-price,
.pricing-table table.table thead:first-child tr:first-child th,
.pricing-table.style3 table .plan-price sup, .pricing-table.style2 table .plan-price sup,
.pricing-table ul li.row-title, .pricing-table ul li.row-price,
.pricing-table.style2 ul li.row-price sup, .pricing-table.style3 ul li.row-price sup,
.tabs .tab-title, .left-bar .left-titles .tab-title-left,
.right-bar .left-titles .tab-title-left, .slider-container .show-all-posts,
.bc-type-variant2 .woocommerce-breadcrumb,
.bc-type-variant2 .breadcrumbs, .post-single .post-share .share-title,
.toggle-element .toggle-title, 
.product-thumbnails-slider .slides li.video-thumbnail span,
.coupon label, .product-image-wrapper .out-of-stock,
.shop_table .product-name a, .shop_table th, .cart_totals .order-total th,
.page-heading .row-fluid .span12 .back-to {
    font-family: Montserrat;
}

And it dawns on me: no human being did this. Right? This has to be CSS generated by some GUI. And here you have an execellent example of why I have never used GUIs to build web sites. I have never seen a tool that produces what I would even consider reasonable code. I remember looking at one mumble mumble years ago that used tables and single pixel gifs with various heights and widths applied to fix the layout. The markup was a sight to behold, let me tell you.

I get that it's a difficult problem to solve. No disrespect to those trying to crack this nut. Somehow I think a GUI that produces clean-ish code is going to take some serious AI to get it right.

CSS Pet Peeve #17

Aug/21/2014

It's amazing how often I see something like this:

.btn-submit, .btn-save-changes, .btn-send-email, a.btn-see-addresses,
a.btn-see-cards, a.btn-see-order-history, a.btn-see-wish-list, a.btn-share-wishlist,
a.btn-add-new-address, a.btn-add-new-card, a.btn-share-your-wishlist,
a.btn-continue-shop, .btn-update, .btn-add-wishlist, .btn-tell-friend,
.btn-continue-blue, .btn-no-thanks, .btn-add-wishlist-blue, 
a.joinnow, .btn-check-status{
    background: url("../images/buttons_24.png") no-repeat;
    border:0 none !important;
    cursor:pointer;
    display:block;
    height:25px;
    line-height:25px;
    text-indent: -9999px;
}

First: don't do that. Using a comma to apply the same rule to different styles is fine if you have two, maybe three, styles where it applies. Doing it with this many styles is just absurd. Oh, and I've probably removed a dozen rules from this list already.

Second: This used to be all on one line. Also bad. I know guys who write their rules on one line a la

a.btn-see-addresses{width:135px;background-position:-1697px 0;}

One fellow I know who is a proponent of this style says he likes to being able to see lots of class definitions as he scans down the page. Another says he's usually looking at rules in Chrome dev tools and not the CSS file itself. This way his CSS is already compact and he can skip some kind of build tool to compress his CSS. To each his own, I guess, but I'm not a fan. I think they're really hard to read. Of course, either argument falls apart when you have so many class names on the line you can't even see the rules. I had no idea there was an !important in these rules until just now when I broke it into multiple lines. Which brings me to...

Third: If you have to use !important you've done something wrong.

Here's one way to tackle this particular issue.

.btn-sprite {
    height: 30px;
    line-height: 30px;
    border: 0;
    overflow: hidden;
    text-indent: 500px;
    display:block;
}
.btn-yellow {
    background: url("/images/btn-sprite-yellow.png") 0 0 no-repeat;
}
.btn-red {
    background: url("/images/btn-sprite-red.png") 0 0 no-repeat;
}
.btn-checkout {
    background-position: 0 0;
    width: 140px;
}
.btn-checkout:hover {
    background-position: 0 -30px;
}

<button class="btn-sprite btn-yellow btn-checkout"></button>

I find this much easier to read and maintain. There's nothing wrong with having more than one class on an element. I try and draw the line at three, but that's just me.

Browser Quirks

Aug/12/2014

Here's a new oddity between Firefox and Chrome that I haven't come across before. I've removed the logo in question because I don't know how the company would feel about having their logo on my web site. You'll have to take my word for it that the black line indicates the baseline of the logo on the page.

Notice how the menu renders one pixel lower in Firefox. Now the typical developer loaths pixel pushing and is likely to respond "One pixel? Pffft." The problem is when you have a line on the page—a margin of some kind, a font base line— the human eye will pick up that kind of thing and it does become distracting. There are a few situations where pixel pushing is appropriate. Here's the relevant CSS.

.htlink {
    position: relative;
    top: 36px;
    line-height: 24px;
    height: 24px;
    letter-spacing: .14em;
    text-transform: uppercase;
}

If you look a the computed styles in each browser you can definately see some precision differences in the letter spacing numbers displayed by each browser. It's not unreasonable to assume there are similar precision differences when calculating the pixel location of the text baseline. So what to do? Browser detection and Firefox specific styles? If that doesn't make you cringe, it should. In this case, however, there was only one minor tweak that needed to be done.

line-height: 23px;

That had the desired effect of moving the font up one pixel in Firefox, but leaving the font in place in Chrome. I think I just got lucky this time, but if you face a similar situation, you might try tweaking your line-height or other font property that would exploit the precision differences between the two browsers.