Zimbra 32 to 64 bit migration

Migrating Zimbra to centos 6.3

Zimbra end of life both its 32 bit and centos 5 support for their mail software.  As a result, one needs to upgrade the OS and then migrate their 32 bit zimbra software to this new 64 bit environment. While some software like MySQL has no issues with this, the 32 bit ldap software needs a lot of help and an import/export process is required to make this transition.  If you are concerned about availability and uptime for your users, this upgrade process is at odds with how you might run those services. You will incur an extended outage if you have not prepared the environment and worked through the issues before cut over. Here are the steps and what I learned  having prototyped the best and quickest method to this upgrade. In hindsight, it is a fairly straightforward process once you know how but there are a few places where it can fail also.

First you have to install exactly the same modules that was installed 5 or 6 years ago or whenever you did your production zimbra install when you install your new 64 bit version of zimbra. Here is a quick way to find those modules on the old system:

[zimbra user] %zmprov gs mail.everythingmail.com |grep zimbraServiceInstalled
zimbraServiceInstalled: antivirus
zimbraServiceInstalled: antispam
zimbraServiceInstalled: logger
zimbraServiceInstalled: mailbox
zimbraServiceInstalled: memcached
zimbraServiceInstalled: mta
zimbraServiceInstalled: convertd
zimbraServiceInstalled: stats
zimbraServiceInstalled: imapproxy
zimbraServiceInstalled: snmp
zimbraServiceInstalled: ldap
zimbraServiceInstalled: spell

It is important to install the same modules and also update the ham, spam, and admin accounts to be the same during the initial configuration as it makes things easier later.  They ask you to use the same name, ip address, etc.  I found that modifying /etc/hosts and running a private dns server with different ip addresses as the normal dns server allowed me to work on the new server while the old one continued to serve its users.  This bought me time to experiment and learn about the failures before cutting over to production.  If you don’t have the same address space you will need to do this since various files like postfix will be wrong because mynetworks will have your old ip address space and your users can not relay until you change these type of problems.

According to this official link and numerous posts by zimbra on its support board this should be painless and is well document. Well its not and fails in numerous ways which I will point out.  My environment started out as zimbra 5 open source and eventually became the networking pro version soon after.  Others have found problems and created their own solutions which did not work for me which I will explain later why that was the case.  Here is a link that is close to the solution but would not work in my case.  The assumption is that you will have an extended outage and that you can use the same ip address.  That is not realistic for my case usage.

Problems:

  • mynetworks contained the old ip addresses and values – Solution here.
  • DB_CONFIG contained 5.0.x or previous LDAP setup even though I was running zimbra 7.2.1. The fix is to edit the DB_CONFIG file and update the log path with the /opt/zimbra/data/ldap/hdb/logs path.  Since I started out as zimbra 4 that might have explained.  If you fail to do this ldap will not get started and give a very misleading error about database corruption.  The restore ldapadd commands have to be run as the zimbra user and not root or it will fail also with a misleading message.
  • amavisd.conf – uses mynetworks so it will contain your previous ip address and not the current servers. Fix the mynetwork first and postfix and other programs will have the correct value.
  • Backup and other admin functions failed because of ssh permissions and port 22 – not sure why but rebuilding keys and zmsshkeygen did the trick
  • /opt/zimbra/conf/salocal.cf – /opt/zimbra/bin/zmtrainsa could not parse a ipv6 address that was now present in trusted_networks and the old ip address was present here.

Various links:

Various links that were good backup material for this task but didn’t work for me:

Once up there are a bunch of things to tune if you are on a virtual environment as version 8 takes a lot of memory and I run my zimbra instances on 512MB and now 2GB environments.  The ldap DB is 80GB sparse file now so be careful if you rsync or copy the directory structure without accounting for sparse file options.  I need to update this article with the scripts that I created to make this quick and easy.