The pear.php.net server has been fully restored after we had to witness a fatal hard drive crash on 2015-11-29.
Our server sponsor eUKhost quickly provided us with a new machine after we told them the old had failed, and the last two weeks were spent setting it up to provide the same functionality as before:
- pear.php.net with all packages, bug tracker, API documentation and the manual
- downloads.pear.php.net containing all package files (tar and tgz)
- pear2.php.net with Pyrus and compatible packages
All those things are back again.
Why did it take so long?
I contacted eUKhost the next morning, and they had a new server available in the evening. Unfortunately it was CentOS and not Debian (which I’m more proficient with), so I asked them if they could put Debian on it. They tried again, but neither Debian nor Ubuntu had the necessary hardware RAID drivers, so we had to go back to CentOS.
Tuesday evening I began to setup the server (note that I have a day job and kids, so I did the resetup in the evenings/at night).
The sources for pear.php.net are PEAR packages themselves, originally written for PHP4. The previous server had PHP 5.3, but now with PHP 5.6 I got serious errors and had to fix the website code at first. And not only the pearweb code itself, but also the dependencies..
Getting into the old code and all the little dependencies and hidden settings (hello, PEAR_BOX constant!) took a big while to get into.
After 8 days, the new server went online with 90% functionality.
Today after 12 days, everything is restored.
Didn’t you have a backup?
The PEAR package files and the REST XML file structure got synced every 4 hours to my personal server which also acts as PEAR mirror; de.pear.php.net.
What was not backed up is the website and blog database, and the patch files attached to bugs in the tracker. I had a manual database backup from 2015-03-xx, but nothing regular. Luckily I was able to transfer raw MySQL database files before the disk fully died, so nothing was lost.
Manual and API docs could be regenerated from package files and the peardoc sources, so there is no need to back up the rendered files.
What definitely was not backed up were all the little scripts that e.g. cronjobs called to render the manual, the CHM files and such. They are lost and had to be recreated.
What else went wrong?
The firstname.lastname@example.org people switched DNS to let pear.php.net point to my mirror server, which went active late in 2015-11-30. Unfortunately my server did not have a SSL certificate for pear.php.net, so people ran into SSL issues when automatically fetching the PEAR installer via HTTPS.
My mirror server did not contain the installer (go-pear.phar and pear-install-nozlib.phar), so downloading via HTTP did also fail until late 2015-11-30.
The PEAR installer itself does not really ready to use mirror servers:
- It contacts the main server when you want to set a mirror (#11181)
- It downloads the files from the main server because the REST XML files contain full URLs with the main server’s domain name (#20995)
So the currently only workable option is to map the original hostname to the mirror IP to be able to continue using the PEAR installer (which we did by updating the DNS entry).
I’ve seen people complaining that we did not communicate enough, but I think that setting up the new server has higher priority than tweeting. Nevertheless here is the communication timeline:
- 2015-11-29 12:20 E-Mail to pear-dev mailing list, telling about the failure and providing instructions how to work around the problem.
- 2015-11-30 Tweet about the server failure
- 2015-12-07 20:56 Blog post about server restoration
- 2015-12-08 04:29 Tweet about the post
- 2015-12-11 22:26 This post
I’ll setup regular database backups in addition to the package file backups.
The new server has RAID 1, so that the failure of a single disk will not bring down the whole machine.
I don’t think that more people get involved in PEAR again; activity has been declining over the last 8 years as composer and github gained traction. Server maintenance thus will continue to be a one-man spare-time show.
Are people still using PEAR?
If you read tweets and forums you’ll get that impression that nobody uses PEAR anymore, and everyone is using composer now.
Fact is that there are a lot of old projects using PEAR, and also some new ones – we do have a couple of nice libraries that are documented, unit tested and PHP7 compatible. A lof of the packages are available on packagist.
We had 212.000 downloads in the first three days after the release of the PHP7-compatible PEAR 1.10. And no, this aren’t distro package installs – Linux distributions have their own packages.
For me, PEAR packages definitely will stay revelevant.