IMAP Migration WordPress Plugin

So I turned the script that I had created earlier into a WordPress plugin. It will give you fillable boxes inside of your WP installation, and all you have to do is make sure you know your login details for both the old server and the new one. Let me know if it works for you!

It uses imap_append (part of php) including the 5th date variable so there is a 1:1 copy of your email between both servers. All emails are marked as read. If you limit it to 1000 emails at a time it has very few issues with memory, provided you don’t have extremely large attachments.

Screen Shot 2015-07-23 at 9.57.36 AM

Here is the link to download the script!

Download imap-migration.zip

Drupal 6 to WordPress Image Migration

 

I created a plugin that is a companion to a php script that I used recently – http://anothercoffee.net/drupal-to-wordpress-migration-tool/. This basically picks up where the other script left off, and migrates images after the content.  It uses WordPress’ built-in media sideload library and ties it directly to the post ID, so it will automatically be entered as a featured image on the post or page. This requires that the Drupal database is on the same server as the WordPress database or has been backed up and restored onto the same server.

Screen Shot 2015-07-23 at 10.02.43 AM

Here’s the PHP script if you want to try it out – and I have attached it as a download too:

Download migrate-images.zip

<?php
/*
Plugin Name: Migrate Images from Drupal 6 to WordPress
Plugin URI: http://www.codingconcepts.net
Description: A custom form plug-in created by Brendan Carr
Author: Brendan Carr
Version: 1.0
Author URI: http://www.codingconcepts.net

Copyright 2015 Coding Concepts

*/

function new_attachment($att_id){
$p = get_post($att_id);
update_post_meta($p->post_parent,’_thumbnail_id’,$att_id);
}

register_activation_hook(__FILE__, ‘cc_migrate_images_install’);
register_deactivation_hook(__FILE__, ‘cc_migrate_images_uninstall’);
add_action(‘admin_menu’, ‘cc_migrate_images_settings_page’);

function cc_migrate_images_install() {

}

function cc_migrate_images_uninstall() {

}

function cc_migrate_images_settings_page() {
add_management_page(‘Migrate Images’, ‘Migrate Images’, ‘manage_options’, __FILE__, ‘cc_migrate_images_admin_page’);
}

function cc_migrate_images_admin_page() {

if ( ” == DB_USER )
$db_user = __( ’empty’, ‘adminer’ );
else
$db_user = DB_USER;

if ( ” == DB_PASSWORD )
$db_password = __( ’empty’, ‘adminer’ );
else
$db_password = DB_PASSWORD;
?>

<div class=”wrap”>
<h1>Migrate Images Plugin</h1>
<p>This simple plugin will migrate images for posts based using post ID, from Drupal 6 to WordPress. Simply click the
<em>”Migrate Images”</em> below and any posts that do not have a slug will have one generated using the
post title. Has been tested to 4.2.2. </p>
<h2>License</h2>
<p>By using this plugin, you acknowledge that this plugin (and all its source code) is distributed <strong style=”color:#990000″>AS IS</strong> with no implicit or explicit warranty as to the plugin’s fitness for any specific purpose,
and is released under the <a href=”http://www.gnu.org/licenses/gpl-2.0.txt”>GPLv2</a> license. It is strongly recommended that you make a backup of your WordPress installation before using this plugin.</p>
<p>If you agree to the licensing terms above, click the Migrate Images button now to begin. The following variables have been automatically filled based on your WordPress installation. Please change them as needed!
<form action=”<?php echo str_replace(‘%7E’, ‘~’, $_SERVER[‘REQUEST_URI’]); ?>” method=”post”>
Hostname:<input type=”text” value=”<?php echo DB_HOST; ?>” name=”hostname_db” placeholder=”hostname”><br />
WordPress Database:<input type=”text” value=”<?php echo DB_NAME; ?>” name=”database_db1″ placeholder=”hostname”><br />
Drupal Database:<input type=”text” value=”<?php echo DB_NAME; ?>” name=”database_db2″ placeholder=”hostname”><br />
Username:<input type=”text” value=”<?php echo $db_user; ?>” name=”username_db” placeholder=”hostname”><br />
Password:<input name=”password_db” type=”text” value=”<?php echo $db_password; ?>” /><br />
URI: <input type=”text” value=”<?php echo $_SERVER[‘HTTP_HOST’]; ?>” name=”host” placeholder=”host”><br />
Document Root: <input type=”text” value=”<?php echo $_SERVER[‘DOCUMENT_ROOT’]; ?>” name=”docroot” placeholder=”docroot”><br />

<input name=”migrate-clicked” type=”hidden” value=”1″ />
<input type=”submit” value=”Migrate Images” />
</form>
</p>
</div>
<?php
if (isset($_POST[‘migrate-clicked’])) {

$hostname_db = $_POST[‘hostname_db’];
$username_db = $_POST[‘username_db’];
$password_db = $_POST[‘password_db’];
$database1 = $_POST[‘database_db1’];
$database2 = $_POST[‘database_db2’];

$docroot = $_POST[‘docroot’];
$host = $_POST[‘host’];
$db = mysql_connect($hostname_db, $username_db, $password_db, MYSQL_CLIENT_INTERACTIVE) or die(mysql_error());

$db_selected = mysql_select_db($database1, $db); //select WP first

set_error_handler(
create_function(
‘$severity, $message, $file, $line’,
‘throw new ErrorException($message, $severity, $severity, $file, $line);’
)
);

/*
Theory for Migration of images:

1. select all posts in wordpress database, get ID, push into array
2. take that ID and query drupal database – nid column contains post id, field_blog_image_fid contains image id
3. take image id and aquire filepath using row filepath, filemime = jpg/gif/png Ex: “sites/default/files/blog-photos/Blog 2015 07 20.jpg”
4. grab image using curl, upload to tmp folder on server
5. sideload into media library and attach to postID
*/

//Step 1:

$query1 = “SELECT * FROM wp_posts WHERE post_status LIKE ‘publish'”;
$result = mysql_query($query1);
$postids = array();
while ($row = mysql_fetch_row($result)) {
$postids[] = $row[0];
}

mysql_free_result($result); //free result set
//print_r($postids);

//Step 2:

$db_selected = mysql_select_db($database2, $db); //select WP first
$images = array();
foreach ($postids as $post) {
$postID = $post;

$query2 = “SELECT * FROM content_field_blog_image WHERE nid = “.$postID;
$result = mysql_query($query2);
$row = mysql_fetch_row($result);
//print_r($row);
$imageID = $row[2];
mysql_free_result($result);
//echo $postID.” = “.$imageID.” | “;
if ($imageID > 0) { $images[] = array($postID, $imageID); }
}
//print_r($images);
//Step 3:
$combo = array();
foreach ($images as $postimage) {
$postID = $postimage[0];
$imageID = $postimage[1];

$query3 = “SELECT * FROM files WHERE fid = “.$imageID;
$result = mysql_query($query3);
$row = mysql_fetch_row($result);

$file = $row[3];

//echo “<a href='”.$file.”‘>”.$file.”</a><BR>”;

$combo[] = array($postID, $imageID, $file);

}//for each postimage/imageid combination

//print_r($combo);
mysql_close();

//Step 4:
foreach ($combo as $postimage) {
$postID = $postimage[0];
$imageID = $postimage[1];
$file = $postimage[2];
$path_parts = pathinfo($file);
$old_uploadfolder = $path_parts[‘dirname’];

$tempfile = false;
$uploadOk = 1;

$file = “http://”.$host.”/”.$file; //for copying from another server
$file = str_replace(” “, “%20″, $file);

try {
$tempfile = file_get_contents($file);
}
catch (Exception $e) {
echo $e->getMessage().”<BR>”;
}

restore_error_handler();

if ($tempfile !== false ) {
$filebase = basename($file); //original filename minus folder structure
$uploadfolder = “/wp-content/uploads/”.date(‘Y’).”/”.date(‘m’).”/”;
$target_dir = $docroot.$uploadfolder;
$target_file = $target_dir.$filebase;
//echo $target_file.”<BR>”;

/* Image Checking */
$imageFileType = strtolower(pathinfo($target_file,PATHINFO_EXTENSION));

if($imageFileType != “jpg” && $imageFileType != “png” && $imageFileType != “jpeg” && $imageFileType != “gif” ) {
echo “Sorry, only JPG, JPEG, PNG & GIF files are allowed.<BR>”;
$uploadOk = 0;
} else {

/* End of Image Checking*/

/* Upload the Image and attach to file */

if ($uploadOk == 1) {
//Step 5:
echo “Copying to “.$target_file;

//if (copy($tempfile, $target_file)) {
if (file_put_contents($target_file, $tempfile)) {
echo “- OK!<BR>”;
} else {
echo “- Failed!<BR>”;
}

require_once($docroot.’/wp-admin/includes/media.php’);
require_once($docroot.’/wp-admin/includes/file.php’);
require_once($docroot.’/wp-admin/includes/image.php’);

add_action(‘add_attachment’,’new_attachment’); // gets the attachment ID at the time of sideload

$return = media_sideload_image($file, $postID, “Blog Image “.$postID, “src”);

remove_action(‘add_attachment’,’new_attachment’); // removes the action after it has been sideloaded

$imageID = get_post_meta($postID , ‘_thumbnail_id’, true);
print_r($return);
if (!is_wp_error($return) ) {// update post meta if no error

update_post_meta( $postID, ‘upload_photo’, $imageID );
update_post_meta( $postID, ‘_thumbnail_id’, $imageID );
echo “thumbnail id:”.get_post_meta($postID , ‘_thumbnail_id’, true).”<BR>”;

} else {
echo “There was an issue updating post meta for post id #”.$postID.”<BR>(http://”.$host.$uploadfolder.$filebase.”)<BR>”;
}

} // if uploadOK = 1

/*End of Upload*/

}// if imageFileType is GOOD

} //if tempfile returns a proper file

} //foreach image filename etc

}//migrate-clicked
}//function
?>

ModSecurity

Configured brute force protection for WordPress logins on the server today after I noticed some increasingly high CPU usage. cPHulk already monitors all of the other ports and blocks accordingly, but I needed something that would let me affect all WordPress installations at once with minimal involvement moving forward (which is why all sites have Wordfence installed too). I followed the instructions at this link (here) and within about 20 minutes it was back to normal, but I had 1060 pages of blocked IP logs in those 20 minutes. The new WHM makes it super easy to add these rules, and it handles it much faster and using less resources than WordPress.

Phew

A big day and busy week! Moving to live customer testing on the database/directory project that has been in the works for the past few months, the Excelerate Conference site has their schedule of events calendar ready to go and should be completed and launched today, I have the mail server set up and ready to go and am actively transferring about 40,000 emails (from one client) to it using a new IMAP migration script I completed yesterday. I’m also doing wine labels for the wedding, prepping another new computer to donate to the Fraser Valley Humane Society,  and tonight will be migrating the final 3 domains and 30 email users to the new server. The plug will be pulled on the old one tomorrow. We aren’t even halfway through July here, people!

IMAP Migration PHP Script

So after a bit of digging I was able to complete a working IMAP mailbox-to-mailbox migration script to help move accounts between servers. A lot of time and effort will be saved, as it is run directly on the server rather than linking your mail client to two accounts and transferring it using drag and drop. You can use it to sync your emails using IMAP, however it won’t check if emails already exist before transferring, so you will end up with duplicates.

It uses imap_append (part of php) including the 5th date variable so there is a 1:1 copy of your email between both servers. The only downside to it at this point is script timeouts and running out of memory on the server, as it uses quite a lot. If you limit it to 1000 emails at a time it has no issues.

Here’s the PHP script if you want to try it out:

<?php

$fromMboxServerPath = “{servera.com/notls/imap:143}”;
$fromMboxMailboxPath = “INBOX”;

$toMboxServerPath = “{serverb.com/notls/imap:143}”;
$toMboxMailboxPath = “INBOX”;

$fromMboxMailAddress = “username”;
$fromMboxMailPass = “password”;

$toMboxMailAddress = “username”;
$toMboxMailPass = “password”;

$fromMboxConnStr = $fromMboxServerPath.$fromMboxMailboxPath;
$toMboxConnStr = $toMboxServerPath.$toMboxMailboxPath;

//Increment these (ie: 1001 to 2000) to continue transferring
$fetchStartSeq = 1;
$fetchEndSeq = 1000;

function myLog($str)
{
echo “Log [“.date(‘Y-m-d H:i:s’).”]: $str\n<BR>”;
}

myLog(“Connecting to mailbox”);

function mboxConn($connstr,$addr,$pass)
{
if(!($mbox = @imap_open($connstr, $addr, $pass)))
{
myLog(“Error: “.imap_last_error());
die;
}
else
{
myLog(“Connected to: $addr $connstr”);
return $mbox;
}
}

function mboxCheck($mbox)
{
if(!($mbox_data = imap_check($mbox)))
{
myLog(“Error: “.imap_last_error());
die;
}
else
{
myLog(“Mailbox check “.$mbox_data->Mailbox.” OK”);
myLog($mbox_data->Nmsgs.” messages present”);
return $mbox_data->Nmsgs;
}
}

$fromMbox = mboxConn($fromMboxConnStr, $fromMboxMailAddress, $fromMboxMailPass);
$toMbox = mboxConn($toMboxConnStr, $toMboxMailAddress, $toMboxMailPass);

$fromMboxCount = mboxCheck($fromMbox);
$toMboxCount = mboxCheck($toMbox);

/**
* Loop on mails
*/

$fetchStartUID = imap_uid($fromMbox,$fetchStartSeq);
if ($fromMboxCount < $fetchEndSeq)
{
$fetchEndSeq = $fromMboxCount;
}
$fetchEndUID = imap_uid($fromMbox,$fetchEndSeq);

/**
* Loop on mails
*/

myLog(“Do stuff and backup from UID [$fetchStartUID] to UID [$fetchEndUID]”);

for ($i=$fetchStartSeq;$i<=$fetchEndSeq;$i++)
{
$pfx = “Msg #$i : “;
$h = imap_header($fromMbox, $i);
$fh = imap_fetchheader($fromMbox, $i);
$headerinfo = imap_headerinfo($fromMbox, $i);
$internal_date=date(‘d-M-Y H:i:s O’,$headerinfo->udate); //important!
$fb = imap_body($fromMbox, $i);
$message = $fh.$fb;

$msgUID = imap_uid($fromMbox,$i);

$struct = imap_fetchstructure ($fromMbox, $i);

/**
* We do some logging
*/

myLog($pfx.”Date: [“.$internal_date.”]”);
myLog($pfx.”UID [“.$msgUID.”] SEQ [“.imap_msgno($fromMbox,$msgUID).”] Flags: [“. $h->Unseen . $h->Recent . $h->Deleted . $h->Answered . $h->Draft . $h->Flagged.”]”);
myLog($pfx.”From: [“. htmlspecialchars($h->fromaddress) . “] To: [“.htmlspecialchars($h->toaddress).”]”);
myLog($pfx.”Subject: [$h->subject]”);

/**
* Transfer email using imap_append – All will be marked as read using 4th variable SEEN
*/
if (!($ret = imap_append($toMbox,$toMboxServerPath.$toMboxMailboxPath,$message,”\SEEN”,$internal_date)))
{
myLog(“Error: “.imap_last_error());
die;
}
else
{
myLog(“everything ok, mail [$fetchStartUID:$fetchEndUID] downloaded and moved in $newMailboxNameMOVE”);
}
}

/**
* End
*/

imap_close($fromMbox);
imap_close($toMbox);

myLog(“Connection closed”);

?>

Tech Support

Arguing with tech support yesterday and today – trying to explain to them that emails can be sent between accounts on the same server, which bypasses any spam filtering I have in place. They don’t get it.

This is one of the reasons I’m moving servers. I also won’t have to ask to remove blacklisted IPs, increase storage space, fix email problems, etc. It will be great all around to have this much control over everything, rather than going through intermediate tech support in India that needs you to explain every little detail.

Softaculous

I set up Softaculous on the server today. Did you know it’s only $12 a year? Only $1 a month, to save the amount of time and effort each manually created WordPress installation takes. Not to say there’s not still time involved but it is considerably less, and absolutely worth that amount.

New Server

After a lot of searching and comparison, I’ve decided to purchase a new VPS server for my clients and my own personal sites. I’ve run into a few email issues in the past few months, and despite the fact I’ve been with d9 for three years, and their service is great, I need to think about what’s best for the 50 or so domains I currently host and manage. A VPS means faster sites, more bandwidth, and more control overall of the server. I look forward to setting everything up and moving everyone over!

Hello world!

Ah yes, a little cliche, but that’s what you get from a developer.