Migrate/Import your data into Drupal 8

Drupal is a popular open-source CMS (Content Management System) for designing robust and flexible websites. It has been the preferred platform for Internet applications and enterprise-level systems. Features like flexibility, Ease of Use, Scalability, global acceptability, no licensing cost, etc. make it the best choice among others.

 

Drupal is consistently releasing new stable versions to improve the platform for producing the world’s best digital experiences.

 

To leverage the enhanced features of the latest versions of Drupal 8, organizations are switching from previous versions of Drupal or legacy CMS system to Drupal 8. Data migration to Drupal is an important step in this upgradation. If you want to migrate your older Drupal versions like Drupal 6 and Drupal 7 to Drupal 8, then it is straightforward. Since Migration has been added as a utility in Drupal Core that offers the upgrade path from Drupal 6.x and 7.x versions.

 

The migration process is an ETL (Extract-Transform-Load) process. The extract phase is called the source the, transform phase is where the work actually takes place, and the destination is the load phase, where all transformed data is accumulated. This process can be a bit complex when the legacy system is not on the Drupal platform. For this migration task, you need to hire a Drupal developer who is well experienced in Drupal development.

 

Plugins are involved in all cases, Source Plugins extract the data from the source, Process Plugins transform the data, whereas Destination Plugins save the data to Drupal 8.

 

For instance, your existing system has a Mysql database, and you want to import/migrate data into your new Drupal 8 based system.

 

As there is no standard solution using contribute modules to accomplish this; you have to write custom code and convert all data into a .CSV file and write a script to import it into your Drupal 8 System.

 

Here we present a solution for this complex scenario, without creating a .CSV file.

 

Drupal Queue to process Migration

 

Drupal Queue Worker (or the Queue API in Drupal) allows you to handle any number of tasks at a later phase. That means you can add several items to your Queue and process them later. Queue API works based on the First-In-First-Out concept. You can run Queue API manually using Cron – a background process, which runs at certain intervals of time. Drupal Queue is more efficient in handling resource-intensive tasks. This API enables you to revert the item to the queue when any failure occurs. Further, it allows you to run multiple queues simultaneously without disturbing the existing workload.

 

Drupal 8 Migration Approach

 

• Create a custom module.
• Make a connection to the existing Mysql database in the Drupal system.
• Fetch all needful data into an array and add it to Drupal Queue.
• Write a plugin to processes Queue data.

 

Step 1: Add new configuration in the settings.php file to connect your legacy MySQL database. Drupal’s database connections are defined in this setting file.

 

$databases['external']['default'] = [
'database' => 'old_app_database',
'username' => 'xxxx',
'password' => 'xxxx',
'host' => 'localhost',
'port' => '3306',
'driver' => 'mysql',
'prefix' => '',
'collation' => 'utf8mb4_general_ci',
];

 

Step 2: Create a custom module and write a controller in it to migrate data from the source database to Drupal 8.

 

Use the path like
webroot/modules/custom/custom_data_migration/src/Controller/ImportDataController.php

 

This controller will add data to the Drupal Queue. You can view all data added in Queue, by installing a module (https://www.drupal.org/project/queue_ui).

 

query($sql1);
$result = $query->fetchAll();
if (count($result) == 0) {
print "No data for import.";
}
// connect with drupal 8 database
Database::setActiveConnection();
foreach ($result as $key => $value) {
$data = ['title' => $value->title, 'body' => $value->body ];
// Add in drupal queue.
$queue = \Drupal::queue('custom_node_import_queue');
$queue->createQueue();
$queue->createItem($data);
print " - Node added successfully in Drupal Queue 
 ";
}
}
else {
// do some thing
}
$element = [
'#markup' => 'Data from , ' . $from . ' - ' . $to . 'Added in Drupal 8 queue',
];
return $element;
}
}

 

Step 3: Create a plugin to perform Queue task that will fetch data from the source and add it to the Queue.

 

Use the Path:

webroot/modules/custom/custom_data_migration/src/Plugin/QueueWorker/ImportNodeQueue.php

This plugin will have an actual node insert script. No matter how much data you have, your Drupal cron configuration will take care of your data. Whenever Drupal cron starts, it will process the item in the Queue.

title = $data['title'];
$node->body = $data['body'];
$node->save();
\Drupal::logger('custom_data_migration')->notice('node added successfully !');
}
else {
\Drupal::logger('custom_data_migration')->notice('Failed to create node As title is empty !');
}
}
}

The above code will keep you away from an error like “max_execution_time exceed”, or a blank page during script execution. You might want to see the status of imported data, for that you can install the queue_ui module and see each record in detail and more.

 

The main reason we use Drupal Queue instead of direct import is it can spread out the heavy processing over several page requests. The process is no longer interrupted due to PHP timeout. It can also reduce out-of-memory situations.

 

Hope this overview provides you with an effective solution to migrate data from SQL database to Drupal 8. Irrespective of its simplicity, this process needs a Drupal Expert who can perform the migration and debug situations where the tool is unable to proceed.

 

Author Bio: Ghanshyam Singh is an enthusiastic Drupal developer, with a passion for working on the best technology for different solutions. Having more than 10 years of experience in Drupal development, worked on 100+ Drupal-based applications based on Drupal 6,7 & 8.