Introduction

The here described way of importing relies on a customized version of the Migrate module. The content is not 'streamed' to the server via REST, but uploaded in a couple of XML files (one per content type) plus a folder with binaries, and then inserted by Migrate.

Prerequisites

  • A Drupal 7 installation
  • Write access to the Drupal installation (folders and files)
  • Administrator access to the Drupal CMS for turning on/off the modules and using the Migrate UI
  • PHP knowledge to customize the Migrate module to the content types you need (already done in the example, but this will be hard to test directly since we can not provide the specific Drupal installation here)
  • A dataset to migrate, stored in UDM. An exampe of this is included in the 'db' folder of the attached connector package. You can import this into MongoDB to see how the XML-generating robot works.

Unified Data Model Setup

Read about this in the MediaWiki export connector page, because that is based on the same migration project as this article about Drupal. While UDM aims at being universal, thus independant from specific information systems, that was not totally the case for this project, because the standard decorators were not defined yet at the time. Instead, the decorators have been named after the source content types and the fields after the source field names. The whole UDM definition can be viewed (and inserted into UDM if you wish) with the robot ContentTypes.xill in the package attached to this article.

Solution

The name 'mysite' or 'My Site' is chosen as project name here. You can change it to your own project name if you want, but note that this must be consistent between file names and file contents (PHP class names and such).

You might want to read this 'Solution' chapter in reverse order to connect your content from the target to the source, even though your scripts will run from source to target.

Build the import XML

This is the Xill part of the attached connector package, which you can test by running BuildImportXML.xill. To do this, first make a project in Xill IDE with the 'bots' folder of the package as root.

The content type mapping is done here with the object sourceTargetContenttypes, and the field mapping is done inside the same robot like this:

function fillPublicationDoc(doc,newdoc) {
    newdoc.field_author = doc.Publication.author;
    newdoc.field_involved_organisations = doc.Publication.publisher;
[...]
}

The field names author and publisher are chosen because they exactly matched the source field names of the source system (a WikiMedia website) in this particular project, and the field names field_author and field_involved_organisations are chosen because they exactly matched the target field names of the Drupal target system. This makes re-using the robot difficult, since your project probably does not use the same source- and target field names. Instead of hardcoding these names, for some fields (where possible) it would probably be better to give them a universal name in UDM and use functions coupled to (Excel) sheets to determine the source- and target field names. This makes a single project a bit more complex, but increases re-usability for next projects that have the same source- or target system. That is the whole idea of the universal data model after all.

For a start though, if you want to import in Drupal, you can import the database files included in the connector package in MongoDB and run de robot BuildImportXML.xill. This should write the import XML in the folder '/import/xml' that was actually used in a Xillio project, and using the debugging functionality you can see how exactly it is generated. In short, it creates a target object for every source object and then uses XML templates to generate the correct import xml.

Put the Migrate module, xml and binary files in the right place

For the Migrate modules in Drupal to find all the content it needs, the XML and binary files have to be in the correct place.

First, the modules Migrate and Migrate My Site have to be uploaded to the Drupal installation. The versions we used are included in the 'Import package' folder of the zip file. The best location for these folders is probably inside a /modules/custom folder. If there was already a version of Migrate in place, disable and remove that module from Drupal first. Once the modules are uploaded, check if you can enable them (including Migrate UI) and check if you can reach the Migrate tab from the Content page (/admin/content/migrate) without errors. If you don't see the module in the CMS, you might need to do 'Reports > Status Report > Update Database System'.

Then, the import XML has to be copied to migrate_mysite/xml and the binaries (files and images) in migrate_mysite/xml/binaries, also on the Drupal server. If everything is in place, all that's left to do is selecting a migration and clicking 'Execute'. But before that, you will need to customize the Migrate My Site module to the target content types.

Customizing the Migrate module

This might be the hardest part, and not related to Xill IDE but written in PHP. Our tweaked version of Migrate (included in the import package) might work for you as well, as long as you provide the right content types and field mappings in de custom 'migrate_mysite' module. The following files are important for that:

  • migrate_mysite.info - this contains the name of the module and a list of all files that should be included to use it. There is a base file and one file for every content type.
  • migrate_mysite_[content type].inc - one file for each content type, which contains a class that extends XMLMigrationBase from the migrate_mysite_base.inc file. There are several ways of mapping fields here:

    • addSimpleMappings - for fields that are mapped one to one, with the same name in the xml as in the target cms
    • addTaxonomy - for taxonomy terms or 'taxonomy_term_reference' (you can use strings or ids to import these)
    • addForeignKey - for entity references. Put 'true' as last parameter if you want to ignore the dependency (needed for references to the same content type)
  • migrate_mysite.migrate.inc - this file specifies migrations, migration groups and the name they have. This is linked to the migration name you have to use in the migrate_mysite_[content type].inc files for addForgeignKey, because every 'migration' keeps its own list of ids.
Here you can see whether you (or actually I) mapped the fields of content type 'Expert':
 

Drupal 7 migration

The top ones are mapped in the migrate_mysite_expert.inc file. The red ones below are not, because there was no need to import them:

 

After adding new content types (and therefore new PHP classes) to Migrate, you will probably need to 'Register statically defined classes'.

 

-

 

If you get this step right, which might take a lot of time and maybe some experience with PHP, then you should be ready for import. Mind that there are quite a lot of different properties to every field though (which values are allowed? multiple or not?) and you have to think about the order of your migrations to be able to make the references you want.

 

Closing Remarks

Hopefully, this article helps in setting up your own Drupal import. If something written here is not clear to you, please ask for a clarification.