089 Migrate in Core with Moshe, Melissa, Mike and Chx - Modules Unraveled Podcast

## Migrate in Core!

* Why was the Migrate module chosen to get into core?

This has been discussed for a very, very long time. The problem with the upgrade path is it runs Drupal 8 code against a Drupal 7 database so a lot of APIs need to be re-implemented in a more restricted fashion but obviously no one is too thrilled to waste time on code that runs essentially once per site at most so we tried to reuse as much code as we can with often unsatisfactory results. Code you’d think will work against a D7 database might turn out to work against a D7 *test* database but not the real ones.

* How did this get into core even though Drupal is “frozen”?

Well, ask Alex Pott. He raised the idea at DrupalCon Prague. You should remember Drupal 7 was held back by the upgrade path by about half a year. Decoupling the ready levels of any sort of data upgrade path and release is certainly a wise idea. Realizing some part of Drupal is broken beyond recognition and making a smart decision on how to solve the problem is something we love Alex for.

* What was the process involved?

Getting into core is really easy if a core committer starts the discussion. On Monday, Alex said, let’s do it, we said, great idea. He got Dries on board at breakfast on Wednesday, they announced the move at the Core Q&A on Thursday. By then I got approval from my boss at 10gen (MongoDB) to spend my paid time on this. I also got Melissa Anderson on board by this point. I really hoped funding will happen for her as well but that didn’t happen so far. Anyone interested please get in contact.

This was an easy decision -- we hit a wall with the upgrade path and we did climb that wall in Drupal 7, noone wants to do that again. Here’s an (in hindsight funny) moment about it: one of my life regrets is not capturing the face Barry Jaspan made at DrupalCon Copenhagen after an extremely fruitful discussion when he realized we do need to update disabled modules. I could’ve framed that photo and sold it with a title like “Damnation” or something like that. By the way, that particular completely ad-hoc conversation around a whiteboard, cemented my (and others) conviction of the helpfulness of core conversations (which just started at DrupalCon SF the same year). We easily shortened the D7 cycle by a month in about half an hour.

The reason MongoDB (name of the company and database both) came on board, this is an enormous opportunity for MongoDB: they paid me to make sure Drupal 8 runs on MongoDB alone, without any SQL and I believe this to be possible since about February (provided someone writes all the drivers but many are already written, actually). Unlike upgrade, Migrate uses the same APIs as a normal Drupal 8, it is just another module like any other so if you put these two together it means that without any further work, if migrate works against an SQL database as a target it will also work for MongoDB as a target. People will be able to leave behind not just antiquated Drupal versions but also antiquated databases too :)

## Use Cases

* What does the migrate process look like for sites on D6, D7, and older?

We will ship a default UI which will be super simple: you can provide database credentials, a file path, two buttons, one to migrate configuration, one to migrate content. I will be the first to admit this UI will be an alibi as it won’t work in many, many cases. After all, upgrade didn’t work either. But what happens after a fail is crucial. Upgrade function are intertwined and just impossible to reuse and themselves not working against in many installations. If user_update_8002 fails against your database, you can’t run node_update_8013 and above (this is a real world example). Migrate is different. It will with ship reusable, mostly independent migrations but even more importantly the insides of a migration are not closely coupled. We worked very hard on this. It will ship with many, many independent plugins. So, one level, if you hacked the user table apart and because of this the user migration fails, you can still migrate nodes. On the next level, if you need to write your own user source plugin that’s all you need to do, the rest of the migration might just work or need small changes. Say, your user table doesn’t have unique emails. This obviously will fail to migrate. But the migration module ships with a deduplication plugin, so you can add a few lines to the migration entity YAML, adding that to the ‘mail’ pipeline of the user migration and be done.

Getting back on how this will look like. If you decide to rebuild your site in a Drupal 8 way and only migrate users, content and such without migrating configuration, you will be able to do that either via a contrib UI or drush. This is the important part, really. So the way this is going to happen for some people: install an empty Drupal 8, try to migrate configuration, tweak the configuration, migrate content. Others will rebuild the site and then pick and choose which migrations to run. Many, especially coming from Drupal 6 will code their custom migrations. Remember the user example. It won’t be very different for custom data either. You need to write a source plugin which deals with reading from your custom table, very often, this is just a simple query. Then you need to write the migration itself using this source plugin, and using the migrate provided process plugins and of course an entity destination.

We strive to make the whole Drupal version change as painless as possible. Instead of having a “take all or nothing” upgrade path, migrate has a simple core UI, will hopefully have a more sophisticated contrib UI, will allow minimal custom code to augment existing migrations and a little bit more custom code to create new migrations. Unlike the “No Child Left Behind” program which resulted in every child left behind, we truly believe in no Drupal site left behind: come to Drupal 8, we have cookies! (And web services and mobile UI and in place editing and deployable configuration and….)

## History of Migrate Module. Who has used code before me?

Moshe to describe prior success stories about Economist, Examiner, Martha Stewart, Warner Music, ...

## Questions from Twitter

* [Damien McKenna](http://twitter.com/DamienMcKenna)

How much configuration will be migrated (variables, blocks table, etc), or is it going to be data-only?

* [socketwench](http://twitter.com/socketwench)

How different will the migrate framework be for developers in 8 compared to 7? Will there be a configuration migration path?

* [Cathy YesCT](http://twitter.com/YesCT)

#MUP089 What would make it easier for the team? What do they need (aside from eventual people to do small bits of migration/tests).

* [Cathy YesCT](http://twitter.com/YesCT)

#MUP089 For each person, what is their proudest moment so far in working on migrate in D8?

* [Damien McKenna](http://twitter.com/DamienMcKenna)

#MUP089: While working on the D8 codebase is there any consideration for backporting changes to the D7 Migrate module?

* [Cathy YesCT](http://twitter.com/YesCT)

yay! as [@1z411](http://twitter.com/lz411) said "post partial work" #MUP089 Great way to get quick good feedback that will help people learn.

* Where can people go to get involved?

## NodeSquirrel Ad

Have you heard of/used NodeSquirrel?

Use "StartToGrow" it's a 12-month free upgrade from the Start plan to the Grow plan. So, using it means that the Grow plan will cost $5/month for the first year instead of $10. (10 GB storage on up to 5 sites).