Test DBs and developer workflows

[totten]

This post is mostly to verbalize some thoughts on test-DBs. CRM-14605 ( https://issues.civicrm.org/jira/browse/CRM-14605 )  aims to improve the workflow for administering the test DBs. I want to see if the proposed patch ( https://github.com/civicrm/civicrm-buildkit/pull/38 ) brings us to a world which is simpler and more robust than the status-quo. In this world, there are basically 4 things to know:

• The build-config determines what goes into a pristine DB. With build-config "drupal-clean", the "pristine DB" is a barebones Drupal+Civi installation. With build-config "drupal-demo", the "pristine DB" has Drupal+Civi with several modules, extensions, and config options.
• A pristine DB snapshot is saved as soon you build the site ("civibuild create"). It's updated automatically if you rebuild/reinstall the site ("civibuild create --force" or "civibuild reinstall"). You can manually restore from the pristine DB snapshot ("civibuild restore") or update the pristine DB snapshot ("civibuild snapshot").
• There are two physical DBs based on the same pristine DB snapshot -- CIVI DB and TEST DB. CIVI DB is used when you view a web page or run a web-test. TEST DB is used when you run a unit-test.
• CIVI DB and TEST DB are administered in tandem. If you run "civibuild create" or "civibuild restore", then the end-condition will be that both CIVI DB and TEST DB are loaded with the pristine DB snapshot.

Now, let's list out some day-to-day situations and see what happens:

• You run the unit-tests on TEST DB. The TEST DB gets munged, but that's basically OK -- as long as TEST DB keeps the current schema and required records in a few tables, you can run the unit-tests over-and-over again.
• You run the web-tests on CIVI DB. The web-tests don't systematically munge the entire DB. They may have lesser side-effects which you can often ignore (like leaving an extra contact record), but if you feel it's a problem... then run "civibuild restore".
• You make a change in the DB schema. Run "civibuild reinstall". Both TEST DB and CIVI DB will use the new schema.
• You work on an extension and want to write some installation logic. So go ahead and write some logic for the installer; then, perform the installation (eg "drush cvapi extension.install key=org.example.myext") and see what happens.  If it doesn't work, then run "civibuild restore" and repeat.
• You work on an extension and want to write some test-cases.
  
  • For a quick/temporary fix, activate the extension and run "civibuild snapshot". The pristine DB snapshot now includes your extension. Next, to ensure that TEST DB and CIVI DB are both using the pristine, run "civibuild restore". (TODO: Consider making this a single step -- as soon as you save a snapshot of CIVI DB, we should automatically reset TEST DB. Post-condition: both DBs match the new snapshot.)
  • For a more durable fix, update the build-config (or add a new build-config) which includes your extension. This ensures that the extension will be activated if you rebuild or if you install on another computer.

[totten]

I merged the test DB support into civicrm-buildkit's master branch and added some upgrade documentation:

https://github.com/civicrm/civicrm-buildkit/blob/master/UPGRADE.md

I plan to do a tag/release/announcement for another version of buildkit this month.