CiviCRM Community Forums (archive)

I'm enamoured with the idea of Behat functional tests, human-friendly descriptions of things that should be happening on a site which can be tested automatically.

In particular, I'd like to be able to test the integration of my Drupal site with Civi. Civi itself is so big that I can't realistically afford to test it's UI. But I can test the various bits of mission-critical end-user exposed functionality on my site which are supposed to do something in Civi.

For example, when someone completes a donation page (civi profile), I'd like to test that this contribution is actually appearing in Civi.

Feature: Make  donation.
   In order to power fundraising
   As a site visitor
   I need to make a donation via the donate page.

Scenario: Make donation (simplified!)
  Given I am on "/"
  When I follow "Donate"
    And I fill in "Amount" with "10"
    And I do not check "Recurring"
    And I press "Donate"
   Then I should see "Thankyou"
    And there should be a new contribution record with amount "10" and recurring "FALSE"

(I'm sure' that's not the right fields for civi contribution records but you get the idea ...)

Drupal's behat extension should AFAIK already allow me to see that the user gets the thankyou page, but it won't know how to assert the presence of a contribution record. To do that, we need to teach it some new Behat step definitions that assert against the civi database(I think using PHPUnit). (My understanding comes from here: https://github.com/silverstripe-labs/silverstripe-behat-extension/blob/master/docs/tutorial.md) I'm planning to get a developer to create these and contribute them.

Does this sound like a good idea?

If so, any suggestions for what would be the most helpful new steps to have defined?

I'm interested in definitions that would power steps like:
- Given I (do not) have a contact synced with my Drupal account
- Given there is no contact with my email "fred@bloggs.com"
- Given the civiCRM test email system is enabled
- Then there should be a new contact with email "fred@bloggs.com" and with name "Fred Bloggs"
- Then there should be a new (recurring) contribution record (for me) (with amount "") with  and ...
- Then there should be a new email to "fred@bloggs.com" with the subject "Hello" containing "Hi Fred"
- Then there should be a new event registration (for me) (in the name of "Jane Bloggs")
- Then I should be in the group "cool guys"
- Then I should have the tag "coolest"

(For thoughts on test emails in Drupal with Behat see https://www.previousnext.com.au/blog/advanced-testing-drupal-emails-behat-and-testingmailsystem)

Jonathan

Hi Chris,

exciting to hear from you, I'm very keen to collaborate on this. Sorry for the delay in replying.

When you say "I'm definitely keen to see this more opened up" and "working together to outline common steps" did you mean
1) working to create a generic BDD base that is shared by both Codeception and Behat or
or
2) working to figure out the commonly-needed functionality that Behat steps should exist for?

I'm at a very early point in trying to figure this all out, but it matters to me.

I'd like to look at the existing civi tests, but  I'm not able to locate the tests in the civiCRM code base - the only path I've found on the wiki is for a tools/scripts folder that no longer seems to exist. Can you point me in the right direction?

Setup and teardown seems to be one of the most important issues to address. Totten outlines a variety of approaches used now in civi: https://forum.civicrm.org/index.php/topic,35627.0.html
The Rollback approach new in 4.6 seems promising, but he says that it does not work with webtests. It would be good to learn more about why. Drupal's behat extension offers an API mode as well as a blackbox mode - perhaps it is not constrained in the same way and could use Rollbacks. I'll ask PM Totten and ask him to weigh in.

The crudest (effective but not the fastest) way to setup and teardown for every step might be to load the whole civiCRM example data, and then truncate all the civi tables. It's not ideal as we're loading more test data than we need for any particular step, but might do for a first version?

I'll start a google document where we can collaborate on outlining the steps we want and post a link to it here.

The Drupal extension for Behat can auto-detect Behat steps & subcontexts provided by contrib modules. I'm wondering if what we're working towards is a .behat.inc. file that belongs in the CiviCRM Drupal module?

Jonathan

I've initiated https://github.com/jhedstrom/drupalextension/issues/194 to bring support for raw entities into the Behat Drupal Extension.

Hopefully then that will open the door up to simple handling of setup and teardown for simple civiCRM test data, by treating civi entities as if they were Drupal entities, no further coding needed.

We'll also want to be able to test emails sent through civiCRM. I've opened https://github.com/jhedstrom/drupalextension/issues/195 to bring support for email testing into the Behat Drupal extension. This won't automatically extend to civiCRM, but hopefully we can reuse the same steps syntax and just have an @civiMail tag that switches the logic over from checking the Drupal test mail stash to the civi test mail stash (the one linked with "define('CIVICRM_MAIL_LOG', 1);").

I hope to get both these functionalities into the Behat Drupal extension within the next 4 weeks.