CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site
This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion (Moderator: Donald Lobo) »
  • behat and mink as web testing framework?
Pages: [1]

Author Topic: behat and mink as web testing framework?  (Read 2559 times)

JoeMurray

  • Administrator
  • Ask me questions
  • *****
  • Posts: 578
  • Karma: 24
    • JMA Consulting
  • CiviCRM version: 4.4 and 4.5 (as of Nov 2014)
  • CMS version: Drupal, WordPress, Joomla
  • MySQL version: MySQL 5.5, 5.6, MariaDB 10.0 (as of Nov 2014)
behat and mink as web testing framework?
May 10, 2013, 08:59:53 am
behat (http://docs.behat.org/quick_intro.html) seems to provide a good framework for behaviour driven development testing, and via mink (http://mink.behat.org/) provides nice integration of the existing selenium tests.

I'm not sure if the lack of support for Selenium 2/ Webdriver API for PHP has been issue at all.

Any thoughts on whether this is useful as a high-level tool enabling requirements engineering on new development to be connected to resulting tests?
Co-author of Using CiviCRM https://www.packtpub.com/using-civicrm/book

Michael McAndrew

  • Forum Godess / God
  • I live on this forum
  • *****
  • Posts: 1274
  • Karma: 55
    • Third Sector Design
  • CiviCRM version: various
  • CMS version: Nearly always Drupal
  • MySQL version: 5.5
  • PHP version: 5.3
Re: behat and mink as web testing framework?
May 10, 2013, 11:23:56 am
chris gervais from koumbit was doing some work on this at the sprints - might want to get in contact with him.
Service providers: Grow your business, build your reputation and support CiviCRM. Become a partner today

JoeMurray

  • Administrator
  • Ask me questions
  • *****
  • Posts: 578
  • Karma: 24
    • JMA Consulting
  • CiviCRM version: 4.4 and 4.5 (as of Nov 2014)
  • CMS version: Drupal, WordPress, Joomla
  • MySQL version: MySQL 5.5, 5.6, MariaDB 10.0 (as of Nov 2014)
Re: behat and mink as web testing framework?
May 10, 2013, 11:33:58 am
Thanks, Michael. Does Chris hang out in irc ever, and if so, what is his nick?
Co-author of Using CiviCRM https://www.packtpub.com/using-civicrm/book

totten

  • Administrator
  • Ask me questions
  • *****
  • Posts: 695
  • Karma: 64
Re: behat and mink as web testing framework?
May 10, 2013, 11:51:02 am
Chris (aka ergonlogic) posted his work here:

https://github.com/ergonlogic/civicrm_behat/

It's a proof-of-concept for testing the CiviCRM installer UI under Drupal.

Behat is very appealing. I'm a little nervous about overselling it -- like Selenium IDE, its ambition is to let less technical folks work on tests. That seems more like "the holy grail" of testing than "real feature." With that said, it still has several appealing attributes:

 * The stack is thinner than PHPUnit-Selenium (i.e. doesn't require web browsers or Java). That *should* mean better test performance / easier installation. (I like to think that "fast and easy tests" are "tests that people will actually use".)
 * The language is manifestly more *readable*.
 * It does feel more realistic to "write the test first" when using Behat notation than when using PHPUnit notation.

My own preference would be to decouple the test framework (PHPUnit, Selenium, Behat, Mink) from the fixture management (linking to the DB, determining test user+URL, resetting tables, activating extensions, etc) in a way that lets one use either framework (with either extensions or core).

JoeMurray

  • Administrator
  • Ask me questions
  • *****
  • Posts: 578
  • Karma: 24
    • JMA Consulting
  • CiviCRM version: 4.4 and 4.5 (as of Nov 2014)
  • CMS version: Drupal, WordPress, Joomla
  • MySQL version: MySQL 5.5, 5.6, MariaDB 10.0 (as of Nov 2014)
Re: behat and mink as web testing framework?
May 10, 2013, 12:13:08 pm
Very cool, Chris!

I think your caution is understandable. It appears there will still be lots of work to move from a user level readable description to a working test.

I was intrigued by the possibility of having a single tool that could do certain tests in a headless mode when js wasn't needed, and also include Selenium 2 tests controlling a browser for the many ajaxified pages and functionality in CiviCRM. I like your idea of separating fixture mgmt from test framework. And I like the idea of having common testing infrastructure for extensions and core.

The current tests using Selenium take too many hours to complete for them to be run on every commit to the repo. Shifting to headless for some of these tests *should* allow them to run more quickly. Lots of delays that have been introduced with the intention of ensuring page content has been completely downloaded.

Was there any discussion at the sprint about the possibility of getting additional test machines so that we could run Selenium tests in parallel using http://docs.seleniumhq.org/projects/grid/ ?
Co-author of Using CiviCRM https://www.packtpub.com/using-civicrm/book

totten

  • Administrator
  • Ask me questions
  • *****
  • Posts: 695
  • Karma: 64
Re: behat and mink as web testing framework?
May 10, 2013, 12:29:23 pm
We started setting up another VM for running webtests (but got delayed by external factors).

For scaling out to more machines, it's easier to use Jenkins' slave feature than to use Selenium Grid. With Selenium Grid, the PHPUnit process and the browser process run on different machines -- which breaks several test-cases that use file-uploading. (For example, to test the importer, we need to upload a CSV file. The CSV file is created by PHPUnit and then read by the web browser.) With Jenkins' slaves, we can have multiple machines running concurrent test suites -- but PHPUnit+Selenium can still be on the same machine.

JoeMurray

  • Administrator
  • Ask me questions
  • *****
  • Posts: 578
  • Karma: 24
    • JMA Consulting
  • CiviCRM version: 4.4 and 4.5 (as of Nov 2014)
  • CMS version: Drupal, WordPress, Joomla
  • MySQL version: MySQL 5.5, 5.6, MariaDB 10.0 (as of Nov 2014)
Re: behat and mink as web testing framework?
May 10, 2013, 06:26:44 pm
Thanks for responding totten. Good to know about Jenkins vs grid, and very good to hear about setting up another VM to start parallelizing the webtests. So many good ideas, and so little time to get them all working. ;)
Co-author of Using CiviCRM https://www.packtpub.com/using-civicrm/book

JoeMurray

  • Administrator
  • Ask me questions
  • *****
  • Posts: 578
  • Karma: 24
    • JMA Consulting
  • CiviCRM version: 4.4 and 4.5 (as of Nov 2014)
  • CMS version: Drupal, WordPress, Joomla
  • MySQL version: MySQL 5.5, 5.6, MariaDB 10.0 (as of Nov 2014)
Re: behat and mink as web testing framework?
October 18, 2013, 09:15:32 am
Just a note that there was a very positive response from Drupal community to behat and mink and even aspirations to bring it in to D9 core: https://groups.drupal.org/behat
Co-author of Using CiviCRM https://www.packtpub.com/using-civicrm/book
Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion (Moderator: Donald Lobo) »
  • behat and mink as web testing framework?

This forum was archived on 2017-11-26.