CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site
This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion (Moderator: Donald Lobo) »
  • Black box testing
Pages: [1]

Author Topic: Black box testing  (Read 1009 times)

sdague

  • I’m new here
  • *
  • Posts: 17
  • Karma: 0
  • CiviCRM version: 4.1
  • CMS version: Drupal 6
  • MySQL version: 5.1.58
  • PHP version: 5.3
Black box testing
April 15, 2012, 05:22:19 am
After running into 2 show stopping bugs in my 4.1.1 / D6 deploy (both looked to be fixed in SVN, eagerly awaiting 4.1.2), I'm really starting to wonder about black box testing for CiviCRM. Granted, testing the email bounce processing would be hard, but the public event issue wouldn't be too bad. The unit tests clearly aren't catching enough.

Has anyone started going down this path? Some sort of CiviCRM burn in test that would use SimpleTest to just interact with the site like an administrator, add a bunch of content (tagged in a way that it could be easily deleted), manipulate data, and make sure everything is working? I'm going to be building this for myself so I can verify our environment, but if someone else started down this path I'd much rather be collaborating than writing it all from scratch.

Thanks much,

   -Sean

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Black box testing
April 15, 2012, 09:47:08 am

We have quite a few unit and web tests which we run on a regular basis: http://tests.dev.civicrm.org/

we do ensure that we have a fairly high rate of success before doing a release.

However we only test the combination of D7 + Civi 4.1 The webtests take a fairly long time (3+ hours) and hence a lot harder for us to run it across different variations.

D6 + Civi v4.1 is maintained and managed by the D6 community (eileen et al). We'd prefer that the tests are also managed by interested folks.

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

sdague

  • I’m new here
  • *
  • Posts: 17
  • Karma: 0
  • CiviCRM version: 4.1
  • CMS version: Drupal 6
  • MySQL version: 5.1.58
  • PHP version: 5.3
Re: Black box testing
April 15, 2012, 10:28:12 am
Is there some instruction on how to get these running? I'm interested in helping get these running with D6.

Also looking at http://tests.dev.civicrm.org/trunk/results-Web/ < 50% of tests are passing, which seems somewhat problematic.

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Black box testing
April 16, 2012, 10:25:21 am

the trunk results are actually 4.2 (i fixed the index file a few mins back). the current stable has a pretty high pass rate

How to set up your personal machine for tests is here:

http://wiki.civicrm.org/confluence/display/CRM/Testing

Those docs can use a bit of cleanup/automation. Ping us on IRC if u get stuck

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions
Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion (Moderator: Donald Lobo) »
  • Black box testing

This forum was archived on 2017-11-26.