CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion »
  • Unit Testing (Moderator: Michał Mach) »
  • Rant: don't give us excuses not to run the tests
Pages: [1]

Author Topic: Rant: don't give us excuses not to run the tests  (Read 3059 times)

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Rant: don't give us excuses not to run the tests
October 31, 2011, 03:41:37 am
Hi,

Small issues, but that are honestly a roadblock on my tests

say that I modify api/v3/Whatever.php and the corresponding tests/phpunit/api/v3/Whatever.php

I'd be on the root of my civicrm install and alternate between
me@computer:/var/www/civicrm$vi  api/v3/Whatever.php
me@computer:/var/www/civicrm$vi  tests/phpunit/api/v3/WhateverTest.php

(or whatever editor you want that you prefer joe is quite irrelevant here michal ;)

However, to run the tests I have to:
cd tools
me@computer:/var/www/civicrm/tools$php scripts/phpunit -udb_username -pdb_password -hdb_host api_v3_WhateverTest


This command line is genuinely blocking me and I never remember it (I tend to do the tests when I have no other distraction, ie when I can't google to find that wicked syntax because I don't have internet access).

My issues:
1) I expect the test script to be under tests, or bin, or tool/bin but NOT under scripts
2) I hate when a script can't run from the root of civi (where I am to edit)
3) I never remember the passwords I put for the db. Why on earth do you have the password as a mandatory param whilst you forced me to hardcode the name of the db?
I expect to have a config file (under sites/test or wherever you want) where I store whatever user/password/dbname and what else is needed and never having to type them
4) test_class_name. I want to be able to put the file name, not the class name. Having to use the previous line and forced to replace "/" by "_" is strangely blocking me

Moreover, I'd want to have an easy way to skip the db reset stage (that is taking forever)

So to wrap up to make me (us?) an happy tester,

me@computer:/var/www/civicrm$vi  tests/phpunit/api/v3/WhateverTest.php

me@computer:/var/www/civicrm$php tools/bin/phpunit.php  tests/phpunit/api/v3/WhateverTest.php

or optionally php tools/bin/phpunit.php  tests/phpunit/api/v3/WhateverTest.php --no-db-reset


It would have been proven convenient to have an option to choose different dbs (eg to run the test on a real db or one that has custom fields or whatever), but can leave without

I haven't followed the magic done on cli.php at the latest sprint, could it be put in action here too?

http://wiki.civicrm.org/confluence/display/CRM/Setting+up+your+personal+testing+sandbox+HOWTO
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Rant: don't give us excuses not to run the tests
October 31, 2011, 07:54:15 am

At the various code sprints we fixed the tests to do the foll:

1. Take your user name / password from ~/.my.cnf  if availaible. This avoids you needing to send in username/password/host

2. Allow a different db name to be sent instead of civicrm_tests_dev

3. Recent tests do not recreate the DB, do they? I've not checked what the current status of this is, but tests are a lot faster these days. If they do, adding a new flag to it should not be too hard

4. You can write a small wrapper shell script to do a:

% pushd tools
% scripts/phpunit WHATEVER_ARGS_YOU SENT TO THIS SCRIPT
% popd

For extra credit u can also convert filename to classname

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: Rant: don't give us excuses not to run the tests
October 31, 2011, 08:29:00 am
Quote from: Donald Lobo on October 31, 2011, 07:54:15 am

At the various code sprints we fixed the tests to do the foll:

1. Take your user name / password from ~/.my.cnf  if availaible. This avoids you needing to send in username/password/host


Ok, will check thx.
I'd prefer that it's a different user than mine, cause I'm likely to have more access rights than just the civicrm test db (and I don't think that the test will go full rampage on other db, but I'd rather have it block at the db access level if they do)

Quote from: Donald Lobo on October 31, 2011, 07:54:15 am

2. Allow a different db name to be sent instead of civicrm_tests_dev


How? Did I miss a doc somewhere?

Quote from: Donald Lobo on October 31, 2011, 07:54:15 am
3. Recent tests do not recreate the DB, do they? I've not checked what the current status of this is, but tests are a lot faster these days. If they do, adding a new flag to it should not be too hard

I'm not sure, it seems to be "thinking" a lot in the init phase before working on the tests.

Quote from: Donald Lobo on October 31, 2011, 07:54:15 am

4. You can write a small wrapper shell script to do a:

% pushd tools
% scripts/phpunit WHATEVER_ARGS_YOU SENT TO THIS SCRIPT
% popd

For extra credit u can also convert filename to classname

Ok, will hack something.
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

Eileen

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4195
  • Karma: 218
    • Fuzion
Re: Rant: don't give us excuses not to run the tests
October 31, 2011, 11:40:40 am
Good to hear there have been some improvements. I went to write a test night but when I did an svn up nothing seemed to be working - I couldn't check it against the tests on the server as the results aren't showing up there.  I didn't know if the problem was at my end or the server so I just handled the bug in my own code & didn't write the test

I think not having visibility on the test pass rate for trunk (plus active branches) is a blocker on writing tests. Ideally we would be able to see not just the current day but the trend (e.g. to see when they all started failing).
Make today the day you step up to support CiviCRM and all the amazing organisations that are using it to improve our world - http://civicrm.org/contribute

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Rant: don't give us excuses not to run the tests
October 31, 2011, 03:10:02 pm

yes agreed. We do need to have better test infrastructure and coverage on more branches, including trunk.

however, we are also quite maxed out in time and resources (and i know you are too!). so we need to find someone who can help with test infrastructure and rig up various processes :) Know anyone suitable? Lets see if we can manage to find someone to do this in the community

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion »
  • Unit Testing (Moderator: Michał Mach) »
  • Rant: don't give us excuses not to run the tests

This forum was archived on 2017-11-26.