Sneak Peak: PR Testing

[Coleman Watts]

What's the current status? When I submit e.g. https://github.com/civicrm/civicrm-core/pull/3745 (which makes a change to the api), what tests are being run? Looking at the console output it looks like the upgrade suite is running and a subset of CiviTest (not sure how that's being determined) but not the api test suite?

[Eileen]

It's either because you are above questioning or because that api is so awful it runs away screaming like I do every time I see it.

ps  last time I added that function to the BAO (where I think it belongs) but it got removed because there are forms that run it too so it got run twice :-( I never found out when it happened or I would have taken it out of the forms & put it back in the BAO

[Coleman Watts]

Eileen I agree that's the "right" fix but this late in the release cycle this'll have to do. At least now it works.

[Eileen]

yes - the right fix was pulled out of 4.4 during alpha I think (or maybe even before it went alpha)

[totten]

What's the current status? When I submit e.g. https://github.com/civicrm/civicrm-core/pull/3745 (which makes a change to the api), what tests are being run? Looking at the console output it looks like the upgrade suite is running and a subset of CiviTest (not sure how that's being determined) but not the api test suite?

Correct, it was set to only try a handful of tests. I've broadened it to include api_v3_AllTests for now (but not much of CRM_AllTests and none of WebTest_AllTests). That should apply to any future test runs. In this configuration, I'd guesstimate 30-40min turn-around between submitting/updating a PR and getting the results, but we'll see when someone submits/updates a PR...

[Eileen]

submitted - https://github.com/civicrm/civicrm-core/pull/3746 - now we can see :-)

[Eileen]

so that PR failed on pre-exising test failures from what I can see.