CiviCRM Community Forums (archive)

There has been renewed interest in replacing DB_DataObject:

http://civicrm.org/blogs/xavier/database-layer-evaluation

One important question in planning such an effort (among several) is this: To what extent does CiviCRM actually depend on the features of DB_DataObject? How many times is it (directly or indirectly) instantiated? Which functions are actually used -- and how often?

TECHNIQUES

In statically-typed languages (like Java), one can rely on static-analysis of the source code files to assess dependencies on functions and classes. (This is why things like automated refactoring work in Java... if you use an IDE like Eclipse/Netbeans with Java code, you can get a quick-n-dirty list of dependencies by previewing a "rename" operation.) But this probably isn't reliable in PHP.

In PHP, we *can* do runtime-analysis. There are existing tools for profiling and for aspect-oriented programming in PHP. I don't have much experience with them, so instead I hacked a small wrapper class for DB_DataObject (attached). This class intercepts and logs every function call which would otherwise be handled by DB_DataObject. (I also tried to intercept property-assignments using PHP's magic methods, but that attempt didn't work as I'd expected.)

DATA FORMAT

The attached patch produces a CSV. Each line corresponds to an invocation of a function in DB_DataObject. The columns are:

1. Concrete class: The full name of the concrete class for which the function executed. This should always be a subclass of DB_DataObject.
2. Injection technique: How did the logger receive a notification that the function was called? Was it using a overloaded function? Or PHP's magic function?
3. Operation type: Function "call", property "set", or property "get"
4. Function name or property name
5. Caller file
6. Line of caller file

For example, consider this line:

CRM_Core_DAO_Domain,overload,call,selectAdd,/home/abc/svn/civicrm/CRM/Core/BAO/Setting.php,205

This indicates that line 205 in CRM/Core/BAO/Setting.php depends on DB_DataObject because it calls selectAdd on an instance of CRM_Core_DAO_Domain.

DATA COLLECTION

To do a runtime-analysis, we need to run Civi through a variety of scenarios and accumulate log data. The results of running CiviCRM 3.4 (~3.4.4) through three test suites (api_v2_AllTests, api_v3_AllTests, and CRM_AllTests) are available here:

http://arms.dl.locker10.com/devel/apiv2_apiv3_CRM.csv.bz2

Please note that the coverage of this dataset is only as good as the coverage of the chosen unit-tests.

INITIAL OBSERVATIONS

There are a few hundred unique lines of code which invoke functions in DB_DataObject. The number goes down a bit if you exclude self-invocations (i.e. incidents in which DB_DataObject calls itself):

$ grep ,call, apiv2_apiv3_CRM.csv | cut -d, -f5,6 | sort -u | wc
    589     589   33206
$ grep ,call, apiv2_apiv3_CRM.csv | grep -v packages/DB/DataObject | cut -d, -f5,6 | sort -u | wc
    549     549   30933

The non-static DB_DataObject::query function (which accepts arbitrary SQL) is invoked many times, but there's only one line of code with a hard dependency on it:

$ grep call,query, apiv2_apiv3_CRM.csv | wc
  34353   34353 2721695
$ grep call,query, apiv2_apiv3_CRM.csv | cut -d, -f5,6 | sort -u | wc
      1       1      46
$ grep call,query, apiv2_apiv3_CRM.csv | cut -d, -f5,6 | sort -u
/home/armsys/svn/civicrm/CRM/Core/DAO.php,149

The non-static DB_DataObject::find function (which constructs SQL based on object-attributes) is also invoked many times, but there are many more files/lines which reference it:

$ grep call,find, apiv2_apiv3_CRM.csv | wc
  21591   21591 2088771
$ grep call,find, apiv2_apiv3_CRM.csv | cut -d, -f5,6 | sort -u | wc
    205     205   11567


Magic methods like __call() or __set() can be used to dynamically define class methods/properties. They're often used for generic programming/bindings.

http://php.net/manual/en/language.oop5.magic.php

I imagine we could capture test names. I'm not a phpunit expert,  but other xunit implementations provide a global variable to track the active test.

Haven't really used sqlite... is it easy to autocreate the db/schema?

At the risk of taking a hack too far, I cleaned it up a bit more; a revised patch is attached.

FWIW, the class "SQLite3" doesn't seem to be available on PHP 5.2, but PDO seems to work similarly.