CiviCRM Community Forums (archive)

Justification
At raSA (http://rasantiago.com/), we're writing an Application in Ruby On Rails that uses CiviCRM as a back-end datastore. Rails has a core library called "ActiveResource" which is meant to map REST APIs to Ruby Objects (It's almost an ORM, except that REST is never quite a relational database). Rails is implemented as "Opinionated Software" -- when things are done in an industry standard way (or at least in the way that the Rails platform developers think is best), they work with minimal configuration. ActiveResource is, perhaps, more opinionated than most - it is essentially unconfigurable. For an API to work
with ActiveResource, it must:

• Represent objects as XML
• Always use XML nodes that have names that either represent what type of object they are (like <contact> ) or have a type attribute (like <ResultSet type="array"> )
• Use ID fields named "id" for every class of object
• Give unique URLs to each object, by ID, like: http://servername/contacts/1
• Respond differently to each of the HTTP protocol's methods: GET, POST, PUT, DELETE
• Accept raw XML in the POST body of a HTTP request (not in the traditional application/x-www-form-urlencoded style)
• Use the HTTP "Status" header to denote errors
• Report the URL of a newly created object in the HTTP "Location" header

CiviCRM's existing rest.php interface only meets the first requirement on this list. After long consideration, we decided that it would be more useful to the community as a whole to create an interface in CiviCRM that follows these conventions, rather than making a new driver for Ruby or Rails that speaks CiviCRM's existing dialect of REST.

Process
In our local branch of the CiviCRM code, we have forked extern/rest.php and CRM/Utils/REST.php into a new interface, which I'm currently calling REALLY_REST (I'm open to suggestions for a better name). The existing REST.php interface uses URLs that look like this:
http://localhost/civicrm/extern/rest.php?q=civicrm/contact/get&contact_id=1
The equivalent REALLY_REST URL looks like:
http://localhost/civicrm/extern/really_rest.php/contact/1
Note that there are no query parameters on this URL -- all of the information about the action is encoded into the the path (plus the HTTP method, which in this case is "GET"). On many web servers (including Apache), it is legal to append arbitrary data to the URL of a PHP script - this is used to simulate virtual directories. Also, Apache supports PUT and DELETE on PHP scripts without any additional configuration.

CiviCRM's API v2 is already organized in a way that is conducive to building method names dynamically:
GET http://localhost/civicrm/extern/really_rest.php/contact/1        => civicrm_contact_get( 'contact_id' => 1 )
GET http://localhost/civicrm/extern/really_rest.php/contact?name=Bob => civicrm_contact_search( 'name' => "Bob" )
PUT http://localhost/civicrm/extern/really_rest.php/contact          => civicrm_contact_add( $parsed_XML )
POST http://localhost/civicrm/extern/really_rest.php/contact         => civicrm_contact_add( $parsed_XML ) (Civi's API does not distinguish between 'add' and 'edit' operations)
DELETE http://localhost/civicrm/extern/really_rest.php/contact/1     => civicrm_contact_delete( 'contact_id' => 1 )

Roadblocks
One problem with this system of building method names dynamically is that it requires absolute consistency in the API. As soon as I started testing a second model, I started finding that several of my assumptions about the API methods were failing:

• Some methods require named ID parameters like "contact_id" and some require just "id"
• Some objects have separate "get" and "search" methods for find-one and find-many, while some only have one or the other -- and there isn't consistency about whether they return arrays or single records
• It isn't always clear what parameters are mandatory for adding various objects. Also, some objects have read-only fields, and trying to edit them via the API doesn't raise an error.

I've been gradually modifying the API methods so that they can take (for example) either "group_id" or "id" parameters, which should be a back-compatible change. On the other hand, fixing the "get" and "search" methods to return a single record and an array of records, respectively, is an incompatible change. I hope to eventually have these changes merged into the main codebase -- is there a way I can make these changes that will do the least amount of damage to other uses of the API?

Also, I don't currently have a working unit test infrastructure for CiviCRM -- I've been using ruby rspec tests through the REALLY_REST interface. I'm not sure that I understand the current state of unit test infrastructure for CiviCRM -- I hear that it was discussed at CiviCamp -- is it currently working well enough that we could be writing API unit tests?

Other API enhancements
The API currently only exposes a small subset of CiviCRM's functionality: there are only sixteen modules in /api/v2, while the CRM libraries are divided into thirty-four namespaces, and CiviCRM's database has one-hundred-and-thirteen (!) tables. We're trying to gradually expand the API -- our first goal is to deal with Roles and ACLs, but other major areas that need work are with Pledges and Memberships. Are CiviCRM's objects dealing with these subjects considered stable?

Future possibilities
Even these changes only scratch the surface of REST's potential. CiviCRM's API currently treats all objects as flat dictionaries, while sometimes a more correct XML rendering of an object would have nested structure. A simple example is the Contact object: the API currently mixes the user's address fields into the toplevel of the Contact's XML, even though contacts are allowed to have multiple addresses -- the API's behavior in this case is ambiguous. It is not obvious to me how to implement this without dramatically changing the behavior of CiviCRM's API methods.

• Some methods require named ID parameters like "contact_id" and some require just "id"

for the core object we should always use id. For example when updating a contribution, the contribution object should be called id, BUT the contact that is associated with it will be called contact_id. For most of the objects, you will have the main ID and at least one other associated id. If we are inconsistent, please file one issue for all the places where you've noticed this and we will fix

• Some objects have separate "get" and "search" methods for find-one and find-many, while some only have one or the other -- and there isn't consistency about whether they return arrays or single records

IMO: get should always return a single record (i.e. an array of name=>value pairs), search should always return an array of records (each record is another array). Please file another issue with all the places where you've noticed a discrepancy and we will fix

• It isn't always clear what parameters are mandatory for adding various objects. Also, some objects have read-only fields, and trying to edit them via the API doesn't raise an error.

We need to figure out how to have a unified "validation" system that both the form system and the API can use. This is a big issue and I suspect we should tackle (on a small scale to begin with) in 3.0

Also, I don't currently have a working unit test infrastructure for CiviCRM -- I've been using ruby rspec tests through the REALLY_REST interface. I'm not sure that I understand the current state of unit test infrastructure for CiviCRM -- I hear that it was discussed at CiviCamp -- is it currently working well enough that we could be writing API unit tests?

We use simple test and the drupal simple test module (hacked for our use in packages/drupal/simpletest). All of the api/v2 tests should be working as of 2.1.2

lobo