European Code Sprint 2-5 Oct 2010 - suggestions

[Erik Hommel]

I am currently in a project where we use the API's through the REST interface. Doing that I have noticed some issues that I did not dive into, and there were also some posts on the forum. These dealt mainly with the Location API, but I had some issues with Custom Data too. I would like to use the sprint to sort out those errors and upgrade the documentation on the API's on the wiki (the book is already pretty good!)
There was also an idea on permissions in the API, and something Xavier looked at for error reporting in the API's.
Erik

[xavier]

Add to the list:

Put acl apis on the ajax interface (eg for anonymous)
Put acl apis on the REST interface (contact search)
write unit tests
Implementing a quick search api (for the autocomplete) that doesn't suck

And come back to the list we started with helen

[Erik Hommel]

Another addition to the list:
internationalization of street parsing (see forum post http://forum.civicrm.org/index.php/topic,14035.msg59928.html#msg59928)

[xavier]

Trying to find what are the right values to put in the param is a bit of a guesswork (the ajax interface greatly helps the trial, but still a hit and miss game).

Is it id, cid or contact_id ?
get the id with company_id and the name with organization_name
...mail_to_household_id and household (I think)
...

They have been various mentions of work starting to be able to list the valid parameters. Is there any progress ? Could we push it further ?

Also, having a strict mode that properly spit an error message instead of silently ignoring params it doesn't know how to handle would be really useful.

[Donald Lobo]

might help while we are doing this to also do the foll:

1. modify the code so that contact id is ALWAYS contact_id etc and make things more consistent

2. put all object ids as class variables. this helps hooks peek into a form/page object and use them if/when needed

lobo

[xavier]

Right now, the api doesn't "translate" between what the BAO does and what it presents outside and its incoherency is the mirror of the ones in the BAO.

You mean to change the api do add a layer of translation or change the BAO/DAO ?

(eg employer_id (sic wasn't company_id) vs organization_name comes straight from the SQL)

X+

[xavier]

Example of incomplete coverage:

- There is now one or two unit tests on contact_search, but it obviously doesn't cover most of the criterias, and options to return various fields.

Right now in the 3.1 (or 3.2) serie,  a regression has been introduced and the search (on api) doesn't work anymore.

I'd like to understand how to add some tests, and test the search better.

X+

[Donald Lobo]

i think we should use the same names everywhere

i.e.

db name == bao name == api name

this breaks down when u want to use it, and in most cases we use TABLENAME_id which seems reasonable

lobo

[xavier]

An important one:

Handling better the errors. Eg if you try to create an entity that is already existing, you will get a big sql fatal error that will stop your program that is using the api.

I would much prefer getting an is_error than having my program die.

(not sure if it's possible to trap an exception and process, but if it is, I'd love to fix it in briton)

X+

[xavier]

Hi,

Every time it searches on name OR email, the request takes several 100th of ms for a db > 5k contacts.

SELECT SQL_NO_CACHE sort_name name, ce.email, cc.id FROM civicrm_email ce INNER JOIN civicrm_contact cc ON cc.id  = ce.contact_id WHERE ce.on_hold = 0 AND cc.is_deceased = 0 AND cc.do_not_email = 0 AND ( cc.sort_name LIKE '%q%' OR ce.email LIKE '%q%' ) AND cc.is_deleted = 0 LIMIT 0,20 ;

1) introduce a minChar of 2 or 3 on the autocomplete/token (can be a const in settings
2) php merge two simple and fast queries  (select from civicrm_email + select from civicrm_contact)

X+

[xavier]

Hi,

Just ran scripts that I wrote for 2.x:

I 2.x I could have several addresses in the create contact, I can't anymore and have to use location

In 3.0 I could have several locations in the add_location api, I can't anymore on 3.2

1) We need proper unit tests for these cases
2) I'd like to have the location api going back in an user friendly state

2.1) the get and create should have compatible formats on the params
2.2) I should be able to set several locations on the create contacts and add_location

[xavier]

The format is
{ 42 : {contact_id:42;sort_name...}
   123: {contact_id:123

I would be more convenient (eg when you fetch from smarty or as json for an autocomplete) to get an array of contacts instead

[
{contact_id:42
{contact_id:123

I'd suggest adding a param (nohash/indexresult/flatten_result...whatever) that allows to switch to this mode

X+

[Michael McAndrew]

Hey there,

Two (UK centric) pet projects that I'd like for us to work on at the code sprint

1) Integration of http://mapit.mysociety.org/ with CiviCRM to provide mapping and area information for UK organisations using CiviCRM.  This API provides all publicly freely available area (political and administrative) information in the UK and would be invaluble for campaigning / canvassing organisations working here.

2) Giftaid functionality - don't need to go over (again) why this is important

Plus I'd love 1/2 an hour to get the address block into CiviCRM as a token: http://civicrm.org/blogs/michael-mcandrew/creating-address-block-token

[xavier]

For mapit: the problem is that what you can get aren't standard fields. Not sure how to make something "generic enough"

X+

[Michael McAndrew]

can you expand on 'aren't standard fields'?

http://mapit.mysociety.org/postcode/E5 0DR

gives all we need for Green Party of England and Wales, no?

Code: [Select]

{
    "wgs84_lat": 51.552439490404979,
    "coordsyst": "G",
    "wgs84_lon": -0.040020225207082163,
    "postcode": "E5 0DR",
    "easting": 535986,
    "areas": {
        "900000": {
            "codes": {},
            "name": "House of Commons",
            "country": "",
            "type_name": "UK Parliament",
            "parent_area": null,
            "generation_high": 13,
            "generation_low": 1,
            "country_name": "-",
            "type": "WMP",
            "id": 900000
        },
        "900001": {
            "codes": {},
            "name": "European Parliament",
            "country": "",
            "type_name": "European Parliament",
            "parent_area": null,
            "generation_high": 13,
            "generation_low": 1,
            "country_name": "-",
            "type": "EUP",
            "id": 900001
        },
        "900002": {
            "codes": {},
            "name": "London Assembly",
            "country": "E",
            "type_name": "London Assembly area (shared)",
            "parent_area": 900006,
            "generation_high": 13,
            "generation_low": 1,
            "country_name": "England",
            "type": "LAE",
            "id": 900002
        },
        "900006": {
            "codes": {},
            "name": "London Assembly",
            "country": "E",
            "type_name": "London Assembly area",
            "parent_area": null,
            "generation_high": 13,
            "generation_low": 1,
            "country_name": "England",
            "type": "LAS",
            "id": 900006
        },
        "2247": {
            "codes": {
                "unit_id": "41441"
            },
            "name": "Greater London Authority",
            "country": "E",
            "type_name": "Greater London Authority",
            "parent_area": null,
            "generation_high": 13,
            "generation_low": 1,
            "country_name": "England",
            "type": "GLA",
            "id": 2247
        },
        "8651": {
            "codes": {
                "ons": "00AMGL",
                "unit_id": "11210"
            },
            "name": "King's Park",
            "country": "E",
            "type_name": "London borough ward",
            "parent_area": 2508,
            "generation_high": 13,
            "generation_low": 1,
            "country_name": "England",
            "type": "LBW",
            "id": 8651
        },
        "2508": {
            "codes": {
                "ons": "00AM",
                "unit_id": "11199"
            },
            "name": "Hackney Borough Council",
            "country": "E",
            "type_name": "London borough",
            "parent_area": null,
            "generation_high": 13,
            "generation_low": 1,
            "country_name": "England",
            "type": "LBO",
            "id": 2508
        },
        "11824": {
            "codes": {
                "unit_id": "41451"
            },
            "name": "North East",
            "country": "E",
            "type_name": "London Assembly constituency",
            "parent_area": 2247,
            "generation_high": 13,
            "generation_low": 1,
            "country_name": "England",
            "type": "LAC",
            "id": 11824
        },
        "65752": {
            "codes": {
                "unit_id": "25071"
            },
            "name": "Hackney South and Shoreditch",
            "country": "E",
            "type_name": "UK Parliament constituency",
            "parent_area": null,
            "generation_high": 13,
            "generation_low": 13,
            "country_name": "England",
            "type": "WMC",
            "id": 65752
        },
        "11806": {
            "codes": {
                "unit_id": "41428"
            },
            "name": "London",
            "country": "E",
            "type_name": "European region",
            "parent_area": null,
            "generation_high": 13,
            "generation_low": 1,
            "country_name": "England",
            "type": "EUR",
            "id": 11806
        }
    },
    "northing": 185548
}