Developing with / Extending CiviSMS

[Michael McAndrew]

Hey there,

We are developing some SMS based functionality for a client.  They want to do a couple of things.

1) chain SMS, i.e. send a second SMS based on the answer to the first SMS
2) put data from SMS messages into custom data fields in CivICRM

We are presuming that there is nothing in the current version of SMS that will allow us to do the above two tasks.

If that, I am happy to write the code for chaining SMS and updating custom fields based on the replies in SMS.

I'm presuming this will not be too hard but that I would benefit from / am thinking about developing

a) an SMS hook which is called when an SMS is received (although maybe there is a hook which is fired when activities are created that I can check for type == 'inbound SMS') that allows us to call...
b) an SMS action for the contact API

At some point, we are going to have to work out how to define the custom data field to update with this SMS, but we can delay answering that question for a while.

Interested to know your thoughts / want to develop this in a reusable way.

[totten]

I agree that it makes sense to provide hook+API for receiving+sending messages

There's a lot of room for developing different bits of business logic for SMS interactions -- whether to create activities, create contacts, update contact fields, send followups, etc. I think the best thing we can do is to keep core minimal and focused on the interfaces and helper-functions. Then we should encourage downstream developers to experiment with the business logic by writing extensions.

The biggest tension, I think, is creating something that can be configured by non-coders. We should perhaps encourage two tracks of development:
 1. Downstream PHP devs can write adhoc extensions that address pressing business needs
 2. A semi-official "SMS uber extension" should be developed which allows administrators to configure more robust policies in the web UI -- making decisions like "when to auto-create a contact? when to auto-create an activity? what keywords should we auto-reply to? how to handle questions and answers?" Like CiviMobile, this extension could target Civi 4.2+ and be released on its own timeline (separate from core).

[Michael McAndrew]

Hey there,

1. Downstream PHP devs can write adhoc extensions that address pressing business needs
 2. A semi-official "SMS uber extension" should be developed which allows administrators to configure more robust policies in the web UI -- making decisions like "when to auto-create a contact? when to auto-create an activity? what keywords should we auto-reply to? how to handle questions and answers?" Like CiviMobile, this extension could target Civi 4.2+ and be released on its own timeline (separate from core).

Feelin' that.

To add a bit more flesh to chaining, here is the use case

Possible Text one:

Hi, as part of its Future First Network {school name} is keen to know what you’re currently doing, so which of the following best describes the main thing you’re up to? Text A for working, B for in education, C in training or D for other. Replying to this text is free.

Text two:

If A: Great thanks, we hope you’re enjoying it. One final question, job title or place of work?

If B: Great thanks, one final question what sort of further education are you doing? Text 1 for university, 2 for A levels, 3 for BTEC, 4 for other.

If C: Great thanks, we hope you’re enjoying it. One final question what are you doing your training in?

If D: Great thanks, can you give us any more details about what you’re up to?

Will also have a look at http://drupal.org/project/smsframework and see how they do it

[totten]

That's a helpful example use-case.

I think smsframework is a good example of how to do a basic SMS API. That's probably the design we're looking for right now.

Another interesting project to look at is http://drupal.org/project/voipdrupal . The scope is a broader than smsframework -- it's more about scripting interactions for voice/touchtone/SMS, and they've created a special scripting language for the task. The difference is that smsframework is essentially stateless -- your app receives a hook notice about an incoming message, but smsframework provides no way to determine whether the message is the first in a new conversation -- or if it's the fifth response to a series of questions. One *can* use smsframework to build a scripted interaction -- but you'll need to implement your own way to track the progression through the script. By contrast, voipdrupal does this automatically. (FWIW, the project lead on voipdrupal -- Leo Burd -- was interested in adapting it to Civi. He felt it would take a few manweeks to work on a CMS-agnositic CiviCRM integration.)

[Donald Lobo]

would be great if leo burd could start moving voip drupal independent of drupal

I'm not sure that the estimate of a few man-weeks is realistic though

lobo

[totten]

would be great if leo burd could start moving voip drupal independent of drupal

I'm not sure that the estimate of a few man-weeks is realistic though

Fair enough -- and I'm probably forgetting an important qualifier about what was in the scope of that ballpark estimate (eg "porting over the APIs/interpreter" vs "porting over the admin UI" or "porting VOIP applications built on that API").

[Michael McAndrew]

The difference is that smsframework is essentially stateless -- your app receives a hook notice about an incoming message, but smsframework provides no way to determine whether the message is the first in a new conversation -- or if it's the fifth response to a series of questions.

just to clarify, i think that is a limitation of SMS rather than smsframework or voipdrupal, right? or am i misunderstanding you?

My thinking was that we assume messages received from a number we have recently sent a question to are replies perhaps if they are sent within a certain timeframe which you can specify. What do you think?

[totten]

just to clarify, i think that is a limitation of SMS rather than smsframework or voipdrupal, right? or am i misunderstanding you?

My thinking was that we assume messages received from a number we have recently sent a question to are replies perhaps if they are sent within a certain timeframe which you can specify. What do you think?

You're right -- SMS (like vanilla HTTP) is stateless. But if you store some extra information on the server (eg "the time and substance of the last question sent to each phone number"), then you can write code which infers "This incoming message came from 2025551212, and we recently sent a question to 2025551212; therefore, the incoming message is probably a reply to the question." In smsframework, you explicitly write this; in voipdrupal, I believe they handle it automatically.

[Michael McAndrew]

Hey there,

Some notes on what I am finding as I look through this.

It seems to me that SMS are completely stateless, by which I mean, there is no way to tell that a MT (mobile originated, i.e. inbound) message is in reply to a MT (mobile terminated, i.e. outbound) message.

http://drupal.org/node/363259 is about processing incoming messages using Drupal's SMS Framework.  It doesn't mention any way of knowing what SMS this is in reply to.

That pretty much resonates with what my real world understanding of SMS is.

It makes me think that if we want to do chaining, then we have to presume / superimpose / overlay this chaining.

It might look something like this.

1) when we receive a message, decide if this message is an 'Answer SMS'.

We can decide this based on a number of factors. But probably the major one is 'Did we send this person a 'Question SMS' in the last x seconds/hours/days?

2) if it is an Answer SMS, work out what question it is an answer to and based on the answer, do something.

something could be anything   We probably want to define a data structure to store questions and answers.  Questions could possibly be extensions of SMS templates.  Answers could be stored in a table with three columns
* question
* answer (regular expression)
* weight (order in which answers are checked for validity)

I suspect looking at http://drupal.org/project/voipdrupal/ 's scripting components might be handy here, since this stuff might be already written.

[leoburd]

Hello there,

For some reason, I've missed this entire conversation!

Is anyone currently working on the integration of CiviCRM with SMS and eventually voice calls?

Is this topic still relevant to the CiviCRM community?

Best,

Leo

[Michael McAndrew]

SMS integration is complete (google 'civicrm sms').

There was some ideas for VOIP as part of GSOC 2014, but maybe you know about that: http://wiki.civicrm.org/confluence/display/CRM/Google+Summer+of+Code+-+2014#GoogleSummerofCode-2014-CiviCRMPhoneIntegration