Dreams for Civicrm - a top 10 list

[The Next Wave]

I looked through the forums for these issues- and didn't find recent discussions. I'm not criticizing- I'm just hoping for these things to make it to the forefront of the core team goals.
1) update process- at least in WordPress- it needs to be as easy as a WordPress plugin update.
2) Terminology- a standard vocabulary across all of Civi- so that there isn't confusion between a "campaign" and a "survey" or a "Fundraiser" - the more accessible the program is- the easier it is to adopt- the more it will grow.
3) Responsive- mobile friendly is essential- and- the addition of a mobile app for canvassing at the least, is critical.
4) Batch updating- I'm sorry, but 1mb uploads is insane. Just one county in Ohio takes 118mb. Why can't we upload a CSV to the server- and Civi parses and updates and generates an error report. Data that isn't easy to update is data that becomes stale.
5) Better form design- since no mobile app is available- printed walk lists should be able to be custom designed using an AJAX interface. Right now- we export the data and import into a page layout in InDesign so we don't waste paper- and have a nice looking walk list.
6) A better system for assigning tasks- and managing teams of canvassers. The Reserve Respondents function is archaic- and requires too much management. Gamification would help- or a different approach. Assigning task ranks- and then allowing workers to check out tasks and self defining their responsibilities- would make this much easier.
7) Optimizing geocoding- with USPS barcodes- routing for walk lists- (ie- shortest routes, and also- most efficient- or having them end near start points.
A business layer- using the CiviCRM framework as a basis for a business crm- so that we can increase the size of the community- and funding for developers- to grow the community.
9) Social integration- the ability to add twitter feeds, facebook, linkedin, instagram, etc to profiles of individuals to help optimize understanding of our communities. In order to have a meaningful conversation with a contact- wouldn't knowing more about them be useful?
10) A total UI overhaul- too many menus, options- this thing is an ugly baby. There I said it. How many different search options should I need in a menu?
Thanks-
and, no, I'm not smart enough to do much more than write UI specs and test- and I can't fund any of this. Cheers.

[xavier]

Hi,

Good list, thanks for sharing. However, transforming dreams into code needs time and sweat... and most developers are humans enough to need money to feed themselves....

Could you detail more (probably a new thread) the social network integration? It's a regular request that tend to mean a bit everything. If you had some examples some missing features and how they could be used, would be great.

X+

[The Next Wave]

When I was working on the UI for vTiger CRM I started an analysis of every crm I could find.
There are over 50 different CRM systems cataloged in my teambox driven collaboration tool.
The one that promised a simple implementation- was address two. Here is there video that explains it: http://youtu.be/r8s8dC0sxLg
Right now- I can't remember what other crm's included social - but this was a growing trend 2 years ago.

I hope this helps clarify -
and although twitter is nice- Facebook is the bedrock of every non-profit I deal with- so, having their Facebook comments etc- show up as activities when they post to our page- or share- would be great- esp. if you look to adding gamification layer to Civi.
I just did a quick google search- found this post with 5 apps to investigate- http://www.practicalecommerce.com/articles/3924-5-Social-CRM-Tools-for-Small-Business although when I first looked at Batchbook I wasn't impressed.

[JonGold]

Hi there,

Your list is an interesting one - though as I look at it, I think some of these goals are pretty use-case specific.

Two stand out for me though:
4) Batch updating- I'm sorry, but 1mb uploads is insane. Just one county in Ohio takes 118mb. Why can't we upload a CSV to the server- and Civi parses and updates and generates an error report. Data that isn't easy to update is data that becomes stale.

This is possible: https://civicrm.org/blogs/xavier/api_batch_tools

1) One-click upgrades - in my experience, many groups want customizations to CiviCRM that need to be tested before upgrading - a one-click upgrade would break a lot of sites.  I see the value of easier upgrading for folks for whom that's not their use case, but unfortunately, those are the folks least likely to pony up developer time/money to make the improvement.

[The Next Wave]

4) Batch updating- I'm sorry, but 1mb uploads is insane. Just one county in Ohio takes 118mb. Why can't we upload a CSV to the server- and Civi parses and updates and generates an error report. Data that isn't easy to update is data that becomes stale.

This is possible: https://civicrm.org/blogs/xavier/api_batch_tools

Wow- this would almost be great- except for two things-
1) if it was really meant for users- it would have a note on the import contacts pages- telling you there is a way around the 1mb upload-
2) You wouldn't have this part in the description of the link you sent:

Why is that a shell tool? Isn't there a web interface too I can almost hear you asking?
Because the cli is a much more convenient tool to handle these batch tasks and deal much better with files that are a lot bigger (eg if you start exporting/importing loads of contacts). It's also made to be scripted, so for instance sending you by email weekly the organisations above is a trivial oneliner. Really, you should have a server that offers you a ssh access and spend the time to learn bash, at least for the basic stuff.
This being said, if enough of you are interested of having a web interface too, you can make it happen...

Why would anyone suggest this is available to users- when it requires the ability to code?
I know a lot of you think that keeping the workings of Civi a mystery to mere mortals is some kind of job security- but, it's really killing off users.
This is a FUNDAMENTAL requirement of EVERY POLITICAL CAMPAIGN- uploading voting lists.
Why this hasn't been turned into a super smooth web interface at version 4.4 is an epic #fail from a UX perspective.

The one click upgrades from WordPress are why Wordpress is running 20% of the web.
If you want Civi to be the major player-
code has to be segregated- customizations from core- and the customizations should be built with a plugin structure.
I know this is probably an epic change in the way Civi is built-
but- these are the makings of a best in class web app.
Having everyone on the most recent versions- having standard installs- having easy ways to turn functions on and off- is a proven work flow.
This should be a long term goal of the project.
Thanks for letting me know about the api batch tools-
I think I'm going to have to work my way to this for the next batch update.

[xavier]

Wow- this would almost be great- except for two things-
1) if it was really meant for users- it would have a note on the import contacts pages- telling you there is a way around the 1mb upload-
2) You wouldn't have this part in the description of the link you sent:

Why is that a shell tool? Isn't there a web interface too I can almost hear you asking?
Because the cli is a much more convenient tool to handle these batch tasks and deal much better with files that are a lot bigger (eg if you start exporting/importing loads of contacts). It's also made to be scripted, so for instance sending you by email weekly the organisations above is a trivial oneliner. Really, you should have a server that offers you a ssh access and spend the time to learn bash, at least for the basic stuff.
This being said, if enough of you are interested of having a web interface too, you can make it happen...

Why would anyone suggest this is available to users- when it requires the ability to code?

I'm not sure where you see we mentioned you need to code. I just said you need ssh access. See, the web is pretty much made to forbid you uploading big files and have long running processes. It's massively more complicated/expensive to try bolting that into an interface that is trying to stop you doing it than using one that is made for that.

This being said, there is a solution. If you want to improve it, your code/funding would be welcome.

I know a lot of you think that keeping the workings of Civi a mystery to mere mortals is some kind of job security- but, it's really killing off users.

I've discussed with a lot of us, and all don't aim at keeping civi a mystery, and even less think it'd improve security... as for wishing to kill users....

This is a FUNDAMENTAL requirement of EVERY POLITICAL CAMPAIGN- uploading voting lists.
Why this hasn't been turned into a super smooth web interface at version 4.4 is an epic #fail from a UX perspective.

Because we failed at explaining that every political campaign should consider that some of their budget would be wisely spent on IT. Some do think it's the case and improve civi where they think is needed. Some others prefer investing in ALL UPPERCASE comments it seems;)

The one click upgrades from WordPress are why Wordpress is running 20% of the web.

As a corollary, 80% of the web have other criteria and chose another CMS. Neither is wrong, but possibly a sign that IT is complicated and you can't be everything for everyone?

[The Next Wave]

Another "social crm" is Batchbook: http://batchbook.com/tour/social-crm.html
One of the things that I get CRM envy over is how pretty the interface can look-
Why are civi dashboards so butt fugly?

[The Next Wave]

So I tried to get some pretty smart people to help me out with the batch update using the link you suggested JonGold-
and here is what he said:

"You have a Wordpress installation not a "real" installation of CiviCRM utilizing Drupal - nearly all the documentation references the typical drupal install and it seems there are many bugs in the wordpress version. Most of the batch  import/export scripts seem to target the drupal install which provide a restful style API to these objects. (a good reason why drupal is better at managing content than wordpress)

If you choose Entity=Contact and action=getfields you can see a listing of all the 122 different values you can map to. If I'm reading this properly you'll have to map the column header name to the 'name' of the field you care about in the data. You should be able to rename the header using excel or the tool I showed you Saturday - if then export this as a .csv file you should be able to theoretically utilize the batch scripts you linked to. However - this again seems to really be geared towards drupal installations - not wordpress and from reading the bug reports and usage scenarios - this is really intended to be utilized with drupal... I'm doubtful it'd even work. You are probably better off just using excel/google refine to just sort and delete the ones you care about and uploading them in chunks using the web interface (which you've already taken the time to do the mapping)."

So- I say again-
4) Batch updating- I'm sorry, but 1mb uploads is insane. Just one county in Ohio takes 118mb. Why can't we upload a CSV to the server- and Civi parses and updates and generates an error report. Data that isn't easy to update is data that becomes stale.

Somehow- this needs to be simplified.
The fact that I also have to run this through a data cleanser- Google Refine is very helpful-  or now- "Open refine" http://openrefine.org/
says that there is room for improvement.

[JoeMurray]

Thanks very much The Next Wave for your list and your on-going engagement. I think it is a very good wish list. If you know of any ways to work on getting funding for development of these features the core team and other community members would be interested in helping you. Unfortunately, as an open source project we rely on the community of users and their funders to pay for enhancements. A few points.

I'm in the midst of enhancing core so that multiple website fields for a contact can be added to a profile (http://issues.civicrm.org/jira/browse/CRM-9288). I'd love your feedback on http://forum.civicrm.org/index.php/topic,31928.0.html.

This will not provide deeper integration with social media in terms of pulling in a contact's posts to their social media accounts.

Separately, I am working with http://attentive.ly on an integration that we will release as an extension. Version 1 will synchronize contacts back and forth, display on a contact's dashboard the social media posts available in attentive.ly for them, and integrate watch terms between CiviCRM and attentive.ly.

With respect to the import, you can modify the upload the maximum size of upload files at Administer > System Settings > Misc. I think that will change the import file size restriction, but you'd need to check. However, normally browser uploads are only good for a maximum of 8-10MB; bigger than that and you should move to ftp/sftp. While I do NOT think the following is appropriate for users who are not programmers , I thought as an interim workaround you might find the following recipe I created many years ago for a poor political party of potential interest: http://wiki.civicrm.org/confluence/display/CRMDOC/Import+optimization+-+Pre-screening+for+new+and+changed+records

Thanks again for your great list.

[joshmccormack]

For the uploading data issue, could using Google Drive spreadsheet be an option? Uploading spreadsheets up to 20mb is easily done and I think there are sdks and APIs that could be use to connect from CiviCRM.

[xavier]

Hi,

There are already sql and csv as options, so it theory possible to add a 3rd one.

I'm not sure about the API and stuff like authentication on google side.

Something that would probably help is to properly separate the "upload" part from the "import into civi" part. This is more or less done already at the code level (using temporary tables) but would benefit from a proper UI.

Import is a part of civi that would benefit from a redo from scratch and using the new tools planning for the next big version (doctrine...). However IMO, Import is a part that is just not broken enough that users would want to fund (MIH) a brand new one. But I'd love being proven wrong.

X+

[joshmccormack]

It's not an she for me, but might be good if people had a ballpark effort estimate so they'd know what that work would take. Thoughts?

[Eileen]

I think being able to import from a google doc would be really useful- I'm not sure what would be involved though - presumably some sort of OAuth authentication - my guess is that I'd want to have a clear week in my schedule to tackle it. I think I'd have a go in the csvimport extension rather than in core if I were going to try. That's quite an intriguing idea.

Re FB integration - one SAAS I reviewed recently included not only the ability to post events to FB but the ability to book through FB too - presumably unpaid events only? Or possibily pay later too? That is quite appealing. I'd love to spend some time figuring out what that would take too. However, I have no idea without a bit of discovery what would be involved & my suspicion is that enhancing the csvimport extension to import from google docs would be easier

(CSVImport is like Xavier's tool - it imports to the api directly rather than having a bunch of specific coded mappings like the inbuilt tool - which is a strength & a weakness)