Merge 'Custom Fields' administration with 'Profiles'

[Matt2000]

In my experience, users just don't grok the differentiation between administration of Custom Field Groups and Profiles, and I think the UI could be streamlined quite a bit by making 'Custom Fields' something that happens in the background as apart of Profiles.

The separate permissioning is also prone to causing errors which breach security/privacy. I've seen it. Another frequent confusion is the difference between making a field 'required' in the custom field group versus making it 'required' in the profile. I've never yet run into a case where the user really wanted a custom data field to be universally required

A very common task when first configuring CiviCRM is for a user to want to collect some custom information during user registration. Currently, the process is:
Create custom field group > Create Custom Field > Permission Custom Field Group > Create Profile > Add Fields to Profile > Ensure Anonymous users have permissions for Profiles. I'm not even going to count the number of page loads this requires.

My thought is that this can be greatly simplified by making custom fields a 'feature' of Profiles, rather than a separate entity. Those blocks of data that appear on a contact's page as a field set or tab? They are "profiles" too (which might not be exposed to the world, and contain only custom data). A profile is then simply defined as a grouping of fields, whether for input or display, whether custom or standard, whether displayed as a standalone page, or as a part of  contact record page, or as a table or search results. How do you create custom fields? You create a profile, because obviously you'll need a way to input and display that group of fields.

Basically, the concept of custom field 'groups' would be removed. Custom fields would be treated like any other field. Profiles would be permissioned by ACL in the way that Custom groups are now; or for a bigger project, move to field-based ACLs for all fields rather than group-based ACLs.

I think this would greatly improve usability and workflow efficiency. Any thoughts on this from the Development team?

[Michał Mach]

Hey Matt,

Your proposition seems reasonable when it comes to custom fields extending Contacts, however, custom data groups extend all kinds of objects - Activities, Relationships, Contributions, Memberships, etc etc - you can create them for other purposes than just building profiles. Therefore, we cannot remove whole concept of custom field groups.

However, your reasoning on Profiles/Custom Data complexity makes sense. Could you try to discuss this further in this thread, taking above into consideration?

Thx,
m

[Donald Lobo]

Some other things to consider:

1. profiles are a combination of custom data and built-in fields (like name, address etc)

2. custom data can be reused across different profiles

lobo

[Matt2000]

@Michael,

I'm sure this needs further thought, but Profiles as they are now are not limited to Contact records either. They an be added to Event Registration, Membership sign-up, etc.

This may not be so much a change in architecture as a change in terminology & the admin interface. We add a 'Profile' to a Relationship, Activity, whatever, though in the background, it may function exactly as custom groups do now.

It might however, open the opportunity for allowing a custom field to be used across different objects, much like CCK allows a field to be used across different node types in Drupal. (Sometimes a field might have a slightly different meaning when associated with a different record type, but I bet there a performance advantage to sharing custom fields)

For example, an organization might want to track something called 'Initiation Date' on Contacts, Relationships, Memberships, and Groups, which is different from the built-in "Start Date" fields. (Maybe it marks the end of a Training period for new volunteers, for example, and would be tracked on the Contact record, and on the Activity Record for the training activity itself, and on  the relationship record for the Trainer-Trainee relationship.)

I realize the above is probably an outlaying use case, but it does demonstrate the usefulness of rethinking the way concepts in CiviCRM may be unnecessarily divided. I think numerous usability improvements can be made by mering concepts. Custom Group = Profile. Membership Organization = Domain Information. Membership = Group. Event = Activity.

To continue the general principle: Is it really useful to sharply divide Contacts into Individual, Organization, and Household? Another recent thread discussed the frequent blurring between Organization and Individual for some uses. Could one record represent both, if a user wanted it too? There are no Organization-Contacts per-se, there are just contacts who happen to have the Organization-Profile filled out. For most users, there would be no change in the way they work with or think about their data. They think "I need to look up an Organization" and the software knows, "Only look at contacts with at least one completed field in the Organization profile." But the removal of divisions in general opens up the possibility for innovation and more creative use of the tools CiviCRM provides.

(By the way, I'm not actually proposing the above change. I'm just brainstorming. Some actual usability testing would go a long way toward smashing my wacky abstract theories about usability.)

The general concern is simplifying the complex machine that CiviCRM has become over the years through the addition of new features. Sometimes less is more, and sometimes you can make it feel less overwhelming without actually taking away features. But I digress...


@Lobo,

1. Yes, but a profile can also be only custom fields, in which case it would function exactly like a custom group does now. In the case of displaying my profile-hybird in a contact record, I might suggest display redunant built-in fields when the profile-hybrid is displayed as a tab, but only showing the custom fields  of the profile when it is displayed as a fieldset along with built-in fields.

2. It still could be. I'm simply suggesting combining the steps of creating and custom field and adding it to it's first profile. Once it's been created, it appears in the selector for future profiles along with all built-in fields.

While this is under considering, I've been thinking or along time about the idea of build a flash or AJAX widget for a new drag-and-drop interface for building profiles, more like SugarCRM's UI, if I remember it correctly. Would there be demand for that sort of thing? Maybe that would be a better way to implement these ideas.

[Donald Lobo]

my 2 cents ...

1. I think the questions etc, are absolutely the right thing to do We need to simplify things while adding functionality at the same time We do this, but does not hurt to make radical changes periodically

2. profiles and custom data is confusing to a lot of early users and merging / simplifying this in some way is a good step forward.

3. Now is a good time to start these discussions since we can start working them into a 3.0 release (maybe after a quick 2.2 release?)

4. We should focus on a few things to cleanup / simplify. From the below list, i'd vote for: Custom Group / Profile, cleanup / improve contact distinctions (i.e make them extensible and add workflow or merge them)

5. Finally, talk is cheap. So matt, looking forward to your ajax widget to build profiles And please dont do it in jQuery (for 2.x at least)

lobo

[Dave Greenberg]

I'm reading this while on a "break" - so not fully absorbing etc. However it occurred to me that one approach to addressing some of the workflow / usability issues that Matt is addressing (prior to 3.0 and hence w/o a major architectural revamp) would be to allow folks to create custom fields in the context of building a Profile.

- Create Profile
- Add some existing fields
- Notice that I need a field for this Profile (form) that doesn't exist yet
- Click "create new (custom field)"
- Create the field (with the option inline to put it in an existing custom field group or start a new one)

Along these lines we could look at having default custom field groups (which are represented as tables in the current architecture) for each top-level record type. This would potentially allow folks to bypass thinking about creating custom field "groups" if they didn't need that level of complexity in their data.

[Matt2000]

Dave gets it. :-)

[Michał Mach]

Matt,

Seems like I missed quite some discussion over the offline weekend. :-) A few remarks without getting into the core of discussion:

First of all, thanks again for raising the issue and taking it further in the discussion. IMHO, it's really important for us (the team) to be able to think about such ideas, discuss over doubts and get constructive feedback, like yours.  :-)

The general concern is simplifying the complex machine that CiviCRM has become over the years through the addition of new features. Sometimes less is more, and sometimes you can make it feel less overwhelming without actually taking away features. But I digress...

Good kind of digression. :-) It seems like it's an important discussion to have - let's come back to it periodically. As Lobo stated, we really should spent some time ruminating over usability before 3.0 decisions are made.

While this is under considering, I've been thinking or along time about the idea of build a flash or AJAX widget for a new drag-and-drop interface for building profiles, more like SugarCRM's UI, if I remember it correctly. Would there be demand for that sort of thing? Maybe that would be a better way to implement these ideas.

I'm personally sure there would be demand.

Thx,
m

[Michał Mach]

Dave gets it. :-)

Matt, since you started the discussion, do you want to take the honour and file an issue with Dave's suggestion, and perhaps extend it with your ideas if needed? :-)

We can continue the discussion on other usability improvements and file issues as needed, this one can be an immediate outcome and step forward.

Thx,
m

[Matt2000]

@Lobo

5.) Talk is indeed cheap. that's why I give it away for free. :-) Development, however, is not so cheap....

That said, here's a VERY rough UI mock-up of what I'm thinking for a new Profile-Builder widget:

http://www.ninjitsuhosting.com/profiledemo/index.html

I'm fairly confident I could build the back-end to this using the APIs as they are now.

How much demand is there for something like this? Is finding sponsorship for this a feasible goal?

@Michael,

Will do.

[Donald Lobo]

1. the UI mockup is quite nice and packs a lot of functionality in one screen load. If you have not done so, you should also check the views2 package for an ajaxy view of how to do something similar. I'm not sure if u r planning on using openlaszlo going forward with this?

Which brings us to the question, of whether we should think about moving the "admin" based functionality of CiviCRM to a more desktop like environment like adobe flex / laszlo etc

I think having a more streamlined, simpler intuitive UI will always be in great demand sponsorship is a bit more murky? not sure how many folks will step up and help fund it. We've not had great success getting funding for civivoter / civiphonebank / civipledge when we thought that the need was quite evident out there

lobo

[Matt2000]

Well, I've got funded to do my own take on Phone banking, which I do intend to release in some form, so once that's done, I guess I'll start by hitting up my own client base for ProfileBuilder funding. :-)

By way of ballparking the cost, my initial thoughts are that my cost to develop this with my own team would be between $3k - $6, so it's not a HUGE undertaking, compared to what the core team might do.

My intention was to continue with lazslo, but I probably should spend some more time with views2 first. Although, that would would limit the functionality to Drupal users only. Not that I'm against that.... :-)

[petednz]

Without wanting to piggyback/freeload etc  keen to know if there is anything I could feed in/feedback on that might make your offering more internationally useful

[Matt2000]

All feedback and input is welcome. But personally, I have zero experience with internationalization, so I'll be dependent upon whatever is built into CiviCRM for translations. I'll be pulling field labels, etc, by API, so as long as the API takes care of translations, field names will be translated. If anything more is needed, it will be dependent on the community.

[Matt2000]

This thread has shifted a bit from it's original purpose, so I've started a new one to discuss the ProfileBuilder project:

http://forum.civicrm.org/index.php/topic,4627.msg20180.html