2000+ Tags management

[nadfas]

Hi,

We've got number of  issues working with huge amount of tags (2000+).  Essentially, we've tried to tag our lecturers by specializations, thus Tag Set has been created with 4 parent tags and 300-800 child tags for each parent one. Import went fine, but later we've found that it's impossible to load *Manage Tags* page ( /civicrm/admin/tag?reset=1 ). Guess, paging would fix this, but not sure where to start, since we've just started with custom programming in civicrm. Besides, I have a little hope that paging for tags management page deserves implementation at *global* level, not as custom modification for solely purposes of one organization. Please advice, should it be added as feature request in Jira? And what would be a possible workaround while/until request will be accepted/deployed?       

Also, tags autocomplete field at Add new Individual page suggests only 4 parent tags, not single child; there is canceled GET request ( civicrm/ajax/rest?entity=tag&action=getlist&json=%7B%22params%22%3A%7B%22parent_id%22%3A%228%22%7D%2C%22input%22%3A%22C%22%2C%22page_num%22%3A1%7D&_=1424001982479 ), guess it's relevant.  Most likely, it's something with ajax performance, since all tags we have are visible at Views Exposed Filter in Drupal ( just a plain drop down, w/o hierarchy though).  Any advice of how to fix this?

Thank you in advance for any suggestions!

[Coleman Watts]

This would be a great feature for your organization to sponsor. If you are interested we can supply you with an estimate for the work.

[xavier]

When does the problem happen? is this that the page timeout before being displayed or that the page is properly sent but never displayed because too long?

If the former, you can increase the duration php can work on a page before giving up (set_time_limit() in php, by default 30 sec, try to increase to one of two minutes)

At the implementation level, each tag contains its parent id and based on that we can rebuild the hierarchy... that is CPU expensive. They are other ways (eg. materialised path) of storing the hierarchy that offer faster reads.
https://vadimtropashko.files.wordpress.com/2011/07/ch5.pdf

X+

[nadfas]

This would be a great feature for your organization to sponsor. If you are interested we can supply you with an estimate for the work.

So you don't expect this feature to be useful for anyone? Just trying to figure out difference between feature request and sponsored development. But yes, please, PM me estimation details.     

[nadfas]

When does the problem happen? is this that the page timeout before being displayed or that the page is properly sent but never displayed because too long?

If the former, you can increase the duration php can work on a page before giving up (set_time_limit() in php, by default 30 sec, try to increase to one of two minutes)

At the implementation level, each tag contains its parent id and based on that we can rebuild the hierarchy... that is CPU expensive. They are other ways (eg. materialised path) of storing the hierarchy that offer faster reads.
https://vadimtropashko.files.wordpress.com/2011/07/ch5.pdf

X+

Thank you for response and question. Page never displays, page load just consumes PC resources and freezes browser tab before anything appears.   

[Coleman Watts]

So you don't expect this feature to be useful for anyone?

It would be useful to you, and anyone else who might have this problem in the future. By sponsoring this feature you would be joining in with hundreds of others who have done the same thing over the years - fund development for the functionality they need. The beauty of open-source is that when one person steps up to make something happen, everyone benefits.

Just trying to figure out difference between feature request and sponsored development.

The difference is between talking about something vs actually doing something. Developing features costs money, so while discussions and ideas are nice, in the end it doesn't happen unless someone steps up to fund it.

People who use CiviCRM are all benefitting from the hundreds who have come before them to fund and volunteer for development of essential features. It's a great system that benefits everyone, and ultimately saves all organizations who use it a ton of money. If you can download for free a CRM that meets 99% of your needs, and only have to pay to develop the 1% that's missing, it's a huge savings over paying 100% yourself! And it works because people who came before you have chipped in their contribution to make CiviCRM the full-featured system it is today.

[nadfas]

So you don't expect this feature to be useful for anyone?

It would be useful to you, and anyone else who might have this problem in the future. By sponsoring this feature you would be joining in with hundreds of others who have done the same thing over the years - fund development for the functionality they need. The beauty of open-source is that when one person steps up to make something happen, everyone benefits.

Just trying to figure out difference between feature request and sponsored development.

The difference is between talking about something vs actually doing something. Developing features costs money, so while discussions and ideas are nice, in the end it doesn't happen unless someone steps up to fund it.

People who use CiviCRM are all benefitting from the hundreds who have come before them to fund and volunteer for development of essential features. It's a great system that benefits everyone, and ultimately saves all organizations who use it a ton of money. If you can download for free a CRM that meets 99% of your needs, and only have to pay to develop the 1% that's missing, it's a huge savings over paying 100% yourself! And it works because people who came before you have chipped in their contribution to make CiviCRM the full-featured system it is today.

Off top on:

Thank you for reply. Sorry if question seemed to be non polite, it hadn't meant to. And it's not about money and getting for free something others invested their time and efforts to. We've started civicrm adopting process  so trying to figure out best way of support seeking (apart of googling and stackexchanging). Please correct me if I'm wrong. CiviCRM community has:
#0 Online documentation (including API Explorer)
#1. Forum for "self-support" by asking questions and searching for answers and for supporting others.
#2. JIRA for bug reports and (among other) features request which could but haven't to be reviewed / implemented by volunteering civi staff team.
#3. "Sponsored development" channel where it is possible to ask  for support for money: you've just shown that this possibility exists.

It's not clear at these very first steps which channel to use when question arrives and we don't want to bother others by, say, creating useless forum posts which will remind unanswered or filling JIRA requests which aren't important and so on. That's why question about sponsorship has been asked. 

Off top off



Please shoot me a PM with detailed *sponsorship package* proposal. Thank you in advance!

[Coleman Watts]

No worries, your questions are good and very appropriate for the forum. There really are no "bad questions" here and asking for clarification is great.

Your take on the various channels is pretty accurate. The issue tracker tends to be a lot more focused, and issues posted there which are not sponsored improvements, code contributions, or reproducible bugs tend to get referred back to the forum, which is the "catch-all" for general discussion. "Sponsored development" isn't its own channel but uses 1) this forum (for the initial discussion), 2) private email (for the contract negotiations), and 3) the issue tracker (for logging progress).

Another clarification: there are no "volunteer staff" at CiviCRM. The core team is paid (by sponsoring organizations and occasional grant funding) to do "big picture" architecture, release management, sponsored features, and critical bug fixes. Other volunteers are people like, well, you, who have a need for an improvement or bug-fix, and "volunteer" to implement it themselves. But there are no volunteers (that I know of) who just sit around all day waiting for someone else to suggest an improvement that they can work on. Wouldn't it be nice though if that were the case; I have a very long list I could give them...

Will put together sponsorship info for you shortly...

[nadfas]

Let me get back to this topic with partially solution: since CiviCRM utilizes Datatables jQuery plugin, we've been able to enable Tags paging in jsortable.tpl.php; however this solution (better to say hack) arises a question of how it would be possible to enable paging for Tags management only? My worries is about jsortable might be used anywhere else so such "global" paging solution may impact other parts of civi we haven't mentioned yet. Besides, now we need to re-apply our hack each time civicrm upgrade occurs, thus it would be nice to have a custom jsortable: is it possible at all? Thank you for any suggestion(s)!