Notes from the hosting BoF at CiviCon London 2013

[mathieu]

Hi everyone,

We had a "birds of a feather" (BoF) aka informal discussions about hosting & CiviCRM, on the various challenges and strategies. I took a few notes from our discussions. Feel free to add & comment!

What the people present are using, and pros/cons are by no means exhaustive, only some of the points raised during the introductions:

* Dreamhost shared hosting
** Pros: low cost, fully managed.
** Cons: variable performance, no native gettext (more about that below)
* CiviHosting.com: (edit: see response by Hershel below)
** Pros: good service, low cost.
** Cons: jurisdiction
* Virtual private servers (VPS), ex: Linode, Rackspace, Koumbit, OVH.
** Pros: more control on the environment, more resources (memory/CPU)
** Cons: more expensive, requires more sysadmin knowledge, backups may not be included
* Server physically located in the office
** Pros: full control on the environment, hardware, physical security
** Cons: expensive (physical security, hardware maintenance,
   requires a lot of sysadmin knowledge.

Challenges:

- reliability / performance
- jurisdiction / privacy regulations
- backups
- automation of deployments and upgrades (how fast can you deploy a security patch on your sites?)
- enabling native gettext (requires making sure that the host supports the correct locale/language, may require admin rights to configure; however, in another language than US English, using the default php-gettext implementation will cause a 20% overhead).
- learning sysadmin / understanding what kind of server is required for your CiviCRM installation, where the bottlenecks are (hosting providers can be very vague, pretend their systems are all optimal, not be able to identify the bottleneck).
- mysql optimisations: difficult to get proper control on the hosting, mysql config, SSD disks
- caching: memcache, PHP opcode cache (APC, or PHP 5.5) - need more documenting on how it is used by CiviCRM, and how to configure it.
- using nginx, Varnish, optimizing Apache (AllowOverrides, APC)

Hosting providers wiki page:

http://wiki.civicrm.org/confluence/display/CRM/Hosting+provider+information

- wiki page is un-usable
- not clear what works and for what type of requirements (how many contacts, type of use-case ex: SSL, mailings)
- would be nice to have on the official site 1 page per hosting, proper categorisation, comments
- we could let hosting providers advertise on those pages (ex: Joomla), which would bring in new income for sustainable civicrm.

Large imports:

- do them offline, because it will overload your server and can take a lot of time. Better to copy your SQL/files to a local host, or on a separate cloud instance.
- use the CiviCRM API to import large batches (although that involves PHP programming..)
- "inotify" in Linux can automatically launch a commande (API import) when a user uploads a file.
  This way they can use API imports without having to launch the script manually.

Backups:

- do you have any?
- are they reliable, monitored?
- if the server crashes mid-day, what is the impact of losing half a day of data?
- keep e-mail BCC for contributions, quick & cheap way for last result backup
  and make sure you do not loose any data of the contributions for the last few hours.

Thanks to everyone who participated!

I think this conversation identified a few concrete issues that make it hard to start using CiviCRM. What are your thoughts on how we can better address those issues?

Mathieu

PS: edited to put emphasis on the sentence about the pros/cons list not being exhaustive, and mentioned CiviHosting's response.

[billa]

Mathieu,
Thanks for leading that BoF session yesterday.
For reference inotify-tools info (https://github.com/rvoicilas/inotify-tools/wiki) should help people get an idea of what's possible. You can also (for example) monitor the site directory tree as a whole and do backups on change, etc.

Andy

[Hershel]

* CiviHosting.com:
** Pros: good service, low cost.
** Cons: jurisdiction
* Virtual private servers (VPS), ex: Linode, Rackspace, Koumbit, OVH.
** Pros: more control on the environment, more resources (memory/CPU)
** Cons: more expensive, requires more sysadmin knowledge, backups may not be included

I would like to clarify a few points about CiviHosting, for the record. By way of disclaimer, CiviHosting is my company.

• You mentioned "jurisdiction" as a "con" of CiviHosting. It's not 100% clear from your post what that means, but if the issue is having your site hosted on a server in Europe, CiviHosting has servers in Europe.
  
• You did not list "fully managed" by CiviHosting, yet all of our servers, including VPS's, are fully managed by our staff.
  
• You did not list CiviHosting in the list of VPS providers, yet we do have a VPS package. With the regard to that, I would also note that in your list of "cons" for VPS, you noted:
  • "requires more sysadmin knowledge" -- With a VPS from CiviHosting, no sysadmin knowledge is required whatsoever as we manage the server for you
    
  • "backups may not be included" -- Our VPS's come with full automatic daily backups stored on a separate server for 15 days

- would be nice to have on the official site 1 page per hosting, proper categorisation, comments

The CiviCRM team is well aware that the wiki page on hosting is not useful and this solution you mentioned above is actually presently in the works and should be released in the near future.

Thank you.

[Eileen]

Hershel,

In light of the recent security release can you please clarify CiviHosting's procedures around patching clients (ie. is it on the client or did you do some pro-active patching?)

Eileen

[mathieu]

@ Andy: thanks for the tip!

@ Hershel: thanks for the clarifications! I did put as a disclaimer: "What the people present are using, and pros/cons are by no means exhaustive, only some of the points raised during the introductions".

I think that the points raised by the participants would make a good list of criteria to help people choose what is the best hosting for them. For what it's worth, your website does not indicate anything about which jurisdiction you operate in, e.g. in which country the company is registered in, and where are your servers physically located.

http://civihosting.com/about-us
http://civihosting.com/contact

This applies to all providers. Please don't take this personally against CiviHosting. I wish that all providers were as responsive on the forums as you are (and civi partners!).

EDIT: I modified my original post to put emphasis on the sentence about the list not being exhaustive, to make sure that people do not use that list as an evaluation criteria.

[Hershel]

In light of the recent security release can you please clarify CiviHosting's procedures around patching clients (ie. is it on the client or did you do some pro-active patching?)

To be very clear, CiviHosting is not SAAS--it is "traditional" hosting meaning that the client has full access to his webspace, including files, databases and email accounts. We offer a service to install a CMS and CiviCRM for our clients, but our clients are also free to install whatever software (including packages completely unrelated to CiviCRM) they want and then to customize that software as they see fit.

As a service to our clients, we do provide a service to run upgrades to CiviCRM for a small fee. The client must order that from us, however, and such upgrades we do on a per-account basis.

Our clients run a large variety of CiviCRM versions and, in general, security of their web software (including CMS and CRM) is their responsibility. We are not able to provide patches for each security advisory for every version of CiviCRM. We do, however, take pro-active measures to scan our servers on a regular basis to look for malicious software and we do sometimes find such. When a site runs insecure software that allows a hacker to install malicious software, there's not a whole lot a host can do to guard against that.

Sometimes, we are able to. For example, regarding the CIVI-SA-2013-001 - OpenFlashChart File Upload security advisory, we did find instances of hackers exploiting this vulnerability, and we were able to safeguard all of our servers from any further incidents by blocking access to the vulnerable PHP file. Thus we protected all of our clients from this issue.

With other security issues, such as the latest CIVI-SA-2013-009 - SQL Injection Vulnerability security advisory, there is not an effective solution we can apply on the server end to protect sites against this. For such issues, our clients must take measures to protect their own sites. They certainly can simply hire us or another service provider to upgrade their site, but anyhow this is their responsibility.

For what it's worth, your website does not indicate anything about which jurisdiction you operate in, e.g. in which country the company is registered in, and where are your servers physically located

For clarity, our site actually does mention right on the home page that "we have servers in the USA and in Europe," but perhaps the type there is too small to be prominent. The site furthermore requires one to choose a server location of USA or Europe on the hosting purchase pages, which makes that quite clear, but perhaps European visitors don't ever get to those pages if they think our servers are only in the USA.

At any rate, it seems we should endeavor to make this more clear for first time visitors, and we will work on that. Thank you for the feedback.

[Upperholme]

I wasn't able to get along to this BOF session - I sat in on the Wordpress discussion instead. But it looks to have been a good discussion, and thanks to Mathieu for taking and writing up some notes from it. Useful to learn that work is underway to provide some better hosting related information on civicrm.org.

For the record I use Civihosting's Europe-based hosting for a couple of sites I work with, and have found it to be very good value. Very reliable service and excellent support. Not the cheapest, but you tend to get what you pay for with hosting.

I also use a VPS provided by webarchitects.coop in Sheffield. Again, not cheap, but a good service, despite some recent hardware issues. Their servers are highly energy efficient and powered by renewable energy, if that's an important factor for anyone (if it isn't it should be). And for anyone with particularly sensitive data, they also run http://ecodissident.net which offers fully encrypted servers based in Iceland, powered by geothermal energy (not sure if that classes as renewable?). Webarchitects.coop is structured as a multi-stakeholder cooperative, and as a user of their services I am also a member of the cooperative. Interesting model.

[mathieu]

Indeed, you often get what you paid for! (although that more often applies to the cheap hosting solutions, not always to the more expensive ones)

Thanks everyone for the interesting comments. We now have a lot of good material to work with for building a more useful hosting providers page.