Author Topic: yahoo geo coding changes (Read 1048 times)

jamie · November 30, 2012, 11:00:01 am

Yahoo released a blog post in which they said:

Quote

If you are looking to continue experimenting with the standalone PlaceFinder and PlaceMaker services, we are excited to continue supporting you through our YQL tables. These tables will be limited to 2,000 queries per day and are meant for non-commercial usage. We intend to shut down the current free versions of PlaceFinder and PlaceMaker on November 17, 2012.

I'm pretty confused trying to understand their documentation for their APIs, but it seems like the current Yahoo geocoding services in CiviCRM are using the free versions of PlaceFinder that Yahoo says they are going to shutdown on November 17.

Also, from experience, I can say that these have not been shutdown as of today.

I took a look at the YQL stuff that is supposed to remain free and can't figure out how to coax geo coordinates out of it for an address.

It's possible that Yahoo geo coding is about to break.

PTP is starting to look for alternatives for geocoding - we came across a PHP api that provides a list of providers (I'm not suggesting we use the API - just referencing the page because of the useful list of available providers). I wonder if anyone has had experience with any of those providers and if we should consider working on adding them to CiviCRM?

Note - i opened a related issues about the ugly error you get when you exceed your Yahoo limit

xavier · November 30, 2012, 01:05:18 pm

It would be great to provide openstreet map indeed.

However, I've read that it's still behind google. In fact, the only one that seems to be in the google map league seems to be bing from microsoft.

Are you using google map? Is this a technical choice not to use it?

Donald Lobo · December 01, 2012, 04:26:17 pm

seems like we should pull yahoo out of the release for 4.3 for sure

Jamie, any thoughts if we should pull it for 4.2.7 also?

lobo

flug · February 15, 2013, 10:15:34 pm

FYI, here is code that is working with the new YQL system, based on the existing yahoo.php file in /CRM/Geocode/Yahoo.php:

Code: [Select]

<?php
/*
  +--------------------------------------------------------------------+
  | CiviCRM version 4.2                                                |
  +--------------------------------------------------------------------+
  | Copyright CiviCRM LLC (c) 2004-2012                                |
  +--------------------------------------------------------------------+
  | This file is a part of CiviCRM.                                    |
  |                                                                    |
  | CiviCRM is free software; you can copy, modify, and distribute it  |
  | under the terms of the GNU Affero General Public License           |
  | Version 3, 19 November 2007 and the CiviCRM Licensing Exception.   |
  |                                                                    |
  | CiviCRM is distributed in the hope that it will be useful, but     |
  | WITHOUT ANY WARRANTY; without even the implied warranty of         |
  | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.               |
  | See the GNU Affero General Public License for more details.        |
  |                                                                    |
  | You should have received a copy of the GNU Affero General Public   |
  | License and the CiviCRM Licensing Exception along                  |
  | with this program; if not, contact CiviCRM LLC                     |
  | at info[AT]civicrm[DOT]org. If you have questions about the        |
  | GNU Affero General Public License or the licensing of CiviCRM,     |
  | see the CiviCRM license FAQ at http://civicrm.org/licensing        |
  +--------------------------------------------------------------------+
*/

/**
 *
 * @package CRM
 * @copyright CiviCRM LLC (c) 2004-2012
 * $Id$
 *
 */

/**
 * Class that uses Yahoo! PlaceFinder API to retrieve the lat/long of an address
 * Documentation is at http://developer.yahoo.com/geo/placefinder/
 */
class CRM_Utils_Geocode_Yahoo {
  

  /**
   * server to retrieve the lat/long
   *
   * @var string
   * @static
   */
  //update to new yahoo YQL
  //The new http query looks like this when assembled
  //http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20geo.placefinder%20where%20text%3D%223916%20Main%20St%2C%20Anytown%20GA%2032333%22&diagnostics=true
  
  //Change to yql; old code was:
  //static protected $_server = 'where.yahooapis.com';
  static protected $_server = 'query.yahooapis.com';
  /**
   * uri of service
   *
   * @var string
   * @static
   */
  //Change to yql
  static protected $_uri = '/v1/public/yql';
  //static protected $_uri = '/geocode';
  /**
   * function that takes an address array and gets the latitude / longitude
   * and postal code for this address. Note that at a later stage, we could
   * make this function also clean up the address into a more valid format
   *
   * @param array $values associative array of address data: country, street_address, city, state_province, postal code
   * @param boolean $stateName this parameter currently has no function
   *
   * @return boolean true if we modified the address, false otherwise
   * @static
   */
  static function format(&$values, $stateName = FALSE) {
    
    CRM_Utils_System::checkPHPVersion(5, TRUE);

    $config = CRM_Core_Config::singleton();

    //switching this to yahoo's yql server
    $add="q=select%20*%20from%20geo.placefinder%20where%20text%3D%22";

    //$add = "appid=" . urlencode($config->mapAPIKey);

    //$add .= '&location=';
    //addslashes needed for YQL because it chokes on
    //" \ and nul byte. It should be fine with ' but escaping it
    //does no harm.
    if (CRM_Utils_Array::value('street_address', $values)) {
      $add .= urlencode(addslashes($values['street_address']));
      $add .= ',+';
    }

    if ($city = CRM_Utils_Array::value('city', $values)) {
      $add .= urlencode(addslashes($city));
      $add .= ',+';
    }

    if (CRM_Utils_Array::value('state_province', $values)) {
      if (CRM_Utils_Array::value('state_province_id', $values)) {
        $stateProvince = CRM_Core_DAO::getFieldValue('CRM_Core_DAO_StateProvince', $values['state_province_id']);
      }
      else {
        if (!$stateName) {
          $stateProvince = CRM_Core_DAO::getFieldValue('CRM_Core_DAO_StateProvince',
            $values['state_province'],
            'name',
            'abbreviation'
          );
        }
        else {
          $stateProvince = $values['state_province'];
        }
      }

      // dont add state twice if replicated in city (happens in NZ and other countries, CRM-2632)
      if ($stateProvince != $city) {
        $add .= urlencode(addslashes($stateProvince));
        $add .= ',+';
      }
    }

    if (CRM_Utils_Array::value('postal_code', $values)) {
      $add .= urlencode(addslashes($values['postal_code']));
      $add .= ',+';
    }

    if (CRM_Utils_Array::value('country', $values)) {
      $add .= urlencode(addslashes($values['country']));
    }

    $add .= "%22&diagnostics=true";
    
    $query = 'http://' . self::$_server . self::$_uri . '?' . $add;
    //query that generates an error, for testing the error routines
    //$query='http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20geo.placefinder%20where%20text%3D%22Lee%22s%20Summit%22&diagnostics=true';    
    //print_r($query);    
    require_once 'HTTP/Request.php';
    $request = new HTTP_Request($query);
    $request->sendRequest();
    $string = $request->getResponseBody();
    //print_r($string);
    
    $xml = simplexml_load_string($string);
    //print_r($xml);
    //exit;
    
    if ($xml === FALSE) {
      // account blocked maybe?
      CRM_Core_Error::debug_var('Geocoding failed.  Message from Yahoo:', $string);
      return FALSE;
    }

    if ((integer)$xml->Error != 0) {
      CRM_Core_Error::debug_var('Geocoding failed.  Message from Yahoo:', (string)$xml->ErrorMessage);
      return FALSE;
    }
    //YQL BOSS seems to return errors with <error></error> as the top-level
    //object and <description> as the text of the error.
    if (is_a($xml->description, 'SimpleXMLElement')&&strlen((string)$xml->description)>0) {
      CRM_Core_Error::debug_var('Geocoding failed.  Message from Yahoo:', (string)$xml->description);
      return FALSE;
    }


    //print_r($xml);
    //echo "1";
    //if (is_a($xml->Result[0], 'SimpleXMLElement')) {
    if (is_a($xml->results->Result[0], 'SimpleXMLElement')) {
      //echo "1.1";
      $result = array();
      //yql
      //$result = get_object_vars($xml->Result[0]);
      $result = get_object_vars($xml->results->Result[0]);
      //print_r($result0);
      //$result = get_object_vars($result0->Result[0]);
      //print_r($result); 
      //exit;
      foreach ($result as $key => $val) {
        if (is_scalar($val) &&
          strlen($val)
        ) {
          $ret[(string)$key] = (string)$val;
        }
      }

      $values['geo_code_1'] = $ret['latitude'];
      $values['geo_code_2'] = $ret['longitude'];
      //echo "2";
      if ($ret['postal']) {
        $current_pc = CRM_Utils_Array::value('postal_code', $values);
        $skip_postal = FALSE;

        if ($current_pc) {
          $current_pc_suffix   = CRM_Utils_Array::value('postal_code_suffix', $values);
          $current_pc_complete = $current_pc . $current_pc_suffix;
          $new_pc_complete     = preg_replace("/[+-]/", '', $ret['postal']);

          // if a postal code was already entered, don't change it, except to make it more precise
          if (strpos($new_pc_complete, $current_pc_complete) !== 0) {
            $msg = ts('The Yahoo Geocoding system returned a different postal code (%1) than the one you entered (%2). If you want the Yahoo value, please delete the current postal code and save again.',
              array(
                1 => $ret['postal'],
                2 => $current_pc_suffix ? "$current_pc-$current_pc_suffix" : $current_pc
              )
            );
            CRM_Core_Session::setStatus($msg);
            $skip_postal = TRUE;
          }
        }
        
        if (!$skip_postal) {
          //echo "3";
          //echo "replacing zip code values" . $ret['postal'];
          $values['postal_code'] = $ret['postal'];

          /* the following logic to split the string was borrowed from
             CRM/Core/BAO/Address.php -- CRM_Core_BAO_Address::fixAddress.
             This is actually the function that calls the geocoding
             script to begin with, but the postal code business takes
             place before geocoding gets called.
          */

          if (preg_match('/^(\d{4,5})[+-](\d{4})$/',
              $ret['postal'],
              $match
            )) {
            $values['postal_code'] = $match[1];
            $values['postal_code_suffix'] = $match[2];
          }
        }
      }
      return TRUE;
    }

    // reset the geo code values if we did not get any good values
    $values['geo_code_1'] = $values['geo_code_2'] = 'null';
    return FALSE;
  }
}

CiviCRM Community Forums (archive)

News:

Author Topic: yahoo geo coding changes (Read 1048 times)

jamie

yahoo geo coding changes

xavier

Re: yahoo geo coding changes

Donald Lobo

Re: yahoo geo coding changes

flug

Re: yahoo geo coding changes