08 February 2015

So now let’s configure the importer to automatically Geocode the addresses as they’re imported.

###Geocoding

Geocoding isn’t the process of obtaining latitude and longitude for a physical address like “55 Second Street, San Francisco, CA 94105”. This can be done a number of ways, but the easiest way to do this on import is to enable the Geocoding API for the Location module in Drupal.

Navigate to Administration » Configuration » Content authoring

#WHITESCREEN

So while writing this article, I did the previous step and got a white screen. After investigation I figured out the location module’s geocoding currently has an outstanding issue preventing it from correctly geocoding. YAY!

Basically, Drupal tries to obtain some data dynamically from a resource on google spreadsheets that is no longer available. Setting aside the wisdom of relying on an external resource like this, we’ll just say that we need to completely move this solution off the Location module and move it to Geocoder/Geofield/Addressfield/GeoPHP.

Sigh. Lesson learned: Don’t make your module dependant on external data sources unless its stated up front. Scrape your shoe in the grass and move on.

So the modules you need to download and install are Geocoder, Geofield, Addressfield, and GeoPHP. Download and enable.

Add an Addressfield to our content object “Retail Partner”. The defaults should be sufficient. Make sure you enable a country-specific address form under “form handlers”.

</a>

After that, we’re going to create a single geo field to hold the geo information. There’s an option on the geofield config to get the geofield’s data by geocoding the addressfield you just created.

</a>

So now there’s just one problem: Google has strict limits on how much you can geocode in any given day. In order to get past those limits, you have to enter a credit card and have google charge your card for overages. Its been my experience that that never amounts to much until you’re geocoding a LOT of points (20k+). I have about 1500 data points to geocode, which importing the list once will put me at or near the usage limits for the day, so I go ahead and get the google account set up to be charged. But that means the geocoding action needs to include my personal API key so google can recognize its me and allow me past the limits.

Currently the shipping version of the Geocoder module does not do that. I found an issue to add the Google API key to geocoding requests and a patch that will be included in a future version of the module. I used git to clone the development version of the module and then patched it and voila! Google geocoding with an API Key.

Now lets expose the the two new fields to the programmable API using the RESTful project:


/**
 * @file
 * Contains RestfulExampleArticlesResource.
 */
class AnkiRestfulPartnerResource extends RestfulEntityBaseNode {

  var $distanceQuery;

  /**
   * Overrides RestfulExampleArticlesResource::publicFieldsInfo().
   */
  public function publicFieldsInfo() {
    $public_fields = parent::publicFieldsInfo();

    $public_fields['nid'] = array(
      'property' => 'nid',
    );
    $public_fields['vid'] = array(
      'property' => 'vid',
    );
    $public_fields['status'] = array(
      'property' => 'status',
    );
    $public_fields['uuid'] = array(
      'property' => 'uuid',
    );
    $public_fields['language'] = array(
      'property' => 'language',
    );
    $public_fields['address'] = array(
      'property' => 'field_addressfield'
    );
    $public_fields['geo'] = array(
      'property' => 'field_geo'
    );
    
    return $public_fields;
  }

  public function getList() {
    $request = $this->getRequest();
    $autocomplete_options = $this->getPluginKey('autocomplete');
    if (!empty($autocomplete_options['enable']) && isset($request['autocomplete']['string'])) {
      // Return autocomplete list.
      return $this->getListForAutocomplete();
    }
    $entity_type = $this->entityType;
    if (array_key_exists("lat", $request) || array_key_exists("zip", $request)) {
      return $this->proximityQuery($request);
    }
    else {
      $result = $this
        ->getQueryForList()
        ->execute();
    }


    if (empty($result[$entity_type])) {
      return array();
    }

    $ids = array_keys($result[$entity_type]);

    // Pre-load all entities if there is no render cache.
    $cache_info = $this->getPluginKey('render_cache');
    if (!$cache_info['render']) {
      entity_load($entity_type, $ids);
    }

    $return = array();

    foreach ($ids as $id) {
      $toReturn = $this->viewEntity($id);
      $toReturn['location'] = array_merge($toReturn['addressfield'], $toReturn['geo']);
      $return[] = $toReturn;
    }

    return $return;
  }

  /**
   * Overrides RestfulEntityBase::getQueryForList().
   *
   * Expose only published nodes.
   */
  public function getQueryForList() {
    $entity_type = $this->getEntityType();
    $entity_info = entity_get_info($entity_type);
    $query = new EntityFieldQuery();
    $query->entityCondition('entity_type', $this->getEntityType());

    if ($this->bundle && $entity_info['entity keys']['bundle']) {
      $query->entityCondition('bundle', $this->getBundle());
    }
    if ($path = $this->getPath()) {
      $ids = explode(',', $path);
      if (!empty($ids)) {
        $query->entityCondition('entity_id', $ids, 'IN');
      }
    }


    $this->queryForListSort($query);
    $this->queryForListFilter($query);
    $this->queryForListPagination($query);
    $this->addExtraInfoToQuery($query);

    $query->propertyCondition('status', NODE_PUBLISHED);

    return $query;
  }

  public function proximityQuery($origin = NULL) {
    if ($origin == NULL) {
      return [];
    }

    $max_age = variable_get("page_cache_maximum_age", 0);

    if ($max_age !== 0) {
      drupal_add_http_header("Expires", gmdate(DATE_RFC1123, strtotime("+ ".$max_age." seconds")));
      drupal_add_http_header("Cache-Control", "public, max-age=".$max_age);
    }

    $toReturn = ["origin" => []];
    if (array_key_exists("zip", $origin)) {
      $origin += (array) $this->geocodeZip($origin['zip']);
      $toReturn['origin']['zip'] = $origin['zip'];
      //drupal_add_http_header("Expires", date(DATE_RFC850, strtotime("+2 weeks")));
    }
    if ($this->validateOrigin($origin) === true) {
      $toReturn['origin'] += [
        "coords" => [
          "latitude" => $origin['lat'],
          "longitude" => $origin['lon'],
        ]
      ];
    } else {
      return [];
    }

    if (array_key_exists("limit", $_REQUEST) && $_REQUEST['limit'] <= 20) {
      $limit = intval($_REQUEST['limit']);
    } else {
      $limit = variable_get("anki_partner_result_query_limit", 20);
    }


    //variable_set("location_default_distance_unit", "miles");
    $distance_unit = variable_get("location_default_distance_unit", "km");

    $query = db_select("field_data_field_geo", "geo");
    $query->join("node", "n", "n.nid = geo.entity_id and n.vid = geo.revision_id and n.type = geo.bundle and n.status = 1");
    $query->fields("geo", array("field_geo_lon", "field_geo_lat", "entity_id", "revision_id"));
    $query->addExpression($this->earth_distance_sql((float) $origin['lon'], (float) $origin['lat']), "distance");
    $query->condition("geo.entity_type", $this->getEntityType());
    $query->condition("geo.bundle", $this->getBundle());
    $query->condition("field_geo_lon", 0, "!=");
    $query->condition("field_geo_lat", 0, "!=");
    $query->orderBy("distance");
    $query->range(0, $limit);
    $results = $query->execute();


    if ($results->rowCount()) {
      while ($result = $results->fetchObject()) {
        $node = $this->viewEntity($result->entity_id);
        $node += [
          'location' => array_merge($node['address'], $node['geo']),
          "raw_distance" => $result->distance,
          'scalar' => round($result->distance / (($distance_unit == 'km') ? 1000.0 : 1609.347), 1),
          'distance_unit' => $distance_unit
        ];
        $toReturn[] = $node;
      }
    }


    return $toReturn;
  }


  /*
   * Returns the SQL fragment needed to add a column called 'distance'
   * to a query that includes the location table
   *
   * @param $longitude   The measurement point
   * @param $latibude    The measurement point
   * @param $tbl_alias   If necessary, the alias name of the location table to work from.  Only required when working with named {location} tables
   */
  function earth_distance_sql($longitude, $latitude, $tbl_alias = '') {
    //TODO: remove hardcoded location module dependency
    if (!function_exists('earth_radius')) {
      require_once(DRUPAL_ROOT."/sites/all/modules/contrib/location/earth.inc");
    }
    // Make a SQL expression that estimates the distance to the given location.
    $long = deg2rad($longitude);
    $lat = deg2rad($latitude);
    $radius = earth_radius($latitude);

    // If the table alias is specified, add on the separator.
    $tbl_alias = empty($tbl_alias) ? $tbl_alias : ($tbl_alias . '.');

    $coslong = cos($long);
    $coslat = cos($lat);
    $sinlong = sin($long);
    $sinlat = sin($lat);
    return "(IFNULL(ACOS($coslat*COS(RADIANS({$tbl_alias}field_geo_lat))*($coslong*COS(RADIANS({$tbl_alias}field_geo_lon)) + $sinlong*SIN(RADIANS({$tbl_alias}field_geo_lon))) + $sinlat*SIN(RADIANS({$tbl_alias}field_geo_lat))), 0.00000)*$radius)";
  }

  static function geocodeZip($zip) {
    module_load_include("module", "gmap", "gmap");
    module_load_include("inc", "location", "geocoding/google");
    return (array) google_geocode_location(["postal_code" => $zip]);
  }

  function validateOrigin($origin) {

    if (( array_key_exists("lon", $origin) === FALSE || array_key_exists("lat", $origin) === FALSE ) )
    {
      if (array_key_exists("zip", $origin)) {
        throw new RestfulNotFoundException("Postal code was given but unable to resolve its geolocation.");
        return false;
      }
      else {
        throw new RestfulNotFoundException("Unable to locate. lat/lon params are required.");
        return false;
      }
    }

    if (((float)$origin['lat'] == 0 || (float)$origin['lon'] == 0))
    {
      throw new RestfulNotFoundException("Longitude/Latitude or Zip/Postal code needs to be a non-zero number (float)");
      return false;
    }

    if ((int)$zip == 0) {
      if (((float)$origin['lat'] == 0 && (float)$origin['lon'] == 0))
      {
        throw new RestfulNotFoundException("Longitude/Latitude or Zip/Postal code needs to be a non-zero number (float)");
        return false;
      }
    }

    return true;
  }

}