Code

Regular expression (regex) to remove double encoding of html entities

30th March, 2011 - Posted by david

When you have users copying and pasting in data to forms on your website, which then gets stored in your database, you invariably end up with all sorts of ways of encoding and storing special characters. Ideally, these will end up in your database as the correct characters (such as € for the euro symbol), which will then get encoded as HTML entities when you display this data on your website (so, € becomes € in the HTML).

However, with older systems, especially those built in-house, you end up with the HTML entity version of certain characters in your database. It’s pretty much a fact of web development. Let’s use the example of a string that says “Price: €100” but gets stored in the database as “Price: €100”. When you go to display this text on your encoded web-page, you end up seeing things such as “Price: &euro;100” in your browser. This is a result of double encoding, as the & in € is first getting encoded as &.

In order to remove these, I came up with the following function, that uses a simple regular expression to tidy such instances up.

1
2
3
4

function remove_double_encoding($in)
{
return preg_replace('/&([a-zA-Z0-9]{2,7});/', '&$1;', $in);
}

What this does is looks for any 2 to 7 letter strings with & immediately before them and ; immediately after. When it finds a match, it simply replaces the & with &. It does this for all instances in your input string.

Update: Forgot that you can also have purely numeric codes here, so added ‘0-9’ to the regex.

Planned site downtime: how to handle it with PHP

25th January, 2011 - Posted by david

Today I read an interesting article on Google’s Webmaster Central blog about planned down time, for maintenance or whatever. Basically, you need to communicate to users and potential site-crawlers/indexers that the site is down, but it’s ok, it’ll be back up soon!

My suggestion would be to do a mod-rewrite (if using Apache, but other web servers have equivalents) on all incoming files to your 503 file, something like

RewriteRule ^(.*)$ 503.php [L]

If you put this rule first, you can leave the rest of your rules where they are as the “L” at the end of the line tells Apache that this is the last rule it needs to process, i.e. it can ignore the others.

Then, in your 503.php you need to tell any web-crawlers/site-indexers that the service is down but will be back at a certain time using PHP’s header function, as well as telling your users the same using a plain simple descriptive paragraph:

<?php

$back = 'Tue, 1 Feb 2011 12:00:00 GMT';

// tell any web-crawlers
header('HTTP/1.1 503 Service Temporarily Unavailable');
header('Retry-After: '.$back);

// calculate time to wait for users (I assume here it'll only be a matter of hours)
$vals = array('hour' =--> 3600, 'minute' => 60, 'second' => 1);
$back_in_seconds = strtotime($back) - time();
$back_text = '';
foreach ($vals as $key => $time) {
$cur_time = floor($back_in_seconds / $time);
if ($cur_time) {
$back_text .= $cur_time.' '.$key;
if ($cur_time != 1) $back_text .= 's';
$back_text .= ', ';
$back_in_seconds %= $time;
}
}
$back_text = rtrim($back_text, ', ');
?>


We are currently undergoing scheduled downtime to upgrade the site.
We'll be back in . Thanks for your patience.

Should all be pretty straight-forward, but it’s good to let people know that you know you’re down and when you expect to be back.

Web programming with Python and web.py: my experience

2nd November, 2010 - Posted by david

Quick Introduction

I’ve written the front page and an ajax handler for a web application using Python and web.py. Here I give my thoughts on using a niche product (web.py) and some of the coding issues I had to figure out on my own, in case some one has similar problems.

Google Maps API V3 Overview Control

9th October, 2010 - Posted by david

In mid-2009, Google launched version 3 of their fantastic Google Maps API. This new version was a complete overhaul of the existing system, moving to an MVC-based architecture, with the primary focus on speeding up the code so it would work better on mobile devices. It has now taken over from Maps API v2 as their primary Maps AP v2 now deprecated.

One of the things missing from v3 that was in v2 was v2’s GOverviewControl, which was the mini-map you’d see on the bottom right of the main map, which would be zoomed out further than the main map. When I started work on overhauling Daft.ie’s mapping system, we originally used Maps v2 and had such a control in place. We decided to use Maps v3 for the new system and it was then I noticed that the Overview Control was missing. At the time I was still learning how to use the API and have since become reasonably proficient in it, enough so to complete our maps overhaul! Taking these skills, I’ve now written a jQuery-based JavaScript extension that enables users to add the Overview Control to their map, along with a number of customisation options.