Bug Solution

Include comments in search in The Bug Genie

4th April, 2013 - Posted by david

Back in July, I was given the task of finding a decent, preferably PHP-based, bug-tracking system and opted for The Bug Genie, as it had pretty much everything we wanted: email notifications, tickets grouped by project, various levels of user access and more. One thing we noticed however, was that when you searched all issues for a particular string, The Bug Genie only searched the main issue text and omitted any text in the comments. This wasn’t ideal, so I went through the source code, found where the searches were being performed and wrote a short hack to get what I wanted. I must stress, this is a hack and I’m sure could be done alot more elegantly! I just didn’t have the time to tidy it up.

What the code does is perform a simple LIKE on the comments table for your search term, gets each comment’s parent’s issue ID and includes any issues with the set of matched IDs in the main search results. The code snippet below is to go in function findIssues in core/classes/B2DB/TBGIssuesTable.class.php. I’ve included the few lines of code directly above where my code needs to be inserted:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

// ... rest of function findIssues above
if ($filter_info['operator'] == '=')
{
$ctn = $crit->returnCriterion(self::TITLE, $searchterm, Criteria::DB_LIKE);
$ctn->addOr(self::DESCRIPTION, $searchterm, Criteria::DB_LIKE);
$ctn->addOr(self::REPRODUCTION_STEPS, $searchterm, Criteria::DB_LIKE);
$ctn->addOr(TBGIssueCustomFieldsTable::OPTION_VALUE, $searchterm, Criteria::DB_LIKE);
//****** my code begins here
// manually connect to DB
$c = mysqli_connect(Core::getHost(), Core::getUname(), Core::getPasswd(), Core::getDBname());
// search comments table for the text you're looking for
$query = mysqli_query($c, 'SELECT target_id FROM tbg3_comments WHERE content LIKE \''.mysqli_real_escape_string($c, $searchterm).'\'');
// if we've matches, build up an array
$ids = array();
while ($row = mysqli_fetch_row($query))
{
$ids[] = $row[0];
}
if (count($ids))
{
// add clause to map any found target_ids on the comments table to actual id's on the issues table
$ctn->addOr(self::ID, $ids, Criteria::DB_IN);
}
//****** rest of function can continue on
}
else {
// etc.

Safari and long lines in Google Maps Javascript API Directions Service

11th October, 2012 - Posted by david

I’ve been working on building the mobile version of CarsIreland.ie lately and one of the features we want is to offer directions from a user’s current location to any of our car dealerships, assuming we have their latitude and longitude co-ordinates. Getting the directions and displaying them on a map are ridiculously simple, thanks to Google’s Direction Service API. Going into the detail of getting this working is beyond the scope of this post, but what I hope to show here is how to resolve an issue in mobile (and possibly desktop) Safari, and maybe even some other browsers.

When you have instantiated your new google.maps.DirectionsService() and called it’s route function, passing your source and destination (among other things) as parameters, you’re supposed to give a callback function with response and status parameters. Assuming the call to route was successful, the response parameter should be a big JSON block of at least one route and step by step guides detailing how to get from source to destination. Conveniently, Google also provide a very handy object called a DirectionsRenderer, which has a function called setDirections, which can generate a nicely formatted HTML table of directions. See https://google-developers.appspot.com/maps/documentation/javascript/examples/directions-panel for an example.

The problem I experienced and am aiming to solve here is that some of the directions (in Ireland at least) involve very long motorway/freeway names, where each junction is separated by a ‘/’, but with no spaces. This can lead to very long strings of the format ‘Long/And/Winding/Junction/Name’. Add in the fact that they also include the Irish/Gaelic translation for some words and it gets even longer! When viewing this steps on Android’s Dolphin or even a desktop Firefox, the browser recognizes that it can split the line at the ‘/’ and thus doesn’t widen the page. Safari unfortunately doesn’t do this and forces very wide page widths, which makes the page look awful. So, today I figured a way to resolve this.

What I did was traverse the response object you get back from Google, looking at each instruction step, trying to find one of these long strings, and replacing the ‘/’ with ‘ / ‘, i.e. a space character on either side, so Safari will then break the lines and not force a wide page. Doing a simple string replace wasn’t sufficient, as some of the instructions contain HTML tags, which can have ‘/’ in them that we ideally want to keep.

So first up is the Javascript regular expression to find matching instances of “long/string”. In simple terms, it’s any alpha-numeric character, followed by a ‘/’, followed by another alpha-numeric character. In Javascript I came up with:

1	var patt = /([a-z0-9]{1})\/([a-z0-9]{1})/gi;

The gi at the end of the pattern means global search (i.e. don’t stop at the first match), case insensitive (hence no ‘A-Z’ is required).

Now, all we have to do is cycle through the response, looking for routes, legs, steps and instructions and replacing as necessary, via a string’s replace function:

1
2
3

// e.g.
var repl = "$1 / $2";
instruction = instruction.replace(patt, repl);

So, to loop through the response and do the replacing, we need a few nested for loops, our pattern and our replace sequence, as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

if (typeof response['routes'] != 'undefined') {
var patt = /([a-z0-9]{1})\/([a-z0-9]{1})/gi;
var repl = "$1 / $2";
// cycle through each route
for (var i=0; i<response['routes'].length; i++) {
var route = response['routes'][i];
if (typeof route['legs'] != 'undefined') {
// cycle through each leg in that route
for (var j=0; j<route['legs'].length; j++) {
var leg = route['legs'][j];
if (typeof leg['steps'] != 'undefined') {
// cycle through each step in that leg
for (var k=0; k<leg['steps'].length; k++) {
var instructions = leg['steps'][k]['instructions'];
// if we've found an instruction with a matching pattern
if (instructions.match(patt)) {
// do the replace
response['routes'][i]['legs'][j]['steps'][k]['instructions'] = instructions.replace(patt, repl);
}
}
}
}
}
}
}

So, hopefully this well help people experiencing the same problem I was having with the long strings caused by a lack of spaces between ‘/’ characters! As an alternative, one may just wish to only have a space after the ‘/’, in which case, the replace pattern becomes "$1/ $2".

window.onload firing too early in IE

12th May, 2012 - Posted by david

The Problem

Where I work, our tool supports 6 different languages, where translations are contained in various sets of language files. One of these sets consists of the translations stored in Javascript files as JSON blocks

php, error_log and newlines: a solution

15th March, 2012 - Posted by david

When it comes to coding and debugging, I generally keep things simple, opting for a basic text editor (Sublime Text 2 with Vim key-bindings being my current choice) and simple debug output statements when trying to track down a bug. With my current job, I deal with an API over AJAX so it’s not easy to send debug statements to the browser, hence had been using a combination of PHP’s error_log and print_r($var, 1) to get the value of a variable on the server. When debugging this way, I’d usually be doing a tail -f on my error log file, monitoring changes to that as I go.

This was all fine, but got very frustrating when dealing with arrays, objects and even strings with newline characters. The reason for this frustration was that newline characters were being displayed in the error log as \n and no new line, so something like

1 2	$var = array('zero' => 0, 'one' => 1); error_log(print_r($var, 1));

would produce:

1	Array\n(\n [zero] => 0\n [one] => 1\n)

instead of the nicer:

1
2
3
4
5

Array
(
[zero] => 0
[one] => 1
)

Not too bad for small arrays but for large objects it’s a nightmare! Googling around didn’t have an easy answer. I’m sure it’s some setting deep in php.ini or my Apache config, but I managed to come up with a pretty neat solution, which I’m going to document here.

I started by creating my own error log in /tmp/derror (David’s Error = derror!) and letting any one write to it (I realise this could be slightly cleaner/more secure):

1 2	touch /tmp/derror chmod 766 /tmp/derror

Next I needed a nice debug function to write to this file. I wrote one that’ll take any number of parameters and for each one, output the current time along side a dump of their value. If it’s an object or class, use print_r, otherwise just display the value:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

So, that function works fine but after a while I realised by doing my tail -f on my new error file, I was missing out on the real server errors. So, I had 2 different terminal tabs open, flipping between them both, which got annoying after about 2 errors! Luckily, I quickly thought of a way around this: route PHP’s error logging to my new file. The way Apache and virtual hosts are set-up this was quite easy and you can even log to 2 different files, thus preserving your ‘normal’ error file too. so, in your /etc/apache/sites-available/default or whatever, in the <Virtualhost:*.80> block for your site, copy the line for CustomLog and point that to your error file, i.e.

1	CustomLog /tmp/derror combined

This should now route all Apache errors to your new file, so you have the best of both worlds!

To prevent the file getting to big over time, I set-up a cron job to simply delete the file and create a new one every Monday morning.

MySQL reached max integer limit

23rd August, 2011 - Posted by david

Whenever anything generates a MySQL error at work, the whole technical team gets an email about it, with full debug info. We recently got one for an area that I look after, so it was up to me to investigate. The title of the error was a slightly cryptic

Error: Duplicate entry ‘0’ for key 1

Clearly this had something to do with the primary key, which was a simple unique integer ID. As this is a pretty large table, I had a feeling that we had reached the upper limit of what could be stored for the field type (MySQL’s MEDIUMINT, signed). Looking at the maximum value for the ID, I saw it was 8388607; according to this table on the MySQL website, this value is the maximum that can be stored in that type of field, so this was clearly the problem. (N.B. an auto-incrementing field shouldn’t ever be defined as signed, as you’ll never go into the negative indices, but that was done before I came along!)

The solution? Surely a simple readjusting of the key to be unsigned, or to further future proofing, changing to the larger INT… That’s what I thought and quickly (in test first, of course!) did a

1	ALTER TABLE table_name CHANGE COLUMN id id MEDIUMINT UNSIGNED NOT NULL AUTO_INCREMENT

This should then change the maximum value allowed in the ID field to 16777215. So, to test this, I went to insert a row, specifying NULL for the primary key value. However, I still got the same error. Doing a DESC on the table told me that the ID field had been changed to MEDIUMINT UNSIGNED correctly, so that wasn’t the issue. After further research I determined that what happend was that MySQL’s internal counter for that auto increment field was still set to 0, due to the rollover caused after reaching the maximum integer value. To overcome this, you need to point the internal counter back to where it should be, i.e. the current maximum value of your ID field, as follows:

1	ALTER TABLE table_name AUTO_INCREMENT = 8388608

So, you set it to a value that’s one greater than the current maximum ID.

Google Maps Zoom Control missing in Internet Explorer (IE): a solution

23rd February, 2011 - Posted by david

When I was working on rebuilding daft.ie’s mapping system for the modern web, it involved writing alot of Javascript (JS) code and loading this via a couple of JS source files, one external one from Google (i.e. the core Google Maps JS) and another from our own web-servers.

It all worked fine in most browsers, except of course Internet Explorer (IE). To be fair, IE8 was fine; however, seemingly at random, the zoom control would sometimes not appear on the map for IE7 and IE6 and you’d get one of IE’s very unhelpful error messages. At Daft we like to ensure that the site is available and consistent on all browsers, so it was imperative to get this issue sorted.

Because of the aforementioned unhelpful error messages, it proved to be a very difficult bug to nail down. I began to suspect that it was due to the code from Google loading after the map is set-up in the daft.ie hosted file, i.e. IE was calling Google-specific functions before it had the code. As it was all wrapped in a jQuery $(document).ready I didn’t think this could happen, but turns out that in the older versions of IE it could! I tried moving the JS around, loading the Daft-specific JS as late as possible, to try and give the Google JS the maximum amount of time to load, but that was to no avail.

So, I needed to find a way to load the Daft JS when I was 100% certain the Google JS had arrived. After various futile attempts, the solution ended up being pretty simple – change my

1	$(document).ready(function() {

1	$(window).bind("load", function() {

That way, the bulk of my JS is only executed once everything else has loaded and sure enough, the issue disappeared!

HTML Div’s layered onto Google Streetview: Chrome issue

16th December, 2010 - Posted by david

While working on Daft‘s Map Search facility, we naturally decided to integrate Google’s Street View (SV) as a feature, especially since we were one of the launch partners for the product in Ireland. We decided to do this as follows:

For each property icon on the map, when you click on it, bring up a small info window with a picture, minimal details, a link to the property’s main page and, where applicable a link to the SV image for that property
When clicking on the SV link, the entire map is replaced with the SV window
A button would then exist to close the SV window and revert back to the map

Inconsistent JSON arrays using PHP’s json_encode

14th September, 2010 - Posted by david

I’d be amazed if anyone ever has come across this problem before, but it’s one that stumped me recently and I feel would make a good first ‘real’ post for this new blog.

Firstly, a bit of background: in work, we have an API that uses the XML-based SOAP protocol over HTTP. You can request data either using a PHP SOAP plug-in, or by passing your parameters as encoded JSON over a HTTPS connection. The data you get back is usually an array of properties, with various fields set. When data is requested in JSON, it’s returned in JSON. On the main server, we use memcached, which gets checked for a given query before we hit the main database. Whenever a query is made to the API, if there’s nothing in the cache, we hit the database and then cache the result for 5 minutes, to reduce the impact on our primary database; pretty standard stuff for a big website.