I’ve recently started building a new web app and have decided to build it in Symfony (and hopefully ultimately Vue for the front-end), as a way of learning what looks like a really cool framework. I’d had an introduction to Symfony through work, then did some online courses, including a load of the excellent SymfonyCasts, so have a reasonable amount of knowledge now on the subject.

One cool feature of Symfony is its form handling abilities. Using simple annotations and generated code, you can get alot of functionality around submitting, validating and saving your form data for very little effort. When rendering a form to the user, Symfony’s template handler, Twig, has a load of built in functions to render each field, such as form_widget to display the field’s HTML tag, form_label to display a field’s label, form_row to combine the above 2 and 2 more, etc. These can be overwritten, so you can style/theme your form using your own CSS, while keeping Symfony and Twig’s powerful form functionality. More details can be found on Symfony’s form customization page.

One thing I found was that sometimes you don’t really need to modify the original form_widget, you just need to say wrap its contents in a div with a specific class. So, ideally there’d be a way to overwrite form_widget but still call the parent/original one. Luckily there is, but it was a little tricky to figure out.

Let’s say I need to add a <div class="select"> around a particular field. For this, we can overwrite the choice_widget function, which generates <select> tags. In your form theme you can do something like:

And you’d think that would work no issues! However, I kept getting errors along the lines of “Can’t call parent function” or “parent function does not exist”, or something like that. I found it surprising, because you can simply call choice_widget before you add your code to overwrite it, so the function must be in the global scope, right? Well, somehow it isn’t! Luckily the fix is simple: import the base Twig theme at the top of your own theme:

And voilá! You should now have your select wrapped in its stlystyled div.

Read more...

Facebook have long had the ability for developers to write custom apps and embed them as tabs in people’s or company’s Facebook profile pages. What I’m talking about here is when you write your own HTML app and host it at e.g. https://fb.mysite.com, which is then embedded into profile pages via an iframe. These apps then have URLs like https://www.facebook.com/profile.site/app_123456789, where 123456789 is your app ID within Facebook.

I’ve written one such app, in PHP, which has sub-pages, so while the root is https://fb.mysite.com, using the app will call pages such as https://fb.mysite.com/product/1. Seeing as this is within an iframe, the URL within the browser remains at https://www.facebook.com/profile.site/app_123456789 while you browse around the app. I recently had a request from a client on how they could link to a sub-page from within a post on their profile page, so they wanted to post something like ‘Check out my product at <link>’, where clicking on the link will load up the iframe app and bring them to the specific product. This is achievable, but it’s not exactly straight-forward and requires some work on the developers behalf. In an ideal world the link would simply be https://www.facebook.com/profile.site/app_123456789/product/1!

The way I managed to achieve this was using Facebook’s app_data parameter. Here, you can pass any data you want to the variable and it’ll end up as part of the signed_request variable in $_REQUEST at https://fb.mysite.com/index.php . The way we’re going to structure these deeplinks is to pass a JSON object to app_data containing a page key, with the sub-page we want, in this instance products/1, so our deeplink is going to look like https://www.facebook.com/profile.site/app_123456789?app_data={page:products/1} . Not exactly elegant but it’ll have to do! You could simply set app_data to products/1, but there may come a time when you want to pass other data in addition to the page, so I opted to go down the JSON route.

Now that we know what to expect, we need to decode $_REQUEST['signed_request'] (which should be available to your https://fb.mysite.com/index.php), json_decode app_data from the result, validate page, then redirect the browser accordingly.

To decode $_REQUEST['signed_request'] I used Facebook’s PHP SDK. Once we have the signed request as an array, we decode the JSON from app_data. Then, we check for the presence of page, validate it (I’ll leave the validation code up to yourself!) and send them on their way. This is pretty straight-forward, so is probably best illustrated with some code:

Pity it’s not more straight-forward, but this is the way I got it to work. Hope this helps!

Read more...

I often see different ways of storing and displaying opening hours for various businesses when browsing the web. Some seem to simply store them as an unstructured blob of plaintext and spit that back to the user. Others will store the exact times for each day and display all 7 days on 7 separate rows.

I think neither of these options are great and I’ve come up with what I think is the ideal solution. Basically, we want to store the exact time for each day, but group similar days together. So, you could have lines like ‘Mon – Thurs: 9 – 5’, but if it’s say Tuesday at 10am, you’d also know that the business is currently open. I also think you could have alternative text to display when the business is closed (indicated by open and closed being NULL), so you could have something like ‘Sat – Sun: Open by appointment’.

So, let’s start with our table layout, see below. The primary key is id, which is just your standard auto_increment value. business_id is if you have multiple businesses, each with their own opening hours, as was the case for me. You might want to build an index on this field too. If you’re just storing your own opening hours, dow could be the primary key and you could drop those last 2 fields. open and closed are just simple strings, to store the time in 24-hour ‘HH:MM’ format. Doing it this way, you can still to comparisons like WHERE open > '09:00' and get the result you were expecting.

Next up, I want to show you a quick function to format the time. Being a programmer, the time ’13:30′ is easily translated to ‘1.30pm’, but for the general public it might not be so simple, so this function will display your time in a more human readable format, such as the example given above. Basically, we want to drop any leading 0s, any 0-value minutes, convert the time to a 12-hour version with ‘am’ and ‘pm’ and change the ‘:’ to a ‘.’ (this last bit is probably a bit region-specific). For midnight, we’ll store that as ’24:00′ (which I know is technically the start of the next day!) and display that to the user as ‘midnight’, instead of the slightly confusing ‘0am’.

OK, so displaying the data in a nice table, with similar days grouped together, is the next bit. We’ll have 2 columns, one for the day/days, the other for the time. If a day has a value for optional_text, then that value will be displayed and the times are ignored. I’m also going to add another block of optional text ($extra_text below) that will be displayed at the end of the table and is applied for all days, to be used for something like ‘phone anytime’. Finally, there’s a $short_day_names option, so you can choose between say ‘Mon’ and ‘Monday’.

I should also mention at this point: I’m returning a block of HTML here from a function, as well as mixing business logic with display logic; I realise this is generally a bad idea and some of this could be split into a function and a template, but seeing as it’s a simple 2-column table, I just kept it all together.

So, with the following data…

you should end up with the following table:

Read more...

At work we use memcache as our local variable cache and the excellent memcache.php from Harun Yayli to give us a simple way of viewing what’s in the cache.

One use case we came up with that was missing from the original memcached.php script was a way to group similar variables and see how much of the cache they’re taking up. For example, for searches done on the site, we generate a key by concatenating search- to an md5 of the SQL, then store the result of that query in the cache with that key. Another example might be to cache an ad, so the key could be ad-1234, for the ad with ID 1234. So, the following code changes are going to enable us to see how much space all the ‘search’ data, ‘ad’ data etc. takes up in comparison to each other.

It works by starting off with a list of known key prefixes (i.e. search- and ad- in the examples above), then uses existing memcache commands to get a list of slabs, then queries each slab for each item it contains. From this list of items, it looks for our known keys, calculates the size of the item and adds it to a running total. Once it has all the totals, it generates a nice pie chart with a legend, using Google’s Chart API.

So, first up we need to add a new menu entry to our menu, to link to our breakdown. This is simply done by editing the getMenu function in src/display.functions.php and adding a new menu entry to it, as follows:

Next up, we need to add the big block of code that’s going to generate our pie chart. You’ll see in memcache.php a switch block around $_GET['op']. This is where we want to add our block for our new operation 16, as follows:

So, now you should see a new menu option and clicking on it, should hopefully bring up a nice pie chart, something like the screenshot below (I’ve had to blur out our cache variable names).

Memcache Breakdown

Read more...

Back in July, I was given the task of finding a decent, preferably PHP-based, bug-tracking system and opted for The Bug Genie, as it had pretty much everything we wanted: email notifications, tickets grouped by project, various levels of user access and more. One thing we noticed however, was that when you searched all issues for a particular string, The Bug Genie only searched the main issue text and omitted any text in the comments. This wasn’t ideal, so I went through the source code, found where the searches were being performed and wrote a short hack to get what I wanted. I must stress, this is a hack and I’m sure could be done alot more elegantly! I just didn’t have the time to tidy it up.

What the code does is perform a simple LIKE on the comments table for your search term, gets each comment’s parent’s issue ID and includes any issues with the set of matched IDs in the main search results. The code snippet below is to go in function findIssues in core/classes/B2DB/TBGIssuesTable.class.php. I’ve included the few lines of code directly above where my code needs to be inserted:

Read more...

On the website I work for, when a user uploads an image for an ad, we generally keep 3 versions of that image, each a different size, simply referred to as ‘small’, ‘main’ or ‘large’. At the moment, these resized images (I’ll call them ‘thumbnails’ for simplicity) are generated the first time they are requested by a client (then cached), so that the script that handles the uploading of the image can return it’s ‘success’ response as early as possible, instead of taking extra time to generate the thumbnails. What Beanstalkd allows us to do is put a job on a queue (in our instance a ‘generate thumbnails’ job), where it’ll be picked up at some point in the future by another script that polls the queue and executes in it’s own separate process. So, my uploading script is only delayed by say the 0.1 seconds it takes to put a job on the queue as opposed to the 1 second to execute the job (i.e. generate the thumbnails). This blog post is how I got the whole thing to work on a Ubuntu 12.04 server, using PHP.

This post was largely inspired by an article on the blog Context With Style, which was written for a Mac. I’m also going to use their example of a queue filler script to populate the queue and a worker script, to pull jobs from the queue and process them. I recommend you read that post for a better idea.

One other thing, most of these UNIX commands need to be run as root, so I’ll assume you’re in super-user mode.

Beanstalkd

Installing Beanstalkd is pretty straightforward:

We don’t need to start it just yet, but for reference, to run it you can do

Pheanstalk

Pheanstalk is a PHP package to interface with a Beanstalk daemon. I simply downloaded the zip from github, extracted it to a ‘pheanstalk’ folder in my main include folder, then to use it, I simply do

Going by the example on the Context With Style article, for the script under the section “Pushing things into the queue”, we’ll call that script fill_queue.php. We’ll call the script in “Picking up things from the queue” worker.php. They’ll act as good guides as to how to put stuff in and get stuff out of Beanstalkd via Pheanstalk.

So, the idea is we’ll have our worker.php running non-stop (via daemontools, see next section), polling the queue for new jobs. Once we know our worker.php is ready, we can manually run fill_queue.php from the command line to populate the queue. The worker should then go through the queue, writing the data it reads to a log file in ./log/worker.txt. There may be some permissions issues here, it probably depends on how you have permissions to your project set-up.

Daemontools

First up we need to install daemontools, which is

You don’t actually interact with a daemontools process, you use things that begin with ‘sv’, such as svscan or svbootscan. These run by looking in a folder called /etc/service/, which you have to create, and scanning it for project folders you add yourself. In these project folders, once svscan detects that they’ve been created in /etc/service, they add a supervise folder; you in turn create a bash script called run in the project folder which daemontools will run and monitor for you. Don’t worry, all these steps are outlined below!

Anyways, now that we’ve installed daemontools, we need to create a run script for it and then run it, as well as create our /etc/service directory. Some of these tips are thanks to this post.

Hopefully, now if you do a ps aux | grep sv, you’ll see at least svscan running.

Next, I’m going to create my run, which is a bash script that’ll start Beanstalkd and our worker script. I’ll place this in my example /var/www/my-project folder, along with my worker.php, fill_queue.php and log/worker.txt files. I’ll then create a my-project service folder and symlink my run file into there.

Hopefully when you do the tail, you’ll see data that corresponds with the output from fill_queue.php. This will indicate that your worker is running, polling the queue for new jobs. If you re-run fill_queue.php, your log file should expand accordingly.

Read more...

When it comes to coding and debugging, I generally keep things simple, opting for a basic text editor (Sublime Text 2 with Vim key-bindings being my current choice) and simple debug output statements when trying to track down a bug. With my current job, I deal with an API over AJAX so it’s not easy to send debug statements to the browser, hence had been using a combination of PHP’s error_log and print_r($var, 1) to get the value of a variable on the server. When debugging this way, I’d usually be doing a tail -f on my error log file, monitoring changes to that as I go.

This was all fine, but got very frustrating when dealing with arrays, objects and even strings with newline characters. The reason for this frustration was that newline characters were being displayed in the error log as \n and no new line, so something like

would produce:

instead of the nicer:

Not too bad for small arrays but for large objects it’s a nightmare! Googling around didn’t have an easy answer. I’m sure it’s some setting deep in php.ini or my Apache config, but I managed to come up with a pretty neat solution, which I’m going to document here.

I started by creating my own error log in /tmp/derror (David’s Error = derror!) and letting any one write to it (I realise this could be slightly cleaner/more secure):

Next I needed a nice debug function to write to this file. I wrote one that’ll take any number of parameters and for each one, output the current time along side a dump of their value. If it’s an object or class, use print_r, otherwise just display the value:

So, that function works fine but after a while I realised by doing my tail -f on my new error file, I was missing out on the real server errors. So, I had 2 different terminal tabs open, flipping between them both, which got annoying after about 2 errors! Luckily, I quickly thought of a way around this: route PHP’s error logging to my new file. The way Apache and virtual hosts are set-up this was quite easy and you can even log to 2 different files, thus preserving your ‘normal’ error file too. so, in your /etc/apache/sites-available/default or whatever, in the <Virtualhost:*.80> block for your site, copy the line for CustomLog and point that to your error file, i.e.

This should now route all Apache errors to your new file, so you have the best of both worlds!

To prevent the file getting to big over time, I set-up a cron job to simply delete the file and create a new one every Monday morning.

Read more...

When you have users copying and pasting in data to forms on your website, which then gets stored in your database, you invariably end up with all sorts of ways of encoding and storing special characters. Ideally, these will end up in your database as the correct characters (such as € for the euro symbol), which will then get encoded as HTML entities when you display this data on your website (so, € becomes € in the HTML).

However, with older systems, especially those built in-house, you end up with the HTML entity version of certain characters in your database. It’s pretty much a fact of web development. Let’s use the example of a string that says “Price: €100” but gets stored in the database as “Price: &euro;100”. When you go to display this text on your encoded web-page, you end up seeing things such as “Price: &amp;euro;100” in your browser. This is a result of double encoding, as the & in € is first getting encoded as &.

In order to remove these, I came up with the following function, that uses a simple regular expression to tidy such instances up.

What this does is looks for any 2 to 7 letter strings with & immediately before them and ; immediately after. When it finds a match, it simply replaces the & with &. It does this for all instances in your input string.

Update: Forgot that you can also have purely numeric codes here, so added ‘0-9’ to the regex.

Read more...

Today I read an interesting article on Google’s Webmaster Central blog about planned down time, for maintenance or whatever. Basically, you need to communicate to users and potential site-crawlers/indexers that the site is down, but it’s ok, it’ll be back up soon!

My suggestion would be to do a mod-rewrite (if using Apache, but other web servers have equivalents) on all incoming files to your 503 file, something like

If you put this rule first, you can leave the rest of your rules where they are as the “L” at the end of the line tells Apache that this is the last rule it needs to process, i.e. it can ignore the others.

Then, in your 503.php you need to tell any web-crawlers/site-indexers that the service is down but will be back at a certain time using PHP’s header function, as well as telling your users the same using a plain simple descriptive paragraph:

Should all be pretty straight-forward, but it’s good to let people know that you know you’re down and when you expect to be back.

Read more...

I’d be amazed if anyone ever has come across this problem before, but it’s one that stumped me recently and I feel would make a good first ‘real’ post for this new blog.

Firstly, a bit of background: in work, we have an API that uses the XML-based SOAP protocol over HTTP. You can request data either using a PHP SOAP plug-in, or by passing your parameters as encoded JSON over a HTTPS connection. The data you get back is usually an array of properties, with various fields set. When data is requested in JSON, it’s returned in JSON. On the main server, we use memcached, which gets checked for a given query before we hit the main database. Whenever a query is made to the API, if there’s nothing in the cache, we hit the database and then cache the result for 5 minutes, to reduce the impact on our primary database; pretty standard stuff for a big website.

Read more...