Grunt & WordPress development IV: Another task for internationalisation

grunt

This is part 4 in a series looking at using Grunt in WordPress plug-in/theme development.

  1. Grunt & WordPress development
  2. Grunt & WordPress development II: Installing Grunt
  3. Grunt & WordPress development III: Tasks for internationalisation
  4. Grunt & WordPress development IV: Another task for internationalisation

WordPress has recently (since 3.5) seen a shift towards a more JavaScript codebase. This shift is still minor (it currently accounts for less than 15%, according to its GitHub repository, in 3.9). But the introduction of Backbone.js and the re-factoring of particular the editor in the WordPress admin (media manager, shortcodes “objects” etc.) are testament to Matt Mullenweg’s comment:

I forgot to mention our biggest architectural change, which is already ongoing: an ever-increasing percentage of our codebase is shifting to Javascript instead of PHP. At some point I expect it to be the vast majority.

Matt Mullenweg, https://news.ycombinator.com/item?id=4744542

However, a JavaScript codebase presents difficulties for WordPress in terms of internationalisation. I ran into this while working on Event Organiser Pro’s 1.7 release. This new release saw the booking form customiser rewritten to use Backbone. With so much JavaScript replacing PHP, there became a massive need to allow strings to be translatable (in a sane way).

Received wisdom is that you should use wp_localize_script() to allow strings in javascript files to be translated. Pippin covers the method excellently here but essentially when you enqueue your script you use wp_localize_script() to make a variable available which contains all you translated strings.

function my_load_scripts() {
    wp_enqueue_script( 'my-script', plugin_dir_url( __FILE__ ) . 'my-script.js');
    wp_localize_script('my-script', 'mynamespace', array(
         'helloworld' => __('Hello World', 'mydomain' )
    ));
}
add_action('wp_enqueue_scripts', 'my_load_scripts');

Then in your JavaScript file, instead of the string “Hello World” you would use mynamespace.helloworld. Although this method is common, there are a couple of things wrong with it when you pit it against some sort of gettext function:

It makes code harder to read. For me when reading code it’s much easier to see the actual text rather than a variable. (More so when reading other people’s code as this helps you link the sourcecode with what you actually see)

<script>
  alert( mynamespace.welcome_msg ); //ok
  alert( mynamespace.gettext( "Welcome to..." ) ); //better
</script>

At the very best its slightly more cryptic and uglier.

It makes it harder to maintain code. – When editing a JavaScript file, a gettext function allows me to edit the string there and then. If I use wp_localize_script(), I need to track down the .php file responsible for that and change it there – and then not forget that that string might have been used elsewhere.

It makes the translator’s job harder. – Hands up anyone who is very poor at providing translators with comments or providing a context when appropriate. Me at least. Regardless, .po files provide a line number so that, if necessary, translators can look up that line to get some sort of context for the string they are translating. It’s not very helpful when that line number points them to a large array in some obscure php file, rather than where the text is being used. Nor is it immediately obvious which JavaScript file(s) are using the string.

You may think that my reasons here are weak and pinnikity… and you might right… But I prefer using gettext-esque function for translating strings.

The problem(s)… (and how Grunt helps solve them)

There a couple of problems with using trying to use a gettext function in a JavaScript file, but they are all easily solved:

  1. There is no native gettext function
  2. How do you get the translations from .po to your JavaScript file
  3. How to get translatable strings from your JavaScript file to your .pot

There is no native gettext function in JavaScript

A very simple solution is to roll your own. Below are four functions which handle translatable strings, plurals and contexts. They all expect the translated strings to be found in mynamespace.locale.

mynamespace.gettext = function( msgid ){
    if( this.locale[msgid] !== undefined ){
        return this.locale[msgid];
    }
    return msgid;
};

mynamespace.ngettext = function( msgid1, msgid2, n ){
    var key = ( n > 1 ? msgid1 + '_plural' : msgid1 );
    if( this.locale[key] !== undefined ){
        return this.locale[key];
    }
    return ( n > 1 ? msgid2 : msgid1 );
};

mynamespace.pgettext = function( ctxt, msgid ){
    if( this.locale[msgid+'_'+ctxt] !== undefined ){
        return this.locale[msgid+'_'+ctxt];
    }
    return msgid;
};

mynamespace.npgettext = function( ctxt, msgid1, msgid2, n ){
    var key = ( n > 1 ? msgid1 + '_' + ctxt + '_plural' : + '_' + ctxt + '_' + msgid1 );
    if( this.locale[key] !== undefined ){
        return this.locale[key];
    }
    return ( n > 1 ? msgid2 : msgid1 );
};

You may have noticed that only one plural form is supported (so a string is either plural or singular), but some languages use more (and some less). There are ways around this, but the limitation is also a result of the Grunt task that’ll we’ll use later. Plural strings and strings with a context expect _plural and _{context} modifiers – I personally think this is less than ideal, but again is forced upon me by my choice of Grunt task. (This is just a start, and I’d like to see these limitations lifted).

Getting translations from .po to your JavaScript file

This is a two-step process:

  1. Use a Grunt task to generate a .json file for each .mo file
  2. Depending on the user’s choice of locale, load that .json file and use wp_localize_script() to make it available in you JavaScipt file.

I went with grunt-i18next-conv to generate the .json files. I found that converting .po to .json included untranslated strings, so I recommend you opt for converting .mo files to .json. (If you need a Grunt task for generating your .mo task, I recommend po2mo task I covered in my last article). It’s this task, by way of the format of the .json file it produces, that imposes some of the limitations already mentioned.

Next you, when enqueuing your JavaScript file, you use wp_localize_script() to ‘attach’ the relevant .json file to it. In the following I expect that the .json files are of the form /languages/mytextdomain-{locale}.json

 $locale = array();
 $file = plugin_dir_path( __FILE__ ) . 'languages/mytextdomain-'.get_locale().'.json';
 if( file_exists( file ) ){
      $locale = json_decode( file_get_contents( $file ), true );    
 }

 wp_localize_script('my-script', 'mynamespace', array(
      'locale' => $locale
 ));

Getting the translatable strings from your JavaScript file to .pot

If you’re using grunt-pot this is easy. Simply include the functions above in the ‘keywords’ option:

  keywords: [ 
        ...
        'namespace.gettext:1',
        'namespace.ngettext:1,2',
        'namespace.pgettext:1c,2',
        'namespace.npgettext:1c,2,3',
        ...
       ]

and ensure the files to search include your JavaScript file.

Limitations

As discussed above there are currently two limitations:

  1. Poor support for plurals other than ‘single form plurals’
  2. Awkward ‘.json’ structure (not a massive issue…)

For the time being, however, and for use in Event Organiser’s booking form customiser, this method was ideal.

Grunt & WordPress development III: Tasks for internationalisation

grunt

This is part 3 in a series looking at using Grunt in WordPress plug-in/theme development.

  1. Grunt & WordPress development
  2. Grunt & WordPress development II: Installing Grunt
  3. Grunt & WordPress development III: Tasks for internationalisation
  4. Grunt & WordPress development IV: Another task for internationalisation

Internationalisation Tasks

One aspect of WordPress plug-in development that involves a lot of mundane work is that of internalisation: ensuring WordPress’ localisation functions are used correctly, generating a .pot file, compiling submitted .po files to .mo files. The latter two you can do with Poedit – but this still involves manually opening the .po/.pot file. These tasks can be completely automated so let’s do that:

po2mo – Compiling to .po files to .mo

The po2mo plug-in automatically compiles given .po files and produces a .mo file of the same name.

To install:

npm install grunt-po2mo --save-dev

<em>Please not an earlier version of this article executed the above as a super user (`sudo npm`). As pointed out by Lacy in the comments, this necessary and can cause permission issues with the npm cache.</em>

The following set up looks in the languages directory for any .po files and compiles them, creating the corresponding .mo in the same directory:

po2mo: {
    files: {
        src: 'languages/*.po',
        expand: true,
    },
},

Finally load the task by adding grunt.loadNpmTasks('grunt-po2mo'); at the bottom of your Gruntfile.js, just after grunt.loadTasks('tasks');. Then whenever you add or change a .po file:

grunt po2mo

(You can see a live example of this task, and the others listed below, here.

pot – Create a .pot template file

For users to be able to translate your plug-in you’ll need to create a .po template file ( a .pot file). The pot plug-in does exactly that.

You just need to provide it with:

  • The files to search in,
  • The keywords to search for (and indicate which arguments are translatable strings, and which are context specifiers)
  • A text domain (used only for naming the the .pot file)
  • The directory where you wish to output the .pot file.

To install:

npm install grunt-pot --save-dev

Then

pot: {
      options:{
          text_domain: 'my-plugin', //Your text domain. Produces my-text-domain.pot
          dest: 'languages/', //directory to place the pot file
          keywords: [ //WordPress localisation functions
            '__:1',
            '_e:1',
            '_x:1,2c',
            'esc_html__:1',
            'esc_html_e:1',
            'esc_html_x:1,2c',
            'esc_attr__:1', 
            'esc_attr_e:1', 
            'esc_attr_x:1,2c', 
            '_ex:1,2c',
            '_n:1,2', 
            '_nx:1,2,4c',
            '_n_noop:1,2',
            '_nx_noop:1,2,3c'
           ],
      },
      files:{
          src:  [ '**/*.php' ], //Parse all php files
          expand: true,
      }
},

Finally load the task by adding grunt.loadNpmTasks('grunt-pot'); to the bottom. Then to generate your .pot file:

grunt pot

checktextdomain – Verify localisation functions have been used correctly

Having generated a .pot file, gathered translations for your plug-in and then compiled them – it would be entirely wasted if you haven’t used the WordPress localisations functions properly. In particular, if you had failed to specify the correct domain, your efforts would have been wasted.

When coding it’s easy to forget to specify a text domain, or to mistype it. Or perhaps you’ve been using a variable for the domain, and now want to switch to a literal string.

The checktextdomain – not only checks if you’ve used the correct textdomain in the localisation function it can also correct it for you.

Simply provide it with:

  • Files to look in,
  • Keywords to look for (important: you must provide a domain argument specifier)
  • A text-domain to check against
  • Whether you want mistakes corrected (it will not add missing domains… yet).

The plug-in will then

  • Warn you if some keywords have been used without a text domain
  • Warn you if some keywords have been used with an incorrect text domain (optionally correct it for you)
  • Warn you if some keywords have been used with a variable text domain (optionally correct it for you)

There are various options for this plug-in to enable you to check (and correct) the things you want to. You can see all the available options for this Grunt plug-in on its Github page.

To install:

npm install grunt-checktextdomain --save-dev

You’ll notice that the keywords option is very similar to grunt-pot. There is an important distinction. For this plug-in to work you must extend the keyword specifier and indicate where the domain should be.

E.g. 2d indicates that the domain should be passed as the second argument of the localisation function

checktextdomain: {
   options:{
      text_domain: 'my-plugin',
      correct_domain: true, //Will correct missing/variable domains
      keywords: [ //WordPress localisation functions
            '__:1,2d',
            '_e:1,2d',
            '_x:1,2c,3d',
            'esc_html__:1,2d',
            'esc_html_e:1,2d',
            'esc_html_x:1,2c,3d',
            'esc_attr__:1,2d', 
            'esc_attr_e:1,2d', 
            'esc_attr_x:1,2c,3d', 
            '_ex:1,2c,3d',
            '_n:1,2,4d', 
            '_nx:1,2,4c,5d',
            '_n_noop:1,2,3d',
            '_nx_noop:1,2,3c,4d'
      ],
   },
   files: {
       src:  [ '**/*.php', ], //All php files
       expand: true,
   },
},

Finally load the task by adding grunt.loadNpmTasks('grunt-checktextdomain'); to the bottom. Then to check your files:

grunt checktextdomain

I’m planning on improving this further to warn you of missing contexts which using functions that expect one.

Final remarks

Remembering to add grunt.loadNpmTasks(...); at the bottom of your Gruntfile.js, just after grunt.loadTasks('tasks'); is easily forgotten. But there’s a way around this which I’ll discuss in my next post.

Just before publishing Brady Vercher announced his Grunt plug-in, which allows you to utilize the internationalisation tools that WordPress uses. There’s a bit more set-up involved, but a notable advantage over grunt-pot is that it recognises theme template headers as translatable.

Grunt & WordPress development

grunt

This is the first post of a five part series looking at WordPress plug-in/theme development with Grunt.

  1. Grunt & WordPress development
  2. Grunt & WordPress development II: Installing Grunt
  3. Grunt & WordPress development III: Tasks for internationalisation
  4. Grunt & WordPress development IV: Another task for internationalisation

Back in August the WordPress core team announced they were going to use Grunt in WordPress’ development. This in my view is a major stride forward for WordPress (more so than the much celebrated ‘features as plug-ins’ – which itself marked an improvement to WordPress’ development cycle).

This series of articles however, will be focussed on using Grunt for WordPress plug-ins and themes (which are fundamentally the same thing), and the tasks I use in development. In this first post I’ll discuss the what and the whys:

What is Grunt?

Grunt is a javascript based task runner from Ben Alman. It performs repetitive task such as compression, unit testing, linting, concatenation, preprocessing etc. Almost any task in the development, building or deployment of your WordPress plug-in which can be automated can be performed by Grunt – freeing you from those tedious, and potentially human-error-prone routines.

(Once Grunt is installed) there are two files which set up Grunt for use in your project:

  • package.json – which details your project (in this case: a WordPress plug-in) and it’s dependencies (in this context, Grunt and any Grunt plug-ins you want to use).
  • Gruntfile.js – listing the tasks you wish to perform and their configuration

Those tasks can be executed simply by running

 grunt [task name] 

in your command line. You’ll probably have multiple tasks that you’d want to run one after than other (e.g. one task to copy/generate files from your development environment to a build directory, and another to upload that directory to an Amazon S3 server). Instead of calling each manually Grunt allows you to create tasks which simply call a collection of other tasks. For example

 grunt test

might be configured to trigger unit-test and linting tasks.

Why Grunt?

The idea of automating deployments, unit testing, compressing images, scripts & stylesheets and other tasks you may wish to perform in your plug-in’s development, build and release cycle is certainly not unique to Grunt. Before I switched to Grunt I had a home-grown Makefile to perform a lot of my routine tasks.

Grunt however, brings this all under one roof: giving a familiar command line procedure to execute task(s). Importantly it allows (Grunt) plug-ins, and their end-users to add structure to their tasks. By this I mean tasks being able to call other tasks, being able to initiate an entire list of tasks, being able to configure all your tasks in only one file and easy of portability. In fact, if you download a development repository for a plug-in which includes the package.json and Gruntfile.js files, in one command you can install all the Grunt plug-ins it requires for use in testing, building and deploying that plug-in. (This assumes you have Node.js installed, which I’ll cover in part two).

Grunt doesn’t offer much new – but it does offer a much better solution.

It’s also popular – and popularity is key for any library, platform or tool to succeed and to grow. Popularity brings greater number of developers, they drive the growth of plug-ins & features and the increased functionality drives popularity. (Perhaps not dissimilar to WordPress’ own growth). Grunt’s ecosystem is already very substantial: there’s phpunit for PHP unit testing, jshint for Javascript linting, there is uglifyjs for compressing javascript files and imagemin to optimise images.

The point is: it has a large, and growing ecosystem. In the majority cases, for any task you might want to perform, there will exist a grunt plug-in to perform it.

And if there isn’t? Grunt is incredibly well documented, open-source and easy to dive into. If you find a gap in its armoury, the chances are it’s easy enough to fill.

What’s next?

If you weren’t already sold on Grunt, hopefully that will do it. The next post will be on installing Grunt and executing your first task.