Grunt & WordPress development IV: Another task for internationalisation

grunt

This is part 4 in a series looking at using Grunt in WordPress plug-in/theme development.

  1. Grunt & WordPress development
  2. Grunt & WordPress development II: Installing Grunt
  3. Grunt & WordPress development III: Tasks for internationalisation
  4. Grunt & WordPress development IV: Another task for internationalisation

WordPress has recently (since 3.5) seen a shift towards a more JavaScript codebase. This shift is still minor (it currently accounts for less than 15%, according to its GitHub repository, in 3.9). But the introduction of Backbone.js and the re-factoring of particular the editor in the WordPress admin (media manager, shortcodes “objects” etc.) are testament to Matt Mullenweg’s comment:

I forgot to mention our biggest architectural change, which is already ongoing: an ever-increasing percentage of our codebase is shifting to Javascript instead of PHP. At some point I expect it to be the vast majority.

Matt Mullenweg, https://news.ycombinator.com/item?id=4744542

However, a JavaScript codebase presents difficulties for WordPress in terms of internationalisation. I ran into this while working on Event Organiser Pro’s 1.7 release. This new release saw the booking form customiser rewritten to use Backbone. With so much JavaScript replacing PHP, there became a massive need to allow strings to be translatable (in a sane way).

Received wisdom is that you should use wp_localize_script() to allow strings in javascript files to be translated. Pippin covers the method excellently here but essentially when you enqueue your script you use wp_localize_script() to make a variable available which contains all you translated strings.

function my_load_scripts() {
    wp_enqueue_script( 'my-script', plugin_dir_url( __FILE__ ) . 'my-script.js');
    wp_localize_script('my-script', 'mynamespace', array(
         'helloworld' => __('Hello World', 'mydomain' )
    ));
}
add_action('wp_enqueue_scripts', 'my_load_scripts');

Then in your JavaScript file, instead of the string “Hello World” you would use mynamespace.helloworld. Although this method is common, there are a couple of things wrong with it when you pit it against some sort of gettext function:

It makes code harder to read. For me when reading code it’s much easier to see the actual text rather than a variable. (More so when reading other people’s code as this helps you link the sourcecode with what you actually see)

<script>
  alert( mynamespace.welcome_msg ); //ok
  alert( mynamespace.gettext( "Welcome to..." ) ); //better
</script>

At the very best its slightly more cryptic and uglier.

It makes it harder to maintain code. – When editing a JavaScript file, a gettext function allows me to edit the string there and then. If I use wp_localize_script(), I need to track down the .php file responsible for that and change it there – and then not forget that that string might have been used elsewhere.

It makes the translator’s job harder. – Hands up anyone who is very poor at providing translators with comments or providing a context when appropriate. Me at least. Regardless, .po files provide a line number so that, if necessary, translators can look up that line to get some sort of context for the string they are translating. It’s not very helpful when that line number points them to a large array in some obscure php file, rather than where the text is being used. Nor is it immediately obvious which JavaScript file(s) are using the string.

You may think that my reasons here are weak and pinnikity… and you might right… But I prefer using gettext-esque function for translating strings.

The problem(s)… (and how Grunt helps solve them)

There a couple of problems with using trying to use a gettext function in a JavaScript file, but they are all easily solved:

  1. There is no native gettext function
  2. How do you get the translations from .po to your JavaScript file
  3. How to get translatable strings from your JavaScript file to your .pot

There is no native gettext function in JavaScript

A very simple solution is to roll your own. Below are four functions which handle translatable strings, plurals and contexts. They all expect the translated strings to be found in mynamespace.locale.

mynamespace.gettext = function( msgid ){
    if( this.locale[msgid] !== undefined ){
        return this.locale[msgid];
    }
    return msgid;
};

mynamespace.ngettext = function( msgid1, msgid2, n ){
    var key = ( n > 1 ? msgid1 + '_plural' : msgid1 );
    if( this.locale[key] !== undefined ){
        return this.locale[key];
    }
    return ( n > 1 ? msgid2 : msgid1 );
};

mynamespace.pgettext = function( ctxt, msgid ){
    if( this.locale[msgid+'_'+ctxt] !== undefined ){
        return this.locale[msgid+'_'+ctxt];
    }
    return msgid;
};

mynamespace.npgettext = function( ctxt, msgid1, msgid2, n ){
    var key = ( n > 1 ? msgid1 + '_' + ctxt + '_plural' : + '_' + ctxt + '_' + msgid1 );
    if( this.locale[key] !== undefined ){
        return this.locale[key];
    }
    return ( n > 1 ? msgid2 : msgid1 );
};

You may have noticed that only one plural form is supported (so a string is either plural or singular), but some languages use more (and some less). There are ways around this, but the limitation is also a result of the Grunt task that’ll we’ll use later. Plural strings and strings with a context expect _plural and _{context} modifiers – I personally think this is less than ideal, but again is forced upon me by my choice of Grunt task. (This is just a start, and I’d like to see these limitations lifted).

Getting translations from .po to your JavaScript file

This is a two-step process:

  1. Use a Grunt task to generate a .json file for each .mo file
  2. Depending on the user’s choice of locale, load that .json file and use wp_localize_script() to make it available in you JavaScipt file.

I went with grunt-i18next-conv to generate the .json files. I found that converting .po to .json included untranslated strings, so I recommend you opt for converting .mo files to .json. (If you need a Grunt task for generating your .mo task, I recommend po2mo task I covered in my last article). It’s this task, by way of the format of the .json file it produces, that imposes some of the limitations already mentioned.

Next you, when enqueuing your JavaScript file, you use wp_localize_script() to ‘attach’ the relevant .json file to it. In the following I expect that the .json files are of the form /languages/mytextdomain-{locale}.json

 $locale = array();
 $file = plugin_dir_path( __FILE__ ) . 'languages/mytextdomain-'.get_locale().'.json';
 if( file_exists( file ) ){
      $locale = json_decode( file_get_contents( $file ), true );    
 }

 wp_localize_script('my-script', 'mynamespace', array(
      'locale' => $locale
 ));

Getting the translatable strings from your JavaScript file to .pot

If you’re using grunt-pot this is easy. Simply include the functions above in the ‘keywords’ option:

  keywords: [ 
        ...
        'namespace.gettext:1',
        'namespace.ngettext:1,2',
        'namespace.pgettext:1c,2',
        'namespace.npgettext:1c,2,3',
        ...
       ]

and ensure the files to search include your JavaScript file.

Limitations

As discussed above there are currently two limitations:

  1. Poor support for plurals other than ‘single form plurals’
  2. Awkward ‘.json’ structure (not a massive issue…)

For the time being, however, and for use in Event Organiser’s booking form customiser, this method was ideal.

Grunt & WordPress development III: Tasks for internationalisation

grunt

This is part 3 in a series looking at using Grunt in WordPress plug-in/theme development.

  1. Grunt & WordPress development
  2. Grunt & WordPress development II: Installing Grunt
  3. Grunt & WordPress development III: Tasks for internationalisation
  4. Grunt & WordPress development IV: Another task for internationalisation

Internationalisation Tasks

One aspect of WordPress plug-in development that involves a lot of mundane work is that of internalisation: ensuring WordPress’ localisation functions are used correctly, generating a .pot file, compiling submitted .po files to .mo files. The latter two you can do with Poedit – but this still involves manually opening the .po/.pot file. These tasks can be completely automated so let’s do that:

po2mo – Compiling to .po files to .mo

The po2mo plug-in automatically compiles given .po files and produces a .mo file of the same name.

To install:

npm install grunt-po2mo --save-dev

<em>Please not an earlier version of this article executed the above as a super user (`sudo npm`). As pointed out by Lacy in the comments, this necessary and can cause permission issues with the npm cache.</em>

The following set up looks in the languages directory for any .po files and compiles them, creating the corresponding .mo in the same directory:

po2mo: {
    files: {
        src: 'languages/*.po',
        expand: true,
    },
},

Finally load the task by adding grunt.loadNpmTasks('grunt-po2mo'); at the bottom of your Gruntfile.js, just after grunt.loadTasks('tasks');. Then whenever you add or change a .po file:

grunt po2mo

(You can see a live example of this task, and the others listed below, here.

pot – Create a .pot template file

For users to be able to translate your plug-in you’ll need to create a .po template file ( a .pot file). The pot plug-in does exactly that.

You just need to provide it with:

  • The files to search in,
  • The keywords to search for (and indicate which arguments are translatable strings, and which are context specifiers)
  • A text domain (used only for naming the the .pot file)
  • The directory where you wish to output the .pot file.

To install:

npm install grunt-pot --save-dev

Then

pot: {
      options:{
          text_domain: 'my-plugin', //Your text domain. Produces my-text-domain.pot
          dest: 'languages/', //directory to place the pot file
          keywords: [ //WordPress localisation functions
            '__:1',
            '_e:1',
            '_x:1,2c',
            'esc_html__:1',
            'esc_html_e:1',
            'esc_html_x:1,2c',
            'esc_attr__:1', 
            'esc_attr_e:1', 
            'esc_attr_x:1,2c', 
            '_ex:1,2c',
            '_n:1,2', 
            '_nx:1,2,4c',
            '_n_noop:1,2',
            '_nx_noop:1,2,3c'
           ],
      },
      files:{
          src:  [ '**/*.php' ], //Parse all php files
          expand: true,
      }
},

Finally load the task by adding grunt.loadNpmTasks('grunt-pot'); to the bottom. Then to generate your .pot file:

grunt pot

checktextdomain – Verify localisation functions have been used correctly

Having generated a .pot file, gathered translations for your plug-in and then compiled them – it would be entirely wasted if you haven’t used the WordPress localisations functions properly. In particular, if you had failed to specify the correct domain, your efforts would have been wasted.

When coding it’s easy to forget to specify a text domain, or to mistype it. Or perhaps you’ve been using a variable for the domain, and now want to switch to a literal string.

The checktextdomain – not only checks if you’ve used the correct textdomain in the localisation function it can also correct it for you.

Simply provide it with:

  • Files to look in,
  • Keywords to look for (important: you must provide a domain argument specifier)
  • A text-domain to check against
  • Whether you want mistakes corrected (it will not add missing domains… yet).

The plug-in will then

  • Warn you if some keywords have been used without a text domain
  • Warn you if some keywords have been used with an incorrect text domain (optionally correct it for you)
  • Warn you if some keywords have been used with a variable text domain (optionally correct it for you)

There are various options for this plug-in to enable you to check (and correct) the things you want to. You can see all the available options for this Grunt plug-in on its Github page.

To install:

npm install grunt-checktextdomain --save-dev

You’ll notice that the keywords option is very similar to grunt-pot. There is an important distinction. For this plug-in to work you must extend the keyword specifier and indicate where the domain should be.

E.g. 2d indicates that the domain should be passed as the second argument of the localisation function

checktextdomain: {
   options:{
      text_domain: 'my-plugin',
      correct_domain: true, //Will correct missing/variable domains
      keywords: [ //WordPress localisation functions
            '__:1,2d',
            '_e:1,2d',
            '_x:1,2c,3d',
            'esc_html__:1,2d',
            'esc_html_e:1,2d',
            'esc_html_x:1,2c,3d',
            'esc_attr__:1,2d', 
            'esc_attr_e:1,2d', 
            'esc_attr_x:1,2c,3d', 
            '_ex:1,2c,3d',
            '_n:1,2,4d', 
            '_nx:1,2,4c,5d',
            '_n_noop:1,2,3d',
            '_nx_noop:1,2,3c,4d'
      ],
   },
   files: {
       src:  [ '**/*.php', ], //All php files
       expand: true,
   },
},

Finally load the task by adding grunt.loadNpmTasks('grunt-checktextdomain'); to the bottom. Then to check your files:

grunt checktextdomain

I’m planning on improving this further to warn you of missing contexts which using functions that expect one.

Final remarks

Remembering to add grunt.loadNpmTasks(...); at the bottom of your Gruntfile.js, just after grunt.loadTasks('tasks'); is easily forgotten. But there’s a way around this which I’ll discuss in my next post.

Just before publishing Brady Vercher announced his Grunt plug-in, which allows you to utilize the internationalisation tools that WordPress uses. There’s a bit more set-up involved, but a notable advantage over grunt-pot is that it recognises theme template headers as translatable.