q

q is the second of my tiny npm modules, even smaller than the last. It’s to be used with browserify and gives arrays rather than NodeLists from DOM queries.

q wraps querySelector and querySelectorAll in one function, it returns arrays except when there is a single element in which case it returns the element itself:

var q = require('@artcommacode/q')

q('ul li')
// => [ <li>...</li>, <li>...</li>, <li>...</li> ]
q('ul li')[0].textContent
// => $1
q('ul li')[0] === q('ul li:first-of-type')
// => true

Pass an element in as the second argument to run a query on it:

var ul = q('ul')
q('li', ul)
// => [ <li>...</li>, <li>...</li>, <li>...</li> ]

q will return an empty array if no elements are found:

q('ul div')
// => []

The meat of q is only 10 lines:

function toArray(nodeList) {
  return [].slice.call(nodeList)
}

module.exports = function q(query, element) {
  var root = element && document.body.contains(element) ? element : document
  var elements = toArray((root).querySelectorAll(query))
  return elements.length === 1 ? elements[0] : elements
}

And finally, for testing I used substack’s tape and testling. Testling will open a tab in your browser and run tests against an HTML file to display the results in your terminal. It’s a very neat process but the module is now unfortunately unmaintained.

var test = require('tape')
var q = require('../')

test('test q', function (t) {
   var ul = q('ul')
   var li
   t.equal(q('ul li').length, 3)
   t.equal(q('ul li')[0].textContent, '$1')
   t.deepEqual(q('ul li:first-of-type'), q('ul li')[0])
   t.deepEqual(q('ul li'), q('li', ul))
   t.deepEqual(q('ul div'), [])
   t.deepEqual(q('div', li), [])
   t.end()
})

I released this module because I found myself writing something very similiar each time I had to interact with the DOM, wanting to use Array methods such as forEach, filter and reduce on returned elements. It also helps that q() is much shorter than typing document.querySelectorAll()!


The Development of 2015 in Development

  1. Less jQuery, more Browserify and npm packages.
  2. Less grunt and gulp, more npm scripts.
  3. Less underscore and lodash, more Ramda and pure JS.
  4. Less Sublime Text, more Emacs, particularly with Spacemacs.
  5. Less EJS and HTML, more Jade.
  6. Less Node.js and JavaScript, more Clojure and ClojureScript.
  7. Less MongoDB, more Postgres.
  8. Less devops, more Heroku.
  9. Less note-taking apps, more Markdown.

Overall I’m moving towards simpler, more focused tools that I can combine to produce smaller, more easier to understand packages.


npm Directives

"scripts": {
   "build-js": "browserify src/js/script.js | uglifyjs -mc > build/js/script.js",
   "build-css": "cat src/css/*/*.css src/css/*.css | uglifycss > build/css/style.css",
   "build-html": "jade src/jade/*.jade -o build",
   "build": "npm run build-js && npm run build-css && npm run build-html",
   "watch-js": "watchify src/js/script.js -o build/js/script.js -dv",
   "watch-css": "catw src/css/*/*.css src/css/*.css -o build/css/style.css -v",
   "watch-html": "jade src/jade/*.jade -o build -w",
   "watch": "npm run watch-js & npm run watch-css & npm run watch-html",
   "start": "npm run build && cd build; http-server",
   "start-dev": "npm run watch & npm start",
   "postinstall": "npm run build"
 }

I recently replaced my 100 line gulp file with 10 lines of npm directives which have already proven to be simpler and more reliable.

Not happy with the hacks that were piling up in my gulpfile.js I started looking at the alternatives. Makefiles are too magical for me but I haven’t ruled them out completely. For anybody interested in using them I highly recommend James Coglan’s post “Building JavaScript projects with Make”.

Instead I found substack’s post “task automation with npm run” and have liberally copied from it. Because this current project is a static site and not one of my usual Express applications I’ve also added Jade to convert my .jade files to HTML and http-server to serve the site.


Tumblr API & Node.js

function getFacebooks(id) {
   var fb = require('fb')
   fb.setAccessToken(config.facebook.accessToken)
   return new Promise(function (resolve, reject) {
     fb.api(id + '/feed', function (res) {
       if (!res || res.error) return reject(res.error)
       return resolve(res.data.filter(filterFbPost).map(formatFbPost))
     })
   })
}

function getTumblrs(name) {
   var tumblr = require('tumblr.js').createClient(config.tumblr)
   var getPosts = Promise.promisify(tumblr.posts, tumblr)
   return getPosts(name).then(function(data) {
     return data.posts.map(formatTumblrPost)
   })
}

Thank you Tumblr for having the best API documentation I’ve seen in this week of scraping posts from every social media site under the sun. Maintaining an official npm library is an added bonus.

(the differences between scraping from Tumblr and Facebook can be seen above)


Writing About Me

Ryan Smith is an Australasian immigrant, living in Scotland and working for Graphical House where he is a Senior Developer and technology wrangler for money. His interests are many and diverse (owing to his millennial attention span) and some of them include design, photography, hardcore punk and obscure fashion no-one has ever heard of.

His current foci are the functional programming languages — primarily Clojure, Haskell, and if he squints hard enough, Javascript — and he spends his private development time arguing about them at length over the internet. You can read some of his opinions here.

Recently commissioned my copy-writing girlfriend to make up some words about me as part of a series of talk proposals I’m putting together.


Doing Away with Express-Messages

Although I was recently appointed as a maintainer of express-messages by the expressjs organisation I no longer use it any of my projects. There’s a lot I don’t particularly agree with – HTML as strings, lack of support for templating engines and particularly the way it locks you into a structure – but I can’t deny it’s been very useful to me in the past and I hope I can keep it that way for others in the future.

So I’m going to show you what I do instead. Lets get rid of that app.use(require('express-messages')()) and replace it with:

app.use(function (req, res, next) {
  var flash = req.flash()
  res.locals.messages = Object.keys(flash).reduce(function (messages, type) {
    flash[type].forEach(function (message) {
      messages.push({type: type, message: message})
    })
    return messages
  }, [])
  next()
})

What we’re doing here is taking the object of arrays generated by connect-flash and turning it into an array of objects so I don’t have to do any fiddly logic to include it in my templates. If I’m using Jade (my personal favourite) I can do this:

ul.messages
  each message in messages
    li(class="#{message.type}")= message.message

and if I’m using EJS I do this:

<ul class="messages">
  <% messages.forEach(function (message) { %>
    <li class="<%= message.type %>"><%= message.message %></li>
  <% }) %>
</ul>

Now instead of asking express-messages to concatanate some strings into HTML for us to include as a function in our template we get a nice messages array on the locals object that we can include wherever and however we want.


Reading the Problem

Yesterday a friend asked on Twitter if I had any JavaScript/jQuery snippets to “detect that something has scrolled into view and trigger an event”. Pretty easy I thought:

$(window).bind('scroll', function() {
  var $e = $('.some-element')
    , scrolltop = $(window).scrollTop() + $(window).height();
  if (scrolltop > $e.offset().top) {
    console.log('in view');
  } else {
    console.log('not in view');
  }
});

Of course, anybody that actually read the question (unlike me, apparently) would realise that the above snippet doesn’t answer it at all, all we can do with this is constantly log whether or not the element is view. Lets try again:

var currentlyIn = false;
$(window).bind('scroll', function() {
  var inView = ($(this).scrollTop() + $(this).height()) > $('.some-element').offset().top;
  if (inView === currentlyIn) return true;
  $('.some-element').trigger(inView ? 'in-view' : 'out-view');
  currentlyIn = inView;
});

Which – unless you already know what it’s for – is almost as useless as the first attempt. Time for some comments:

var currentlyIn = false; 
// we need somewhere to hold state so that we don't continuously emit events
$(window).bind('scroll', function() {
// bind the scroll event from the window so this function's called when the user scrolls
  var inView = ($(this).scrollTop() + $(this).height()) > $('.some-element').offset().top; 
  // check if the top of the element is less than the bottom of the window
  if (inView === currentlyIn) return true;
  // if the new state is the same as the old then don't do anything
  $('.some-element').trigger(inView ? 'in-view' : 'out-view');
  // 'else' trigger an event depending on whether the element has gone in or out of view
  currentlyIn = inView;
  // update the state
});

Done!

… and then I woke up this morning and realised that I still hadn’t answered the question. (this might explain why I do so poorly in written examinations) While the code would detect when the element comes in from the bottom of the screen, what happens at the top of the viewport? Lets fix that right up and turn it into a little jQuery plugin too:

// the plugin
(function($) {
  var currentlyIn = false
    , $window = $(window);
  $.fn.inViewport = function() {
    var $e = this;
    $window.bind('scroll', function() {
      var aboveBottom = ($window.scrollTop() + $window.height()) > $e.offset().top
        , belowTop = ($window.scrollTop() < ($e.offset().top + $e.outerHeight()))
        , inView = aboveBottom && belowTop; 
      if (inView === currentlyIn) return true;
      $e.trigger(inView ? 'in-view' : 'out-view');
      currentlyIn = inView;
    });
    return this;
  };
}(jQuery));

// and to call it
$('.some-element')
  .inViewport()
  .on('in-view', function() {
    // in view
  })
  .on('out-view', function() {
    // out of view
  });

Overall, quite different from where I started.


A Type Driven Approach to Functional Design

Michael Feathers’ (@mfeathers) “Type Driven Approach to Functional Design” is a perfectly-sized 20 minute talk on how to think about solving design problems in a functional way.

It also serves as a gentle introduction to Haskell’s Type Annotation which I personally find useful even when not working in Haskell, or in languages without proper typing at all! (JavaScript I’m looking at you)


Join Me on My Functional JavaScript Journey

I have some pages, each of these pages has a string called “tags” containing a list of comma-separated tags. What I need is a way of grabbing all of these tags and counting them:

var allTags = {};
for (var i = 0; i < pages.length; i++) {
  var tags = pages[i].tags.split(', ');
  for (var j = 0; j < tags.length; j++) {
    allTags[tags[j]] = allTags[tags[j]]
      ? allTags[tags[j]]++
      : 1;
  }
}

Okay, that’s not even remotely functional and it’s also pretty ugly. Lets take advantage of Array.reduce to functionify this up a little:

pages.reduce(function (allTags, page) {
  page.tags.split(', ').forEach(function (tag) {
    allTags[tag] = ++allTags[tag] || 1;
  });
  return allTags;
}, {});

That’s a little better! We’re just reducing the pages into an empty object ({} at the end there) that contains the summed-up tags. It works quite well and is looking pretty good but I’m not happy with the nesting of that forEach loop. Lets try again:

pages.reduce(function (allTags, page) {
  return allTags.concat(page.tags.split(', '));
}, []).reduce(function (countedTags, tag) {
  countedTags[tag] = ++countedTags[tag] || 1;
  return countedTags;
}, {});

Much better. On the first pass we go through each page and return an array containing every occurrence of a tag, the second pass simply takes this array and sums up those occurrences.


The Right Solution to the Wrong Problem

So I thought I’d come up with a pretty neat solution to organising a collection of posts in the format “YYYY [title]” by year:

Collection.findAndPopulate({
  title: this.controller
}, function (error, collection) {
  if (error) return callback(error);
  collection.pages = _.filter(collection.pages, function (page) {
    return _.first(page.title.split(' ')) === req.params.year;
  });
  callback(null, collection.pages);
});

Pretty cool right? Underscore is great when working with lists. However, a few hours after pushing the new code to production I realised (while in the shower) I could’ve skipped the Underscore and just done:

Page.findAndPopulate({
  title: new RegExp('^' + req.params.year),
  collection: this.controller
}, callback);