hello.

Recently I’ve noticed a lack of resources on advanced Node.js topics. There are plenty of tutorials on the basics, but you’ll find very little written on maintainable design or scaleable architecture. I created The Node.js Handbook to address this gap. You can read all past topics here.

If the V8 Engine is the heart of your Node.js application, then callbacks are its veins. They enable a balanced, non-blocking flow of control and processing power across applications and modules. But for callbacks to work at scale, developers needed a common, reliable protocol. The “error-first” callback (or “Errorback”) was introduced to solve this problem, and has since become the standard protocol for Node.js callbacks. This post will define this pattern, it’s proper use, and exactly why it is so powerful.

Why Standardize?

Node’s heavy use of callbacks dates back to a style of programming older than Javascript itself. Continuation-Passing Style (CPS) is the old-school name for how Node.js uses callbacks. In CPS, a “continuation” function is passed as an argument to be called once the rest of the code has been run. This allows for different functions to safely pass control asynchronously back and forth across an application.

Almost all Node.js code follows this style, so having a dependable callback pattern is crucial. Without one, developers would be stuck looking up and remembering the form and arguments of each and every callback. The error-first pattern was introduced into Node.js core to solve this very problem, and has since spread to become today’s standard. While every use-case has different requirements and responses, the error-first pattern can accommodate them all.

Defining an Error-First Callback

There’s really only one rule for using an error-first callback:

  1. The first argument of the callback is always reserved for an error object. The following arguments will contain any other data that should be returned to the callback. There is almost always just one object following ‘err’, but you can use multiple arguments if truely needed.
    Example: function(err, data)

When it’s time to call an error-first callback, there are two scenarios you’ll need to handle:

  1. On a successful response, the ‘err’ argument is null. Call the callback and include the successful data only.
    Example: callback(null, returnData);

  2. On an unsuccessful response, the ‘err’ argument is set. Call the callback with an actual error object. The error should describe what happened and include enough information to tell the callback what went wrong. Data can still be returned in the other arguments as well, but generally the error is passed alone.
    Example: callback( new Error('Bad Request') );

That’s it! Easy, right?

Implementing an Error-First Callback

1
2
3
4
fs.readFile('/foo.txt', function(err, data) {
  // File Returned
  console.log(data);
});

Above you have an error-first callback in action. readFile takes in a file path to read, and calls your callback once it has finished. If all goes well, the file data is returned in the data argument. But if somethings goes wrong, readFile won’t have any data to return. In that case, the first argument will be populated with an error.

1
2
3
4
fs.readFile('/foo.txt', function(err, data) {
  if (err) throw err; // Check if an error occurred!
  console.log(data);
});

It’s up to you to check for that error, and handle it as best it can. The error object will usually contain some extra information describing what went wrong, which you can use to detect and safely handle all possible problems.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
fs.readFile('/foo.txt', function(err, data) {
  if(err) {
      if(err.fileNotFound) {
          console.log('File Doesn't Exist');
         return;
     }   
     if(err.noPermission) {
         console.log('No Permission');
         return;
     }
     console.log('Unknown Error');
      return;
  }
  console.log(data);
});

Unfortunately the information isn’t normally as explicit as .fileNotFound, but handling errors properly is always worth the investment. Ignoring an error entirely can be disastrous for your application. Don’t cut corners on this.

Err-ception: Propagating Your Errors

The error-first pattern is the backbone for safe communication between Node.js modules. When a method passes its errors to a callback, it no longer has to make assumptions on how an error should be handled. Instead of choosing between throwing or ignoring, errors get handled at the level that makes sense.

When you’re consistent with this pattern, errors can be propagated up as as many times as you’d like. This is especially important when throwing isn’t possible, like within a web application. If some method threw an error while attempting to read a file the whole server would crash. By propagating it up the callback has a chance to handle the error. And if that specific route doesn’t know to handle it, it can propagate up even farther to be handled with a 500 or 404 error page.

1
2
3
4
5
6
7
8
if(err) {
  // Handle "Not Found"
  if(err.fileNotFound) return res.send('File Does not Exist');
  // Handle "No Permission"
  if(err.noPermission) return res.send('No Permission');
  // Propogate Other Errors, Express Will Catch Them
  return next(err);
}

Slow Your Roll, Control Your Flow

With a solid callback protocol in hand, there are endless possibilities for controlling your asynchronous flow. There are parallel flows, waterfall flows, and many more creative ways to direct asynchronous code. If you want to read in 10 different files, or make 100 different API calls, you no longer have to make them one at a time.

The async library can handle all this and more. And because it uses error-first callbacks, it’s incredibly easy to work with. Async can simplify and speed up your code, and hopefully keep you out of callback hell….

Bringing it all Together

To see all these concepts come together, check out some more examples on Github. And of course, you can always choose to ignore all of this callback stuff and go fall in love with promises… but that’s a whole other post entirely :)

Whether you love terminal customization or feel more Rand Paul with your ‘.bash_profile’, this tip will change your life. This is one of the first commands I run on a new machine, and it has been an invaluable tool for as long as I can remember. 

git config --global alias.lg "log --color --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr)%C(bold blue)<%an>%Creset' --abbrev-commit"

― via Filipe Kiss

This command creates a new ‘git lg’ alias to use instead of the default ‘git log’. What does running git lg do exactly? It turns this mess:

git log example

into this:

git lg example

Beautiful, isn’t it? You get more information above the fold by removing the noise. And as an added bonus, you can see your branches ebb and flow across your repository. This can be super helpful for navigation and rebasing, and invaluable when used in merge-based workflows like GitHub’s.

#lifehackz.


Follow @FredKSchott for more discussion and git tips. Got any favorites of your own? I’d love to see them. Leave a comment below and spread the productivity/joy.

A new stable version of Node was released last Wednesday, and with it came the newest version of npm. This update included a lot of big fixes, but the most visible change is that ‘install —save’ now prepends a caret (^) instead of a tilde (~).

What does this mean for you? Well, first you should understand the difference between the two. In the simplest terms, the tilde matches the most recent minor version (the middle number). ~1.2.3 will match all 1.2.x versions but will miss 1.3.0. This has been the default behavior of ‘—save’ since the start, and you are probably already comfortable seeing it in your package.json. The caret, on the other hand, is more relaxed. It will update you to the most recent major version (the first number). ^1.2.3 will match any 1.x.x release including 1.3.0, but will hold off on 2.0.0. npm’s semantic versioning parser clarifies the distinction:

~1.2.3 := >=1.2.3-0 <1.3.0-0 “Reasonably close to 1.2.3”.
^1.2.3 := >=1.2.3-0 <2.0.0-0 “Compatible with 1.2.3”.

― isaacs/node-semver (emphasis added)

The difference between “reasonably close” and “compatible” can be traced back to semantic versioning (SemVer) semantics. From the spec:

Given a version number MAJOR.MINOR.PATCH, increment the:

  • MAJOR version when you make incompatible API changes,
  • MINOR version when you add functionality in a backwards-compatible manner, and
  • PATCH version when you make backwards-compatible bug fixes.

― semver.org

Only major versions are allowed to break compatibility, so a developer should be able to switch between minors and patches with ease. The caret trusts that this promise is kept, but the tilde is a little more pessimistic.

Of course every rule has an exception, and semver’s lies in pre-release versioning:

Major version zero (0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable.

― semver.org

Unlike 1.0 and above, 0.x software has only one promise: “Anything can change at any time.” While early developers are now free to rapidly increment versions without constraint, most will still maintain a basic level of sanity. Patches are still commonly reserved for bug fixes & internal changes, while the minor version is used for everything else.

npm’s semver parser relies on this sanity, so the tilde’s behavior for matching “reasonably close” versions can remains unchanged. The caret, however, won’t work with this new rule. To maintaing compatibility, it must be similarly conservative and only match the minor version. So for version zero software, the caret behaves like the tilde. Both ~0.1.2 and ^0.1.2 will match the most recent 0.1 software, but ignore 0.2.x since it could be incompatible.

You probably won’t notice anything immediately, and your package.json file is still fine as it is. But next time you save a dependency you may notice a caret has crept into your package.json.

Eventually you may want to consider switching all dependencies to one or the other, or even just a select few. But this decision is entirely up to you, and should be based on your use case, the packages themselves, and just how much risk you’re willing to take on.

Whether you are a big fan of terminal customization or feel more Rand Paul with your .bash_profile, this tip will change your life. This is one of the first commands I run on a new machine, and it has been an invaluable tool for as long as I can remember. 

from Filipe Kiss:

1
git config --global alias.lg "log --color --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr)%C(bold blue)<%an>%Creset' --abbrev-commit"

What does your new ‘git lg’ alias do exactly? It turns this:

git log example

into this:

git lg example

You get more information above the fold, with a focus on the most relevant bits. As an added bonus, the graph view shows how your branches are laid out across your git log. This can be super helpful for navigating your brances, and invaluable when using a merge-based workflow like GitHub’s. #lifehackz

Got any other favorite git tips? I’d love to see them. Add them to the comments section below and spread the productivity/joy.

In synchronous code, handling your context is easy. If you want to grab your current user or request data, you usually just pull it out of the ether. In Node.js it’s not that simple. There’s no thread to hold your context, and no ether to be found. So when you want to do something like logging, how do you attach meta-data without passing it explicitly down your call chain?

In this talk I introduce Continuous-Local Storage and how it can solve this problem for you. Attach meta-data at the top of your call-chain and access it at the bottom. No more overloading the ‘req’ object & no more hijacking domains. It’s as simple as that.

Slidedeck Includes Annotations
Additional slides have been added to the deck below to improve overall readability.

Slides

Video