JavaScript Architecture: RequireJS Dependency Management

03.20.2012

Updated Aug 11, 2012 to reflect current library versions.

In JavaScript Architecture: Organization and Quality, we discussed the importance of breaking apps down into very small, decoupled pieces following the single responsibility principle. Small pieces are generally easier to comprehend than a mess of large peices.

Some devs coming from other languages feel like they can only build a few large, spaghetti JavaScript files for their app. I tend to think this may be caused by three reasons:

  • That’s the way JavaScript has been done in the past.
  • Loading many JavaScript files requires many HTTP requests resulting in longer load times.
  • Dependency management is hard in JavaScript.

Don’t stoop to these excuses. We can do better than that and today there are fantastic libraries and standards that can help us out. However, let’s see what the above problems mean.

That’s the way JavaScript has been done in the past. Unfortunately, that’s true. For years many teams developed within one or two large JavaScript files containing thousands of lines of code. It goes without saying that maintainability was very poor. It’s difficult to wrap our minds and IDEs around thousand of lines of interoperating code and also a nightmare with version control in a team environment. I do believe this is a large contributor to why JavaScript has had a bad wrap in the dev community for so long.

Loading many JavaScript files requires many HTTP requests resulting in longer load times. It’s true. Each time a JavaScript file is loaded it requires the overhead of an inidiviual HTTP request and, unfortunately, if you list many script tags in your page the files will load in sequence, not in parallel.

Dependency management is hard in JavaScript. By splitting code out into many files, you must keep track of which files depend on each other and make sure they’re loaded in the right order. This can be a messy long list and easy to mess up later down the line. For example:

<script src="script3.js"></script>
<script src="script1.js"></script>
<script src="script13.js"></script>
<script src="script7.js"></script>
<script src="script6.js"></script>
<script src="script12.js"></script>
<script src="script4.js"></script>
<script src="script11.js"></script>
<script src="script5.js"></script>
<script src="script9.js"></script>
<script src="script8.js"></script>
<script src="script10.js"></script>
<script src="script2.js"></script>

This is troubling enough on a single-man project but even more-so within a team.

Modules

Let’s get nostalgic for a moment. A few years back, using JavaScript on the server-side was just starting to get hot. Server-side libraries and JavaScript engines were being deployed but they didn’t have a good, standard API for working with one another and defining dependencies. Getting one library to work with another required some finagling and it was obvious that if JavaScript were to scale it would need some common APIs.

In January 2009, Kevin Dangoor wrote a blog post titled What Server Side JavaScript Needs outlining just that–what server-side JavaScript needed. The list included:

  • Cross-interpreter standard library
  • Standard interfaces
  • Standard way to include other modules
  • Package up code for deployment and distribution
  • Install packages and their dependencies
  • Package repository

As a means to fulfill these needs, he created a Google group named ServerJS where like-minded folk could collaborate. Soon enough, the group realized that many of these goals weren’t necessarily limited to the server-side and renamed the group to CommonJS.

One of the standards that CommonJS worked toward was that of a module. A module is a contained piece of code (How’s that for vague?) that defines not only that it is a module itself but which other modules it depends on to correctly function. When calling for module B, module B might call for module G and module M, which might call for modules D and W. By having a module standard, dependency management becomes easier. Rather than keeping some sort of implicit master list that must be kept in order, each module just defines its own dependencies and that mapping can be used to determine required resources and the order in which they must be loaded.

AMD

The module concept was great for server-side development as it addressed how to synchronously load modules based on dependency definitions, but the JavaScript devs on the browser-side got a bit jealous. Why should such awesomeness be confined to the server? Sure, module loading needs to be done asynchronously in the browser, but that didn’t mean the concept of modules and dependency definitions couldn’t be applied.

The Asynchronous Module Definition, or AMD, was born for this purpose. It takes the module and dependency definition API from the server-side and applies it to the asynchronous paradigm of the browser.

RequireJS

So what does RequireJS have to do with this? Well, even though we can define our modules and their dependencies with AMD, we need something smart that can take this dependency map, load the modules, and execute the modules in order. That’s the role RequireJS plays. Both RequireJS and AMD are open source, popular, and well-curated by James Burke.

Defining and Requesting Modules

At this point let’s jump straight into some code and hopefully the concepts will start to solidify. Almost always, a module is defined within a single file and, vice-versa, a single file only contains a single module definition. Defining a module, at its core, is as simple as the code below. Let’s assume it’s within a file called book.js.

define({
    title: "My Sister's Keeper",
    publisher: "Atria"
});

We now have a book module. define() is a RequireJS function. When we call it, we’re essentially saying, “Register what I’m passing you as a module.” By default, RequireJS assumes the module name is the file path following the base url (don’t get your panties in a twist–we’ll get to that in a minute) excluding the extension. In this case, that means that “book” is our assumed module name. When other code asks RequireJS for the “book” module, RequireJS will return the object we just defined above. Now let’s make a “bookshelf” module in a new file named bookshelf.js and see how we can request the book module into it.

define([
	'book'
], function(book) {
	return {
		listBook: function() {
			alert(book.title);
		}
	};
});

Notice this one’s a bit different than the book module. The book module didn’t have any dependencies so it was a bit simpler. Here, we’re defining an array of dependencies for our bookshelf–in this case, book. The second parameter is a callback function. If book.js hasn’t been loaded into the app yet, RequireJS will go fetch it from the server. Once book.js is loaded and the book module is registered, RequireJS will execute our callback function and pass the module (the book object we defined previously) in as a parameter. The argument name isn’t technically significant. We could have just as easily said function(a1337Book) or used whatever name we wanted. In this case, it makes sense to have our argument name match the module name. Whatever object we return from this callback function will be registered with RequireJS as the bookshelf module. In our case, it’s an object with a listBook() method that alerts the book’s title.

RequireJS tries to be as efficient as possible when loading multiple modules. For example, if multiple dependencies are listed, RequireJS will load all the dependencies as fast as possible in parallel.

App Setup

So…how do we get this party started? Let’s first start by getting our html page set up. Here’s how it looks:

<!DOCTYPE html>
<html>
	<head>
		<title>RequireJS Example</title>
		<script data-main="js/main" src="js/libs/require.js"></script>
	</head>
	<body/>
</html>

Quite literally, you can build a large application without adding anything else to your html file just by manipulating the body’s content using JavaScript and loading html snippets/templates using Require. We’ll get there. For now, take notice that there’s a data-main attribute. This tells RequireJS where our bootstrap file is–in our case, it’s main.js that lives under a js directory (it assumes main has a js extension).

main.js is what we could call our bootstrap file. We’ll make ours look like this:

require([
	'bookshelf'
], function(bookshelf) {
	bookshelf.listBook();
});

Because we specified this file as our data-main in our html file, RequireJS will load it as soon as possible and it will be immediately executed. You’ll notice this has a lot of similarities to our previous modules but instead of calling define() it calls require(). The define() function–at least when dependencies are defined–really does three things: (1) loads the dependencies specified, (2) calls the callback function once dependencies are loaded, and (3) registers the return value from the callback function as the module. The require() function only does #1 and #2, not #3–hence the function names “define” vs. “require”. In the case of main.js, it’s just a bootstrap file. I don’t need to have a “main” module registered with RequireJS because nothing will be calling for it as a dependency.

Our bootstrap lists our “bookshelf” module as a dependency. Assuming the bookshelf module hasn’t already been registered with RequireJS, it will load bookshelf.js. Once it loads bookshelf.js, it will see that bookshelf has the book module listed as a dependency. If the book module hasn’t already been registered, it will then load book.js. Once that’s done and book and then bookshelf have registered their respective objects as modules with RequireJS, the callback in main.js will be executed and bookshelf will be passed through. At that point we can do whatever we want with bookshelf. If needed, we can list multiple module dependencies and they will all be passed into the callback as soon as they’re all loaded and registered with RequireJS.

Configuration

Previously I mentioned the concept of a base url. By default, the base url is whatever directory contains the bootstrap file. In my case, I put main.js under a js directory along with book.js and bookshelf.js. This means, in my case, my base url is /js/. Now let’s say instead of placing all my js files directly under the js directory, I moved book.js and bookshelf.js to /js/model/. Now my main.js would need to look like this:

require([
	'model/bookshelf'
], function(bookshelf) {
	bookshelf.listBook();
});

Now main knows the correct location of bookshelf.js. Likewise, bookshelf.js would list the book dependency as model/book even though book and bookshelf live in the same directory.

This brings us to RequireJS configuration. At the top of my bootstrap file, I usually perform my configuration. main.js might look like this:

require.config({
	baseUrl: '/another/path',
	paths: {
		'myModule': 'a/b/c/d/myModule'
		'templates': '../templates',
		'text': 'libs/text',
	}
});
 
require([
	'bookshelf'
], function(bookshelf) {
	bookshelf.listBook();
});

In this case, I’ve manually changed my base url to something completely different. Personally, I’ve never needed to configure this but it’s there to demonstrate one of the many configuration options available. I also demonstrated how to configure paths. Paths are really just shortcuts. In this case, rather than having to type out “a/b/c/d/myModule” when listing dependencies, I can just type “myModule”. I’ve set up a path (shortcut) for accessing a templates directory that is a sibling to my js directory. I’ve also set up a path (shortcut) for accessing the RequireJS text! plugin more easily.

Class Modules

So far, our modules have all been object instances; bookshelf was an object and book was an object. In reality, modules are very often classes. When using Backbone, modules are classes much more often than not. The bookshelf class needs the book class, for instance. It can then instantiate a bunch of books or whatever it needs to do using the book class. If an instance of the class needs to be passed to different Backbone views, for example, it would be passed through the recipient view’s constructor or other method. You’ll start to see that listing dependencies is similar to listing imports in languages like Java or ActionScript. It’s essentially giving you access to another class that you can then use within the class you’re defining. You just so happen have the ability to also define and retrieve object instances as modules if you really want to. Hopefully this will become more obvious in the next example.

RequireJS + Backbone

Using the knowledge from the previous articles in this series and our new-found knowledge of modules, let’s see an integrated example. In essence, we have a collection of book models. We’re calling the collection a bookshelf. We have a bookshelf view that loops through the books in the bookshelf, creating a book view for each book in the bookshelf. The book view displays information about the book. The views, frankly, are over-architected for the simple stuff we’re doing here, but I think this demonstrates the integration of RequireJS and Backbone nicely. If it’s still unclear, please post a comment and I’ll do my best to answer.

index.html:

<!DOCTYPE html>
<html>
	<head>
		<title>RequireJS Example</title>
		<script data-main="js/main" src="js/libs/require.js"></script>
	</head>
	<body/>
</html>

js/main.js:

require.config({
	'paths': {
		'jquery': 'libs/jquery',
		'backbone': 'libs/backbone',
		'underscore': 'libs/underscore'
	},
 
	shim: {
		'backbone': {
			// These script dependencies should be loaded 
			// before loading backbone.js
			deps: ['underscore', 'jquery'],
			// Once loaded, use the global 'Backbone' 
			// as the module value.
			exports: 'Backbone'
		},
		'underscore': {
			// Use the global '_' as the module value.
			exports: '_'
		}
	}
});
 
require([
	'view/bookshelf-view'
], function(BookshelfView) {
	$(document).ready(function() {
		new BookshelfView({
			el: $('body')
		});
	});
});

js/view/bookshelf-view.js:

define([
	'backbone',
	'underscore',
	'model/bookshelf',
	'view/book-view'
], function(Backbone, _, Bookshelf, BookView) {
	return Backbone.View.extend({
		initialize: function() {
			this.collection = new Bookshelf([
				{
					title: 'A Tale of Two Cities',
					author: 'Charles Dickens'
				},
				{
					title: 'The Good Earth',
					author: 'Pearl S. Buck'
				}
			]);
 
			this.render();
		},
 
		render: function() {
			this.collection.each(function(book) {
				var bookView = new BookView({
					model: book
				})
				this.$el.append(bookView.$el);
			}, this);
		}
	});
});

js/view/book-view.js:

define([
	'backbone',
	'underscore'
], function(Backbone, _) {
	return Backbone.View.extend({
		initialize: function() {
			this.render();
		},
 
		render: function() {
			this.$el.html('Title: ' + this.model.get('title') +
					'; Author: ' + this.model.get('author'));
		}
	});
});

js/model/bookshelf.js:

define([
	'backbone',
	'model/book'
], function(Backbone, Book) {
	return Backbone.Collection.extend({
		model: Book
	})
});

js/model/book.js:

define([
	'backbone'
], function(Backbone) {
	return Backbone.Model.extend({
		// Intended attributes:
		// title
		// author
		// genre
 
		defaults: {
			genre: 'historical'
		}
	})
});

Loading Non-Modules

Some libraries you want to use in your project don’t conform to the AMD spec. In fact, Backbone and Underscore have had their flirts with conforming to AMD but, at the moment of this writing, aren’t AMD modules. Backbone has no concept of modules, dependency loading, RequireJS, or any of that. It just sets a global variable called Backbone and expects you to use it. This means if we were to just add the backbone.js file to our project and list Backbone as a dependency from one of our modules, it wouldn’t work. RequireJS will load backbone.js, but nothing in backbone.js registers itself as a module with RequireJS. RequireJS will throw up its hands and say something like, “Well, I loaded the file, but I didn’t find any module in there.”

For this reason, RequireJS has provided us the ability to specify a “shim” configuration which you can see in the example above. I’ll repeat it here:

require.config({
	...
 
	shim: {
		'backbone': {
			// These script dependencies should be loaded
			// before loading backbone.js
			deps: ['underscore', 'jquery'],
			// Once loaded, use the global 'Backbone'
			// as the module value.
			exports: 'Backbone'
		},
		'underscore': {
			// Use the global '_' as the module value.
			exports: '_'
		}
	}
});

Like the comments say, for Backbone we need to make sure underscore and jQuery are loaded first since they are dependencies of Backbone (that’s the deps part) and then we’ll make the global Backbone variable act like a module (that’s the exports part). Underscore doesn’t have any dependencies but it’s not AMD compatible so we’ve configured the global variable _ to act like a module.

Now, we don’t always have to make everything act like a module. Let’s say we just want to load a single non-AMD JavaScript library file we downloaded off the web. It doesn’t depend on anything else and we don’t want to set up a shim for it for whatever reason. We just want to make sure the file is loaded and then just access the global variable like we normally would if we weren’t using RequireJS. That’s not a problem; we can load it in and access it using the global variable. Here’s an example of how we would do so:

require([
	'js/libs/coolLib.js'
], function() {
	console.log(window.coolLib);
});

A few things to notice here: First, our path to the file includes everything from the directory where our html file is located. This is unlike loading a module which would include everything after our “base url” (read above if you don’t remember what this is). Second, our path includes the file extension. This is unlike loading a module which would exclude the extension. Third, because it’s not a module, RequireJS won’t pass anything into our callback function. At that point we just reference coolLib however the coolLib library dictates. RequireJS is just loading the file and, once it’s loaded, calls our callback function. Nothing special.

Loading Templates

Back in JavaScript Architecture: Underscore.js, we discussed how to break our HTML into templates to prevent it from mingling too closely with our JavaScript. In the examples we used, we put our HTML templates within script tags. I then posed the question, “You might rightfully be wondering how this is scalable if you start to have hundreds of templates in your html file or how you might be able to load them asynchronously only when needed.” This is also where RequireJS helps out.

RequireJS has a nice plugin called text!. You might think I’m really excited about it because I put an exclamation point next to it. While I do admit it’s pretty south-beach sexy, really that’s just the nomenclature of RequireJS plugins because you type something like “text!myTemplate” when using them. The text! plugin allows us to load in files of text–in our case, HTML templates. Let’s see how this works.

First we’ll download and place the text! plugin file in js/libs.

We’ll place book.tpl.html located under a templates directory that’s a sibling to our js directory:

<span class="label">Title:</span><span class="value"><%= title %></span><br/>
<span class="label">Author:</span><span class="value"><%= author %></span><br/>
<span class="label">Genre:</span><span class="value"><%= genre %></span>

Add some paths to our RequireJS configuration in our main.js to make it easier to use in our modules:

require.config({
	'paths': {
		'jquery': 'libs/jquery',
		'backbone': 'libs/backbone',
		'underscore': 'libs/underscore',
		'templates': '../templates',
		'text': 'libs/text'
	}
});

Then use the template in book-view.js:

define([
	'backbone',
	'underscore',
	'text!templates/book.tpl.html'
], function(Backbone, _, template) {
	return Backbone.View.extend({
		_template: _.template(template),
 
		initialize: function() {
			this.render();
		},
 
		render: function() {
			this.$el.html(this._template(this.model.toJSON()));
		}
	});
});

Notice that the template argument is a string variable holding the exact html as found in our template html file. Then, we use underscore to convert it to a compiled template. We run our book model attributes through the template on render(). Finally, we update our element’s inner html with the result.

So what about all those potential HTTP requests? Here’s where we optimize…

Optimization

We’ve broken down all our JavaScript and HTML into granular pieces. Now we potentially have hundreds of files and unless we do some optimization we would be making hundreds of HTTP requests. Fortunately, we can leverage the RequireJS optimizer.

Generally, you set up the optimizer to run on your server before deploying code. The RequireJS optimizer takes your app files, minifies them (shortens your code to make the files really small), and then concatenates them (smashes them together to make a single file). In the end, you end up with a single file that contains both your JavaScript and your HTML templates. After index.html loads RequireJS, it will load our main.js file. Our main.js file, this time, will not only contain our expected main.js code but all the minified, concatenated code of the rest of our app–both JavaScript and HTML templates. Everything in the file will register itself with RequireJS. When main starts asking for dependencies and those dependencies start asking for dependencies, RequireJS will recognize that all those modules have already been loaded and forego loading them again.

The optimizer has a smattering of options. You can optimize your app down to a few different files representing sections of your app instead of a single large file. You can also use different minifier libraries, exclude files from concatenation, or even minify CSS.

Learn More

We’ve covered a lot of content but there’s plenty more to learn about dependency management. I’ve written a separate article on the Adobe Developer Connection that might catch your interest. Also, the RequireJS website is a great place to start. If you’re interested in seeing more examples of Backbone + RequireJS, dig into this Todos App example. And, as always, feel free to provide feedback or ask questions below!

<< JavaScript Architecture: Backbone.js Routers

Tags: , , , , , ,


Comments

03.23.2012 / Dakota Reier said:

This is a terrific series and is really intentional about reinforcing best practices! Thank you for covering backbone.js in such detail. Obviously business requirements will complicate decisions, but I think you’ve identified some key candidates for architecting and developing a client side app.

I would love to see this series branch out to cover other client-side topics like preprocessors for CSS (like LESS and SASS), JS (although the require.js module cs! probably does the trick here) and even HTML (like slim and haml or more specifically haml-js). The require.js optimization requires a build step, so it seems preferable to find ways to consolidate the different types of compilation required under the scope of require js — and they’ve started to explore this with CSS minification (but they need a less! module like their cs! module).

Obviously you’ve focused on JS architecture here, but I think these tangent topics will pop up again and again for any JS-based web app, and I think there will always be advantages to using a DSL to make up for some of the shortcomings of the base language (HTML, CSS, JS). Each of these candidates (esp. LESS, Coffeescript, Haml) is especially appealing to me because they play nicely with JS (aka. have good JS-based converters), and so offer the potential to leverage a totally JS-based compilation step, resulting in having the native languages rendered and cached. I need to explore require js some more to see how the cs! module is designed and whether there’s potential for a less! and/or haml! module to allow these types of files to be pulled into backbone views and precompiled/cached in conjunction with the require js build step using the individual compilers that already exist (in JS code). Since require js is the part of the application that is low-level enough to understand all of the dependencies, it makes sense in my mind to leverage require js for as much of the preprocessing/compilation/caching as possible.

Anyway, I guess that turned into a ramble, but hopefully it will inspire you to continue writing about client side topics, now that you’ve covered the basics of backbone js.

Do you think the trend toward transplanting everything into JS (or atleast pushing it through a JS-ish filter) is faddish or reasonable?

Cheers!

03.28.2012 / Trouble said:

That’s what im talking about! just briliant! Thanks @Aaron to spread this awesomeness!

@Dakota Reier well i think the most difficult now is like you said to realize a Business Requirements with Backbone… there is Backbone-Relational but really nothing about a good Tutorials for it(search google)… Yeah dont forget that an Article about the Backend(nodeJS + requireJS + backboneJS) can also be awesome… but that’s all Wishes! let see what @Aaron got next for us :D

@Dakota Reier for me i’m loving the way JS now takeover! WinRT for example!! JS has good days ahead! :D

Cheers!

03.28.2012 / Aaron Hardy said:

Thanks for your feedback!

Dakota, I’d love to cover those parts of JavaScript. So many things…so little time. We’ll see what floats to the top of the priority stack.

I wouldn’t go so far as to say transplanting everything to JavaScript is reasonable, but transplanting a vast majority of “online apps” to JS is reasonable given the convergence of browser improvements and the need for cross-device, multi-resolution support. It used to be satisfactory to consider publishing an app as a “desktop browser version” and/or a separate “desktop installed version” and calling it good. Now we’re dealing with desktops, laptops, tablets, smartphones, televisions, and everything in-between–each with their own batch of languages and resolutions. Rather than trying to target each one using separate code bases, it seems to behoove us to target a common denominator–HTML/CSS/JavaScript using responsive layouts. Of course, when you have something as popular as iOS and companies willing to drop serious cash on their mobile efforts, we’ll still see some one-offs for a specific OS or device. Beyond that, we also have specific niche items like video and gaming where HTML and JavaScript just don’t cut it at the moment beyond the basics.

That said, there are a lot of JS fanboys for fanboy’s sake. I find too much fanaticism prevents us from viewing weaknesses objectively and driving advancements. JavaScript and browsers still have a long, long way to go. Their success, in my view, is more based on their history, pervasiveness, and the industry environment (what I described) than the actual quality of the language and browsers.

08.21.2012 / Backbone tutorials - Adriel Blog | Adriel Blog said:

[...] JavaScript Architecture: RequireJS Dependency Management Be Sociable, Share! TweetNo TweetBacks yet. (Be the first to Tweet this post) Categories: html5 Tags: HTML5 Comments (0) Trackbacks (0) Leave a comment Trackback [...]

12.02.2012 / Modularisation with Require.js « watsonbs said:

[...] is well covered ground. Smarter people with more experience working with it have got the basics at [...]

01.07.2013 / Re-Learning Backbone.js – Require.js and AMD | BarDev said:

[...] Aaron Hardy – JavaScript Architecture: RequireJS Dependency Management [...]

01.24.2013 / Kris said:

Hi Aron, nice writeup, well explained. I wish you would include testing in this context.
I chose Backbone and RequireJS for the reasons you mention here, since my app will be really large and needs good structure, modularization and therefore dependency mgmt. The app is coming along fine and I decided its time to start setting up a test framework, thinking it should be really easy to mock out any dependencies since I’m using require… But after a week of trying every solution I can find I’m starting to despair. This is obviously an area that is in dire need of examples!

02.11.2013 / Aaron Hardy said:

Hi Kris. Thanks for stopping by. There are some nice libraries for helping mock dependencies. Check out these if you haven’t already:

https://github.com/mattfysh/testr.js
https://github.com/jrburke/requirejs/wiki/Test-frameworks

Those should help you out. I’d be curious to know if you still run into problems.

03.18.2013 / Kunwar Sangram Singh said:

Recently got interested in web app development and starting exploring frameworks and declarative ways of glueing javascript and other modular components.
Googled for hrs read blogs but most of the post lead to further confusion and explaining kool functions then actual pattern.
Finally found this series. Thanks A TON !! :)

11.14.2013 / Nir said:

One of the best RequireJS articles that I havefound on the internet.
Good job!
Can you provide an explanation or an example of multi page web application (where there is no single main.js module), please?

Thanks!

11.21.2013 / Tutorial RequireJS and PhoneGap | Fovea said:

[…] I took the liberty to include ideas and references from this interesting series of blog posts about javascript applications architecture by Aaron Hardy, more specifically this post. […]

12.22.2013 / Jose Lopez said:

“… south-beach sexy …” just threw me off course.. love it. Nice article.

10.14.2014 / Aalok said:

Hello Aaron! Thanks for the article, it helped a lot.
I think there’s a mistake in the example where you demonstrated how to load a single non-AMD JavaScript library file with no dependencies and no shim.

It should be
require([
'js/libs/coolLib.js'
], function() {
console.log(window.coolLib);
});

instead of

define([……..

correct? And the particular code snippet should be written in main.js, yes?

10.15.2014 / Aaron Hardy said:

Yeah, it should be require since we’re not defining a module in this particular example. I’ve fixed it. Thanks!

This doesn’t have to be written in main.js. That’s probably where it would end up though in order to maintain a single place from which third-party libraries are loaded. Good question.


Leave a Comment

Your email address is required but will not be published.




Comment