View slides as PDF

Array/Object/Function manipulation

As we create software is seems like we come in contact with the same patterns over and over. Usually, we end up re-writing them over and over as well. Take an array of user objects, each with a username property. We need an array of all the usernames from all the objects. So, like many times before, we create a new array to populate, create a for loop, snag the object at the current index, grab the username and push it into the array.

To me, that’s boring. It’s mundane. Underscore makes it fun again. With Underscore, we just use pluck():

var usernames = _.pluck(users, 'username');

Ah…concise, fast, and boilerplate is gone. Want to find all objects within an array that pass a specific test? Use the filter() function. Just want a reference to the first one that passes the test? Use the find() function. Want to retrieve the union of two arrays, that is, retrieve a single array of all unique objects contained within multiple other arrays? Try the union() function. Merge properties of multiple objects into a single object? Use extend().

Once you grasp the power of Underscore you’ll find yourself being more productive with less code while having more fun. Some have called it the bowtie for jQuery’s tux. I concur.

Templating

The template() function is only one of the many listed for Underscore, but its purpose is very important.

Let’s say you’re building an app to manage your todo items. When the user creates a todo, it’s your job to create a todo row for the view. And maybe you want to allow them to select the row using a checkbox, delete the row, or edit the row in-line. Without using a templating engine, you might end up with something like this:

var todoTemplate = '' +
	'<div class="todo ' + (done ? 'done' : '') + '">' +
		'<div class="display">' +
			'<input class="check" type="checkbox" ' + (done ? 'checked="checked"' : '') + ' />' +
			'<div class="todo-text"></div>' +
			'<span class="todo-destroy"></span>' +
		'</div>' +
		'<div class="edit">' +
			'<input class="todo-input" type="text" value="" />' +
		'</div>' +
	'</div>';

This is troublesome. First, your html is likely buried somewhere inside your JavaScript code. This makes re-skinning an app much harder since you now have to track down all your html strings scattered throughout your logic. Second, concatenation takes time, is risky, and falls under the categories of mundane and frustrating. You have to deal with escaping and indentation. Your IDE probably isn’t going to color-code, code-hint, or auto-indent as well as it could if it were plain html. It’s easy to make a mistake and, since JavaScript is loosely typed, you might not find these mistakes until runtime. Also, if you dish your code off to HTML/CSS gurus, they now have to deal with these problems too.

Templating engines provide a better way of managing the situation. What if we added this at the bottom of our body tag instead:

<script type="text/template" id="item-template">
	<div class="todo <%= done ? 'done' : '' %>">
		<div class="display">
			<input class="check" type="checkbox" <%= done ? 'checked="checked"' : '' %> />
			<div class="todo-text"></div>
			<span class="todo-destroy"></span>
		</div>
		<div class="edit">
			<input class="todo-input" type="text" value="" />
		</div>
	</div>
</script>

There are a few things to note here.

• The code is plain HTML except for those things that can be dynamic which are inline (<%=...%>).
• The dynamic portions reference a variable called done. We’ll talk about that in a minute.
• The code is wrapped in a script tag with a type of text/template meaning it won’t be interpreted as JavaScript (the default script type) by the browser. If it were, the browser would throw errors.
• The script tag has an ID which allows us to later access its content.

At this point, the code inside the script tag is just innate text. The browser won’t try to execute it or anything of that nature. That’s where Underscore’s templating engine comes in. In our app JavaScript code, we can then take action like this:

var template = _.template($('#item-template').html());

The $('#item-template') code uses jQuery to reference the script element. Calling html() on the element then returns the inner html of the element–in this case the html that will represent a todo item. Here comes the underscore templating engine: _.template(). At this point a compiled template has been built. This allows us to now pass a context object into the compiled template to produce html we can then use for our todo item. This might look like so:

var todoHTML = template({done: true});

The portions of the template that depend on the done variable will be evaluated using the done property on the context object. The resulting todoHTML will be a string of html that looks like:

<div class="todo done">
	<div class="display">
		<input class="check" type="checkbox" checked="checked" />
		<div class="todo-text"></div>
		<span class="todo-destroy"></span>
	</div>
	<div class="edit">
		<input class="todo-input" type="text" value="" />
	</div>
</div>

You can then use jQuery to create an element out of the html. It’s as easy as

var element = $(todoHTML);

I should mention that a portion of this code is pulled from Backbone.js‘s Todos example.

Template management

You might rightfully be wondering how this is scalable if you start to have hundreds of templates in your html file or how you might be able to load them asynchronously only when needed. If you’ve asked yourself these things, congratulations! These are questions architects should be asking. We’ll talk about how to manage this in a later post.

Templating syntax

Underscore by default uses ERB-style template syntax (<%=...%>) which to me feels nasty and isn’t fun to type. You can change that to some other style like {{...}} as follows:

_.templateSettings = {
  interpolate : /\{\{(.+?)\}\}/g
};

Much better.

Other templating engines

The concept of a templating engine is not unique to Underscore or even JavaScript. There are many others in many languages. I feel the most notable ones in JavaScript outside of Underscore are Mustache.js and Handlebars.js. I use Underscore because it’s a top-notch templating engine, includes other Underscore functionality I use anyway, and it’s a dependency of Backbone.js which we’ll be discussing in a later post.

Where to use templates

After reading this, you may think templates are only appropriate for elements whose count is undetermined until runtime like our todo items. If so, may I suggest considering that everything should be a template?

What if you took everything in your body tag, broke it up into reasonably small templates, and left your body tag with nothing in it except for your templates? What if each DOM element and its contents produced from each small template were managed by a single JavaScript object? What kind of freedom would that bring your code? Could you move these components around more freely? Ship them over to be used in a different app? Would they be less coupled from one another and provide stronger APIs? Could you more appropriately add, remove, re-arrange, and manipulate these components rather than having a big fat HTML file written up front? Could you provide your users an all-around more dynamic experience? I hope you catch the vision. In upcoming posts we’ll see how this takes shape.

Justin Meyer, the main guy behind JavaScriptMVC, said something I feel is a very simple principle every architect should ingrain into their brain:

The secret to building large apps is NEVER build large apps. Break up your applications into small pieces. Then, assemble those testable, bite-sized pieces into your big application.

I don’t think any other principle will carry more weight in architecture–especially with JavaScript. You may be thinking, “Oh, my application isn’t big enough to follow this principle.” Re-think this. Every application I’ve architected–even the smallest of the small–have benefited from this principle. The pieces of your application must be decoupled and cohesive as much as humanly possible to withstand the test of time. Each piece should be as black-box as possible–pass information over the wall and the next component does its job. And I don’t just mean different views of your app; I also mean individual, small components of your views, models, and everything you build. This presentation on Scalable JavaScript Application Architecture does a great job of explaining these principles in more detail.

When you think of JavaScript apps you may think of a few gigantic, spaghetti JavaScript files and a few beefy HTML files and a mess throughout. You might even think of fat strings of html text hand-concanated, escaped, and declared right within your JavaScript. Escape this mindset. That, indeed, is a rightful stigma of JavaScript apps because so many exist that fall into this category. We’re not going to be following that trend. We want to favor many small classes, files, and HTML templates over a few large ones. Small classes lead to small scope. Small scope leads to better maintainability and easier debugging because there is less your mind needs to comprehend. It also encourages strong, well-planned APIs and low coupling.

You may be thinking, “But JavaScript doesn’t have namespaces and everything’s asynchronously loaded and depends on everything else! And what about all the requests to the server? And how do I get all those HTML templates into my app? This is going to be a nightmare!” Don’t worry about it. We’ll get there. Don’t fall into the trap of making excuses for poor design just because you’re in JavaScript.

Admit what you don’t know

The only thing that’s guaranteed not to change is change. Therefore, never act like you know how your app will grow and change. Nicholas Zakas said it well:

The key is to acknowledge from the start that you have no idea how this will grow. When you accept that you don’t know everything, you begin to design the system defensively.

Naming conventions

There are many conventions in JavaScript–some more popular than the others. I find it important to maintain consistency with most JavaScript code out there. Unfortunately, I haven’t found empirical evidence showing which conventions are most popular, but I have done some rudimentary research and will propose a few seemingly most-popular conventions. Again, these are only suggestions. The most important point is that you agree on a standard as a team and stay consistent.

• Names of directories and files should be all lowercase. Add hyphens between words.
• Variable names should be camelCase.
• Objects (functions) meant to be instantiated should start with a capital letter.
• Private scope doesn’t exist in JavaScript except through closures. See the poorly-named module pattern for more info but I don’t really love the consequences of the pattern. Instead, I suggest prefixing names of members meant to be private with an underscore. It doesn’t provide true privacy, but it’s a pretty solid red flag that you shouldn’t use it publicly.

Testing

Strongly-typed languages (e.g., Java, C#, ActionScript 3) specify the type of each variable throughout the application. This means the compiler can raise red flags when you try to use an object in a manner that wasn’t intended by its type. In essence, strong typing can be considered to be a bunch of built-in, simple unit tests that are run each time the app is compiled.

JavaScript is not a strongly-typed language but instead is a very loosely-typed language. It’s much easier to make mistakes by using objects in ways that weren’t intended or just fat-fingering a variable name and you won’t find out about it until you run the code and see the error (a good IDE can also help). For this reason, I believe testing is even more important in loosely-typed languages.

There are many JS libraries available to help you test your code. QUnit seems to work well and is popular. If you don’t already have a favorite, I suggest to start there and research outward.

JsTestDriver can also be very useful (and happens to have a QUnit adapter). JsTestDriver allows you to run your tests within a variety of browsers. This can be very helpful since browsers aren’t consistent in their implementation. For example, I might write code using myArray.indexOf() which will work on most browsers but not older Internet Explorers. By running your tests within actual browsers these issues will crop up and you’ll find out about them before your users do.

Strict mode

The 5th version of ECMAScript (the standard upon which JavaScript is based) provides an option with which you can tell browsers to be more strict with your code. Newer browsers will respect it while older browser will stay lenient. Why would you want browsers to be more strict with your code? Because you want to decrapify your code. Strict mode provides the following (copied from MDN):

• Strict mode eliminates some JavaScript pitfalls that didn’t cause errors by changing them to produce errors.
• Strict mode fixes mistakes that make it difficult for JavaScript engines to perform optimizations: strict mode code can sometimes be made to run faster than identical code that’s not strict mode (Firefox 4 generally hasn’t optimized strict mode yet, but subsequent versions will).
• Strict mode prohibits some syntax likely to be defined in future versions of ECMAScript.

I highly recommend invoking strict mode to keep your code squeaky clean.

Code Analysis Tools

A couple popular code analysis tools exist to likewise help decrapify your code. They scan your code and raise flags when they see you’ve done something dangerous or undesirable. They mainly focus on syntax, style, and structure. They do not verify that your logic is correct.

The first is JSLint. It’s “the original”. The second is JSHint. JSHint explains the difference:

JSHint is a fork of JSLint, the tool written and maintained by Douglas Crockford.

The project originally started as an effort to make a more configurable version of JSLint—the one that doesn’t enforce one particular coding style on its users—but then transformed into a separate static analysis tool with its own goals and ideals.

Both tools have online versions where you can copy-paste a snippet into a textarea and have the code analysis do its thing. This works okay for bits of code here and there, but even better is setting up the analysis code to run on your full app. Both tools have integrations to you allow you to analyze your app’s code from your IDE or command line. On our team, we run the tool from an Ant script that we can then run from the command line, our IDE, or our deployment servers. I highly recommend you run these tools often right off the bat. Note that each have flags you can set to be more or less strict in its analysis.

For the past several years I’ve been an architect in enterprise-level RIAs. This is a fancy way of saying I oversee the design and construction of apps that are web-based but have a lot of the same characteristics as desktop applications. Some of the applications I work on are in fact desktop applications but heavily communicate with the web. The line becomes very blurry, but the main point is–I deal with applications which in my world are quite different than what is often considered a “website” even though they both live within a browser.

There are others in the industry who do what I do or would like to do what I do. Of those there are some who are new to JavaScript. This is my intended audience. An architect arriving at JavaScript for the first time has a lot to learn and many decisions to make. As the lead architect on a product and being fairly new to JavaScript (well, modern JavaScript) I likewise had to go through this process and have spent countless hours re-learning JavaScript, testing IDEs, discovering communities, exploring deployment procedures, learning what questions to ask, evaluating libraries (mvc, dependency management, dom manipulation, testing, deployment, utilities) and putting all these things together to lead a team and produce an application worthy of enterprise consumption. My hope is to help others along the way.

You might think this is a broad effort, but my goal is not to simply regurgitate everything I’ve learned but to help boil things down to what’s important to get you on your way. We’ll discuss core principles that are common in dynamic apps regardless of the languages or libraries used. However, to illustrate them and hopefully give you enough of a concrete example to help you hit the ground running, I’ll be using specific, hand-picked libraries for this series. In your own research you may find other libraries are more appropriate for your particular project. That’s fine; the principles in this series are still valuable and applicable.

Choose wisely

JavaScript is merely a language. HTML is, for the most part, a set of dilapidated, relatively primitive components. To reduce boilerplate, abstract browser inconsistencies, and provide engineers with higher-level tools to be more productive, all kinds of libraries have come out of the woodwork. Because JavaScript and HTML leave out some basic structures and components that are so commonly needed, these libraries often overlap in functionality and features. This also means some don’t play well together. This is both exciting because you have so many options to choose from and yet frustrating because you have so many options to choose from.

As an architect, specifically in an enterprise setting, these decisions will be very important and can ultimately determine the fate of your product. If I were farting around on a side-project, I’d probably feel comfortable picking the most awesome-sounding libraries and running with them. In the corporate world though, we have to answer some important questions: Is our product part of a larger suite? If so, what is the rest of the suite using? If the rest of the suite were to standardize on a particular library, how hard would it be for us to refactor? What if we chose a “full-stack” library and a portion of it is or becomes inferior to other options? What’s the chance that a different library may overtake my library in community and technical progression? What’s the community like around those libraries? Which libraries are gaining steam or fizzling out? What’s their learning curve? How easy is it to hire developers for these libraries? It’s a very different ballgame so take care in your decisions. In my case, I found the best option was to pick a handful of high-quality tools with healthy communities that worked well together but were decoupled in their implementations. Grooveshark provides a very interesting related case study.

Rolling your own

There comes a time in every developer’s life where we get caught up in rolling our own libraries. Maybe we think our work is superior, maybe we don’t do enough research on what’s already available, or maybe there’s just one small thing that isn’t quite right in a library so we rewrite it from scratch. I’ve done it and you likely have too. When determining what your team will use, it’s very important to keep in mind some of the significant benefits of going with a wisely-chosen 3rd-party/open-source library:

• Free development resources
• Free documentation
• Free bug fixes
• Free testing by automated tests and other community members
• Collaboration amongst generally intelligent industry leaders
• A place to learn more and ask questions

Far too often I’ve seen very intelligent architects roll their own libraries because they feel their skills are superior or are OCD that the libraries work exactly like they want and carries no extra baggage. Very often, if that architect were to stay on the project forever, this would probably work out quite well and the product would be successful. In reality, the architect leaves. Almost always they were time-crunched and didn’t fully document the library, set up testing, or cross-train their co-workers. They developed the library for the purpose of fulfilling the specific needs of a project and even though the library may have been a fantastic work of art, the engineers left with that work of art don’t understand its intended usage or how to make modifications. From there, the engineers begin using it incorrectly and if they have questions they have nobody to ask and nowhere to look except for undocumented, increasingly morbid code. Inevitably, the product starts its death-spin, the engineers blame the architect that left, and technical debt grows until the product declares bankruptcy. At this point it’s time to rebuild or sink.

Truthfully, most of the fault really does fall on the architect. The role of the architect is not only to code awesomeness, but also to understand the software circle of life and make wise decisions.

Don’t get me wrong–there is a time and place for rolling our own libraries. If you come across this time and place, make sure you have the necessary resources behind it. Do it right. Where possible, create the libraries as though other companies might also use them. Document them–not only in code but in API documents, examples, and tutorials. Open-source them on GitHub and make it a home for external resources to congregate around and contribute. Pay attention to others’ contributions and incorporate their ideas and code where possible. Many if not most of the great libraries available today originated from this concerted effort.

One last note about open-source libraries. The historical perception is that these libraries are siloed away in some remote part of the web and their developers are anonymous and unapproachable. This usually isn’t the case anymore–especially with GitHub. Oftentimes if you have a legitimate concern the community is very approachable and will respond quickly and respectfully. A great example of this is when I recently found that underscore.js lacked a function for finding the symmetric difference between multiple arrays. I provided the underscore.js community an example of where it would be beneficial and five days later it was added to the trunk with a unit test and documentation. Good libraries have good communities. Choose wisely.

We don’t need no stinkin’ frameworks!

A long-standing debate among some in the community is whether formal frameworks are needed at all. The argument against frameworks usually goes something like, “We’re all smart engineers–we can architect an app well without having to resort to a formal framework.” That may be true, but it’s usually a fallacy. If it’s not a fallacy, it’s very difficult to do with most teams. If you are able to successfully execute on that concept, you very likely unconsciously re-wrote large portions of a framework you could have used in the first place.

When working in a team, a framework provides an important benefit of consistency and a game plan. Good frameworks generally prescribe examples of best-practice usage but provide some flexibility in implementation. No doubt engineers can abuse the best of frameworks, but having a blueprint of how architecture and implementation should, in general, be performed goes a long way in getting everyone on the team to have a consistent vision and understanding.

Learn JavaScript

Learning JavaScript is not a one-time event. You’ll be learning over a long period of time, but it’s important that you actually make it a priority to learn the language. A lot of the libraries we’ll be looking at do a lot of abstracting for you–keeping you away from the nitty-gritty of JavaScript. That’s very helpful and can speed up your development dramatically, but it should not be used as a crutch. Far too often people learn libraries but not the language. This is a mistake and will bite you in the end.

I’ll be the first to admit that JavaScript has some really nasty parts. Frankly, it’s not my favorite language. But you’ll be living in it. Accept it and learn how to take advantage of its good parts and avoid its bad parts. To that end, I recommend starting with Douglas Crockford’s JavaScript: The Good Parts. Ironically, it’s a small book and very approachable by JavaScript noobs.

IDE

Get yourself a quality development environment. If we’re talking web we’re primarily talking HTML, JavaScript, and CSS. There are all kinds of free web-oriented IDEs (Aptana comes to mind) but I recommend you pick up a copy of WebStorm by JetBrains. A good IDE will give you great JavaScript code-hinting to shield you from some of the woes that come with the dynamically-typed language. Other things I look for: coding workflow, local and remote debugging, version control integration, remote server deployment, plugins, etc.

Have comments or questions? Post them below!

For those who don’t know, Nimbus is a Flex MVC framework (or micro-architecture for the technically-inclined). Sponsored by Rain, we develop it primarily for use on our applications but make it available for public use and encourage the community to contribute.

Nimbus pulls core concepts from Cairngorm but is meant to cut out a lot of the plumbing developers groan about when they hear “Cairngorm.” It’s light, but it’s not fluffy. Baked in are those rare and tasty flavor morsels you thought only existed in those Funfetti cupcakes your grandmother bakes with love on your birthday.

Until the end of 2008, Nimbus had been shaped, re-shaped, patched, and plugged. A complete re-write was merited so that’s what we did. And really, what makes for a better Christmas than sitting down with your stocking stuffed with jelly beans and re-writing an MVC framework. A few months later, the code is settling down along with the sugar and we’re ready for action.

Here are some features of Nimbus:

• Notification system of command completion so portions of the application can know when a command has finished executing.
• Nestable “macro” commands. Think “I want commands A and B to run in parallel, then C, then D and E in parallel, but inside D I want F and G to run sequentially…”
• Undo/Redo functionality.
• Progress feedback system. You as a developer can associate a processing message with each command or macro command to provide feedback to users.
• Interaction blocking system. While commands are executing, easily disable user interaction to protect your service calls.
• Combinable commands. Ensure that similar command instances can be grouped appropriately in the undo/redo history.

Nimbus may not be right for everyone. Call your system administrator if your scalability problem worsens or is not better within 7 to 10 days. This may mean that there is another condition causing your RTE problems. Do not take Nimbus with alcohol. Call your system administrator right away if after taking Nimbus you are unable to walk, drive, eat, or engage in other activities. In rare cases, severe allergic reactions can occur. Most MVC frameworks carry some risk of dependency. Do not use MVC frameworks for extended periods without first talking to your system administrator. You are encouraged to report negative side effects of Nimbus. Visit http://code.google.com/p/nimbus-as3 for more information.