Work on Ben McCormick

Coding is deciding

Fri, 22 Aug 2025 21:09:00 -0400

We often talk about computer programs purely as collections of “code”. Lately I’ve been thinking about it a bit differently: a computer program is the encoding of a huge series of nested decisions, made intentionally or otherwise.

Let’s say we’ve decided to build a calculator app. That overall goal is the first decision, but there are many more decisions nested underneath it.

What features does our calculator app need? (Just basic arithmetic or are we buliding a graphing calculator?)
What platform are we building for? (web, mobile, desktop, VR?)

Once we know generally what we’re trying to do, there’s still a bunch of fundamental decisions to be made before we write any code

Is this a client only app or is there a server component?
What framework should we use?
How will the app be hosted and distributed?

Then once we start actually writing code, we find even more nested decisions.

How are we going to structure our files?
How will we break functionality up into components, classes and/or functions?
Is this a single file, a single app, or some sort of distributed system?

And then even within a given class we nest even more decisions

What functionality should the class have?
What is the high level logic of each function?

And then we come down to the lowest level of decisions

What syntax will we use for that? (For example: for loops or iterators to run through a list)
Will we use a library or write some functionality from scratch?
Tabs or spaces? Single quotes or double quotes?

We can think of these as functional, infrastructure, architecture, logical and syntax level decisions.

The history of computing has mostly involved changing which of those decisions a software developer needs to make as tools have changed. Many of those changes have removed the need to make decisions in favor of a generally acceptable standard implementation. The vast majority of programmers no longer worry about the details of the bytecode or assembly definitions of programs — those have been standardized by higher level languages. Most modern languages also remove or significantly reduce decision making around memory management, cloud platforms reduce¹ the decisions many devs need to make around infrastructure for their apps, and API frameworks like Django or Ruby on Rails reduce the decisions somebody needs to make about how to structure their API code.

Each of these abstractions have allowed developers to focus more on the decisions around what their program should do and less on the implementation details of how it does it. They’re also all leaky abstractions to varying degrees, and there are times and situations where a developer will need to go without or work around the standard tool to make a different decision.

Applying this model to think about LLM based coding tools, it’s useful to talk about the use of LLM in terms of what decisions we’re making vs offloading to the coding tool. LLMs are essentially tools for replacing decisions that you might have had to make while generating text with a reasonable standardized option. However the way we use them can lead to different sets of decisions being made by the tools.

Vibe Coding

Vibe coding, the buzzword of the year in tech, is commonly understood to mean that you’re creating an application solely through prompting and iterating based on the observed output without reading the code. This is effectively choosing to make decisions at the functional level (and maybe the infra level) and let AI handle the rest.

Tab Complete

We can contrast this to a tab complete interface like those provided by Github Copilot or Cursor, where an IDE will offer to insert a chunk of code based on context. Here AI is mostly deciding on syntax, with more ambitious implementations like Cursor possibly moving up to the logical level

Feature Development With Agents

There’s a lot of room between these 2 extremes. Tools like Claude code, Cursor and Copilot can take on tasks at a variety of levels depending on how they’re prompted. You can ask a coding agent to generate a new feature for instance, which might live completely at the logical / syntax level or could end up infringing on architecture if the feature has new architectural considerations or the coding agent introduces a new pattern or dependency.

Who decides what?

As I’m personally moving back into more of a heavy coding role this month, I find this framework of who is making decisions helpful.

While building enterprise software at work, I’m ultimately responsible for the quality of all the code I write, so while I find it helpful to move faster by using AI tools to get a rough solution in place, I encode some decisions through the prompts I give and end up reviewing and tweaking everything to make sure what comes out of PRs I open represents my decisions. I’m also quickly building an intuition for which tasks a coding agent is likely to produce a useful output for that I need to tweak, vs one where I’m unlikely enough to agree with the decisions that its faster to just write it myself².

On the other hand if I want to experiment with something quick and dirty or prototype a concept, the freedom these tools give to ignore every level below the functional one is wonderful (I just don’t want to maintain such a system).

Working with these tools has reminded me once again that reality has a surprising amount of detail. There is no magic in the world and decisions are always going to have to be made. If coding tools help us to go faster by focusing more on the decisions that really matter, I’m excited to see what they help us bulid.

Ok given the huge ecosystem of choices and configuration in cloud computing its probably more accurate to say that it changes the decisions devs need to make ↩︎
Given the rapid evolution of the tools of course, this is something that needs to be checked over time. Claude Sonnet can do things reliably that Chat GPT3.5 would consistently fail on, and the agentic system seems to have a good deal of impact on quality as well. ↩︎

Retraining our Brains To Review LLM-augmented work

Tue, 22 Jul 2025 10:05:07 -0400

Something I worry about with generative AI in business and commercial use: almost no one fully reads anything in those environments.

Now imagine when even the author hasn’t read what was written… yikes. How does AI writing and reading impact this reality?

Steven Sinofski has an interesting post here about “slop” at work. I think about this in terms of heuristics. Our brains are lazy and we all naturally develop “shortcuts” to evaluating a piece of work. Historically seeing a PRD or tech spec that was fully fleshed out with diagrams and clear explanations of context was a good indication that some serious work and discovery had gone into the document. A PR full of detailed test cases indicated attention to detail, and that thought had gone into all the possible edge cases. But with LLMs it is easy to take a few half thought through bullet points and make them look credible, or generate plausible test cases at scale.

We’re going to need to retrain our brains.

A Simple Path To Building Trust

Tue, 25 Jun 2024 14:22:10 -0400

Being trusted is a superpower at work and in life. When we’re trusted things move faster. Communication is more open, people become more cooperative, and work becomes more enjoyable as relationships can grow deeper.

I’ve found that there’s a very simple¹ formula to building people’s trust.

Here’s the formula:

Identify a need
Say you will meet the need. Say when you’ll do it by and with what quality.
Go and do it the way you said you would
Tell people you did it
Repeat

That’s the whole process. Do that enough as an individual and you will develop a reputation for reliability and trust. You’ll find that people tell you things and give you more opportunities. If your team does that you’ll find that other teams become easier to work with.

A few notes:

Misses count more than makes

Fairly or unfairly, one missed commitment is more memorable than 4 that were completed on time. If you’re operating at a trust deficit it will take time to rebuild trust, but the process is the same.

When you do miss, always try to miss early and clearly. Nothing erodes trust like discovering that somebody has been hiding failure.

Sometimes you’ll discover that while you don’t have to miss, your commitment no longer makes sense after making it: priorities have changed, new information has emerged, or another opportunity has presented itself. You can choose to make changes in this situation. If you communicate the change and your reasoning early to the people you’ve committed to², this probably won’t be counted as a true miss. However it probably won’t build trust either, and if you’re regularly doing this you may create the impression that no plans can be relied upon if you’re involved. The tradeoffs here will vary based on context.

People can hold you to implicit expectations

You can lose trust if you share a secret, even if you never explicitly promised not to tell anybody the secret. You may lose trust at work if you don’t meet some standard of quality that other team members are hitting, even if the expectation wasn’t explicitly stated. This is also why it’s useful to say when and at level of quality you’re committing to something - if you think you’re committing to a rough draft of a tech doc by next week but your manager is expecting a polished doc to be done by Friday, you’ll lose trust even if you hit what you think you’re committing to. Since misses count more than make its important to understand and clarify any implicit expectations you might be held to

People trust kindness

When you’re kind to others they want to trust you and are more likely to overlook or forgive misses. Jerks fight an uphill battle to earning others' trust.

As always, simple does not mean easy. ↩︎
For work tasks it may not always be clear who is counting on your commitments: other departments might have been promised this work by somebody else, or be relying on another project that is dependent on your work. Be careful here. ↩︎

Setting Expectations: 4 key areas for performance management

Sun, 12 Feb 2023 17:00:00 -0400

Performance management processes have been different at every company I’ve worked in, ranging from “basically non-existent” at a 15 person startup to “months of work for managers each review period followed by clear expectations that growth feedback will be given throughout the year” in big tech world. Some of these differences are good and bad, most are more about the context of the company. I’ve found that regardless of process there are 4 things that it’s important for a team member to align with their manager on throughout the year. Hopefully there’s process supporting this, but if not both the team member or the manager can take the initiative to make it happen.

Focus - Where should the teammate be putting their energy? Depending on seniority and the level of change within the organization, this might be something to resync on every 6 months, monthly, weekly or even daily[^1].

Baseline Expectations - It’s important to agree not just what you’re trying to do but what results are expected. This means being clear on what a team member is and isn’t responsible for, what success looks like, and giving feedback when the manager believes things are off track or a miss has occurred.

Opportunities - This is about above and beyond. What things aren’t expected but could be an opportunity for growth (to help the team/business, show that the teammate is ready for a promotion, or ideally both)? It’s useful to clearly delineate these from expectations, so that you don’t risk a lack of clarity on what is “good enough”, which can cause unnecessary stress/burnout or poor prioritization.

Risks - I find that this part is usually left implicit, but its helpful to align up front on what likely areas of failure might be, where the teammate and manager see the most risk of missing expectations (or finding out that the expectations weren’t correct!). This could be due to ambiguity, a known skill gap from the team member or those they’re collaborating with, a new type of challenge, or business level needs/expectations that aren’t realistic.

[^1] Long term syncing daily on priorities/focus areas might be micro-managing or a sign that the team member is struggling to manage their own work, but it can be useful in a rapidly changing environment or during onboarding for a time

3 Pillars For Effective Work

Fri, 13 Jan 2023 07:33:53 -0400

There are 3 primary reasons why one human may be more effective than another in a given work situation.

Caring - All things being equal, somebody who cares more about a problem / product will generally be more effective¹. Caring can range from “startup founder” level obsession to being completely checked out, and effectiveness will vary on that spectrum.

Capacity - The amount of time and mental resources a team member can bring to a problem matters and will vary over time. Illness, stress, a busy personal season or competing work priorities can reduce this temporarily for anyone. Some people will have longer term lower capacity than others due to factors like life circumstances, health, and outside of work commitments.

Leverage - People’s skills, experiences, tendencies, relationships and abilities can allow them to be disproportionately effective or ineffective in different situations. Some of this is situational: “he wrote that library and knows all the details”, “she is an expert on this technology”, “he has admin privileges on this system”, “she knows the tech lead on that team”. Other factors like communication ability, broad technical experience, or credibility within an org tend to persist across problems.

When trying to understand why somebody is more or less effective than [their peers | what you expected | your own performance], it is useful to pull out this model². Trying to help somebody who is capacity constrained level up their skills likely won’t move the needle. On the other hand, if you have team members who have the skills but lack capacity or don’t care about their work, there may be an opportunity to help address those underlying challenges and get a better outcome. Similarly if you’re trying to “outwork” your way to a promotion while you see peers who seem to be doing less than you getting better outcomes, its worth looking into what leverage they might have (skills, access, relationships) that are helping them be more efficient.

A caveat: be careful not to prejudge caring vs capacity issues in others. It can be difficult to tell whether somebody seems disengaged because they don’t care, or because they don’t have capacity to care. Best to work through that with them before solutioning.

Caveat – if the level of care for a sub-section of a product/problem/org is disproportionate to the business value, this can become a negative. Think of an employee who spends their time improving line-level code quality of a feature that the team intends to deprecate. ↩︎
I don’t recommend it for hiring however – trying to judge “caring” can tend toward unconscious biases, and most long term capacity issues fall under protected categories / more explicit biases. Better to focus on proven previous effectiveness and testable “leverages” (tech/people skills / knowledge) ↩︎

2022 Books

Tue, 03 Jan 2023 10:47:00 -0400

I read 13 “work related” books in 2022¹. They’re broken up by category below. I didn’t think any of these books were bad, but the ones I would recommend have a star next to them.

Decision Making

⭐ How To Decide by Annie Duke - Annie Duke takes her poker background and applies it to making decisions in other contexts – lots of practical decision making strategies here

⭐ The Scout Mindset by Julia Galef — somewhat similar ground to to “How You Decide”, but more of a focus on viewing the world objectively. I really enjoyed reading these 2 books together.

Productivity

⭐ Make Time by Jake Knapp and John Zeratsky — A nice book full of practical tips for managing your time and reclaiming control from distractions

Product Process

⭐ The Messy Middle by Scott Belsky — The best description of what it’s like working in a “startup land” situation (either a real startup or high change/growth in a larger company) that I’ve read. Plenty of good practical wisdom as well

⭐ Escaping the Build Trap By Melissa Perri — This book lays out a great vision for what a healthy product organization looks like. As a non-product manager, I’ve found it mostly helpful for recognizing when things are going wrong and giving feedback.

⭐ Kill It With Fire by Marianne Bellotti — An engaging review of how to work with legacy systems (the title is ironic) with lots of real world examples and principles that apply even for newer systems.

The Lean Startup by Eric Ries — This book is very well regarded and I don’t remember disagreeing with anything as I went through it about a year ago, but it left no impression on me. Possibly I’d already absorbed the main ideas through other sources?

Soft Skills

The Speed of Trust by Stephen M.R. Covey — A look at how to build trust in business and why its important

Rituals For Virtual Meetings by Kursat Ozenc and Glenn Fajardo — This was more of a workbook, with a ton of examples of possible “rituals” that you could implement during remote meetings. I in theory like this idea, in practice most of these felt too hokey for me to try – but if you’re willing to go full “icebreaker game” for your team meetings, this has good ideas.

The Culture Map by Erin Meyer — Another book that I read a year ago that I don’t feel I’ve retained very well — but the big idea was that business communication and practice varies wildly across cultures and its important to understand the background of the people you work with.

Career

⭐ The Staff Engineer’s Path by Tanya Reilly — My most recently completed book, this was a great exploration of the “advanced IC” career path. I’ll have a longer review coming later, but this was my favorite work related book of the year.

Miscellaneous

Product Led Growth by Wes Bush — This was pretty specific to a work project, but a decent review of the business requirements for moving to a product led growth model rather than sales driven. Useful for folks who are bootstrapping a new SaaS business or trying to move to PLG.

Building For Everyone by Annie Jean-Baptiste — This is a high level overview of accessibility and inclusive design practices by a design leader at Google. I wanted to like this more than I did – a lot of the content here is about changing large organizations to institute more inclusive design practices, I had been hoping for more “local” things that I could learn to get better here.

Going Forward

One of my 2023 goals is to spend more time with fewer books, including taking deeper notes on some books that I’ve read in the past which had interesting ideas that I don’t feel I’ve retained well. In particular I want to revisit Scout Mindset and Escaping The Build Trap from this year and am already writing up notes for Staff Engineer’s path.

You can see non-work related books on my “non-tech” blog ↩︎

7 Months of Lifting At Home

Wed, 28 Dec 2022 15:38:00 -0400

I picked up a new hobby this year ¹! For years I viewed weightlifting as a form of benefial drudgery that I occasionally tolerated but one I have mostly avoided post-college. Then last November I joined a cross-fit style gym and was introduced to Olympic/Powerlifting style barbell lifting. The gym didn’t stick for reasons that had more to do with schedule than anything else, but I pulled my favorite part of it out and have been consistently doing barbell lifting at home since May.

This has been an unexpectedly rewarding experience for me. It’s allowed me to develop a consistent exercise routine with 2 kids in the house (something thats eluded me since Elena was born 4 years ago), reminded me that I can still learn new things at 35, and helped me feel physically the best I’ve felt in a decade. My original goal was to help myself have energy for my kids, and I definitely think I’ve achieved that.

I’ve been running through the Starting Strength program, unmodified for 3 months and then shifting to a 4 day schedule for the last 4 months. In practice this meant primarily focusing on 5 exercises: the Squat, Bench Press, Overhead Press, Deadlift and Power Clean ². Initially the schedule looked like this, alternating between A & B 3 days a week

Workout A		Workout B
Squat	3x5	Squat	3x5
Bench Press	3x5	Press	3x5
Power Clean	3x5	Deadlift	3x5

Doing that much squatting became exhausting around the summer, the workouts became long as I needed longer breaks between sets and I decided I wanted a bit more upper body focus (and better legs for when I was playing basketball and going for walks). So I switched to a 4x a week schedule:

Tuesday		Wednesday		Saturday		Sunday
Squat	3x5	Bench Press	3x5	Deadlift	3x5	Bench Press	3x5
Power Clean	3x5	Press	3x5	Light Squat	3x5	Press	3x5

Those are the core lifts – this doesn’t include warmups (I usually did a Peloton strectch workout and worked my way up to heavy weights) or accessory exercises (I tried to do rows / ab workouts / pushups on days I was feeling good and having time in the 4xweek schedule)

Because I had effectively never trained for any of these lifts in a serious way before, I got to see a lot of relatively easy progress just from staying consistent in lifting. The chart below shows the change in my 5 rep max #s throughout the year (these are training numbers – I haven’t been trying to set up for specific max attempts).

Other than a lag in August / September when I traveled and then got sick, I’ve been able to make fairly consistent progress on all of the lifts throughout the year, with some exceptions where I was reworking my form/technique (for DL in June and Squat/Powerclean recently). Overall I’m really pleased with my progress.

Exercise	Initial 5RM	Best 5RM	% increase
Squat	175	320	82%
Bench Press	135	240	77%
Press	95	145	52%
Deadlift	205	385	87%
Power Clean (3RM)	65	170	161%
Pendlay Row	130	190	46%

I do think I’m reaching the end of my “linear progression”, and going heavy on legs also wears me out for basketball, which is still my favorite form of exercise. So going into the new year I’m going to be switching to 5/3/1 with a goal of slower but steady progress while also keeping my legs a bit more intact for basketball and exploring other goals rather than just moving numbers for the core lifts.

Technically 2, I also started smoking meats – a nice complementary habit to lifting :) ↩︎
The original program also includes back extensions and chinups – I’ve been swapping in rows instead, but I want to get to chinups in the new year ↩︎

Mental Models For Managing Change

Mon, 24 Jan 2022 20:00:00 -0400

This article was first published in my newsletter, Herding Lions

At this point I’ve spent the large majority of my career working in startups at different stages. While life can look very different in a 15 person early stage startup and a 300 employee growth stage startup, one thing that is constant across every startup I’ve encountered is a rapid pace of change and a relatively short time horizon for planning. Plans aren’t made for too long in advance, and yet often still manage to change significantly before reaching the original goal. Even in more stable environments, the reality is every software project is a unique undertaking to bring something new into the world. That brings an unavoidable level of unpredictability, meaning that everyone working in software must accept dealing with changing plans and priorities.

Some common questions predictably come with this state of affairs. How much change is too much? For a specific change, why are we doing this instead of sticking to our original plan? As engineering leaders we need to be able to both make choices about what changes to advocate for and push back on, and also learn how to explain those choices to our teams and other folks we work with.

I’ve found 2 different models helpful for thinking about changes in plans.

Understanding the level of the change

Changes in roadmap can come in at a few different levels. Understanding the level can help you understand how to think about it, and how to talk to your team about it.

A consequence of existing ambiguity This occurs when a team didn’t have clear committed plans previously or didn’t communicate them clearly elsewhere. This is less of a change and more a symptom of ongoing chaos. If a situation like this persists, it is the Engineering Manager’s problem to fix.
A higher level need A team is sometimes asked to do something that doesn’t directly tie into its mission, but is important for the broader company. Examples include helping another team with their project, or working on a revenue driving sales ask that wasn’t part of the original product roadmap.

Higher level requests are important to take seriously and often are the right thing to do. But if they become a regular pattern it can be a sign that your team doesn’t have the right mission, or you’re getting sucked into thinking too short term.

A tactical change When working on a project a team often learns about new and better ways to achieve their goals. A different approach to code, a new feature that moves the same goal, or putting different people with a new perspective on a problem are all new pieces of.information that serve as opportunities to make a better plan.

Tactical changes are normal and a sign that you’re paying attention to what you learn as you go. However, If you’re making these type of changes repeatedly, they’re not tied to a clear lesson from the work you’re doing, or you’re making similar tweaks on every project (looks like we need to add unexpected time for more testing again) it might be a sign that you didn’t properly plan or the team isn’t aligned on a goal.

A strategic change Sometimes teams change plans because their overall approach to achieving their goals has changed. Maybe they’ve decided to focus on a specific segment of their customer base or realized they need to focus on building up our quality before releasing new features.

The most important thing with strategic changes is clear communication – do your team and the stakeholders around your team understand not just what new work you’re doing, but also the strategic change behind it? If not, that misalignment can quickly move a project into muddy waters.

A change in mission Sometimes a team’s whole purpose can change. This often corresponds with a reorg or a higher level strategic change in the company. These changes should be rare, but are a good opportunity for a team to increase its impact.

After a change in mission expect to spend a lot of time learning, helping teammates work through the change, and handling a period of other change requests – because by default you’ve found your way back to ambiguity and it is your job to work your way out of that.

Track Your Stability Over Time

Beyond examining individual changes it can be helpful to keep track of how often you’re changing your plans over time. Some teams are naturally more stable, while others change plans more often. These aren’t necessarily descriptions of virtue – a team that never changes its plan can end up being slow to respond to environment changes and continue for months on low impact work, but a team that is constantly changing plans may never hold a direction long enough to make a dent in any problem. It’s best to consider this along 2 axes, level of change and level of impact.

Teams that are stagnant may need to consider more aggressively seeking feedback on their approach from users and consider making changes, or finding a higher value mission that is unlocking more impact over time.

Teams that are churning on the other hand might need to commit to finishing more, even if that means being a bit less responsive to new requests or opportunities. A discipline of making and then completing small time bound goals can be helpful here.

Of course there’s a fine line between stagnant and steady, churning and adaptive. An approach that was working before might go stale, and a team that appears stagnant may be in the early stages of unlocking a big iterative return. So this model should be used carefully, but it is helpful to have an opinion on where your team stands today as you consider new requests to change your plans.

More Thoughts On Time

Mon, 10 Jan 2022 21:00:00 -0400

This article was first published in my newsletter, Herding Lions

In my last post I listed some process tips for how to manage your time as an engineering leader. Continuing that thought, in the spirit of new habits in the new year this week will explore the question of how to choose what to focus on with the time that you do have. After all, efficiency is only valuable if it is pointed in a worthwhile direction.

Generally the time on my schedule can be divided into 3 categories - ongoing rhythms, one off must-dos, and opportunities. Each of these requires a different strategy.

Rhythms

Rhythms are my word for any activity or task that takes place on a regular cadence (daily / weekly / monthly / quarterly). As EMs we’re likely to have rhythm meetings focused around our teams (standups, 1-1s, quarterly planning, performance reviews), as well as upward and outward rhythms (staff meetings or regular cross-departmental syncs), and personal rhythms (clearing time each week to write a status report, or do code reviews or review the backlog).

If you get one percent better each day for one year, you’ll end up thirty-seven times better by the time you’re done. - James Clear

Healthy rhythms are extremely valuable. When you thoughtfully invest in relationships or valuable work on a regular basis, that investment can compound and redefine the type of impact you and your team can achieve. And regular maintenance of important relationships or projects can also help you stay informed and aware when things are off. When I plan out my quarters and years, I spend much more time on figuring out what rhythms and habits I want to maintain or develop than setting goals, because it’s the regular investment of time that ultimately leads us wherever we end up.

However, not all rhythms are healthy and inefficient rhythms that are not thoughtfully moving you toward some broader goal can be a painful tax on your productivity and your sense of well-being. Almost everyone who has done time in the corporate world has had the experience at some point of regularly attending a meeting that seemed to provide little value to many or all of the participants, where engagement was low and morale was lower. This type of “rhythm” can easily have negative externalities where it wastes not just the hour in the meeting but time before where you’re dreading it and time after when you’re recovering as well. Better time management with regard to rhythms means designing your rhythms. That means understanding the why behind each recurring calendar event and taking steps to remove or redeem the low and negative value ones. I covered this for meetings a few weeks ago, but the principles apply to any regular use of time. It also means taking the time to evaluate your goals and focus areas and consider whether there are any rhythms you should be investing in that might allow you to unlock the type of compound gains that require regular time. This could be clearing time for networking, building a new skill, 1-1s with key peers, or just dedicated “focus time” that you block and protect for deep work.

Must-Dos

Ok so this is the tricky one — what if you’re drowning in “must do” work? There may be seasons where you really are filling your calendar with mostly one-off non-recurring work. If this is a short term season with a clear ending (a big project push, a switch to a new team or company) that may be fine, but this is an area to tread carefully. When we perceive a large chunk of work as things we must do, it becomes much harder to be thoughtful about balancing our priorities.

It’s a cliché, but the key for this type of work is developing a healthy ability to say no, as well as good rhythms of prioritizing and evaluating your time. EMs who say yes to everything will quickly overwhelm themselves, and even worse will overwhelm and burn out their teams; so the ability to say no in a way that is effective and doesn’t burn bridges within your organization is a key skill. But often time work that we’re pushed to do is valuable. So the key is to make sure you’re prioritizing thoughtfully, and that takes time. I have a few questions that I commonly ask when considering external requests (or marching orders passed from above)

Is this a high leverage activity? I love Andy Grove’s model of manager time being valued in terms of leverage, or how much impact an action has relative to time/effort commitment. For asks of both me personally and my team, it’s helpful to see whether this is an incremental win or something that will enable more value over time. You can also think about this in terms of return on investment.
Can somebody else do this? For EMs, almost any work that can be done by somebody else on their team should be done by somebody else. Especially if you feel like you’re spending a high amount of time on urgent “must do” work. Give away your Legos, develop your time and clear your time to work on higher level work.
Can I do less here? When people are making external requests of you, most of the time they’ll ask for everything they think they might want up front, and are not thinking about your schedule. If the work isn’t core to your responsibilities, it’s always worth asking if there is a scrappier solution that will still make people happy. There’s a fine line to walk here — you don’t want to become known for skimpy/sloppy work, but it’s important to develop a habit of understanding what level of quality is needed for a task. People are used to making cost vs speed vs quality choices, and can usually help you understand their priorities if you frame things in that light (I can do it like this now, or do the full thing next month. If you need the full thing now it will mean stopping all work on some other thing they care about)

Opportunities

The final category of work is the most likely to get pushed off your category. This is work that won’t blow up if its neglected, might not ever be asked about, and in many cases isn’t visible to the rest of your org at first. This is taking the time to do a deep dive on understanding the quality issues on your “problem feature”, writing a reusable onboarding plan for your team to make it easier to add new teammates or clearing time to organize a brainstorming session on future roadmap activities. If you don’t already have ideas of what this might be for you, you could always start with clearing time to think about what areas your team could most improve in and your top ideas for addressing those problems.

This type of strategic deep work is an important part of having a team that is continuously improving. Without time to identify and understand problems, and time to address those issues, most teams will stagnate. And while that leadership can come from anywhere, on many issues it is often most impactful and most practical for it to start with an EM. The key skill for opportunities is to keep an identified list of your top strategic opportunities and priorities so that as time does become available to work on them, you’re ready to take advantage and don’t automatically fill that time with lower priority work.

Every EMs calendar is going to shift over time, with some months bringing more opportunities and others resembling more of a fire fighting session. But consistent habits of designing your rhythms, carefully evaluating the asks that come in to you and your team, and identifying and prioritizing key opportunities can go a long way. An Engineering Manager’s calendar abhors a vacuum and time will fill up. Invest in making sure that it goes to the right things.

How I Think About Time

Mon, 13 Dec 2021 20:00:00 -0400

This article was first published in my newsletter, Herding Lions

One of the biggest challenges every engineering leader faces is making decisions on how best to use their time. Every engineering team I’ve ever seen has more opportunities than capacity, and leaders can always do more.

My day always ends when I’m tired and ready to go home, not when I’m done. I am never done. - Andy Grove

Below are 4 lessons I’ve learned about using time well during my time as a manager.

Time is your primary resource as a manager, and it is finite. The Andy Grove quote above is challenging. When we can always do more, there aren’t natural boundaries that tell us when to stop. For individual contributors this can get managed without much intention through the rhythms of sprints / projects / the rate that product managers can assign work. EMs tend to be expected to generate their own work and there are fewer natural cues to tell you that you’ve done “enough”. As a result, management tends to be next level time management, and that means you need to start by defining some boundaries (when will you choose to go home and sign off?) and learning to be intentional with your time. A lack of boundaries, or setting your boundaries based on demand in an unbounded way is a sure path to burnout. This is a lesson I’ve had to learn several times personally, as demands on my time have scaled up.
Be careful with the Maker’s Schedule, Manager’s Schedule mindset. One of the most famous articles on schedule management in tech is Maker’s Schedule, Manager’s Schedule. In it, Paul Graham breaks down 2 ways of managing your time: manager schedules, where days are neatly divided into hourly blocks, and maker schedules where time is ideally at least in half day uninterrupted blocks (due to the need to do Deep Work / avoid context switching). I think most engineers take the right lesson from this article when they read this and work to protect blocks of time on their calendar better. And many managers empathetically learn from it that they should be considerate of their reports calendars and be careful how they schedule meetings. But there’s a dangerous assumption that people can take from the labels: that once they become a manager, they must be on the manager schedule. Now, plenty of manager’s are on a pure manager’s schedule (I’ve been there). But if you can, working at least partially on a maker’s schedule can unlock a lot more opportunity for creative and strategic work that is difficult to fit into 30 minute blocks scattered throughout your week. The article itself is essentially Graham explaining why he as an “important business person” chooses to operate on a maker schedule against social expectations. So read the article, learn from it, but don’t typecast yourself. Structure your time in the way that lets you have the most impact.
Run your calendar, don’t let your calendar run you. On the subject of manager schedules, becoming a manager was the first time in my life I really ran my schedule based off an external calendar, as opposed to primarily relying on my memory for what I needed to do on a day and using calendars and todo lists as a poorly utilized backup plans. Even if you have fewer direct reports and more strategic or technical duties, it’s hart to avoid becoming calendar driven as you move into leadership roles. That said, it’s important to take charge of your calendar, and not let other people structure it for you. That means being judicious about what meetings you accept, and regularly reviewing your recurring meetings for opportunities to prune them, blocking time for strategic and creative work in lengths that allow for deep work and is appropriate to your obligations, and defragging your calendar to be as efficient as possible. When I’m running on the busier side I’ve also found it helpful to fully block my day the night before the day starts (if it isn’t already) to prevent last second invitations from sapping up prep or break time in my schedule. Your mileage may vary on that depending on how much you culturally feel you need to be available for last second invites. If you do this I recommend explaining it to your direct reports and give them a path to escalate urgent things to you in order to not appear unavailable or “too busy”.
Clear time to review and improve how you use your time. The retro is my favorite “agile ritual”, and I think its difficult to overuse for both teams and individuals. One great use for personal retros is a regular review of where your time is going. Most people seem to have a bad intuitive understanding of this, its helpful to do a short term tracking exercise or reconstruct at the end of a week based on your calendar. This is a good time to prune meetings as mentioned above, identify “important but not urgent” work that is falling through the cracks, and also do a quick burnout triage.

When Everything Falls Apart

All of the above lessons have been very helpful to me, but I’ve struggled to follow some of them at specific points where I feel completely overwhelmed. So my 4 additional lessons here are patterns that I fallback to when I need to dig my way out of insanity.

Do 3 things a day. I’ve never been great with task management / productivity systems. I’ve wandered through a bunch of tools and patterns without really developing consistent habits. So I may be a poor person to give advice on this topic. However the most effective habit I’ve had, which I’ve picked up multiple times when things have gotten overwhelming is very simple – pick 3 things to do each day, write them down and then try to do them. Limiting myself to 3, not keeping a backlog and writing the 3 things somewhere I can visually see it has helped me focus and make choices when life felt overwhelming. Long term I’m not sure this is an optimal system – the lack of backlog and long term planning can leave blind spots. But it’s simple, productive, and easy to implement, and a great way to build momentum and prioritize when you’re underwater.
Distinguish between core work and non-core work - When things are going well and can feel easy to follow Grove’s advice at the top. I do stuff and then go home. However when you’re juggling 3 plates that are each stacked too high, it can be really hard to know how to stop. At this time it’s very important to draw lines between the essential and nice to have. One dimension I think about with this is core work vs non-core work. There are things I do at work to grow my career, to help others, or just because I’m interested in them. That’s healthy and should be part of my normal rhythms. But in crisis mode it’s important to identify these as things that can be cut down on or dropped. And then actually doing it, even if it’s painful.
Distinguish between glass and rubber balls. Similarly, it’s useful to distinguish between glass balls – responsibilities at home or work that won’t come back if you drop them, like being there for a child’s graduation or executing on a career altering project, vs those that we will be able to recover from more easily because there will be more opportunities to make up for it. The distinction between glass and rubber balls will be different for everybody, but Scott Eblin has some good advice here for distinguishing the two.
Choose between doing more things acceptably or a few things well. One of my biggest recent challenges as a manager was learning when my own standards for what “doing my job” were making me less impactful. At a time when I had more on my plate than I could reasonably do well, I made a choice to drop responsibilities where I had a real chance to provide value, because I didn’t think I could perform them to my own standards. That’s a defensible choice, but I didn’t really make it intentionally – instead I freaked out and protected myself. An alternative that I didn’t really consider until I’d gotten to a calmer place was accepting a lower standard of performance (and communicating that expectation to others) in return for being able to keep some projects going without having to disrupt others. Neither of these things are globally the right strategy, it will depend on circumstances and what responsibilities you’re balancing. But making the choice to not just commit to something, but to commit to a maintainable level of quality is an important skill and can help you scale yourself as a manager when under high demand.

Values Are How You Scale Culture

Mon, 29 Nov 2021 20:00:00 -0400

This article was first published in my newsletter, Herding Lions

I recently had the pleasure of working with a team to help define Kustomer’s engineering values. We went through a brainstorming process, got feedback from the team and ended up with a set of values that we believe reflect the best of our culture and how we work. These days this is the type of work that often feels most impactful to me; we’re investing in culture. But earlier in my career I probably would have been deeply suspicious of it. Aren’t values just babble-speak that managers use to justify their existence? Have I become what I once hated?

What I’ve learned through seeing this modeled in healthy organizations (both before and after shifting to management) is that values are how you scale culture as a company grows and evolves. Good values help employees make hard choices, help new employees know how to be productive in an organization, and allow hiring for culture fit in a way that doesn’t just translate to “this person looks like me”.

This of course doesn’t happen in every (most?) organizations that define a set of values. For values to be useful you need meaningful values that are regularly used well. Otherwise they’re just an easy to make fun of list of words.

Meaningful values are opinionated tradeoffs and applicable to day to day decisions. Opinionated means that you’re making a choice – values are a decision to prioritize one good over another good, not just to reject something obviously bad. The agile manifesto is a good example of this: instead of saying “Quality is good” (duh!) they say prefer working software over comprehensive documentation – a guiding preference that some devs may not embrace and can practically be used to guide decisions. From Kustomer’s values, we choose to “Optimize For Impact”, which means we may move off of something before getting to perfect, and may explicitly ignore some issues if they won’t move the needle. We value “Do What You Say” because we’re targeting a culture of accountability and trust over one that is simply “nice”. These values directly guide how I as a manager might choose to give feedback in a situation, and how an engineer thinks about calculating scope and an estimated due date for a project. Meaningful values are also specific to their context. Instead of writing our own values, Kustomer could have just taken Google’s values. And we certainly did look at other companies for inspiration. But ultimately values are useful for defining the culture of a company and the behaviors that are successful there, and that will not be the same between 2 companies. If you ask your employees to start behaving like Google employees that will likely lead to some interesting results, but your context isn’t Google’s and ultimately you’ll likely find that what works for a huge multi-national tech giant may not make sense for you. And realistically the values and behaviors that benefited Google as they were a small growing startup may not be the right ones for them today – they’ve even famously shaded away from “Don’t be evil”. Instead, understand what behaviors and priorities are useful and valued in your current context (some level of aspirational values are appropriate but mostly this should be descriptive of what is effective in your context today), and find ways to communicate those clearly and crisply, preferably using language that you already hear in your company today. It doesn’t mean that you can’t share values with another place or even use a term that is somewhat generic – it just has to have a clear meaning in your culture. Kustomer’s value of “Trust” is somewhat generic as a buzzword, but in practice its a term that has been important throughout our history and internally provides a clear set of priorities – work that threatens our customers’ trust always comes first, and we should always be proactive rather than reactive about quality issues. We also value going fast, but quality comes ahead of velocity. Every new engineer goes through a training where we unpack these concepts, so the value was simply naming something that already existed for us.

Once you’ve established a set of meaningful values, you still need to regularly put them to use in order to see value. That means referencing them when you make decisions (“We shouldn’t work on this because we need to optimize for impact” or “We can’t ship this without a monitoring strategy – we need to keep our customer’s trust, and we can’t do that if we don’t know when things break”). It also means rewarding behaviors that align with our values. At the company level, Kustomer monthly recognizes an employee who embodies our “Don’t just talk about it, be about it” value with a DJTAIBAI award. I’ve seen other companies experiment with “micro-awards” where employees have a budget for sending other employees rewards that get tied to their values. This also can work into your normal processes of evaluation and recognition by emphasizing behavior that aligns or deviates from values in performance reviews and 1–1 coaching. The more the values get talked about and rewarded as expected behavior, the more they have the power to scale culture.

Values are a tool for naming the culture you want to have and starting to iterate to it. They don’t guarantee that you’ll get there, and if the values you list aren’t already a meaningful reality for at least part of your company, they’re probably not going to transform your team. But they can help you scale what you have as you grow, and they can help set a bar that will open the path to the difficult conversations needed when things need to change.

Strategies For Building Team Resilience

Sun, 14 Nov 2021 20:00:00 -0400

This article was first published in my newsletter, Herding Lions

In Thinking In Systems Donella Meadows observed that when maintaining a system the properties you can choose to optimize for include productivity, stability, and resiliency. Productivity is about producing outputs – for a software team this might be completed projects, fixed bugs, or releases shipped. Stability is about consistency of output; can you maintain the same level of productivity over time? Resilience is the property of being able to recover from perturbations and unexpected changes. For a software team that might mean being able to handle losing a teammate, adapt to remote work or maintaining productivity during a tumultuous time for an organization.

Meadow’s interesting observation was that because resilience is less observable than productivity or stability, people tend to systematically underinvest in it. We can see outputs go up, and we can measure how they change over time, but resilience is by definition about response to rare and unexpected events, and therefore is hard to measure in normal circumstances. But the last year has been a reminder for many just how important resilience is; whether dealing with moving teams to remote work, the great resignation, supporting parents whose kids were suddenly home with them during the day, or the trauma of illness or loss from the pandemic, we’ve all had to deal with out of the ordinary circumstances when leading teams this year.

So how do we cultivate resilience on our teams? Here are a few things I’m thinking about:

Reduce Work In Progress - In addition to other benefits, reducing the # of things you’re working on a time helps team be more resilient by encouraging collaboration / shared knowledge.
Leave Margin - Don’t fill 100% of your sprint/quarter/annual planning with roadmap projects. Leave meaningful slack that can be used on technical improvements, unplanned requests, maintenance and learning tasks. That work is valuable to start with but the margin that planning for it provides also can make your team more resilient to changes in plan or scope creep surprises.
Build Psychological Safety - Teams that trust each other and can talk about difficult things are less likely to have issues fester and explode, and are more able to weather storms. Building a safe environment is the right thing to do for your teammates regardless of effectiveness, but the business case here is also easy to make.
Prioritize Developer Tooling & Velocity - Its easy to adjust to change when you can move fast. Velocity makes everything easier, and great tooling also makes it easier to onboard new team members and reduces reliance on the head-canon context of your more experienced team members.
Hire and Train For Redundancy In Your Team - Teams of specialists lack resiliency. If only one person can do a particular task, then you’re vulnerable to anything that removes that person: leaving for a new job, an illness, a vacation, or just an opportunity for them to be promoted into new responsibilities.
Have Everybody Take Real Vacations - Vacations are good for your team, but when people truly take the time off they’re also great “chaos engineering” experiments for your team to see how well you can function without different team members. If your output is limited when somebody is on vacation, it’s a smell that you can try and correct with some of the other tactics here.
Rotate Responsibilities - Similar to the last 2 items, we can build redundancy by making sure that we don’t rely on a single person to always fulfill certain tasks. It’s great to have experts and “owners” but those titles shouldn’t be exclusive and part of the responsibility should be helping others do the work and grow into co-owners.
Give Everyone Context - When everybody understands the why behind a team’s work, it’s easier to adapt to changing conditions because that can happen throughout the team – the leader doesn’t need to understand every change or detail
Write Documentation - Another way to make it easier to ramp people up or move people between roles is to write things down. This means processes, architecture, how-tos. Thoughtful, well-maintained docs can add significant resilience to a team. Unfortunately this item is also the one that I’ve never really seen done well at a team level and may in practice be unrealistic. At an org level though this is extremely important for resilience.

Good Meetings Are Designed Not Born

Tue, 02 Nov 2021 20:00:00 -0400

This article was first published in my newsletter, Herding Lions

There’s something deeply human about wanting to excel at the work we find ourselves doing. When we aspire to be effective at our jobs, how we describe and understand the key concepts in our work matters a lot. Last week I talked about having a better model for productivity — measuring productivity at the team rather than the individual level can cause significant change in what we choose to work on and the environment we create for our teammates. This week I want to consider the beliefs we have around meetings. Do you believe that bad meetings are just an inevitable cost of the working world that we all must endure? Or can we work to create better meetings?

I didn’t think very much about meetings early in my career; I just showed up where I was told, when I was told. Sometimes meetings were interesting, sometimes I just sat and waited for them to be over. For the especially long and irrelevant meetings, I learned that people didn’t object to me “taking notes” on my laptop, and I soon learned how to tune out without anyone caring.

As I moved into the startup world and began to care more about my work though, I started to get into more meetings that were genuinely interesting to me, and bad meetings started to bother me more. When I began learning about management and taking more leadership, I read more about the theory behind meetings and started paying closer attention to what worked and didn’t in meetings. What I learned is that good meetings are designed, not born. It’s possible to have meetings that are good experiences for everyone involved, but it takes intentionality.

There are 4 levers I use when designing a meeting: choosing the audience, crafting the agenda, picking the right cadence, and shifting aspects async.

Audience - Meetings without key decision makers or subject matter experts often end without conclusions, and decisions can get overturned quickly afterwards. On the flip side, while it may feel kind to invite anyone interested to a meeting, a meeting with too many people can become chaotic, waste people’s time, or stifle contribution if some people aren’t comfortable being open and honest in front of the full group for reasons of timidity, confidentiality, or simply unfamiliarity with the participants.

Agenda- Coming into a meeting with a clear agenda doesn’t always lead to a great experience, but without one it’s pretty much impossible to guarantee a well run meeting. A thoughtful agenda sets a direction for a meeting, makes goals clear and helps people prepare. Written agendas are great, but a simple verbal “the goal for this meeting is” at the beginning of a less formal meeting can go a long way. It’s never too late to reset everybody towards a plan.

Cadence - Should a meeting occur daily? Weekly? Monthly? On an as needed / ad hoc basis? This decision has a big impact on a meetings tone and impact. Set this too regularly and you’ll find you’re wasting people’s time and losing engagement. Too spread apart and you might not really be accomplishing your goals.

Async - We’ve all been in a classic “could have been an email” meeting. But even for meetings that clearly do merit time together, it’s worth asking if parts of it can be split out async. Maybe updates can go into Slack while we meet to talk about strategic discussions or open problems. Maybe instead of demoing a new design and then discussing it in a meeting, a designer could send a video of the design, take some async comments and then meet to discuss the more controversial aspects of the design that weren’t fleshed out in comments.

The process for designing a meeting with these levers is straightforward. Break down what needs to be accomplished (goals), identify the audience for each goal, consider whether a goal can be fully or partially accomplished async, figure out the cadence for the meeting(s), and then write an agenda for each meeting you’re creating.

My single most important meeting design advice is never start with a meeting, start with your goals. Some of the worst meetings are those that try to do 5 things poorly rather than 1 or 2 things well. When you start by breaking down and defining what you want to accomplish, it becomes much easier to pick the right audience, agenda and cadence for your goals. For instance I was recently part of a weekly cross-org meeting that was attempting to push forward tactical code level changes as well as serve as an announcement/discussion place for important strategic announcements. Attendance was poor and the meetings were often low on engagement. The organizers were able to greatly improve this meeting by identifying and breaking apart these goals, shifting to 2 meetings: a monthly strategic meetings for managers and a weekly tactical meeting for engineers involved in the initiatives.

Once you have a set of goals, you should figure out the audience for each one. If the audiences of your goals are sufficiently different, consider splitting your goals across multiple meetings or async mediums rather than trying to cram them into a single meeting. Even if it feels inefficient for you as an organizer, it can greatly streamline the process and bring better outcomes for everyone as in the above example. If your goals all need roughly the same audience, just think carefully about who really needs to be included. There’s a lot of value in keeping information open, sponsoring folks for opportunities, and listening to all voices. However there’s a delicate balance between pursuing those goals and inviting chaos or wasting people’s time. If you’re having trouble cutting a meeting, consider inviting a smaller group but sending the minutes to a larger one to keep them in the loop, or workshopping a proposal with a small group and then presenting it to a different group of stakeholders.

At this point you should be able to see if you really need a meeting or whether your goals can be accomplished async, and what their cadence should be. Sometimes some goals may need a meeting while others can be handled async. For instance when we redesigned my team’s standups last year we determined that our “individual status updates” could be handled async through Slack, and the purpose of our standup Zooms would be connecting as a team, checking in on project status with a quick confidence check and an open floor for people to discuss issues and blockers. This allowed us to remove some boilerplate from the meeting discussion, honored my team’s desire to have more dedicated “team time” during the pandemic, and led to deeper and more relevant discussions. When in doubt I recommend starting with something happening async until it proves that it needs a meeting. If you do believe you need to meet in person consider starting with meetings scheduled on a case by case basis rather than immediately jumping into a recurring cadence. You’ll get a sense of how often the meeting is relevant.

Finally, make sure to have an agenda. If you’ve started with goals this is easier. Each goal covered by a meeting should be an agenda item. I’m currently running a daily project check in sync where we determined the goals were to make sure we stayed on track, make sure we were working tightly together across roles, and quickly resolve blockers as they arose. As a result the agenda for that meeting is a 2 minute review of our progress against project tickets in Jira, an open time for developers and our designer to demo any new updates to the team as they complete work, and then a period for raising blockers or issues for the team to discuss and hopefully resolve. Each section of the meeting is directly connected to a goal so the agenda was easy to write. For one off or informal meetings you may not have a written agenda, but I still recommend that you call out the goals as you start the meeting, and circle back at the end to make sure you’ve addressed each goal. This practice has made a big difference for me in terms of running effective meetings.

The quality of a meeting isn’t the result of a dice roll. You can make a meeting better with planning. Take the time to do it.

New Newsletter: Herding Lions

Tue, 26 Oct 2021 20:00:00 -0400

This blog has always been a bit of an eclectic assortment of topics that have reflected my shifting interests over time. Recently I’ve been primarily interested in writing about Engineering Management, my role for the last 3.5 years. I’ve also been craving more conversation on these topics and want to be more interactive in my writing.

As an experiment towards those 2 goals, I’m starting a new newsletter Herding Lions focused on Engineering Management. It will have a mix of shorter pieces from me, interesting links, and questions that I’m hoping to engage with folks on. I’m excited to hear from you there.

You can see the first issue and subscribe here.

Thinking Fast & Slow: Lessons For Managers

Wed, 20 Oct 2021 20:00:00 -0400

I recently read Thinking Fast & Slow by Daniel Kahneman, and it was my favorite read of this year. I had been generally familiar with Behavioral Economics previously, but this was the deepest I’d gotten into it. Kahneman is a very strong writer, and it left me with lots of things to think about. A few of the ideas in particular felt relevant to management, and I thought I’d highlight them here.

Everybody thinks about their lives and careers as a story. When we evaluate how good a story it is we tend to think about the high points, low points and most recent feelings only, neglecting how long any of those things lasted. [Summarized from the chapter 36 “Life as a Story”]

This made me think of how a bad week, or a very bad incident in the past can negatively color the story of an employee with an otherwise happy tenure leading to what Michael Lopp calls shields down moments. This can feel unfair for managers who are doing things right and want to maintain their teams, but the main lesson I take is to not let little annoyances fester without addressing them, as little things can shift people’s attitudes by more than is justified by a big picture view.

Humans have a number of cognitive traits that make us bad at predicting and estimating — we ignore evidence that is not currently on our mind, become confident on relatively small pieces of information, and have a tendency to simplify complexity away by answering easier questions than what we’ve been asked. We also over-weight “causal thinking” and underestimate regression to the mean. [Summarized from Chapters 18-24]

This was easy enough to accept personally – I’m perennially overconfident in my estimates. The lesson here in my mind is that more process around predictions may lead to better outcomes; left to our own devices we’ll think less about predictions and not notice that our confidence is based on flimsy evidence. For more important predictions, processes that force us to be more thorough should help.

Humans are very sensitive to how questions are framed — we hate losing things more than we like gaining them, are very associative in our thinking, and often aren’t consistent in how we make choices depending on external factors like a suggested answer, another question we were just asked, or how close it is to a meal. [Summarized from Chapters 25-34]

I see this type of stuff in behavioral economics mostly quoted in terms of “nudge” like behavioral manipulation, but this mostly has me thinking about the importance of communication plans. It’s easy to think that getting info passed to a team or individual is more important than how you do it, but actually how choices and changes are communicated has a real impact on how people feel about them, which can make a big difference to team morale and retention over the course of many communication opportunities. Lara Hogan has some good resources on communication plans including this article.

We all overweight things that are easy for us to see vs things that are not (availability bias). We tend to see our own contributions as bigger and more meaningful than others, our own challenges as more painful, and our own skills as more impressive. Because of course we see all the evidence for those and only a fraction of what others are doing. [Summarized from Chapter 12 and some references elsewhere]

This was a good reminder to me to be humble and not overweight my own accomplishments first. But it also is a good thing to keep in mind the next time a teammate feels underappreciated or that they may have been unfairly passed up for an opportunity. We’re all prisoners of our own perspectives, and while that may sometimes lead us astray in terms of how we stack up a little grace can go a long way here.

Kustomer Is Hiring

Wed, 22 Sep 2021 20:00:00 -0400

👋 Hi There!

It’s been a bit of a quiet year on the blog as I’ve been figuring out life with 2 children, work and a pandemic that doesn’t seem like it wants to go away. I’m going to try writing a bit more soon, but in the meantime wanted to add a quick plug for Kustomer.

I’ve had the pleasure of working at Kustomer the past 2 years now – it’s been a crazy rocket ship experience, my first time in a high growth unicorn startup, and we’re very much just getting started in many ways – we’re looking forward to joining Facebook in the future and all the possibilities that will bring to the table.

In the here and now though, we’re building an extremely cool product – improving customer service experiences for millions of people by giving agents better info and better tools and making it easier for people to find the info they want themselves without long hold times or delays between emails. We’ve got lots of fun technical challenges as we scale and work to improve performance of our system at every level (you can get a taste of a few of these at the recently started Kustomer Engineering blog). And we have a fantastic team of engineers who are extremely collaborative and focused on building the best experience for our customers (and their customers).

We’re growing like crazy and have a ton of ambitious plans for the future, which means we need more folks. We’re hiring for

Engineering Managers: EMs at Kustomer get to lead 1 to 2 teams of 3-7 engineers each of which have a specific product focus. You get to wear a lot of hats – technical leader, domain expert, coach, chaos-tamer, and are pretty much guaranteed to learn a lot. My 2 years in this role has not always been easy, but it has pushed me to grow more than any other stretch in my career. It’s a great role for people who like a challenge and thrive on taking on different types of work depending on the day.
Frontend Engineers at Kustomer get a large amount of autonomy to solve business and user experience problems in their domain of expertise. Every team has a dedicated Product Manager and Designer that you’ll be collaborating closely with, and we’re at a fun point in our growth where we’re taking a product that has grown organically and finding ways to standardize best practices, improve performance, and be more consistent in building great user experiences, not just “working” experiences. It adds up to a lot of great technical challenges, and we’re big enough to have experienced engineers to support and teach while still being small enough to have lots of room for people to come in and own important problems. Our core stack is React / Redux / TypeScript / Jest / Cypress / Webpack.
Backend Engineers have similar collaborative and ownership opportunities as FE devs, and get to work in a modern Node microservice based codebase with its own set of interesting technical challenges, especially as we scale the product. We spend a lot of time thinking about reliable message processing, storing and performantly accessing large amounts of data, and applying machine learning and intelligent automation techniques to customer service data. Our core stack is Node / Express / Docker / ECS / AWS / MongoDB / ElasticSearch.

*All of these roles will eventually be New York City based, but are currently operating remotely. *

If any of the above roles sound like the right type of challenge for you please feel free to reach out and apply. You’re also welcome to contact me with any questions about working at Kustomer. Helping bring new awesome people who are excited about what we’re doing into the company is the most important thing I can do both for them, for Kustomer and for making my own job great, so always happy to talk.

Engineering Management: How To Give Feedback

Sat, 06 Mar 2021 20:00:00 -0400

I’m currently taking some time away from my job on parental leave, and it seemed like a good time to reflect on the lessons I’ve learned the last few years about engineering management. This is my fifth post, you can see a list of past posts on the series page

Giving good feedback to others is a core piece of any manager’s job, but it’s also one that is rarely taught formally. We all give feedback in some form or other every day: we leave reviews, give compliments, complain, or thank people. But making the feedback effective is a lot harder, and as a manager it is well worth the effort to consider how you can improve. Let’s talk about the why, when and how of effective feedback.

The majority of people-related disasters I’ve created originate with my choice to not say the hard thing. On my short list of critical leadership skills, the ability to “say the hard thing” is right after “delegate until it hurts.” - Michael Lopp

As managers we give feedback because it is the core way we drive improvement. We are not generally doing the work, instead we evaluate the work and give feedback. That means making it clear to the team when we see things that are or are not working with the end product, the process to build that product or the inner dynamics of the team. And it means being clear with individuals what is or isn’t working. Improvement for teams and individuals is a function of practicing the relevant tasks over time with useful feedback loops. Teams will get some natural feedback loops from the result of their work, peer feedback, and self evaluation but those are unreliable and/or slow. As a manager you are responsible for making sure that your teammates have tight feedback loops that allow them to understand quickly when things are going off track and correct them, or when they are doing something well and have an opportunity to do even more.

“if it hurts, do it more often, and bring the pain forward.” - Continuous Delivery by Jez Humble and David Farley

It’s important to understand that feedback isn’t something that should happen only at performance reviews – good feedback is continuous. It’s helpful to have regular times for giving and soliciting feedback like a weekly 1-1, but even those should be a backstop – giving feedback directly after you observe a behavior or impact is often the best time. We want to minimize the feedback loop so that your teammate can respond quickly, and also minimize distance from the event which can often bring skewed memories from both parties. For those who find giving feedback hard or scary, the idea of doing this regularly might seem overwhelming. In this case though we can benefit from the devops principle of doing hard things more often. When feedback is regular and low intensity it can be easier to give and absorb, whereas a mountain of feedback at a performance review can be overwhelming and scary to give. Early feedback also allows for quicker course correction, hopefully making those difficult review conversations less common. Performance reviews should never contain surprises, they should just be formalization of an ongoing conversation.

“Radical Candor” is what happens when you put “Care Personally” and “Challenge Directly” together - Radical Candor by Kim Scott

Radical Candor has influenced my thoughts on feedback more than any other single source, and while the book is worth reading in full, its core idea is easy to summarize: good feedback comes from caring personally about the individual and being willing to challenge them directly. We’ve all dealt with the jerk who is happy to give unfeeling and unwelcome feedback on anything – that’s challenging without caring, what Kim Scott calls “obnoxious agression” in the book. More common and insiduous though is what she calls “ruinous empathy” and “manipulative insincerity” – caring or not caring without challenging and thus simply avoiding feedback altogether. The single biggest antipattern around feedback that I’ve seen is choosing to not give it until it is much too late. People who give good feedback have plenty of empathy and understand the impact of their words – but they also understand how people can be hurt when nobody tells them the truth, so they let that empathy drive them to choose their words carefully and judiciously speak out with courage.

The Situation-Behavior-Impact method is simple and direct: You capture and clarify the Situation, describe the specific Behaviors observed, and explain the Impact that the person’s behavior had on you. Center For Creative Leadership

If you’re struggling with how to express a certain piece of feedback it can be helpful to prepare in advance. I usually benefit from writing down my thoughts and have found the SBI method from the Center For Creative Leadership very helpful here. For any feedback you want to give, frame it in terms of the situation (when and where did this happen – any relevant context), the behavior you observed from the person receiving the feedback (what did they do), and the impact on you or that you observed. This model is helpful in giving a clear way to write feedback as well as avoiding a bunch of potential traps. It leads you away from guessing at anyone’s intent or motivations, grounds the feedback in a real situation and behavior, and makes the severity of the feedback clear. Optionally you can continue to a 4th step – asking the person receiving the feedback what their intention was and giving them a chance to share their thoughts, hopefully leading to a productive 2 way conversation.

Feedback is a gift - Michael Lopp

So far in this piece I’ve focused on giving feedback, but it’s important to remember that feedback is a two way street. Giving feedback well gets easier when we develop an attitude of curiousity and gratitude when we receive feedback. Remember that all feedback is a gift – even if some pieces of feedback are easier to work with and better delivered than others. Creating an environment where others are comfortable giving you feedback is hard – if you’re a manager you will have to work upstream to overcome some natural power dynamics if you want real honest feedback. It means being persistent in asking over time, building trust by proving you care, and being willing to take and be grateful for hard feedback when it does come. Ultimately there is almost nothing you can do that will help you more as a manager than creating an attitude where people are comfortable sharing unfiltered truth with you – you’ll grow faster and be more effective as you learn what is really happening around you, and it is much easier to share hard feedback with people who trust that you’re listening to them. Feedback is a gift. Give it, receive it and value it.

Engineering Management: How To Delegate

Thu, 25 Feb 2021 20:00:00 -0400

I’m currently taking some time away from my job on parental leave, and it seemed like a good time to reflect on the lessons I’ve learned the last few years about engineering management. This is my fourth post, you can see a list of past posts on the series page

In my last post I discussed the importance of delegating work to your team and giving them room to do important work. Delegation like this is crucial for effective managers, it’s the only way that you’ll ever have time to grow and it’s the only way that you can help your team grow. But it isn’t always easy, and effective delegation certainly isn’t as simple as giving somebody a task and walking away. I’ve picked up a few principles over time on how to delegate well, as well as a few practices that have been helpful for me in finding the right mix of delegation. The principles that have been most helpful for me when delegating work have been to “trust by default”, thinking about people’s ability in terms of “task relevant maturity”, “not being essential”, and “giving objectives and context rather than just tasks”.

The most important step to beginning delegation is to trust by default rather than making people prove themselves before giving them opportunities. Trusting in this context doesn’t mean blindly handing people arbitrarty work and expecting it to go well. But it means being willing to give people reasonable stretch opportunities and then properly supporting them through them, and doing that starting early on in your management relationship with them. Ideally trust will build as you work with a teammate, but if you default to a “prove it” attitude it can be tough for teammates to ever gain your trust. If this proves difficult for you it may point to a higher level problem like you being too attached to doing work yourself or not having the right people on your team for the work you’re trying to accomplish. Don’t jump to that last conclusion though – it’s important to look in the mirror first.

When evaluating people’s ability to handle a task you’re considering delegating to them, you shouldn’t be looking at their title or general track record, but instead focusing on what Andy Grove calls their task relevant maturity – how experienced they are at the specific task under discussion. It’s possible to give brand new employees tasks that are easily within their wheelhouse and very senior team members tasks that are outside of anything they’ve done before. If you base your support and expectations solely on how senior somebody is or how successfully they’ve handled generic delegated work in the past, you’re likely to mishandle the delegation.

Adjusting well based on task relevant maturity means giving teammates who have less experience of success with a task more support and monitoring their progress more frequently. This doesn’t mean micro-managing, but being a resource to the employee, checking in on the task regularly during 1-1s and reviewing any incremental steps in the process for signs that you’ve become misaligned on the objective or that things are off track. The key is that you’re a support resource and a quality checker, not laying down specific restrictions on how the task is done. For teammates with progressively more experience with the task you can reduce check-ins and explicit offers of support to match their comfort level, though you should never completely ignore projects or lose track of what is going on with anything important.

A 3rd principle of delegation is more a warning post that you’re not delegating enough. I never want to be essential for any task as a manager. If I get hit by a bus or leave the company, my ideal is that everything in the teams reporting to me will hum along perfectly for a long time, with my absence being felt only in lost potential in how I could have helped people grow, or solved higher level problems in the company. That means that if I see areas where I am essential for success, rather than allowing that to inflate my self importance I should view it as a sign that I need to grow somebody on my team into that role. If there are people who could be doing it but I’m not giving them the opportunity, it’s time to delegate. If there’s nobody who can do the work, then it’s a gap on the team that needs to be filled by training or hiring.

Finally, it’s important to delegate at the right level. Delegating completely defined tasks with a prescribed solution is just asynchronous micro-managing. Instead you should delegate objectives, and give people the context they need to make good decisions. For technical work this can come through in how a Jira ticket or Github issue is written. Does it describe a technical change that needs to be made without explaining why? Or does it describe a user problem, with context around what we know about past attempts to solve it or any relevant constraints¹? When having a tech lead replace them in an interview, a manager could choose to give them the list of questions they have been asking and ask the tech lead to read from the sheet, or they could explain what they’re looking for in the position, any constraints around what will be accepted by the wider organization, and then have a discussion around the interview questions that the tech lead might want to use. The second way will engage the tech lead’s brain, give them ownership and maybe help the manager learn something.

Practical Activities To Delegate Better

It’s good to understand the principles and theory behind delegation, but what if you’re not getting it right in practice? When I discovered last year that I was not delegating nearly enough, there were 2 activities I went through that helped me do a better job with this: a weekly review and being extra explicit about ownership

After getting feedback from multiple folks that I was trying to do too much work myself, my first step to getting better was a simple 30 minute weekly review where I went through all of my activities from the week and listed out anything I’d spent time on that wasn’t part of what I considered my “core responsibilities”. From that first list I then considered 2 questions: who else could be doing this work, and are there good reasons why I’m doing it instead? If you’ve never thought this through before you may find some good low hanging fruit where you’re doing work that somebody else on your team would benefit from doing without a compelling reason. It’s perfectly fine if many of the things on this list are more complicated though. There may not be an obvious person for every piece of work, or there may be good reasons why you’re picking it up. In those cases the goal of the exercise is to bring clarity and give you time to think about what would need to happen to move these tasks to a teammate who would benefit from the work and/or be a more appropriate fit. Maybe you need to invest in coaching a teammate, have a difficult conversation with a peer about ownership, or just need to document the work so that other people can do it.

I started this exercise focused on “non-core” work, but scaling yourself as an EM means constantly giving up work and letting others grow into it. So feel free to consider all of your responsibilities and see if there’s an opportunity for others to grow into some of those while you grow into broader and new opportunities or just get more time back to invest in people.

Another thing I found helpful when starting to document more was being explicit about ownership. If you have been handling a type of task yourself and want others to step into it, there are benefits to creating a clearly defined role or set of work that you can give somebody. Without this it can be easy for some piece of the work to fall back to you or be missed. For instance if you’ve been doing most of the project management on tech projects and want engineers to step into that, it might be helpful to create and document a “project tech lead” position that you assign for each project. You can then be explicit that the project tech lead is responsible for keeping the project on schedule and identifying or escalating threats to the project plan along with any other duties that make sense. You can also be explicit about what ways you expect to be supporting a project tech lead and any responsibilities that still fall on you as an EM.

Why Delegate?

Delegating work well is an incredibly rewarding activity. You give others opportunities to grow. You can reduce your own stress and increase your capacity. You’ll learn from seeing other people take different approaches to work you have done in the past. Over time this will help create a team that is excited about their work since they’re empowered to do meaningful work and continue growing over time, and it will allow you to continue tackling new challenges as well. So take some time today to figure out where you can find new places to share your opportunities and create new ones.

Engineering Management: Handling Accountability

Mon, 22 Feb 2021 20:00:00 -0400

I’m currently taking some time away from my job on parental leave, and it seemed like a good time to reflect on the lessons I’ve learned the last few years about engineering management, this is my third post, you can see a list of past posts on the series page

When moving into an engineering manager role, one of the toughest transitions for me was mentally adjusting from being responsible for tasks and projects that I owned to being accountable for the output of a team. As I discussed in What Engineering Managers Do, I like using Andy Grove’s definition of a manager’s output: the output of their team as well as the output of any teams that they’re influencing. Manager’s thus bring value by improving those outputs. This potentially allows you as a manager to have a much larger sphere of impact than an individual contributor working on a single project at a time. But it also means that you will be held accountable for actions that you may not have had direct control over. This can be distressing for new EMs (it was for me!) as you realize that “working harder when things go wrong” may no longer be a viable strategy for avoiding failures.

The first time a manager gets negative feedback for work that wasn’t personally in their control it is tempting to prevent it in the future by micromanaging or using themselves as a safety net. Micromanaging has a horrible reputation, but it is actually a very natural reaction for engineers who are used to competently completing tasks and then being praised for it who face criticism when a task they know they could complete is not done adequately. It is a very human response at this point to say “well I know how to do that, let’s make sure you’re doing it the way I would”. When that is repeated over time it leads to the familiar and awful rhythm of low trust between the manager and teammate, low initiative from the teammate and frustratingly slow progress.

Acting as a safety net for employees is a more subtle task for managers. They may know they don’t want to micromanage, but instead they might choose to tackle the complex, likely-to-fail parts of a task themselves, or perform a “diving save” taking over a project when it looks like it might not complete successfully. This appears better than micro-management – employees have ownership of their work, at least until it goes off the rails. But it leads to many of the same issues as pure micromanagement: employees who don’t grow because they’re not learning to do the hard work themselves, a disconnection from consequences that means a lack of ownership from the team, and a lack of trust that will eventually erode morale. Like micromanagement though, this approach doesn’t scale. You and your team will both be locked to a limited amount of responsibility that is based on your capacity as a manager to be in the details of everything. Growing as a leader and a team requires a different approach.

The only solution for handling accountability that scales is developing a high performing team. It isn’t a problem to be held responsible for the output of a group that you trust and believe in. And high performing teams have a tendency to generate new leaders, allowing you to do less and/or focus on higher level work and grow your career. But creating a team like that requires effort and humility over time. This means as managers we have to be willing to take some short term failures.

While your team is in a growth stage (any of the forming, storming, norming steps in a group’s development) you’ll need to expect that their output will not always meet your individual standards. There are a few things you can do to mitigate this from an accountability perspective.

Firstly it is important to match your expectation setting and commitments to your team’s maturity. If you’re not confident that a team can make a commitment without you doing the work yourself, don’t sign on for the work. And certainly try to limit the number of simultaneous challenging projects you’re taking on so that you can focus on enabling your team when you do have challenges. This may sound naive and frustrating to people who feel they have to take on the work. The key thing to understand are that you can’t do this forever – eventually the team has to grow it’s capacity to something the organization can accept, or either you or your management is failing. But in most circumstances it’s better to give an honest no up front than make excuses for a failed project 3 months down the road. There are organizations which won’t reward that calculus – in which case you should probably document your concerns and continue anyway, hopefully developing some credibility for the next such discussion.

When you’re in this type of growth stage, it’s also important to understand the impact of the work that is coming in. I like the analogy that Gregg Dedrick shares here of juggling crystal and rubber balls. Some tasks are like crystal balls – if you drop them there are expensive consequences and no second chances. But most tasks are more like rubber balls. If you drop one you’ll get to pick it up and try again. It’s important to understand which are which. If your growing team is being asked to pick up crystal ball tasks it’s important to exercise caution. This is where it’s extra important to be clear about your team’s capacity, and is a case where it may be appropriate to get more involved personally to ensure success. Of course if most of the tasks coming your way seem like crystal balls, it’s worth digging into that more and understanding what is driving that, and if some of them are actually more droppable than you think.

As your team grows you’ll be able to start shifting the work you take on and saying yes to more things. At this point it’s important to nurture the ownership that your team is feeling. Be generous with credit for successes while continuing to own accountability for failures. Within the team, make sure you are holding high standards and giving clear feedback when needed. Nothing undercuts an empowered team like somebody who isn’t contributing at the level of the rest of the team and isn’t held accountable for that. That said, leave room for failures to continue happening. Even high performing teams have room to grow, so the principles for a growing team still apply – let people take on rubber ball tasks that are a stretch,

Accountability for a high performing empowered team

Managing a team that feels ownership of their work and is growing into higher levels of capability is more rewarding, less stressful, more educational, and more powerful for your career than micromanaging. It’s also ultimately morally good – you’re creating an environment where other people can thrive and use their gifts and abilities.

Engineering Management: Choosing What To Work On

Sat, 20 Feb 2021 20:00:00 -0400

I’m currently taking some time away from my job on parental leave, and it seemed like a good time to reflect on the lessons I’ve learned the last few years about engineering management, this is my second post here, a followup to my post on What EMs do

In my last post I listed a bunch of different things that engineering managers can do to produce value. Given the sheer number of things that engineering managers can consider doing to help their teams and make an impact, the next question quickly becomes: how do I balance my time and work on the right things? I think of this in terms of maintaining rhythms, finding high leverage work, and following some principles for maintaining sanity.

There are a number of rhythms that I try to maintain every week. For me these include 1-1s with my team, our agile sprint rituals, preparing weekly updates for our VPE and CTO, and dedicated time set aside for strategic work. I focus on protecting these times and as much as possible, but also maintain a pretty high bar for what type of thing I set up as a recurring meeting or obligation. I draw a distinction here between things I do a lot, and things I try to do every day/week/sprint/month. Making it a regular commitment prevents these things from being washed away by busy periods and unlocks the compounding benefits of something that is routinely practicied.

Rhythms are a great fit for activities that are high value but low urgency, like maintaining important relationships, keeping up good information flow, and protecting time for longer term creative work. The important thing here is being selective and as I’ll discuss more below, making sure to review regularly to make sure they’re still a good use of time. Rhythms become an anti-pattern when they get disconnected from value – when you keep doing something that is no longer providing value – a zombie standup meeting or a regular meeting with another team that could just be handled through async communication. I doubt that there is a universal ideal ratio of rhythms to unstructered time, but for me personally I end up having about 20 hours each week that is preblocked for the same activities every week, and that has felt like an appropriate tradeoff between the urgent and new with the consistent and important.

With my remaining time I try to focus as much as possible on high leverage activity. As in my last post I’m borrowing language from Andy Grove’s High Output Management here – leverage is simply a term for an activity’s value relative to its cost in time/effort. So training somebody to do repeating work is often higher leverage than doing it yourself; you pay a one time cost to get an ongoing benefit of the task being completed and growing one of your teammate’s careers. If the term leverage doesn’t speak to you, you can think of the pareto principle and trying to work on the 20% of stuff that brings 80% of the value, or working “smarter not harder”. The point is to figure out what activities are going to have outsized impact and push to prioritize them.

Will Larson has a good guide on identifying work that matters. It’s focused on staff-plus engineers, but the principles apply to EMs equally well. Once you’ve exhausted low effort high value efforts, there’s more value in taking on the bigger problems that will truly impact a company. Swarm on existential issues, but otherwise try to find places to work where there’s room to make impact but you will get support. You can scale your impact by investing in people and nudging or editing other folks approaches without having to own everything yourself or by helping them finish projects that need a bit of help getting to done. And it’s important to see if there are only things that you can do. I can’t phrase this better than Will did so I would highly recommend reading the whole thing on his site. I would add that for managers ultimately all of our value will disappear if we do not have a high-performing team that is able to stick together and make progress over time. I’ve never regretted spending time investing useful time into my team.

Of course in real life, picking high leverage activities is not an exercise where we calmly sit back and pick from a neatly arranged menu of options. Instead as an EM you will be bombarded with requests for your time; meetings with other teams and departments, project meetings, compliance documentation, onboarding new employees, performance reviews, mandatory company trainings, process improvement discussions, status update requests and more. Some of these things will be high value and important, depending on the culture of your company almost all of them may appear urgent. It isn’t a productive goal to only ever work on the most important things at all time. There will always be urgent work that you’ll find you feel you need to do. However there are better and worse ways to handle this flood of urgency.

Effective managers know the importance of saying no to many things in order to focus on the essential. There’s space to grow beyond that though – a no that preserves a manager’s integrity is better than a yes that they fail to follow through on, or one that leads to them working on the wrong things. But even better is if they can work with the people making requests on their time to find ways to make the urgent request smaller, or help get it delegated to people who will grow through the work. Ultimately managers are as effective as their relationships and finding ways to say yes without getting distracted from the most important work is a valuable advanced skill. Of course the temptation can be to trade away these hard decisions and problems by simply giving more time.

Working more hours is the ultimate short term time management power move. It’s also a quick path to burnout. I try to follow a few principles to keep my schedule sane.

First it’s important to reconsider your time commitments regularly, especially any recurring meetings/commitments. Many recurring meetings degrade in usefulness over time, and even ones that should be a bedrock of a manager’s schedule like 1-1s may need to be adjusted as the needs of the participants change. I try to reconsider my calendar as a whole at least once a quarter, and anytime I have what feels like an unproductive meeting or a scattered day I try to think through whether I can make any adjustments to my schedule to improve in the future. This can be as simple as Lara Hogan’s concept of defragging my calendar to put similar things together and clear creative space, or involve reworking a meeting that has gone stale to be more useful for the team – maybe even reducing its frequency or canceling it and replacing it with an async process.

My second principle is that constraints are good. Any EM will have an essentially infinite amount of work. There’s always more training, strategy, preparation and planning that can be done. If you’re still comfortable coding that represents another near-infinite pool of potential contribution. A work till it’s all done approach simply won’t succeed. There is always more. So I try to embrace a schedule and accept that schedule as a constraint to solve for in my work. At points this has been a literal daily stopping time for me – more commonly in my career I’ve simply held a value of being there for my family and protecting family time, as well as protecting my health and time to take care of it. That has meant that at times I have flexibility to pour a lot of time into work, and other times I’m more limited in my total time to devote to work. In each case though I’m able to take a finite amount of time and do my best to optimize it for the challenges in front of me.

Stay Sane, Keep Your Integrity, and Remember That People Matter The Most

Managing your time in more senior roles is hard. You’re not given the direction that you may have gotten used to as an IC engineer, and a lot of the most important work won’t have a timer or anyone yelling at you to get it done. If you’re struggling with this I recommend giving yourself grace. And if you take anything from this piece I hope you remember to protect yourself with constraints so that you stay sane, avoid compromising your integrity by over-committing and ultimately err on the side of investing in people. In 5 years it will be the people you’re interacting with today who are going to have the biggest impact on your career, and as I look at the scope of the rest of my life I expect it to be my impact on people that I remember for better or worse, much more than the results of a project or whether I nailed a certain process for my team.

Engineering Management: What do EMs do anyway?

Wed, 17 Feb 2021 20:00:00 -0400

I’m currently taking some time away from my job on parental leave, and it seemed like a good time to reflect on the lessons I’ve learned the last few years about engineering management, starting with a base assumption of what it is that engineering managers actually do

There are a lot of different ideas out there about what engineering managers actually do, from pithy one liners like “nothing” or “meetings” to detailed expectations that sound like a superset of tech lead, project manager, and people manager roles. When I got started in management I quickly found that at fast growing companies people will happily flow as much work as possible to EMs, and it’s important to have a clear idea of what your job really is, because the workload will quickly become larger than one person can do – at which point effective managers excel primarily because of good time management and prioritization. Those skills start by understanding your core responsibilities.

In High Output Management Andy Grove summarizes a manager’s output as the sum of the output of his team and the output of neighboring teams that he influences. A manager’s goal then is to maximize those outputs – a framing that is helpful to come back to when it comes to prioritizing work. Making good choices of what to work on is a different article, but for now I’d like to consider a semi-comprehensive list of the different areas a manager can focus to influence those outputs. In my career and reading I’ve seen a pretty consistent set of 3 high level areas that effective managers invest in: facilitating information flow, driving progress forward, and building a sustainably productive team.

Facilitating information flow includes a mix of information gathering and information sharing. Engineering Managers are responsible for doing the work to understand what is happening in their team and also understand what is happening in the broader organization. Effective EMs then find useful ways to synthesize and share that information to help their teams be more effective and give other teams and executives clarity into what is happening with their team. There are a multitude of anti-patterns here including undersharing or oversharing in either direction, manipulating people via information flow, and failing to extract and emphasize the most critical pieces of information. But doing this right means bringing clarity to everyone an EM works with, pushing the most important information to the people who need it and making it easy for people to pull other information when needed.

Tools for facilitating information flow include 1 on 1 meetings, building or advocating for processes and tools that expose team and project status clearly, encapsulating decisions and reasoning in documents that can be shared widely and referenced later, and asking good questions in standups and project meetings that lead to better understanding of problems and any misalignments between the team and the business. Getting good at this stuff means being a good written and verbal communicator, keeping an attitude of curiousity and humility that lets you learn from others and stay up to date with what is going on, and being intentional about building relationships with all of the key people you work with.

Driving progress forward breaks down into 3 parts: decision influencing, delegating/monitoring and “doing the work”. The first item here is phrased as decision influencing rather than decision making because while you will have to make many decisions as an Engineering Manager it is even more common and in many cases more desirable that you’ll be providing input and influence on decisions but not ultimately “owning” the decision yourself. Delegating work to others is an important part of getting things done and scaling yourself as a manager, but to again quote Andy Grove “delegation without follow-through is abdication”. So even delegated tasks still require the work of monitoring and supporting, and thus take time. Finally, EMs sometimes take on work that is actually directly part of their teams day to day – writing code for a feature, designing architecture for an upcoming project, or debugging a production issue. That last point is the most controversial – there are plenty of “engineering managers shouldn’t code” zealots out there. I simply add it here as something an EM can do to increase their team’s output. Whether it is an effective or impactful activity is going to be highly context specific and is out of the scope of this article.

Progress on a team’s work has the shortest feedback loop of the 3 top level areas: missing information can often go unnoticed for a while, and unhappy teams' troubles may take a while to manifest. But everybody notices when your team is not succeeding at it’s goals. Thus the anti-patterns here are often trading off team and communication for an appearance of progress: doing too much work yourself at the expense of your team’s development and long term scalability, failing to take responsibility for your team’s output – often by delegating without supporting a teammate, or making decisions that optimize for making yourself look good at the expense of your team or other parts of your organization. Getting really good at driving progress as an engineering manager means learning to delegate with an appropriate amount of support and monitoring, learning how to influence work usefully without having to own it end to end and being strategic about what you yourself try to own.

Building a sustainably productive team involves a mix of hiring, culture shaping, team development, and what Michael Lopp calls the care and feeding of engineers. Hiring new engineers is arguably the highest leverage activity we undertake as engineering managers – it’s a one time process that will have impacts on your team for years. Engineering managers have outsized influence on their team’s culture: what work is valued, what the expectations are for meeting preparation and timeliness, how people are treated, what type of people and ideas are welcome. This is often influenced as much by what EMs neglect as what they do. EMs can help their teammates develop through thoughtful feedback, coaching, sponsorship and training. And we work in an industry where our teammates usually have many great work options – setting aside the moral case for being a compassionate manager, managers that do not care for their employees, understand their needs, concerns and ambitions, and help them thrive are unlikely to develop a sustainably productive team over any length of time.

The biggest anti-pattern here is simply neglecting the work – I’ve worked for and around managers who simply didn’t take the time to regularly talk with their team, were not mindful of the impact their actions had on culture, or treated their employees as replacable cogs in an industrial machine. Even hiring is often not given the investment it deserves based on impact: many managers are content to offload sourcing and evaluation to other people or put minimal time into preparing for interviews and new job postings. Getting really good at building a sustainable team means thinking of yourself as a role model for the behavior you want to see, investing heavily in hiring and supporting your current team, regularly giving and seeking meaningful feedback and being organized about helping your team in their career growth.

It’s all about outcomes

I labeled this list as semi-comprehensive because while most of the valuable work I do as an engineering manager falls under one of the categories above, managing engineers is fundamentally different than the type of work someone who has just been promoted from a mid-senior engineering role might be used to. Instead of taking tasks or projects to completion, you’re now operating at a level of an organization where the expected deliverables get a little hazier and you’re being asked to achieve higher level business outcomes.

So don’t be afraid to shift the time you spend on different parts of this list over time or even do work that seems a bit far afield from your role. If it’s grounded in a desire to produce good outcomes for the business and help your team and it won’t get done without you stepping in, consider it part of the job.

Driving Change Without Authority

Mon, 18 Jan 2021 20:00:00 -0400

I recently had a conversation with a coworker where they mentioned that they didn’t feel like they had the same power to change other people’s behavior that I did since I was a manager. It’s a common sentiment that has some truth but usually will make a manager start laughing or crying if you suggest that it represents reality. Managing people well involves very little imposing your will on a situation and a lot more driving change indirectly. The good news is that most of the techniques for this type of indirect change not only can be used by non-managers, many of them are actually more effective coming from engineers directly. A lot of these techniques deserve posts of their own and I intend to keep these short, so consider this a brainstorming list for the next time you want to change something on your team and don’t feel empowered to do so.

Build the thing you want to exist

A software engineer’s ultimate power move is the ability to create new things. If you’re in an argument on the best path forward for a project, just going ahead and building the thing you think should exist can be incredibly powerful. This could be a proof of concept for larger things, or the full thing for smaller features. Opposition and bikeshedding often melts away in the face of a working example of the future. This is especially effective if the argument has been about whether your preferred solution is possible.

Note that this path can be overused, is helped along by a gentle touch, and I wouldn’t recommend it if you put so much time and effort into it that you’d be crushed if people still said no. But it is a powerful way to move discussions forward.

Make it easier

Do you want everybody to write more/better tests? Create some best practice examples and documentation. Maybe make some training videos showing how you write tests. Write some helpers to encompass common testing patterns. Make sure your testing tools are updated to versions with the latest features, and consider whether there are different tools you can use to make adoption easier. This type of work is also great for establishing yourself as an expert in the area (see below).

Make your preference a default

Similar to making it easier, you can often change behavior simply by making it the path of least resistance. Want people to use spaces instead of tabs? Encode that into a linter and set it to run against every Pull Request. Want to make sure people add information to new issues? Create an issue template. Humans are very susceptible to nudges and default behaviors, and are much more willing to accept new behaviors when they don’t require conscious action from them.

Identify allies and opponents

If you want to change behavior in your team, it can be helpful to have multiple people fighting for a change. A great place to start here is just identifying who agrees with you. Not everyone who is willing to accept a change might want to fight for it as much as you, but establishing an understanding of where people stand is helpful for you to identify potential allies as well as people who you may have to work harder to convince.

When you know who disagrees with you, those people can actually be the most helpful in helping you move a change forward if they’re willing to give clear feedback, since they’ll help establish the conditions under which a team might accept a change.

If you find that you’re primarily fighting apathy rather than opposition, it can be a good time to pivot to other methods like “making it a default” or “show the future”.

Show the future

In his 1936 classic How To Win Friends and Influence People, Dale Carnegie said that one of the fundamental techniques in working with people was “Arouse in the other person an eager want”. When we want to drive forward a change and are faced with apathy, often times the issue is that other people don’t understand why the change we’re advocating would be valuable for them.

A great technique in this case is to be crystal clear on what we expect to get from the change – and to do so in a way that resonates with them. If we want to have more automated tests, we might explain to our fellow engineers that more tests can help prevent future rolled back releases, resulting in less crunch time work. Meanwhile, our product manager might be more interested to know that it will result in fewer angry customers and missed deadlines due to rework.

It’s important not to oversell here. If you make it sound like adding a few tests will remove all quality issues with your product this route can backfire with the team quickly dropping support when the initial results underwhelm. But understanding and explaining the “why” in a way that resonates with every relevant teammate is a powerful technique.

Ask authentic questions

When you are looking to change things and it feels like you’re the only one who sees the value in what you’re doing a great place to start can be asking authentic questions of others. For instance, if we went back to our testing example you might ask “Do you think writing more tests would be valuable?”, “why do you think we don’t write more tests now”, or “how do you think our test suite should evolve from here?”.

If asked sincerely those questions will help you refine your understanding of the challenge in front of you, will open the minds of the people you’re asking toward the challenge, and may win some converts just by including people. The sincerely part is important here though - people can generally tell when you’re actually listening to what they have to say versus including perfunctory “feedback requests” on your way to pushing an agenda.

Work in “public”

When you’re trying to encourage a particular practice it can be helpful to demonstrate the work “in public”. So if you want people writing more tests, share screenshares of yourself writing tests, run a brown bag lunch where you do some live coding, or share your test PRs in a public slack channel where people will see it. You might even write public blog posts about the work you’re doing, or speak about it at a meetup.

There are a few goals here: to show people what it looks like to do the work, to “normalize” the practice, and to give it visibility. Doing work publically like this can be a way to attract other interested folks who you might not have expected to be sympathetic, it can raise your own profile, and it can be a way to get great feedback.

Explain your work

One mistake I see people who are trying to drive change make is an assumption that everybody understands what they’re doing. When you’re first introducing a new technology, process, or improvement to your team you should always assume that you need to make the case for it, starting with clearly explaining what the change is and what you’re intending to accomplish with it.

This technique can be combined with many of the other ones here, and can be accomplished in many ways. A short RFC style document where you outline the change you want can be great. Putting extra detailed comments and descriptions into pull requests where you’re implementing a desired change can be helpful, as can screencasts, blog posts, or brown bags. For smaller changes this may just mean going out of your way to be extra explicit about the “why” when you make pull request comments requesting people to move in a certain direction.

Establish your expertise

As I said at the beginning, “change by authority” is not generally the best path. People are happier doing things because they see the value, not because somebody told them to do so. That said, sometimes change does need to come through authority. When that is the case though, there are two types of relevant authority in most engineering organizations. Hierarchical authority (IE directives from somebody with a managerial title above you in a reporting relationship) and expertise authority. While hierarchical authority changes slowly and comes with all sorts of other pros and cons, anyone can develop the authority of expertise with a level of dedication and work.

If you want to drive testing culture on your team it can be helpful to establish yourself as an expert on testing. The bar for this might be quite high or quite low depending on your existing culture, but the path is similar either way; do the work consistently over time, educate yourself, and demonstrate the impact of the work you’re doing. A little bit of this might not go very far, and if the organization explicitly doesn’t value the change you’re trying to make it is possible that expertise won’t change the equation here. But if there is willingness to consider growth, establishing yourself as an expert will ensure that you will be part of the conversations when it is time to make a change, and you’ll get to contribute to making change happen.

Play Other Positions!

Thu, 31 Dec 2020 20:00:00 -0400

This fall ESPN published an article about Steelers QB Ben Roethlisberger’s high school stint as a Wide Receiver and how it benefited him as he moved forward in his career.

Before he set records as a quarterback his senior year, before he became the Pittsburgh Steelers' top draft pick out of Miami (Ohio), and before he won two Super Bowl rings as one of the NFL’s best signal-callers, Roethlisberger spent his junior year of high school playing wide receiver.

It gave Roethlisberger a deeper understanding of the game he took with him the rest of his career – something he found again last season as he stood on the sideline following a season-ending elbow injury in Week 2.

“[Findlay] coach [Cliff Hite] was always like, ‘It benefited you as a senior playing quarterback from playing wide receiver your junior year,'” Roethlisberger said. “I never understood it at the time, but I think it does to a certain extent because you can see the other side of it.

The article highlights 2 advantages of the previous receiver experience for Roethlisberger.

Roethlisberger’s stint as a receiver enables him to be clear with his instructions. Nothing gets lost in translation when the quarterback can give his directives in a way the receiver clearly understands.

“He can talk our language,” said Ryan Switzer, who played with Roethlisberger in Pittsburgh for two seasons. “He can talk leverage, he can talk technique, he can talk lean and press, he can talk all these things, which makes the communication a lot more fluid between us and him. … He’s able to communicate really well, which helps translate to being on the same page more often on the field.”

The carryover from one position to another goes beyond communication. Iriti sees it in the way Roethlisberger eludes the pass rush and finds a receiver on a broken play.

“It’s like he knows where he’s at, and he knows where you’re trying to go, so he can wait until that last second, just make that little move,” Iriti said. “It’s not fast, it’s not quick, but it’s like it’s premeditated that he knows where you’re going. So it makes him seem more agile and quick than he actually is. He’s not a blazing-fast guy, but instinctively he’s really smart.”

Playing a position that he later had to collaborate with allowed Roethlisberger to communicate with his teammates better and make better adjustments when things went wrong and he needed to understand or anticipate his teammates reactions.

Reading Roethlisberger’s stories reminded me of some of my own practices as a high school basketball player. Our coaches would force us to switch positions around when practicing plays. The theory was that you didn’t really know a play until you could execute on every position’s part of it. As a tall but gawky and uncoordinated high schooler, I was never likely to move away from my forward position to become a full time point guard, but the exercise was useful both for the reasons called out above, as well as the flexibility it gave us to play different lineups for shorter stretches during a game. If we had problems with injuries or foul trouble, we could get away with playing people at a different position for some stretch of time, or play different groupings of players that let us take advantage of matchups.

So what does any of this have to do with the purported topic of this blog, productive software teams? It turns out that modern software teams have a lot in common with sports teams. They’re a collection of individuals with different skillsets who take on different roles to work together towards a set of goals over time. Our work has less black and white scorekeeping, and the lines between “games” are a lot blurrier, but there is a lot we can learn from the world of sports.

The lessons from my high school basketball days have stuck with me as I’ve progressed through my software career. I find that I collaborate better with my teammates when I have some sense of what it is like to do their job. A benefit of spending the last 7 years in startups was the opportunity to wear a lot of different hats on a short term basis. I’ve always done an mix of front end and backend development. I once was assigned the role of being primary test engineer for a major release while our normal QA lead was taking a few weeks off out of the country. As my career advanced I got to get more involved with the product management side of things, collaborating closely with PMs to help shape how products got built. I even spent a memorable and painful couple months as my team’s primary designer after a round of layoffs. As both an individual contributor and an engineering manager, these experiences have helped me work well with a variety of different people, identify times when teammates in different roles were talking past each other, and plug gaps when the need arose. It also gives me empathy for people who are facing the tough parts of their job, and a level of humility when approaching discussions that comes from knowing there are always many perspectives on a particular problem or process.

Unlike sports, we don’t usually consciously “practice” in software development. That may be a mistake, but given it is the world we live in, how do we actually develop this type of experience? I’ve found 3 primary things helpful: doing the work, talking with people experienced in the role, and reading about the role. This isn’t that different than the learning process if you actually wanted to pursue a particular role, the time commitment is just lower.

If you have the ability to just do a piece of a project in a different role or fill in for someone for a short time, that can be the easiest way to quickly develop some intuition about a role. For some role combos that’s pretty easy. A software engineer can always take a shot at doing more of a test engineer role. Look at another developer’s code and think about how you would test it. You probably won’t be doing what a full time test engineer would do, but just getting into the mindset can be helpful. It can feel harder to cover other roles - how would you “try out” being an engineering manager as a junior engineer? But you might be able to ask the person in that role if there’s some part of their job you can help them with. Then you get to try out a piece of the role and potentially get some guidance to go with it.

One problem with “just doing it” is that when we first try a new skill or role, we don’t know what we don’t know. It’s easy to perform a parody of the role if we don’t get any meaningful feedback on our performance. So it is helpful to talk with somebody who is experienced in the role, ideally an expert. They can give you feedback on your attempts or just tell you about what they do and how they see the world. The ability to listen well and ask good questions can take you far here.

Finally, there are a lot of written resources out there for learning about different roles. To help work with people in a certain role it can help to read some of the “classic” essays or books about a role. A learning-focused person in the role will be able to make recommendations, but if you don’t have a good resource there you can get a similar effect with careful googling for “best books for product managers” or similar. Some books I’ve found helpful for understanding different roles include Inspired for product managers, The Manager’s Path for engineering managers, and The Design of Everyday Things for product designers.

For pretty much every piece of career advice out there, there are pathological variations that should be avoided. Almost everything can be taken too far or stripped of context to the point where it becomes harmful. I’ve seen at least one common way that the pursuit of understanding our teammates roles can go wrong. That is simple ego; it can be tempting to believe that because we’ve walked a mile in somebody’s shoes we’re the experts and our voices should now hold heavy weight when making decisions in our teammates area of expertise. We should always be listening to our teammates and taking in different opinions. However, it is still valuable to have “owners” for a role, and spending a week or a month or even a year working in a role is not the same as making it your career. There are going to be times where a more experienced team will need to support somebody in their role, especially if they’re new. But that’s not the same as you “being able to do it better” or “having veto power”. This can get quite bad; I’ve known engineers who genuinely believed that they could step in and do any role in their company better than the people who had made it their careers. This is the opposite of the empathy we’re trying to create.

I’m wary about aggrandizing this advice. Trying other roles is a simple technique for working better with your teammates and it isn’t going to change the world. But I’m writing this on the first day of 2021. 2020 was a year that was marked by a lot of suffering and struggle across our world, and also one that seemed to me to lead many people away from empathy and connection and toward conflict. Consider this as a simple way to build empathy and connection with your teammates this year.

2020 Roundup

Tue, 29 Dec 2020 20:00:00 -0400

2020 was the 7th full year of this blog, and it was by far the quietest in terms of output. For reasons that are both obvious (global pandemic!) and personal (toddler, puppy, still learning a new job) writing has been on the backburner for a year. I hope to change that in 2021, but with another child on the way in February I’m not foolish enough to make any promises here.

I only published 5 posts this year, with the posts generally focused on what I was reading and learning as an engineering manager.

Simple Burnout Triage - This post got picked up by The Changelog which led to a pretty large spike – but it also is obviously a topic that a lot of people are working through in 2020. Given that this got 70x more views than any of my other new posts this year, it can hog the summary spotlight all by itself this year.

Languages and Libraries I used a lot of in 2020

JavaScript (ES6 + TypeScript), React, Node + Express, lodash, Jest, Redux

But realistically this was my first year spending significantly more time focused on people & product management than hands on software work

Tech I want to try in 2021

My code time is limited these days – but I’d like to go deeper into some of the cool new stuff around React like Server Components, and also learn Terraform.

Blogs I started following in 2020

The Beautiful Mess by John Cutler

Blogs I read every post from in 2020

Software I used (nearly) every day in 2020

MacOS/iOS, Google Search, Chrome, GMail/MailPlane 3, Twitter/Tweetbot, Feedbin/Reeder, iMessage, Slack, Jira, 1Password, VS Code, iTerm, Fish Shell, Github, Fantastical, Spotify, Bear -> Roam Research, CloudApp, Dash, Postman, Complice

Best Technical Books I read in 2020

Designing Data-Intensive Applications - An information rich exploration of the technologies behind modern “data heavy” information systems. I learned a lot from this and expect that I’ll be referencing it regularly in the future
Site Reliability Engineering - A set of essays describing how Google maintains their production systems that has served as a template for the SRE role at other companies. Was an interesting read, even as I’m skeptical of applying “gigantic-corp” strategies at smaller or less technically savvy companies. With essays like this I always find it more interesting to understand the problems that companies were seeing and how they approached finding a solution than I am with the details of the final solution, and this book does a good job laying that out.

Best Technical Leadership Books I Read in 2020

How To Measure Anything - This was a great read on how to think about measuring things that defy easy measurement. It starts with a lot of great general principals before going into some more technical / detailed topics in the back half of the book. Most people could probably benefit from reading at least the opening chapters of this book, with the rest open to skimming based on interests.
Inspired - This was probably my favorite “work book” I read all year. It’s aimed at product managers, but was helpful for me as an engineering manager to understand what “great product process” looks like and think through how I can work with PMs well and collaborate on our shared goal of building a great product.
Making Work Visible - A breezy walkthrough of the principles behind Kanban. I don’t use Kanban to organize my teams at work, but because of the focus on foundational ideas I still found this bok extremely helpful.
Team Topologies - This is pretty specific and nerdy, but for engineering leaders looking to figure out the best way to evolve teams and split off new ones over time as their companies grow and change, this is a great read. It argues persuasively for the importance of team structure to productivity and product success. Highly recommended.

Best Non-Technical Books I read in 2020

Andrew Carnegie - A detailed biography of a fascinating historical figure who defied easy characterization. Carnegie was an immigrant born in poverty who through hard work and good timing became one of the richest men in the world, a union buster who viewed himself as a “friend of the workers”, a philantropist who wanted to give away his whole fortune, and a political force who aimed for world peace – and protection for his own interests. It was a great reminder in the sometimes overly black and white world of 2020 that real humans are complicated and don’t fit neatly in our tight political boxes.
Salt, Fat, Acid, Heat - I’m not the primary cook in my family – I’m blessed to regularly enjoy my wife’s far superior cooking. But I enjoy it as an occasional activity and this book helped spark even more interest. Unlike most cookbooks this book is focused on the science and “whys” of cooking rather than procedural formulas for making a particular dish. It felt empowering and exciting rather than my typical frustration with recipes that didn’t turn out like the images in the cookbook.
Atomic Habits - Over the last few years I’ve made a personal transition to focusing more on the habits and routines I’m making part of my life and less on ambitious long term goals. This book gave me a framework for thinking about that transition as well as practical tips for doing it better. A quick and helpful read.
How To Talk So Little Kids Will Listen - An incredibly practical book for how to interact with toddlers. I’ve found a bunch of helpful parenting books, but none beat this for giving parents practical actionable advice that will reduce stress in your house and help you build a relationship with your kids.
House of X / Powers of X - For the comics fans out there, I’ve been really enjoying Jonathan Hickman’s run on the X-Men, starting with this graphic novel that soft-reboots the whole world of mutants

How Teams Go Faster

Fri, 04 Sep 2020 20:00:00 -0400

I recently discovered John Cutler’s Twitter and blog. Everything I’ve read from him so far is excellent and highly recommended, but this table in particular felt like it was worth sharing.

13/13 Feels like fast ... is actually faster pic.twitter.com/c87USFeu6b
— John Cutler (@johncutlefish) June 30, 2019

It’s amazing to me how unhelpful our intuition often is when we’re under pressure to speed up. I’ve observed clear examples of several of these items personally over the last several years.

I’ve seen my team go significantly faster with less stress by shifting projects to being serialized with pairing and collaboration rather than giving everyone their own project ¹
I’ve watched attempts to get ahead on planning be thwarted by changing business priorities. This can be frustrating in the moment until you realize the problem is a planning process that can’t adapt to new information cleanly.
I’ve been part of another team that doubled in size and then got cut back in half without much discernable impact in actual output over time.
The most effective change my team has made this year has been de-emphasizing filling sprints in favor of being clear about what we’re trying to accomplish in the sprint. I don’t think we’ve gone far enough in that direction yet.

Some items I’d add to John’s list:

Manual QA / Deployments vs Test Automation as part of development²
Senior devs filling up on feature work vs Senior devs mentoring, enabling and unblocking other devs³

Besides the creative and redundancy benefits of pairing, shared resources like PMs/EMs/QA become bottlenecks as you parallelize. ↩︎
This relates to the “handing off” QA point in John’s list. If the original people responsible for creating the feature aren’t writing automation as part of development, there is likely to be inefficiency or gaps in the testing. ↩︎
Even worse if this is a manager doing this, something I struggled with as I transitioned from being a dev-manager hybrid to managing a larger team. ↩︎

Simple Burnout Triage

Sun, 30 Aug 2020 20:00:00 -0400

I have a one question triage process when I’m concerned that I or one of my teammates is headed towards burnout:

If you take the pace & quality of the last 2 months of your life and repeated it again and again, how long would you be able to sustain it?

Why 2 months? It’s long enough to be a representative sample of your pace and cut out the rush of a specific project deadline or medium sized stressful life event¹. But it’s a small enough sample to check in regularly, and most people have a fairly clear sense of what their last 2 months have been like for them and can visualize what repeating it would look like.

When I ask myself or a teammate this question, it tends to elicit one of 3 responses:

”I can’t go on like this.” The current pace is unsustainable and we’re probably already well into the burnout phase where tasks are being dropped, personal needs are not being met, and everyday life has become overwhelming.

”I can make this work, but…” The most interesting result of this question to me is that it often makes it clear where a priority is being dropped. “I’m doing good, but I wish I was doing a better job making time for my kids”. “I’m getting by, but I feel like I only have time for the urgent and I haven’t had a chance to invest in myself”. “I can do this for a while longer, but I have to figure out a way to live in the moment in the evenings and leave my work at the office”. The act of visualizing a plausible future often causes people to hone in on what they’re missing.

”I love my life, and I can keep doing this forever.” If this is where you’re at, treasure it and check in regularly to make sure it’s still the case.

What next?

The question above is laser focused on triage. Exactly how to handle partial or complete burnout is situation dependent. But I try to hold to principles of optimizing for the long term and tying my time to my values. That means while it sometimes may make sense to live in an unsustainable place for a season, I’m always going to weigh that impact on myself, my family and my teammates against the longterm value and take it seriously². And if my time spent doesn’t line up with my principles and values for a significant length of time, I know it’s time to implement a plan for change.

For large life changes like job switches, family emergencies, society-changing pandemics etc it is probably better to assume that short term data is not representative, err on the side of scaling back as needed, and take stock of things on a regular basis. ↩︎
A note on taking things on for a season; It’s only a season if it has a defined endpoint or offramp. If you and your spouse live in different cities to optimize for your career, or you’re working 70 hour work-weeks because your job demands it, and you don’t know when these things will end, you’re not in a season. This is your life. ↩︎

Better than a wild guess

Sat, 22 Aug 2020 20:00:00 -0400

I’m currently reading Doublas Hubbard’s How To Measure Anything and it is driving home a point I’ve observed anecdotally throughout my career; when people don’t know something exactly, they often throw away a ton of things that they do know. You can see this in project estimation, where engineers often resist providing any estimate since “it’s hard to tell before we actually build it”. You can see it in product planning, where teams I’ve worked with struggle to provide clear usage or business impact goals since “it is hard to tell how impactful this will be till we build it”.

Hubbard’s advice for making properly calibrated estimates would probably help in multiple places in the software world:

Use 90% confidence intervals when there’s uncertainty, not point estimates or vague categories like “medium risk” - Saying a piece of software work is “large” is much less actionable and informative than “we’re 90% confident that it will take between 3 and 15 days to complete.
Use absurd scenarios and alternative bets to calibrate a confidence interval - You may think you have no idea how long a novel new software project will take, but even for projects with high uncertainty this is untrue. You can start by asking questions like “is there any chance this will be done tomorrow” and “is there any chance this project will take more than 5 years”. If the answer is no, then you’ve established a baseline confidence interval, and you can start working inward from there. When you get to the point that something is feasible if unlikely, its a sign that you’re probably near the 90% confidence interval¹. Once you have a starting place, you can further consider a hypothetical bet where you had to choose between a lottery with a 90% payout of $1000 and a scenario where you got the $1000 when the item ended in your range. If you would clearly prefer one of those scenarios to the other, your range is probably too tight or too loose.
Model your estimates with simple monte carlo simulations - Hubbard recommends using Monte Carlo simulations (picking random numbers according to the distributions set up by your confidence intervals ~1000 times) in order to get a better intuition of what real world outcomes might look like. Doing this right does mean understanding not just the confidence range, but also what the distribution looks like. Estimating simple software feature work for instance, is likely not a normal distribution. Many tasks will be straightforward and be completed quickly, but we’ll hit occasional tasks that require significant restructuring of a program to complete. It’s a classic “long tail” scenario.
Break down estimates into component parts - One of the most powerful techniques for making better estimates under uncertainty is “Fermi Decomposition”, the process of breaking an estimation problem into smaller estimation problems (which may be more tractable) and then combining the estimate. In software this can mean breaking up a large project into the component pieces of work and then estimating those. Doing this right of course means capturing our uncertainty on the sub-estimates, so that we can roll that up into the larger estimate. If we optimistically point estimate every element of a project, we’re going to end up with a total estimate that is actually less likely, since we need everything to go well to hit that total estimate. I commonly see engineers handle this by doing optimistic estimates and then adding a buffer period. It would be better to be clear about uncertainty and do your decompositions with a range of values (although it does make sense to factor in uncertainty about tasks that may be missing).
Consider what measurements will be valuable - When we start viewing our estimation goal as reducing uncertainty rather than eliminating it, its easier to see that some easy measurements might go a long way toward improving our projections, even if they won’t take us to an exact value. For instance if we want to know how many tickets we might close during an upcoming sprint, it can be easy to say that there’s no way to know. The new work is unique and isn’t necessarily directly comparable to work we’ve done in the past. However just because we can’t make an exact projection doesn’t mean we know nothing. If we look at our last 10 sprints and see how many tasks we completed in each of them, we can feel confident that this upcoming sprint is likely to be between the minimum and maximum value in that range, or at least close. If we’re currently planning like we will accomplish something outside of that range, this could be very useful information.

All of this definitely has me thinking about the good and bad of software estimation, and how I might incorporate this into my team’s rhythms. Would love to hear stories of effective estimation processes in software. What do you do that is better than wild guessing?

Remember that 90% total interval means that there’s a 95% chance that a value is greater than the lower bound and a 95% chance that the value is less than the upper bound. ↩︎

The Management Resources I Keep Coming Back To

Fri, 24 Jul 2020 20:00:00 -0400

Although I don’t read Hacker News much these days, I do subscribe to a newsletter that rounds up the best links every week, and there are almost always one or two gems in there.

This week my favorite link was this roundup of Essays on Programming I Think About A lot by Ben Kuhn, cataloguing essays on the internet that he comes back to or references repeatedly.

I’m now ~2 years into managing people and because Engineering Management is not a promotion, I’ve spent a lot of time leading up to that change and since reading about management in an attempt to avoid being completely clueless at my job¹.

Like Ben, I have a few key essays and books that I reference and go back to over and over again. These are a mix of books and essays that have shaped how I think about the job of engineering leadership. The books here are mostly well known, while the essays are more obscure, and might be a surprising choice to highlight if you asked their authors. But each one has taught me something specific that has helped shape my leadership style.

Essays

Leadership Style Colors | Lara Hogan

It’s totally natural to embody one a lot—that’s your style! But it’s important to be aware of the other styles/approaches available in your leadership toolbox. A strong leader understands that different styles can be valuable when an environment or situation calls for it. As Herminia Ibarra says in my favorite article on this subject, “Small changes—in the way we carry ourselves, the way we communicate, the way we interact—often make a world of difference in how effectively we lead.”

Taste The Soup | Rands

In a meeting where an individual or team is presenting a complex idea or project, my job as the leader is soup tasting. It’s sampling critical parts of the idea to get a sense of how this soup has been or will be made. Who are the critical people? What are the critical parts? Which decisions matter? I don’t know. I do believe that a pre-requisite for leadership is that you have experience. You’ve had trials which have resulted in both impressive successes and majestic failures. These aggregate lessons define your metaphoric soup tasting ability, and when your team brings you a topic to review, it is this experience you apply to ask the critical soup questions.

Engineering Management: The Pendulum Or The Ladder | Charity Majors

If you like engineering management, your tendency is to go “cool, now I’m a manager”, and move from job to job as an engineering manager, managing team after team of engineers. But this is a trap. It is not a sound long term plan. It leads too many people off to a place they never wanted to end up: technically sidelined.

Trapped in a Values Oasis | Will Larson

The rule of thumb here is to lead through ambiguity, and advocate through disagreement. It’s important to diagnose your situation correctly, because when you get it wrong, it’ll still feel like you’re making progress, but it’s wholly dependent on you and it’s progress that is likely to come at the cost of undermining both you and your team within the broader organization. It can be extraordinarily frustrating to “disagree and commit” to a policy or value that goes against your personal values, but any worthwhile measure of successful leadership needs to consider your team’s success more highly than your own.

Friday Wins & A Case Study in Ritual Design | Kellan Elliott-McCrea

Culture is what you celebrate. Rituals are the tools you use to shape culture. Yet very few of us think much about ritual design.

Books

Radical Candor | Kim Scott

“It’s brutally hard to tell people when they are screwing up. You don’t want to hurt anyone’s feelings; that’s because you’re not a sadist. You don’t want that person or the rest of the team to think you’re a jerk. Plus, you’ve been told since you learned to talk, “If you don’t have anything nice to say, don’t say anything at all.” Now all of a sudden it’s your job to say it. You’ve got to undo a lifetime of training. Management is hard.”

The best thing I’ve read on how to give feedback well.

Death By Meeting | Patrick Lencioni

The hard truth is, bad meetings almost always lead to bad decisions, which is the best recipe for mediocrity.

A lot of this book is focused on very specific advice for specific types of meetings, but the philosophical takeaway it left me with was much simpler; meetings aren’t non-negotiable things that we all must endure. They can be designed, and that design matters.

The Common Rule | James Early

All those who want to be attentive to who they are becoming must realize that formation begins with a framework of habits.

This is a book written through the lens of the author’s Christian faith, so if that bothers you, you might prefer Deep Work, which covers similar ground in a secular manner. But for me personally, the rituals and worldview laid out in the Common Rule has been life changing, and has been my most helpful tool as I navigate the “Advanced Time Management 501” lifestyle of engineering management

That’s the list for now, but I’m sure I’ll add more as time goes on.

I would like to think this has worked, but Dunning-Kruger being what it is, I don’t know how I could really tell you with confidence. ↩︎

Fun with Ascii: Putting some console art on my site

Tue, 31 Dec 2019 20:00:00 -0400

I’ve always enjoyed sites that had easter eggs in their consoles, whether they’re simple and whimsical like Ponyfoo.com

or practical like Facebook’s

I’m also part of a fast growing company and we’re always hiring, so I thought I’d see if I could leave a fun message in the console, and play around with ASCII art while I was at it.

Kustomer’s logo is Kusty; a smily speech bubble with apostrophes for eyes:

So how did I get Kusty showing up in the console on this site? I started with a bit of a cheat: I asciified the logo using asciify.me/. That got me a nice starting place, I had an Ascii version of Kusty to work with:

 ' ``.'` . -_+")cTtfyyyszztszszszztzfztfsfzsfzsstttyt17vr>, .   ` '-`- `
  ..`.'`.~)71tyyzzfzfzsyststztyftsytsystyztzzyzsyyytyfsytf1l?!  -.` `'`.
-.- - `+}Tzztsfyffztzzyyztsffyytfyyzztyfzyfzyfystzftszfytfysyt{+--'.- `.
 `--.>Lyffttyztsfyztyt11fYtyyzYTz1yT1yfzfTt1TsYfftsttzzysffssyyt7+  ''`-
 -`!iysttfstffzTi?"+,,!~^~!^__~!_:,:^^^,!~_~~:_,,!:^+/i1zyytfyztzzi~''-.
. !}tsftzffyTv+~..  -`-.' '. .`-   .'.-- ` ` . -'- .-..~+v1fsfttszf}< . 
 ,xftyyysszv, -. ```' '. .-.``'.-   ''.'--'- '  ` ...''  .!vytyzsstsx^- 
'/zsstzzzf*` `` '  ` `- '`- ` `.`  -.-   - `.'-- -` - '` '``=1szfyzssi' 
+Yzftsztf? `  `-` ` -- ` "vje4J>' ...` `  '~iteau(.'` `-- ``'/tzszszfY+'
isttsstzl:     `.-  .`'[qQQRgQBt  - `-.' rkDQWRQg6  -''- `   ^]ffyzstyi'
xyfssytyv''- .''-   .:#QBW&j}v+'  '`. `'s@gR@$Lv"-``   - - .'`?ystysztL`
xtyytzsyv .`-.'-  `' JRgBV:...``.' -.  [HWHd+  .'-- `-     '' /ysyzftsl'
cyyfysfyi  '-`'``' -`NBWW^'    .'`` - '2gBQ{`    .-`-'. . `-'.(zsyyzzzL_ 
{ytyztfti    '-`'..''DBRWSS%9wk"-  .-  nWWHOwE%%5x    `' -  `'|ftzttytx- 
{ttszfysi.`-' `.'' -.DWHgggBRgH2-`.''  2BWHgWRHgB&' .`. '  . '/sfztfsyc` 
7zzsfsyzi - '`  '' ` &BHgQWRgBHj-'- ' 'JQWHBBHHHQ@   ` .-```- /ttfttyy7' 
lsfsttfti-'.`  -` -'`DBHgHHRWRHu.. `'`.nWHRgRRBgg&.`'` '`   '-?fsysfss7.
xstfsytti `- `.'''`.'pBBRQQRBgBn. '' ' CQBWBQBBBQO--'`   - -- )sstfzyz] 
}fzztyfsv.  ' -   '..+[yfsfyzyv!-    ` !ittfftztc+' '  '  -. ')zzzzstsL`
7ttszfssi'- '-  '`  -``-- - -`` -.-.-`` `` '..- . .''- .  '.- /sfyyftsL`
{fsyysyzv-' ' .-  '`  '    - - -  ```-...' .' -  - .-'`-.'-  `/fstyzszl'
7{yzzyttfi '- '.'.' ' ' . - - . -'  -'-- ''`  .  -'   `'- .  '`(ffstfss}`
{ztfzttsi -   -' ..'` ``'`.--  .  -. '. .'''   '  -' -''.-. ' )zssfyzs]-
{tzszzzyi'.'-`.--  ' `"&QQQHQWWgHWWgBQWWBggBgWW&/ `` '.' '`- -/zfszzzs}-
xyffzzzsi -.`-`'''- ' ']MHgHRHHgRQHBBWQWHHgBQQO1-`.`'.--. ''` |zzztfysl' 
ittsfsttc^`   -  .'   '`~zMHHQWgBQRQgHQWggHQMt:'' '`- .'   `-,{fttttzzi`
+Yzftsstt|'. .  ' .   .  .-vJONHHBWQggBB&MFi. ` ' -.  `.'--  \yyzszyf1>`
 /tfyyyzyf"  - `.`'..``  -`- -+\c1f2yTc/<. '``.-`.`'-.'`-'` rfztyssfsi'' 
 :Ltzzytsstv,- ` '.`'  '` ' ```- '`    .`.- `'. . - ` ``--,itftfsfyt{,'`  
--^LtfsfsftsYi+:'`-. `  .  .-` - `.--''`` -   `. ``'`  ~>iTztyyyfzs[< `.
.. ^iftstzzfsssYv)r,^__!_^!!~'.-'. ..- -'' `,_:_~_~:>)i1zsfzsstfzfi!. `-
.`  .>]ytfyyzsyszftsz1TYY1Y11v>-     -.'-'. {TYTTYstzttfzsfftsstl<'   '`
-.-.-. >lfsszzsyytszysyzzzzysfy1?< '       `7zzffsyystzsttztzfx>` ' - .'
`.' `''`.^(c1zfttfztfyssyyffttzyyf}r~.- '''-7tfyzszzztysfy1L(!-.-`-.`.-`
-'    `' -. !+"([TyyzyzzsztststfsszyYi*_` -`{zyzfzfffT[v">!'`..`-` .`.-`

At this point I could have tossed that into a console.log and called it a day, but I wanted to take this to the next level. And that means color!

This isn’t something you run into every day while writing for the web, but it turns out that you can apply css to text in the console using a %c macro. The format goes like this:

console.log('%cbold text %cred text', 'font-weight:bold', 'color: red')

That code outputs bold text red text.

Of course its a pain to split that giant Kusty ascii block into multiple console logs with different colors. So I minimized the pain and wrote a script that would let me take a set of ranges and render characters in one of 3 colors, and using either solid characters A-Za-z1-9+{}<>; or space characters `.,_-’. I did eat the cost of writing out the map characters manually based on the generated ascii. If I was going to do this again, I’d probably write out a script to generate the map from the original ascii image, and just fill in the color mappings manually.

//we use these characters for the solid lines
const solidChars =
  'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ123456789+{}<>;';
// and these ones for empty space
const spaceChars = " `.,_-'";
// these maps chars and colors
const chars = [spaceChars, solidChars];
const colors = ['#E0E0E0', ‘#ED6159’, ‘black’];
// this is the mapping to generate kusty
const map = [ [13, 0, 0], [45, 1, 1], [13, 0, 0], null, [9, 0, 0], [53, 1, 1] /* etc */ ]
// generate the ascii string from the map
const ascii = map.reduce((agg, val) => {
// nulls mean spaces
if (val === null) return agg + ‘\n’;
agg = agg += ‘%c’;
let characters = chars[val[1]].split('');
const randChar = () =>
characters[Math.floor(Math.random() * characters.length)];
for (let i = 0; i < val[0]; i++) {
agg += randChar();
}
return agg;
}, ‘');
const colorStrings = map
.filter(x => x !== null)
.map(
val => color:${colors[val[2]]}; font-family: monospace, font-weight: bold
);
console.log(ascii, …colorStrings);

The code is fairly straightforward, going through each segment and generating a string of random characters based on the map. The interesting parts are probably:

We use null to denote new lines. Could have been anything, but null is pretty obvious when you write out the mapping markup.
I usually use lodash for this, but I wrote some custom code to pull a random item from an array: const randChar = () => characters[Math.floor(Math.random() * characters.length)];
I added a rare for loop in JavaScript code: even though map/filter/forEach etc work better for most things, a for loop is still the best way to do things when you’re given a number of times you want something to occur.

It works pretty well!

Next Steps

Now that I’ve done this it would be pretty easy to reuse this code to create a terminal Kusty with chalk. Maybe next New Years.

For everyone else, if this seemed fun to you, you might be interested in the job openings at Kustomer. Feel free to apply or reach out to me directly if you’re interested in working with us.

2019 Roundup

Mon, 30 Dec 2019 20:00:00 -0400

2019 was the 6th full year of this blog, and it was a bit up and down posting wise. For the first half of the year I published more or less a post a week, with 24 posts from January till July. But this is only my 2nd post since then.

Turns out that starting a new job while learning to be a parent sucks up a lot of time! I’m not making any promises for 2020, but I expect that my writing rate will fall somewhere in between the first 7 months and last 5 months of 2019. In the meantime, here’s a brief look back at the year on the blog.

This was my first year where I spent the full year managing people, and that was reflected in the blog’s topics as well as where I spent my time.

Reusable Code In React: Inheritance, Composition, Decorators and Mixins The most popular post for the year was also one of my favorites: a dive into the various mechanisms React provides for reusing code.
Improving Code Readability With Async/Await I really like async/await but its not always clear how to move older patterns to use it. This was just a quick example of taking some ugly callback code and converting it to use async await.
What’s involved in Front End Architecture? Another one of my favorite posts to write, this is a look at what types of things developers should be thinking about when they consider front end architecture.

Languages and Libraries I used a lot of in 2019

JavaScript (ES6), React (including hooks), PostgreSQL, lodash, Jest, Immer, CSS Modules, Redux,

Tech I want to try in 2020

We’re starting to roll out TypeScript at Kustomer and I’m excited to use it in production for the first time. My code time is more limited these days, so that will probably be my main area for learning as I focus more on helping other people get stuff done.

Blogs I started following in 2019

Irrational Exuberance! by Will Larson - a great engineering / technical leadership blog

Blogs I read every post from in 2019

Overreacted by Dan Abramov, Lara Hogan’s Engineering Leadership blog and newsletter, Rands In Repose

Software I used (nearly) every day in 2019

MacOS/iOS, Google Search, Chrome/Safari/Firefox, GMail/MailPlane 3, Twitter/Tweetbot, Feedbin/Reeder, iMessage, Slack, Jira, 1Password, VS Code, iTerm, Fish Shell, Github/Bitbucket, Fantastical, Spotify, Bear, CloudApp, Dash, Postman

Best Technical Books I read in 2019

The Pragmatic Programmer 20th Anniversary Edition - A huge update from the 1st edition, this remains my #1 book recommendation for any professional software developer
Accelerate - A great look at the impact improving devops/release practices can have on software teams/companies

Best Technical Leadership Books I Read in 2019

Resilient Management - A great collection of management wisdom from Lara Hogan. Nothing revolutionary here for those who already follow her blog/newsletter, but its great to have it all together in one place.
The Manager’s Path - A super helpful book for anyone thinking about their career path in tech, especially if they’re considering management. Lots of helpful tips for those already managing people as well.
An Elegant Puzzle - Another “blog book”, this one is a bit of a grab bag of topics, but the content is so high quality it was completely worth it.
Turn The Ship Around - Not tech-specific, but one of the best books on leadership I’ve read, in the form of a history of a nuclear submarine crew getting ready to deploy.

Best Non-Technical Books I read in 2019

I Hear You - A small practical book on validation that will make you a better friend / spouse / coworker.
Why We Sleep - A fascinating book on the (pop) science behind sleep
Team Of Rivals - A very readable history of the Lincoln presidency focused on his relationships with the other candidates for President in 1860. Incredibly well done.
Evicted: Poverty and Profit in the American City - A sobering look at how laws, norms and expectations around rentals and evictions in America contribute to spirals of poverty. Another very well written book, make sure to read the end to understand the research that went into it and allowed for the very personal feel.
The Common Rule - This is a Christian look at how ritual and habit can help us overcome modern distractions and focus on the things that are important. I’ve read a lot of books in this genre in recent years (Deep Work / Indistractable / The Tech Wise Family) and this was my favorite.
The Foundation Trilogy - Classic scifi, and a great read. Fascinating to read a mid 1900s view of the future and realize that while space travel and force fields sounded great, it was inconceivable that casual tobacco use in the home might go away.

How Software Engineers Produce Value

Tue, 17 Dec 2019 20:00:00 -0400

I had the opportunity this week to walk a software engineer who is new to the field through some lessons I’ve learned over the years about how software engineers add value to the business. It seemed generally applicable, so here’s a cleaned up version of what I shared.

What Is Value?

It can be easy to think of software engineering work as a series of task where you’re given instructions and you make the computer do what the designer / PM / CEO asked. Or as a series of fun problem solving challenges that you’re somehow getting paid to complete. Both of those views will eventually lead you to make suboptimal decisions when it comes to helping your business and growing in your role though. Instead consider the following definition:

Software Developers create value through shipping working code that solves important user problems in a predictable manner.

This captures a number of important things.

Shipping

Real Artists Ship - Steve Jobs

Software Engineer’s super power compared to other roles is that we get to build. We create new things, add new behaviors, and ultimately (for a product company) control the shape of the product our company is built around. This only has value when software actually ships though. When we’re not shipping, it’s actually worse than just being slow to deliver value, because of a few non-obvious factors.

First, unshipped code rots. When we leave code in a branch 90% done, it doesn’t just stay the same. It’s quite common for our situation to get worse. Our teammates check in code that may cause merge conflicts, or worse just doesn’t take into account the unshipped code and sets us up for subtle bugs. And when we do get back to the work we’ve often lost valuable mental context that makes any remaining work or testing harder and more error-prone.

Second, we learn from shipping! It’s quite common in software development for a developer or team to complete 90% of their feature according to the spec, only to find that they have another 90% of the work to go. Doing the work to push things through to production inherently de-risks things, because we learn what the challenges are, and we know how long the work took. Up until then, we will always have unknowns. And when software goes out to users, we get to hear their feedback as well. Shipped software lets us learn from our users (and from the performance of our system) and make adjustments that we wouldn’t have known about if we sat on the code.

So how do we get to shipping? First, it’s important to push things through to completion rather than allowing ourselves to jump from project to project, making progress without finishing, and incurring a lot of cost from context switching. Soemtimes this may mean imposing kanban style work in progress limits on ourselves. Second, its often helpful to break things into smaller chunks. Small code reviews are easier to complete, test, review and verify. So multiple small changes will often go through faster than the same changes bundled together into a big chunk. This can go too far, but as long as your changes work in isolation and provide value, I’d encourage erring on this side when in doubt.

Working Code

[We Prefer] working software over comprehensive documentation - The Agile Manifesto

Software engineers are responsible for their code working. At the simplest level that means delivering code that doesn’t crash when run in the intended way. As we grow and mature as developers however, we should aspire to learn to write code that will scale to any reasonable load the system might see (and maybe even to unreasonable loads), code that handles odd input or context gracefully, and code that is easy to modify and maintain. We should also take responsibility for proving that it works; writing tests, making the code easy to monitor in production, and providing good logging for debugging all fall under a software engineer’s domain. We also should make sure that our definition of working matches what the business needs, not just a pedantic reading of whatever spec we’re working off of. More on that in a moment.

Solving Important User Problems

Just really funny- What the Customer Wanted expanded pic.twitter.com/kdr7HKTSDy
— Frank Spillers (@farreaching) August 13, 2016

Software Engineers can be quite inventive in the ways they write code without solving important user problems. Some common routes:

Working on their own pet projects over the prioritized business problems - Tech improvement work and engineer-driven projects are part of a healthy organization, but if they consistently replace work that was scoped based on user requests / feedback, they can cause problems
Matching the spec without solving the original problem - This is partially a design/product problem, but its often possible to meet all the stated requirements for a project without solving the original user pain point. Great engineers understand exactly what problem they’re trying to solve with a feature and will always step back to evaluate whether they’re succeeding throughout the development process.
Matching the spec and causing new problems - Oftentimes our efforts to fix cause or expose new problems. When we dodge responsibility by ignoring or hiding those problems instead of openly evaluating tradeoffs or working to improve the solution, we’re not providing all the value we’re capable of.

These are ultimately issues of ownership. When we’re invested in our work, we will want to work on useful problems and make sure they produce real value. If you’re consistently not feeling that way, it may be time to find something else to work on.

In a predictable manner

The challenge and the mission are to find real solutions to real problems on actual schedules with available resources - Fred Brooks

Ok, now we come to the section that is the boring part for many software engineers. But as somebody who has recently converted to the management side of the fence, the ability to ship predictably is incredibly important.

It’s important for software engineers to ship predictably in 2 senses:

There is ongoing understanding of when something is likely to ship among the people who need to know
There is ongoing understanding of how something will work when it will ship among the people who need to know.

Estimating software engineering projects is not an exact science. Nobody reasonable expects estimation perfection from software engineers, especially those new to the field. Instead we aim to be as accurate as possible, and especially to be transparent as those estimates change over time.

When we encounter obstacles, it’s important to be realistic about what they mean to schedules and update the important people in a timely manner.

For those working on a product team that may be a manager and a PM, for those building something solo, that may be the customers or internal users of the software directly. It’s never fun to say something is going to be late, but its always better to say that we’ll miss a date that is 2 weeks away than to tell somebody for the first time that you’ve missed the date that was 2 days ago.

Communicating about the scope of work is just as important as schedule. If we find out a piece of a feature is unfeasable or we dropped it due to time pressure, its important to communicate that. Depending on your team/product, that may mean sharing with people directly, updating documentation, or just making the new behavior clear in an interface.

It’s a process

Nobody starts out their first software engineering job with a fully evolved sense of how to estimate, scope and document their work. Most have no clue how to think about building a system to be scalable or when to escalate problems to other members of the team. These are skills learned through hard experience. Make sure to always be reflecting on what you’re experiencing though. Experience combined with reflection is a powerful mixture, and eventually will lead to growth. Good luck!

Book Review: Resilient Management

Sun, 21 Jul 2019 20:00:00 -0400

Resilient Management by Lara Hogan is a short and useful read for engineering managers who want to think more systematically about how they support their team. Lara covers concepts like how to connect with a new team, how to encourage growth in your teammates, and how to communicate difficult news. This is a blog+ book: most of the material here has appeared previously in Lara’s blog or newsletter, but for the book it has been cleaned up and organized together in a coherent fashion that makes it greater than the sum of its parts.

If you’ve followed my newsletter for a while, you’ll have seen me link to Lara’s work on a regular basis. Lara’s strength as a teacher is building meaningful exercises for thinking through abstract topics, and that plays out here. Almost every piece of advice and insight comes with a suggestion on how to start implementing it, or an exercise to help you think deeper about yourself or your situation. I’ve found her leadership colors and her MadLibs worksheet for leadership philosophy particularly helpful for me.

In addition to the helpful exercises, the highlights of the book for me were Lara’s look at the BICEPS model for understanding people’s needs at work, the way she structured her book around a groups evolution over time, and the chapter on communication plans. I’ve made notes to reference each of those areas the next time I hit a relevant situation.

Overall I’d recommend this book to pretty much any early-career engineering manager. If you’re not already familiar with Lara’s work, there should be some fantastic new ideas for you in here. If you have followed it, I recommend buying out of appreciation, and to get the concepts bundled in a nice cleanly organized packages. These are ideas worth paying for.

Resilient Management is available from A Book Apart

Avoiding Derived State In React

Sun, 23 Jun 2019 20:00:00 -0400

A lot of advice in the React community is focused around avoiding explicitly storing “derived state” of some type or another. Some examples of this type of “derived state” that the community warns against:

Storing data that is directly derived from another piece of data in redux

const reducer = createReducer(initialState, {
  ADD_USER: (state, action) => {
    state.users = [...state.users, action.user];
    // numUsers is derived state: it will always be tied to users
    state.numUsers = state.users.length;
  }
})

Storing state that is directly derived from a prop in a component


const Widget = props => {
    // the hasError state is derived from the error prop
    let [hasError] = useState(!!props.error);
    //... continued ...
}

Storing state that is directly derived from another piece of state

class NameForm extends React.Component {
    constructor(props) {
        super(props);
        this.state = {
            firstName: '',
            lastName: '',
            fullName: '',
        }
    }
    // because fullName is directly associated with firstName and lastName, we have to update it
    // any time they are updated
    updateFirstName(newFirstName) {
        this.setState(state => ({
            ...state,
            firstName: newFirstName,
            fullName: newFirstName + ' ' + state.lastName,
        }))
    }
    updateLastName(newLastName) {
        this.setState(state => ({
            ...state,
            lastName: newLastName,
            fullName: state.firstName + ' ' + newLastName,
        }))
    }
    // ... continued ....
}

These examples are all based on real-world code I’ve seen. And they all were in code that worked, the program executed as expected. But they’re still problematic. This type of derived state is error-prone; when new code is written to update the state, each new programmer touching the code has to understand the relationship, or the whole thing can get out of sync. When state is coming from props, like the error example above, you have to make sure that the state gets updated properly when the props change. And when the state gets out of sync, it is possible to end up in strange states where a programmer’s expectations are violated. A program shows a blank state saying no users have been added, even though one has already been created. A form is expecting to have an error, but the error text is empty or set to a default value. Or a site has 2 different names for an individual that show up differently in separate parts of the interface.

This type of problem isn’t a new thing in Computer Science. When programmers do it by accident they usually call it a DRY violation. When they do it intentionally they call it caching. And when they do it intentionally but screw things up, they make jokes about off by one errors. Fortunately, that means that the tradeoffs are pretty well understood here. So how do we handle situations where it looks like derived state may be necessary?

Just Derive State At Render Time

For many many things, including the 3 examples above, if you need information at render time that can be derived from other pieces of state, it’s totally fine to just calculate it at render time. For the first name example above, you can just include a let fullName = this.state.firstName + ' ' + this.state.lastName; line at the top of your render function and move on¹. Storing derived information is unnecessary and harmful when calculating it at runtime adds minimal performance overhead or repetition.

Memoization

But Ben, my component is slow now!

…are you sure it’s actually slow? Have you tested it? Have you proven that the derived state calculation is actually a meaningful part of this?

Deriving state can be slow when you have to do complicated calculations or reformatting of data. This is still not usually a problem, but if you’re in a performance sensitive area of code and have proven it to be a problem, you can start considering caching solutions. You still don’t want to store your cached values in state or your redux stores though. Instead, you want to use a computed value that is recalculated when its inputs change. The exact code will depend on your state management strategy, but some examples out there include

All of these methods use memoization, a type of caching where arguments are passed to a function, and if the function has been called with those arguments before, it returns the previous result rather than recomputing. If for instance we had a component and wanted to get the names of all users with the current type based on props, and we knew that users was sometimes quite large, making the calculations slow, and we expected the component to re-render often for some reason or another we might write code like this:

const UsersList = ({users, type,}) => {

    let userNames = useMemo(() => users.map(user => user.type === type), [users, type]);
    // ... continued ...
}

Now userNames will automatically be recalculated only when users or type changes². The MobX and Redux solutions work similarly. Memoization isn’t a performance silver bullet: at some point, you might have to look at how you can make your calculations less expensive or take them off of the rendering path completely. But for most simple data transformation tasks, deriving state at render time and using memoization to keep it performant where needed should help you tame any derived state problems you encounter.

But you should probably read about all the ways that names can be more complicated than you think ↩︎
Note that useMemo (and most memoization solutions generally) doesn’t guarantee the function will never run more than once. Most solutions have a limited number of results they will store, so if the inputs change often, the result might get recalculated. ↩︎

Lessons From Managing A Distributed Team

Sun, 16 Jun 2019 20:00:00 -0400

Working with people over the internet is different from meeting in an office five days a week. This isn’t a moral statement. I’ve done and enjoyed both. Working with a distributed team requires a different set of skills, especially when you’re a leader. Today I want to share the most helpful lessons I’ve learned over the last year managing a distributed team.

Evaluate process and outcomes, not inputs

Focusing on output` is great advice for any team, but it’s easier and more important when managing remote teams. When you’re not in the office with people, you won’t see when they start in the morning or finish at night. You won’t be able to tell if they’re taking long coffee breaks or checking Amazon between tasks. You’ll have to evaluate them on their output and on the methods they use to get work done.

Evaluating on outputs is tricky for tech work in the short term. It’s often hard to tell how hard a specific task should have been. But over the long run, you’ll need to find ways to see if developers are meeting their goals, and focus on that over the exact times they did work. And this is good, because that is actually what matters.

Output is a lagging indicator though. In the short term, what you can evaluate if “butts in seats” isn’t an option. I recommend process: is work being documented, are tests being written and are code reviews being done in a timely manner? The nice thing about this is that you can check these things in low touch ways; by reviewing your issue tracking and version control systems rather than having status meetings or sending distracting slack messages. This idea comes up in Camille Fournier’s book The Manager’s Path, where she argues persuasively for always doing your information gathering in ways that don’t disrupt your team first.

1-1s are extra important

Let’s say Sarah has some lingering concerns about her current project that she’s not yet ready to raise to the full group. When I work in the same office with Sarah, she can drop by my desk anytime to talk through a small issue. When we’re working remotely, that may not feel as comfortable for a variety of reasons. Good 1-1s provide space for the little things to come out.

When you get in-person time, focus on the relationship

Currently I get together with my team (and my manager) about once a quarter. Initially my instinct was to save the “big work stuff” for that time. And I still try to get a few big conversations in each time, mostly around career development. But the most value from these visits has come in the gaps, when I go out to lunch with my team and chat, or we get together with a larger group after work. Time spent together builds trust, and that trust lingers and pays off when you’re back on the other side of the internet the next week.

Async communication is great. Don’t neglect “face to face” time.

“Embrace async communication” is common remote work advice. When you get stuff done through email/slack/Github/Invision/etc you have fewer meetings and more flow time for your team. Which is great! My team takes advantage of “async” tools every day. But I also try to be intentional about working in “face to face” time through Zoom. For my team that’s a daily standup and weekly 1-1s, but it could be different for another team’s process. The key is to not only rely on writing. Writing is an amazing way to communicate, but it will always lack the context of the human face. For now I’m aiming for a mix of mostly async with regular “internet face to face”, and occasional “in person”.

Make sure to personalize your communication as well. If Jerry tends to miss all nuance when you communicate in text, try to spend more time talking over zoom to make sure you’re communicating. On the other hand if Jennifer has a tendency to lose track of what you’re asking her to do, you may want to give instructions through a text medium, where you can use it for accountability later.

Hire for remote success

Not everyone is going to thrive in a remote role. Most companies hiring remote recognize this and try to filter for it in some form. A lot of companies' response here is to “only hire senior”. Which is probably a workable heuristic in some ways. Developers that have a proven track record of success probably have built up some habits that will help them succeed in remote work even if they aren’t naturally inclined to it. But hiring only senior folks is expensive, and makes things like diversity and career growth paths harder.

I haven’t done this enough to draw any hard conclusions on remote hiring yet. But the qualities I want to evaluate for are independence, technological comfort, writing skills, and a history of taking the initiative to seize opportunities (in whatever domain). Obviously that’s in addition to normal hiring criteria.

Tools and Process matter

The worst way to work remote is to not commit to it. Commitment means aligning your process toward remote work. Some lessons here.

First, Hybrid meetings are harmful to your team. There are 3 ways to do meetings: in person, remote, or hybrid. An in person meeting involves putting a number of humans in a small room for an hour and letting them talk through an issue. A remote meeting connects various people from the comforts of whatever work venue they’ve chosen, and discussion ensues over the internet. In a hybrid meeting, some people are in the room, and some are connecting remotely. Hybrid meetings are hell for the remote folks. Usually they can’t hear at least one person well. They can’t match faces to voices, lose context of what is happening in the room, and are sometimes just plain ignored by the folks in the room.

I really didn’t understand how bad this was until I did it myself a few times. If you’re a leader who uses or tolerates hybrid meetings, I highly recommend trying to call into them remotely a few times. As the boss you’ll get a slightly different experience, but ask yourself: are you a first class meeting participant when you’re calling in remote? If not, is it acceptable to you to treat your remote workers as second class participants?

Second, Use good tools. Bad collaboration software is a drag if you’re working with people in person. It’s a killer for remote teams. Use software that lets you communicate in timely and useful ways, keep your project critical information organized, and lets your team focus on getting stuff done. There are hundreds of good tools out there these days, so there’s no excuse for using bad ones. My recommendations, which are not particularly controversial and also not the only good options: use Zoom for video chat, Slack for messaging, Bitbucket, GitHub, or Gitlab for Version Control, Trello or Jira¹ for task tracking depending on the size of your team, and Google Drive for keeping documents and documentation together².

I… don’t love Jira. But Trello doesn’t scale forever, and I haven’t seen anything significantly better than Jira for large teams. ↩︎
My team doesn’t actually use Google Docs for documentation, we use Confluence. Which is… worse than Jira. So this is more of a “don’t be miserable like me” recommendation than one based on experience. I’d be curious to hear if anyone has a documentation/wiki tool that they love. ↩︎

Book Review: The Manager's Path

Tue, 11 Jun 2019 20:00:00 -0400

Titles in the software engineering world are strange. Without much work, it’s easy to find Senior Engineers with 2 years of professional experience, junior developer job descriptions requesting 5 years of experience, tech lead roles focused purely on technology, tech lead roles where the lead is managing a team, and who can tell the difference between a CTO and a VP of Engineering anyway? This results in weird title shifts when developers switch jobs, odd or unfair salary distributions, and a world where it is very difficult to understand what a career path looks like in this industry.

Camille Fournier’s book The Manager’s Path steps into this breach by laying out a clear look at an engineering management-focused career path. The book runs through a series of roles: Mentor to Tech Lead to Manager to Manager of Managers all the way up to Executive. For each role she describes what the job is, walks through what type of challenges you might face in the role after a promotion, and offers practical advice on overcoming those challenges.

I found it to be an extremely engaging read, and the best description of what it actually looks like to advance in technical leadership roles I’ve ever read. Because she focuses on skills and responsibilities more than titles, it’s possible to apply her advice across large and small companies, even those with wonky title structures. It helped me think about what skills I need to be building in my career, and gave me a glimpse of what next steps look like.

The focus on common challenges and how to face them was also helpful, though whole books could (and have) been written on those challenges and how to face them at each level. The Manager’s Path serves as a good starting point here, identifying the challenges and giving good solid general advice for facing them. For those new to a role it could help them identify where they’re weak or uncomfortable, and serve as a starting point to a deeper dive into those areas.

As I read the book, I immediately thought of 4 or 5 people at various points in their career I knew who could benefit greatly from reading this book. If you’re working in the software world and think that you long term are interested in advancing in your career beyond your current role, there will probably be interesting tidbits in here for you. Even those who are sure they don’t want to be in management could benefit from understanding the perspectives of what engineering managers actually do. The book is also structured so that the first few chapters are highly relevant for both technical and manager track developers; tech lead is an important role on both tracks. If you fall into one of those buckets, I highly recommend The Manager’s Path.

Thinking About Values

Sun, 02 Jun 2019 20:00:00 -0400

As a way to help myself grow in my first year as a manager, I’ve been working through Google’s New Manager Training which they have generously published online. One exercise that stood out to me was a “values selection” process, where they encourage managers to select a ranked list of 5–10 words that resonate with them as values from a list of around 400 “values words”.

Choosing values is an odd process, because you’re choosing between “goods”. When I first went through this sheet, I circled over 40 words as values that “resonated” with me. They’re things that I value and strive for, or wish that I was more like. Deciding between adventurousness and cowardice is not a “values decision”. A real values decision means that you’re trying to decide whether good traits like adventurousness, belonging, or clear-mindedness are more important to you.

Management often forces you to choose between goods: Will we ship quicker? Aim for higher quality? How do those choices impact work/life balance for our teams?. It’s hard to know if we’re making good choices, because the results routinely lack clarity. Indicators lag, causation is fuzzy, and we don’t get to compare to a control group. Values can be guideposts through ambiguity. I’ve seen the power of this at a company level in my current gig, where having a set of values that are kept in the spotlight often allows us to weaponize them as decision making tools [1].

Going through the list, I eventually cut things down to 20, 12, 10 and then 8. That still feels like a lot to me, but for each of the 8 I can easily cite work situations I’ve faced in the past month where my response came directly out that value. In the end I ended up with Integrity, Thankfulness, Community, Prudence (in the classical sense), Generosity, Continual Improvement, Accountability, Justice. I find that these are easy to connect to my everyday work when I write them out as more detailed sentences though.

Act with integrity.
Live with thankfulness for what I’ve been blessed with.
I want to work and live life in community.
Make decisions based on a rational analysis of reality, not feelings and first impressions (Prudence).
Don’t save it all for myself. Leave margin and be generous.
Continual Improvement + Goodism are a powerful combo[2].
Be accountable for my responsibilities and hold others to the same.
Actively work for justice

I’ve published them here with a bit more detail.

Those are “guideposts” that I can easily quote to myself or post up somewhere, but they still have enough substance to say something meaningful when I’m considering how to handle a low-performing employee, organizational changes, or personal career decisions. I’ve already seen these values bring a clarity to my thinking about tough things in a short period of time. If you’re interested, I highly recommend checking out Google’s training.

I’ve also seen how quickly this becomes useless when those values are not universally embraced or enforced. ↩
Probably a separate post, but goodism has always been my response when asked if I’m a perfectionist: “I’m a goodist. I care about doing work that is good and meets the needed standards, but tend to move to the next challenge when I pass that bar and hit diminishing returns”. ↩

Herding Lions

Sun, 26 May 2019 20:00:00 -0400

I used to find myself using the term “herding cats” a lot when confronting inertia, disorganization or ambiguity in work situations. Urban Dictionary actually endorses this:

The phrase herding cats comes from the common saying that something involving coordination of many different groups or people is as difficult as herding cats. One of the commonly encountered uses of the term in technical fields is the phrase “Managing programmers is like herding cats” or “Managing engineers is like herding cats”. In education it would be “Managing students is like herding cats”.

The phrase correctly captures the difficulty of getting a group of programmers (or even worse teams of programmers) on the same page. But there’s an inherent arrogance to it. The implication is that the herder is the adult in the room and everyone else’s pointless individualism is getting in the way.

There are days that I think I could fix everything if I could just get the *&$% cats to move in the right direction. In those situations I’ve started telling myself that what I’m actually doing is herding lions. How do you herd a group of lions? I have no idea. But I know that if you don’t start with a healthy respect for the lion’s capabilities and agenda, it won’t end well for you.

When you treat people like lions, it’s an opportunity to stop and figure out why they’re not just getting on board with your obvious plan. It’s an opportunity to learn, to find new ways to motivate people, and maybe even to empower somebody who had been viewing themselves as more of a kitten. Ultimately the best teams flourish where people can be strong contributors, take risks and do work they find meaningful. They’re teams of lions.

Obvious Things

Fri, 17 May 2019 20:00:00 -0400

I saw a tweet thread a few weeks ago that made me think:

There’s a bit of a meme in some parts that there is value in saying, out loud, “I realize that I am surprised.” For similar reasons, I often subvocalize “I believe I now understand the type of situation the ‘trivial’ advice was preparing me for.” https://t.co/JzdYmSvJWH
— Patrick McKenzie (@patio11) April 9, 2019

And then you repeat it to someone and they say “lol that is obvious” and you wonder if you know the advice well enough yet to correct them on how not obvious it is going to be for their next 10 years.
— Patrick McKenzie (@patio11) April 9, 2019

I realized that I’ve encountered plenty of pieces of wisdom in my software career that have this property; I see them as clear and obvious when put in the abstract, but only learn to actually apply them by repeatedly failing to do so. This is what people talk about when they say something is “hard earned experience” I guess.

Here are some examples from my career:

In the long run, rate of improvement (or regression) matters more than current level - This has applications in hiring, code quality, and finding/staying in a job. But it’s often easier to observe how good something is now than to see how its changing, and I often fail to properly estimate what a trend means for a situation 1-3 years down the road. It’s also easy to mis-apply this principle when you for get about the “long run” part. A new employee with tons of potential won’t help you when you need an experienced hand now and fixing technical debt can easily cost you your ability to hit a quarterly objective. Knowing how to balance these long and short term goals is something I still struggle with all the time.
Rewriting from scratch is (almost) never worth it - Joel Spoelsky wrote about this way back in 2000 and I remember reading that article nearly a decade ago and nodding along. I seem to have infinite capacity to convince myself that my current situation fits in those parentheses though. In my career this has mostly been a problem where I’m able to more accurately forecast the pain that would ensue from fixing what exists (because that code exists and is easier to work with), while I appear to systematically underestimate the pain of building the new thing (because the internet told me it would be great, and the code doesn’t exist for me to nitpick yet).
Always consider multiple options - Pretty much any decision (technical or otherwise) benefits from having multiple contenders. This seems obvious. But I have often let myself fall into situations where I’m choosing between just 2 options, or really just choosing whether or not to do 1 thing. This has applications in technology choices, job choices, responding to another person’s provocations, and more. I intuitively know that more options are good, and I constantly miss opportunities to add options.

So how do we improve on these situations where we fail to apply obvious wisdom? I’ve been learning a few things lately.

Always review what you’ve done on a regular basis - Do weekly reviews about how your week went. Do quarterly reviews with yourself about how your job is going. When you make big decisions, make sure to find a time to check back in later and evaluate how it went. We make better decisions when we take the time to measure our results
Note Missed Opportunities - When you do those reviews, make sure to pay attention to when you missed an opportunity to apply something you knew. Sometimes we make mistakes because we just didn’t know something. It’s more common though that we didn’t properly apply our own principles and knowledge.
Identify Cues - Identify the thing that you know you’re going to have to handle different next time. Maybe you say “if I’m looking for a job, I should consider at least 4 or 5 different companies and not focus in on just one”, or if I’m asked to fix a legacy codebase, I will spend at least 3 months trying to improve it before considering a re-write,and I’ll look for real world comparisons to base a rewrite estimate on". Whatever it is for you, when you can identify situations where a specific action should take place, it can help you be better the next time instead of repeating the same old mistakes.

How To Get Better Feedback On Your Code

Sun, 21 Apr 2019 20:00:00 -0400

Deliberate practice is when your attention is focused tightly on a specific skill you’re trying to improve … you receive feedback so you can correct your approach to keep your attention exactly where it is most productive.

Cal Newport, Deep Work

In Deep Work, Cal Newport explains that research has shown the primary difference between experts and amateurs in a typical field is not innate talent; instead it is the amount of time they’ve spent in Deliberate Practice within their field of study that sets them apart. Deliberate practice is not merely time spent working on the craft, but includes a feedback loop aspect. To get better at something, we need to receive meaningful feedback on whether what we’re doing is working.

When working with a team, one of the best ways to get this type of feedback is quality code reviews. When you’re reviewing, you get to see other developers approaches to code, and when you’ve written code, you hopefully get high quality feedback from a different point of view.

In reality, code reviews don’t always provide the value they could. Many developers don’t give much feedback at all on code reviews, give unhelpful aesthetic feedback, or may even use it abusively to try to show off their “superiority”. I can’t solve all of those problems, but here are a few tips I’ve found that help me get better feedback when my code is being reviewed.

1. Annotate Pull Requests

Most people assume that the inline commenting features in Github and Bitbucket code reviews are for use by the people reviewing the code. But I’ve found that it can be helpful to preemptively add comments to my PRs to give reviewers context. This is especially useful for information that may be relevant during a code review but isn’t long term relevant, like the original reason for extracting some reusable code and where it came from. Below is an example of a comment I left demoing how React hooks could be used to pull out shared logic in my work project.

These types of comments are also good ways to point a reviewer to the important parts of a review. This matters for reviews where there are many small changes across a large number of files, with a few high impact changes. It’s always better to make atomic changes (see more here), but when you must make a large change, this technique can help a lot. I have received noticeably better feedback on my large pull requests since I began annotating

2. Encourage Questions

If you’re receiving feedback from more junior members of a team, they may not always be comfortable giving direct feedback on code. One way to get better feedback in those situations is to encourage them to ask questions about things they don’t understand. This gives them opportunities to learn, but also let’s you see which parts of your code are unintuitive or unfamiliar to the rest of the team. In some cases the questions will expose bugs that a more confident reviewer may have pointed out straight out, and that experience can help grow a reviewers confidence and lead to even better feedback next time.

Of course asking questions rather than assuming understanding can be a good tactic when giving feedback in general, and is worth considering when doing your own reviews regardless of whether a reviewer has encouraged it.

3. Find More Than One Reviewer

This applies to more than just code, but if you’re not getting the feedback you need right now, ask somebody else. In a work setting, this might be finding different team members to review your code. If that isn’t possible at work, finding other ways to write code and get feedback on it can make a big difference in your career. Not everyone has time or capacity for this, but contributing to open source, project nights at local meetups and hackathons can all be great ways to learn and get feedback from others. If you can find a mentor to talk through code challenges with, that can be beneficial even if they aren’t on your team and going through your code line by line.

Ten Soft Skills That Will Shape Your Career

Sun, 14 Apr 2019 20:00:00 -0400

One of the most popular posts in the history of my blog was a list with the click-baity name Ten Things A Serious JavaScript Developer Should Learn. It explicitly focused on the set of technical skills that you need to become a “senior” JavaScript developer. While I’m happy with that post, I’ve always wanted to revisit the idea, because it is my strong belief that once you’ve hit a minimum level of technical capability, only about 20% of career success is determined by your technical skills. The other 80% is composed of skills that can be loosely bundled under “soft skills”.

You may quibble with my choice of percentage, but I don’t think the underlying idea that things other than technical ability is particularly controversial. Even if they bitterly dismiss such things as “politics” most developers are aware that their careers are impacted by things that go beyond their ability to write algorithms. What I haven’t seen much of however, are attempts to catalogue what these soft skills are, and how somebody might get better at them. The second part is outside of the scope of a single post, though I will attempt to list resources here. But the first is achievable, and that is what I will attempt to do here.

The following is not an exhaustive list, but it covers 10 “soft skills” that I’ve seen have a clear impact on my career in software and those of others. The best part? These are things you can grow in that will have an impact in your life outside of work, and in any career you might transition into in the future.

The Skills

1. Clear Communication

While I’m making bold statements: A developer’s ability to communicate clearly is the single most important skill in determining the trajectory of their career. Poor communicators are forced to rely on others to interpret their work, even when it is brilliant. They will often create problems for others through miscommunications or lead teams to make suboptimal decisions because they were not able to explain the technical considerations involved in a situation.

Great communicators on the other hand, enlighten. They help teams understand what is really going on so that non-technical team members can make good decisions, and technical folk can work with confidence. Great communicators cut through misunderstandings, expose ambiguity and poor assumptions, and can translate for team members who are having trouble making themselves heard.

Communication skills are important for those on a management track, but they are just as important for those on a technical trajectory. Architects, Principal Engineers and Tech Leads must coordinate with many teams, teach less experienced developers, explain technical tradeoffs and document decisions and code. Those are core parts of a senior technical employee’s job, and all of them come down to communication.

Resources

2. Attention Management

This one is more commonly referred to as “time management”, but I agree with this recent NYT Op-Ed: the mechanics of how we organize our time is less important than whether we end up spending it on the most important things. This broadly encompasses the ability to prioritize, to prevent distractions from wreaking havoc on productivity, and the ability to find ways to get back on track when things go wrong or priorities change.

This was a weakness for me early in my career: I got things done but often did them haphazardly or inefficiently. One of my biggest sources of growth in the last few years has been finding ways to structure and prioritize my time better, while cutting out distractions. This has been necessary due to life changes: I’m now a parent and have people reporting to me at work. I wouldn’t be able to meet my current responsibilities with the habits I had as a 25 year old. For me this has been largely about focusing in on a small amount of essential things and actively removing less important things from my life. For others it may be more about establishing patterns and limits. Either way, this requires planning and introspection first to be effective.

Resources

3. Initiative

If you see an opportunity or a problem, do you do something about it? I’ve been reading a good amount of “manager books” recently, and in many of them a surprising amount of focus is dedicated to the question of how we create conditions when people are “empowered” to act. And there certainly is an environmental factor here. People are more likely to take initiative and risks when they’re not afraid of being punished for it. But the individual also controls a lot here. Developers who are willing to step up and take initiative in a situation can make an outsize impact.

Do you have a piece of code or process on your team that everybody acknowledges is a bit wonky, but nobody has ever taken the time to fix even though it is slowly taking a toll on the team over time? Are there simple things that you know could make your product better, but nobody has quite taken the time to get to them? Those are the opportunities that initiative can turn into value.

Resources

Fix Like No One’s Watching

4. Empathy

There is no meaningful software product that has been created in isolation. One person may have written the code (though that is quite rare for software with any scale of impact), and maybe even didn’t collaborate with anyone to handle distribution / hosting etc, but if that software is used by others there is always support, training and marketing to consider. You can create a todo app for yourself without talking to anyone. But nobody makes a career in this business without talking to other humans.

The thing about us humans; we work well with people who we trust, and who we perceive as caring about us. We also are for the most part exceptionally good at discerning when somebody is faking it. So if we can’t get away from humans, and we can’t be fake, what can we do? We are left with, in the words of Dale Carnegie “becoming genuinely interested in other people”. If you’re currently filled with red-faced frustration at your annoying co-worker or manipulative boss, this may seem like asking for a miracle. And arguably any time we look outside ourselves it is a miracle. But regardless of your philosophical outlook on such things, there are definitely ways we can improve in how we work with and care for the people around us. And it starts with trying to empathize and seeing the people around us as human beings and not obstacles or tools.

Resources

5. Negotiation & Persuasion

You may not think that Software Engineering involves much negotiation. But that is only true if you use a shallow definition of negotiation. Every time a team debates implementation strategies for a feature, which tickets should end up in a sprint, and how much time to spend chasing a difficult to reproduce bug, they are engaging in negotiation, and team members are attempting to persuade. And there is a wealth of knowledge out there describing how to improve on these skills.

A word of caution here; unlike some of the other things on this list, these skills can often be dangerous when they’re improved in isolation. When developers complain about politics trumping technical ability, they’re often experiencing decisions being made based on the ability of some employees to persuade and negotiate. When those employees are not actually providing the best ideas this is suboptimal. For the technically skilled employees, the answer is to learn to persuade better. But for those who can negotiate we always need to make sure that we’re persuading for good. Just like the loudest voices shouldn’t always win, when you can communicate well, make sure to stay humble and boost other’s good ideas when you see them.

Resources

Negotiation Genius - Deepak Malhotra and Max Bazerman

6. Continual Learning

Were you paying attention in high school physics when they taught about the difference between distance, velocity, and acceleration? Distance is the amount an object has traveled. Velocity is the speed at which it is currently moving in a given direction. Acceleration is the rate at which the object’s velocity is changing. If you want to know where an object is right now, distance is the only thing that matters. But as time progresses, velocity starts to matter more, and in the very long term any difference in acceleration will be the only thing that matters.

The same ideas play out when acquiring skills. If you need to evaluate your career potential for the next few months, the skills and accomplishments you’ve acquired so far (distance) are the only things that matter. But if you want to think about your career potential over the next 30 years, the new skills you’re acquiring (velocity) and even more importantly your ability to get better at learning new things (acceleration) will matter much much more. Learning is a skill unto itself, and the earlier you invest in it the more dividends it will pay over time.

Resources

7. Self Care

Another lesson that I’ve learned the hard way in my career is the reality of my own mortality. None of us are indestructible, and our bodies are not perpetual energy machines. They require care and maintenance. Our souls too are not meant to run on work alone, and we can burn out hard without sufficient rest. This is all aside from deeper issues of disease and mental health that can change our lives in even more profound and difficult ways.

Self care is a career skill because frankly we are great at deluding ourselves into thinking we’re functioning normally while we’ve compromised ourselves. Research shows that we are horrible judges of whether our judgement and creativity are compromised by lack of sleep. Because we can’t run controlled studies on ourselves, we don’t really know whether our lack of exercise or 60 hour workweek induced burnout are hurting our creativity. But research suggests it is for pretty much everybody.

Healthy socialization, sleeping, eating and exercise ultimately lead to better emotional stability, ability to focus, and creativity. But even aside from the career benefits, when we’re devouring our bodies and minds for work, it leads to a question of what we’re doing this for in the first place.

Resources

8. Business Awareness

Do you know what your company actually does? Do you understand why the features you’re building are important, and the relative priority of them? Can you explain the business tradeoffs involved when you’re making an software architecture decision between performance and security?

In companies larger than a few people it’s not ultimately developers' jobs to make business decisions. But understanding the context for those decisions helps us better prioritize our time, make good tradeoffs, and call attention to inefficiencies or counterproductive behavior in our products. In addition, nobody wants to spend time working on things that don’t matter. But when we try to innovate without understanding the broader business, we often end up doing just that.

Resources

Choose Boring Technology - Dan McKinley

9. Marketing and Branding

Marketing is a dirty word in engineering circles. Many take the attitude of the Dilbert comics that marketing is a synonym for exaggeration lying, and fraud. But at it’s core marketing is just making sure that the people who could benefit from your services are aware of those services. This has several applications to developers. In the job market, finding ways to let employers know who you are and what you’re capable of is important. But in medium to large companies it also is important to do some level of internal marketing; to make sure you’re credited for the work you’ve done and that people understand it’s values. Developers who consider themselves above this or dismiss it as politics do so at their own peril.

Resources

7 Common Content Marketing Mistakes Developers Make and How to Avoid Them - Stephanie Morillo

10. Character

The final item here is in some ways an oddball. While the other 9 things on this list are skills that you can work on, character is something deeper. Questions of what good character is and how you develop it are way outside the scope of this post. And I would personally consider any attempt to improve your character for solely utilitarian career reasons to be ultimately hollow. But character is undeniably important in your career.

Do the people around you trust you? Do they believe that you’re trying to do the right thing? If the answers are no, you may find obstacles in the most unexpected places. If the answers are yes, you will eventually find help from people you would never have looked to.

In addition, as an industry, technology has the power to shape the world, and as practitioners we all get the chance to impact our little corners of the world. It’s worth considering not just whether we’re “succeeding in our career”, but what impact we’re having on the world around us.

Resources

Future Ethics - Cennydd Bowles

Consistency

Sun, 31 Mar 2019 20:00:00 -0400

The last few weeks I’ve had several conversations with a coworker about consistency in code: why do we do things one way here, and a different way there? Some of this is macro level; Where do we split chunks of related functionality into different files? Some of it is micro level; what should the rules be for when we use an inline function instead of pulling it out into its own named method? Some of it is trivial; Do we use _ or - to separate words in file names? In our current codebase we’re not always consistent on many of these issues. As we’ve discussed the various issues, I realized that some of them bothered me more than others, and (more interestingly to me) my ideas have changed here over time.

The Platonic Ideal

In a perfectly consistent system, 3 developers writing the same module would produce line by line identical code

There’s an extreme point of view out there that says “while a perfectly consistent system might not be possible, the closer we can get to that the better”. The benefits of consistency are many: consistent naming helps with code search and connecting different parts of a system. Consistent code structure makes code easier to scan once a developer is familiar with the system, and it makes exceptions stand out as something intentional and interesting. Consistent formatting removes potential distractions. So what’s not to love? Shouldn’t we always try to add more consistency? I see 2 primary reasons why it might not be worth trying to enforce every single rule we can come up with.

We’re only human

The first reason not to enforce “everything” is that our human attention spans can only internalize so many rules at once. When we’re trying to remember a large number of rules, its easy to miss one. But more importantly, when we’re writing code or reviewing it, focusing too much on human enforced consistency rules can distract from the actual purpose of the code, making it harder to notice the substance of code because we’re so focused on the form.

Standards too soon

The other thing that can happen when you try to standardize everything is a loss of creativity. Sometimes it’s helpful to experiment for a while to find the right pattern in a codebase. In an ideal situation experiments like that will happen on the side, or be explored in branches, but for small teams or new codebases, it will likely happen at least partially in production.

User Friendly Consistency

So what can we do to get the benefits of consistency? The 4 ways I like to see this happen are:

Automation

The best way to enforce stylistic consistency and other straightforward things like naming conventions is to use automation. Linters like eslint and auto-formatters like Prettier. My rule here is that any linter rule that is about stylistic consistency should be auto-fixable, if not it isn’t clear enough or isn’t worth it. For “correctness rules” it can be ok to flag errors without auto-fixing, if human judgement is needed.

Code Review Checklists

Code reviews can be a good way to limitedly enforce consistency; but its good to limit scope to avoid developers trying to become human compilers. If you try to enforce too many things in code reviews, its easy for people to get caught up in going through a list and forget to apply human judgement to the context of the current code. But code review checklists are a good way to make a coordinated push on high value items that are either new or often neglected. This checklist should consist of a few important things to check during every review. The list should be re-examined on a regular basis and ideally kept to 5 or less items.

Focused Refactoring

Sometimes its appropriate to have “refactoring initiatives” to standardize on a particular pattern over time. This can be good for standardizing on a particular library for a task, or removing an older pattern that persists in the codebase after a better way has been found. My advice here is to only do one of these at a time, and complete them before starting a new one. One of the biggest causes of inconsistency is when a team attempts to “standardize” on a solution, then abandons that for a new solution without finish the migration, resulting in a lava layer architecture.

Focus on the high level

If there is one place to apply human consistency checks its at the structural level. If your high level patterns are consistent, the code base will be easy to navigate, and modular code written in a standardized high level structure will be easy to delete. Overall low level code that is inconsistent or poorly written is easier to overcome than high level code as long as you’re taking advantage of code reviews and the other methods suggested here to improve new code, because most low level code is only read occasionally. So high traffic “messy code” can be a focus of refactoring efforts, and the rest can continue working without disturbing anything.

Messy code is much more problematic at a structural level though, because when different areas of the code are arranged in different ways, it becomes much harder to find functionality when debugging. This type of inconsistency also tends to make bulk improvements harder, as you have to have a much greater understanding of all the ways a codebase is put together.

Lessons from Six Months As A Manager/Developer Hybrid

Sun, 24 Mar 2019 20:00:00 -0400

For the last 6 months I’ve been serving as a hybrid developer/engineering manager. It’s been an interesting experience, and because this is a fairly common way to transition into engineering management¹, I thought it might be worthwhile to share some lessons I’ve learned so far.

1. Time discipline is key

The biggest challenge of this transition was learning to be a manager. It’s a whole new job! But the biggest challenge specific to the management/developer split has been managing my time.

I’ve always been a bit loose with my time management. I do a lot based on my mental state at a particular time; if I’m tired or uninspired I may lean more into grunt work like documentation, code reviews, or more “scripted” code work. If I’m in the zone I’ll dive into big tasks, and if someone comes to me with a problem I tend to context shift into it and go from there.

This “time management strategy” hasn’t held up at all while trying to balance manager and development strategies. Basically because of the blend of maker and manager schedules. Developers need long uninterrupted periods of time to be productive, while managers tend to operate on a much more social and interrupt driven schedule. They’re naturally opposed.

Working with this tension requires discipline and planning. For me personally I’ve started planning my schedule for the next day at the end of the previous day, batching my checking of Slack/email and being more thoughtful when somebody asks for my time.

Planning my schedule for the next day is a tip I picked up from Deep Work. To close out the day, I review the things I’ll need to handle the next day, and block out my calendar with time for different tasks for the next day. This does mean that my calendar always looks completely booked for the current day. So far this has served as a feature not a bug; I can still move things if something urgent comes up and it has a way of encouraging people to schedule less urgent things in advance. Note that scheduling like this doesn’t mean you can’t change plans; I do all the time. It just means that you start with a plan and can be thoughtful about how you vary off of it.

Batching Slack and email has been something I’ve known would be good for me forever. It feels good in the moment to be the guy who is constantly available, but the cost of distraction is higher than you think. So far, management has been a catalyst for me buckling down here.

Finally, I get a lot more unexpected requests for my time now than I did previously. I’ve found that batching my email and Slack reading combined with a willingness to suggest a meeting later in the day or the next morning rather than right now has helped me protect blocks of productivity and focus, without making me an unresponsive manager. Most things can sit for 2 hours without blowing up anyone’s day. The key is to ask enough to identify the 5% of things that can’t wait.

Those 3 tactics have worked for me, but overall it comes down to being intentional about your time. If you let yourself be blown by the wind while splitting dual responsibilities, one of them will almost always receive the short end of the stick.

2. Team success metrics are the only ones that matter

When you’re acting as both a developer and a manager, it can be tempting to hold yourselves to 2 sets of metrics: management metrics and development metrics. This is a mistake. It will cause you to over-prioritize your own tasks and hurt your team’s long term growth.

Focusing on personal development metrics and goals can lead to over-prioritizing them for a few reasons. First, development goals tend to be much “cleaner” than management goals; it’s easier to tell what you’ve achieved. Second, if you’re transitioning into this role from a dev role, you probably are much more comfortable with these tasks, and so they can act as a safe place where you avoid having to stretch yourself. And finally, if you moved into this role from a senior dev position, it can be easy to tell yourself that this is actually the best way to get stuff done. And you may be right… in the short term.

So what’s the antidote? Focus on team productivity and view your own work as just an extra card you have to play in your hand. It’s an advantage you have over less technical managers, but if you don’t set to working developing other strengths, you’re not going to be able to grow, and if you’re monopolizing the important work your team won’t be able to step up into new roles. So play that card, but do it with caution, and always as a means rather than an end.

3. This isn’t long term sustainable; have a plan

Hybrid Management / Developer roles aren’t a long term game. Aside from the scheduling tension I mentioned above, once you move into this role you’re either going to be slowly giving up responsibilities and watching your tech skills slowly move off the cutting edge as you focus on learning how to manager your team and developing others to fill the roles you once did, or you’re going to be failing as a manager as you try to cling too tightly to your senior individual contributor responsibilities. So if you want to avoid that, it’s important to be evaluating how you want your career to evolve. A role like this can be a first step up a chain, or part of a dev to manager pendulum. You also might discover that management isn’t for you and decide to move back. I’m committed to this for a set period of time, and that’s helped me to work through the days when I wonder what I actually accomplished, and think strategically about what I need to get better at.

4. Best Management resources

So this isn’t a learning so much, but as a newbie manager, probably the best thing I can do is point you to the resources I’ve found helpful. So here are some of the best.

On Engineering Management

Rands In Repose by Michael Lopp and the book he pulled out of the blog are great little snippets of engineering management gold
Lara Hogan’s Newsletter is full of useful advice, and she’s really good about including exercises to help practice what she’s describing.

On Management Generally

Radical Candor is the single most helpful thing I’ve read while trying to figure out this whole management thing
Patrick Lencioni’s books are all told as “leadership fables” which can seem a bit weird, but in the squishy imprecise world of management, stories can be very effective ways to teach. I’ve read The Five Dysfunctions of a Team, Death By Meeting and Silos, Politics, and Turf Wars. They’ve all been great and I plan to read more of his stuff.

On Structuring My Time

I reviewed it last week, but Deep Work was super helpful in thinking through how to balance my time.

On Career Path

I linked this one above, but if you didn’t click through then Charity Major’s post on the Engineer/Manager Pendulum has been really helpful for pushing me to think through where I want to go next.

I’m aware that lots of folks would say that this is a very bad way to start management and that you should stop coding when you become a manager, but it isn’t always possible to fully control what opportunities are available. ↩︎

Utility Functions

Sun, 10 Mar 2019 20:00:00 -0400

As I’ve gone through projects over time, I’ve found a few functions that I end up creating for every significant project that I’ve worked on. Some of the details differ from project to project, but the basic function remains the same. I thought it would be helpful to do a rundown:

React/JSX

Some of my utils are focused on making JSX nicer to deal with. 2 situations that I’ve run into a lot in JSX are conditionally showing an item in JSX, and determining which of several classes to add to an element based on logic. Scenarios like these:

<div>
    {shouldShowCreateButton ? <Button
            onClick={createWidget}
            type = 'primary'
            size = 'Large'
            disabled = {!createEnabled}
            className='create-button'
        >
            Create Widget
        </Button> : null}
</div>

let classNames = 'create-button'
if (extraClass) {
    classNames += ' ' + extraClass;
}
if (createEnabled) {
    classNames += ' is-disabled');
}
if (hasCreateError) {
    classNames += ' has-error';
}
return <Button
        onClick={createWidget}
        type = 'primary'
        size = 'Large'
        disabled = {!createEnabled}
        className={classNames}
    >
        Create Widget
</Button>

These are both ok, and you’ll see plenty of examples of code like them out there, but the first puts a big chunk of code in a ternary, and it’s not always going to be clear to a reader where that : null came from at the end, or why it’s there by the time they reach the end.

In the second example, we’re dealing with a lot of ugly string concatenation. There are other ways to handle this, like using template strings + ternaries but generally they each provide their own messy pieces.

To solve these situations, I like to define these 2 small functions:

import isString from 'lodash-es/isString';
import filter from 'lodash-es/filter';
// A helper for React Components to reduce ternary use
export const showIf = (condition, element) => (condition ? element : null);

// A helper for handling class lists where some values may be undefined
// filters out anything that isn't a string and then adds spaces
export const classes = classList => {
    return filter(classList, c => {
        return isString(c) && c.length > 0;
    }).join(' ');
};

The first just formalizes the most common use of ternaries in JSX (conditionally showing an item) and adds an easy to understand name to it.

The second lets us work with an array of classNames instead of handling string concat ourselves, and will automatically filter out undefined or empty classNames. This implementation does silently swallow errors on anything that is not a string, which may not be your preference. Usage looks like this:

<div>
    {showIf(shouldShowCreateButton, <Button
            onClick={createWidget}
            type = 'primary'
            size = 'Large'
            disabled = {!createEnabled}
            className='create-button'
        >
            Create Widget
        </Button>)}
</div>

the showIf reads much more clearly than the previous ternary, especially for language newbies who might not be used to ternaries.

let classNames = ['create-button', extraClass];
if (createEnabled) {
    classNames.push('is-disabled');
}
if (hasCreateError) {
    classNames.push('has-error');
}
return <Button
        onClick={createWidget}
        type = 'primary'
        size = 'Large'
        disabled = {!createEnabled}
        className={classes(classNames)}
    >
        Create Widget
</Button>

Here we’ve gotten rid of the extra string code and are working on arrays, which is a nicer data structure for dealing with lists of stuff. Also since we’re auto-filtering falsy stuff, we don’t have to worry about whether extraClass was defined if it was passed as a prop to our component or something.

Formatting

So the next few may be different for everyone. The details here aren’t important, the important thing is that you centralize your formatting of numbers and dates. For instance, here are functions for formatting dates, normal numbers and percentages from a current project:

import { format } from 'date-fns';
import isNumber from 'lodash-es/isNumber';

// takes either a string or a date object and formats it as a standard date with time
export const stdFormat = dateStr => {
    if (!dateStr) {
        return dateStr;
    }
    let date = new Date(dateStr);
    return format(date, 'MM/DD/YY hh:mm A');
};

export const commafy = (num, minDecimals = 0) => {
    if (isNumber(num)) {
        return num.toLocaleString('en-US', {
            minimumFractionDigits: minDecimals,
        });
    } else {
        return num;
    }
};

export const percentify = (num, minDecimals = 0) => {
    if (isNumber(num)) {
        return num.toLocaleString('en-US', {
            style: 'percent',
            minimumFractionDigits: minDecimals,
        });
    } else {
        return num;
    }
};

The first one will take a date object and format it to something like 05/30/17 01:16 PM. The second will take a number and return a string formatted with appropriate commas: 1234567 -> '1,234,567'. The last one will convert a decimal to a percentage string: 0.5 -> '50%'.

Note that none of these functions are very long. I’m relying on a date library for the first piece and toLocaleString for the other 2. So I could inline this in most places and not have it get much longer. The benefit of these types of functions is not in saving typing. It’s in standardizing how you format or don’t format things throughout an application. This has 2 benefits:

It’s easy to fix bugs, add features or switch between libraries for these common tasks across your application because all of the logic is centralized. If I needed to start supporting more locales than just the US for percentage formatting, or use moment for my date formatting library I could easily do so with edits to only one file. That file is also well unit tested, so those transitions would be safe.
It lets us control which formats are allowed. For instance we’re allowing the function caller to specify how many decimal points to show for the numeric functions, but not giving options to change the date format for the formatting function, because we want that to be the same everywhere. If we need to support multiple formats for different scenarios, like a date-only format or a format for communicating with the API, we can add options for that which don’t open up all formats, or expose them as separate functions. With dates in particular, I want imports of libraries like moment or date-fns to be limited to 2 or 3 files at the most, and not used ad-hoc throughout the codebase.

Note that all my examples so far have utilized other utility libraries like lodash and date-fns. The best utility functions are those that you don’t have to maintain yourself. I do recommend looking for libraries that you can tree-shake and only take what you need though. Other ones like moment.js can make a big difference on your JavaScript bundle size if you let them.

Timing

Timeout functions are useful, but the browsers built in functions are callback based and don’t play well with promises or async await. We can fix that:

export const sleep = secs => new Promise(res => setTimeout(res, secs * 1000));
export const pause = () => sleep(0);

I now have a promisified version of setTimeout (sleep) and a special case version with a 0 timeout (useful for things that should happen as soon as possible after the current event loop executes. I prefer dealing with things in seconds over milliseconds, so I changed the API to use that. That is probably not everyone’s cup of tea, so feel free to copy and modify.

Usage looks like this:

// function to kick off a "job" of some type and then
// check every 30 seconds to see if the job is complete
const createJobAndShowResults = async () => {
    createJob();
    // don't do this without a timeout/failure
    // condition in real life!
    while(true) {
        await sleep(30);
        let jobStatus = checkIfJobIsCompleted();
        if (jobStatus.complete) {
            return jobStatus;
        }
    }
};

And Even More!

This is just a sample of the small functions I usually create in projects. Pretty much all of them exist for one or more of the reasons that were already discussed:

Smoothing over awkward library syntax like multiple classes in JSX or HTML templates
Making user intention clear with a well named function, like showIf or stdDateFormat
Ensuring consistent application-specific behavior across common low-level operations like formatting or logging
Making it easy to replace 3rd party utility libraries like date-fns by wrapping them in an application specific function
Improving browser level APIs, or shaping them to have useful defaults for the current application, like we did with toLocaleString and setTimeout.

Book Review: Deep Work & The Common Rule

Fri, 08 Mar 2019 20:00:00 -0400

I’m trying something different today. This is a review of 2 books I’ve read recently; I’m combining them because their similarities and differences are just as interesting to me as the practical steps they encourage.

Deep Work is a 3 year old book from Cal Newport focused on encouraging readers to spend more time on their most important work. It starts out with sections defining and motivating “Deep Work”: focusing without distraction on cognitively demanding tasks. Then it goes on to give a series of recommended strategies for creating more and better deep work time in your life.

The Common Rule is a new book by Justin Earley focused on building habits that allow living life with purpose. It is a “Christian book”, and recommends practices like daily prayer and fasting. But it is mainly focused on encouraging habits to combat some universal challenges of modern American life: distracting technologies, all consuming work environments, and increasing isolation.

A Shared Diagnosis

Deep Work and The Common Rule share a common view on the challenges facing many American adults today: they each tackle tech distractions, work expectations of constant availability, and time spent on shallow activities disconnected from our core goals and desires.

Both authors are skeptical of modern “networking tools” like social networks, chat tools, news sites and email without dismissing them. The primary issue both authors identify as the problem with technology is our non-critical acceptance of new tools into our lives as progress, the addictive nature of these services, and how they then impact our brains. Because we turn to technology for distraction and easy comfort whenever we’re bored, we lose our capacity for deeper engagement with the world and can lose sight of more important things.

Each book also decries the problems with “constant connectivity culture”. Deep Work comes at it from a pure productivity standpoint: expectations of constant availability create distractions that prevent us from getting into the flow of meaningful work, and when we’re answering emails at all hours of the day, we don’t get the context shifts where our brains can rest and help us be more productive later. The Common Rule shares these concerns but is more interested in how these distractions can shape our thinking and emotions: when we get sucked into the world of social media news for instance, is the anger we feel about issues a reflection of our true priorities or a path that we’ve fallen down. It also asks us to consider how long term media-consumption may shape these believes and not just distract from them.

Finally, both authors touch on the amount of time we spend on meaningless activities. Whether it is busy work, meetings, social media usage we feel we “have to do” or “real work” that just isn’t the most important work, both books encourage readers to value their time and be strategic about how to spend it. They share a rare humble view of the human condition: we really do need to work with only 24 hours a day. How to “make the most of them” is a core focus of each book.

A Suprisingly Similar Recommendation

In addition to a shared view of the problems people face, both books share a fairly similar approach to solving them. Both books recommend ritualized structuring of life to reduce time spent on shallow, distracting activities and enable time for “the good life”.

So… what is “ritualized structuring”? Both books spend significant time outlying a series of recommended habits/rituals that combine to help make us more aware of how our time is spent, and also serve as enabling spaces for “the good life”. Deep work recommends (among other things) structured work hours, work-free evenings, social media avoidance and end of the day “shutdown rituals” where you clean up todos and make a schedule for the next day. The Common Rule advocates for taking an hour away from your phone each day, a meal with others each day, curating media to 4 hours each week as well as more obviously religious habits like daily prayer and a sabbath day.

Each book offers these habits as “focusing tools”, ways to spend time on what is important and focus on the “good life”. So what is the “good life”? Well…

A Subtly Different Motivation

The biggest difference between Deep Work and The Common Rule is what they’re aiming to enable. What is the “good life” that we want to unlock as we wade out of the shallows?

Deep Work sees “Deep Work” as both a good in itself and a means to career success. Deep Work allows for craftsmanship and an impact on the world greater than we’re normally able to achieve. And because it is both important and implicitly discouraged by the modern office environment, clearing space for it is a tremendous competitive advantage when it comes to career success.

The Common Rule is explicitly focused on habits that turn us outward from ourselves; habits like a daily meal with others and a weekly hour of conversation with a friend are about connecting with the people around us, while daily prayer and weekly fasting are about connecting with God. And ultimately they’re about health: when we connect well with God and others, we’re able to live in a way where we can accept our limitations and thrive where we are.

I don’t think the 2 philosophies are incompatible per say: you can work efficiently at work while valuing your relationships, as long as you’re willing to accept the inherent limitations of the 24 hour day. But reading these 2 books back to back reminded me that “efficiency” is never an end to itself. It’s always important to make sure you’re heading efficiently in the right direction.

Recommendation

I highly recommend both of these books. They are full of fantastic suggestions that I plan on implementing over time (I’m trying to avoid my classic blunder of trying to change everything about my life at once and look for bits and pieces to attempt first). But as you take in the ideas, make sure to reflect on what the “good life” looks like to you.

Amazon Links

The Importance of Writing For Software Developers

Fri, 01 Mar 2019 20:00:00 -0400

Once somebody hits a minimum threshold of technical skill to hold a job in the industry, about 80% of their ability to succeed in Software Development is determined by their communication and people skills, not their technical abilities

- Me, Christmas 2018

I think I caught my sister in law by surprise a bit when I answered her questions about the importance of writing in my job that way. She’s an assistant professor who teaches writing to college students, and has found that her students pursuing technical degrees generally fail to see the value in learning to write. The contrast between their expectations and my perspective after a decade working in software was strong. I promised her at the time that I’d write down some thoughts on why writing skills are valuable for software engineers. So here it is, 6 reasons that writing skills are crucial for working in the tech industry¹.

1. You will write every day

Every industry has truths that are obvious to those who have spent time working in the industry, but may be surprising to those on the outside. One such truth for software engineers: our jobs involve an awful lot of writing. Here are a few types of writing that I’ve done daily or weekly for the last decade:

Discussing technical implementations through chat apps like slack
Writing Bug Reports
Writing comments in my code and technical documentation to help others and my future self understand what was done
Writing reviews of other developers code to help our team improve and coordinate on best practices
Answering questions from less technical employees or clients, often through email or slack.
Creating design documents for complex work during planning times

That list is illustrative but not exhaustive. Any software engineer who wishes to build software that is used by others will end up doing plenty of writing.

2. The ability to explain complex topics will set you apart

The ability to take a complex topic and explain it in terms that somebody who doesn’t have all the background you do can understand is a critical one for software engineers. This can help negate two of the most frustrating experiences for a developer:

A business asking for work that doesn’t make sense because the business folks don’t understand how the product works
Less experienced members of the the team routinely making mistakes and breaking things because they don’t understand how the product works

Senior engineers who can explain and document complex pieces of software save everyone frustration by helping junior members of the team understand how things work and be more productive, and by aligning business expectations and plans to the reality of how their software systems actually work. These are hugely valuable activities that can make a big difference in your career.

3. Writing will help you find a (better) job

When I’ve interviewed software engineers, I see a variety of problems: they fail to present their strengths, don’t manage to show a sincere interest in the job, and get caught flat-footed on fairly standard questions. Most of these mistakes come down to a lack of preparation, not tailoring their message to their audience, and not knowing how to structure the information they know about themselves in a compelling way. These are the skills that you’ll learn from a good class on writing, and develop through practice writing.

Interviews are important, especially early on in your career. Your compensation as you go throughout your career tends to be anchored to where you started, especially if you stay one place for a long time. An increase of $5,000 in salary on a job you take today would mean an additional $92,000 of accumulated income over the next 15 years assuming a salary increase rate of 3% a year².

In addition if you write publicly on a blog like this, it can help you show proof of your knowledge in a way that a resume never will. Not everybody needs to be publicly blogging, but if that appeals to you it can be a great career booster.

4. Remote work increases the importance of writing

One of the biggest trends in the tech industry these days is the rise of remote work. More teams are embracing partial remote work, and there are major companies like Buffer, Invision and GitLab that are entirely remote. Remote work is partially enabled by video chat technology, but written communication is the lifeblood of a successful remote team. Collaborating via tools like Slack, Github, Trello and Jira is part of most teams' workflows, but when everybody is remote, being able to express yourself clearly on these platforms is critical.

5. Your career trajectory will depend on your ability to influence and persuade

Most developers when they begin their first software job will be given a set of tasks to do and the expectation is that they will create work to the spec that somebody else has designed. They are implementors. But as a developers career progresses there is an opportunity to be more than an implementor. Since developers understand the system best, they often have ideas that can help shape product designs. Developers also often need to work with other teams whose goals may not be completely aligned with their team. These are not situations where even a senior engineer is likely to have direct authority. Instead the developers who continue to grow in their careers are those who are able to influence and persuade without authority.

This is again a skill that writing classes can cultivate. Persuasive writing practice can help give you the clarity of thought and intuition for structuring arguments that will help both when you’re trying to make a point through writing or doing it verbally.

6. Writing skills translate to writing code

This last one is really specific to software engineering, but I’ve found that applying writing principles to code has helped me write more maintainable code. Most code written in a professional context will be read many times by other people (or by the original author after they have long lost track of their original context). Being able to write code that can be understood by both a computer and by human beings is a skill developed over time, but many of the same “brain muscles” that you develop in writing prose can be applied to code: it’s important to know your audience, structure your information in a logical manner, and use judgement when finding just the right phrase (or code structure) for a situation. That could be a whole post in itself.

I write based on the tech industry and software development specifically because that is what I have lived, but most of these points should be equally applicable to any engineering career and would likely require very little modification for science and research as well. ↩︎
Ok real world promotion rates in the tech industry are likely to be choppier than that, but the example holds. ↩︎

Strategic Coding

Sun, 17 Feb 2019 20:00:00 -0400

I recently read A Philosophy Of Software Design by John Ousterhout. In an early chapter he paints a picture of a “tactical tornado” programmer, a programmer who is always focused on solving the current problem by the most expedient method possible, with no thought to the long term implications. I suspect anyone with even a few years in the programming has seen programmers like this, and has also from time to time written “tactical” code like this themselves. Ousterhout’s recommendation is instead to write code strategically rather than tactically, taking more up front time to plan the structure for the benefit of longer term productivity.

Ousterhout focuses his examination of tactical vs strategic coding on the issue of complexity. Complexity as he defines it is anything that makes the system difficult to work with. It is the root issue behind much of the ease of change concern I identified in my recent post. But we can expand this to encompass all of the front end concerns. We can easily impact reliability, performance, ease of change, accessibility, security and capability when we make short hand tradeoffs.

Let’s consider the problem of getting a new sign up page set up for a site. There are several different approaches we could take. By far the simplest is a basic HTML page using normal CSS and little or no JavaScript, that posts to a backend service after the signup and then redirects to a different static page. We could set this up on S3 or any other service that will host static files. On the other extreme, we could set up a new server side rendered Typescript-based React application that uses Next.js to give us server side rendering, which we deploy using Docker and a fully automated CI build system with chatbot ops. Which one of these approaches is better?

A tactical tornado would likely prefer the first solution. They could get the whole thing up and running in an hour, stamp it and call it done. Other developers might be attracted to the cool approach and want to avoid the “outdated” approach. But we can only really answer the question by stepping back and asking some questions:

What is the future of this application? Is this a standalone page, or the first page in a larger application?
What feature requests are we likely to get for the page? Is it probable that we’ll eventually need form validation, dynamic success and failure states, complex behavior that depends on the use case?
What browsers are we supporting
What are our performance benchmarks?

Questions like that can tell us when we’ll benefit from an up front investment in certain areas, and when it’s wiser to be quick and dirty. They also allow us to evaluate whether any options we’re considering might cut us off from achieving long term goals. For instance, if we know that we’ll eventually need to scale this site into a large application with responsive interactions, personalized content and information maintained between page loads, building this out as a series of static pages which POST to a backend would eventually limit our capabilities or require a refactor. But if this is only a one-off landing page for an event happening next week that won’t be maintained beyond that point, developing a long-term maintenance strategy would be wasted effort. This is strategic coding, and it doesn’t stop at the beginning of a project, it’s an ongoing mindset that you can take as you add each new feature to an application.

Ousterhout makes a case that aligns with my experience, that strategic programming is a long term winner. I’ve seen that over time improvements to ease of change will often help a team save time lost, significant hits to performance, security, reliability may hurt them more than a fast solution saves them, and solutions that unreasonably limit us in terms of system capability or accessibility will ultimately get rejected. The weasel words there exist for a reason… sometimes it really is the best idea to just get something working. But a strategic programmer will understand the context of their project, and make wise choices about where and how to invest in the codebase.

TL;DR

It is possible to program either tactically or strategically
Tactical programming is focused on solving the immediate problem as quickly as possible
Strategic coding considers the context of the problem, and any architectural concerns involved in a change
Strategic coding doesn’t mean never hacking and always investing, it means letting context inform the amount and direction of investment in a codebase

PS: I put a full review of A Philosophy of Software Design up on the site a few weeks ago. Check it out :)

Reusable Code In React: Inheritance, Composition, Decorators and Mixins

Sun, 10 Feb 2019 20:00:00 -0400

Once you get past the initial complexity it creates by bucking conventions, React is the most developer friendly UI library I have ever used. A big reason for that is the way it uses and encourages a variety of patterns to share codes and concepts across a code base. Let’s look at how 4 traditional code re-use patterns are used by React and in the React ecosystem, using a simple counter example to explore why each pattern is used.

Inheritance

Inheritance is a code-reuse technique associated with object oriented programming. If you’re like me and learned to program in school using Java, it probably is one of the very first Computer Science concepts you learned. For self-taught JavaScript devs it may be much less familiar. Inheritance is a pattern for extending an existing class or object with additional methods or data, possibly overriding the implementations of other methods in the process. A classic example of inheritance looks like this in JS:


class Automobile {
    constructor() {
        this.vehicleName = automobile;
        this.numWheels = null;
    }
    printNumWheels() {
        console.log(`This ${this.vehicleName} has ${this.numWheels} wheels`);
    }
}

class Car extends Automobile {
    constructor() {
        super(this);
        this.vehicleName = 'car';
        this.numWheels = 4;
    }
}

class Bicycle extends Automobile {
    constructor() {
        super(this);
        this.vehicleName = 'bike';
        this.numWheels = 2;
    }
}

const car = new Car();
const bike = new Bicycle();
car.printNumWheels() // This car has 4 wheels
bike.printNumWheels() // This bike has 2 wheels

Inheritance is useful for highly structured objects where you have a lot of shared behavior across similar but slightly different objects and you want to be able to treat all of the objects as interchangeable from an API perspective. This is especially valuable when that interchangeable API is well defined and unlikely to change too much. However it tends to lead to brittle structures when nested across multiple levels, and is a poor fit when dealing with dissimilar objects that need to share a few behaviors.

React Components are objects that we want to treat as interchangeable from an outside API perspective, and they have well defined points where behavior may need to be different between components but don’t always need to differ: the lifecycle methods like render, componentDidMount, componentWillUnmount, etc. Thus React relies on a single level of inheritance as the foundation of its class components. Class components inherit from React.Component, and only implement the life cycle methods that they’re interested in. But all of them inherit useable setState and forceUpdate functions, and React can verify that by checking the isReactComponent property on the object.

Here is a basic counter component. Notice how we don’t have to define setState on our Counter class. Instead it inherits it from React.Component.

//counter.js
class Counter extends React.Component {
    constructor(props) {
        super(props);
        this.state = {
            count: 0,
        }
    }
    render() {
        let {count} = this.state;
        let increment = () => this.setState(state => {
            return {
               count: state.count + 1
            };
        });
        return <div className="fancy-layout">
            <div className="fancy-display">
                Current Count: {count}
            </div>
            <button
                className="incrementCounterBtn fancy-btn"
                onClick={increment}
            >Increment</button>
        </div>
    }
}

Outside of this single layer of inheritance for defining Class components, inheritance isn’t used much in React development. It turns out that better patterns have emerged for sharing application code.

Inheritance Pattern: Used sparingly to share common code across React class components.

Composition

Composition is a code reuse technique where a larger object is created by combining multiple smaller objects. Composition is in some ways the natural pattern of the browser, as HTML uses it. We build a table in html, not by adding a bunch of configuration to a single table object, but by combining a table tag with trs, tds, and ths like this:

<table>
    <thead>
        <tr><th>Web Framework</th><th>Language</th></tr>
    </thead>
    <tbody>
        <tr><th>Rails</th><th>Ruby</th></tr>
        <tr><th>Django</th><th>Python</th></tr>
        <tr><th>Flask</th><th>Python</th></tr>
        <tr><th>Play</th><th>Scala</th></tr>
        <tr><th>Play</th><th>Java</th></tr>
        <tr><th>Express</th><th>JavaScript</th></tr>
    </tbody>
</table>

React has fully embraced the HTML style of composing UI components, and it takes thing to the next level by allowing components to be stateful and pass values down into children. For instance, if we wanted to split the counter component above into a separate control and display, perhaps using a standard set of UI components we could do it by extracting FancyLayout, FancyButton and FancyDisplay elements out into their own components and then composing them back into a single counter component. This type of thing is useful for creating and reusing standard UI elements, and might look something like the following:

// UI components defined elsewhere
const FancyLayout = ({children}) => <div className="fancy-layout">{children}</div>;
const FancyDisplay = ({children}) => <div className="fancy-display">{children}</div>;
const FancyButton = ({children, className, onClick}) => <button
    className={`fancy-layout ${className}`}
    onClick={onClick}
    >
        {children}
    </button>;


// counter.js
class Counter extends React.Component {
    constructor(props) {
        super(props);
        this.state = {
            count: 0,
        }
    }
    render() {
        let {count} = this.state;
         let increment = () => this.setState(state => {
            return {
               count: state.count + 1
            };
        });
        return <FancyLayout>
            <FancyDisplay>
                Current Count: {count}
            </FancyDisplay>
            <FancyButton
                className="incrementCounterBtn"
                onClick={increment}
            >Increment</FancyButton>
        </FancyLayout>;
    }
}

In this example we now have 4 components instead of one. Three of them are laser-focused on defining a specific element type, and are “display only” with no logic of their own. The 4th contains all of our logic and defines how the display pieces come together.

In the seminal 1994 book Design Patterns the authors argue that composition should be strictly preferred over inheritance where possible because it creates less tightly coupled objects, and encourages designs where objects are small, focused and readable. Composition also lets us hide complexity inside of child objects, because parent components only have to concern themselves with the APIs exposed by those child objects. One of the keys to React’s success has been making this type of composition clean and painless by encouraging strong component interfaces and utilizing JSX to provide a comfortable syntax for describing composed UIs while keeping all component concerns in the same place¹.

It is worth noting that after this refactor, our counting logic is still tightly tied to the layout of our counter component. Let’s explore 2 more patterns that can help us extract our logic from the display completely.

Composition Pattern: The core pattern for separating concerns while creating complex UIs with React.

Decorators

Decorators are a pattern for editing an instance of a class to give it additional behaviors than it had previously. In contrast to inheritance, decorators are not part of the class definition, but are modifications to the class at run time to allow a subset of objects to have additional data or behavior ². In classic object oriented programming, this was implemented by having a decorator class that shared an interface with the class to be decorated, and called its methods through to that class, with additional behavior before or after the pass-through. The React community has generally implemented this pattern slightly differently, by writing functions that take a component class and return a new component class that calls through to the original component, possibly with additional props passed in, extra components rendered around it, or conditional logic that causes it not to render in some cases. The React community generally refers to this pattern as Higher Order Components.

Here’s an example of using a higher order component called withCounter to separate out the display and logic portions of our Counter component.

const withCounter = WrappedComponent => {
   class ComponentWithCounter extends React.Component {
      constructor(props) {
         super(props);
         this.state = {
            count: 0
         };
         this.increment = () => this.setState(state => {
            return {
               count: state.count + 1
            };
         });
      }

      render() {
         let {count} = this.state;
         return <WrappedComponent count={count} increment={this.increment} />
      }
  }
  return ComponentWithCounter;
};

// counter.js

// this is a display only version of the component with no logic
// very easy to test and simple to change
class CounterDisplay extends React.Component {
    render() {
        let {count, increment} = this.props;
        return <FancyLayout>
            <FancyDisplay>
                Current Count: {count}
            </FancyDisplay>
            <FancyButton
                className="incrementCounterBtn"
                onClick={increment}
            >Increment</FancyButton>
        </FancyLayout>;
    }
}

// this adds the logic back to the display only version
const Counter = withCounter(CounterDisplay);

Now we’ve taken our refactored Counter and split it into 2 more pieces. The display logic remains in a simplified CounterDisplay component that doesn’t have any state or business logic of its own. CounterDisplay takes 2 new props, count and increment that allow it to implement the counting logic without having to know how it is implemented.

Meanwhile, we split the counting logic out into a Higher Order Component. withCounter takes a React component and wraps it with another component that is solely responsible for keeping track of counts and providing the increment function. In this case that is our CounterDisplay component, but if we decided we wanted to also have fancy counters that incremented when the user clicked a checkbox or hovered over a box or typed the word increment, it would be easy to reuse this logic.

Higher Order Components helped many React libraries like Redux and React Router define elegant APIs when they first came on the scene, but they’ve always been a power user technique. There are many ways to screw them up, and although they allow for theoretical loose coupling, they still require a certain level of shared understanding between the decorator and decoratee, including knowing what props to make available, whether any methods have to be proxied through to the decorated component, and more. The community as a whole has tended to favor render props over the past year³, and now looks poised to move towards our 4th pattern: mixins.

Decorator Pattern: Used to provide a nice interface for separating out logic shared by multiple components and centralizing it.

Mixins

The newest pattern that React has added to its bag of code reuse tricks is also one of the oldest. In older versions of React there was a feature called “mixins” that was later deprecated due to its ease of mis-use. “Mixins” are gone now, but the “Mixin pattern” is back! React is in the process of introducing hooks, a new way of doing mixins.

So first, what is a mixin? It is yet another way of defining shared behavior onto a class, in this case providing methods and/or data that are “mixed in” to one or more classes. It differs from inheritance because a class using a mixin doesn’t have to use the mixin as a parent class, and the class may use several mixins at once. Mixins that are used together in one class may not be used together in another. Mixins also different from the decorator pattern, because the mixin is part of the class definition, not something that is applied dynamically later.

Mixins are useful when you have classes with different purposes that shouldn’t share an inheritance tree but do have some shared behavior. In practice though they have historically been tricky to use because it is easy to tightly bind mixin code to the implementation of the code that it is mixed into. Poorly implemented Mixins can result in naming collisions, inflexible code, and hard to trace errors.

React Hooks may not look like mixins, since they are used in a function, not a class. But they fill the exact same role as traditional mixins, while avoiding some of the footguns (no naming collisions!). Let’s look at an example from Dan Abramov’s blog post introducing hooks.

function useWindowWidth() {
  const [width, setWidth] = useState(window.innerWidth);

  useEffect(() => {
    const handleResize = () => setWidth(window.innerWidth);
    window.addEventListener('resize', handleResize);
    return () => {
      window.removeEventListener('resize', handleResize);
    };
  });

  return width;
}

This is a hook that provides the width of the current window to a React component, updating each time the window is resized. You would use it in practice like this example of a header bar that replaces its menu items with a hamburger menu on smaller screens:


const Header = () => {
    let windowWidth = useWindowWidth();
    let showHamburgerMenu = windowWidth < 800;

    return (<div>
        <Logo>
        {showHamburgerMenu ? <HamburgerMenu/> : <MenuItems/>}
    </div>)
}

This usage is different from a normal helper function, because it enhances the component; because this is using React’s state code within the hook, this component will automatically update when needed. So it is a way of reusing component behavior like Higher Order Components. But we lose a lot of the disadvantages of HOCs. useWindowWidth and Header are extremely loosely coupled. useWindowWidth doesn’t know anything about the header bar structure or logic, or the rules for when things should show. It is only providing information. At the same time, if we refactored it to get the window size in some other way, or to throttle updates, we wouldn’t have to change anything about Header, since all it knows is that useWindowWidth will occasionally provide it an updated windowWidth.

The functional style makes this look different than a traditional object oriented “mixin pattern” example, but it is using the same concepts and solving the same problems. This way of doing things has advantages; because it is “just a function” that gets called, we never need to worry about it having a naming collision with a different class method, or having to be aware of the structure of the outer component. If it needs component specific information, we can just pass it an argument.

With Hooks, we can create a much cleaner version of our Higher Order Component example from the decorators section.

// defined in a library file
const useIncrement = (initial = 0) => {
   const [count, increment] = useReducer(val => val + 1, initial);
   return [count, increment];
}

// counter.js
const Counter = () => {
    let [count, increment] = useIncrement();
    return <FancyLayout>
        <FancyDisplay>
            Current Count: {count}
        </FancyDisplay>
        <FancyButton
            className="incrementCounterBtn"
            onClick={increment}
        >Increment</FancyButton>
    </FancyLayout>
}

As with decorators we’ve extracted the logic in a reusable form, but we’ve done so with much less boilerplate. I’ve removed the intermediate CounterDisplay here, because if we’re not using it for anything else, it is easy enough to just include the hook directly in the definition. But if we wanted, we could easily use that CounterDisplay component with a hooks enabled wrapper component:

const Counter = () => {
    let [count, increment] = useIncrement();
    return <CounterDisplay count={count} increment={increment}/>;
}

Mixin Pattern: Hooks use a variation on the Mixin pattern to allow sharing related behavior and data between unrelated function components easily.

Summary

If I took one thing out of this analysis personally, it was a renewed appreciation for considering the needs of a situation and finding the right approach for the job, rather than trying to fit every problem to my favorite solution. I’m over 2000 words and I haven’t even addressed all the different strategies React and the React community use for code re-use. Sorry render props 😢. So next time you run into an architecture problem, consider a few different solutions and their tradeoffs.

TL;DR

React uses a variety of patterns for code reuse. Inheritance Pattern: Used sparingly to share common code across React class components. Composition Pattern: The core pattern for separating concerns while creating complex UIs with React. Decorator Pattern: Used to provide a nice interface for separating out logic shared by multiple components and centralizing it. Mixin Pattern: Hooks use a variation on the Mixin pattern to allow sharing related behavior and data between unrelated function components easily.

Clunky JS based interfaces for specifying child components are visually difficult to scan and understand, while solutions that use external template files spread logic about a single component to multiple files, making it harder to keep everything in the developers head. ↩︎
Note that the decorator pattern is not the same thing as the TC39 proposal to add a decorator syntax to JavaScript, or the TypeScript feature of the same name. Both are related to modifying class behavior, but the mechanisms are not the same. ↩︎
Render props ultimately qualify as an advanced composition technique and so didn’t get their own heading here, but they’re definitely worth understanding. ↩︎

Book Review: Info We Trust

Tue, 29 Jan 2019 20:00:00 -0400

Sometimes surprises are good. I bought RJ Andrews new book Info We Trust expecting to get some practical tips on how to create better data visualizations. But frankly there’s very little in the way of “how to” in this book. Instead I was treated to a deeply engaging book that’s part history of data visualization, part exploration of human psychology, and 100% a love letter to “data storytelling”, with brief detours into linguistics, museum curation sentence diagramming and propaganda posters. If you’re looking to know when to use a bar chart vs a pie chart in your powerpoint slides, this book will be a disappointment. But if you want to learn how to think deeply about telling stories with data, or just enjoy seeing somebody who loves their topic nerd out about it for 200 pages in a beautiful formatted book, this may be for you.

Let’s start with that beautiful book. This is not a read that you want to pick up on Kindle. The book is littered with fascinating illustrations, all hand drawn by the author (!!!), and almost every page has interesting quotes, examples and tangents in the marginalia. Here are a few random pages that I snapped with my phone while writing this; it turns out that every single page is this beautiful.

In addition to the eye candy, Andrews mostly avoids giving readers rules, but instead dives deep into the whys of data storytelling. Things I was left thinking about:

The way I think about bars vs columns differently based on my different assumptions about distance vs stacks of items
How the plethora of data available in the world today has a cost: our attention. That has left me thinking both about designing interfaces that manage attention, but also how to reclaim my own distracted attention.
The difference between data and information, and how each can be useful. How in the product I work on we can help users get access to enough raw data to explore on their own, but also give them meaningful “information” derived from that data.
I was motivated to get better at drawing, and explaining things with images in my day to day work. I don’t do this enough.

If all of that sounds too high level and impractical for you, this is probably not the book for you. But anyone who is already a data viz geek will find something to love, and as somebody who is tangential to the field it certainly expanded my mind and gave me a lot to think about. All in all, this one is easy to recommend. Check it out!

Improving Code Readability With Async/Await

Sun, 27 Jan 2019 20:00:00 -0400

When it first became a part of JavaScript, I wasn’t sure how much I was going to use async-await. I could see that it made things prettier in some cases, but I was pretty happy with my promise chains and thought that async-await might just be some extra language baggage. Since then I’ve fully converted; it’s one of my favorite features added to the language in recent years. The reason? It helps you write code in a linear manner.

A principle of readable code:

Code should where possible be written in the order in which a reader needs to understand it.1

For complex classes, this ideally means that the “entry points” are towards the top, with the helper methods below to be read as needed¹. But for “linear” code that reflects a workflow, we want the code to progress in chronological order as much as possible. With imperative code that tends to happen by default. But JavaScript’s asynchronous constructs have tended to obscure chronological orders.

As an example, here is a React component that displays the time remaining in a user session based off of some cookie values. It checks a cookie on a regular interval to see whether it should display a warning about the session timeout. Once it is showing the warning, it begins checking the cookie more frequently in order to show a timeout. Once the session expires, it can stop checking.


// When time is less than this, we should show warning
const MAX_TIME_TO_HIDE_WARNING = 1000 * 60 * 2;

// intervals in seconds to update timeout count
const CHECK_INTERVAL_WHEN_HIDING = 30 * 1000;
const CHECK_INTERVAL_WHEN_SHOWING = 0.5 * 1000;

class ShowSessionTime extends React.Component {
    constructor(props) {
        super(props);
        this.state = {
            timeRemaining: MAX_TIME_TO_HIDE_WARNING + 1,
        };
    }

   /* Update the remaining time till session timeout */
    updateRemaining() {
        let currentTime = Date.now();
        let timeRemaining = getCookie('serverExpiry') - currentTime;
        this.setState({
            timeRemaining,
        });
        return timeRemaining;
    }
    componentDidMount() {
        // while the modal is open, update every cycle but stop when the time remaining is 0
        const whileOpenCheck = () => {
            let timeRemaining = this.updateRemaining();
            if (timeRemaining >= 0) {
                setTimeout(whileOpenCheck, CHECK_INTERVAL_WHEN_SHOWING);
            }
        };
        // while the modal is closed, update every cycle but switch to the open cycle when we reach the threshold
        const whileClosedCheck = () => {
            let timeRemaining = this.updateRemaining();
            if (timeRemaining > MAX_TIME_TO_HIDE_WARNING) {
                setTimeout(whileClosedCheck, CHECK_INTERVAL_WHEN_HIDING);
            } else {
                setTimeout(whileOpenCheck, CHECK_INTERVAL_WHEN_SHOWING);
            }
        };
        // kick off our timeout loop
        setTimeout(whileClosedCheck, CHECK_INTERVAL_WHEN_HIDING);
    }

    render() {
        let {timeRemaining} = this.state;
        let showWarning = timeRemaining > MAX_TIME_TO_HIDE_WARNING;
        return showWarning ? <div>Time Remaining In Session: {timeRemaining} </div> : null;
    }

The componentDidMount method is the interesting part here. Notice how the setTimeout based flow has resulted in the flow being reversed?

    componentDidMount() {
        // THIS PART HAPPENS LAST IF AT ALL
        const whileOpenCheck = () => {
            let timeRemaining = this.updateRemaining();
            if (timeRemaining >= 0) {
                setTimeout(whileOpenCheck, CHECK_INTERVAL_WHEN_SHOWING);
            }
        };
        // THIS IS THE FIRST RECURSIVE LOOP
        const whileClosedCheck = () => {
            let timeRemaining = this.updateRemaining();
            if (timeRemaining > MAX_TIME_TO_HIDE_WARNING) {
                setTimeout(whileClosedCheck, CHECK_INTERVAL_WHEN_HIDING);
            } else {
                setTimeout(whileOpenCheck, CHECK_INTERVAL_WHEN_SHOWING);
            }
        };
        // THIS IS THE ENTRY POINT
        setTimeout(whileClosedCheck, CHECK_INTERVAL_WHEN_HIDING);
    }

The code is super hard to follow because its defined all out of order. You have to understand the whole thing before you can wrap your mind around any piece of it, and your eye is going to be wandering back and forth. Compare that to this reimplementation with async/await:

    async checkRemainingAtIntervals() {
        while (this.updateRemaining() > MAX_TIME_TO_HIDE_WARNING) {
            await sleep(CHECK_INTERVAL_WHEN_HIDING);
        }
        while (this.updateRemaining() >= 0) {
            await sleep(CHECK_INTERVAL_WHEN_SHOWING);
        }
    }

    componentDidMount() {
        this.checkRemainingAtIntervals();
    }

We’ve pulled the logic out into a separate async function and suddenly everything is so much simpler: we’re just using basic loops, with each loop condition updating the remaining time and then checking to see if we should keep looping. Sleep in this case is just a promisified setTimeout:

export const sleep = millisecs => new Promise(res => delay(res, millisecs));

In fact this version is so much simpler that when I originally wrote the code that this post is based on, I immediately saw a bug that I had missed in the complex version: the time remaining in a session can go up as well as down. If a user makes a request after I show the warning, the interval will go up and we should re-hide the warning. But my first few takes both ignored that complexity. Fortunately it’s easy enough to add:

    async checkRemainingAtIntervals() {
        let remaining = this.updateRemaining();
        // keep checking until the session expires
        while (remaining >= 0) {
            let warningIsShowing = remaining > MAX_TIME_TO_HIDE_WARNING;
            let interval =  warningIsShowing ?
                CHECK_INTERVAL_WHEN_HIDING :
                CHECK_INTERVAL_WHEN_SHOWING;
            await sleep(interval);
            remaining = this.updateRemaining();
        }
    }

We’ve actually gotten even simpler here, with a single loop that runs until the session expires. Each time the loop runs, it checks to see whether it should wait for a long or short interval before updating the remaining time again. Everything happens in ~8 highly readable lines that will be easy to follow for anyone who has used a while loop.

Compared to our original version we have

Cut out several “intermediary variables” that were mostly managing complexity around async, but weren’t part of the core problem
Matched the visual flow to the workflow
Simplified enough to catch a bug that I missed originally

I’m going to continue to use async-await as the basis for most of my asynchronous code going forward for benefits like this. If you’ve been holding out, browser support has gotten very good and this is a great time to jump in.

TL;DR

Async Await can help readability by matching code structure to the program workflow
Code that is structured more naturally can be easier to debug and see errors in

React’s convention of putting render at the bottom of class components has always annoyed me as a result of this. ↩︎

Designing Front End Apps For Reliability

Sun, 20 Jan 2019 20:00:00 -0400

In my recent post on Front End Architecture I identified Reliability as a key concern of front end architecture. I define reliability as follows:

An application’s ability to provide a consistent functional experience to all users in the intended audience over time.

Let’s break that down a bit:

Consistent - Reliable applications function the same way over time or degrade gracefully, even on days with heavy load, leap years, and weekends.

Functional - Reliable applications allow the users to do the things they need to do, and are forgiving when a user attempts actions that weren’t part of an ideal use case.

All Users In The Intended Audience - Reliable applications work for users with strange data who live in different locales, and those with bad network connections and old browsers, at least within the limits of what what the business intends to support. There is some overlap here with the concept of accessibility.

Over Time - In the real world, no application is ever completely reliable. Things break down, there are bugs, and new edge cases are discovered. What we can aspire to though, is an application where things that are working consistently stay working consistently, and as bugs, regressions and edge cases are discovered they get fixed in permanent ways.

Things Break Down

So why are applications often unreliable? There are a fairly predictable set of issues that most applications run into that degrade reliability.

Unexpected Usage Patterns

Users are agents of chaos and the real world is stranger than expected. Your sample database and QA testing doesn’t come close to giving you a true idea of how your application will be used.

External Failures

Google never has bugs right? Good luck with that. Even if the rest of the world always wrote perfect software, front end code usually has to deal with the network, even when your user is on a plane, accessing your site over 2G in the mountains, or just dealing with crappy coffeeshop wifi.

Bugs

Bugs happen. When they happen consistently and repeat themselves, you have a reliability problem.

Scaling Problems

Some things work great on small data sets or with a few users but break down when things get popular. This isn’t just a backend issue. Data scales as well as people, and your program needs to be robust to slower backend responses when under load.

Keeping Things Running

I’d like to offer some practical strategies for overcoming the challenges I’ve listed. These are things you can consider during code reviews and when setting up your dev processes.

1. Consider all states, and limit them if you can

One of the big leaps in JavaScript based UI development over the past decade has been moving from systems that we modeled as an initial UI state followed by a series of mutations (server generated HTML + jQuery) to systems that were modeled as partially driven with event systems that caused mutations to the UI (Backbone, Angular 1, Knockout), to systems that model the UI as a function of state (React, Vue, Angular2+, Modern Ember) State-driven systems tend to be more robust, because it is often more clear when a particular state might cause a problem, and it is more obvious when a problematic state occurs for a user how it happened.

Visibility into state is a huge step forward. The next step is to define the possible states. You can do this in your head, but it’s much easier when using types or by thinking in terms of state machines. If you can map out different portions of your applications into a limited set of states and define behavior for each of them, reliability will be much easier to achieve.

The final step is to limit these states where possible. Some interfaces depend on user input or external data enough that they naturally are effectively “infinite state machines”, but when you can limit states by cutting scope, rearranging components or cordoning “infinite state machines” off into a smaller area of your application, you gain a lot of benefits.

2. Don’t trust the network

This one is straightforward, and easy to check in code reviews. Don’t assume that anything over a network will return quickly. Don’t assume it will return correctly. And in fact, don’t assume it will return at all. You may be limited in how you can handle these possibilities, but the baseline competence here is handling them. Have strategies for loading states, timeouts, error handling and bad formats. Things like service workers can help here, but also make the problems around the network even more complicated by introducing a new caching layer. When you start making calls over a network, you’re entering the realm of distributed systems and things get tricky fast. But the least you can do is recognize the complexity and make a plan for it.

3. Consider variations in data

Do you have a test data set that you develop against? Is that your primary source of validation before you send code off into the wild? Some of that may be unavoidable and out of your control, but remember that users are agents of chaos. You will encounter data you didn’t expect, so plan ahead for challenges. How would your data visualization function if the dataset you ended up displaying was 2x the size of your test database? 10x? 1000x? What if the list of products to display in your table contained product names 200 characters long? Or if some of them were empty? The real world is strange and you should be ready.

4. You should have testing

This is another topic that’s too big to cover fully here, but basically there’s a reason that QA engineers have jobs. You should have a plan for testing. That plan will be better if it contains meaningful amounts of automated regression testing, with a clear process for making sure bugs are tested automatically once they’ve been discovered and fixed. If you don’t know where to start, always focus on testing the most important things well over testing everything mediocrely. And if you’re not in a position to do meaningful automated testing, consider what it will take to get there.

5. Monitoring

Finally, sometimes it’s just not possible to catch problems before they ship. But you can still catch them before too many of your customers do if you put in the work. I’ll be honest, I’ve never had a monitoring setup that I’ve been happy with at work, but on the front end good monitoring can mean sending console logs over the network so that they’re shared with whatever backend logging solution you’re using, tracking feature usage through analytics to look for any large problems or lack of usage that may indicate bugs, and even more advanced stuff like session recording in some cases.

TL;DR

Reliability is an application’s ability to provide a consistent functional experience to all users in the intended audience over time.
Reliability breaks down due to unexpected usage, external failures, bugs, and scaling problems
You can boost reliability by defining and limiting application states, distrusting the network, thinking about variation in data, testing and monitoring

6 Keys To Valuable Code Reviews

Sun, 13 Jan 2019 20:00:00 -0400

I’ve read 2 things lately that really reminded me of the value of code reviews. One was A Philosophy Of Software Design by John Ousterhout, which I reviewed here. Philosophy of Software Design is a practical guide to software design, and Ousterhout emphasizes code reviews as the best point for “practicing architecture” in real world projects. The 2nd was Sophie Alpert’s recent blog post Why Review Code, where she lists the various advantages she’s seen from having code reviews. I can’t improve on her list, but I can endorse it. I’ve seen all of the things she listed as benefits from code reviews in the past. However I’ve seen these benefits come out in some environments more than others. So I wanted to share 6 practical tips to getting the most valuable possible out of your team’s code reviews.

1. Make sure that code reviews are multi-directional

When you start doing code reviews, it can be tempting to only have senior developers review code. After all, they’re the ones who know the system best. But not only do you risk burning out your senior staff that way, you also miss out on some of the biggest benefits of code reviews. You get different benefits with different team combinations. I’m using Junior and Senior here relatively speaking, the roles may change over time or in different areas of the codebase.

Senior Reviewing Junior - This is the first thing most folks think about. This is an opportunity for a more experienced developer to catch bugs in a less experienced developer’s code and also give suggestions for improving architecture-level structure, introducing new ways of doing things, or catching when there are opportunities for code consolidation that the original author might not have known about.
Junior Reviewing Senior - When a less experienced developer reviews the code of a more experienced developer, its an opportunity for them to see new ways of doing things, and also get “cheap” exposure to new areas of a codebase. It’s also a great opportunity for them to ask questions about things they don’t understand. But it shouldn’t be a one way street. Often times a fresh pair of eyes are great at catching bugs, and if a code review is particularly confusing to a less experienced dev, it’s a sign that the code may need to be rewritten or better documented so that it can be maintained in the future.
Peer Reviewing Peer - When 2 developers have roughly equal skill and experience with the codebase, code reviews are still an opportunity for a second pair of eyes, but they also have the benefits of building team collective knowledge about the codebase and exposing times when there are differences in approach across the team. Conversations at this level are a great place to standardize on best practices and team conventions.

2. Hold reviewers responsible for the code they review

If a bug makes it past initial development efforts and is caught by QA or ends up in production, who is responsible for it? Although there will ideally always be layers of accountability here, most developers I know naturally tend to hold themselves responsible when code they wrote breaks something, but also apply the same standard to others. This can lead to rubber-stamp code reviews when things get busy, since there’s no “skin in the game”. The benefits of code review tend to come out though, when developers take that same responsibility for the code they review. A team culture that sees the reviewer and the developer as equally responsible for maintainable code, bug-smashing, and other architecture concerns will lead to better code reviews. This means treating code review as a core job responsibilities for devs and rewarding those who do it well, rather than just tacking it on.

3. Match technology to your process

The teams I’ve worked on that didn’t have real code review processes also didn’t have an easy setup to make it work. They were reviewing diffs manually in their editors or over email. Tools like Github, Bitbucket and Gitlab have made this much easier. If you want this to be part of your process, make it as easy as possible.

4. Use checklists where appropriate

In moderation, I’ve found checklists a useful tool for code reviews. If your team has specific areas of focus that you want to improve compliance on, it can be helpful to give reviewers a list of items to sign off on. For instance:

Verify change supports internationalization
Verify change doesn’t add any new code with legacy framework
If change is user-facing, verify it includes documentation updates

You can take this too far, the review process shouldn’t be reduced to repetitive box-ticking. But these can be great as guides for implementing new initiatives or addressing problem areas in code.

5. Make code reviews as atomic as possible

10 lines of code = 10 issues.

500 lines of code = "looks fine."

Code reviews.
— I Am Devloper (@iamdevloper) November 5, 2013

Huge code reviews are overwhelming, and much easier to rubber stamp. When you develop a culture of creating small focused issues and reviewing each of them, it leads to more interaction and better results. Single issue (atomic) code reviews are best. As a rule of thumb, if you can describe the change without using the words “and”, “rewrite”, or “a couple of different”, you’re probably doing alright.

6. Leave space to make changes

Finally, code reviews will have the most value if as a team you’re committed to acting on the feedback received. That means being supportive when a code review leads to a re-write or refactor. Sometimes that will need to be delayed of course, but in a healthy process there should usually be time to act on feedback. This is of course easier if you’ve followed the previous suggestion and kept your code reviews small.

What's involved in Front End Architecture?

Sun, 06 Jan 2019 20:00:00 -0400

The word “architecture” gets thrown around a lot in software discussions. But we don’t seem to always use it the same way. Some example usages:

Our app uses Redux for our architecture.

Sometimes we assume that because we are using a framework or data management library, that must be our architecture.

I created a diagram of our dependency graph, showing all the imports in the code, as a way of seeing our architecture.

We could also say the way your program is structured (which files depend on which other files) represents your architecture. If we’re fancy, then maybe we have UML diagrams representing this structure.

Bob is our architecture guy: he decides what dependencies we can bring in, is always asking us to refactor code to make it “cleaner”, and runs a design meeting before each feature to decide how to structure the code

This final quote sounds like architecture is more about the decisions you’re making as you go through a project.

Appealing to authority?

So if the community isn’t using the term consistently, let’s ask the experts. Here are 2 definitions from folks who have written popular books on software architecture.

Robert Martin (Uncle Bob) uses the following definition in Clean Architecture:

The goal of software architecture is to minimize the human resources required to build and maintain the required system.

Martin Fowler gave his definition in a talk titled Making Architecture Matter. It is worth watching in full, but paraphrased, his definition is:

Software architecture is those decisions which are both important and hard to change

These 2 definitions turn out to be complementary. Uncle Bob motivates architecture: we’re focused on minimizing the work put into the system. Martin Fowler’s definition shows what parts of the system architecture is concerned with: the parts that are “important” and hard to change.

The next question of course is… what is important? The unsatisfactory answer is that it depends. It will change between projects, and will change within a project over time. But when we scope the question to the domain of front end development, common themes emerge. Specifically there are 6 areas of concern that I see coming up again and again in public and private front end architecture discussions. They’re not exclusive to the front end, but they’re a good starting point for anyone thinking about building web applications.

Ease of Change

When new requirements arrive, how quickly are developers able to make changes and release new code to production? When there are bugs, how easy is it to find the problem and fix it? Are developers able to consistently deliver correct code? Or are multiple iterations often required?

Security

Can a malicious user inject code into your site? Are you unnecessarily exposing information about your users or code structure? And you are working with your API developers to use secure authentication and authorization practices right?

Accessibility

This concern is broader than the way the word is often used in web dev circles. Is the full audience of this site or application able to use it with as few limitations as possible? Users with disabilities? Users with older but supported browsers? How about international users or those with slow connections? Does your site work on mobile? Offline? On Apple Watch? Do you support AMP? And do your pages appear nicely on social media? Do you even care about any of these audiences? Maybe you’re optimizing for people who aren’t your users?

Reliability

Is a site or application consistently available to users? Does it often fail, and if so how does it recover? If things are breaking, are the developers aware of it quickly? Do different users in different scenarios receive a consistent experience?

Performance

Is a site or application consistently quick to load and interact with? If there are animations or highly interactive elements, are they responsive and usable, or do they freeze/jank? Is the user disrupted in their tasks by strange loading behavior?

Capability

Is your site limited in fulfilling its purpose because of foundational decisions or tradeoffs with the other concerns? Have you made decisions that let you serve your audience in unique ways because of your tech stack or development processes? Do your technical decisions even make sense in the context of your site or application’s purpose?

Coming Soon

I’m hoping to look at each of these areas more in depth in the coming weeks, as well as exploring how we make decisions that impact multiple areas simultaneously. In the meantime, as you’re looking at decisions and tools that will impact the long term structure of your front end application, consider how they’ll impact each of these areas.

TL;DR

Architecture is about decisions that are important and hard to change, with a goal of improving the efficiency of a project
Front End architecture is often concerned with the areas of Ease of Change, Security, Accessibility, Reliability, Performance and Capability

Book Review: Radical Candor

Mon, 31 Dec 2018 20:00:00 -0400

Have you ever read a non-fiction book that you just couldn’t put down because you kept hitting paragraph after paragraph of useful, practical wisdom? That was me reading Radical Candor by Kim Scott this summer. It’s a ~200 page masterpiece on communication, leadership and being real at work. The ideas in Radical Candor have already been a huge help to me as I’ve transitioned into a manager role over the past few months, and I fully expect to refer back to them a lot in the years to come. It was the best thing I read in 2018, so I wanted to write a quick review to start the new year.

Radical Candor is split into 2 parts: part 1 is philosophical, and builds out a philosophy of working with and managing people that’s based on honest guidance and feedback backed by genuine care. The second half of the book is tactical and focused on techniques that managers can use to implement the philosophical ideas. Overall it is a book for managers, but is also relevant and useful for anyone who has a manager or who wants to work better with others in a professional setting.

The heart of the book is the 2x2 grid diagram that Scott sprinkles throughout the chapters, showing the 4 quadrants that can characterize an interaction, relationship or communication style. People who don’t care about those they work with and also don’t challenge them fall into manipulative insincerity and should be avoided at all costs. The Steve Jobsian category of Obnoxious Aggression is when somebody is willing to challenge directly but doesn’t care about the people they’re challengin. To Scott’s mind, Obnoxious Aggression is preferred over Ruinous Empathy; when people who hate the idea of hurting somebody so much that they fail to tell them important truths. But the target quadrant is Radical Candor, a state where you deal with people in a way that is backed by real humanity while still being willing to be exceptionally honest and challenging.

This core idea wasn’t super new to me, I had seen it formulated in Church contexts as Grace + Truth = Growth in the past. But Scott puts in the work to show how these ideas play out in a wide variety of professional situations and relationships, and her examples left me teeming with new ideas on how to interact better with my coworkers. The theme is good, but the implementation ideas and little nuggets of wisdom and experience sprinkled throughout are better. As a result, Radical Candor is one of the best books on relationship and communication I have ever read.

I would recommend it for anyone who

Manages people
Wants to manage people
Has a manager
Has to deal with people

Do yourself a favor and check it out.

Book Review: A Philosophy of Software Design

Sun, 30 Dec 2018 20:00:00 -0400

A lot of the software books out there these days are marching towards their expiration dates the day they’re written. When I see titles like “Angular 2 Development”, “Learning JavaScript with React”, “Writing Native Apps with Ionic 2 and Lodash 4” I know that the book will soon have out of date code examples and will not contain the latest ways a community is using the libraries they’re discussing. There is value in these types of books and they often represent a monumental amount of thought and time from their authors, but 5 years from now they will no longer be relevant. I love reading books that I know will last.

A Philosophy Of Software Design by John Ousterhout is a book that will last. It’s full of timeless software writing advice that isn’t tied to a particular language, library or point of time. Although not formally organized this way, it seems to be divided into ~4 parts: 2 chapters on the idea of complexity, a concept which Ousterhout uses as the unifying theme of the book, 8 chapters on what I’d describe as the structural design of software systems, 7 chapters on more “code level concerns” like naming conventions, comments and making code obvious, and then a final 2 chapters that feel a bit more like appendixes which discuss some common software trends and designing for performance.

The best part of the book is those first few chapters where Ousterhout eloquently describes the problems that complexity can cause in a software project and lays out the distinction between a tactical and strategic approach to coding. This is the heart of the titular philosophy of Software Design: the idea that software engineering is ultimately a fight against complexity. It’s a strong vision clearly articulated.

The “structural” sections of the book are all sound in an opinionated way. The book makes an argument for larger implementations hidden behind smaller abstractions, which Ousterhout calls “deep modules”, and much of the structural section is through that lens. This isn’t as open-minded as architecture advice I’ve seen elsewhere, but the benefit is that it’s prescriptiveness makes it much easier to use as a guide or code review tool than more situational or caveated advice. That practical aspect is clearly a goal of the book, and the approach therefore makes sense.

After chapter 11, the book takes a sudden detour from higher level concepts into much more concrete realms. There is an extensive treatment of commenting style, naming conventions and other ways to make code readable and obvious. There is a coherent through-line of building maintainable systems throughout, but the transition felt strange. The content is still good though.

The book ends on a chapter addressing modern software trends through the lens of the concepts considered in the book, which felt like a good place to wrap up, and then a chapter on designing for performance. Given the books focus on complexity and the size of the performance topic, this chapter is content to argue that simple designs are friendly to performance, without trying to unpack many performance related topics. It felt a bit tacked on to me, but it makes sense as a piece of objection handling for those who feel that they can’t simplify their architecture for performance reasons.

A Philosophy of Software Design is going to be my new go to recommendation for developers who are able to write code that runs but want to start thinking about how to write better code. It doesn’t have the depth of something like Clean Architecture, but it is extremely readable and approachable, and I’ve never seen an architecture-level book that was so focused on actionable advice. If you take the advice in the first chapter and consult the book during code reviews, every chapter in the book has something to consider when looking at a piece of non-trivial code. That makes it a perfect introduction to higher-level software design thinking. If you think that would be a fit for you, go ahead and check it out.

Want to work with me?

Fri, 02 Nov 2018 20:00:00 -0400

Hey all. The blog is still on pause, but I wanted to take a moment to share that I’m hiring for my team at Whatcounts.

This is a mid-level position focused on coming in and helping me build a new React-based UI for an older Java based SaaS application. You’ll get to work with modern React/JavaScript code and help shape the project¹. I’m looking for folks who are comfortable building out complex pages quickly and are knowledgeable about JavaScript, ideally you’re at least starting to get comfortable with these 10 concepts and you want to keep growing as a developer.

I’m hiring both in Atlanta GA and Durham NC², so if you’re located there and think you’d be a fit feel free to apply on Linkedin or reach out to me directly.

I’m actively considering pulling in some other cool tech like Typescript and Storybook, and the person who fills this role will have the opportunity to help make some of those decisions. ↩︎
No remote at this time, sorry folks, although there will be considerable flexibility to work from home a few times a week. ↩︎

Pressing Pause

Fri, 12 Oct 2018 20:00:00 -0400

Due to a plethora of life events happening this year, I’m going to be putting the blog on pause for a while. So no more updates through the end of the year. Will re-evaluate then and hopefully start things back up. I have plans to take the weekly links stuff I’ve been doing the past year and turn it into a real email newsletter with a better template, and some exciting ideas for a new blog series. But I lack the capacity to follow up on them for now. So Happy Holidays in advance everybody, I hope to be back early next year.

Weekly Links: September 30

Sat, 29 Sep 2018 20:00:00 -0400

Javascript & the Web

Principles of Component API Design | Chris Schmitz

Good advice for designing React components

Programming Tools

Sublime Merge

A new Git tool from the makers of Sublime Text. Looks lightning fast, good looking and usable. Exactly what you’d expect. I’m not a big UI guy for Git, but I’ll be playing with this, and it might be my new go-to recommendation.

Book Review

The Everything Store | Brad Stone

Still finishing this one up, but can already recommend it. A great look into the creation of one of the most important companies in the world. I was pretty interested to read about how intense Jeff Bezos is, and how hard he ran the early employees of Amazon. I hadn’t realized that he fell into the Steve Job’s school of CEO-hood.

Weekly Links: September 23

Sat, 22 Sep 2018 20:00:00 -0400

Javascript & the Web

The Rise of Immer | Shawn Wang

I’m a few weeks late on this one, but it is worth your time. Shawn breaks down a new trend in React state management: the increasing popularity of Immer as a base for improving state management APIs. I’m particularly interested in react-copy-write as a new simpler paradigm for React state management. Right now Redux has “won” that space, and MobX is the clear second option, but it still feels like there’s room for improvement and new options. Will be cool to see what people build on top of immer.

Leadership

How to fail as a new engineering manager | Brad Armstrong

As a new engineering manager, I found this list super helpful. I can already feel the temptation to fall into many of these traps, and it’s encouraging to hear about them from somebody who has gone ahead of me.

Book Review

Every Good Endeavor | Tim Keller

This is a “Christian book”, but I think it would be interesting to anyone who is interested in thinking about why we work, and how we find meaning in our work. If you are a Christian, there’s lots of insight here connecting the truths of our faith to the rhythms of our work lives. If you’re not, there is plenty to provoke thought, because the questions this book raises; why do we work? why is work often so unfulfilling? How do we find our purpose?; are universal questions regardless of whether you agree with Keller’s answers or foundational beliefs. Highly recommended.

Weekly Links: September 16

Sat, 15 Sep 2018 20:00:00 -0400

Javascript & the Web

The 12 Things You Need to Consider When Evaluating Any New JavaScript Library | Sacha Greif

This is a smart set of criteria to consider when looking at new tools. I’d add that it’s always worth comparing a library to a “do it without the library” approach, to make sure you understand the benefits that you’re getting for your specific project.

Leadership

Newsletter | Lara Hogan

I’ve already linked several of Lara Hogan’s blog posts in this space, and now she’s started an email newsletter. Worth checking out.

Book Review

Ready Player One | Ernest Cline

I’m about 7 years late on this (and even 6 months late for the movie hype), but Ready Player one was a great little sci-fi novel that uses 80’s pop culture as a background for a bunch of insightful commentary about our modern tech culture and where it may be heading. A lot of fun, and worth reading if you’ve missed it for this long like me.

Weekly Links: September 7

Thu, 06 Sep 2018 20:00:00 -0400

JavaScript And The Web

State of JS Survey | Raphaël Benitte, Sacha Greif & Michael Rambeau

This survey is always a great look into the JavaScript community, and it just opened for this year. If you’re reading this, please fill it out. I’ll look forward to seeing what they’ve learned in a few months.

Books For Programmers

It’s a bit of a slow week links wise, so this week I want to share some of my favorite “timeless” programming books. These are books that will still be relevant in 10 years.

The Pragmatic Programmer | Andy Hunt & David Thomas

My favorite book on general programming “tactics”. Lots of code level tips and tricks here, as well as some higher level architecture thinking. Some of the specific examples have aged out, but the general concepts discussed in this book are timeless.

Clean Architecture | Robert Martin

I reviewed this one earlier this year, and it’s a relatively new book, but I’m confident that it is built to last. This is a great book on programming “strategy”, the high level construction of maintainable programs.

How To Win Friends And Influence People | Dale Carnegie

Not a programming book, but for most programmers who have achieved a minimal level of technical proficiency, the best thing they can do to get better in their career is not more technical growth, but instead growth on their “soft” skills. Communicating complex topics, helping teams work well together, working through interpersonal issues, negotiation and persuasion are all huge skills for professional software developers.

There are lots of “business” books out there that are worthwhile, but this is the best book I’ve ever read on the simple yet endlessly deep topic of relating to our fellow human beings. It has been relevant and helpful for 82 years, and I wouldn’t expect that to change soon.

Weekly Links: September 1

Fri, 31 Aug 2018 20:00:00 -0400

JavaScript And The Web

Reverse Engineering React | Angular In Depth

A really cool look at how you can approach a new code base and start learning how it works quickly

Programming Tools

Bowler | Facebook Open Source

This looks like “JSCOdeshift for Python” to me: a way to quickly update codebases to use new patterns and practices. I love it. You can check out my experience using JavaScript Codemods here

bat

bat is a prettier cat: dump a file into the terminal, but use syntax highlighting and line numbers. Good stuff for anyone who lives on the command line.

Weekly Links: August 18th

Fri, 17 Aug 2018 20:00:00 -0400

JavaScript and the Web

Bits | Ethan Marcotte

I’ll pretty much always link performance posts that do a good job laying out the challenges in a way that respects both the technical and business POV. Ethan advocates here for a more structural view to performance problems on the web and wonders how we can find more performant ways to solve businesses revenue problems.

React Community Resources | Egghead

Egghead has accumulated a pretty amazing set of beginner React resources that are free for non-members. If you’re trying to get into React, this is a pretty good place to start.

Books

Radical Candor | Kim Scott

I may write this up as a full review later, but Radical Candor was the best book I’ve read this year. It’s a book for managers, but it really was just one of the best books I’ve ever read on communication and relationships. If you want to work better with others, Radical Candor is worth buying.

Weekly Links: August 5th

Sat, 04 Aug 2018 20:00:00 -0400

JavaScript and the Web

The Cost Of JavaScript in 2018 | Addy Osmani

This was a great breakdown of how exactly large JavaScript bundles cause sites to be slow and how that impacts business needs. These type of advocacy pieces have a tendency to clump all businesses together a bit too much¹, but Addy is pretty reasonable about that and it’s important for every front end dev to understand these tradeoffs and problems, even if they don’t need to make them top priority for every project.

Programming Tools

On Switching Code Editors | Chris Coyier

I’m in the middle of possibly switching to VSCode from Atom, which I switched to from Vim, which I switched to from Sublime Text, which originally introduced me to a world beyond overweight IDEs like Eclipse. So I can relate to this article, and it’s a great framework for thinking through switches. I will say, I don’t necessarily think I’ve made the best choices here. I’m not sure that since I’ve learned Vim keybindings, any of my switches have actually increased my productivity. There’s a cost to learning new tools, even when they have shiny new stuff. grumbles a bit and then goes back to chasing the shiny.

People Are Going Crazy Over Framer X | Aaron Adler

My twitter feed was blowing up with people talking about Framer X this week, a tool for designers to build interactive designs/prototypes based on React. This post is a little old but was the best summary of the concept I could find. Framer X is still in a private beta, so it will be interesting to see what happens when it goes public.

Even in 2018, not everyone is serving 3rd world businesses or gets a large chunk of their traffic from mobile. And performance always needs to be traded off against other business goals. ↩︎

Feedback Loops

Wed, 01 Aug 2018 20:00:00 -0400

How do we get better faster? It’s amazing to consider the amount of money, thought and effort that has gone into solving that one simple problem in software development over the past 2 decades. Here’s a small sample of buzzwords, trends and methodology that are concerned with reducing the time between making a change to software and seeing the results:

Agile - Agile is all about delivering working software sooner, so that you produce value faster and learn lessons quickly.

Test Driven Development (TDD) - Test driven development is a process designed to let you know as soon as possible when a change has had the desired effect during development, and also sets up the conditions under which you know as soon as possible when a change has had an undesirable side effect.

Devops - Devops has been a push over the past decade to automate and control all parts of the software creation, deployment and maintenance life cycle. Automation like continuous integration servers that run your test after every commit, chat bots that alert you based on site performance metrics, and servers that automatically build a copy of every branch for testing in a virtual machine are all examples of Devops being used to speed up the point when you know that a change worked or didn’t work.

Developer Experience - This is a more recent buzzword that I’ve seen thrown around a lot in the JavaScript community, talking about improving developer tools. Developer tool features like hot reloading, nicer error messages, specialized debugging and profiling tools, and fast build times all fight this same problem of getting a result as fast as possible.

Feedback Loops

All of these tools are concerned with feedback loops, the process by which developers, designers and product managers make changes, learn about the effect of those changes, and then iterate. Teams with slow feedback loops spend a lot of time planning and thinking about every change, making sure to get everything right. They feel the cost distinctly each time they make a mistake, and it generally leads to a more cautious mindset. Teams with fast feedback loops can plan too. But they also have the option of taking a more experimental attitude and learning from their attempts. This can happen at multiple scales:

A developer can try out 5 different CSS rules in the UI in a few seconds if his UI is hot reloading and he doesn’t have to wait for a whole SCSS bundle to recompile
A QA team can immediately know if somebody pushes a change that would breaks tests if they have a CI server set up to run tests against every new commit to each branch.
A product team can see quickly that a new feature is problematic if he’s able to see detailed analytics on usage, instead of waiting to do enough user interviews to get a full picture
A company can change direction if they ship an MVP early instead of attempting to wait and release the perfect product

Closing Your Loops

As a developer who wants to make your product better, one of your best power moves is finding ways to speed up feedback loops. Here are 4 suggestions for how to do that:

Ask Questions Early: One of the most important feedback loops for a developer is between yourself and the individual in charge of defining what you’re building. If you simply take requirements from your PM/Manager/Lead/Designer/CEO/Client and go off and build your giant amazing piece of beautiful software, without any clarification, there’s a pretty good chance you’re going to get to build it again, this time twice as fast without the budget. Asking clarifying questions early prevents you wasting your time and exposes understanding gaps and technical barriers while there’s still a chance to deal with them.
Get Feedback as you go: Similarly, anytime you have a chance to get feedback on work while it is still in process from somebody who might object in the end, do it. This means instead of building everything at once, work on smaller chunks. This often takes time and thought, but the earlier you create complete chunks of code that can be evaluated and ideally shipped on their own, the more likely you are to ship something great in the end.
Optimize your environment: Have you spent time figuring out how to work as efficiently as possible? Have you done it recently? If not it might be worth re-evaluating your setup on a quarterly basis or so, figuring out what tools you can use to get quicker feedback, and also learning the shortcuts, tricks and new features for the tools that you do use.
Ship Code: Ultimately the best feedback comes when people are using what you’ve written. Whether that’s compliments, bug reports, feature requests, anger or viral adoption, shipping will tell you things that you would never have been certain of while iterating on your own. If you’re struggling to get code out, always fix that first.

Weekly Links: July 28th

Fri, 27 Jul 2018 20:00:00 -0400

JavaScript and the Web

Getting to Know a Legacy Codebase Harry Roberts

I like this advice on approaching a new codebase. Some of it is CSS specific but the general ideas of tackling the important bits, experimenting with changes, and using tools as much as possible to catch regressions apply across the board.

A one year PWA retrospective | Zach Argyle

TL;DR: Pinterest saw great results from supplementing their mobile apps with a PWA web app.

Mission: Improve the Web Ecosystem for Developers | Dion Almaer

I don’t know Dion, but this was a wonderful list of challenges that people looking to improve the web platform might want to tackle. I’m excited to see people tackling this stuff. I’m especially interested in what better tools for creating performant sites would look like.

Technical Deep Dives

These are long technical articles, and more back end focused then I usually write about on this blog but both touch on things I’m interested in and I found them fascinating.

Centrifuge: a reliable system for delivering billions of events per day || Calvin French-Owen

I had to build an event handling system like this a few years ago, and it’s fun to see how the optimal solution changed for Segment as they scaled. On the backend designs that make no sense for small projects start becoming very real options as a project scales. That’s why it’s always worth having some skepticism over the idea that because AppAmaGooFaceSoft are using a technology it must be “best in class” and we should use it in our small 20 person development team that is facing problems of a completely different scale.

ripgrep is faster than {grep, ag, git grep, ucg, pt, sift} | Andrew Gallant

This piece is structured as a comparison of performance between cli file search tools, but the cool part is the deep insight into text search algorithms and performance tricks littered throughout the benchmark discussions. It’s fascinating to see how heavily optimized for speed these tools are. Also, I’ve been using Ack for 5 years even though I know it’s not the fastest tool for searching, because for me it was always “fast enough” to avoid searching. But ripgrep appears to be dramatically enough faster in the relatively rare circumstances where I’m searching truly large directories of files, that I may finally be up for converting to a new tool.

Book Review

Front-End Reactive Architectures | Luca Mezzalira

I’ve been very excited to check out this book since I first saw the title pre-release. I finally had a chance to purchase and read it last month, and I’m sad to report that I had fallen prey to the classic mistake of judging a book by it’s cover. I had hoped that this would be a deep dive into architecture and how we structure front end applications, and how reactive programming concepts can impact that. There’s a real dearth of quality high level UI architecture resources out there, so I was genuinely excited about this.

Unfortunately the book is structured more like a series of loosely related blog posts, first with an overview of older architectural models like MVC, MVP and MVVM, followed by sections looking at several different libraries that use reactive principles: RxJS, Cycle, and MobX. The latter sections focused more on the nuts and bolts of using the libraries to create simple example apps and less on the impact that using them would have on a large application. So this is a good book if you want to see some simple code examples of how these libraries work in practice. But I’m still waiting for a great high level front-end architecture book.

Weekly Links: July 20th

Thu, 19 Jul 2018 20:00:00 -0400

JavaScript and the Web

Building the Google Photos Web UI | Antin Harasymiv

An amazing detail-laden look at the design and engineering decisions that go into a top of the line web photo experience. One of the best front end reads I’ve seen this year.

What’s Happening With the Pipeline (|>) Proposal? | James DiGioia

An update on the proposal to add a pipeline operator to JavaScript. I am not the target audience of this proposal, as I don’t regularly write in the nested miniature function style that is aided by this proposal, but I would really love to see some real world code (not fabricated examples) that would have their readability improved by this proposal. The smart pipelines proposal in particular seems like it’s adding a bunch of new syntax for developers to learn and tooling to support. Since this is a “sugar” feature and doesn’t add new capabilities, I really wonder whether it’s worth it.

Leadership and Teams

The P Rules | Andrew Bosworth

Design Reviews are one of my favorite parts of my job, and this is a great set of rules for having good ones.

Feedback Equation | Lara Hogan

So this is back to back weeks with links from Lara, and there’s a good reason for it. This is a great framework for speaking the truth to somebody with humility.

Startups

Why I love B2B over B2C | Justin Kan

I’ve worked at B2B companies my whole career, so I’m biased I guess, but unless you just like playing the lottery, its tough to see why you’d prefer B2C from a pure financial perspective.

Random

Grace Hopper on Letterman in 1986 | YouTube

This clip is just wonderful. A little slice of history, touching the computer industry, the greatest generation, and the roots of our current late night comedy scene. “There was a time when everyone in this country did one thing together” is an amazing statement in our current political environment.

Quick Tip: Logging In Breakpoints

Mon, 16 Jul 2018 20:00:00 -0400

Stop me if this sounds familiar: you see a bug on your site. You want to know what’s going on. You fire up the developer tools, put a breakpoint in the area you think is a problem, and… the error doesn’t occur while it’s being debugged. Probably it’s a timing issue, but I’ve also seen situations where pausing on certain breakpoints will actually cause a browser tab to freeze up. Whatever weirdness you’re dealing with, you can’t get a breakpoint running in the right spot to see the error. These are known colloquially as Heisenbugs, bugs that disappear when measured. Ideally at this point you would add some logging, but if the issue is on production, and you don’t know exactly what needs to be logged, that could delay a fix by an unacceptable amount.

I actually ran into this last night, and decided to test out a tactic I’d heard about in a recent JS Party episode. Did you know that you can use conditional break points to add temporary logging to a live application?

Conditional breakpoints are a browser debugger feature where you set a breakpoint, and specify a condition. The breakpoint only stops the program execution when that condition is truthy. Here’s a video describing the feature in Firefox:

Conditional breakpoints are useful when you’re running through a code path many times during program execution, but you want to stop during a specific run. But they can also be used to solve our heisenbug problem. Because the conditional breakpoint executes the code inside of it in the context of the line we’re trying to break on, we can actually put logging inside the breakpoint.

An example of a console.log inside a breakpoint

That console.log will execute, and because console.log returns false, the breakpoint will never actually stop. It’s a heisenbug-safe version of watching variables when you’re at a breakpoint, and there’s no redeploys required.

Update January 2019: Chrome will soon be building this hack right into the devtools with a new “logpoint” option for breakpoints that formalizes this method. Very cool!

New feature in Chrome 73: Logpoints. Log messages to the Console without cluttering up your code with console.log() calls. https://t.co/aFl2e6GmnF https://t.co/fJ6CUNN0Lg
— Chrome DevTools (@ChromeDevTools) January 24, 2019

Prime Generation Revisited

Sat, 14 Jul 2018 20:00:00 -0400

I’ve been wanting to try learning a new programming language again for a while, and this weekend decided to play around with Go for the first time in a while. Go is a statically typed, compiled language useful for systems programming, which makes it a nice complement to my 2 primary languages, JavaScript and Python. To help myself learn, I decided to revisit the Sieve of Eratosthenes problem that I wrote about last year. I already had implementations in Python and JavaScript, and thought it would be interesting to see how the code and performance compared.

As a refresher, the Sieve of Eratosthenes is a method for generating prime numbers by iterating through numbers and marking all multiples of primes we encounter as not prime. The original algorithm is useful for finding small primes up to a limit, but will eventually consume infinite memory for computers or infinite time for humans. I used a modified algorithm in my JavaScript implementation that only marks numbers as they become relevant, you can read more details about the problem in the original post but using the JavaScript implementation to generate the 1 millionth prime looks like this:

function* generatePrimes() {
    const markedNotPrimeMap = new Map();
    let valueToCheck = 2;
    while(true) {
        if (!(markedNotPrimeMap.has(valueToCheck))) {
            yield valueToCheck
            markedNotPrimeMap.set(valueToCheck**2, [valueToCheck])
        } else {
            let primes =markedNotPrimeMap.get(valueToCheck)
            primes.forEach(prime=> {
                let nextMultipleOfPrime = prime + valueToCheck;
                if (markedNotPrimeMap.has(nextMultipleOfPrime)) {
                    markedNotPrimeMap.get(nextMultipleOfPrime).push(prime);
                } else {
                    markedNotPrimeMap.set(nextMultipleOfPrime, [prime]);
                }
            })
            markedNotPrimeMap.delete(valueToCheck);
        }
        valueToCheck += 1
    }
}
let counter = 0;
for (let p of generatePrimes()) {
    counter += 1;
    if (counter >= 1000000) {
        console.log(p);
        break;
    }
}

That executes in ~10.5 seconds on my Macbook Pro, and the similar Python implementation averages about 23 seconds. I was curious how fast a Go solution would execute, since it is known for being fast.

I put a Go version together like this:

package main

import "fmt"
import "strconv"

func generate_primes() chan int {
    c := make(chan int)
    var marked_not_prime_map = map[int][]int{}
    go func() {
        for value_to_check := 2; ; value_to_check++ {
            var primes, in_map = marked_not_prime_map[value_to_check]
            if !in_map {
                c <- value_to_check
                var values []int
                marked_not_prime_map[value_to_check*value_to_check] = append(values, value_to_check)
            } else {
                for _, prime := range primes {
                    var next_multiple_of_prime int = prime + value_to_check
                    var next_multiples, next_in_map = marked_not_prime_map[next_multiple_of_prime]
                    if next_in_map {
                        marked_not_prime_map[next_multiple_of_prime] = append(next_multiples, prime)
                    } else {
                        var values []int
                        marked_not_prime_map[next_multiple_of_prime] = append(values, prime)
                    }
                }
                delete(marked_not_prime_map, value_to_check)
            }
        }
    }()

    return c
}

func main() {
    c := generate_primes()
    var counter int = 1
    for true {
        var p int = <-c
        if counter >= 1000000 {
            fmt.Println(strconv.Itoa(p))
            break
        }
        counter++
    }
}

The Go version is about 50% longer than the JavaScript version due to boilerplate but is more or less a direct 1 to 1 translation of the JavaScript code. In Go, generators are created using channels and goroutines. Goroutines are special functions that can run concurrently to the main thread in Go, and channels are a mechanism for passing data from them, that block the goroutine until the message is received. So a goroutine that will pass multiple messages through to a channel is essentially equivalent to a Python or JavaScript generator using yield. Beyond that, most of the syntax difference is around the ceremony required for using slices in Go , and checking whether an item exists in a map. Slices are Go’s version of a dynamic length array/list, they require an append function which returns a new slice to add a value. Go doesn’t have a has method or equivalent on it’s map types. Instead when you attempt to access a value from a map, the map returns 2 values: the received value or the “zero value” of the map’s type and a boolean saying whether the access was successful. So you have code like this:

commits := map[string]int{
    "example": 100,
}
var val, has_val = commits["example"]  // val == 100, has_val == true
var val2, has_val2 = commits["example2"]  // val == 0, has_val == false

So does all of this ceremony buy us any performance? Only a minimal amount in this case. After timing each script several time the Python code averaged out to 23 seconds, the JavaScript averaged 10.5 seconds, and the Go version averaged 9.5 seconds. It’s the fastest, but only by minimal amounts over the JavaScript. A good reminder that the algorithm will generally matter more than the programming language for many things, and also that large companies have optimized the heck out of JavaScript in recent years to the point where it’s pretty competitive on single threaded performance for many tasks.

Weekly Links: July 13th

Thu, 12 Jul 2018 20:00:00 -0400

JavaScript and the Web

What is JSX? | Kent C Dodds

If you were introduced to React “all at once” and didn’t learn it from the basics up, JSX may feel a little bit like magic syntax. This is a solid explanation of what JSX syntax actually compiles to under the covers. It demystifies the process and shows that it is all “just JavaScript” in the end.

Defining Component APIs in React | Brent Jackson

Setting up good interface boundaries between your React components is one of the most important steps you can take to make a React application more maintainable. This is good solid advice, that I wish I’d had when I started building React apps.

Business and Leadership

Leadership Style Colors | Lara Hogan

Lara has put together a thoughtful article about the different ways that leaders need to be able to relate to others.

The open-plan office is a terrible, horrible, no good, very bad idea | DHH

I appear to be in a weird middle ground where I neither love nor hate open offices. I don’t think they’re good for my productivity, but I enjoy the people I work with and build more connections with coworkers in an open office than I did when I worked in cubicles. I also find that articles like this tend not to fully grapple with the alternatives. Most companies aren’t mostly remote like Basecamp, and budgeting private offices for all developers is not realistic (especially for the startups called out in this piece due to unknown scaling needs). So it’s remote, open office or cubicle farms, and all three have their detractors.

Design

Designing for accessibility is not that hard | Pablo Stanley

I’m embarrassingly ignorant about accessibility concerns, so this was a helpful article for me. I’m going to try and walk through some of these steps on this site to help improve my knowledge here.

10 years of the App Store: The design evolution of the earliest apps | 9to5Mac

This was a really fascinating look at how iPhone app designs have shifted over the years. 9to5Mac took 10 popular apps that were there when the App Store first launched 10 years ago (I’ve used 8/10 of these at some point or another) and showed how their UIs and logos had evolved over time. Interestingly it looks like early 2013, right before the new iOS7 look and feel launched, was the peak point for interface diversity and distinctiveness, as each of these apps had evolved along their own paths. iOS7 imposed a level of standardization and sameness, and I think apps are just now re-evolving to the point of having truly distinct brand identities on top of that new base.

Books

I’ve been on vacation this week, so here are 2 quicky non-tech book reviews:

Golden Days: West’s Lakers, Steph’s Warriors, and the California Dreamers Who Reinvented Basketball | Jack McCallum

This was a fun book for basketball lovers. The sections on Jerry West and the Lakers of the 60s and 70s are more detailed and colorful than the alternating chapters on the modern Warriors, but both are fun, and those interested in how modern Silicon Valley has interacted with the rise of the most popular basketball team in the world will find some interesting tidbits in the modern chapters as well.

American Gods | Neil Gaiman

A fantasy novel set in modern America. This one was too crude for me to recommend unreservedly, but it contains a huge number of cool images and ideas, and I don’t regret reading it.

Book Review: The Five Dysfunctions of a Team

Sun, 08 Jul 2018 20:00:00 -0400

The nice thing about team books is that they’re talking about a subject that almost everybody can relate to. The Five Dysfunctions of a Team by Patrick Lencioni is a book for leaders, but it’s also a book for anyone who wants to better understand why they problems they see in their teams occur, and what needs to change to make them better.

The Five Dysfunctions is a leadership fable, a story told to illustrate the principals of a business model. I’ve found that I enjoy this teaching style, I also loved Death By Meeting and The Goal, which employee the same style. The beauty of fables is the way they allow you to soak into a scenario, and connect it to your own experiences and relationships. Then as the principles the author wants to teach unfold, they take advantage of those connections to spark ideas for application. At least that’s the theory.

The Five Dysfunctions is of course, about how teams go wrong. I won’t spoil the contents too much, but ultimately, team failures aren’t about technique or distinct decisions, but human beings and how we relate to each other. This book both clearly defines a set of those problems, and then gives concrete suggestions for improving a team when they’re present. It’s a classic, and I found it as thought provoking as I expected. I definitely recommend it for anyone who wants to better understand how teams work, and how they fail.

Weekly Links: July 8th

Sat, 07 Jul 2018 20:00:00 -0400

A small list this week as it’s a holiday in the states. So fewer articles out there, and I’m sure I missed some of them.

Programmer Life

My Lessons from Interviewing 400+ Engineers | First Round Review

I’m nowhere near Marco’s experience level interviewing, but his advice in this “interview on interviewing” resonates with my experience. Things that stood out to me:

Don’t overoptimize on a specific skillset as a startup, since you don’t know what skills you need yet.
It’s incredibly important (but also quite difficult) to be consistent in your standards across all interviewees
Team interviewing works better than 1 on 1 interviews for evaluating technical skills and team fit.

Digg’s v4 launch: an optimism born of necessity | Will Larson

An engineering horror story.

Anti-Flow | Rands in Repose

Rands remains my favorite software blog because of his ability to capture truths about the engineer condition with beautiful words.

Evil JavaScript

Tue, 03 Jul 2018 20:00:00 -0400

JavaScript developers sometimes complain that their language is unjustly maligned for having too many confusing features. Plenty of other people are out there explaining why this is misguided, but for this post I want to lean in. Rather than falling into the traps that JavaScript exposes you to, what if we embrace our language’s reputation instead, and see what code we can write if we don’t care about the feelings of others?

The “Evil JavaScript” you’ll see today takes advantage of an array of language features. Many of these techniques still work in other languages, so please don’t worry that I’m limiting your pursuit of the dark side. But JavaScript certainly has a flair for evil that is hard to match. If you write code that other people have to work with, the opportunities to annoy, confuse, aggravate and bamboozle are limitless. The following is a small selection of choice techniques.

Modify objects in your getters

JavaScript objects can define getters, functions that let you access the result of a function as a property. Used normally, that looks like this:

let greeter = {
  name: 'Bob',
  get hello() { return `Hello ${this.name}`}
}
console.log(greeter.hello) // Hello Bob
greeter.name = 'World';
console.log(greeter.hello) // Hello World

Used evilly though, we can come up with fun self-destructing objects!

let obj = {
   foo: 1,
   bar: 2,
   baz: 3,
   get evil() {
      let keys = Object.keys(this);
      if(keys) {
         delete this[keys[0]]
      }
      return 'Nothing to see here';
   }
}

Here, every time we read obj.evil, we delete one of the other properties on the object, with no indication to the code accessing obj.evil that anything extraordinary is happening. But this is just the start of how we can introduce unexpected side effects.

Nobody expects a Proxy!

Getters are cool and all, but they’ve been around for years, and many developers know about them. We now have an even more powerful variation on the same theme with Proxies. Proxies are an ES6 feature that let you put a wrapper class around an object that lets you control what happens when a user accesses or updates any property. So we can, for instance define an object that will, one third of the time return a value from a random key whenever a user tries to access a key.

let obj = {a: 1, b: 2, c: 3};

let handler = {
    get: function(obj, prop) {
      if (Math.random() > 0.33) {
        return obj[prop];
      } else {
        let keys = Object.keys(obj);
        let key = keys[Math.floor(Math.random()*keys.length)]
        return obj[key];
      }
    }
};

let evilObj = new Proxy(obj, handler);

// this is example output that I received running the code
console.log(evilObj.a); // 1
console.log(evilObj.b); // 1
console.log(evilObj.c); // 3
console.log(evilObj.a); // 2
console.log(evilObj.b); // 2
console.log(evilObj.c); // 3

Our sneakiness is undermined a bit by devtools, where evilObj will be identified as a Proxy, but we can still lead other developers on a merry chase before we’re caught.

Contagious functions

So far we’ve talked about how objects can modify themselves, but we can also create innocuous looking functions that infect the objects that are passed to them with strange behavior. For instance lets say we had a simple get function to do safe property lookups on an object that may not exist:

let get = (obj, property, default) => {
   if(!obj) {
      return default;
   }
   return obj[property];
}

It would be easy to make that function contagious by subtly changing the object it was passed. For instance we could make it so the retrieved property no longer showed up when the user enumerates the keys.

let get = (obj, property, defaultValue) => {
   if(!obj || !property in obj) {
      return defaultValue;
   }
   let value = obj[property];
   delete obj[property];
   Object.defineProperty(obj, property, {
      value,
      enumerable: false
   })
   return obj[property];
}

let x = {a: 1, b:2 };

console.log(Object.keys(x)); // ['a', 'b']
console.log(get(x, 'a'));
console.log(Object.keys(x)); // ['b']

This is super subtle, since enumeration is neither rare nor particularly common. Since the actual bug wouldn’t be tied to the infected object at all, it could persist in a code base for a long time.

Messing with Prototypes

Sometimes the old ways are best. One of the most criticized features of the JavaScript language is the ability to modify built-in prototypes. This was used in the early days of JavaScript to enhance built in objects like arrays. For instance we can add a contains function for arrays like this:

Array.prototype.contains = function(item) {
  return this.indexOf(item) !== -1;
}

It turns out that doing this in real libraries can ruin the language for everyone else. So providing additional useful prototype methods is a good long con for malicious developers with patience. But for the impatient sociopaths reading along, we also have some short term fun we can unleash. The fun thing about prototype manipulation is that it affects all code running in an environment, even if that code is running in modules or closures. So if we introduce the following code as a 3rd party script (maybe an ad or analytics script?), we can introduce subtle bugs throughout a site.

Array.prototype.map = function(fn) {
   let arr = this;
   let arr2 = arr.reduce((acc, val, idx) => {
      if (Math.random() > 0.95) {
         idx = idx + 1
      }
      let index = acc.length - 1 === idx ? (idx - 1 ) : idx
      acc[index] = fn(val, index, arr);
      return acc;
   },[]);
   return arr2;
}

We’ve rewritten Array.prototype.map to work just like normal map with the exception that we will randomly switch the order of 2 values 1/20th of the time. An example result:

let arr = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15];
let square = x => x * x;
console.log(arr.map(square));
// [1,4,9,16,25,36,49,64,100,81,121,144,169,196,225
console.log(arr.map(square));
// [1,4,9,16,25,36,49,64,81,100,121,144,169,196,225]
console.log(arr.map(square));
// [1,4,9,16,25,36,49,64,81,100,121,144,169,196,225]

We ran map 3 times, and the first was subtly different than the next 2 runs. It’s subtle enough that it won’t always cause anything to happen, and the crazy thing about this hack is that there’s no real way to know it’s happening without actually reading the source code that caused it. Our function doesn’t show up weirdly in devtools, and it doesn’t error under strict mode or anything else. This is how developers go mad.

Naming things is hard

As one of the 2 hard problems in Computer Science, you don’t have to be evil to come up with bad names¹. But it certainly doesn’t hurt. We’ll pass over the chump change of misleading names and outdated comments.

// initialize a date
let arrayOfNumbers = { userid: 1, name: 'Darth Vader'};

No let’s get to the good stuff. Did you know that much of unicode can be used to name variables in JavaScript? If you’re into fun and games of course, this means that emoji are in play:

let 💩 = { postid: 123, postName: 'Evil JavaScript'}

But we want evil JavaScript, so let’s go straight to lookalike characters.

let oｂj = {};
console.log(obj); // Error!

The ｂ in oｂj may look normal, but it is actually a fullwidth latin small letter b instead of a latin small letter b, and that distinction means that anyone typing in the variable directly is probably going to be very confused.

Don’t be evil

Despite appearances, it is the official position of this blog that evil JavaScript should not be encouraged and may in fact be harmful. That said, it’s always good to know the ways things can go wrong. Each of these patterns exist out in the wild somewhere. Hopefully in less malicious forms than I’ve shown here. But lack of intent doesn’t make them easier to debug, and knowing just how bad things can get is a good start to seeing these bugs coming. You never know what bug is sitting out there waiting to strike. Just because you’re paranoid doesn’t mean that they’re not out to get you.

This may be hard to believe for old time Linux folks of course. They’ve had years to associate the worst naming offender in the industry (Microsoft) with the deepest forms of evil. ↩︎

Book Review: Atomic Design

Sun, 01 Jul 2018 20:00:00 -0400

The Unix philosophy for designing software, “Make each program do one thing well”, is a piece of wisdom that goes beyond just programming. For instance, it also applies to non-fiction writing. When learning, I find that I gravitate towards small focused resources, rather than monolithic books or websites that aim to teach everything about a specific topic.

Atomic Design by Brad Frost is a book that does one thing really well: it explains what a design system is, and how you can implement one in your organization. It doesn’t try to be a book about CSS, design styles or any of the other myriad topics that touch against design systems. That focus serves it well, and I found it to be well worth my time.

Design Systems are an organized toolkit for creating digital products. They’re a set of elements, widgets, patterns, and layouts (atoms, molecules, organisms and templates in the language of the book) that can be combined to build complex sites and applications. The idea is pretty simple. But there are some real challenges in implementing one.

What content belongs in a design system? Where do we draw the boundary as to what belongs and what doesn’t?
How do you build a tool for your design system so that patterns are easily findable, usable, and understandable?
How do you keep a design system relevant over time as the product(s) change?
How do you convince budget-conscious organizations that maintaining a design system is a good use of time and money?

After a brief introduction to the ideas behind design systems and pattern libraries, these are the challenges that Atomic design tackles. It discusses a hierarchy for patterns, and methods for pulling them out of existing applications, then moves on to talking about tools for implementing a pattern library, and techniques for navigating organizational politics when starting and maintaining a design system.

So this book will be helpful for you if you are

Struggling to deal with inconsistencies in your interfaces, or constantly doing custom UI work for each new feature
Considering implementing a design system/pattern library for your organization
Hoping to learn more about design systems because somebody else in your organization is pushing for one
Having trouble maintaining an existing pattern library

It won’t be helpful for you if you are

Looking for tips on how to make your interface beautiful
Seeking hands on information about specific design, CSS or JavaScript techniques

I enjoyed Atomic Design a lot, and hope that I’ll get to use what I’ve learned in the near future. As far as I can tell, Atomic design is only available on Brad Frost’s site. You can read the book online there, or buy the paperback version. If you fall into one of the 4 categories I listed above, I’d highly recommend checking it out.

Weekly Links: June 30th

Fri, 29 Jun 2018 20:00:00 -0400

JavaScript & The Web

Headless User Interface | Merrick Christensen

UI mixins are one of the hardest problems to solve. Sharing “behavior” across multiple UI components is difficult to do right. Merrick has documented an emerging pattern for doing this with React.

React Native at Airbnb | Airbnb Engineering

This has gotten plenty of attention around the internet, but Airbnb is moving away from React Native after years of heavy use and contributions to the community. Worth reading as an explanation of the pluses and minuses of React Native, probably not worth reading too much into the move beyond that.

An Adventure in Sparse Arrays | Remy Sharp

For anyone who read my “sparse array code golf” post the other day, this is a better holistic look at sparse arrays in JavaScript. I learned several new things here.

Programming Practices

Understanding Performance Regions | Rico Mariani

This is a nice high level post about a different way to think about performance monitoring and improvement.

Fun

Paul McCartney Carpool Karaoke

An incredibly fun video. Paul McCartney showing James Corden around Liverpool.

The Legend of Nintendo | Bloomberg

A great writeup on Nintendo’s last few years and where they stand now.

JavaScript “Stale Practices”

Mon, 25 Jun 2018 20:00:00 -0400

Best practices don’t last forever. This is especially true when a field is changing fast, and JavaScript development has changed a lot over the past 10 years. The old best practices go stale, and new ones take their place. Here are 5 JavaScript best practices that have gone stale recently.

Using the strict mode pragma to opt in to a sane JavaScript subset

When ECMAScript 5 was released, it introduced a special strict mode that tweaks the behavior of some obscure features of the language. Now, when a function or script file starts with the string 'use strict', code in that scope works differently than the same code without the 'use strict' string. Expressions that look like a mistake are flagged as an error instead of being ignored, code is restricted in ways that make it easier to optimize performance, and terms that may be used in future JavaScript versions are reserved as keywords and you can’t use them as names for variables.

'use strict';
var objWithDuplicates = {x: 1, x: 2}; // Fine normally, an error in strict mode
with(objWithDuplicates) { // with expressions are syntax errors in strict mode
    console.log(x);
}
var private = 'top secret'; // this is a ReferenceError in strict mode

Strict mode was a nice small improvement in JavaScript, and a good way to move past some of the mistakes of the past like with expressions. But even though it was a good idea to use, and I was adding the silly 'use strict' string to the start of all my functions for a few years, this is no longer necessary for many people.

The use strict string was necessary because the language designers wanted to maintain backwards compatibility, and they needed a way for developers to signal that they were writing new code that they were comfortable holding to a higher standard. ECMAScript 2015 (ES6) provided a better way of doing that. JavaScript Modules are a new type of script, and browsers can be confident that scripts loaded that way have been edited in a ES2015+ world. Because of that confidence, JavaScript modules are strict mode by default. So if you are using import and export in your code, you can stop adding 'use strict' everywhere and just rely on the browser to do the right thing. If you’re compiling through Babel or other tools you’ll still be ok, as they add the strict mode pragma for you when they compile your classes down to ES5 code.

Using anonymous functions when dealing with async code in loops

For many years, JavaScript’s many async use cases combined with the way var works to create a classic programmer trap. Here’s some jQuery code to put a listener on a series of buttons and alert a different number for each one of them.

for (var i = 0; i < buttons.length; i ++ ) {
    $(buttons[i]).click(function() {
        alert(i + 1);
    });
}

The idea is that if we have a list of 10 buttons, after we run this code clicking the first one will alert 1, the next one 2, and so on. The problem is… it doesn’t work. If there is a list of 10 buttons, all of them will alert 10 when clicked. But why? The problem is that var doesn’t create a new variable for each run of the loop. The technical term here is that var isn’t block-scoped. It isn’t limited by a set of curly braces, when var is called, it creates a variable that is accessible anywhere inside the function that contains it, or it creates a property on the global object if there is no containing function. So when we run our loop, we aren’t creating 10 separate i values, we’re creating 1 i value and updating it 9 times. Since the click handlers don’t run until the user has clicked the button, i has been updated all the way to 9, and every button references it and alerts 10.

For years, the recommended way of working around issues like this was to create an inner function. Because JavaScript variables are function scoped, we can create a new function, pass it the current i value, and execute the function immediately. When the click handler executes, it will have a reference to the arguments from the inner function, and will receive the correct variable value.

for (var i = 0; i < buttons.length; i ++ ) {
    (function(innerI) {
        $(buttons[innerI]).click(function() {
            alert(innerI + 1);
        });
    })(i);
}

This example works, but it’s obviously a bit convoluted, and would not look straightforward to a new JavaScript programmer who came across it. It gives off some unmistakable “magic function” vibe. Fortunately we can do better nowadays.

ECMAScript 2015 introduced the block scoped variable types let and const. Now we can create loop iterators that stay scoped to the loop portion that they were assigned in. This allows us to re-simplify our code, but have it actually work.

for (let i = 0; i < buttons.length; i ++ ) {
    $(buttons[i]).click(() => alert(i + 1));
}

Using closures for all of our private variables

JavaScript has never had a first class method for creating objects with private state. Any normal object you create will have all its properties available for reading and editing by any code that has access to it. But there has always been a workaround. We can once again take advantage of JavaScript’s function scope. This time we can write a function that creates an object. The methods of that object will have access to any other variables inside that function. This is called a closure. A closure is a function that returns another function that maintains access to the outer function state. In this case the returned function will be a method on an object. Here’s an example with a user, where the id is only readable and the name can only be updated with explicit get and set commands, which won’t accept a falsy value:

function createUser(name) {
    let id = uuid();
    return {
        getID() { return id },
        getName() { return name },
        setName(_name) { name = _name || 'Unknown'},
    }
}

This object factory pattern is still useful today, but sometimes we want something a bit more object oriented. Fortunately there are now more ways of doing these things. ES2015 introduced the Symbol keyword. Symbols are unique values that can only be accessed by code that has a reference to the symbol. So we can use a symbol as a more advanced version of our factory code above, creating a class that accesses it’s private state via a symbol.

let ID = Symbol();
let NAME = Symbol();

export class User {
    constructor(name) {
        this[ID] = uuid();
        this[NAME] = name;
    }
    get id() { return this[ID]}
    get name() { return this[NAME]}
    set name(name) { this[NAME] = name || 'Unkown'}
}

This allows us to keep all of our private state local to the object rather than in a separate closure, and our objects with private state can be created with class constructor syntax when that is preferred or required.

I want to note that this new best practice will likely become stale itself soon. TC39 is considering a proposal to add explicit private state to classes. It is currently in stage 3 with TC39 and is likely to become a part of the language. In the proposal, private state would look like this:

export class User {
    #id = uuid()
    #name = 'Unknown'

    constructor(name) {
        this#name = name;
    }
    get id() { return this.#id}
    get name() { return this.#name}
    set name(name) { this.#name = name || 'Unkown'}
}

Reading `arguments` to create variadic functions

Variadic functions are functions that may contain different numbers of arguments when called. For instance, a sum function that takes multiple arguments and adds them all together. Prior to ES6, this was accomplished by using the special arguments variable.

function sum() {
    let len = arguments.length;
    if (len < 2) {
        throw 'Sum requires at least 2 arguments'
    }
    // convert arguments to a "real array"
    let args = Array.prototype.slice.call(arguments);
    return args.reduce(function(a, b) { return a + b}, 0);
}
// sum(1,2) === 3
// sum(1, 10, 20) === 31

Using arguments is a bit messy. First, it’s a magic variable like this that is automatically available in the scope of any function. This isn’t obvious to people who don’t know about it. Second, it’s not a “real array”, it’s an “array-like object”, and thus it doesn’t have useful methods like map, filter, reduce available on its prototype. You always have to deal with the whole object, even if you have some named arguments and some unnamed ones. But ES6 provided a better solution with rest parameters.

function sum(...nums) {
    if (nums.length < 2)
        throw 'Sum requires at least 2 arguments'
    }
    return nums.reduce((a, b) => a + b, 0);
}

Rest parameters allow us to capture a varying-length group of arguments and read them as a real array with a meaningful name we choose. Because it captures a varying-length list of arguments, rest parameters must be the final parameter defined for a function, but they don’t have to be the first. You can define a few static parameters and leave a rest parameter to capture the remainder.

let createBlogPost = (name, date, ...tags) => ({
    id: uuid(),
    name,
    metadata: {
        date,
        tags,
    },
});

Rest parameters can do everything arguments do¹, but they do it in a less magic way, without the confusion of managing a fake array. At this point arguments should be considered an anti-pattern.

Using slice to convert something to an array

The previous snippet contained an extra outdated practice. Did you notice this line?

let args = Array.prototype.slice.call(arguments);

That was previously the best way to handle fake arrays like arguments, NodeLists, or just {3: 1, length: 4}. slice makes a copy of an array, optionally adding or deleting elements. Here we’re only passing arguments as the this value of the function, so it works the same as arr.slice(). When called without arguments, slice is essentially a clone function. But this code looks pretty bizarre if you haven’t seen it before. It also slides into the “wait did I mean slice or splice?” vortex of array method confusion. So it isn’t a particularly readable solution.

ES6 introduced the Array.from method to improve this situation. Array.from takes an array-like object and creates a real array copy of it. So it can be a clone method for existing arrays, or a conversion method for fakes like arguments. Array.from(arguments) certainly reads a lot cleaner than the slice method. It should be noted that they aren’t completely equivalent. slice will leave missing indices as holes, while Array.from will add those indices and set them to undefined. So you can see the following difference

let fakeArray = {1: 3, 3: 5, length: 4};

// convert object to a real array
let slicedArray = Array.prototype.slice.call(fakeArray);
let frommedArray = Array.from(fakeArray);

// filter Out 5s
let slicedArrayNoFives = slicedArray.filter(x => x !== 5);
let frommedArrayNoFives = frommedArray.filter(x => x !== 5);

console.log(slicedArrayNoFives) // [3]
console.log(frommedArrayNoFives) // [undefined, 3, undefined]

You can see my last post for more on holes in arrays.

Moving Forward

The fun of this industry is that everything is always changing. The frustration of this industry is that everything is always changing. I’m sure ES2020 or whatever they call it by then will give us plenty of new “best practices”. In the meantime, no need to throw away all your code right away. But if you’re setting an example for others, it would be a great time to update some visible examples of stale practices to point to the newer, more friendly solutions. Even if you don’t have your own blog posts or documentation to change, you can update public documentation. Did you know that anyone can update the MDN JavaScript documentation? I was able to add information about module strict mode to the MDN article while working on this post. And you can upvote modern answers to popular old questions on StackOverflow.

Or just do some refactoring of high traffic code in your own code bases, so that older code isn’t copy and pasted, and you have good examples of the better way to do things.

Ok technically arguments has an arguments.callee property that gives you a reference to the calling function, but you can recreate this by giving your function a name, and using arguments.callee is an error in strict mode. ↩︎
Yeah… that one is my answer. I promise I’m not just trolling for upvotes :) ↩︎

Weekly Links: June 23rd

Fri, 22 Jun 2018 20:00:00 -0400

JavaScript & The Web

Announcing Gatsby v2 beta launch! | GatsbyJS

Gatsby continues to be one of the most exciting things happening in web development. If you’re unfamiliar, Gatsby is a tool for building performant content-focused sites with React and GraphQL. This blog has been built on it for the last 18 months or so. Updating to V2 is pretty straightforward, I’ve been running it on here for the past week with no problems. And V2 offers some nice improvements in terms of hot loading speed, library updates (it’s React 16, Babel 7 and Webpack 4 friendly) and API improvements. It’s also obvious that since they announced that Gatsby contributors were forming a company the docs and marketing materials have been quickly moving from “indy open source” level to “real company” level, which is cool to see.

Programming Practices

Big Ball of Mud | Brian Foote and Joseph Yoder

This is an older piece (1999), but a cool reflection on “Big Balls of Mud”: computer systems that have been designed haphazardly and are now difficult to change. Besides giving these systems a great name, Foote and Yoder are thoughtful about why these systems come to be, and how and when that may be appropriate.

Overcoming Intuition in Programming

Some interesting reflections here on how we use frameworks to reduce the cognitively challenging parts of programming. I can definitely recall feeling less useful outside of my comfort zone when I’ve spent too much time working with “intuitive” code.

Books

JavaScript Allongé | Reg Braithwaite

I realized the other day that I had never written a review of one of my favorite JavaScript books on this site. This won’t count as a full review, but suffice it to say that if you’re interested in functional programming, want to get a deeper understanding of JavaScript, or really like JavaScript and coffee, this is the book for you.

On Writing Well | William Zinsser

Over the past year, I’ve been going through a challenge to read 30 books in a year. I am well aware of my limitations as a writer, so one of the books I added to the list was “On Writing Well”, recommended from multiple sources as the best resource for non-fiction writing.

Reading a 42 year old book¹ about writing is a nice view into what has changed over the last half century and what hasn’t. The basic principles the book lays out are as applicable as ever, however its specific examples often feel a bit dated. It lacks any direct instruction about the type of writing that now happens in emails, slack and social media², but it pointed me back to a time where our public writing was created with more care and resulted in more eloquence than you usually find in today’s frenzied discourse. Parts of the book felt dated to me, but there was plenty of practical advice to glean, and even more sections I found inspiring. I finished up wanting my writing to be better and had some practical ideas about how to make that happen. And I learned a little bit about painting birds and the salt caravans of the Sahara along the way. What more can you ask for?

It was last updated in 2006, so it’s not completely that old. ↩︎
There is a great opportunity sitting out there for somebody to write the definitive book on modern business writing skills. ↩︎

Code Golf: Sparse Arrays

Tue, 19 Jun 2018 02:00:00 -0400

Today on Twitter, Axel Rauschmayer posted a fun little coding challenge:

Code golf: check if an Array has holes. I’ll reply with two ideas of mine in a few minutes.
— Axel Rauschmayer (@rauschma) June 19, 2018

For context, code golf is a type of challenge where participants attempt to find the shortest possible way to express a program. Arrays that have holes, or sparse arrays, refer to arrays that don’t have values at every index between their maximum and minimum values. You can define a sparse array like this:

let arr = [1];
arr[100] = 2
// arr.length is 100

or like this

let arr = [1,,2];
// arr.length is 3

This is different from let arr = [1, undefined, 2], because the index keys are not defined. If this is difficult to understand, you can picture arrays as simple objects, with access to the Array.Prototype methods and a special length property that is aware of numeric keys. The important thing is not whether a value is defined for a specific index, but whether a key was ever explicitly set. Axel’s suggested solutions take advantage of that distinction:

// Does someArray have holes?

someArray.findIndex((x,i,arr) => x === undefined && !(i in arr)) >= 0
Object.keys(someArray).length < someArray.length
someArray.filter(_ => true).length < someArray.length
— Axel Rauschmayer (@rauschma) June 19, 2018

I’m going to standardize these into a function syntax for the purpose of comparison. I’m only going to consider one liners, so we’ll define a simple arrow function isSparse, and compare the length of the function bodies.

// Solution 1
let isSparse = someArray => someArray.findIndex((x,i,arr) => x === undefined && !(i in arr)) >= 0
// Solution 1 condensed (49 characters)
let isSparse = a => a.findIndex((x,i,b)=>x===undefined&&!(i in b))>=0
// Solution 2
let isSparse = someArray => Object.keys(someArray).length < someArray.length
// Solution 2 condensed (30 characters)
let isSparse = a => Object.keys(a).length<a.length
// Solution 3
let isSparse = someArray => someArray.filter(_ => true).length < someArray.length
// Solution 3 condensed (33 characters)
let isSparse = a => a.filter(_=>true).length<a.length

Axel’s solutions all rely on the fact that sparse items don’t have keys, and that Object.keys and Array.prototype methods therefore don’t iterate over them. That’s a good insight and a good foundation for getting even more concise. We can use a few tricks to do that.

Let’s start with his second solution, which was the shortest and clearest in practice and modify it a bit

// We're comparing the length of the list of keys to the array length
let isSparse = a => Object.keys(a).length<a.length
//But we can use reduce to do this instead (28 characters)
let isSparse = a => a.reduce(x=>x-1,a.length)==0

So our first step is to use reduce to save 2 characters. If you haven’t seen reduce before, I recommend checking out this great explainer by Sarah Drasner that I linked to in my Weekly Links post last week. Essentially though, reduce is a generic method for iterating through an array to produce a new value. It takes a function and an initial value, and uses the items of the array to iteratively transform the initial value. In this case, I’m just decrementing the value for each defined item in the array, starting at length and seeing if it equals 0. Which ends up being shorter than taking the length of the keys array. But we can still do better!

Our next step will take advantage of a property of numbers in JavaScript. When converting numbers to booleans, 0 is considered falsy, while every other number is considered truthy. Since we want our function to produce a boolean, we can take advantage of this property, and write our function like this.

// Boolean version (27 characters)
let isSparse = a => !!a.reduce(x=>x-1,a.length)

That seems like a bit of a local maximum. But getting to this point made me realize that we weren’t actually saving anything by using the boolean version over a comparison, since we can increment instead and end up at the same length.

// Counting up to length (27 characters)
let isSparse = a => a.reduce(x=>x+1,0)<a.length

Doing it this way though, we can actually save 2 more characters, because if reduce doesn’t have an initial value, it just uses the unmodified first element in an array as its initial value, and goes from there:

// Final (broken) Version (25 characters)
// UPDATE: This doesn't work
let isSparse = a => a.reduce(x=>x+1)<a.length

Update: This last solution only works in situations where the array starts with 1. So 27 is the best generic solution I can find at this point. Thanks to Axel for correcting me. Code golf is hard 😔.

So there you have it. 27 characters to identify sparse arrays. I’d love to hear from you if you can do better. Hit me up on Twitter or by email.

A Codemod Survival Guide

Sun, 17 Jun 2018 20:00:00 -0400

Do you like making repetitive syntax updates across many files? Enjoy the thrill of painstakingly combing through 1000 line diffs that all look the same to see if you made any typos? Relish that feeling of doubt in the back of your head as you wonder whether you actually caught all instances of the syntax you needed to update when you switched to the next major version of a library? If so, you should probably stop reading now, because you won’t like codemods at all. For the rest of us, codemods are a tool for assisting in large refactoring; they save you time and reduce typos.

Codemods originated at Facebook, first with a tool named codemod that served as a helper for regex or python based refactoring, allowing you to make iterative changes with pretty diffs, then with jscodeshift a tool for doing more advanced JavaScript refactors that work directly against a data model representation of the code, rather than relying on regex or direct programmatic changes to text. jscodeshift is extremely powerful, but I found it intimidating to approach. Once I took the plunge though, it wasn’t that hard to start writing codemods. So here are a few quick tips for anyone else who might be interested.

1. Start with a goal in mind

It’s always helpful to dive into a new technology by trying to solve a concrete problem. That’s especially useful for codemods, because the ideas behind them can seem abstract and poorly documented until you begin focusing on a specific issue. In my case I was originally motivated to look at codemods when updating an application from using MobX version 3 to MobX version 5. In MobX3, when you write code without using decorators, a typical model class will look something like this:

import { extendObservable, computed, action} from 'mobx';

export class Example {
    constructor() {
        extendObservable(this, {
            foo: false,
            bar: false,
            currentData: computed(() => {
                return foo || bar;
            }),
            otherData: computed(() => foo && bar),
            setFoo: action('Set Foo', val => {
                this.foo = val;
            }),
            setBar: action('Set Bar', val => {
                this.bar = val;
            }),

        });
    }

    unrelatedMethod(key) {
        return key;
    }
}

I was updating an app with dozens of these model classes. Unfortunately, in Mobx5, the syntax for these classes has changed, and it now needs to be rewritten like this:

import { extendObservable, action} from 'mobx';

export class Example {
    constructor() {
        extendObservable(this, {
            foo: false,
            bar: false,
            get currentData() {
                return foo || bar;
            },
            get otherData() {
                return foo && bar;
            },
            setFoo: val => {
                this.foo = val;
            },
            setBar: val => {
                this.bar = val;
            },

        }, {
            setFoo: action('Set Foo'),
            setBar: action('Set Bar')
        });
    }

    unrelatedMethod(key) {
        return key;
    }
}

Computeds must be shifted to gets, action functions have to be pulled out of their wrapper function calls, but the wrapper and action names have to be moved down to a new second object. This is really finicky to do by hand or with a text editor macro, because it’s a lot more complicated than just find and replace. I’d usually be converting from arrow functions that might or might not have blocks and return statements currently, and I’d have to move the actions down to the new object correctly even though some of the models had 10+ action functions. Facing several hours worth of finicky manual or script assisted refactoring work was enough to convince me it was time to learn about codemods.

2. Create a small example and learn about its AST

My first productive step towards creating a Mobx3to5 codemod for my codebase was creating the example code above. I iterated on it a bit as I found new things I cared about; I didn’t realize at first that computeds needed to be changed as well as actions, and I eventually realized I needed to handle 2 types of arrow function syntax. But the initial version I wrote got me going. Having a stripped down example of the code I cared about without the baggage of my real codebase was valuable, and being able to update it to include all the cases I cared about during testing was even better.

Once I created the example, I was able to understand better what I was trying to do by examining its AST. An AST (Abstract Syntax Tree) is a representation of a JavaScript file’s source code as a tree data structure. For instance here’s a portion of the AST for a basic Hello World program:

{
  "type": "Program",
  "sourceType": "module",
  "body": [
    {
      "type": "ExpressionStatement",
      "expression": {
        "type": "CallExpression",
        "callee": {
          "type": "MemberExpression",
          "object": {
            "type": "Identifier",
            "name": "console",
            "_babelType": "Identifier"
          },
          "property": {
            "type": "Identifier",
            "name": "log",
            "_babelType": "Identifier"
          },
          "computed": false,
          "_babelType": "MemberExpression"
        },
        "arguments": [
          {
            "type": "Literal",
            "value": "Hello World",
            "rawValue": "Hello World",
            "raw": "'Hello World'",
            "_babelType": "Literal"
          }
        ],
        "_babelType": "CallExpression"
      },
      "_babelType": "ExpressionStatement"
    }
  ],
}

This is a simplified example that looks only at the semantic structure of the syntax. The full version also contains information about the indenting and exact text of the code, and comes out to about 600 lines when represented as formatted JSON. All of this from console.log('Hello World');. Fortunately there’s a nice tool for exploring ASTs.

AST Explorer is a site that lets you enter some JavaScript code, and then visualize the AST that represents the code. When I put my example code above, in, I was able to see that the computed expressions in my original code showed as a CallExpression with an ArrowFunctionExpression as an argument.

When I put the result code I wanted in, I could see that I would need to switch the property that contained the callexpression to have a type get and would need to replace the call expression with a function expression.

3. Start with a small edit

As a starting point, I decided I wanted to write a script that could find MobX classes and just add the second object argument to extendObservable, with a dummy property. So this:

extendObservable(this, {
    foo: false,
    bar: false,
    currentData: computed(() => {
        return foo || bar;
    }),
    otherData: computed(() => foo && bar),
    setFoo: action('Set Foo', val => {
        this.foo = val;
    }),
    setBar: action('Set Bar', val => {
        this.bar = val;
    }),

});

would become this:

extendObservable(this, {
    foo: false,
    bar: false,
    currentData: computed(() => {
        return foo || bar;
    }),
    otherData: computed(() => foo && bar),
    setFoo: action('Set Foo', val => {
        this.foo = val;
    }),
    setBar: action('Set Bar', val => {
        this.bar = val;
    }),

}, { foo: 'bar'});

To do this, I took an existing script that I found in this repo of small codemod examples and modified it to lookup extendObservable calls and add a second argument. I filtered it to only include calls with 2 arguments, in case I ran the code a second time¹.

module.exports = function (file, api) {
    const j = api.jscodeshift;

    return j(file.source)
        // We're looking for a CallExpression for extend observable
        .find(j.CallExpression, {
            callee: {
                name: 'extendObservable'
            }
        })
        // Verify that it's only using 2 arguments so far, and that the second one is an Object Expression
        .filter(p => p.value.arguments.length == 2 && p.value.arguments[1].type == "ObjectExpression")
        .forEach(p => {
            // Now we want to
            // 1. Add a 3rd argument object
            let args = p.value.arguments;
            args.push(j.objectExpression([j.property('init',j.identifier('foo'), j.literal('bar'))]));
            // TODO: 2. Move action declarations to that argument
            // TODO: 3. Convert action statements to plain functions
            // TODO: 4. Convert computeds to get statements
        })
        .toSource();
};

This is a working script to achieve my simple goal. To break it down:

j(file.source).find(j.CallExpression, {
            callee: {
                name: 'extendObservable'
            }
        })

jscodeshift provides a jQuery style api for traversing a tree. Here we’re looking for CallExpressions that have the name extendObservable. We now have a collection of these expressions, and we can filter it down to only the expressions that have 2 arguments.

.filter(p => p.value.arguments.length == 2 && p.value.arguments[1].type == "ObjectExpression")

Next we want to go through each callExpression and transform it. In this case by finding it’s arguments list and pushing a new one, using the jsCodeshift API to simply generate the AST objects neededed to represent {foo: bar}.

.forEach(p => {
            // Now we want to
            // 1. Add a 3rd argument object
            let args = p.value.arguments;
            args.push(j.objectExpression([j.property('init',j.identifier('foo'), j.literal('bar'))]));
            // TODO: 2. Move action declarations to that argument
            // TODO: 3. Convert action statements to plain functions
            // TODO: 4. Convert computeds to get statements
        })

Finally, we call .toSource() to write the tranformed AST back to the file.

4. Test your edit

Because this type of coding lends itself to experimentation, it’s important to develop a good feedback loop while working on your codemod. If you have a simple example you’re off to a good start. The next step is to find a good way to run your script and see the output. Simply running your test is a bit frustrating, because jscodeshift is destructive. It overwrites your file in place, and doesn’t save a copy. You can get around this though by adding --dry --print flags to your command line call

jscodeshift -t my-codemod.js example.js

will run the transform defined in my-codemod.js against example.js and modify it in place.

jscodeshift --dry --print -t my-codemod.js example.js

will print the transformed file to the console and not update example.js.

For simple codemods and small example files, this is a nice way to iterate quickly. For more extensive codemods with multiple test cases, I’m excited to try out jscodeshift’s Jest support². It has helper methods for working with transforms, and a workflow of Jest watch checking the output of several inputs against several expected outputs would be lovely.

5. Iterate

After I got that first edit working, it was simply a matter of working through my list. I needed to remove the experimental foo:bar and then

Move action declarations to the new argument object
Convert action declarations to plain functions in the existing object
Convert computeds to get statements

I was able to take those one at a time, and my final script looked something like this:

module.exports = function (file, api) {
    const j = api.jscodeshift;

    const buildDecoratorPropertiesFromCurrentObject = currentObj => {

        let decoratorProps = [];
        currentObj.properties.forEach(prop => {
            if (prop.value.type === 'CallExpression' && prop.value.callee.name === 'computed') {
                prop.kind = 'get';
                let fnBody = prop.value.arguments[0];
                if (fnBody.type === 'ArrowFunctionExpression' && fnBody.body.type !== 'BlockStatement') {
                    prop.value = j.functionExpression(null, fnBody.params,
                        j.blockStatement([j.returnStatement(fnBody.body)]),
                        false);
                } else {
                    prop.value = j.functionExpression(null, fnBody.params, fnBody.body, false);
                }
            }
        });
        let dataProps = currentObj.properties.filter(prop => !(prop.value.type === 'CallExpression' && prop.value.callee.name === 'computed'));
        dataProps.forEach(prop => {
            if (prop.value.type === 'CallExpression' && prop.value.callee.name === 'action') {
                // copy action to the decorator area first...
                let args = prop.value.arguments;
                let dupProp = Object.assign({}, prop);
                dupProp.value = Object.assign({}, dupProp.value);
                dupProp.value.arguments = (args.length === 2) ? args.slice(0, 1) : [];
                decoratorProps.push(dupProp);

                // then flatten things out
                prop.value = (args.length === 2) ? args[1] : args[0];
            }
        });
        return [dataProps, decoratorProps];
    }
    return j(file.source)
        // We're looking for a CallExpression for extend observable
        .find(j.CallExpression, {
            callee: {
                name: 'extendObservable'
            }
        })
        // Verify that it's only using 2 arguments so far, and that the second one is an Object Expression
        .filter(p => p.value.arguments.length == 2 && p.value.arguments[1].type == "ObjectExpression")
        .forEach(p => {
            // Now we want to
            // 1. Add a 3rd argument
            // 2. Move action declarations to that argument
            // 3. Convert action statements to plain functions
            // 4. Convert computeds to get statements, or just move them down to the second object?
            let args = p.value.arguments;
            let [dataProps, decoratorProps] = buildDecoratorPropertiesFromCurrentObject(args[1]);
            args[1].properties = dataProps;
            args.push(j.objectExpression(decoratorProps));
        })
        .toSource();
};

That looks rather crazy, but the buildDecoratorPropertiesFromCurrentObject is mostly straightforward data manipulation, looking for some patterns in the data, and replacing them with a different pattern. Everything else is just the same base code from our initial example.

6. Use version control and code reviews!

Actually running a finished transform against a codebase is crazy easy. Just give it a base folder, and jscodeshift will run your script against all JavaScript files in the directory. This is definitely a “With great power comes great responsibility” tool though. Before running jscodeshift across a large code base, make sure you are using version control and that you’ve checked in any other changes. Because the script will update many files at once, you want to be able to isolate the change and review it carefully afterwards, to make sure that you see what is different, and that those changes look right. In my case, I got to be done with my script twice, because the first time I thought it was ready to execute across all of the app files, things still went wrong. It was easy to fix though, since I was able to just checkout my changes and try again.

Resources

If you have a change that you want to make to your codebase, but have been holding off because you think it would take forever, I definitely encourage you to check codemods out. They’re not as scary as you think! Here are some more resources to get started:

This is one of many assumptions I made that won’t be true for MobX classes generally. I was writing a script to convert my app to MobX5 compatible syntax, not write a general case update tool. In the end there were a few examples that violated my assumptions in the codebase, and I had to update 2 files by hand. ↩︎
jscodeshift was created by Christoph Pojer who is also one of the primary maintainers of Jest. ↩︎

Weekly Links: June 15th

Thu, 14 Jun 2018 20:00:00 -0400

JavaScript & The Web

GraphQL Stack

This is a nice overview of the technologies involved with GraphQL, for anyone who has heard of the technology but isn’t quite sure how it all fits together.

Understanding The Almighty Reducer | Sarah Drasner

Moving from a loop mindset to making proper use of forEach/map/filter/reduce was one of the biggest shifts I made when moving from Java development to JavaScript. Reduce especially can be hard to learn initially, and it never becomes simple. This is a great introduction for those who are unfamiliar or haven’t gotten comfortable with writing reducers yet.

Software + Business

On paying for software | Seth Godin

This mirrors my opinions on paying for software pretty well. There’s a place in the world for free apps, and I definitely have passed on apps because of price before. But the apps I’ve used that have had real value to me have almost always been paid, usually these days as an ongoing subscription. The exceptions, like Twitter, tend to come with frustrating downsides that are difficult to disentangle from the free-to-play model.

Tech World

Climate Change Can Be Reversed by Turning Air Into Gasoline | The Atlantic

This seems really promising.

On The Sad State of Macintosh Hardware | Rogue Amoeba

I’m lucky enough to have access to 2 2015 era Macbook Pros (1 for home, 1 for work) and they’re amazing machines. I’m hoping that they both have long lifespans. But I’m increasingly not sure what my next laptop will be whenever I next have a choice. It seems like the laptops since then have lost some things I appreciate about my current laptops (Magsafe charger, HDMI port) and made the keyboard significantly worse, with minimum benefits. I hope that Apple can right this ship.

Weekly Links: June 10th

Sat, 09 Jun 2018 20:00:00 -0400

JavaScript & The Web

The React is “just” JavaScript Myth | Dave Rupert

I’ve been thinking a lot about the reasons that JavaScript development is perceived as intimidating and over-complicated the past few weeks. Might have a full post on that soon, but for now, there’s a good point here.

React is relatively simple at it’s core, but it’s difficult to find examples of using it in the wild, that don’t require you to learn Webpack/Babel/npm at minimum, and many “starting points” will also toss in Redux/React Router/Jest/etc as well.

MobX 5: the saga continues… | Michael Weststrate

MobX is the first major library I’ve seen to take advantage of proxies to give normal looking JavaScript objects/arrays magic powers¹. This sort of “just make it work in the prettiest way possible” API is the type of thing that tends to produce really cool results as long as it works correctly 100% of the time, but becomes a huge pain as soon as it’s flaky. I’ve never had any problems with MobX after using it for 2 years, so I’m hopeful seeing them as an early adapter here. Will be interested to see if they start getting used more widely going forward. If so, I hope that browser start thinking about how they can expose these types of patterns well in developer tools. How do you drop into a side effect caused by a proxy in a debugger? It’s an interesting problem

Meta programming with ECMAScript 6 proxies | 2ality

In case I lost anybody with my discussion of the last link, here’s an old but good walkthrough of what Proxies are and how you can use them to create some pretty magical APIs.

Microsoft Buying Github

🔥 Our reactions to Microsoft buying GitHub |Adam Stacoviak & Jerod Santo

A Changelog podcast that summarizes the reaction of the 2 primary Changelog hosts to the annoucement as well as collecting reactions from around the internet.

github-xp | Mårten Björk

This is just fun. A Chrome extension to render Github in the style of Windows XP.

Even if they chose to announce that in the silliest way possible ↩︎

ES6: The Bad Parts

Mon, 04 Jun 2018 20:00:00 -0400

This month is the 3 year anniversary of the ECMAScript2015 spec, better known as ES6. It was a spec update that brought a ton of features to the JavaScript language, and the start of a new era of improving the language. It was also the last “big bang” release of JavaScript, as TC39 has now moved to a pattern of annual small releases rather than making large multi-year language releases.

There has been a boatload of justified hype around ES6 over the last 4 years. I’ve been writing all of my code using the modern spec during that time¹. So this is a good time to step back and evaluate the features that were added. Specifically there are some features that I used for a while that I’ve stopped using, because they made my code worse.

In his book JavaScript: The Good Parts Douglas Crockford dedicated a chapter to the Bad Parts of JavaScript. The Bad Parts are the features of JavaScript that he felt shouldn’t be used. Fortunately there is nothing as bad in ES6 as some of the old JavaScript stinkers like coercive equality, eval or the with statement. ES6 was a well-designed release. But there are some features and patterns that I avoid. A JavaScript feature makes my Bad Parts list because:

It’s a trap. The feature looks like it does one thing, but has unexpected behavior in some cases that can easily lead to bugs
It increases the scope of the language for little benefit. The feature provides some small advantage, but requires the readers of my code to know about obscure features. This is doubly true for API features where using the feature means that other code that interacts with my code must know about the feature.

In that spirit, here are my Bad Parts of ES6.

const

In older versions of JavaScript, we declared variables with the var keyword². This was mostly fine, but var has peculiarities. It creates variables that are added to the global object, or scoped to a function, but don’t respect other block boundaries. It is also possible to reference a var declared variable before it is declared. These oddities occasionally come back to bite developers. In response ES6 introduced 2 new keywords for declaring variables: let and const. These solve the primary quirks of var; they are block-scoped, so a variable declared inside of a loop is not referencable outside of that loop, and trying to access the variable before it is declared in code results in a reference error. This was a big step forward. It was also confusing, because we were given 2 solutions.

A const variable cannot be re-assigned after it has been declared. This is the only difference between const and let. This seems useful, and it can be. The problem is the name. What it guarantees doesn’t match what most people expect from a constant.

const CONSTANT = 123;
// this will cause "TypeError: invalid assignment to const `CONSTANT`"
CONSTANT = 345;
const CONSTANT_ARR = []
CONSTANT_ARR.push(1)
// this will print [1] without an error
console.log(CONSTANT_ARR)

const prevents reassignment, but it doesn’t make objects immutable. This is a fairly weak guarantee for most value types. Because this feature invites confusion, and const is otherwise redundant with let, I choose to always use let.

Template Literal Tags

const is an example of the spec creating too many solutions to too few problems. Template literals are the inverse. The template literal syntax was TC39’s way of tackling string interpolation and multi-line strings. Then they decided to give it a string macro feature as well.

If you’ve never seen a template literal tag, they’re a little like decorators for strings. Here’s an example from MDN:

var person = 'Mike';
var age = 28;

function myTag(strings, personExp, ageExp) {

  var str0 = strings[0]; // "that "
  var str1 = strings[1]; // " is a "

  // There is technically a string after
  // the final expression (in our example),
  // but it is empty (""), so disregard.
  // var str2 = strings[2];

  var ageStr;
  if (ageExp > 99){
    ageStr = 'centenarian';
  } else {
    ageStr = 'youngster';
  }

  return str0 + personExp + str1 + ageStr;

}

var output = myTag`that ${ person } is a ${ age }`;

console.log(output);
// that Mike is a youngster

This isn’t totally useless. Here’s a roundup of some use cases. HTML sanitization is a useful trait. And this is the currently the cleanest way to do things when you’re having to perform the same operation on all inputs to an arbitrary string template. But that’s a relatively rare scenario, and you can also accomplish the same use cases with a (more verbose) function API rather than using tagged literals. And for most things, the function API won’t even be worse. This feature is not adding new capabilities, but is adding new concepts that future readers of my code have to be familiar with. So I plan on staying clear as much as possible.

Overly crazy destructuring expressions

Some features are fine when used simply, but can spiral out of controls. For instance, I am happy to write ternary statements like

let conferenceCost = isStudent ? 50 : 200

but they become tough to follow if you start nesting them:

let conferenceCost = isStudent ? hasDiscountCode ? 25 : 50 : hasDiscountCode ? 100 : 200;

This is how I feel about destructuring. Destructuring lets you pull variables out of objects or arrays

let {a} = {a: 2, b: 3};
let [b] = [4, 5];
console.log(a, b) // 2, 4

and also lets you rename variables, get nested values, and set defaults.

let {a: val1} = {a: 2, b: 3};
let [{b}] = [{a:3, b:4} , {c: 5, d: 6}];
let {c=6} = {a: 2, c: 5};
let {d=6} = {a: 2, c: 5};
console.log(val1, b, c) // 2, 4, 5, 6

All of which is great, until you start combining those features. For instance, this expression declares 4 variables, userName, eventType, eventDate, and eventId pulled out of different spots in the object structure of eventRecord.

let eventRecord = {
  user: { name: "Ben M", email: "ben@m.com" },
  event: "logged in",
  metadata: { date: "10-10-2017" },
  id: "123"
};
let {
  user: { name: userName = "Unknown" },
  event: eventType = "Unknown Event",
  metadata: [date: eventDate],
  id: eventId
} = obj;

It’s pretty much impossible to follow. Code like this is much easier to read with the destructuring split into parts or eliminated.

let eventRecord = {
  user: { name: "Ben M", email: "ben@m.com" },
  event: "logged in",
  metadata: { date: "10-10-2017" },
  id: "123"
};
let userName = eventRecord.user.userName || 'Unknown';
let eventDate = eventRecord.metadata.date;
let {event:eventType='UnknownEvent', id:eventId} = eventRecord;

I don’t have a clear line for when destructuring has gone too far, but anytime I can’t look at it and instantly know what variables are being declared, I know it’s time to break things down for readability.

Default Exports

One nice thing about ES6 was the way it standardized things that had been accomplished using competing libraries. Classes, Promises and modules all benefited from being folded into the spec after the community had time to absorb lessons learned from competitive user-land implementations. ES6 modules are a great replacement for the AMD/CommonJS format wars, and provide a nice clean syntax for imports.

ES6 modules provide 2 main ways of exporting values: named exports, and default exports.

const mainValue = 'This is the default export
export default mainValue

export const secondaryValue = 'This is a secondary value;
export const secondaryValue2 = 'This is another secondary value;

A module can define multiple named exports, but only a single default export. When importing default exports, the importing file can give the default export any name as there isn’t any lookup on name. Named exports are imported by using the name of the variable in the exporting file, though renaming is possible.

// default import
import renamedMainValue from './the-above-example';
// named import
import {secondaryValue} from './the-above-example';
// named import with a rename
import {secondaryValue as otherValue} from './the-above-example';

Default exports were the preferred syntax of the folks designing ES6 and they intentionally gave it the cleanest syntax. But in practice I’ve found named exports to be better.

Named exports match the names of the imported variable by default, making searching easier for those not using intelligent tooling.
Named exports can be matched to variables in other values prior to an import statement being written, allowing niceties like auto-import for those using intelligent tooling.
It’s possible to use named imports consistently for everything, but that’s only possible with default imports if you never want to export multiple values from a file.³ Using only named modules keep things simpler.

Naming things is good. It promotes consistency in discussion and code. I use named exports because I like clarity.

Thanks Babel! ↩︎
Or global scope, or a property on an object, or a function declaration ↩︎
Exporting objects with multiple values as properties seems like a workaround here, but that loses the tree-shaking value that ES6 Modules provide for bundlers like webpack. ↩︎

Weekly Links: June 3rd

Sat, 02 Jun 2018 20:00:00 -0400

JavaScript & The Web

New in Chrome 67 - Desktop PWAs | Google Developers

Google has a new process for bundling web apps as installable applications, this time based on the emerging Progressive Web App semi-standard¹. An interesting alternative approach for smaller applications that don’t need the full electron runtime, or education apps targeting Chromebooks

Programming Tools

black: The uncompromising Python code formatter | Github

This looks like a Python version of Prettier, and something I’d be interested in playing with.

Jest 23 | Jest Blog

Jest just keeps getting better. This release brings interactive snapshot updating, so that you can approve some changes and not others. That’s a big UX improvement and reason enough to upgrade, but looks like there are some improved matchers and new customizability options for power users as well. A solid release.

Tech & Business

Microsoft has reportedly acquired Github | The Verge

Microsoft seems to be doing everything it can to earn back love and trust from developers. Some of that has been by investing in new stuff like Visual Studio Code, some of it has been attitude, and then things like this are just buying somebody elses love. But I think they’ve earned back enough good will at this point that most developers will be willing to give them a chance and not assume that this will automatically be a bad thing. Which is a big change from 5 years ago.

It’s a little vague still to really be considered a standard, but Microsoft and Google at least are providing broad support for similar sounding things that they both call PWAs, and Mozilla and Apple have been supporting the underlying technology if not the branding as much. ↩︎

Weekly Links: May 27th

Sat, 26 May 2018 20:00:00 -0400

Business & Technology

Gatsby contributors form Gatsby Inc. | Gatsby Blog

This blog is built on Gatsby, and I’ve written and spoken on it a few times in the past. So I’m really excited to see what happens with the tech as it becomes a company, while continuing to be open source. It will be really exciting to see where they take it from here, and I think they have a real chance to be successful, especially if they can get a foothold into the ecommerce world.

Problem Solving

The Cult of the Root Cause | Donald Reinertsen

A really nice short post on problem solving, and how the quest to find the root cause for an issue can sometimes be a distraction.

Books

Unbroken | Laura Hillenbrand

I’ve fallen behind on my quest to read 30 books in a year, but this month I was able to read a fantastic book that multiple friends had recommended: Unbroken. With the second movie based on the book coming out this fall, you’ll probably start to hear about this book again. It’s been around since 2010, but I didn’t read it when it was first popular, and now I wish I had. It is an amazing story of human perseverence, surviving trauma, and ultimately grace. It doesn’t oversimplify the human beings whose lives it chronicles, but still manages to shine on a light on the best and worst of the human condition. Highly recommended.

Weekly Links: May 19th

Fri, 18 May 2018 20:00:00 -0400

JavaScript and the Web

The React Podcast | Michael Jackson

I listen to a decent amount of podcasts, but I’ve usually found it tough to get into programming podcasts. Without visually being able to see code, they’re usually either ultra high level, or get hard to follow when going into the details. The changelog family of podcasts has generally been my favorite though, and the new¹ React podcast has been consistently excellent over it’s first 9 episodes. Worth a listen.

Guess.js | Minko Gechev

This is a crazy look at what I suspect will be the next step in static site performance stuff. This blog does some of the cool stuff mentioned here (we prefetch links on the page thanks to Gatsby, and I use google analytics at build time to determine popular pages and populate the Most Read Articles list on the main page) but this is next level, using analytics to efficiently prefetch. Excited to try it on the site.

Non-Tech

The Vision | Tom King

Ok so I’m pretty late on this one², but Tom King’s The Vision is the best comic book I’ve read in years. He’s got a pretty good run going on Batman now as well, but this is a brilliant self-contained story that holds up as well as anything I’ve seen. Great art as well.

Newly relauched? I’m not totally sure of the history ↩︎
The single edition Vision comics were originally released in 2015, though the hardcover that I purchased was just released this year. ↩︎

MobX Patterns: State Machines & Flags

Sun, 13 May 2018 20:00:00 -0400

A few weeks ago, I was listening to the React Podcast, specifically their episode on State Machines with David Khourshid. About 20 minutes into the podcast, there was a discussion on the benefits of using enums over flags to model the state of objects that can change across multiple dimensions. It reminded me of one of the nice patterns that MobX makes possible, and I thought I’d share it here.

Background

To set the stage here, the podcast discussion centered around modeling an async process like a promise. One possible way given was to use a mix of flags and if statements like this:

// loading object
class ExampleLoader {
    constructor() {
      this.isLoading = false;
      this.hasError = false;
    }

    fetchData() {
      //fetch data from server and return it here
    }

    async load() {
      try {
        this.isLoading = true;
        let data = await this.fetchData();
        this.isLoading = false;
        this.hasError = false;
      } catch(e) {
        this.hasError = true;
        this.isLoading = false;
      }
    }
}

using one of these objects might look like this:


if (loader.hasError) {
  // show error state
} else if (loader.isLoading) {
  // show loading state
} else {
  // show data State
}

In the podcast David Khourshid criticized this as a haphazard way to manage state, that doesn’t show the whole picture. Because we can’t easily see how the various flags combine, it’s tough to keep track of all possible states. For instance, in the above example, did we really want to show the error state if we had an error, but have called load() again and both hasError and isLoading are now true? Maybe that was the intention, but maybe that was just a state that was missed during the original construction of the program. It’s hard to tell.

As an alternative to full blown state machine modeling, David advocated for an easy first step of swapping out flags for enums. In that case the code above might look like:

const STATES = {
  INITIAL: 'initial',
  LOADING: 'loading',
  ERROR: 'error',
  LOADED: 'loaded'
}
// loading object
class ExampleLoader {
    constructor() {
      this.state = STATES.INITIAL;
    }

    fetchData() {
      //fetch data from server and return it here
    }

    async load() {
      let {LOADING, ERROR, LOADED} = STATES;
      try {
        this.state = LOADING
        let data = await this.fetchData();
        this.state = LOADED;
      } catch(e) {
        this.state = ERROR;
      }
    }
}

with usage like this:


switch(loader.state) {
  case STATES.ERROR:
  // show error state
  case STATES.LOADING:
  // show loading state
  case STATES.LOADED:
  // show data State
  case STATES.INITIAL:
  default:
  // show blank state
}

We can see just from using an enum, we’ve cleared some ambiguity: now it is clear that we should be showing a loading state when we’re loading data, even if a previous load resulted in an error. And we’ve also been forced to remember another state distinction that was missed originally, the difference between a successful load and the initial state before load is called. This type of Enum-based approach is really helpful for modeling complex systems and especially async process state. But it’s not perfect. While ENUMs are great for listing out every possible state, sometimes we just need to know if an object is in a particular state. For instance, we might have a special help-prompt component that should only show if our data is in an error state, but is otherwise divorced from the details of the data. In that case loader.hasError is simpler and cleaner than loader.state === STATES.ERROR. It’s a straight forward named expression that doesn’t require importing a constant from another file or matching up to a string value. Enums add some verbosity and ceremony to code that can otherwise be very simple. Fortunately MobX allows us to have our cake and eat it too.

MobX

MobX is a state management solution created by Michael Weststrate. It grew out of the React community but can be used with other frameworks or on it’s own. One of its nicest features are its intelligently updated computed properties. With MobX we can write our loader object like this:

import {extendObservable, computed, action } from 'mobx';
const STATES = {
  INITIAL: 'initial',
  LOADING: 'loading',
  ERROR: 'error',
  LOADED: 'loaded'
}
// loading object
class ExampleLoader {
    constructor() {
      let {INITIAL, LOADING, ERROR } = STATES;
      extendObservable(this, {
          state: INITIAL,
          isLoading: computed(() => this.state === LOADING),
          hasError: computed(() => this.state === ERROR),
          updateState: action(state => this.state = state),
      })
    }

    fetchData() {
      //fetch data from server and return it here
    }

    async load() {
      let {LOADING, ERROR, LOADED} = STATES;
      try {
        this.updateState(LOADING);
        let data = await this.fetchData();
        this.updateState(LOADED);
      } catch(e) {
        this.updateState(ERROR);
      }
    }
}

Now we have both our state enum, and we can check specific properties. So both of the below examples work:

// ENUM example
switch(loader.state) {
  case STATES.ERROR:
  // show error state
  case STATES.LOADING:
  // show loading state
  case STATES.LOADED:
  // show data State
  case STATES.INITIAL:
  default:
  // show blank state
}

//Flags example
if (loader.hasError) {
  // show help component
}

And because MobX takes care of the details of efficiently updating our flags based on state, we know everything will stay in sync and remain performant, without a bunch of ugly code to make sure of that.

Weekly Links: May 5th

Fri, 04 May 2018 20:00:00 -0400

Intriguing Github Projects

MDX

Ever since I started using Gatsby to power this blog, I’ve wanted the ability to write markdown with React elements intermingled in it. This looks like the first working version of that I’ve seen. I’m excited to play with it.

bat: A cat(1) clone with wings | David Peter

This looks like a really cool project, a “cat” clone with syntax highlighting and git line indicators. Lot’s of cool stuff being written in Rust these days.

Tech and Business

Amazon Letter To Shareholders | Jeff Bezos

I originally saw this letter in an article on Stratechery, (and that piece is also great). It’s one of the best short pieces on business that I’ve ever read. The Stratechery piece breaks down the idea of “divinely discontent customers”, which is an amazing image. But the other advice about how to set and maintain high standards is even better. Just great stuff, from the leader of one of the most important companies in the world.

Fun

“What the customer really wanted 😏… “

This is a new variant on one of my favorite programming jokes¹

What the customer really wanted 😏 pic.twitter.com/sDTbQsZb3U
— Jayme Hoffman (@jhoff) May 1, 2018

If anyone knows where the original version of this joke is from let me know and I’ll credit it. ↩︎

Weekly Links: April 27th

Thu, 26 Apr 2018 20:00:00 -0400

JavaScript & The Web

CSS at Scale: LinkedIn’s New Open Source Projects Take on Stylesheet Performance | LinkedIn Engineering

This is a cool new set of CSS tools from LinkedIn. This looks most similar to CSS Modules/PostCSS to my eyes, with a focus on optimizing bundle size for both CSS and templates/components. I’d be very interested in playing with it.

Programming Tools

Strange and maddening rules | Joel Spoelsky

I’m continuing to enjoy Joel’s series on StackOverflow. I’ve always been more sympathetic to their moderation strategy than some, so I’ll leave it as an exercise to the reader whether the explanations presented here hold water, but definitely worth reading if you’ve ever tried contributing to StackOverflow and been frustrated.

Defeating Electron | Felix Rieseberg

Electron doesn’t have the best reputation. Any time an electron based app makes news on Hacker News, three quarters of the comments are bashing Electron and the companies that use it. It’s been criticized recently and prominently on Daring Fireball and seems to generally be looked down on by “real” MacOS developers. So I think this is a helpful response. An Electron community member knocks down some straw-men (Electron doesn’t only exist because web developers don’t want to learn new things) and then lays out what would be needed to replace it.

I think there’s an interesting tension here: This post describes what would be needed to create a more compelling cross-platform solution from the point of view of companies and developers. But the criticism has mostly been focused on the requirements of users, not companies. Specifically that Electron apps tend to be slower/less-memory-efficient/less accessible/less attuned to platform specific conventions. So companies are making the decisions that let them serve the most number of people, while evolving to meet their needs as quickly as possible, with as small cost as possible. But they’re possibly limiting the top level quality they can achieve, and there is a possible social good being missed by the people excluded (those with lower performance machines or accessibility needs¹).

Tech and Ethics

‘I Fundamentally Believe That My Time at Reddit Made the World a Worse Place’ | Noah Kulwin & Dan McComas

A great interview on the massive potential for good or evil today’s tech products can have. This does suggest that the problem is deeper than simply “engineers and designers behaving ethically” though. You can get at Reddit’s issues by following the money.

A Very Good Program

On a lighter note, here’s the real case for engineers and designers needing to behave ethically.

I made a VGP (very good program) that makes it so it looks like I’m typing on slack whenever anyone else is typing, and stops when they stop.

Everyone loves it so far and doesn’t find it annoying at all!https://t.co/W8e2EKuVXX pic.twitter.com/ZZd01GxYqg
— will leinweber (@leinweber) April 25, 2018

I am not an accessibility expert, but I’m aware that it is a criticism of Electron. I can’t evaluate the actual difficulty of making a native vs electron application accessible. ↩︎

The New Hello World

Tue, 24 Apr 2018 20:00:00 -0400

JavaScript has had a weird reputation inversion over the past decade. For years it was the stripped down toy language that could. It didn’t have the features people wanted, and it did have the features they didn’t want, but it ran everywhere and it was easy to just start building things. The last decade has shown that you can actually build some serious applications in JavaScript. But it has also eaten away at the ability to start simple and try new things. State of the art for JavaScript apps now involves compile steps, a long tooling chain, and a million choices for developers to make. So the folks who are helping new developers get started have a few options for teaching JavaScript:

Embrace The Complexity - Want to learn to write Web Apps? Let’s just toss you into a boilerplate that’s using node, webpack, react, redux, redux-saga, react router, glamorous, eslint, prettier, npm scripts, lodash, and babel. This is what production apps look like, so you might as well start learning in a realistic environment.
Hide The Complexity - This is where we maybe use something like Codepen to provide a sandboxed area where we may be using Babel/React/etc, but the user doesn’t have to configure or think about them too much. My understanding is that Glitch tries to go down this route to some extent as well.
Discard The Complexity - Teach coding like it’s 2010 all over again! This approach is to focus more on the JavaScript language as it’s implemented in the browser, using script tags without a build step, and maybe even an older/simpler library like jQuery to explain DOM concepts and CSS. This approach can eventually evolve towards introducing more complexity.

For the past few years I’ve seen a blend of all of these approaches when I’ve looked at other folks' beginner level resources. It’s been hard to determine the best way to prepare new developers for the complexities of the development process. But good news, I think that era is finally coming to an end.

The Ideal Intro

In an ideal world, we’d be able to introduce new developers to the concepts that undergird modern web apps, and teach them syntax that is as up to date as possible without having to entangle them in the implementation details and distractions of production grade tools like Babel and Webpack. So we want them to start out writing JavaScript code that is:

Modular
Cross-environment
Modern (ES6+ syntax as appropriate)
Simple to start with

The last 10 years of tool build up have been primarily focused on making it easy to achieve the first 3 goals in production (along with prioritizing other needs of large scale applications). But production tooling is by nature less concerned with #4. Fortunately native environments are starting to catch up on the first 3, and that means we can do better.

The New Hello World

Let’s say we want to frame an intro through the classic “hello world” program. Our goals are to create a JavaScript function that can print “Hello World”, and then expand on that a bit. And we want to teach this in a way that will still look familiar when our learners get themselves into a production JavaScript code base.

Bare Bones

We might start with something super simple like this:

<!-- index.html -->
<html>
  <body>
    <h3 id='name-display'></h3>
    <script>
      let displayArea = document.getElementById('name-display');
      displayArea.innerHTML = 'Hello World';
    </script>

  </body>
</html>

That’s about as simple as a JavaScript hello world gets. You can access it by opening the file directly from the file system in your browser, and it’s easy enough to edit the text and see changes on refresh.

But in 2018, we can do something cool next. We want to be modular and separate concerns, so lets create a hello world module. This is an opportunity to introduce functions and modules. It introduces our only small tooling step: adding a simple web server.

Hello Modules

On OSX, it’s easy enough to set up a simple web server with python -m SimpleHTTPServer 8080. We can do that with our existing index.html and verify that we can now see it at localhost:8080 in our browsers¹.

After that, we can start doing cool stuff, starting with creating a new module:

//hello.js
export const sayHello = (name='World') => {
  return `Hello ${name}`;
};

This is our first function. Depending on your audience, you may want to strip this down a bit and remove the default parameter, or spend some time explaining the different concepts in play here. But the focus can be on the concept of functions, as well as the ability to export them from a module.

Now we can update our index.js file to use the new function. Assuming both files are in the same directory, that will look like this:

<!-- index.html -->
<html>
  <body>
    <h3 id='name-display'></h3>
    <script type="module">
      import {sayHello} from './hello.js';
      let displayArea = document.getElementById('name-display');
      displayArea.innerHTML = sayHello();
    </script>

  </body>
</html>

Now we have a hello world built on a foundation that can scale towards modern JavaScript environments. We have modules and modern syntax, and the only difficult setup step has been a lightweight web server so far. The rest is just script tags.

Interactive

It’s easy enough now to take our module and turn it into something interactive:

<!-- index.html -->
<html>
  <body>

    <div id='container'>
      <h2> Enter a Name and Hit The Button</h2>
      <input id='name'>
      <button id='change-name'>Say Hello</button>
      <h3 id='name-display'></h3>
    </div>

    <script type='module'>
      import {sayHello} from './hello.mjs';

      let displayArea = document.getElementById('name-display');
      let input = document.getElementById('name');
      let button = document.getElementById('change-name');

      button.addEventListener('click', () => {
        displayArea.innerHTML = sayHello(input.value);
      });
    </script>

  </body>
</html>

This example introduces DOM events and let’s the user decide who to say hello to. It also is a good jumping off point for further experimentation for a new dev.

Crossing Environments

Going from the browser to node, even though it is simpler in some ways, is always going to be a bigger jump for newcomers because of most people’s unfamiliarity with the command line, and the base hurdle of installing node itself. But once we’re ready to move past browsers, our new hello world can be easily translated to node land. The biggest hurdle is in the file names. We will have to rename or copy hello.js to be named hello.mjs (Node’s way of differentiating between files using ES6 syntax). After that, we can write a simple script to start out:

//node.mjs
import {sayHello} from './hello.mjs';
console.log(sayHello());

And then, as long as we’re on an up to date version of Node, we can run it with node --experimental-modules node.mjs. And we will see our Hello World.

There are some annoyances for beginners here. Node only recently added module support, and beginners must

Rename their modules to the currently non-standard mjs ending
Have a recent version of Node (v8.5+)
add the experimental-modules flag and learn to ignore the warning it gives

But overall it isn’t meaningfully harder than learning to use node has been in the past.

Interactive Again

We can make our node.js version accept a command line arg easily enough, along with more opportunities to learn about functions, arrays and conditionals.

//node.mjs
import {sayHello} from './hello.mjs';

const getLast = list => {
  if(list.length) {
      return list[list.length - 1]
  }
  return null;
}

console.log(sayHello(getLast(process.argv)));

The Future

I’m excited to see the new dev experience moving back closer to alignment with the production web app experience. I think that can only be a positive things. Some steps that will help even more if/when they happen:

Module file format standardization - The .mjs extension seems to still be controversial, but it also seems a bit inevitable. So the question will be whether it becomes more standard to use across all modular JS on the browser and node. The current split and experimental support makes for a confusing distraction for new learners
De-flagging - Right now Firefox and Node still require flags for using modules. That reduces beginner friendliness.
Where is the jQuery of 2018 - This is more speculative, but I think the next step to a beginner friendly hello world is a “beginner friendly framework” that is based on the “one way data flow/declarative translation of state to UI” paradigm that has swept across pretty much all modern JS frameworks. That’s an important concept to understand, and there isn’t anything I’m aware of out there right now that let’s you build with those concepts but is as approachable to newcomers as jQuery is. jQuery was a gateway to DOM manipulation, AJAX, and functional array manipulation for a generation of developers. React and create-react-app is probably the closest to that concept today, but it isn’t nearly as simple.

That’s all for now. Code samples for this are on github. Goodbye World.

This should work in the latest Chrome, Safari and Edge out of the box. I encountered a bit of slowness and flakiness on Chrome, but it worked like a charm in Safari. Firefox still has this stuff behind a flag. ↩︎

Weekly Links: April 22nd

Sat, 21 Apr 2018 20:00:00 -0400

JavaScript & The Web

Designing Very Large (JavaScript) Applications - Malte Ubl

This is a great talk, both because of the concepts it delivers, and because it serves as sort of a funhouse mirror look at a JavaScript framework that will probably never be public.

Programming Tools

Hyper 2

A new version of the Electron-based terminal app from Zeit. I switched back to iterm a while ago after trying Hyper for a while, I’m excited to give it another try though.

Tech and Politics

The Facebook Current – Ben Thompson

Ben Thompson is the best person to read if you want to understand the broader implications of tech news, so naturally he’s all over the current Facebook situation

Book Review: The Inmates Are Running The Asylum

Tue, 10 Apr 2018 20:00:00 -0400

For the past few months I’ve been working through a goal of reading 30 books in a year. I made a list sourced from books that I’d wanted to read in a while, and books that friends recommended, and I’ve been working through it. For the most part I’ve gotten through books pretty quickly as they’ve been generally fascinating. Last month was the first time I hit a book that I struggled to get through a bit. Fortunately I stuck with it, and it was worth it in the end.

The Inmates Are Running The Asylum by Alan Cooper is a book that I’d seen recommended all over the place by design-savvy developers and product folk. Cooper is the creator of Visual Basic, a highly regarded programmer who became an even more high regarded designer, pioneering the field of interaction design (laid out in his other book About Face) and the use of personas in software design. While About Face is his explanation of the techniques behind interaction design, The Inmates Are Running The Asylum is his case for why interaction design is necessary as part of the software development process. As somebody who has been working in a partial design role for the past year, it seemed like a great fit.

Unfortunately, the first few chapters were a struggle. The first 4 chapters of the book are devoted to laying out the poor “current situation” for technology when the book was written in the late 90s. Cooper goes through a long list of poor experiences in devices and software like TVs, email clients, car remotes, alarm clocks and more. The problem is that almost universally his examples are out of date. People certainly still write software with bad UX, and struggle especially with software that is replacing hardware. But they struggle in different ways now than they did 15 years ago. Possibly due to the influence of the author, and partially due to some maturation in the field and changes in technology, the particular examples listed in these chapters fail to land. It’s possible to extrapolate some things to present day, but it makes for choppy reading. At minimum the book would benefit greatly from a second edition.

A second issue that made the book harder to apply was also related to the publish date. This is a book that is firmly rooted in a waterfall/long release cycle view of the software development process. In fact if anything it is advocating for a more waterfall-like approach than the waterfall status quo of 2004. I’m genuinely curious how the author would choose to communicate his critiques today in light of the normalization of agile techniques.

If I haven’t scared you off yet, you’re at the same place I was 4 chapters in, wondering if I should cut my losses and give up on a great but outdated work. Fortunately I stuck with it and reaped the benefits. The last 10 chapters are as relevant as ever in todays world.

Chapter 5 describes the complementary roles of design, engineering, and business acumen in successful businesses. Chapters 6-8 make the case for interaction designers being a distinct field that can’t easily be filled by industrial designers, visual designers, marketers or programmers. There is a particular focus on the problems of developers designing the code that they’re implementing. Chapters 9-11 are a high level view of the principals of interaction design. Finally chapters 12-14 describe what implementing design into a software culture looks like. There is a ton of great content in these chapters.

As a programmer who has been designing and implementing for the last year, the critiques of chapters 6-8 rang quite true. I see all the tendencies outlined in the book dragging me away from user-focused design, and it’s a constant battle to remind myself to build for the user and not development convenience.

I was familiar with many of the ideas in chapters 9 through 11 as techniques but these chapters did a really good job of laying out the heart behind them as well as making a business case for their use. Frankly they also made me much more interested in reading About Face, so hopefully I’ll be able to get a review of that up sometime soon (after I finish my current list of books).

Finally, while the implementation chapters at the end struggled a bit due to being written in a pre-agile world, it holds up because it focuses a lot on creating a design culture and isn’t just stuck on prescriptive steps for integrating designers.

Overall I can definitely recommend this book if you’re interested in learning more about the role of design in a healthy software development organization and you know enough about the industry to contextualize the 90s-era advice in 2018. It is definitely a high-level book; Cooper describes it as the “business case for interaction design”, and it is written at a “business level”. If you want a more practical and in the weeds design resource, then About Face is probably a better bet. But The Inmates Are Running The Asylum is still solid reading.

Weekly Links: April 7th

Fri, 06 Apr 2018 20:00:00 -0400

JavaScript and the Web

React Lifecycle Diagram | Dan Abramov

Dan shared a really great illustration of the React lifecycle methods on twitter that can help you internalize how they work if you’re unfamiliar.

Announcing the Second Edition of “Refactoring”

The upcoming second edition of Refactoring will have it’s code examples written in JavaScript. I’d love to see more architecture-level books written for JavaScript developers. I don’t think we have great definitive resources for those questions yet, and rewriting a classic with JS examples is a great first step.

Present web pages to secondary attached displays | Google Developers

This is a small thing, but I thought Chrome’s experimental support for external display controls was cool, and may eventually lead to me actually using one of the many JS based presentation tools instead of Keynote.

Tech

Former Uber Backup Driver: We Saw This Coming | Citylab

May be some sour grapes here, but the piece raises some interesting questions about the transition from our current state to a self-driving car future, and whether a Level 3 self driving car is even a feasible goal.

The Stack Overflow Age | Joel Spolsky

Stack Overflow had a huge influence on my early days learning JavaScript. I love seeing the history here from one of the founders.

Humor

Returning To The Empress Hotel | Nick Burchill (Facebook)

One of the funnier stories I’ve read on the internet in a while.

Weekly Links: March 30th

Thu, 29 Mar 2018 20:00:00 -0400

A big pile of links for Easter weekend. Happy Easter everybody!

JavaScript and the Web

ES modules: A cartoon deep-dive | Lin Clark - As usual, Lin Clark offers a great approachable explanation of a complex topic. Here she gets into the weeds of how ES modules are implemented in browsers, and how the module syntax differs from CommonJS.

Redux - Not Dead Yet! | Mark Erickson - A good explainer of where Redux fits into the React landscape today.

Update on Async Rendering | Brian Vaughn - Important read for React devs. There are changes coming to the core lifecycle methods.

System Fonts in CSS | Craig Hockenberry - I’m always happy to see improvements to the default font situation on the web¹. I don’t love relying on custom loaded fonts for this blog and I wish there were more high quality fonts that were reliably available without resorting to loading your own. This particular feature doesn’t pass the “caniuse test” for now though; Chrome and Safari support it, but there’s nothing from Edge or Firefox yet.

Command Line Tips

git log – the Good Parts | Ian Miell - A good rundown of some of the things you can do with git log. It misses that you can use --author to filter by the committer though, which is a flag I use all the time when I know who did a specific piece of work but am vague on what actually changed.

Why I usually run ‘w’ first when troubleshooting unknown machines - Here’s a useful command line command I didn’t know about. w shows you a list of users currently on a machine and what they’re doing.

Code Style, Documentation and Code Reviews

“Just” | Brad Frost - A good reminder to not assume everybody has our accumulated knowledge. I’ve found a good cure for this is to spend some time working with programming tools I have no experience in. Trying to install gems in Ruby for instance always reminds me how confusing npm was when I started using it.

Avoid Else, Return Early | Tim Oxley - I’ve found this pattern of short-circuiting functions to produce very readable code. I’ve also seen these referred to as guard statements.

Putting the I back in IDE: Towards a Github Explorer| Jane Street Tech Blog - This is an interesting idea, although I was a bit weirded out by the idea of commenting in the source files to leave code review comments. The idea of deeper editor integration is excellent though, and I’d love to see Github evolve further in this direction with their Atom plugin.

Privacy in the news

Facebook scraped call, text message data for years from Android phones | Ars Technica - It’s been a bad few weeks for Facebook. You can maybe get away with trying to get everyone’s data, and you can maybe get away with being careless with user data, but when you’re as big as Facebook is, you absolutely can’t do both.

How Europe’s new privacy rule is reshaping the internet | The Verge - GDPR is a big deal. If you’re involved in web products that store user information, you should understand this. I’m excited to see meaningful regulation on privacy issues that seems to be reasonably well written.

Humor

Cow Astrology | John Cook - When a mathematician examines the implications of astrology’s claims.

Or I would always be happy if there was ever anything to be happy about. ↩︎

Front End Computer Science Basics: Boolean Logic

Mon, 26 Mar 2018 20:00:00 -0400

This is the second post in a series of posts on Computer Science basics for front end developers. The goal is to focus on the practical implications of basic CS theory for JavaScript development.

Modern programmers owe a lot to the efforts of many head in the sky theoretical mathematicians from the 1800s and early 1900s who were searching for mathematical truths and ended up making discoveries that serve as the basis of many modern computing inventions. Boolean Algebra is one of a few branches of mathematics that was developed for purely theoretical reasons that has ended up shaping the computer revolution. It fortunately has the virtue of being a relatively approachable to mathematical newbies.

Boolean algebra is a study of operations on logical values. Or in more understandable language, it is studying the rules of logic when we’re limited to dealing with only 2 types of values, true or false. The basic building blocks of boolean algebra are incredibly simple: in addition to the true and false values, usually encoded as 0 for false and 1 for true, boolean algebra considers 3 basic operations:

The “AND” operator (∧) - a∧b is true if and only if a and b are both true.
The “OR” operator (∨) - a∨b is true if at least one of a or b are true and false otherwise.
The “NOT” operator (¬) - ¬ is a unary operator. It only affects a single value. ¬a is true if a is false, and false if a is true.

Of course this is a whole field of mathematics, so the interesting part comes as you start combining these pieces. Mathematicians have discovered many “laws” that apply to boolean logic. Some of these will probably look familiar to those of you who took high school algebra. Here’s a sample of the laws from Wikipedia.

The last 3 especially are worth remembering. Taking them out of mathematical notation for a second and using pseudocode instead, the rules are:

Double Negation: not not x == x
De Morgan 1 (not x) and (not y) === not (x or y)
De Morgan 2 (not x) or (not y) === not (x and y)

Boolean Algebra in JavaScript

For most JavaScript developers, the preceding description of boolean algebra probably seemed suspiciously familiar. It turns out that JavaScript has all the base units of boolean algebra in its core language. True and false are represented by the appropriately named boolean primitives true and false. AND, OR, and NOT are represented by &&, || and ! respectively.

So a JavaScript statement like

let x = false;
let y = true
if(x && y) {  // false
  willNeverHappen();
}

is equivalent to x∧y in boolean algebra notation. But this being JavaScript, there are some quirks and things to know about.

First of all it’s worth noting that boolean logic in JavaScript is lazy by default. That means that JavaScript will only evaluate enough of a boolean expression to know what the result will be. So in an expression like this:

let alwaysReturnTrue = () => true;
let alwaysReturnFalse = () => false;
let value = alwaysReturnFalse() && alwaysReturnTrue();

the function alwaysReturnTrue will never be called. This will matter when we start talking about real world takeaways.

The second JavaScript quirk gets its own section. Because booleans get more complicated when they interact with the rest of the JavaScript world.

Truthy and Falsy

When JavaScript conditionals like the if statement evaluate an expression, they aren’t strictly evaluating to see if something is the boolean values true or false. Instead they’re looking to see if the expression is truthy or falsy. This isn’t the kindergarten version of booleans, instead it’s JavaScript’s way of handling the fact that not all expressions are boolean. Rather than requiring you to cast your values to boolean or use a statement that produces a boolean, JavaScript instead treats every value as either truthy or falsy. The rules for this are reasonably consistent and easy to follow once you understand what is happening¹. It turns out that most values are truthy, so it is easiest to just learn the falsy values. In JavaScript the falsy values are:

false,
null,
undefined,
0,
NaN,
'', // or ""
document.all // Special case browser hack just for fun

If an expression that evaluates to one of these values is read into a boolean context like an if statement or ternary, it will be treated as false. Any other value will be read as true.

Implications and Hacks

So far we’ve gone through some theory, as well as JavaScript’s own quirks. But now let’s talk about 3 different implications of this stuff for day to day JavaScript programming.

Shortcuts using the && and || operators - It’s fairly common, especially in special context like minified code and JSX expressions in React, to run into code that get’s clever with the lazy behavior of JavaScript boolean operators. Specifically the && statement often get’s used as an inline if statement like this:
```
let userID = showID && getUserID();
```
which is equivalent to
```
let userID;
if (showID) {
  userID = getUserID();
} else {
  userID = showID; //presumably false or null
}
```
The || statement can be used for a similar purpose, usually to provide a fallback value.
```
const = createNewUser(providedRole) {
  let  userRole = providedRole || 'basic-user';
  //...
}
```
It’s important to be careful with this type of use though, because if it would be valid for the user input to be a falsy value like '', null or 0, this can introduce subtle bugs. More on that in a second.

I wouldn’t personally relying too heavily on this technique. Besides the possibility of subtle bugs, it isn’t very approachable for new developers, especially if you try nesting logic this way. But you should know how it works since you will see it, especially in minified code you may need to debug and JSX.
!! for casting - Another common idiom to be aware of is the use of double negation to cast values to a boolean. It is fairly common when you have code that is expecting a boolean and not just a truthy/falsy value to convert the value to a boolean. One way to do that is using the Boolean constructor, like var bool = Boolean(value), but another method is to use double negation. The first negation evaluates to the boolean that is opposite the truthy/falsy value, and the second one flips it back to the original truthy/falsy state, but as a boolean.

I personally find this notation to be less confusing than the &&/|| shortcuts, but your mileage may vary. Either way, it’s important to understand its role since it is common to find !!s in real world code.
0s and empty strings are dangerous - For our last quirk, let’s imagine a code example function getCountFromServer:
```
const getUserCountFromServer = async () => {
    try {
      let response = await fetch('users');
      let users = response.json()
      return users.length;
    } catch (e) {
      // if we can't get the count, return null
      return null;
    }
}
```
This function makes a request from the server, and then returns the number of users, or null if it can’t get the value from the server for whatever reason. Maybe not the best error handling, but certainly something you could see in real world code. If we want to test to see if this function got data correctly, we have to be careful. We might want to simplify our testing by taking advantage of null’s falsiness like this:
```
let userCount = await getUserCountFromServer();
if (userCount) {
  // do something with the info
} else {
  //handle the case where the number failed to load
}
```
Do you see the bug? It’s subtle. With code written like this, if there are no users it will be treated the same as if the number had failed to load at all. This is a pretty common bug in situations where a value might be undefined or null, but could also be a number. Strings have a similar issue.

It’s important to understand that if 0 or empty strings are a valid value for a variable, you can’t rely on a general falsiness check to see if the value is defined. Instead it’s better to either define the exact falsy value you expect for “no value” (null in this case) or to check based on type. For instance in this case, you could rewrite the code to look for a number:
```
let userCount = await getUserCountFromServer();
if (typeof userCount === 'number') {
  // do something with the info
} else {
  //handle the case where the number failed to load
}
```

Feedback Welcome

This is the second post in a new series, and I’d love to hear whether it is helpful for you, and any other topics you’d like to see covered. Feel free to reach out on Twitter or email. And if you see anything that looks wrong or misleading to you, don’t hesitate to open an issue on my blog’s github repo.

The exception for me are empty arrays. They get evaluated as truthy in JavaScript and falsy in Python, and I can never remember which is which without testing. ↩︎

Weekly Links: March 24th

Fri, 23 Mar 2018 20:00:00 -0400

TC39 and the future of JavaScript/ECMAScript

#SmooshGate FAQ | Mathias Bynens - A great explanation both of the background behind the recent twitter controversy over Array.prototype.smoosh as well as the TC39 process generally.

js-classes-1.1 | Allen Wirfs-Brock - This is the new proposal for expanding JS classes I mentioned in my post last Friday. It aims to be a more minimalist take on expanding class capabilities (primarily adding private state syntax) than other proposals that have come along over the last few years.

thoughts.md | Yehuda Katz - This was a response to the JS Classes 1.1 proposal from Yehuda Katz, the lead proponent of adding decorator syntax for JavaScript classes. He compares the new proposal to existing proposals, with specific concerns for how it would interop with decorators.

Computer Science and Software Engineering

basecs | Vaidehi Joshi - While I’ve been starting to consider how to explain CS concepts for front end developers without a CS background, I found this excellent beginner set of CS articles written by Vaidehi Joshi on Medium over the past year. Worth diving into, especially if you like fun visuals.

The 3 Skills That Helped Me Become a Better Software Engineer | Victor Savkin - This is worth sharing solely for the advice about mental models. In my experience the ability to think in abstractions (and understand when you need to dig deeper) is one of the biggest hurdles in programming when trying to grow from a beginner to a more advanced software engineer.

Books

I’ve been working through a goal to read 30 books in a year, and have worked through a few more books recently. Here are two good ones.

The Martian | Andy Weir - Yes I know I’m 3 years late on this one. But I’m sure there are other lost souls like me who entered 2018 without reading this book. If you love tech and science (and I’m guessing you do if you’re reading this), you should find a way to read The Martian. My first thought when I started reading it was “This is great”. My second thought was “How in the world did this end up becoming a Hollywood Blockbuster starring Matt Damon?”. The Martian is unapologetically geeky, and it is “real science” fiction, with a very human core. I loved it.

Death By Meeting | Patrick Lencioni - This was a recommendation from my manager, and is a great read for anyone who wants to improve the meetings they’re in at work. It’s written in a “business fable” style, with a story serving to illustrate the principles it is trying to teach. One thing I love is that it tells the story of the principles being implemented from a lower rung employee, rather than a top-down executive driven approach, so hopefully it can be inspirational to you even if you feel like you wouldn’t be able to change your meeting culture even if you knew what to do.

Front End Computer Science Basics: Variables and Memory

Tue, 20 Mar 2018 20:00:00 -0400

A few weeks ago, I asked for input on the idea of resources on Computer Science for front end developers. Because there was a decent amount of interest, I’m planning on fleshing out the idea a bit with a small series of posts on CS basics. These posts will cover some concepts that would usually be covered in an intro level CS course. If folks find them helpful I may move on to more advanced topics like Data Structures & Algorithms, the software abstractions that modern web development is built on top of, and other math/theory topics.

With these posts I’m not trying to replicate a CS intro course and won’t cover everything. I’m not trying to teach anyone how to program. Instead I’m covering the ideas and details behind concepts that FE devs may have learned by doing but never been exposed to from an theory point of view. I am not an academic, so the goal will be to keep a casual tone focused on the practical implications of theory to a Front End developer. If I’m successful, these posts will give you a better set of mental models when working to understand the code you’re reading and writing every day. That’s plenty of throat-clearing, so now I’m going to jump into one of the most fundamental building blocks of any programming language: variables.

What are Variables? (Not as obvious as it sounds)

Variables in programming languages are symbols that we assign to represent a value. That seems simple enough, but that sentence is too vague to properly account for the complexity of reality. When dealing with variables in the real world, it’s important to have a good mental model of how they really behave, or you may become confused by how your programs act. For instance if you believe that variables are always live references to whatever other variable they were assigned to, you may be surprised by the behavior of this snippet:

let a = 1;
let b = a;
a = 2;
console.log(b) // prints 1

and if you believe that variables always point to a unique value that is created when you assign to them, you might be surprised by this one:

let a = [1, 2, 3]
let b = a;
a.push(4)
console.log(b) // prints [1, 2, 3, 4]

Knowing generally that a variable points to a value is a great start. But a proper understanding of the model that a programming language uses for variables can help you avoid subtle bugs.

How Variables Work

Here’s the most important sentence for gaining a proper understanding of how variables work: “A variable points to a location in memory.” Explaining how memory is organized and allocated is outside the scope of this post, but for now it’s sufficient to just understand memory as a long list of blocks, each of which has it’s own address.

When a variable is assigned to, it points to the address of whatever value is on the right hand side of the assignment expression (if newly created, the value is assigned an open address in memory first).

If another variable is assigned to the first variable, it too will point to that same address in memory

Finally if we reassign the original variable, it will point to a new location in memory, and the second variable will continue to point to the same location.

Function arguments are based on these same concepts. When a function is called, the argument variables are assigned to point at the memory address of the value passed into the function.

Mutable vs Immutable Variables

This memory address stuff can seem pretty abstract, and not very useful, but it’s a great gateway to understanding more practical topics. A while back I wrote a post about mutable and immutable data. If you’re unfamiliar with the concepts you can read the original article for background. But basically, JavaScript values are divided into 2 categories: primitives like booleans, strings, and numbers which are immutable, and objects which are mutable. Immutable primitives can’t be changed, when they’re operated on they can only be replaced with a new value. Mutable objects can be changed in place. That “in place” qualifier can seem a bit hazy without understanding the underlying memory situation, but now we can explain this more clearly: the value for a primitive will never change within a memory location, a variable will instead have to point to a new location in memory. A mutable variable can be modified without changing it’s location in memory. This is the explanation behind our confusing example from earlier.

let a = [1, 2, 3]
let b = a;
a.push(4)
console.log(b) // prints [1, 2, 3, 4]

Because b points to the same location in memory as a, when we mutate a, b continues to point to the same array as a and sees the same changes we performed on a.

Implications and Gotchas

There are a few practical implications of these concepts, and gotchas that folks with the wrong mental model around variables tend to run into.

=== is based on location in memory for objects - While it does compare primitive values directly to see if they’re the same value, for objects, === compares by reference. So the following code works in a way that people might not expect if they don’t understand variables.

let a = {x: 2};
let b = a;
let c = {x: 2};
let sameReference = x => x;
let d = sameReference(a)

console.log(a === b) // true
console.log(a === c) // false
console.log(a === d) // true

Modifying objects in functions modifies them outside of the function scope - This is a pretty common gotcha for folks who don’t understand variables and memory. When they have a bad mental model of function arguments, you get code like the following made up example.

// function to create a contact that is ready to send
let formatContactForSend = contact => {
  contact.fullName = contact.firstName + contact.lastName;
  delete contact.id;
  return contact;
}
// get a contact from somewhere
let newContact = {
  id: 1,
  firstName: 'Ben',
  lastName: 'McCormick'
};
// send along the formatted version of the contact
send(formatContactForSend(newContact));

//oops
console.log(newContact); /* {
  firstName: "Ben",
  lastName: "McCormick",
  fullName: "Ben McCormick"
}
*/

const isn’t really a constant - ES6 introduced the new keywords for declaring variables let and const. Both help with issues around variable scoping¹, but the difference between the two can seem rather confusing. It seems like const might indicate a constant value that can’t be changed. But mutable values declared with const can in fact change:

const arr = [1, 2, 3];
arr.push(4);
console.log(arr) // [1, 2, 3, 4]

Instead const is once again about memory and reference assignment. A variable that is assigned through const can’t be reassigned to a new location in memory. So if it is pointed to a primitive like 123 or true, it truly is a constant, because the value can’t change. But when it’s pointed to a mutable value, it’s reference won’t change, and it will always === any variables that have been assigned to it, but the actual contents of that memory can change.

Feedback is lovely

This is a new series, and I’d love to hear whether it is helpful for you, and any other topics you’d like to see covered. Feel free to reach out on Twitter or email².

A topic for a different post ↩︎
My carrier pidgeons have all fled their coop. I do promise that if you send me smoke signals I will sit back and ponder their meaning. Right after calling the fire department. ↩︎

JavaScript Proposals In Production

Thu, 15 Mar 2018 20:00:00 -0400

This post was adopted from a twitter thread that outgrew it’s medium today. You can see the original thread here.

This morning I saw the following tweet:

*TC39 proposes Array.flatten feature that breaks code that was working fine for 10 years*
"IT'S THEIR FAULT FOR DOING NONSTANDARD STUFF"

*TC39 proposes changing classes feature not yet in standard*
"IT'S NOT FAIR I WROTE CODE USING THAT"
— Dave Methvin (@davemethvin) March 14, 2018

The tweet refers to this proposal for replacing a number of other proposals for augmenting JS’s current class syntax, as well as recent backlash on twitter over a proposal to not use flatten or flatmap for future JS standard syntax due to their previous use in mootools.

Because people have been using Babel or other tools to transpile their code and use the proposed class syntax today, it would be frustrating for many devs if the proposal shifted under them. I’ve been there. And thus, story time:

A few years ago I used a polyfill for a browser API that wasn’t yet fully supported in all browsers. And I used it in production code. Browser support didn’t pick up nearly as quickly as I expected, and then the spec changed. Supporting that code has caused me problems repeatedly since, and there isn’t a lot of support, given that the syntax isn’t standardized. Because I am working with limited resources I still haven’t been able to get rid of the polyfill code completely. It’s baked into one of the most critical portions of my production application. And so I’ve had to live with it and deal with some of it’s non-ideal aspects for a long time. Since then I’ve avoided putting anything in production that isn’t in a spec. And I have absolutely no regrets about that.

The lesson I’ve learned: Polyfilling a partially supported spec for compatibility is fine. Playing around with new proposals is fine (and actually a great idea if you want to influence the future of the language). But I never want to use polyfills/transpiling for a proposed spec in production code. And yes that means no decorators/class properties/pipeline operators in my code for now. But that’s ok. JavaScript is a pretty decent language today. A little patience will help you in the end. Even libraries like MobX that encourage the use of “future specs” will still work without them if they’re reasonably well maintained.

Why Does This Happen?

To some folks the lesson of “Don’t use language features that aren’t standardized yet” probably looks obvious. So why does this happen? I think there’s a community/culture aspect to some of this. For instance, many of the early adopters of React really wanted to push the language forward to achieve better developer experience, and were more likely to propose and advocate for new syntax. This was fine for them, because many of these folks were either

a. Employees at large companies who had the resources to support whatever code extensions they wanted, and dedicate time to a large refactor if they got it wrong (not typical of most devs) b. Consultants/teachers who are always trying to stay on the bleeding edge for teaching purposes and who don’t build projects intended to be maintained long term (not typical of most devs) c. Involved in the language standards process and experimenting with new syntax for the purpose of influencing the language, rather than writing production code with it (not typical of most devs)

Because of this, the need to compile JS for JSX purposes anyway and the heavy emphasis in the early React community on rethinking best practices, I’ve seen a lot more of this type of pre-standardization use of features in the React community than elsewhere. The Ember community for instance doesn’t seem to have the same problems. But it started with early adopters in React, and spread when folks who were less informed about the standards process cargo-culted the code examples. So here we are.

My suggestions

Learn about the ECMAScript standards process. This post can get you started.
Don’t use non-standard code in production. It just isn’t worth the headaches you may get later.

Weekly Links: March 16th

Thu, 15 Mar 2018 20:00:00 -0400

JavaScript And The Web

Bitmasks and the new React Context API | Haukur Hallvarðsson - A nice rundown of one aspect of the new React Context API that is likely to be familiar to low-level developers but will probably be a new idea to many front end devs: using bitmasks to test a condition.

MobX 4: Better, simpler, faster, smaller | Michel Weststrate - MobX continues to be a fantastic state management libary and this update looks like it will make it even better.

Lighter than Lightweight: How We Built the Same App Twice with Preact and Glimmer.js | LinkedIn Engineering - This is a really cool real world comparison of 2 JavaScript frameworks, where they actually built the same app in 2 separate frameworks, with core contributors to each framework involved in the implementation. To be honest, my main takeaway is that among modern frameworks “framework performance” can be a bit overrated, and not a great reason to choose a specific library or framework, as compared to factors like developer experience, community, and fit with the existing codebase.

Rust And WebAssembly

I’ve been thinking about taking the time to learn a new language for the first time in a while. I think WebAssembly and Rust transpiled to WebAssembly in particular are going to become an important part of high performance JavaScript applications and frameworks/libraries. So I’m starting to check it out. Here are a few recent relevant links

Making WebAssembly better for Rust & for all languages – Mozilla Hacks – the Web developer blog - This outlines a plan to improve the developer experience of what I think is the future of WebAssembly in most web apps: interop with JavaScript to improve performance of open source libraries and some core performance sensitive code.

GitHub - atom/xray: An experimental next-generation Electron-based text editor - A non-webassembly example of using Rust selectively to increase performance of an application built primarily using web technologies.

Rust’s 2018 Roadmap | Rust Blog - A good rundown of the areas of focus for Rust this year. I’m particularly interested in their target use cases, since 3 of them (CLI tools, Network Services and WebAssembly/the web) are things that I have built before and will build again.

Classic Book Review: The Visual Display of Quantitative Information

Mon, 12 Mar 2018 20:00:00 -0400

If you have any interest at all in data visualization, you can’t get very far learning about it without hearing the name Edward Tufte. His four books on the subjects have had a huge influence on the industry over the last 35 years, a time where data visualization has blossomed as it has moved from a tedious hand drawn task to a computer assisted activity that even middle schoolers can attempt with the software available on most laptops.

Tufte’s first book, The Visual Display of Quantitative Information is also his most well known and acclaimed book¹. It lays out many of the principles that he would go to expand on elsewhere:

Focus on the data
“Graphical excellence is that which gives to the viewer the greatest number of ideas in the shortest time with the least ink in the smallest space”
Graphics should have integrity. They shouldn’t mislead the reader.
Remove unecessary clutter
and more

The book is also beautiful. Tufte chose to self-publish it so that he could control the exact layout and typography. He also mixes in many old hand drawn visualizations to mix with a fewer new computer assisted graphics. The result is stunning, and makes for a very enjoyable read throughout, as there is always something interesting to look at.

I was pleased to find that the book didn’t back up some of my own pre-conceived notions about Tufte and his work. When I’ve seen him referenced in the past, it has been almost solely in terms of minimalism. He was an expert witness for fewer colors, fewer lines, less ink. But his takes here are more nuanced. He definitely tends towards minimalism, and argues for reducing graphs to their bare minimum ink amount over conforming to familiar forms². But his go to example of great data display throughout the book is hardly minimalist. Minard’s map of Napoleon invading Russia is a complex visualization that tracks the size of an army, time, location and temperature, all in a single image.

Instead Tufte is really arguing for data efficiency. Simple things should be displayed simply. If you have a single number, don’t dress it up in a fancy graph. Just show the number. If you have a series of numbers, a table or simple scatter will work fine, no need for a fancy piece of chart “art”. But when you have truly complex data, a complex visualization can make it accessible in ways that a large page of words and numbers never could on its own.

I found The Visual Display of Quantitative Information fun, informative and inspirational. If you work with data and have never read it, I’d definitely recommend checking it out.

Amazon named it one of the 100 Best Non-Fiction books of the 20th century ↩︎
Some of that may be a relic of this book being originally written in the 80s before computers, his own work and the work of other similar thinkers greatly increased public awareness of data visualization standards like line and bar charts. ↩︎

Weekly Links: March 11th

Sat, 10 Mar 2018 20:00:00 -0400

Rather than posting random stuff that popped up this week like usual, I wanted to instead post a set of “classic” links instead. The theme this week is Learning React. Here are some of the links that helped me to learn different React concepts as I was first tackling it. Note that these are links that helped me learn the concepts behind different parts of the React ecosystem. Some are intro level, while some assume basic familiarity with the concepts they discuss.

The Motivation Behind React

Thinking In React - It’s strange, but I never expect the official docs for a project to be the best explanation of their purpose. Official documentation tends to lean towards the “how” rather than the “why”, and the motivations of the maintainers aren’t always the motivations of the community anyway. But this article from the React docs remains the best explanations of the problems that React is solving as well as the way that it solves them.

Pure UI - Even moreso than Thinking in React, this 2015 article nails the paradigm shift that React represented. The power of UI as a function of data has been the biggest change I’ve experienced in my career building UIs, and this article was like a message in the bottle from the future when it was first written 3 years ago.

Flux, MobX and Redux

A cartoon guide to Flux | Lin Clark - This was a nice early run through of the Flux concepts. Lin Clark is one of the best “front end explainers” out there, and she does a good job here.

The Fundamental Principles Behind MobX | Michael Weststrate - Like most of the other links here, this is a theory link, not a practical introduction.

Getting Started With Redux | Dan Abramov on Egghead.io - Like Thinking with React, this is a great introduction to a library that starts with the problems to be solved and shows how a new library solves that problem, rather than simply telling you what a tool does.

You Might Not Need Redux | Dan Abramov - Another helpful article for understanding what Redux is and isn’t solving for, helpful if you’re trying to decide whether to include it in a project.

React Patterns

Advanced React Component Patterns | Kent C Dobbs on Egghead.io - This is my one paid content link, but its so good that its worth sharing. This was a fantastic course that covers patterns like Higher Order Components, Render props and more. It discusses both the implementation concerns and motivation behind each.

Future Stuff

React’s New Context API | Kent C Dobbs - This is fairly recent, but Kent Dobbs gave a great explanation of React’s new context API

Sneak Peek: Beyond React 16 | React Blog - I posted this last week as well, but this is a great inspirational talk on where React is going next.

Weekly Links: March 2nd

Thu, 01 Mar 2018 20:00:00 -0400

JavaScript And The Web

Sneak Peek: Beyond React 16| React Blog - This is a preview of where React is going next. I’m still processing through this, and how I can use it, but Suspense seems like something that will make dealing with async data much easier in React. Loading states are something I deal with all the time, and having cleaner primitives for expressing them will be a big win.

Programming Tools

The Lost Art of the Makefile |Jesse Hallett - An interesting case for using Makefiles to manage modern JavaScript builds on projects that don’t need the overhead of webpack. I don’t think this is going to be the next trend in JS, but I am interested to see where the JS world leans next when it comes to task automation. When I started using JS, I saw people using Make or tools from other language ecosystems to build projects. Grunt and Gulp really changed that and moved folks towards using the node ecosystem for task automation. Now the community seems to have settled on webpack for build related tasks, and npm scripts for everything else. I’m all good with webpack for builds, it’s complicated but powerful. But npm scripts are frustrating as a task automation solution. There’s no ability to comment, and long operations in nested JSON strings are a bit unwieldy. I’d love to see some standardization on something more like make that allows comments and powerful task composition.

Computer Science In JS

Forde’s Tenth Rule, or, “How I Learned to Stop Worrying and ❤️ the State Machine” | Reginald Braithwaite - A really nice explanation of state machines, and how they can be used in JavaScript development. The article is written at a very conceptual level and really explains the topic well. I was thinking afterwards about how the state machine concepts reminded me of Redux, and fortunately I found another link that expanded on that connection…

How to model the behavior of Redux apps using statecharts | Luca Matteis - I hadn’t heard of statecharts before reading this, but they look like a really interesting tool for talking about large systems. I’d be really curious to play with the redux middleware for automatically generating them and see if the artifacts it produces are useful.

Non-Tech

Am I Going Blind? | Frank Bruni - This is absurdly well written, and it hit me personally since the idea of going blind has always been a personal fear of mine. I think everybody has their one bad thing that they irrationally fear will happen to them and this is mine. The article was good perspective in mutliple ways, reminding me how blessed I am and how resilient humans (and our bodies) are. (Via Daring Fireball)

Weekly Links: February 23rd

Thu, 22 Feb 2018 20:00:00 -0400

JavaScript And The Web

Feature watch: ECMAScript 2019 | 2ality - Not a lot to see here yet, but Axel Rauschmayer has set up a page for tracking the status of the next version of ECMAScript as features advance to stage 4 and become confirmed for the next version of the spec. No features have been confirmed yet, but worth a bookmark. He also has links to his detailed articles about the various candidate features if you’re interested in seeing where JavaScript is headed next.

The JavaScript Supply Chain Paradox: SRI, CSP and Trust in Third Party Libraries | Troy Hunt - This is a great reminder on why including scripts from 3rd party domains is a risky business, and how to do it securely.

Computer Science

After my post on Computer Science this week, I thought I’d share 2 CS-y links I came across this week.

How to think in graphs: an illustrative introduction to Graph Theory and its applications | Vardan Grigoryan - This is a nice “plain language” explanation of basic graph theory. It’s a bit long, but it’s a nice thing to look at for the folks who told me this week they want to know more about data structures.

Learning JavaScript Design Patterns | Addy Osmani - One of the better “CS for front end” resources I’m aware of is Addy Osmani’s book on JavaScript design patterns. Like a lot of things in JS land, parts of it haven’t aged well now that it’s a 6 year old book (it’s very focused on jQuery and DOM manipulation in many of its examples), but since it’s centered around more timeless design patterns, I think it holds up well.

Computer Science for Front End Developers?

Mon, 19 Feb 2018 20:00:00 -0400

I am primarily a front end web developer, and I have a Computer Science degree. In my experience, this is neither rare nor particularly common. Folks seem to get into front end dev because they’re trying to solve particular problems, and those problems exist whether you have a PhD in high performance computing or you’ve never written a line of code before. Front end web development is how you make code real to people, and it is the minimal requirement for making something that people can actually interact with in a web browser. As a result, a dizzying array of folks get into Front end dev, and you’ll find a broader set of experience levels than devs who work further “down the stack”. All of which means that many front end devs have never had any sort of “formal” Computer Science education.

Some CS degree holders turn up their nose at this, and it contributes to the reputation in some quarters of JavaScript and web dev not being “real programming”. Some self-taught devs see themselves as being liberated from the “irrelevant theory” of formal Computer Science and see no need for that type of background for Front End development. We all have the capacity to turn any sort of credential into a tribal mark. My own experience is that developers can function just fine on the front end without a formal computer science background, but that my own education has helped me in my day to day work, mostly in subtle and indirect ways.

I’ve been thinking for a while about what Computer Science topics are actually useful for front end developers. It’s not obvious to me where to draw the lines, because you’ll never directly encounter many of the concepts that are covered in a typical Computer Science curriculum while making web applications. But I still have an intuition that my background in these subjects is helping me in my day to day. This weekend I decided to ask Twitter about it.

Pondering what a blog series on “Computer Science for Front End Devs” would look like.

Thinking this would be a high level of “what you missed if you don’t have a CS degree”, focusing on a subset relevant to FE devs.

Would love input. What would you expect to be covered?
— Ben McCormick (@_benmccormick) February 18, 2018

I got a high degree of interest from that tweet, and so decided to keep digging into the idea. I did some research into the typical structure of a modern Computer Science curriculum, and came up with the lists below. Depending on the school, certain topics may be elevated or diminished, grouped or split, but the general shape tends to look the same for most schools I’ve looked at.

Intro class

Generally there is an intro class or 2 that can be skipped by students with previous experience that covers core programming concepts, usually with an Object-oriented philosophical tilt.

Programming basics
- Variables
- Basic Data Structures
- Control Structures/Recursion/Iteration
- Object Oriented Programming principles

Core Curriculum

There are also often 3-5 other “core” classes that all CS majors must take. They generally break down something like this:

Data Structures and Algorithms
- Big O analysis
- Common Data Structures - Linked Lists, Stacks, Queues Trees, Heaps, Graphs, etc
- Common Algorithms - Searches, Sorts, Graph algorithms, etc
Computer Systems
- Start at the hardware level
- Assembly
- Compilers
- Memory Models
- Concurrency
- Operating Systems
- Processes
- File Systems
- Networking
Mathematical Foundations
- Proofs and logic
- Formal lanugages/Context Free Grammars/Turing Machines
- NP-Completeness
- Probability Theory
- Bayes Theorem
- Hashing
Software Engineering (not always core)
- Architecture of larger programs
- Balancing risk/quality/cost
- Design Patterns
- Designing for security/robustness/scalability

Potential Specialties

Finally, programs usually have a selection of electives that students can take to dive deeper into different specialty areas. These electives usually mirror the research interests of the faculty of the school but may include some of the following:

Systems Design
Human-Computer Interaction
Computer Networking
Artificial Intelligence and Machine Learning
Databases
Computer Vision

Computer Science for the web

So how much of this is relevant to your average front end developer? Some of it is more tightly tied to web programming than others. Most of the basics will be picked up through osmosis for a typical front end dev, but a rigorous look at concepts like object-oriented programming and recursion might be beneficial. While most can usually get by with simple data structures, a background in common “intermediate” data structures like stacks, trees, and graphs can be quite useful in day to day web dev. Software Engineering principles are as obviously useful when building web sites as they are when you’re building firmware or operating systems. On the other hand, Computer Systems and Mathematical foundation concepts are usually not directly applicable in front end work. In my own experience, they serve as useful background for understanding the abstractions that my day to day work is built on. It’s good to have an intuition as to how browsers work, what is happening when we send data to a file, or what “text encoding” means. And you may end up in a crazy situation where you need to understand the code behind Babel or V8, and knowledge of compilers becomes a real part of your job.

I’m trying to figure out the best way to teach some of this stuff to somebody who hasn’t and is never going to spend 4 years taking these classes. So I’d love help from anyone reading:

If you have a CS degree and write front end code, how has it helped you?
If you were going to try and share part of that education with somebody with different experiences, which parts would you say are important? What resources would you recommend?
If you don’t have a CS degree, do you feel like you’re missing out on anything? What useful-seeming concepts have you heard about but found it hard to find resources on?

If you have any answers for those questions I’d love to hear from y’all on Twitter or email.

Weekly Links: February 16th

Thu, 15 Feb 2018 20:00:00 -0400

JavaScript And The Web

Promises are not neutral enough | André Staltz - This is a pretty good dissection of the downsides of Promises as JavaScript’s default async primitive. I’m glad that Promises have replaced callbacks as the default async method for most JavaScript code, but it does have the downside of making it harder for anything better to come along.

Direction Aware Hover Effects | CSS-Tricks - As a relatively pedestrian CSS user, I always am fascinated by the cool things people can do with it. These demos were no exception. (Hat tip to Ponyfoo for this one)

Tech Tools

Slack is the opposite of organizational memory | Abe Winter - This is a fun rant that makes some good points. I have a love/hate relationship with Slack. It’s a fun tool that I enjoy using, but I’m not at all convinced that it’s really making me more productive. I definitely feel the pain of normalizing crazy short reply times, even though I thrive on that on some ways and love the dopamine hit of being instantly helpful to somebody. Slack makes it easy to mistake noise for productivity, and I definitely agree that it is more ephemeral than it pretends to be. But for all that… I’m not sure there’s anything better out there¹, especially for teams that work partially or completely remote.

Cryptocurrency Craziness

Advertising Screens Hacked To Mine BitCoin | Terence Eden - This is just nuts. Bitcoin seems to be enabling a whole new malware economy.

Salon’s Monero mining project might be crazy like a fox | TechCrunch - At least some folks are giving you an ad-free malware experience though.

The truth of this statement of course is going to depend on what exactly you’re hoping Slack will do for you. I consider Slack to be a communication tool and not a project management tool. The linked article seems to conflate the two, which is maybe part of the problem here. ↩︎

Book Review: Clean Architecture

Tue, 13 Feb 2018 20:00:00 -0400

Clean Architecture by Robert “Uncle Bob” Martin was one of the better books on Software Development I’ve read in recent years. It did exactly what I want from a software book in 2018: it tackled an important high level topic with a great amount of depth in a way that helped me as a reader to develop insight and perspective on the topic. These days blog posts and other online sources cover most of my “how to” needs, as well as the simple “what is X and why should I care about it” questions that I used to go to books for. But there is still no substitute for books when you want an in depth understanding of an important broad topic. For this book, that topic is software architecture. The book tackles it by starting with some history, and then starting with a micro view before expanding out its gaze. After giving a broad overview of the different software paradigms, Martin walks readers through 3 levels of architecture principles: class-level design principles, component-level component principles and system-level architecture principles. At each level he shows the similarities between the lower level principles, as well as the unique concerns at each scale. Then he wraps it up with a case study example that shows the principles discussed throughout the book in action.

I found Clean Architecture to be a great reminder of concepts I learned in school¹ that I didn’t properly appreciate at the time due to a lack of practical experience. It brought home to me that while there are some architectural things I do pretty well right now, like keeping code separated enough that it only has one reason to change, there are other things that I don’t do well a lot of the time. For me the biggest realization was how much grief I’ve caused myself the last few years from failing to keep high level policies from depending on low level details. The book gave me a few new lenses to examine my decisions with, and definitely inspired me to spend more time taking a step back to see the big picture as I’m adding new features.

For me, that sense of perspective was the biggest win, but if you’re looking for detailed “how tos”, there’s a decent amount of that as well. Martin goes into details about how to arrange and refactor classes to create maintainable systems that are easy to change, as well as identifying patterns to look for that may be a sign of problems to come. He goes pretty deep into the weeds with examples and class diagrams as he discusses creating proper system boundaries and pulls it all together with a nice case study example at the end.

While I did find Clean Architecture extremely helpful overall, I do have some nitpicks. The biggest for me personally is that while the book mentions dynamic languages like Ruby and Python a few times, it mostly does so in passing to say that the principles mention still apply, but may look different. Since this is usually not expanded upon, and all of the examples given seem to assume static languages that provide abstract interfaces to classes, it is sometimes hard to see how the books principles should be applied to the dynamic languages that I use daily. I would love to have seen more detail there. Also, the final chapter of the book “The Missing Chapter” was a bit of an oddity. It was written by a different author², framed issues that had already been covered differently, and came to some different conclusions than the rest of the book. It was almost a counterpoint to the other parts of the book, which could have been interesting, but lacked an introduction or context to help explain its purpose. Overall it left me a bit confused and ended the book on a discordant note rather than bringing the ideas together and focusing me on the important ideas.

I can recommend Clean Architecture for anyone who wants to get better at thinking about the big picture when it comes to software systems, particularly if you haven’t already been exposed to any of the theory behind designing maintainable systems. If you’re usually working in static languages on larger projects, I would double that recommendation, as you’ll be right in the book’s wheelhouse. For those like me working with dynamic languages on web projects, it may be a bit more hit or miss, but it still is a great way to broaden your horizons and gain some new ways of thinking about the tradeoffs our work entails.

I was a Computer Science major, but if you weren’t exposed to theories of software architecture through a curriculum ↩︎
This was not obvious to me on first reading, I only went back and noticed the acknowledgement on the chapter page when I was confused by a 3rd person reference to the author. ↩︎

Weekly Links: February 9th

Thu, 08 Feb 2018 20:00:00 -0400

JavaScript And The Web

CodeMirror - CodeMirror is a JavaScript library for building text/code editors in the browser. It isn’t new, but I’ve had the pleasure of building something moderately complex with it over the past week, and thought it deserved some promotion. It makes building an online editor that does shiny “text editor” things like linting, find and replace, syntax highlighting and autocomplete a very manageable endeavor. It’s highly extendable and is a nice example of “easy things should be easy and hard things should be possible”, with simple addons for basic stuff, and less simple extension points for custom work. If you have need of an online text editor, make sure to check it out!

Googlebot’s Javascript random() function is deterministic - Tom Anthony - A really interesting peek at a small part of how Googlebot actually functions when rendering and indexing the web.

Non-Tech

Sleep strategies for a racing mind – Pamela Fox - I"ve never had the easiest time falling asleep at night, and it’s taken me a long time to figure out the best ways to manage that. This article lays out a bunch of strategies for coping, many of which are the same things I’ve had to learn the hard way. If you struggle to sleep, definitely worth a read.

Finally I’ll leave you with my personal pick for “tweet of the week”

If at first you don’t succeed... #science pic.twitter.com/2TGojKDcr3
— TwistedDoodles (@twisteddoodles) February 6, 2018

React Architecture Confessions

Tue, 06 Feb 2018 20:00:00 -0400

I’ve been reading through Clean Architecture by Robert “Uncle Bob” Martin this month¹, and while it is great it has had the unpleasant effect of reminding me of all of the mistakes I’ve made over the past few years while learning and using React. I’ve absolutely loved using React at work, and it has made so many things easier. But it’s a very unopinionated tool when it comes to structuring applications, and I made a lot of mistakes as I was using it to build some small side projects, and then transitioned a major production app to React from Backbone/Marionette. These mistakes are all architectural; how I broke out components, communicated between components, and managed data and dependencies within components.

I thought it would be useful to lay out the architecture mistakes I made as I was learning React, both as a solid reflection exercise for myself, and a warning to those of you who are just starting down the “component based UI” road.

My Mistakes

1. Including API calls directly in components

What I did wrong: This is an embarrassing one, but essentially in several places I had components directly fetching data from the server in their constructors or mount calls, and using that information to determine what to render.

Why I did it: This was to some extent a lack of planning. I had all my core “business objects” modeled in some sort of separate object oriented way (either as Backbone or MobX objects) and was able to coordinate syncing them to the server outside of the component layer. But some data in the system hadn’t been modeled in that way, and when I encountered a need for those components (such as some types of system configurations) instead of stepping back and coming up with a design for modeling those configurations on the front end, I instead sometimes fell back to an ad-hoc API call. This was also sometimes a result of lazily copying legacy code that had bad patterns while converting components to React, and other developers copying bad patterns when trying to figure out the new React code.

What I should have done instead: This could be a whole series of blog posts in itself, but essentially I should have been observing a few principles that would have set off alarm bells.

Components in a system of any size shouldn’t know about the server. UI Components are about as “low level detail” as it gets in a UI architecture, and should be passed the data they need from a higher level portion of the system.
If the front end code needs data from the server, it is worth the time to properly model that data in terms of whatever data layer the front end is using.

2. Breaking the “props as the component interface” abstraction

What I did wrong: One of the best things about React is the clear interfaces components provide with their list of props. If properly documented with PropTypes or a type system like Flow or Typescript, it’s easy to look at any React component and tell what data it expects to receive, and therefore how other code is expected to interact with it. 99% of the time, good React components act as a function of (props, state) => UI. It is however, possible to get access to a component instance, and call functions on them. There are rare situations where this is the right way to do things, for instance focusing an input.

When I first started creating React components though, I overused that pattern, and accessed the interface directly to access or change its internal state. I’d have components like this:

class SpecialInput extends React.Component {
  constructor(props) {
    super(props);
    this.state = {
      inputValue: 'abc'
    }
  }
  getValue() {
    return this.state.inputValue;
  }
  setValue(inputValue) {
    this.setState({inputValue});
  }
  render() {
    return <input
      className='fancy-input'
      defaultValue={this.state.inputValue}
      onChange={inputValue => this.setState({inputValue})}
    />;
  }
}

that I would access from parent container like this:


class InputUser

  methodThatGotCalledOnSomeEvent() {
    doSomethingWith(this.input.getValue())
  }

  render() {
    return (<div>
      <SpecialInput ref={el => this.input = el}/>
    </div>);
  }

Why I did it: Early on I was using this pattern in a way that copied other frameworks I was familiar with. A parent component would need the state of it’s child component, so it would maintain a reference to the child and query the child about the state. When I converted Backbone components to React, sometimes I was basically just copy and pasting existing code, and making it work.

What I should have done instead: React doesn’t really encourage parents pulling state from their children. Instead, if there is state that needs to be shared between components, it should be moved up to the parent component and then the state and an updater function should be passed down to the child component. So my example above would look like this:

class SpecialInput extends React.Component {
  render() {
    return <input
      className='fancy-input'
      defaultValue={this.props.value}
      onChange={event => this.props.updateValue(event.target.value)}
    />;
  }
}

that I would access from parent container like this:


class InputUser extends React.Component {
  constructor(props) {
   super(props);
    this.state = {
      inputValue: 'abc'
    }
  }

  methodThatGotCalledOnSomeEvent() {
    doSomethingWith(this.state.inputValue)
  }

  render() {
    return <div>
        <SpecialInput
             value={this.state.inputValue}
             updateValue={inputValue => this.setState({inputValue})}
          />
    </div>

  }
}

3. Interacting with 3rd party components directly

What I did wrong: My first year and a half using React, when I was using 3rd party UI components like react-fixed-data-table I would always use them directly in my components, and take full advantage of their APIs.

Why I did it: Because every React library in the world has documentation showing how to use its components directly in code for a variety of different use cases, and nobody ever tells you to put an abstraction around their code

What I should have done instead: I should have wrapped the 3rd party libraries in one or more wrapper components. This has a few advantages:

When we’re using a UI component like a table in a new place in our App, if I’ve created a wrapper component that has presets for most common settings, I only need to configure the things that are meaningful to this new use case in my app
Targeted wrapper components like a SortableTable component or a DropdownWithAutoComplete component are easy to understand to a new reader of the code. Instead of digging through configuration props, they can read the component name and understand the purpose of that component then focus on the data being passed to it
When making updates to a library it is much easier to test that the changes are safe if the uses of the library are limited to a few focused components, rather than being spread across a whole application
Similarly, if we need to switch to a new library, that is much easier with a limited footprint
It can be easier to create reliable automatic tests using a wrapper component, since you have more control of things like how mocking works and when the component interface changes than you do for 3rd party code

4. Importing other parts of the system directly into display components

What I did wrong: The main application I work on uses a messaging system to allow communication between different parts of the system. It’s a legacy piece from when it existed as a Backbone application, and isn’t a very React-y way of doing things, but that isn’t really a problem. This situation would be the same for a logging component or a data store. What isn’t great is that when I need to use the messaging system, I always import it directly into the component file.

Why I did it: Direct imports are convenient, the messaging system is a core part of my system, and it seemed like a normal way of accessing it. I also wasn’t aware of any better way to do it.

What I should have done instead: The disadvantage of direct imports is that they’re harder to test. If I had passed my messaging system as a prop to whatever component needed it, it would be much easier to test that it was called in the way I expected, without having to mock out the whole messaging system library. I also could abstract away the whole power of the messaging system, and only provide the specific functionality needed by a component. When I first began working on this system though, I didn’t know a practical way to do this. The messenger system was often used by deeply nested components, and it seemed bad to pass it down a whole component tree.

The answer is to use the Provider pattern. The provider pattern is a way to use React’s context API to provide top level properties from a high level parent component to deeply nested child components. It is a perfect pattern for providing important high level pieces of your application to low level components without tightly coupling them, while maintaining testability.²

5. Changing too many things at once

What I did wrong: When my team began converting our application from Backbone to React, it was tempting to try and change many things at once. All the React examples you see online are using Babel/ES6, Webpack, Redux, React Router and tons of other shiny but complicated things. Trying to implement it all in an existing project³ can be overwhelming and lead to lava layers. We actually started out with good discipline and converted files to use ES6 first, then added Webpack, then slowly started to convert UI components to React. Unfortunately as we continued to convert React, we began replacing our Backbone data models with MobX, and began using CSS Modules in some places. When our team size was reduced last year, we were left in a situation where the same functions were performed in different ways throughout the codebase.

Why I did it: It’s easy to see things that could be better in a web application and immediately want to fix them! Especially if parts of that change seem easier than other changes you’re working on

What I should have done instead: I wrote about this at length last month, but there really is no substitute for taking things one at a time. While it’s tempting to think that it will be easier to convert to React if you’re using other “react-y” tools and libraries, your code will ultimately be more maintainable if it is consistent throughout an application. The only reliable way to do that is to make changes completely before you start making other changes. It isn’t sexy, and it will leave you at least a little ways off the bleeding edge, but it’s the way to build large, reliable systems and keep them up to date.

At the time of writing this, it was super on sale on Amazon ↩︎
Note that the Context API that the provider pattern is based on is changing, so I’d isolating out any provider code you write into a reusable component so that it is easy to update later ↩︎
Or even a new project for that matter! ↩︎

Weekly Links: February 3rd

Fri, 02 Feb 2018 20:00:00 -0400

JavaScript And The Web

ECMAScript 2018 | 2ality - The spec for the 2018 version of ECMAScript has been finalized and this is a good roundup of the new features and syntax available. I’m personally excited for rest/spread properties to be an official part of the language.

PWAs are coming to iOS 11.3: Cupertino, we have a problem | Maximiliano Firtman - Interesting look at what is and isn’t available in the newest version of Safari when it comes to PWAs and offline support

Tech and Business

1 Trillion of Real Estate is on the move | Phil Levin - Almost everybody seems to agree that self driving cars are going to happen. It’s only a question of when. But the other question is what the second-level effects of self driving cars will be. Those who can correctly predict the answer to those two questions are going to have opportunities to make money and create interesting businesses. This is an interesting take on how self-driving cars might affect real estate.

Weekly Links: January 19th

Thu, 18 Jan 2018 20:00:00 -0400

JavaScript And The Web

The Brutal Lifecycle of JavaScript Frameworks | Stack Overflow Blog - This was an interesting analysis on how JavaScript frameworks have risen and fallen. Among the reasons that it’s interesting is that the data appears to be a bit of a Rorschach test. The post itself is very negative about the pace of change in the JavaScript framework world. But I saw a lot of twitter takes like this one:

This “new framework every 6-months” FUD is comical. The crux is centered on 4 JS frameworks released over the last decade.

4. In 10 years.

It’s great for my business, but makes me chuckle.

It’s not really moving that fast y’all. https://t.co/BiH7UK0M9V
— Joel ⛈ (@jhooks) January 12, 2018

Personally I lean more toward the side that there’s nothing wrong here. The change in frameworks has come during a large change in use cases and technology used on the web. Think PWAs verse server-side generated sites, and the large amount of new browser APIs and JavaScript language features. These things have naturally brought new best practices which have been encoded in frameworks. While it is true that a jQuery based site from 2009 might be difficult to maintain through today, that has less to do with the lack of support for the technology than it does with the fundamental nature of web development. It would be much more difficult to build and scale a web application like Slack using only jQuery and other 2009 technologies.
Which is why it was surpassed for web application development first by a wave of frameworks that allowed you to model domain concepts in an object oriented way and tie that to a UI (Backbone, Angular1, Knockout), and then by a second wave that allowed you to do that with more declarative component-based UIs (React, Ember, Angular2+, Vue). Those evolutions have come to match a real need.

Can (a ==1 && a== 2 && a==3) ever evaluate to true? | Stack Overflow - This was a fun little question. I don’t think I like it as an interview question for JavaScript, since it borders on trivia, and requires in-depth knowledge of the “bad parts” of JavaScript like == and with or global variable behavior that many new developers are just taught to avoid. But it’s definitely a fun trivia question. And it may even be a good interview question in Python, where you can do it using more commonly accepted parts of the language.

class EqualToEverything:
  def __eq__(self, x): return True
a = EqualToEverything()
print(a == 1 and a == 2 and a ==3) #prints True

Making WebAssembly even faster: Firefox’s new streaming and tiering compiler | Lin Clark - Web Assembly + Rust is quickly moving to the top of my “I’ve got to learn more about that” list.

Keep webpack Fast: A Field Guide for Better Build Performance | Slack Engineering - This is a super helpful and detailed list of performance tips and tricks for Webpack builds, which unfortunately are easy to make slow and challenging to make fast for large projects. Even after having my own fights with webpack last year, I learned a few things from this, and look forward to experimenting with them.

Tech and Business

Birdcage Liners | Joel Spoelsky - Joel’s first post in a while is a great piece on the problems with social media, as well as tech folks' responsibility to think through the choices they’re making, including the secondary effects. What type of future are we making?

My Joke Cryptocurrency Hit $2 Billion and Something Is Very Wrong - I promise I snagged this link to post before bitcoin took a dip this week, but I don’t think it’s properly appreciated in many places just how crazy the cryptocurrency fad is becoming. Meme-based joke currencies are being valued at billions of dollars, and there are many other similar newborn currencies with similar valuations out there. I’m not going to try to give anyone investment advise, but it seems clear for now that at minimum these “currencies” are completely unusable for their theoretical purpose as currency given the price swings.

Weekly Links: January 12th

Thu, 11 Jan 2018 20:00:00 -0400

JavaScript And The Web

Manage Application State with Mobx-state-tree| Michael Weststrate on Egghead - Mobx-state-tree is an opinionated state library built on top of the unopinionated Mobx library. This is a free course explaining why it exists and how to use it. I wrote about Mobx last year and am still enjoying using it a year later. I’m excited to dive into Mobx-state-tree and hopefully sharpen my Mobx skills in the process.

Programming Tools And Practices

The State of Atom’s Performance | Atom Blog - Atom continues to (slowly) become snappier. I’m glad to see how organized they are on this. It’s really hard to take something that was built with a slower architecture fast while it’s being used by tens of thousands of people every day. They’re making great progress.

Clean Architecture | Robert C Martin - I’ll probably be posting more on this soon, but I’ve been reading the new book on architecture by “Uncle Bob” Martin, and it’s fantastic. It breaks down big architecture and program design concepts like the Single Responsibility Principle and Dependency Inversion, and explains them well individually, but also shows how they fit together when designing a large system. I’ll write a full review when I finish the book, but for now just wanted to share that it is worth picking up.

Non-Tech

North Carolina Is Ordered To Redraw Its Congressional Map | NY Times - A little bit of local news here ¹, but it is still important for everyone in the USA. Redistricting is the type of problem where a simple rules based algorithm could be helpful. Nothing opaque or deep-learning-y, just have a machine apply a published set of rules and work with the results.

I’m a proud resident of Durham, North Carolina ↩︎

Tips For Improving a Large Code Base With A Small Team

Sat, 06 Jan 2018 20:00:00 -0400

Over the last year I’ve been acting as the lone JavaScript resource working on a codebase with around 90k lines of JavaScript code. That has left me spending the vast majority of my work time trying to knock out features and deal with bugs across the large code surface area. But I’ve also wanted to do my best to continue to improve code quality with limited resources. Here are a few things I’ve learned about trying to improve code quality across a large project with limited resources.

1. You need to accept not being on the bleeding edge

Let’s be real, in the JavaScript world the only way to always be on the bleeding edge is to never maintain an application for more than a few weeks. And even in other contexts, it often is much less feasible than hype bubbles at conferences or sites like Hacker News might make it seem. Most companies that are shipping software that users actually pay for are using quite a lot of very old technology quite extensively. That’s not to say there’s no place for new and shiny things, but there will always be limits to how new and shiny things get. When you’re low on people resources for the size of your project, those limits get a lot more restrictive.

2. It’s important to identify your pain points.

There are 3 main type of value that developers can add by changing a software system. They can write new features, which give users new capabilities. They can improve some cross-feature attributes of the system like loading speed, accessibility, design, logging or documentation. Or they can make architecture improvements that make it easier to make other changes. Generally engineers don’t have to argue to add features, because the business decision makers will be calling for them constantly. Cross-feature things like performance, design or logging sometimes come from business decision makers, but often only when things have become problematic, and then the conversations around how long it will take to achieve improvements can be tense ones. Architecture concerns pretty much never come from business folks, because they usually don’t see the effects of a bad architecture until it is too late and shipping simple features has become a disastrously slow process¹. It’s part of a developer’s job to push for the latter 2 type of changes when they’re needed. But with limited resources, needed is often hard to quantify.

That’s why it is important to know what is actually causing you pain, or will soon be causing pain. Is your site getting slower and slower? Is your CSS a giant tangled mess that creates bugs with every change²? Make a list, and then prioritize it by importance and expected effort.

3. Take things one at a time

Once you have a list of things that you want to fix, it can be tempting to try to fit those fixes in all over the places. You’re creating a new page, might as well write it in Vue instead of Angular. You’re changing some styles, maybe you can use a CSS in JS library. But the absolute worst thing you can do with limited resources is try to improve too many things at once. You’ll end up with a codebase with many different ways to do everything, so that every change requires a full context reset, and maintenance means dealing with multiple sets of bugs and compatibility issues for each duplicated approach. That’s why prioritization is critical. With limited bandwidth, you need to know what one thing will be the best use of your time.

4. Expand from small footholds, don’t boil the ocean

A few truths to note at this point:

Changing large systems is hard. They have a million different subtle assumptions weaved into their code, and making seemingly the same change in different places can often produce subtly different results.
Practice is how you learn to use a tool or technique optimally. Unless you’ve already used a technology extensively in a different context, you will never be more blind to it’s downsides than at the start of adding it to a project. As you begin using it, you’ll learn more about it and how to use it better in your system.
Well designed systems have good interface points. Interface points are areas where one part of the system doesn’t need to care about the details of how other parts of the system are implemented, they just need to know the interface for interacting with that part of the system. For example, when using React, parent components don’t need to know the details of how a child component work, just what props they take. Similarly when writing HTMl, you don’t need to know the details of how CSS will work for layout, you just need to expose meaningful class names that CSS can use. A website doesn’t need to know what language a web server is written in to make HTML requests to it. These are examples of interface points, and while well designed systems will have more of them and fewer “breaches” of the interface lines, all systems have them to some extent.

The implication of these truths is that there is value in starting small and expanding. If we can pick a small section of the application that is cordoned off from the rest of the code in a meaningful way, we can create a foothold for the new technology. This serves as a learning opportunity, an example, and a starting place. If it goes badly, small changes are easy to revert or fix. If it works well, we can look for more similar interface boundaries, or expand out to convert some of the code that uses the new code.

When I converted a large Backbone app to use React, I took advantage of 2 different interface boundaries. I started by converting small “widget” components to be written in React, and came up with a method of calling these widgets from within Backbone code. Once I had enough of these building block widgets written in React, I started writing new pages in React. Then finally I would convert subsections of Backbone driven pages to React, starting from the bottom up and converting the top level layouts of the page last. I was using 2 interface boundaries: components and pages. That made sense for Backbone to React, where I was essentially converting one type of component tree to another. But the lines may be different for other changes, and you’ll need to use your judgement.

5. Automate what you can

Not every change needs to be hard and slow. Sometimes we can take advantage of automation. Tools like Prettier and ESLint make it easy to automate code style and correctness fixes. Tools like React often provide codemods to help make it easier to update to a new version of the tool or use best practices. Unit tests can help you prevent regression bugs and are easy to automate with a CI server or pre-commit hooks.

If you’re dealing with problems that are more specific to your codebase, it may make sense to invest time into writing your own automation scripts, whether that means hacking together a crazy bash/python script to do a massive find and replace, or using a tool like Ansible to automate some previously manual developer processes. It made a big difference in our company when our QA developer put in the work to automate the creation of consistent local environments.

6. Count the cost of maintenance

With a small team, some improvements may simply not be worth it over time. Genuinely valuable things like test coverage, JSDoc style automated documentation, old browser support, add overhead to future changes as well as any up front cost to implement. These are the type of things that advocates tend to moralize about, and say that it’s the developers job to advocate for them. And that is true as I alluded to earlier. But when faced with limited resources, it’s important to prioritize, and understanding the cost of maintaining certain improvements is part of that.

7. Distinguish between craftsmanship and business needs

Along the same lines, as developers we’re often encouraged to think of ourselves as craftsmen. Think of Steve Jobs talking about using good wood for the back of a cabinet or the aforementioned moralizing about code and process quality. There are great things about this, since it is a developers job to look after the quality and craftsmanship of their product. But it also is good to align that craftsmanship with business needs and create something that is useful rather than merely theoretically beautiful. So there is a place for making changes to improve the “purity” of code, but when investing limited resources, it’s even more important than usual to balance that with business needs.

I’ve gotten to work with plenty of non-devs who respect the need for good architecture. They still aren’t going to be able to identify when changes need to happen. ↩︎
Little known fact: CSS actually stands for “Giant Tangled Mess that creates bugs with every change” ↩︎

Weekly Links: January 5th

Thu, 04 Jan 2018 20:00:00 -0400

Working in The Tech Industry

To Serve Man With Software | Jeff Atwood - This is a great review of the ethical questions that software developer’s face in the modern world. Is your work actually serving anyone who isn’t invested in your employer?

Basecamp doesn’t employ anyone in San Francisco, but now we pay everyone as though all did | Basecamp - On a similar vein, this is a tech employer holding themselves to a higher standard in terms of benefiting their employees¹. Also this quote is a fantastic summary of the type of software companies I want to work for:

Especially in software, which is a profitable business when run with restraint and sold to businesses.

There’s a glorification of the unicorn startup in the tech world, but you can go a long way with low costs and high paying customers (and it’s easier to “serve man with software” without outside pressure to grow like crazy)

Non-Tech (“Less-Tech”?)

Perceptions of Probability and Numbers | Zoni Nation - This is simply a collection of visualizations of a dataset pulled from Reddit of what probabilities people associate with common english phrases used to express uncertainty. I found it fascinating.

Stop Using Facebook and Start Using Your Browser | Foster Kamer - This is a call for people to get out of their social media bubble and internet like it’s 2005. Use bookmarks, type in URLs directly, basically go to sites directly for content instead of relying on social media algorithms to find stuff for you. I think this is easier said than done. Social networking sites are ingrained into our culture at this point. You don’t replace culture by condemning it, and telling everyone to go without but by creating something new or better². So what can we make as Internet users that provides something as easy or easier than the mindless serendipity of social media, but produces better outcomes?

Feel free to be cynical about this as a marketing ploy, but I’m all for companies marketing themselves by actually improving how they compensate their employees. ↩︎
If you’d like to read a whole book making that argument, I recommend Culture Making by Andy Crouch ↩︎

2017 Roundup

Fri, 29 Dec 2017 20:00:00 -0400

2017 was the 5th full year of this blog, and in many ways it was the most productive. I wrote one of the most popular things I’ve written on here, started the weekly links section (on hiatus for New Years, but will return next week), and wrote 21 full articles and 8 weekly links post, the most I’ve ever posted here in a year. Here are some of the highlights from my blog and development this year.

Most Read Posts Published in 2017

Ten Things A Serious JavaScript Developer Should Learn - It turns out that making lists of things is a good way to get traffic. I wonder if anybody else on the internet has figured this out?

In all seriousness, I think there’s a hunger out there for clearer roadmaps for folks who are early out in their front end development career. People are telling them to learn all the frameworks, all the tools and all the compile-to-JS languages, and everyone knows they can’t learn it all. So even an imperfect list that can serve as a starting point for figuring that out can get some traction. This article was my attempt to answer “what should a serious mid-level JavaScript developer have learned and be learning?”.
How To Follow The JavaScript Roadmap - This post was aimed at a similar audience to the “Serious JavaScript Developer” post, but aimed to help folks who want to follow changes in the JavaScript language keep up with new developments.
Orthogonality and CSS in JS - This article was an examination of the architecture tradeoffs of CSS in JS solutions, and also an examination of how people discuss “separations of concerns” in general. If you liked this one, keep your eyes open. I’m planning on writing more architecture level posts in 2018.

Code I wrote in 2017

Most of the code I wrote this year was again for my work at Windsor Circle. But I’ve continued to add new features to this blog, which is based on React and Gatsby, and the source code for that is available on Github.

Talks I gave in 2017

This was a big speaking year for me. I gave 3 different talks: The Vim Survival Guide at Open Source 101, Intro to Gatsby at a local meetup group and Saving Time with Jest at All Things Open. I’d be interested in giving any of those talks again next year, so if you’re looking for a speaker, feel free to reach out.

Languages, Libraries, and Technologies I started using in 2017

Mobx is a JavaScript data modeling library that serves as an alternative to Redux or React state for state management. I wrote about my first impressions at the beginning of the year, but I’ve continued to use it and love it since then.
Yarn was a nice upgrade over npm. My understanding is that npm has since stepped up and closed the gap here, but I’m continuing to use Yarn for now since it has taken over my projects and muscle memory.
When I upgraded this blog to Gatsby 1.0, I had the opportunity to learn about GraphQL. I’ve really enjoyed getting to play with it and love the GraphQL community’s dedication to providing a great developer experience.

Languages and Libraries I used (more or less) every day in 2017

JavaScript (ES6), Python, React, PostgreSQL, Backbone (sigh), lodash, MobX, Jest

Languages and Libraries I want to try in 2018

Victory, Rust, and Pandas/other python data analysis libraries [1]

Blogs I started following in 2017

Mark Erikson’s Dev blog

Blogs I read every post from in 2017

Rands In Repose, Stratechery, Daring Fireball, purposedworking.org, wynnnetherland.com

Podcasts I started listening to in 2017

Planet Money - A long running NPR podcast about money, the economy, and the unexpected ways they affect our everyday lives
The Allusionist - A great podcast on the crazy history behind different words in the English language.

Software I started using in 2017

Bear - I mentioned I was testing this out at the end of last year, but it has become my go-to notes application, and I would recommend it to anyone who wants a stylish notes app in the middle ground between Apple Notes and Evernote (and who uses Apple products).

Software I used (nearly) every day in 2017

MacOS/iOS, Google Search, Chrome/Safari/Firefox, GMail/MailPlane 3, Twitter/Tweetbot, Feedbin/Reeder, Instapaper, iMessage, Slack, Trello, 1Password, Atom/Vim, iTerm, Fish Shell, Bitbucket, Fantastical, Spotify, Bear, Annotate, Dash

Yes this is basically the same list as last year. ¯\_(ツ)_/¯ I mostly learn tech as I have need for it, and it was a very “nose to the grindstone” year focused on improving existing codebases this year ↩

Weekly Links: December 23rd 2017

Fri, 22 Dec 2017 20:00:00 -0400

JavaScript And The Web

73,939,133 / Truncatable Primes | Reg Braithwaite - A nice writeup about how brute force can be useful in solving a Project Euler style math problem.

Programming Tools and Practices

Jest 22: Refinements & Custom Runners | Jest Blog - This looks like a solid refinement release for Jest. One warning: If you’re running jest -o on precommit hooks as I described here, Jest 22 fails when run with 0 tests, so you need to specify the --passWithNoTests flag in order to let things go when you make a change that doesn’t affect any tests.

Introducing The Gatsby UX Research Program | Gatsby Blog - An unusual case of doing real UX research for an open source developer tool. Very cool stuff.

Non-Tech

Google Maps’s Moat - If you have any interest in product design, maps, or computer intelligence, this is the one article you need to read this week. Beautifully put together. It’s an in depth examination of the difference between Google and Apple maps, but it reads as a deeper exploration of how big data combined with computer learning and automation can have a huge impact on product design.

How to cheat at settlers by loading the dice | Mike Izbicki - This is just a fun application of statistics to my favorite board game.

Weekly Links: December 15th 2017

Thu, 14 Dec 2017 20:00:00 -0400

JavaScript And The Web

The State Of JavaScript 2017 - This annual survey of JavaScript developers always has lots of interesting insights. Since it is an internet survey, I view it less as a full snapshot of the industry, but more as a place to see what is new and trending in the JS world. This year Visual Studio Code, Prettier and Vue caught my eye as spiking in popularity.

The Glimmer Binary Experience | LinkedIn Engineering - I had the privilege of seeing Yehuda Katz, one of the Ember core developers, give a talk about Glimmer and bytecode at this year’s All Things Open conference. They’re doing some pretty serious software engineering in the Ember community right now. This write-up covers a lot of the same ground and is worth a read.

Programming Tools and Practices

Atom 1.23 and Visual Studio Code 1.19 were released this week, making this a big week for Electron based text editors. If you use either of them, you’ll get to enjoy a nice set of performance improvements when you upgrade.

Bear 1.4 - Speaking of software that I love to use getting updates, I’ve been using Bear as my primary notes app for the last year, and it is a delight. Getting autocomplete on tags this week solved it’s last real annoyance for me, and I would definitely recommend it to anyone interested in a simple notes app. It’s more powerful than Apple’s notes or text files, but cleaner and more elegant than Evernote. And even the paid level is very affordable.

Weekly Links: December 8th 2017

Thu, 07 Dec 2017 20:00:00 -0400

JavaScript And The Web

Introducing the React RFC Process | React Blog - React is now using a public process for proposing and designing new features, starting with a new context API.

Why we have banned default exports in Javascript and you should do the same | Krzysztof Kaczor - This lines up pretty well with my own experience when it comes to native JS modules. The named exports are pretty much always better than the default ones.

Advanced React Component Patterns | Egghead.io/Kent C. Dobbs - This is a new Egghead course on React Component patterns, and is the best “advanced” React resource I’ve seen. If you’re a React developer, you should either watch it this weekend while it is still free, or pay up to Egghead (it will be behind their subscriber wall starting Monday). He also released a beginner level React course this week that will remain free.

Non-Tech

The Pollyannish Assumption | Stratechery - Ben Thompson gives a pretty good breakdown here of the problems companies face moderating Internet scale content. Human beings capable of both greater heights and darker depths than we like to think.

Weekly Links: December 1st 2017

Thu, 30 Nov 2017 20:00:00 -0400

JavaScript And The Web

Functional pattern: flatMap | 2ality - This post isn’t new, but flatmap and flatten hit stage 3 in the TC39 process for adding new features to JavaScript this week¹. So this is a great time to read up on them.

Stage 3 for Array.prototype.flatMap and Array.prototype.flatten! Thank you so much @jspedant for driving this!https://t.co/9kLCKJcZEv
— Brian Terlson (@bterlson) November 28, 2017

Boiling the Ocean, Incrementally - How Stylo Brought Rust and Servo to Firefox | Bobby Holley - This is a great review of the technical achievement that Firefox 57 represents, and also gives some insight into where Mozilla is going from here. I’ve switched back to Firefox over the past 2 weeks and am pretty happy with it so far.

Programming Tools and Practices

Introducing AWS AppSync | AWS News Blog - AWS announced a bunch of new services this week, but I wanted to highlight AppSync, an Amazon GraphQL service. GraphQL seems to be gaining more and more momentum, and if you don’t know anything about it I’d suggest checking it out. I’ve gotten to play with it since rewriting my blog to Gatsby 1.0, and it is a nice way to work with data.

Non-Tech

This City Hall, brought to you by Amazon | The Seattle Times - I’m amazed by the incentives cities have been willing to offer Amazon to locate a headquarters in their city. I think the outrage in this column is a bit over the top, but it is a startling insight into the power large tech companies have over the cities where they’re located.

Doomsday Clock - #1 - If you have any interest in comic books DC’s Watchmen sequel has begun. It’s a story that crosses over with the mainstream DC Universe, and appears to be a rebuttal of the cynical vantage point of the original work. This is the first of 12 issues over the next year and it was a strong (but slow start). If you’re interested in Watchmen but don’t read a lot of comics, this is the continuation of a story that began with a big DC special and continued in a special Batman/Flash crossover

See this post for more info about staying informed about features that are being added to JavaScript. ↩︎

Implementing The Sieve Of Eratosthenes in JavaScript

Mon, 27 Nov 2017 20:00:00 -0400

As a way of keeping my math skills sharp, I’ve recently started to work through the Project Euler set of computational math problems. They ask you not to post your work publicly, so I won’t be posting about any of my specific solutions, but I did think it would be fun to share about a helper method I’ve been using.

For several of the Project Euler problems I’ve worked on, I needed to be able to generate a list of prime numbers. To do that, I borrowed a Python implementation of the Sieve Of Erathosthenes from this StackOverflow answer. Since I haven’t had much reason to use generators in JavaScript yet, I thought it would be fun to translate that algorithm to JavaScript and share that here.

Sieve of Eratosthenes

The Sieve of Erathosthenes is a very old[1] simple algorithm for identifying prime numbers. In its normal implementation it is a useful way of finding small primes up to a certain number. Beginning at 2, the algorithm iterates upward. For the current number, if the number has not been marked, we identify it as a prime, and then mark all multiples of that number up to our target number. So to find all primes up to 12, we would

start at 2, note 2 as prime, and mark 4, 6, 8, 10 and 12 as non-prime
move to 3, note 3 as prime, and mark 6, 9, 12 as non-prime
move to 4, see that it is marked, and skip it
move to 5, note 5 as prime and mark 10 as non-prime
move to 6, see that it is marked and skip it
move to 7 , note 7 as prime
see that 8–10 are marked and skip them
move to 11, note 11 as prime
move to 12, see that it is marked and skip it

You can see it graphically with this visualization from Wikipedia:

Python Implementation

The Python implementation that I used makes some modifications to the classic version of the algorithm that lets it generate an indefinite number of primes while being memory efficient

def gen_primes():
    """ Generate an infinite sequence of prime numbers."""
    marked_not_prime = {}
    value_to_check = 2
    while True:
        if value_to_check not in marked_not_prime:
            yield value_to_check
            marked_not_prime[value_to_check * value_to_check] = [value_to_check]
        else:
            for prime in marked_not_prime[value_to_check]:
                marked_not_prime.setdefault(prime + value_to_check, []).append(prime)
            del marked_not_prime[value_to_check]
        value_to_check += 1

This version of the algorithm keeps a running set of the relevant matches, and keeps only the ones it needs. We can translate it more or less directly to JavaScript like this:

function* generatePrimes() {
    const markedNotPrime = {};
    let valueToCheck = 2;
    while(true) {
        if (!(valueToCheck in markedNotPrime)) {
            yield valueToCheck
            markedNotPrime[valueToCheck**2] = [valueToCheck]
        } else {
            let primes =markedNotPrime[valueToCheck];
            primes.forEach(prime=> {
                let nextMultipleOfPrime = prime + valueToCheck;
                if (nextMultipleOfPrime in markedNotPrime) {
                    markedNotPrime[nextMultipleOfPrime].push(prime);
                } else {
                    markedNotPrime[nextMultipleOfPrime] = [prime];
                }
            })
            delete markedNotPrime[valueToCheck];
        }
        valueToCheck += 1
    }
}

Let’s break down what this is doing.

function* generatePrimes() {
  //...
}

generatePrimes is a JavaScript generator. Generators “generate” a series of values in a “just in time” fashion; they only calculate the next value when it is needed, making them an efficient solution for working with large or infinite series. You specify a generator function by adding the * to the function declaration. Generators then use the yield keyword rather than return to return their values, and can yield multiple times. After a generator yields, it pauses execution until it is asked for another value. For our generatePrimes function, we are operating inside an infinite loop and will yield continuously without stopping[2].

const markedNotPrime = {};
let valueToCheck = 2;
while(true) {
    //...
    valueToCheck += 1
}

We’re using a plain object to keep track of our marked numbers, and using valueToCheck to keep track of where we are in iterating through numbers. As noted, we will iterate it forever if the generator continues to be called.

if (!(valueToCheck in markedNotPrime)) {
    yield valueToCheck
    markedNotPrime[valueToCheck**2] = [valueToCheck]
} else {
  //...
}

As in the original algorithm, if we get to a number that isn’t marked, we know that it is a prime. So we yield it, which pauses execution. If the generator is queried again, when we resume execution we mark the prime squared as not-prime, and we do that by assigning it an array with the current prime value in it. Using arrays for markers will make more sense in a moment. For now its worth noting that we only need to mark prime-squared as non-prime since every multiple below that is the current prime multiplied by a number less than it, which means it will already be covered, as we will see below.

if (!(valueToCheck in markedNotPrime)) {
  //...
} else {
    let primes = markedNotPrime[valueToCheck];
    primes.forEach(prime => {
        let nextMultipleOfPrime = prime + valueToCheck;
        if (nextMultipleOfPrime in markedNotPrime) {
            markedNotPrime[nextMultipleOfPrime].push(prime);
        } else {
            markedNotPrime[nextMultipleOfPrime] = [prime];
        }
    })
    delete markedNotPrime[valueToCheck];
}

Here’s the clever part of this implementation. When we hit a number that is marked as not-prime, instead of just skipping it and moving on, we use some logic to keep our list of marked values small and memory efficient. This is also what allows us to get an infinite list of primes instead of settling for those under a particular value. What we want to do is keep a list of all the primes that are factors of each marked number. Then when we hit a number that is marked, we mark the next multiple for each prime (valueToCheck + prime), and remove the current value from the object. That keeps checks fairly cheap by eliminating the number of marked numbers, and allows us to only add values to the marked list as we need them.

We can then use this code to get the 1,000,001st prime like this:

let gen = generatePrimes();
for(var i=0; i< 1000000; i ++) {
    gen.next()
}
console.log(gen.next().value)

That executed on my laptop in ~20seconds, approximately the same as the python implementation in python2, a little slower than running it under python3. I was curious whether using a Map for our marked prime list would speed things up:

function* generatePrimes() {
    const markedNotPrimeMap = new Map();
    let valueToCheck = 2;
    while(true) {
        if (!(markedNotPrimeMap.has(valueToCheck))) {
            yield valueToCheck
            markedNotPrimeMap.set(valueToCheck**2, [valueToCheck])
        } else {
            let primes =markedNotPrimeMap.get(valueToCheck)
            primes.forEach(prime=> {
                let nextMultipleOfPrime = prime + valueToCheck;
                if (markedNotPrimeMap.has(nextMultipleOfPrime)) {
                    markedNotPrimeMap.get(nextMultipleOfPrime).push(prime);
                } else {
                    markedNotPrimeMap.set(nextMultipleOfPrime, [prime]);
                }
            })
            markedNotPrimeMap.delete(valueToCheck);
        }
        valueToCheck += 1
    }
}

This version of the code runs about 25% faster, in 15 seconds on my machine. Not bad.

More Resources

You can dig in more on the Sieve of Erosthenes on Wikipedia
Check out Project Euler for some fun computational math problems.

There are references as old as ~100 AD ↩
At least until we hit system boundaries for memory/number size/etc. ↩

Weekly Links: Thanksgiving 2017 edition

Sat, 25 Nov 2017 20:00:00 -0400

In honor of Thanksgiving this week, instead of a list of writing from around the web, this week I’ll be linking to some of my favorite lesser known software tools, apps and gear. These are things that let me get things done, and I’m thankful for that.

JavaScript Libraries

Gatsby - I’ve written and spoken about Gatsby before, but it keeps getting better as a tool for building React-based web sites and applications. This blog is written using it.

date-fns - date-fns is a nice modular lightweight alternative to moment.js for date management in JavaScript. Because JavaScript’s built in dates are weird and hard to work with, and not everybody needs all of the goodness moment provides.

MobX - It’s been overshadowed by Redux, but MobX is a fantastic state management library for JavaScript libraries. Since its an object oriented model that can be used in a variety of ways, it is easier to pull in incrementally

glamorous - My favorite CSS-in-JS library for React, it lets you write styles in a few different ways, which allows you to manage things in a very React-y way, without ever getting stuck with things you can’t do.

Apps

Fantastical - The best app I’ve ever used for OSX and iPhone

Mailplane - If you like GMail’s web app (obviously not to everyone’s taste) Mailplane is a great way to use it as a native app on OSX. I like it because it allows me to use both of my Gmail accounts side by side with their own notification settings and no risk of being lost in a sea of browser tabs.

Gear

Rickshaw Bagworks Commuter Bag - I’ve had the 2.0 version of this bag for the past 4 years, and it has been an amazingly versatile and durable tool for transporting my laptops, notebooks and more. Highly recommended.

Chromecast - I have a Chromecast and a Chromecast audio, and they’re an amazing cheap way to turn TVs and speakers into “Smart devices” that you can control from your phone without having to pay for “smart devices” that probably won’t be kept up to date the way you’d want.

Backbeat Fit - I’ve now ended up buying 4 pairs of these headphones for myself and others. They’re fantastic versatile headphones that work great for exercise headphones but are good enough quality to use for other purposes. I personally have trouble with ear-buds (they don’t stay in my ears), so the over-ear style of the fit is perfect for me.

Weekly Links: November 17th 2017

Thu, 16 Nov 2017 20:00:00 -0400

JavaScript And The Web

Currying In JS | 2ality - A nice round up of the different approaches for using currying (a functional programming concept) in JavaScript. My favorite line: “One of JavaScript’s best traits is how many different styles of programming it can accommodate." This is one of my favorite things about the language as well.

Thoughts on Redux and its similarities with OOP | Dhruv Rajvanshi - I always find it useful to understand when core concepts are shared across very different looking tools and paradigms

Programming Tools and Practices

Entering the Quantum Era—How Firefox got fast again and where it’s going to get faster | Lin Clark - I’ve been using Firefox 57 as my primary browser this week and so far it is very very nice. Other than a single Chrome-specific extension that I’m missing, and a few minor devtool annoyances, everything has worked great. And it is blazing fast.

Introducing Visual Studio Live Share | Microsoft and Code together in real time with Teletype for Atom | Github - In what appears to be a crazy coincidence, both Atom and Visual Studio/Visual Studio Code announced new tools for collaborating with other developers on code, using a google docs-like experience. Atom’s solution is available in beta today, and is a free WebRTC based peer-to-peer solution for editing code together, with the primary user sharing their active tab with anybody who has the uniqueID for your session. It also includes open-sourcing a library that would let other editors implement its document protocol. Microsoft’s solution is more ambitious, supporting both Visual Studio and Visual Studio code, and allowing users access to a whole shared workspace. It is just an announcement for now though, and it’s not clear if it is a peer to peer or centralized solution, nor whether it will have a cost associated with it. Regardless, both solutions look amazingly cool.

As one of the three engineers that built Teletype, I share your curiosity! 🙈
— Jason Rudolph (@jasonrudolph) November 15, 2017

Managing Humans | Michael Lopp (AKA Rands) - This book was a great roundup of some of the best writings from one of my favorite blogs Rands in Repose. Michael Lopp has been a manager and executive at Netscape, Apple, Palantir, Pinterest and now Slack. He’s seen a lot. While the book is aimed first at managers, it’s helpful for anyone who is interested in understanding how engineering teams work, and how to lead or work in one better.

Non-Tech

On the Table, the Brain Appeared Normal | The New York Times - Powerful reading about the dangers of football, a sport that I still love watching. More context here if you’re confused after reading the NYT story.

Security Breach and Spilled Secrets Have Shaken the N.S.A. to Its Core | The New York Times - Interesting to see such mainstream questioning of the NSA’s habit of hording zero-day exploits. Don’t read this one if you want to maintain any naive good feelings about US cybersecurity.

A Quick Browser Compatibility Checkup

Tue, 14 Nov 2017 20:00:00 -0400

I’ve written a few times about the state of browser compatibility and the web platform over the years. Over 4 years ago I linked to a post celebrating the rise of evergreen browsers. Then 2 years ago I was one of several people stressing about whether Safari was going to hold the web back. It’s something I’ve followed closely because supporting many different subtly incompatible execution environments is one of the most unique and frustrating challenges of front-end development.

The last few years have been really good for web compatibility with Microsoft dumping Internet Explorer in favor of the evergreen Edge, and Babel allowing JavaScript developers to write modern JS spec-compliant code and then have it transpiled back into something that runs in a wide variety of environments. But fairly recently things have started to get even better.

For the web application I have spent my workdays building over the past 4 years, we target the last 2 versions of the major desktop browsers (Chrome, Firefox, Safari, Edge). At the time of writing that means we target

Chrome 61 + 62
Edge 15 + 16
Firefox 55 + 56
Safari 10.1 + 11

I get that for some of you this is a very lax browser support standard, and you’re still supporting IE9 or old Android versions. For others you don’t care about browser compatibility at all and are just hacking demos on the newest Chrome. But “last 2 versions” is a real world standard that I know plenty of teams target. And it certainly is a reasonable target for new devs learning about JavaScript and the web. So let’s look at what is natively supported in all targeted browsers if you use this standard today. Spoiler Alert: life is pretty good.

Available Now

All of ES6 and ES2016, and almost all of ES2017

If you’re distrustful of Babel and have been holding off on learning the new features that have transformed JavaScript over the past few years until they were “browser supported”, now is your time. Other than shared memory and atomics (more on that below), every single feature from the recent updates to the ECMAScript spec is available in the last 2 versions of all 4 major browsers.

That includes core ES6 stuff like classes, destructuring, and arrow functions. But also features that were just standardized this summer like async functions.

Modern CSS

Modern CSS specifications like Flexbox, CSS Grid and CSS Variables, and CSS Feature Queries are all fully supported in these browsers, making creating layouts especially much easier than it had been in the past.

Promises and Fetch

Promises have been mostly supported for a while, but they didn’t have native support in IE11, so some folks may still be polyfilling them. With a last 2 versions standard, you not only don’t need to polyfill Promises, but also can drop any polyfills for the promise based fetch api as well (and use them with async-await natively!).

Coming Soon

So there is a ton of stuff that is available today for people targeting the last 2 versions of browsers. But there is also a bunch of stuff that is almost there:

Web Assembly and low level primitives

WebAssembly is now shipping in the latest version of every major browser. WebAssembly is a target bytecode for the web that can interop with JavaScript. Essentially somebody can write C, C++ or Rust code and then compile it to WebAssembly, which runs natively in the browser, potentially much more performantly than equivalent JavaScript code. The latest version of all browsers also support SharedArrayBuffer and Atomics, low level APIs for exploring better parallelism support in JavaScript. In short, it will soon be possible to create much more performant experiences in web browsers than is currently possible.

WebRTC

The latest versions of all browsers also support WebRTC, an API for peer to peer communication through browsers. This potentially opens the door for all types of browser-based text/video chat and collaboration applications.

Service Workers

Service workers are getting close. They’re available in every browser but Safari now, and Safari is working on them. If we get another big x.1 release of Safari like last year, they could be available for early adopters across the board within the next 6 months.

The web is moving forward together

It’s fun to see this level of unified activity across the web platform over the past 2 years. In my 7+ years of front-end development, I’ve never seen a more compatible web than the one we can develop against today.

Weekly Links: November 10th 2017

Thu, 09 Nov 2017 20:00:00 -0400

I started this new thing last week, Friday Links. There’s a feedback form if you’re a regular reader and want to tell me what you think of it. Otherwise, enjoy the links!

JavaScript And The Web

Slaying a UI Antipattern in React – JavaScript Inside - This is an interesting case study of how to work through an architectural problem using React. I love seeing posts like this that take real world scenarios (I deal with this case of having data that might be in one of several loading states all the time), and identify re-usable patterns for solving them.

Netflix functions without client-side React, and it’s a good thing - JakeArchibald.com - A nice look at what Netflix’s experience removing client-side react from their landing page means and doesn’t mean. I’m going to be really curious to see how the JS community gets better at enabling fast initial experiences while still having full rich interactivity when the users need it.

Programming Tools and Practices

Prettier - Prettier put out a new release this week and now supports Markdown as well as JavaScript, TypeScript, CSS, and GraphQL. It has been my favorite new tool of 2017, solving a problem I didn’t know I had: the mental overhead of constantly reformatting my own code.

The Pragmatic Programmer - Andy Hunt & Dave Thomas - Not a new book at all, but I like to recommend it at every opportunity. It’s my favorite resource on the practice of Software Engineering and well worth checking out if you’ve never read it.

Non-Tech

Once considered a boon to democracy, social media have started to look like its nemesis - The Economist - This is a wide-ranging take on the downsides of social media with a focus on it’s deleterious effects on our political systems. If you’re on social media and/or you care about the future of politics in your place of living, it is worth a read.

Improving Gatsby Blog Deploy UX

Mon, 06 Nov 2017 20:00:00 -0400

About a year ago I switched back to using a static site generator for this blog. Specifically, I chose Gatsby and it has been fantastic. It’s incredibly easy to customize and make my own, and I can do fun things like adding metadata to my posts to enable features like the “read next” list at the bottom of each post, and the “most popular” posts feature that is actually generated from google analytics data. The biggest downside of using a static site generator relative to a hosted service though is the annoyance of actually posting. I could edit or write a post to my Ghost blog from anywhere, which was a real perk of using that platform. Until recently I was only able to deploy this site from a single computer, because I needed an API key for Google Analytics to be available. That has been mostly fine, since I usually work on posts for this site over the course of a few days.

But last week I decided to start aiming for weekly shorter linklog posts, and that meant having a bit more flexibility with my deploys. So this week I did 2 things: I set up a Travis CI job to auto-deploy my app when changes hit the master branch, and wrote an initial version of a CLI for managing the blog. The first version just helps me write out a templated new post. My setup is pretty tailored to this blog, but I thought it’d be helpful to share, and maybe inspire some folks looking to do the same thing for their static site.

Travis CI

Travis is a continuous integration environment that is free for open source. I had used it very minimally for some open source projects, but never to deploy anything so I had to learn about the deploy option travis provides. In the end my .travis.yml file was pretty simple and looked like this:

language: node_js
node_js:
- '8'
deploy:
  skip_cleanup: true
  provider: script
  script: scripts/deploy.sh
  on:
    branch: master
env:
  global:
  - ANALYTICS_KEY=./analytics-api-key.json
  - secure: r43TdsxdDrqYtmZiBmBSe+7u0UBdepMWAYmpTGzGJoAt9p0z+CxcR9ZIj4uMPvBtO3n1wcEvylWcmmY6smu57uXNPhQpyYJaw6HlnfKrz5GjRVX6Ti6oDm8Yyhha1IlcS73dJTpApFis30Kv7fsfDvpNcvVxpF1eDUlw2UAYCwI=
before_install:
- openssl aes-256-cbc -K $encrypted_dd5d287df7ed_key -iv $encrypted_dd5d287df7ed_iv
  -in analytics-api-key.json.enc -out analytics-api-key.json -d

Breaking that down a bit:

language: node_js
node_js:
- '8'

I’m using node for this project and I want to deploy on node 8, so these lines just make sure I have that environment available.

deploy:
  skip_cleanup: true
  provider: script
  script: scripts/deploy.sh
  on:
    branch: master

I’ve bundled my deploy steps into a script, deploy.sh. This config tells travis to run the script after my (currently non-existent) tests have completed, and not to remove any built files before starting the deploy. It also makes it clear that this should only happen for changes to the master branch.

env:
  global:
  - ANALYTICS_KEY=./analytics-api-key.json
  - secure: r43TdsxdDrqYtmZiBmBSe+7u0UBdepMWAYmpTGzGJoAt9p0z+CxcR9ZIj4uMPvBtO3n1wcEvylWcmmY6smu57uXNPhQpyYJaw6HlnfKrz5GjRVX6Ti6oDm8Yyhha1IlcS73dJTpApFis30Kv7fsfDvpNcvVxpF1eDUlw2UAYCwI=
before_install:
- openssl aes-256-cbc -K $encrypted_dd5d287df7ed_key -iv $encrypted_dd5d287df7ed_iv
  -in analytics-api-key.json.enc -out analytics-api-key.json -d

This section looks messy, but its all about the environment variables I need for deploying. Specifically I’m pulling in a github key for deploys (thats the secure key), and then also pulling my google analytics key from a json file. The JSON file is stored as an encrypted file in the directory with a key that only Travis can decrypt. Then I use a public environment variable to tell my deploy script where to look up the file. That allows me to set a different path locally and still have local deploys work even though I can’t decrypt the checked in JSON file.

CLI

I’ve started with an extremely basic CLI using Inquirer.js. It asks me a series of questions and then generates a markdown doc with the appropriate YAML headings. Inquirer.js, lodash, and ES6 template strings make this super easy. It looks like this:

const inquirer = require('inquirer');
const map = require('lodash/map');
const find = require('lodash/find');
const { mkFile } = require('../src/utils/file_system');
const categories = require('../src/pages/categories.json').categories;
const moment = require('moment');

let dates = {
  Today: 0,
  Tomorrow: 1,
  'Two Days From Now': 2,
  'Three Days From Now': 3,
};

inquirer
  .prompt([
    {
      type: 'list',
      name: 'type',
      message: 'What Type of Post is this',
      choices: ['Post', 'Page', 'Friday-Links'],
      filter: val => {
        return val.toLowerCase();
      },
    },
    {
      type: 'input',
      name: 'title',
      message: 'What should the title be?',
      validate: value => {
        if (value) {
          return true;
        }
        return "Title can't be empty";
      },
    },
    {
      type: 'input',
      name: 'description',
      message: 'What should the description say?',
      validate: value => {
        if (value) {
          return true;
        }
        return "Description can't be empty";
      },
    },
    {
      type: 'input',
      name: 'key',
      message: 'What should the post key be?',
      validate: value => {
        if (value) {
          return true;
        }
        return "Key can't be empty";
      },
    },
    {
      type: 'list',
      name: 'category',
      message: 'What Category of post is this',
      choices: map(categories, 'title'),
      filter: title => {
        return find(categories, { title }).key;
      },
    },
    {
      type: 'input',
      name: 'tags',
      message: 'Write out any tags you want to use',
    },
    {
      type: 'list',
      name: 'date',
      message: 'When is this going to be posted',
      choices: map(dates, (value, key) => key),
      filter: key => {
        return dates[key];
      },
    },
  ])
  .then(answers => {
    let postDate = moment().add(answers.date, 'days');
    let year = postDate.format('YYYY');
    let formattedDate = postDate.format('YYYY/MM/DD');
    mkFile(
      `/src/pages/posts/${year}/${answers.key}.md`,
      `---
title: "${answers.title}"
date: "${formattedDate}"
permalink: "/${formattedDate}/${answers.key}/"
description: "${answers.description}"
keywords: "${answers.tags}"
key: "${answers.key}"
readNext: "ten-things-js,favorite-interview,jest-matchers-1"
---

    `
    );
  });

Inquirer makes it really nice to gather up all the information, then I use a utility I’ve already written to dump it in a file. I want to do a bunch more here in the future here, starting by giving this script the ability to read and parse my existing posts, which will let me enforce uniqueness on names and keys, as well as offer autocomplete on the readNext column, which I’ve left hardcoded for now, since its hard to type free form. There’s more to come here, and you all may feel free to steal and remix anything you find useful for your own blogs here.

Weekly Links: November 3rd 2017

Fri, 03 Nov 2017 00:00:00 -0400

Today marks the start of a new thing I’m trying with this blog: Friday Links (Edit: Since renamed to Weekly Links). Every Friday I’m going to be gathering some of the best links and content I’ve found that week and sharing it here. It will be primarily but not completely programming focused, with minimal commentary. Some weeks it may be 2-3 links, some weeks it may be 10, but I’m going to give it a run at making this a true weekly feature.

If you’re a regular reader (or not) and you have feelings about this, feel free to fill out this survey and let me know what you think. I’m still working through how this should work, though eventually it will almost definitely get integrated into the site as a different type of post than my main articles.

Without further ado, here are this week’s Friday links:

JavaScript And The Web

Effective Snapshot Testing – Kent Dodds - This is a great set of practical tips for using snapshot testing well with Jest. I’m guilty of sometimes making really large snapshots, and as he says here, I’ve found those tests to be useless because the diffs cause eyes to glaze over.

Safari service workers and other PWA good news stories - mobiForge - I’ve never been a big fan of the PWA terminology. It smells very “marketing-y” to me, in the sense of the word that engineers use as an insult. Exactly what people mean by a Progressive Web App seems hard to pin down and contextual. Is it about the technologies used? The user experience? Is it just that it is installable on phones? Is there a clear checklist that you can check against to see if something is a PWA? In any case, I am a fan of many of the trends and technology that are getting pushed under the PWA banner, so this article, which discusses how the underlying technologies are being adapted in different places, is indeed good news from my point of view.

Programming Tools

What are the Most Disliked Programming Languages? - Stack Overflow Blog - To be honest, I don’t totally buy the premise of this post. I don’t think people always list their most disliked languages as the languages they don’t want to work with on Stack Overflow, I think they also list the languages or technologies that people might expect them to work with, but they don’t want to. Like PHP or CoffeeScript for a web developer, or SQLServer for a DBA. Thats acknowledged but not fully grappled with in the article. That said, it’s still an interesting dataset, no matter how you label the languages that are being excluded.

Modern Vim - Drew Neil - Drew Neil’s first book,Practical Vim, was my introduction to useful text editing. I’ve moved off of Vim for my day to day editing over the last few years, but I still plan on buying this book when it goes paperback, because I know I’ll learn at least a few useful tips, and it’s totally possible I’ll be back to Vim as my day to day editor again someday soon. If you’re a Vim user I definitely recommend checking this out.

Extending the command line - Wynn Netherland - This is a nice roundup of the various ways you can customize your command line commands to aid in recall, reliability and efficiency.

Non-Tech

When Politics Becomes Your Idol - The New York Times - I thought this was an extremely insightful take on the current state of my country’s relationship with politics. How many of us are looking to politics for solutions to problems and seeing it demand more and more for fewer and fewer results?

The Tech-wise Family - Andy Crouch - This is a book that is to some extent written for Christians and to some extent written for parents¹, but I think that it has a lot to offer anyone who has a sneaking suspicion that the tech in their life, while helping with many things, is also distracting them from things that are more valuable. This book was the most challenging thing I’ve read this year, and I highly recommend it.

I am a Christian, I am not yet a parent. ↩︎

How To Give A Perfectly Adequate Conference Talk

Sun, 29 Oct 2017 22:00:00 -0400

Tech talks at conferences or meetups generally end up being experienced in one of 4 ways by audience members.

It’s a disaster, and that leads to rage.

This person is not just wasting my time, I don’t even understand what he’s trying to tell me. Is this a talk about Vue or just him telling his life story?

The talk is executed just fine, but it misses the mark in some way for the audience, leading to boredom.

The first 5 minutes seemed ok, but it didn’t catch my attention, and now I’ve been on Facebook for 20 minutes and woah are we already wrapping up? I don’t really know how Vue relates to my work.

The talk is perfectly adequate, and the audience leaves with some new and useful knowledge.

That was really interesting, I’ll have to check out Vue soon, it sounds like it would really help me write more maintainable web apps.

The talk is incredible and memorable. It leads to inspiration.

Wow I’m looking at web development in a whole new way now. This really changes how I think about my job!

I have no idea how to give a talk that inspires people. But after speaking a decent amount over the last few years, I’ve figured out what helps me give talks that can inform most people, and avoid having people leave bored or enraged. I want to share what I know, because the truth is that giving a talk at a conference or meetup can be fun and rewarding, and I would love it if you too got the chance to enjoy that. So here are my tips:

The Pitch

1. Show up!

The only way to give a talk that will interest and inform other people is to actually attempt it. If you live in a place that has meetups, those are often the easiest place to start. They’re usually run by 1 or 2 people who are going to be grateful to find new folks who are willing to speak. Depending on format, you may be able to give a 5 or 15 minute talk along with other speakers at a meetup to ease into things. If there’s no meetup, you can also apply to conferences. Almost all conferences have open “Call for Papers” periods where they invite applications, and many of them try to save some spots for new speakers. Conference speaking is more of a crapshoot though, since they often have many applications for few spots, and your best bet is to apply widely, and possibly suggest several different talk ideas. This year I applied to speak at 6 conferences, and actually spoke at 2, and that was by far my best ratio since I started applying to speak a few years ago.

2. Pick a topic that you’re interested in

Giving a talk can be a fun experience, but it can also be frustrating for both you and the audience. The best way to suck all of the joy out it is to give a talk about something that you don’t care about. If you’re uninterested in the topic, the work to prepare is either going to be a painful grind, or you won’t do it. In which case sitting through your talk may well be a painful grind for your audience.

Note that I don’t say to pick a topic you’re passionate about. Not every talk is going to be the pinnacle of your speaking career, and sometimes there may be nothing out there that is currently changing your life. You can really help people by sharing the useful, interesting information you have, and this doesn’t need to be more than that.

3. Pick an audience for whom that topic is relevant

So you have a talk topic, and that’s great, but does your topic make sense for the people you’re speaking to? Sometimes when you start speaking you may pick a topic before you’re sure where you’re going to give the talk, other times you may have a place you want to speak, and you need to pick a topic for it. Either way, make sure that you end up with a topic and an audience that match. If you’re talking to a group of beginner web programmers, a deep dive into the Rust memory model might not be the best choice. If you’re speaking to veteran python programmers, a talk about the basics of object oriented programming might miss its mark.

Sometimes you may not know your audience exactly. This is especially true for speaking at large multi-track conferences, where there may be developers (and/or non-developers) from a variety of backgrounds. In this case, you’ll still want to pick the audience you want to target (some subset of the group you know may be there), and then make sure to do a good job of following point #6 below.

4. Pick a “main takeaway” for your audience

It’s not enough to just have a topic and audience that generally match. You’re also going to want to know what you want the audience to get out of it. Talking about “debugging” to a JavaScript audience could be a good talk, but do you want them to learn

how to use Chrome devtools to find performance bugs?
a repeatable step by step workflow for figuring out any bug?
a story about how you solved a specific bug?

Unlike the “topic”, the takeaway should be pretty specific. Given the audience that you’re targeting, what piece(s) of new information do you want them to receive as they walk away from your talk?

5. Pick a talk format

There are a few main formats that most tech talks fall into:

Present a problem, describe several possible solutions (possibly recommending one)
Present a tool, explain what problems it solves and give examples of using it
Present an idea or way of looking at things, and show how it applies in different situations
Tell your story. Explain how a certain technology, idea, or problem in the tech community has affected your life.

All of these formats can work, for a first talk I’d recommend one of the first 2, keeping things concrete and making it easier to get to that “main takeaway”.

6. Write a description that tells your audience what to expect

The final step of putting together a pitch is writing a description that accurately lets attendees know what to expect. A great title and description do a few things.

You actually have something to submit to a meetup organizer or a conferences call for papers.
A good description helps ground you as you prepare the content of the talk. You can look at your slides and see if you’re actually focusing on the things you wanted to communicate¹.
A well written title and description will help attract your target audience to your talk. If you’re giving a talk for beginners or advanced users, or users of a particular technology, make sure that’s clear in the description. People go out of their way to go to talks where they know they’re the target audience, and nothing kills the mood of a talk than a situation where the talk isn’t relevant to most of the audience.

Note that in general, a good description matters more than the exact title of the talk, but if you’re at a multi-track conference, make sure to give a descriptive title, as conferences will often only include the talk name in their schedules, without a top-level description. Naming your talk about the creative uses of Golang “Go-ing Crazy” is fine for a Go meetup, but if people are going to be choosing what session to attend based on your talks name, something more descriptive would be nice, even if it has a little less personality.

The Preparation

7. Have slides

I’ve seen some amazing live-coding talks. I’ve given one myself at a meetup. There is definitely a place for them, as well as for minimally visual talks with 2-3 slides. But if you’re aiming for informative, structuring your talk around a good slide deck is your best bet.

Slides shouldn’t have too much information: they’re the backbone of your talk, and aren’t meant to stand alone². Instead, they provide structure both for you and the audience as you move through your presentation.

8. Have a talk track

Slides are good, but they’re just the start. You need a talk track that is more than just “reading off my slides”. To feel really well prepared for a talk, I usually write out a complete script for the talk that I intend to follow, and then edit that after several practice run throughs. For the actual talk I usually boil most of that script down to bullet points to keep me focused. There will be some experimentation needed there to figure out what works best for you.

9. Make sure your A/V setup works!

I was reminded just last week about the importance of testing that you can give your talk in the venue you’re speaking at with the equipment you have as early as possible. I was forced to give a talk last week on my manager’s laptop after mine failed to connect to the venue’s projector. It ended up being a dumb problem on my side that I could have solved if I’d verified my setup earlier and had more time to figure things out. So that’s my a/v setup parable. The things to remember here:

Show up early and test your setup if at all possible (event organizers are usually happy to help with this and may insist on this)
Bring your own adapters. In the US, at least be ready to connect to HDMI and VGA. I can’t speak to what might be common internationally, but at the very least it’s good to be paranoid
Make sure to bring a power cable. Your slides going black in the middle of your talk is not a good look.

The Delivery

10. Have notes

So now it is time to actually give your talk. Make sure to bring notes. Most presentation software programs like Powerpoint and Keynote have nice built in presenter note features for this (make sure to practice setting it up so that you see the notes and the audience just sees the slide). You also can just use paper notes in a notebook. Notes are important to keep you on track in case you freeze, and may be something you just want to read for portions of the talk.

Here again you have several choices for how to handle notes:

You can read your talk from a script. This lowers your ability to connect with your audience, but it is low risk, and you’ll guarantee that you’ll hit the content you intend to. As a native english speaker, I probably underrate this a bit too. If you’re presenting in a second language and reading can help you confidently deliver the material, it may be worth doing so.
You can work off of bullet point notes, which were ideally pulled out of a full script that you wrote originally. So you’re talking naturally, with some notes to give you structure and help pull you back in if you’re fumbling around.
You can memorize your talk verbatim. This is hard and high risk (if you forget something it can become an awkward stumble as you try to get back to your script), but has a chance at being a very compelling experience

I have found that bullet point notes work well for me, but if you’re great at memorizing things, or would be more comfortable reading and are ok with sacrificing some connection to the audience, all 3 can work.

11. Give context and set expectations (again)

At the start of your talk, you want to answer 3 questions:

Who are you?
Who is this for?
What are you going to be talking about?

You may think you answered these questions with your description, but there will definitely be people at your talk who didn’t read your description, and others who misread it. Giving context at the beginning of your talk helps people prepare to hear the things that matter in your talk. If this is a conference, it lets people decide whether they want to stay for your talk or not. Don’t be discouraged if some people walk out of your talk after your explanation. That’s a sign that you’ve done a good job clarifying the scope of your talk, and honoring their time. Having those folks leave now will be a much better experience than having them staring at Facebook and distracting others through out your talk, and it is now more likely that the people in front of you are the audience you wanted to address.

12. Be Confident and Grateful

When you know your topic and you’ve prepared well, the actual speaking part will usually end up being the easiest part. So be confident. Not because you’re already a great speaker, or because of some feeling of baseless optimism, but because you’ve done the work to be ready and you have the structure around you to succeed. Also make sure to be grateful. A bunch of people showed up to hear you speak. Appreciate them, give them your best and hopefully try to answer their questions. Speaking is a great opportunity, and I hope you get the chance to try it.

More Resources

Rands has a great post on preparing for a presentation. Lots of great little details in there.
Papercall.io is a great way to find conferences that are looking for speakers
If you’re not already connected with local meetup/interest groups, take a look at Meetup.com and see if there are any in your area.

It’s ok to change your description later if things change as you prep the talk. ↩︎
People will often ask for the slides of my talk after I give them. I usually link them, since people ask, but the slides on their own really aren’t a substitute for the talk (and if they are informative to that level, the talk will probably suffer as the audience tries to take in too much information). ↩︎

Taking Advantage of Jest Matchers (Part 2)

Mon, 04 Sep 2017 23:00:00 -0400

This post continues my look at Jest matchers from Part 1.

In part 1 I covered the first 4 types of Jest matchers. In this post I’m going to cover contains matchers, async matchers, snapshot matchers, function matchers and meta matchers, as well as looking at a few extra tips and tricks for using matchers.

Contains Matchers

Contains matchers are exactly what they sound like. They’re matchers that check to see if an array contains an item or if a string contains a substring.

toContain and toContainEqual

toContain and toContainEqual both check to see if a value is contained inside of an array or string. When used against a string, they are both equivalent and check to see if a passed string is a substring of the expression being tested. This means they serve as a more limited version of toMatch that will only take a substring and not a regex. When used against an array or array-like object, toContain looks for matches using strict equality while toContainEqual looks for matches using recursive equality¹. In practice that looks like

// toContain does exact matches
expect(['a', 'b', 'c']).toContain('a') // true
expect({0: 'a', 1: 'b', length: 2}).toContain('a'); // true
expect('abc').toContain('ab'); // true
expect('abc').not.toContain(/a/); // true
expect([{a: 'bc'}, {b: 'cd'}]).not.toContain({a: 'bc'}); // true

// toContainEqual does recursive matches
expect(['a', 'b', 'c']).toContainEqual('a') // true
expect([{a: 'bc'}, {b: 'cd'}]).toContainEqual({a: 'bc'}); // true

Async Matchers

Async matchers handle testing functions that have an asynchronous component. We’ll look at resolves and rejects for testing Promises. But first, it is worth noting that any matcher can be used for asynchronous testing if necessary. You just have to use the callback option that is passed to each test function, like so:

test('Delayed Test', done => {
  let delayedEval = condition => () => {
    expect(condition).toBe(true);
    done();
  }
  setTimeout(delayedEval(1 === 1), 0) // true
})

Jest detects whether the done callback has been defined for the function, and if it is waits for 5 seconds to see if done is called before failing, and evaluates any asynchronous expects that occur before then. This behavior is inherited from Jasmine, and works ok, but can be a bit difficult to work with. Jest makes this easier in 2 ways. First with a set of special matchers for working with promises, and second with a set of meta matchers that can make async tests more reliable. We’ll get to the meta matchers in a second, but first we’ll look at the promise matchers.

.resolves, .rejects

resolves and rejects are chaining matchers like not that let you write async tests with Promises. Using resolves to rewrite the previous example looks like this:

test('Delayed Test with promises', () => {
  let testPromise = new Promise((res, rej) => {
      setTimeout(() => res(1 === 1), 0);
  });
  expect(testPromise).resolves.toBe(true); // true
})

This waits for the promise to resolve, and then compares the resolved output with toBe. If a rejection is expected, you can replace resolves with rejects for a similar test. This ends up being much clearer in my opinion than the callback-based async tests, since the assertion is much more direct, and you don’t have to worry about cases where done might not be called where it should be.

Meta Matchers

Meta matchers pair well with the done async matchers above, if you have to use them. They just let you test that all of your assertions have actually run. The first one, expect.assertions tests that a specific number of assertions have been run at that point in the test.

test('records have a name and an ID', () => {
    let record = getRecord(1);
    expect(record).toHaveProperty('id');  
    expect.assertions(1);  // true
    expect(record).toHaveProperty('fullName');
    expect.assertions(2);  // true
})

This can be useful for async tests to make sure things have run correctly before calling done(), for tests with conditional logic to make sure that the logic played out as expected, and for normal tests as an extra paranoid quality check to make sure that tests aren’t accidentally deleted.

expect.hasAssertions is just a simpler version of this test that checks that you’ve asserted something in the test. It’s useful for the same things as expect.assertions but will be a little less reliable and also a bit less fragile when tests change.

Note that neither expect.assertions nor expect.hasAssertions count as assertions themselves and don’t contribute to the assertion count.

Snapshot Matchers

Snapshot testing is one of Jest’s signature features. It’s been described in depth in many other places, including on this blog. If you’re unfamiliar with snapshot testing, I recommend checking out one of those deeper posts. But at a high level, snapshots are an assertion that data has remained the same since the last time a test was run. Jest has serializers for a bunch of built in data types and React components. But it’s possible to define serializers for different data types to determine how that data should be represented in a snapshot.

The core snapshot function, toMatchSnapshot() is fairly simple to explain. When run the first time, toMatchSnapshot takes the string representation of the data passed to it (generated by whatever serializer is used for that data type) and writes it to a snapshot file. After the snapshot has been written once, future runs of toMatchSnapshot behave differently. If Jest is run normally, toMatchSnapshot compares the serialized data to the value in the snapshot file, and fails the test if they are different. If Jest is run with the -u option, it will instead update the snapshot file with the current serialized value. This behavior lets us write tests that don’t have a pre-defined “correct value”, instead we can use our test cases to see what has changed, and choose to accept it or not.

toThrowErrorMatchingSnapshot

In addition to toMatchSnapshot, Jest offers a second snapshot matcher, toThrowErrorMatchingSnapshot, which catches errors and verifies that they match a snapshot, similar to the toThrow matcher I described in part 1.

Function Matchers

One of Jest’s nicer features is its function mocking system. A full discussion of function mocking is outside the scope of this post, but suffice it to say that you can create “tracked” functions in Jest. The following matchers work with those tracked functions to verify behavior, using an example of a function that takes another function and calls it with the given arguments.

const executeFn = (fn, ...args) => fn(...args)

toHaveBeenCalled, toHaveBeenCalledTimes

The most basic matcher when dealing with functions is toHaveBeenCalled. It works more or less how it sounds, and determines whether a function has executed during the test.

test('does executeFn run the function passed to it?', () => {
  let fn = jest.fn();
  executeFn(fn);
  expect(fn).toHaveBeenCalled();
})

Jest also has a related matcher, toHaveBeenCalledTimes which lets you specify how many times a function has been called.

test('does executeFn run the function passed to it only once?', () => {
  let fn = jest.fn();
  executeFn(fn);
  expect(fn).toHaveBeenCalledTimes(1);
})

toHaveBeenCalledWith, toHaveBeenLastCalledWith

Sometimes you don’t just want to know if a function was called, but want to make sure that it was called with the right arguments. Jest has you covered.

test('does executeFn run the function with the passed arguments', () => {
  let fn = jest.fn();
  executeFn(fn, 1, 2);
  expect(fn).toHaveBeenCalledWith(1, 2);
})

toHaveBeenCalledWith lets you check that the function was called with the right arguments. You have to be careful though, if the function was called multiple times, you’re only verifying that it was called with those arguments at some point. If you want to get more specific, you can use toHaveBeenLastCalledWith.

test('does executeFn run the function with the passed arguments', () => {
  let fn = jest.fn();
  executeFn(fn, 1, 2);
  executeFn(fn, 3, 4);
  expect(fn).toHaveBeenCalledWith(1, 2); // true
  expect(fn).not.toHaveBeenLastCalledWith(1, 2); // true
})

Bonus Content

That’s all for matcher methods, but there are a few other helpful things to know about matchers that I didn’t get to work into any of the matcher categories. So here you go:

Extension Methods

Jest’s expect object has 2 methods for extending what it can do: expect.addSnapshotSerializer and expect.extend. expect.addSnapshotSerializer lets you add a new snapshot serializer as a one off for a specific test file. A serializer is simply an object that defines test and print methods. You can learn more in the docs.

expect.extend allows you to define custom matchers! It takes an object with methods that define the new matchers to add. Each method should be a function that takes the expected value as a first argument, and then expected arguments to the matcher as the rest of the arguments. For instance, here is a matcher to see whether a string is a palindrome.

expect.extend({
  toBeAPalindrome(received) {
    if (typeof received !== 'string') {
      return 'expected ${received} to be a string, but it was a ${typeof received}';
    }
    const reversed = received.split('').reverse().join('');
    const pass = received === reversed;
    if (pass) {
      return {
        message: () => (
          `expected ${received} not to be a palindrome`
        ),
        pass: true,
      };
    } else {
      return {
        message: () => (`expected ${received} to be a palindrome but it was ${reversed} when reversed`),
        pass: false,
      };
    }
  },
});

test('Anna is a palindrome', () => {
  expect('anna').toBeAPalindrome(); // true
  expect('ann').not.toBeAPalindrome(); // true
});

Matcher Wildcards

Several of the matchers I’ve described in these posts are focused on equality of a particular part of a structure. These include the template matchers, contains matchers, and the toEqual matcher. In these cases you’re trying to say that some part of an object is equal to something else. But sometimes you want to be a bit open-minded about part of that. Maybe you want to ensure that an object has an email property that is a string with the character @ in it , or that it has a name property defined, but that could be anything as long as its defined. Jest has wildcards that help you do this.

expect.any(<constructor>) lets you match any object of a specific type
expect.anything() matches anything except null and undefined
expect.stringMatching(regexp) matches any string that matches the regex you pass it
expect.stringContaining(string) matches strings against a substring
expect.arrayContaining(array) matches arrays that have a specific set of values in them
expect.objectContaining(object) matches objects that have a specific set of key/value pairs

Using wildcards looks like this:

test('wildcard example', () => {

  expect('test').toEqual(expect.anything()); // true

  expect({foo: ['bar', 'baz', 'test']}).toMatchObject({
    foo: expect.arrayContaining(['bar', 'test'])
  }); // true
});

expect outside of Jest

As I was finishing up this post, Jest 21 was released, along with the exciting news that jest-matchers has been renamed to expect and is now available as a standalone module that can be used in other test frameworks, including in the browser. So if the matchers in these blog posts have looked appealing to you, but you have an existing test framework that you’re tied to, feel free to check it out on npm.

More Resources

The latest info on all of these matcher APIs is always going to be in their very readable docs. Check them out for more examples and good links to a deeper dive on some of the other topics that these posts have touched on, like snapshot testing and mock functions.

Similar to toBe and toEqual from part 1, strict equality here means it’s the same reference and is equivalent to x === y, while recursive equality means the object’s are structured the same and any object with the same structure and values will be considered equal. ↩︎

Taking Advantage of Jest Matchers (Part 1)

Tue, 15 Aug 2017 00:00:00 -0400

Back in May, my favorite testing library, Jest, released it’s 20th version and rebranded itself as the library for “Delightful JavaScript Testing”. Delightful and testing are two words that usually don’t go together in the software development lexicon. But that claim is the result of 2 years of hard work from the Jest team trying to get the usability details of testing exactly right. One of the version 20 announcements that exemplified this commitment to detail was the addition of several new assertion APIs to an already large list. This post dives into the many matcher options in Jest, why they’re there, and how to make good use of them.

Assertion APIs, or matchers, are the methods that the library makes available for defining the expected value of a variable or expression. The most basic assertion in Jest is the .toBe matcher, which checks to see if one value is the same as the other.

test('Math still works', () => {
  expect(1 + 1).toBe(2);
})

Why so many matchers?

When writing tests, the only assertion api you really need is a method that takes a boolean and determines whether it is true or false. Assuming you can figure out inspecting functions and async code, everything else can be expressed with an assert method like that:

test('Math still works', () => {
  assert((1 + 1) === 2);
})

So why does Jest need 30+ matcher methods? Because they allow you to be specific in your intent, and also let Jest provide helpful error messages. For instance, when you write a test like this:

const getRecord = () => ({ id: 1, 'fullName': 'Ben McCormick'});
test('records have a name and an ID', () => {
    let record = getRecord(1);
    expect(record).toHaveProperty('id');
    expect(record).toHaveProperty('name');
})

it is obvious what the test is trying to check, and you can get detailed output like this:

You get the structure of the object as it is, the name of the property that is missing, the line number and file name where you can find the assertion, all in a concise, colorful and nicely formatted test failure message.

You can categorize Jest’s matchers into 8 different categories: equality matchers, template matchers, numeric comparison matchers, contains matchers, async matchers, snapshot matchers, function matchers, and meta matchers. I’ll go through the first 3 in this post, and cover the rest in a followup.

Equality Matchers

Equality matchers are the workhorse matchers of any test libraries. This is where we see whether an expression evaluates to the value we expect. This being JavaScript however, equality takes a few different forms, and Jest covers each of them, along with some syntactic shortcuts.

toBe and toEqual

toBe, as mentioned previously, is the simplest Jest matcher. It is equivalent to a simple === statement, and verifies that the expression being tested is the same reference as the expression that it is matched against. Since it is reference based, toBe is a good fit for numbers, strings and booleans, or for checking that an object is the exact same reference as another object. But it’s not good for comparing two separate objects. It behaves like this:

expect(1 + 1).toBe(2) // true
expect('on' + 'e').toBe('one') // true
expect(1 === 1).toBe(true) // true
let oneObj = {one: 1};
expect(oneObj).toBe(oneObj) // true
expect(oneObj).toBe({one: 1}) // false!

For comparing object and array equality, it usually is better to use toEqual, which does a deep property comparison:

let oneObj = {one: 1};
let nestedObj = {nested: {one: 1}};
expect(oneObj).toEqual({one: 1}) // true
expect(nestedObj).toEqual({nested: {one: 1}}) // true

toEqual doesn’t line up to any built in JavaScript idea of equality exactly, it just does a recursive equal check. But it behaves like most people who haven’t already encountered the details of JavaScript equality would expect equality to work. If objects share the same structure and values, they’re equal. There also is no Jest equivalent to == with its odd type casting behavior, and that is ok by me.

toBeDefined, toBeNull and toBeUndefined

toBeDefined, toBeNull, and toBeUndefined are all shortcut functions.

// same as expect(x).toBe(null)
expect(x).toBeNull()
// same as expect(typeof x).toBe('undefined')
expect(x).toBeUndefined()
// same as expect(typeof x).not.toBe('undefined')
expect(x).toBeDefined()

One thing worth pointing out here is the use of .not in my example comment. .not is a utility property that you can chain to reverse the output of any matcher.

toBeFalsy and toBeTruthy

These matchers essentially are shortcuts that act like if and else statements.

// same as expect(!!x).toBe(true)
expect(x).toBeTruthy()
// same as expect(!!x).toBe(false)
expect(x).toBeFalsy()

If you aren’t aware of the JavaScript concepts of truthiness and falsiness, the terms refer to whether a value is evaluated as true or false when evaluated in a boolean context.

Template matchers

Template matchers are matchers that don’t check for a specific value, but instead look to see if the expression is consistent with a certain pattern or shape.

toMatch

toMatch is a template matcher for strings. It accepts a regular expression that you can use to describe the acceptable list of strings.

let id1 = '155-60-7723';
let id2 = '15-60-7723';
let socialSecurityNumberFormat = /\d{3}-\d{2}-\d{4}/;
expect(id1).toMatch(socialSecurityNumberFormat); // true
expect(id2).toMatch(socialSecurityNumberFormat); // false

toMatchObject and toHaveProperty

toMatchObject and toHaveProperty are template matchers for objects. toMatchObject works similarly to toEqual and does a deep comparison on objects and arrays. But unlike toEqual, toMatchObject doesn’t require an exact match, it just verifies that the expression has the same properties as the template object. So extra properties on the expression we’re evaluating don’t matter.

let oneObj = {one: 1, uno: 1};
expect(oneObj).toEqual({one: 1}) // false
expect(oneObj).toMatch({one: 1}) // true

toHaveProperty performs a similar function but doesn’t enforce the whole structure of an object. Instead it just tests whether a single property is defined.

let oneObj = {one: 1};
expect(oneObj).toHaveProperty('one') // true

toHaveProperty can take a second “value” argument, at which point it acts as a more limited version of toMatch.

let oneObj = {one: 1};
expect(oneObj).toHaveProperty('one', 1) // true
expect(oneObj).toHaveProperty('one', 2) // false

toHaveLength

toHaveLength is a simple matcher for checking the length of strings, arrays and any other array-like object that has a length property. expect(x).toHaveLength(y) is just a shorthand for expect(x.length) with the benefit of handling undefined values of x safely.

expect('Hello').toHaveLength(5) // true
expect(['Hello', 'From', 'The', 'Other', 'Side']).toHaveLength(5) // true
expect({length: 5}).toHaveLength(5) // true

toBeInstanceOf

toBeInstanceOf allow’s checking the “type” of an object to see if it is an instance of a class, including inherited classes.

test('We can test whether an object is an instance of a class', () => {
  class A{}
  class B extends A{}
  class C{}

  let a = new A();
  let b = new B();
  let c = new C();
  expect(a).toBeInstanceOf(A); // true
  expect(b).toBeInstanceOf(A); // true because of inheritance
  expect(c).not.toBeInstanceOf(A); // true
})

toThrow

toThrow is a bit of a different template matcher, because rather than matching a value, it matches against an exception. except() takes a function, and instead of evaluating that function, it executes the function and looks to see if an exception is thrown. If there is an exception, toThrow can take a string, regex or Class and acts like either toMatch or toBeInstanceOf depending on what it is passed.

test('We can match exceptions', () => {
  const throwingFunc1 = () => {
    throw 'error';
  };
  class SpecialError{}
  const throwingFunc2 = () => {
      throw new SpecialError()
  };

  expect(throwingFunc1).toThrow('err'); // true
  expect(throwingFunc1).toThrow(/err/); // true
  expect(throwingFunc2).toThrow(SpecialError); // true
})

Numeric Comparison Matchers

Numeric comparison matchers are straightforward. They test whether a number relates correctly to another number.

toBeGreaterThan, toBeGreaterThanOrEqual, toBeLessThan, toBeLessThanOrEqual

These are exactly what they sound like.

expect(1).toBeGreaterThan(1) // false
expect(2).toBeGreaterThan(1) // true
expect(1).toBeGreaterThanOrEqual(1) // true
expect(2).toBeGreaterThanOrEqual(1) // true
expect(1).toBeLessThan(1) // false
expect(0).toBeLessThan(1) // true
expect(1).toBeLessThanOrEqual(1) // true
expect(0).toBeLessThanOrEqual(1) // true

toBeCloseTo

This test checks whether a number is the same as another number within a set number of significant digits. toBeCloseTo takes a number and an optional second option that determines the number of significant digits, then tests whether an expression is the same as that number within that number of significant digits. This is useful in JavaScript since floating point math is not precise and 0.1 + 0.2 famously does not equal 0.3 exactly.

test('Floating Point Math Is Hard', () => {
  expect(0.2 + 0.1).not.toBe(0.3) //true
  expect(0.2 + 0.1).toBeCloseTo(0.3) //true
})

Part 2 Coming Soon

There’s a lot to cover here, so check back soon for part 2 of this post, with contains matchers, async matchers, snapshot matchers, function matchers, meta matchers, and a few extra tricks for using matchers.

My Favorite Interview Question

Mon, 24 Jul 2017 07:45:00 -0400

I have a favorite question to ask in interviews for web developers.

“How does the internet work? In other words, when I open a browser and enter a url then hit enter, and a website shows up on my screen what can you tell me about what is happening?”

Usually I get one of a few responses:

1. The clueless shrug

Uh…. ¯\_(ツ)_/¯

Some people have no idea what to do with the question. Things get awkward. This is mostly for junior positions but still isn’t the best sign. Sometimes a few prods will get things back on track.

2. The bare minimum

It is asking a server for the site and the server sends you the site.

Some people can give a very basic response, but don’t give a lot of details. This again mostly corresponds to less experienced devs and can be fine as a starting spot with some follow-up questions.

3. The CliffNotes

Your browser sends an HTTP request which gets routed through a local modem/router and gets sent to a name server for its url extension. That name server routes the request to the correct IP address, which will resolve to some sort of web server. That server will serve up either some static files, or run some backend code in order to generate a resource (probably an html page). When the HTML page is returned, your browser will parse it, which will likely generate more requests, and the cycle will repeat.

Some people know how some of this works, maybe with a few hazy areas and can explain it in a reasonably straightforward way. An explanation like this is a great start.

4. The Geek-out

Let me tell you all about my in depth knowledge of <HTTP_network routing_JavaScript parsing/TCP/other related topic>

Some people dive in and explode with information.

So why is this a good interview question again?

So which answer is best? The geek-out right? Well… its great when somebody gets there, but in my mind this question is more about the journey than the destination.

It’s pretty easy to make the case that this is a pretty bad interview question. It’s not clear what the expected answer is. When I ask it in junior dev interviews, they often don’t have much of a clue how to answer it. And it’s not strictly answerable in full detail by pretty much anyone. But I’d argue that those things actually end up being strengths. I like this question for 5 reasons.

It tests the candidate’s ability to deal with ambiguity. The question is intentionally vague and open ended. Am I just looking for a perfunctory response like “it gets a resource from a server”? Do I want a deep dive into how DNS works? It’s ambiguous. Candidates can respond to that ambiguity in one of a few ways.

They can freeze because they’re afraid of saying the wrong thing.

They can ask a clarifying question.

They can make a guess at what I want and start going down that road, while reading my reaction to see if they’re still on track.

Or they can decide they know what I want (or what they want to tell me) and shoot off like a rocket (potentially for a very long time).

This is super useful information in an interview because developers deal with ambiguity all the time. When a candidate encounters unclear requirements from a product manager, or unclear documentation from another developer, it’s good to have an idea how they’ll respond¹.
It tests the breadth of a candidates knowledge. - For the most part, when I interview people, I ask them questions about the role they’re applying for: Front-End Dev, Backend Dev, QA Engineer, etc. But this question gives a candidate the chance to show that they understand what’s happening up and down the web stack from the code they interact with the most.
It gives the candidate a chance to explain a complex technical topic. - In my mind, good communication ability is the most important skill for a developer once they’ve achieved a minimum level of technical competence. Being able to talk about problems and solutions clearly and explain them to a variety of audiences matters, and it matters a lot. Can the candidate summarize a complex technical topic without getting bogged down in details? Are they communicating with me (verbally or otherwise) to make sure I’m following their explanation? Or are they fading off into a rambling run-on explanation of a trivial detail without any big picture context?
It forces the candidate to confront something they don’t know. - The internet is a perfect example of the Isaac Newton quote about standing on the shoulders of giants. It is the culmination of 50+ years of research and work on networked computer systems. No candidate I interview for a web development position is going to understand every detail of how it works, because you can always go a layer deeper in the stack or afield into crazier edge case behavior. So at some point in questioning (possibly very early), this question exhausts every candidate’s knowledge. The question is how do they react? Some candidates start guessing at things. Some admit they don’t know. Some try to skirt around areas they don’t feel confident in and work at avoiding any questions where they would have to admit they don’t know something. And some will actually ask me how part of the system works. This again is helpful because “not knowing the answer” is such a common experience in real world web dev. It’s great to have some clues to how a candidate handles that.
It gives the candidate an opportunity to show off what they know. I’ve gotten some amazing answers to this question. It turns out some candidates actually know a lot about networking and HTTP/TCP/nameservers. Some can give detailed answers about browsers and how they parse resources and display the DOM. Maybe they just know a lot of random trivia about URLs and DNS. Regardless, this is a chance to show off and prove that they’ve delved deep into something.

To be clear, I don’t think every interview question should be like this one. Most should probably directly relate to the work the candidate is doing, be clearly comparable in an objective way candidate to candidate, and not risk sending the interview down a 10 minute rabbit hole. But this one is my favorite, and I think there’s a place for questions that show how a candidate might handle some real world lack of structure.

Feel free to hit me up on Twitter at @_benmccormick to tell me your favorite interview question. I’ll retweet them.

There is of course a limit to how much any interview is going to teach you about how somebody will behave in a future job situation. We’re all chasing breadcrumbs here. ↩︎

Ten Things A Serious JavaScript Developer Should Learn

Wed, 19 Jul 2017 00:00:00 -0400

There’s been an interesting Reddit thread circling my corner of the internet for the last week or so. It started with a question:

I’m inviting pure opinion here, but what’s your list of ten things that everyone looking for a good javascript role should know and understand. Personally, I’m in a role where what I know is adequate, but I want to be somewhere better; somewhere that has some flippin standards. I’m learning outside of work requirements at the moment, but there is so much to learn out there. The objective is to do good work in a good place. Narrow it down to your ten for me, or for you..

From “What 10 Things Should a Serious Javascript Developer Know Right Now?”

I think this is a great question if you take it seriously, because when you think about it critically, it exposes all of the things JavaScript don’t really need to know. Can you contribute productively as a mid-level JavaScript developer without knowing anything about Prettier, Typescript or React? Sure! Can you contribute productively if you’re writing and testing your JavaScript using <awful uncool code editor that you the reader can’t stand>? Definitely. So here’s my take on answering the question.

Assumptions

I’m making the following assumptions about the question

I’m assuming the audience is a developer who has some level of professional programming experience and some existing JavaScript experience, and that they are primarily interested in “code skill” answers. Thus I’ll exclude things like communication ability, experience with version control, professional conduct, or basic coding ability that are very important to success in a JavaScript development environment but are out of the scope of this question.
I’m optimizing for “what specific pre-existing knowledge will make the most difference between success and failure for a new hire in a mid-level JavaScript role”.

Pre-existing means that these are things that somebody is expected to know before taking the job. Learning the details of a company’s technology stack, style guide and engineering philosophy are all important, but you shouldn’t be expected to know them in advance, and in many cases doing so is impossible.

Specific knowledge means that these are things that you can learn with some effort and study. They should not be broad categories like “security”, “design”, or “server-side development”. They should be things you know or know how to do, not things you “have experience with”. And they should be specific enough that somebody can feel comfortable saying they understand the topic or don’t. Most of the things I list below are topics, but I try to lay out a specific set of things that you would need to know to understand it.

The most difference between success and failure means that these are the things that are really going to make a difference in how effective you are at understanding and completing the work assigned to you. There are many such things, and their importance will vary situationally, so this is just my educated take on a generalists list.

When I say **mid-level**, I'm picturing hiring for a role that pays well but is not the most senior person on the team.  This would be a developer who would be expected to complete tasks with minimal help and oversight but wouldn't necessarily be planning the architecture of everything right off the bat.  To be clear this list is not a set of things that you need to go to get paid to write JavaScript.  It is more along the lines of "next steps" for junior devs.  Its also not a step function where you need to know all of these at once to get to the next level.  These are just things that may limit you if you don't have them.
I’m taking the liberty to assume that JavaScript role implies using JavaScript in some sort of client/UI environment (Browser, React Native, Cordova, Electron, etc).  This list would be at least a bit different for a Node.js server-side role, and somebody else will have to write that post.

The List

1. You should know the core language

JavaScript is not a very complicated language at its core. It can seem difficult to learn because of all of the noise around it. Browser APIs, Library APIs, jQuery, React, TypeScript, the many npm packages out there, etc all can make the language syntax space seem huge. And this isn’t helped by the fact that the language has evolved over time so you’ll see many different ways to do a single task spread out over the internet. But the core language and standard libraries are actually pretty small compared to languages like Python and Java that have more features and larger standard libraries. Eventually you’ll need the ecosystem. But knowing the core language features that are implemented in most browsers is a great start.

This means you should know about basic language features like if/else, error handling with throw/try/catch, array manipulation with push/map/reduce/filter, and string manipulation methods like replace/slice/split. You should also know about how JavaScript handles things like truthiness and falsiness[1], how to work with JSON data with the JSON object methods, and how variable scope works in JavaScript. These days its also becoming more and more important to have at least a basic familiarity with the more important language features that were added in ES6 like Promises, modules, classes, template literals, generators and destructuring. Even if you choose not to use those features, being able to see and understand them will help you learn faster from others.

This list is mostly unordered, but this comes first because learning everything else becomes a lot harder without this knowledge, mostly because it becomes difficult to learn from other people’s code since you won’t understand what it is doing. If you’re starting from a weak base here I’d recommend finding a good book on the core language like Eloquent JavaScript [2] and digging in. If you’ve been using JavaScript in an older code base or haven’t used it in a while and want to catch up, you can see my post on keeping up with new language features.

2. You should understand async code

One of the biggest leaps for new JavaScript developers, even those who are experienced with other languages is the amount of asynchronous code found in idiomatic JavaScript. Asynchronous (async) code is code that is scheduled to run at a later time after some event, delay or response. It can appear a bunch of different ways in JavaScript code, and it’s important to be able to interpret them.

This means learning about callbacks and promises at least. You should also learn about the JavaScript event loop and how it works Ideally some of the standard APIs that use them like setTimeout and fetch. If you’re ambitious you can learn about newer async syntax and library implementations like async-await or RxJS But the most important thing is to spend time using and writing async code so that you get an intuitive understanding of how it works.

3. You should get comfortable with at least one set of developer tools

When developing JavaScript, its important to be able to debug problems and understand how existing systems work. JavaScript has a fantastic ecosystem of developer tools for diagnosing bugs and examining application code. For the most part they’re bundled with browsers (though some IDEs like VS Code and WebStorm have their own debuggers). Learning how to use these tools well will turbocharge your JS dev skills. If you can’t use them though, it will make everything much harder.

Competence in this means comfort with at least the JavaScript debugger panel of a browser’s devtools. You should be able to set and use breakpoints, use watch mode and the console to test for the value of expressions, and understand the stack traces. But its also good to get comfortable using the elements panel to understand and manipulate the DOM, and the network panel to debug network requests.

4. You should be proficient with JavaScript functions

Functions are the core of the JavaScript language. You should be able to use them fluently. That means understanding function scope, closures, how this works, and the difference between normal functions and arrow functions. It also means knowing about how to process arrays with function using methods like map, filter, and reduce. A good resource for “thinking in functions” is JavaScript Allonge by Reginald Braithwaite.

5. You should be comfortable with basic design tasks

No matter how well you learn JavaScript, you’ll be limited career-wise if you can’t manage basic design tasks. This might not seem inherently true; after all isn’t that what professional designers are for? But the truth is that unless you’re a purely server-side developer (excluded by our assumptions above), there will be times when you’re asked to fill in a visual gap that a designer doesn’t have time for, or there will be ambiguities in a designers spec that aren’t worth a meeting. And learning the basics of design will help you communicate better with designers and product managers about technical constraints and design requirements.

When I talk design, I mean both the basics of implementing a visual design with HTML and CSS[3], and enough of an understanding of design basics to create a simple UI that looks ok and isn’t confusing. That last requirement is harder than you’d think. Design is hard and mostly orthogonal to the set of skills most folks pick up while learning to code. For learning design theory, I’d recommend finding a simple resource on visual design first like White Space is Not Your Enemy, and ideally supplementing that with a more UX focused resource like Don’t Make Me Think. The practical side of HTML/CSS is easiest to pick up with experience. If you’re using devtools you can see what other people are doing with CSS and tweak and explore using the elements panel. And there are lots of good online resources for learning about CSS and HTML syntax like MDN for API info or CSS-Tricks for ideas, tutorials and keeping up with new stuff.

6. You should have a basic understanding of networking and HTTP-based APIs

Because we’re assuming that this job involves using JavaScript in a browser or other client context, you should be able to understand the basics of networking. You should know what a client and server is. You should be able to explain a rough approximation of what happens when somebody enters a URL into the browser[4]. There’s a bunch of vocabulary thats good to know here. You should know what an HTTP request is and ideally know the conventions of what GET/PUT/POST/DELETE imply. It will help you to have a working definition of what REST is and what AJAX is. It’s good to get experience working with some HTTP based APIs, either at work or playing around with an API like Twitter or Github. You should also get familiar with at least one way to make HTTP requests in JavaScript (XMLHttpRequest, fetch, jQuery’s ajax methods).

7. You should be comfortable with Node.js based tooling

One of the biggest shifts in the last 5 years in the JavaScript world has been the explosive growth and consolidation of tools to help build JavaScript applications. Tools like eslint, Babel, Typescript, and Webpack are essential to many teams workflows. While you don’t necessarily have to be experienced writing applications for Node.js, it is important that you be able to use those tools. That means knowing how to install Node and use npm to install and update packages. It also means getting comfortable with build tools like Webpack, Rollup or Ember CLI and knowing how to use them. Reasonable people can differ on how many tools and libraries you should be using, or what the right ones to use are, but having a basic familiarity with how the Node ecosystem works has become essential.

8. You should know how to use a framework to structure a moderately sized program

A critical requirement for a developer who is expected to complete work with minimum supervision or hand-holding is the ability to structure their code in a maintainable way. That means having an understanding of (or helping define) the overall architecture of the system, and then writing code that fits into it. It also means being able to identify and help solve situations where the current architecture or code base norms causes problems. I put this in terms of frameworks because pretty much all medium to large JavaScript programs are using some sort of framework, whether it is an “official” framework like React, Angular, and Ember or a collection of helper files and functions cobbled together by a development team over time. For learning purposes, the popular official frameworks will usually be better to learn, since the knowledge can translate elsewhere, the purposes behind the structure are documented on the internet, and at least some sense of best practice architecture should be embedded in the code. To check this box I think it would be best to get experience with at least 2 frameworks/architectures, since it is difficult to tell which decisions a Framework makes are specific to its architecture and constraints without something to compare it with.

This is probably the most controversial item on this list, since I know a lot of folks believe it’s better to learn architecture ideas outside the constraints of a framework. My stance here is that frameworks are practical: they’re most likely to translate to an actual employment background and it’s easy to find practical explanations of how they solve architecture problems.

9. You should know the basics of JavaScript program performance

Like design, performance is a deep well of things you can learn. But knowing the basics is important and can go a long ways. For performance “the basics” is more about process than knowledge. You need to learn how to debug slow processes and understand where the problems are coming from. For browser environments in particular there are some baseline things to understand. You’ll want to be able to tell the difference between code that is running slowly due to parsing slowness, runtime slowness, and network latency. It’s important to learn how to keep a UI responsive and not block rendering with other calculations. And it’s great to understand how to cache data and avoid making too many requests to the server.

This is also an area where learning more about Computer Science theory can be beneficial. Studying data structures and algorithms can help give you good intuitions on how to handle data, and learning about Big O notation gives you a language for discussing performance tradeoffs. This is the type of background knowledge that pays off in occasional subtle ways though, not something that is always obvious applicable in the moment.

10. You should have a process for learning new things

JavaScript development has changed a lot over the last 10 years, and there’s no reason to think it won’t continue to evolve over the next 10. There are also a lot of different technologies that fall under the umbrella of JavaScript, more than any one person can keep in their head. So it’s important to be able to keep up with the new technologies as they come along, and also learn about new technologies quickly when you need them. Ideally this means having at least one place where you can follow news about new JavaScript innovation. Could be Twitter, a set of blogs, newsletters or an aggregator site like Reddit or Hacker News. If you’re looking for an easy place to start, I’d recommend the JavaScript Weekly mailing list.

You also need to get comfortable finding information about specific technologies, either to learn to use them or troubleshoot problems. These resources will likely be more varied, but it’s important to learn to Google well, and also helpful to get familiar with Stack Overflow or some other resource for getting help when you have specific questions or problems.

Do you have a different list? Feel free to post it and then link in the comments!

If you don’t learn all the pitfalls here, at least learn that there are pitfalls. ↩
Disclosure: Some of these book links are affiliate links ↩
Or SASS/LESS/CSS in JS/whatever ↩
This can be explained at several levels, but its good to at least know that a request is sent from the users computer to a server somewhere, and that server returns resources like HTML, CSS, JS, and image files. ↩

How to follow the JavaScript roadmap

Mon, 10 Jul 2017 08:00:00 -0400

It can be difficult to keep up with all the things going on in web development, especially since the JavaScript language itself has been changing over the last several years. After several years of relative stability¹, the ECMAScript spec (ECMAScript is the specification for the JavaScript language) has changed quite a bit over the last 3 years, with more changes on the way. In this post I wanted to highlight a few of the resources I can recommend for keeping up with the language changes.

Learning ES6/ES2015

If you haven’t paid attention to any features that aren’t ready in all browsers yet, or you’re getting back to JavaScript development after being away for a while, you probably could use a resource to catchup on the ES6 JavaScript release from 2015. ES6 is now almost completely implemented in the latest versions of all major browsers, so if you’ve been ignoring it due to compatibility concerns, now is the time to learn. Fortunately there are a lot of great resources for learning about ES6. My favorite quick reference is the Learn ES2015 page from Babel. It’s a short, example filled page with explanations of all the major features of the language update. If you want something meatier, there have been several resources created by JS experts. Understanding ECMAScript 6 by Nicholas Zakas and Exploring ES6 by Axel Rauschmayer are books that go through the new features in detail. If you prefer video tutorials, Wes Bos has a whole video course on ES6.

Keeping up with new features

Things haven’t been changing as fast since the ES6 release. The first 2 “annual releases” of the ECMAScript spec contained only a combined 8 features. but there are still a lot of new features working their way through the ECMAScript pipeline. If you want to get a quick overview of the features being considered for future releases, the best place to look is the TC39 proposals repo on Github. TC39 is the organization that guides the future of the ECMAScript spec. Proposals go through a 4 stage process, where stage 1 can best be understood as “ideas”, and stage 4 is “confirmed for the next ECMAScript release”. Nicolás Bevacqua put up a good post last week describing this process in depth, as well as giving a brief description of the more advanced proposals in the queue. If you’re interested in keeping up with new proposals but want somebody to walk you through them, I recommend subscribing to Axel Rauschmayer’s 2ality blog. In addition to writing the ES6 book mentioned above, he regularly writes blog posts highlighting new proposals with detailed explanations of the feature implementation and motivation.

Testing new features

Almost all of the higher stage proposals on the TC39 list have been implemented by Babel and you can try them out in the Babel REPL or by setting up a small project that loads in Babel.

Keeping up with the conversation

If you’re a twitter user, probably the easiest way to keep up with language evolution is to follow the people who are shaping and teaching the new language features. Some twitter follow recommendations:

Yehuda Katz - TC39 member, Ember core developer
Axel Rauschmayer - JS blogger and trainer
Wes Bos - JS blogger and trainer
Sebastian Markbåge - TC39 member, React core developer
Henry Zhu - Babel maintainer

From the release of the ECMAScript 5 spec to the ES6/ECMAScript2015 spec, the language wasn’t exactly stable in practice because browser implementations didn’t always move quickly, and there were still many incompatibilities. But standard language syntax stayed stable for quite a while, which allowed the browser ecosystem to iron out some of those kinks prior to the adoption of ES6. ↩︎

Adding RSS, Atom, and JSON Feed to Gatsby

Sat, 03 Jun 2017 21:55:00 -0400

For the last few months, this blog has been written in Gatsby, a static site generator for JavaScript. One of the first problems I had to solve when migrating the blog was how to maintain my RSS feed, since Gatsby doesn’t have any built in concept of an RSS feed. This issue came up again this week as I decided to implement the new JSON Feed spec and give folks an additional option for subscribing to this blog. Fortunately this turns out to be a pretty easy problem to solve because of the excellent npm module feed.

feed is an npm module that takes a bunch of information about a blog and its posts, and normalizes them into various syndication feed format. Here’s what setting up a feed option looks like for this blog:

const fs = require('fs');
const Feed = require('feed');
const filter = require('lodash/filter');
const sortBy = require('lodash/sortBy');
const forEach = require('lodash/forEach');
const get = require('lodash/get');
const moment = require('moment');
const markdownIt = require('markdown-it');
const frontmatter = require('front-matter');


//markdownIt is a markdown parser that takes my raw md files and
//translates them into HTML that we can use in the feed
const md = markdownIt({
  html: true,
  linkify: true,
  typographer: true,
});


// getPageContent is a function to help us grab the html content from a post path
const getPageContent = page => {
  let file = fs.readFileSync(__dirname + '/pages/' + page.requirePath, 'utf-8');
  let body = md.render(frontmatter(file).body);
  // handle local links
  return body.replace(/src="\//g, 'src="http://benmccormick.org/');
};

// build feed is our main function to build a `Feed` object which we
// can then serialize into various formats
const buildFeed = pages => {
  let feed = new Feed({
    title: 'benmccormick.org',
    description: 'A blog by Ben McCormick',
    link: 'http://benmccormick.org',
    id: 'http://benmccormick.org',
    copyright: 'All rights reserved 2016, Ben McCormick',
    feedLinks: {
      atom: 'http://benmccormick.org/atom.xml',
      json: 'http://benmccormick.org/feed.json',
    },
    author: {
      name: 'Ben McCormick',
      email: 'ben@benmccormick.org'
    }
  });
  // ignore pages (non posts)
  pages = filter(pages, p => ( !(get(p, 'data.layout', 'page') === 'page')));
  // we only want the last 10 articles to show up in the feed
  pages = sortBy(pages, page => get(page, 'data.date'));
  pages = pages.reverse();
  pages = pages.slice(0, 10);

  forEach(pages, page => feed.addItem({
    title: page.data.title,
    id: 'https://benmccormick.org' + page.path,
    link: 'https://benmccormick.org' + page.path,
    date: moment(page.data.date).toDate(),
    content: getPageContent(page),
    author: [
      {
        name: 'Ben McCormick',
        email: 'ben@benmccormick.org',
        link: 'https://benmccormick.org'
      }
    ]
  }));
  feed.addContributor({
    name: 'Ben McCormick',
    email: 'ben@benmccormick.org',
    link: 'https://benmccormick.org'
  });
  return feed;
};

Now we have a buildFeed method that takes a list of pages and generates a feed object. Then we can use the feed object we’ve created to actually build write some files. We’ll trigger that in the postBuild callback of gatsby-node. Here was my first version to support RSS and Atom:


//mkDir and mkFile are just light wrappers around node's file system APIs
let createRSSFolder = () => mkDir('/public/rss/');
let generateAtomFeed = (feed) => mkFile('/public/atom.xml', feed.atom1());
let generateRSS = (feed) => mkFile('/public/rss/index.xml', feed.rss2());

exports.postBuild = pages => {
  let feed = buildFeed(pages);
  createRSSFolder();
  generateAtomFeed(feed);
  generateRSS(feed);
};

Prior to this week, feed didn’t support JSON Feed, so I had stopped here. But because JSON Feed is super simple to implement (its just JSON!), I decided to try to add it to feed this week. It turned out it was as simple as adding a single function that looks like this:

json1() {
   const { options, items } = this
   let feed = {
     version: 'https://jsonfeed.org/version/1',
     title: options.title,
   };

   if (options.link) {
     feed.home_page_url = options.link;
   }

   if (options.feedLinks && options.feedLinks.json) {
     feed.feed_url = options.feedLinks.json;
   }

   if (options.description) {
     feed.description = options.description;
   }


   if (options.image) {
     feed.icon = options.image;
   }

   if (options.author) {
     feed.author = {};
     if (options.author.name) {
         feed.author.name = options.author.name;
     }
     if (options.author.link) {
         feed.author.url = options.author.link;
     }
   }
   feed.items = items.map(item => {
     let feedItem = {
       id: item.id,
       // json_feed distinguishes between html and text content
       // but since we only take a single type, we'll assume HTML
       html_content: item.content,
     }
     if (item.link) {
       feedItem.url = item.link;
     }
     if(item.title) {
       feedItem.title = item.title;
     }
     if (item.description) {
       feedItem.summary = item.description;
     }

     if (item.image) {
       feedItem.image = item.image
     }

     if (item.date) {
       feedItem.date_modified = this.ISODateString(item.date);
     }
     if (item.published) {
       feedItem.date_published = this.ISODateString(item.published);
     }

     if (item.author) {
       let author = item.author;
       if (author instanceof Array) {
         // json feed only supports 1 author per post
         author = author[0];
       }
       feedItem.author = {};
       if (author.name) {
           feedItem.author.name = author.name;
       }
       if (author.link) {
           feedItem.author.url = author.link;
       }
     }
     return feedItem;
   });

   return JSON.stringify(feed, null, 4);
 }

My pull request was accepted, so feed v1.1.0 now supports JSON feed and my final code to add rss, atom, and json feed to this site looks like this:

const fs = require('fs');
const Feed = require('feed');
const filter = require('lodash/filter');
const sortBy = require('lodash/sortBy');
const forEach = require('lodash/forEach');
const get = require('lodash/get');
const moment = require('moment');
const markdownIt = require('markdown-it');
const frontmatter = require('front-matter');
const {mkDir, mkFile} = require('./utils/file_system');


const md = markdownIt({
 html: true,
 linkify: true,
 typographer: true,
});

const extractMarkdownContent = file => {
 let body = md.render(frontmatter(file).body);
 // handle local links
 return body.replace(/src="\//g, 'src="http://benmccormick.org/');

};

const getPageContent = page => {
 let file = fs.readFileSync(__dirname + '/pages/' + page.requirePath, 'utf-8');
 return extractMarkdownContent(file);
};

const buildFeed = pages => {
 let feed = new Feed({
   title: 'benmccormick.org',
   description: 'A blog by Ben McCormick',
   link: 'http://benmccormick.org',
   id: 'http://benmccormick.org',
   copyright: 'All rights reserved 2016, Ben McCormick',
   feedLinks: {
     atom: 'http://benmccormick.org/atom.xml',
     json: 'http://benmccormick.org/feed.json',
   },
   author: {
     name: 'Ben McCormick',
     email: 'ben@benmccormick.org'
   }
 });
 // ignore pages (non posts)
 pages = filter(pages, p => ( !(get(p, 'data.layout', 'page') === 'page')));
 // we only want the last 10 articles to show up in the feed
 pages = sortBy(pages, page => get(page, 'data.date'));
 pages = pages.reverse();
 pages = pages.slice(0, 10);

 forEach(pages, page => feed.addItem({
   title: page.data.title,
   id: 'https://benmccormick.org' + page.path,
   link: 'https://benmccormick.org' + page.path,
   date: moment(page.data.date).toDate(),
   content: getPageContent(page),
   author: [
     {
       name: 'Ben McCormick',
       email: 'ben@benmccormick.org',
       link: 'https://benmccormick.org'
     }
   ]
 }));
 feed.addContributor({
   name: 'Ben McCormick',
   email: 'ben@benmccormick.org',
   link: 'https://benmccormick.org'
 });
 return feed;
};

let createRSSFolder = () => mkDir('/public/rss/');
let generateAtomFeed = (feed) => mkFile('/public/atom.xml', feed.atom1());
let generateRSS = (feed) => mkFile('/public/rss/index.xml', feed.rss2());
let generateJSONFeed = (feed) => mkFile('/public/feed.json', feed.json1());

exports.postBuild = pages => {
  let feed = buildFeed(pages);
  createRSSFolder();
  generateAtomFeed(feed);
  generateRSS(feed);
  generateJSONFeed(feed);
};

More Resources

You can see the whole spec for JSON Feed on their site. It really is remarkably simple. They also have a list of implementations and example sites.

Atom Productivity Tips

Mon, 29 May 2017 23:20:00 -0400

Atom is a modern hackable text editor. I’ve been using it as my primary editor for a year and a half, and have picked up some tips along the way that I thought were worth sharing. These tips assume that you have at least a passing familiarity with Atom: they’re the things that I’ve picked up over time that have made me more productive.

1. There is more to the Fuzzy Finder than meets the eye

Fuzzy finders have been a key feature on modern editors since TextMate first popularized the ⌘T command¹ for searching through project files. ⌘T (or ⌘P) still works great in Atom, but what you may not know is that Atom provides 2 other versions of the command. ⌘B searches only among currently open files. ⇧⌘B searches only among files that are “dirty” according to git². Both of those options are built in. You can also find other uses for the fuzzy-finder if you start exploring plugins, including atom-fuzzy-grep, a plugin that lets you use the fuzzy-finder interface to search file contents using grep, ack, ag, or ripgrep.

2. Atom is a great Vim replacement

If you’ve been following this blog for a while, you know I’ve written a lot about Vim. I still love the philosophy of Vim, and think that the text editing language it implements is the best way to edit code quickly. But Atom’s vim-mode-plus is the best vi implementation I’ve seen outside of Vim, and unlike other editors, you don’t need to get frustrated with a halfway approach to Vi-keybindings, since you can make pretty much everything work in a vim-like way. That’s because:

3. Atom’s keybindings are powerful and flexible

Any command exposed by the core editor or a plugin in Atom can be bound to a keyboard shortcut. None of the default commands are hard-coded, and the editor provides powerful contexts for controlling what keys have lead to what action when. That means if I want to use ctrl-j/k/l/h to control my navigation between split panes, but also use ctrl-j/k to move up and down in the autocomplete widget, I can just add these lines to my keymap.cson file

'atom-text-editor, body':
  'ctrl-k': 'window:focus-pane-above'
  'ctrl-j': 'window:focus-pane-below'
  'ctrl-h': 'window:focus-pane-on-left'
  'ctrl-l': 'window:focus-pane-on-right'

'body atom-text-editor.autocomplete-active':
  'ctrl-k': 'autocomplete-plus:move-up'
  'ctrl-j': 'autocomplete-plus:move-down'

Different commands are triggered in different context. You can also take advantage of this with vim-mode to have “normal mode only” keyboard shortcuts like this:

'atom-text-editor.vim-mode-plus.normal-mode':
  '] b': 'pane:show-next-item'
  '[ b': 'pane:show-previous-item'

That implements a behavior I’d missed from Vim’s unimpaired plugin of moving between open files with [b and ]b.

4. Make sure to sync your settings

One area where Atom is less ideal than Vim is setting portability. It doesn’t have a single file where all configuration is stored, or a nice settings folder that can be synced. Instead settings are intermingled with other data in a ~/.atom folder, a fact that is obscured by the editor’s UI, which attempts to put a nice UI in front of all of these setting files and steer you away from editing them directly or caring about where they’re stored. Which works great until you need to move your config to a new machine or sync between 2 machines.

Fortunately, there’s a great plugin sync-settings that makes this simple. It does require setting up a gist to serve as your repo, but after that the process is painless and allows you to continue on unaware of the implementation details behind settings.

5. Check Timecop and clean up plugins that you’re not using

Atom is built on web technologies. This is both the best and worst thing about Atom. It leads to its greatest strengths: hackability, a focus on UX early on since they didn’t have to build a whole UI engine from scratch, and quick iteration. But it also has very real tradeoffs, the biggest of which are battery life and startup performance³. There’s not much to be done about the battery life drain; Atom is based on Chromium, and Chromium is a battery hog. It is simply a tradeoff to understand when choosing an editor. But startup performance is often greatly influenced by the packages that you choose to use. Fortunately Atom includes a simple tool for tracking this called Timecop.

You can find Timecop in the command palette (⇧⌘P by default). When you run it, Timecop displays a report of all of your packages and themes and how they contribute to startup time. It will look something like this:

I wouldn’t recommend obsessing too much over timecop. For most packages the benefit you receive is going to significantly outweigh any startup burden. But if your editor starts to feel sluggish, this is a great first step to figuring out if there’s anything you can drop to make an impact.

6. The UI is just HTML and CSS, and that means you can edit it

Everything you see in an Atom UI is an HTML element, with classes that you can use to select and style them. This fact is mostly relevant when creating themes, but you can also use it to make one-off UI adjustments. For instance, when I edit Markdown files, the editorconfig plugin I have installed warns me about a possible conflict with another atom package. The warning shows up as a small colored mouse icon in the status bar.

I find this warning distracting. I know what its there for and have decided I don’t care about it, but I can’t dismiss it normally. So instead I can take advantage of Atom’s flexibility to just get rid of it. All I need to do is open up devtools (⇧⌘I) and find the class.

Then we can edit our styles.less file. Atom will open it for you if you click the your stylesheet link at the top of the Settings>Themes page. My edit to remove the mouse icon looks like this:

.aec-icon-mouse {
  display: none;
}

More Resources

The atom documentation is really great, and worth diving into if you want to go deeper into Atom
If you’re interested in learning more about keyboard driven text editing, check out my series on Vim.

I’m using MacOS default keyboard shortcuts for this post. Windows and Linux shortcuts are usually similar with ctrl often swapped for ⌘. ↩︎
Dirty files are files that have been edited and not yet added to the index or committed. ↩︎
I’ve definitely seen post-startup perf issues at points as well, but I suspect that those are implementation problems that will continue to improve as the editor matures and the underlying Chromium platform continues to push on performance. This may be naive. ↩︎

Mariana Syntax Theme For Atom

Sun, 28 May 2017 00:05:00 -0400

Sometimes the best thing to do is steal somebody else’s good idea. I’ve been using Atom as my primary text editor for a year and a half now, but I still occasionally play with other editors and keep up with their updates. There are a crazy number of good cross platform editors these days, and they all have a bunch of good ideas that get shared and remixed across each other. This week Sublime Text came out with a new release Dev Build 3132 that included 3 new color schemes. One of them was Mariana, which I believe is a variation on the also great Oceanic Next, with possibly some influences from my previous favorite dark theme Tommorrow Night Eighties ¹. It stood out to me right away.

The Mariana syntax theme that shipped in the new Sublime Text is really nice pic.twitter.com/xbmjBTNoWY
— Ben McCormick (@_benmccormick) May 26, 2017

Since I’m no longer using Sublime Text, I decided to port the theme to Atom. I built on the work of the great atom-material-syntax package to build out a new theme. It turns out to be surprisingly easy to build a color scheme in Atom. Fork an existing theme, change some colors and the details in package.json, delete a bunch of git tags ², and then run apm deploy minor, and you’re in business.

You can now download mariana-syntax in Atom. This was a quick port. I’ll be working to optimize it for the languages I use (JS, Python, CSS/Less, HTML and Markdown) over the next few weeks. I would love to see pull requests for other languages or for anything that I’ve missed so far. I look forward to this just getting better and better. Update: Version 0.2.0 has shipped with optimized syntax for JavaScript, Python, CSS, JSON and Markdown.

To be clear, I’m not sure how great of an influence this was. The scheme definitely bears a strong resemblance to both of those color schemes, and the release notes credit both authors as inspiration for the 3 new syntax-themes without naming specific schemes. ↩︎
This Stack Overflow article was helpful https://stackoverflow.com/a/15755041/1424361 ↩︎

Building Normal Distribution Charts Using Highcharts

Thu, 11 May 2017 22:30:00 -0400

This week I got to have a little fun remembering some High School math, and using one of my favorite libraries: Highcharts. Highcharts is a commercial JavaScript data-viz library, and it makes most standard charts and graphs a breeze to implement. I was using it to plot a normal distribution in order to display a 95% confidence interval. It’s a pretty straightforward problem, but its a nice example of a practical use of Highcharts, and comes with a little stats math that many of us may have learned and forgotten.

The Problem

The goal was to display a normal distribution to represent a 95% confidence interval, given the upper and lower bounds of the confidence interval. A normal distribution is the classic bell curve, where the area under any section of the graph represents the probability that the true value of the measure being tracked is within that range. These curves are usually represented something like this:

The x axis of the curve represents the range of values, while the y axis is a function of x that shows the relative probability for different areas. Data sets that are normally distributed conform to the above bell curve and the 68-95-99.7 rule. There’s a 68% chance that the true value is within 1 standard deviation of the middle of the curve (the mean), a 95% chance that the true value is within 2 standard deviations, and a 99.7% chance that the true value is within 3 standard deviations.

Step 1: Generating a Dataset

Highcharts operates by plotting a set of data points, not by handling equations directly. So our first step is computing a set of data points to plot. We can find the equation for calculating a normal distribution’s probability density on Wikipedia.

Yep, we get to play with an equation that has both π and e in it. I hope this feels sufficiently like a high school math class now. But we can actually choose to drop the left side of the equation. It is the normalization constant for the equation: it ensures that the total area under the curve equals 1, but doesn’t change the shape of the curve. Since we’re simply displaying the shape of the graph and care primarily about showing the range along the x axis, we can ignore it and instead use this function:

const normalY = (x, mean, stdDev) => Math.exp((-0.5) * Math.pow((x - mean) / stdDev, 2));

That’s a good start, but right now we only have an upper and lower bound value for a 95% confidence interval. So before we can apply the equation, we need to find a mean, a standard deviation, and a set of x values to run the formula against. The mean and standard deviation are fairly straightforward, and we can use lodash to generate a set of points. For the sake of this example, we’ll plan on generating 100 points ranging from +/- 5 standard deviations from the mean, which should allow us to see all meaningful parts of the distribution, and still have a bit of padding.

const getMean = (lowerBound, upperBound) => (upperBound + lowerBound) / 2;

// distance between mean and each bound of a 95% confidence interval
// is 2 stdDeviation, so distance between the bounds is 4
const getStdDeviation = (lowerBound, upperBound) => (upperBound - lowerBound) / 4;


const generatePoints = (lowerBound, upperBound) => {
  let stdDev = getStdDeviation(lowerBound, upperBound);
  let min = lowerBound - 2 * stdDev;
  let max = upperBound + 2 * stdDev;
  let unit = max - min / 100;
  return _.range(min, max, unit);
}

Now that we have the x axis locations for the points, we can generate a whole data series.

let mean = getMean(lowerBound, upperBound);
let stdDev = getStdDeviation(lowerBound, upperBound);
let points = generatePoints(lowerBound, upperBound);


let seriesData = points.map(x => ({ x, y: normalY(x, mean, stdDev)}));

Step 2: Creating the Chart

Now we have what we need to generate a chart! Given the data, It’s easy to make a simple normal distribution curve with highcharts. All we need to do is create a <div id="container"> element and then run this JavaScript:

Highcharts.chart('container', {
    chart: {
        type: 'area'
    },
    series: [{
        data: seriesData,
    }],
});

Which produces the following graph:

Thats a good start, but contains a lot of “stuff” that Highcharts provides by default that we don’t necessarily want for our normal distribution illustration. Let’s cut the title, legend, y-axis, and tooltips/mouse action. To do that we need a bit more configuration:

Highcharts.chart('container', {
    chart: {
        type: 'area',
        height: 300,
    },
    title: {
        text: ''
    },
    yAxis: {
      labels: {
        enabled: false,   
      },
      gridLineWidth: 0,
      title: ''
    },
    tooltip: {
       enabled: false,
    },
    legend: {
      enabled: false,
    },
    series: [{
        data: seriesData,
    }],
    plotOptions: {
      area: {
        enableMouseTracking: false,
      }
    }
});

This is pretty great, but we still want to show our confidence interval like the wikipedia image at the top. Ideally we’d only be filling in the areas within the 95% curve. Thats pretty easy to do with Highcharts. We can use zones, Highcharts way of applying different styles to different sections of the chart. We want our zones to apply to the x axis, and we’ll define 3 zones: 0 to the lower bound, lower bound to the upper bound, and everything else. In Highcharts that looks like this:

{
  plotOptions: {
    area: {
      zones: [{
        //fillColor gets the inside of the graph, color would change the lines
        fillColor: 'white',
        // everything below this value has this style applied to it
        value: lowerBound,
      },{
        value: upperBound,
      },{
        fillColor: 'white',
      }]
    },
  },
}

We also want to show a 95% label on the graph like the image at the start. There are multiple ways to do that in Highcharts, but for this simple case we’ll just abuse the title attribute and move it down into the middle of the chart:

{
    title: {
        text: '95%',
        y: 200,
    },
}

And with that we’ve taken our original inputs and produced a dynamic illustration of a confidence interval that matches the image we started with.

More Resources

Highcharts has great documentation and you can see more demos on their site, both for Highcharts and their stock and map chart variants. Note that Highcharts is not free for commercial use. I’ve found that it is well worth the money though for any team that is going to be doing many data visualizations, especially if its using relatively standard forms of data visualizations on variable data.
If you need more freeform visualizations, d3 is a great place to look. Here’s an example of D3 being used to draw a similar chart: http://bl.ocks.org/phil-pedruco/88cb8a51cdce45f13c7e It’s more involved than the Highcharts example, but allows for more flexibility as a result. And unlike Highcharts, its free to use for anything.

Grading Applications On Config Portability

Fri, 14 Apr 2017 17:30:00 -0400

This week I got a new laptop, and for the first time in a while I have separate work and personal computers. One thing that this has made clear to me is that it is still non-trivial to keep settings and configurations easily up to date between 2 different PCs.

In my experience there are several levels of “portability” for applications, which are easy to think of as letter grades.

A - Seamless

These applications have built in mechanisms for syncing configs and or data between 2 different environments. When something is updated on one machine, it is either immediately updated on the other, or easy to pull, without having to remember to update anything or doing any previous setup.

B - Easy

B grade applications are similar to seamless applications, but may have a few more potholes. Possibly there is some involved setup required to get syncing working initially, or some small chunks of the application configuration don’t sync.

C - Workable

At this point we’ve moved out of the range of easy syncing solutions. Here it may be possible to import and export settings from a file, without any automatic syncing. Or there may be a straightforward way to automate the setup, but no declarative representation of the config state.

D - Hard

For some applications there is no clear way to sync or import state, but it is probably still possible through scripting if you’re willing to commit to it.

F - Impossible

Some applications have proprietary settings that can’t be exported or scripted. There is no meaningful way to share settings for these programs.

A mixed bag

Setting up my new laptop this week, I found that the software I use a lot was all over the map on this scale. Here are the grades:

Atom - B

Atom doesn’t have native syncing persay (though most configs are in a ~/.atom folder that can possibly be synced with git). But it has a very nice sync-settings plugin that uses gists to sync settings across multiple machines. It loses points due to a little bit of upfront pain (when setting up a new machine you need to either create a new gist or find your old one, and you need to generate github auth keys for each instance), but gains them back with a very nice set of options to selectively restore or choose when to backup, and painless support for all of Atom’s 3rd party plugiuns

Sublime Text - B+

Sublime Text saves all of its configurations as JSON, so its pretty easy to move them over. It loses a few points though for making you figure out how to sync them, and a few more points for sticking user configs by default in the ~/Library/Application Support/ directory, where it is a bit more difficult to sync using things like git and dotfiles.

Command Line programs - A-

Command line programs like Vim, Ack, bash/zsh/fish, and git all nail the idea of declarative configuration files that are stored in a common place. They fall short of a pure A since they don’t provide a sync solution themselves (that would be very non-unixy), but since it is easy to set up dotfile repos to sync and backup these settings, they qualify for a strong A-.

iTerm2 - C

iterm2 is probably the most popular terminal replacement on the mac. It has tons of settings that you normally edit through a GUI menu. It does allow you to export those settings, and import them on another machine. So you need to have access to the original file to generate a snapshot at any given time and need a way to transfer the file(s), but otherwise this is still pretty workable.

Hyper - A-

Hyper is a newer Electron-based terminal application for OSX. Like Sublime Text, it defaults to offering a GUI interface into text based configuration files. Unlike Sublime Text, it stores that file and its plugin files in the users home directory where they can easily be included in a dotfile repo.

MacOS - D-

Unlike most of the applications I run on it, MacOS is extremely unfriendly to syncing configurations. There are a few settings I want the same on all my laptops. Touchpad scrolling should be in the “unnatural”¹ direction, keys should repeat when held, and caps log should be disabled/mapped to control. There is a set of apps that I would always want to have installed. There is no declarative way to configure this, or sync it across computers. Apple’s preferred strategy seems to be having folks use time machine backups, which is great for setting up a new computer for the same purpose as an old one when it is being replaced, but unhelpful when 2 laptops are being used by the same person for different purposes. It is possible to script the setup of a new laptop with scripts like this one but its very involved and when you update a setting the “normal MacOS way”, its not always clear how you would add that setting to a script. Plus a “from scratch” script won’t help keep 2 machines aligned as their configs diverge over time.

Cloud Services - A

Unsurprisingly, the one class of software that really nails this experience is cloud software. For services like Dropbox, Bear, Evernote, Slack, email & calendar clients, etc, most if not all settings and data is stored on servers somewhere else, so getting a new machine up to date and staying in sync over time is just a matter of entering a password. There are reasons to dislike cloud services, but when moving to a new machine, they’re a thing of beauty.

Takeaways

If you are going to be using 2 computers for different purposes as a developer, take the time to setup a good dotfiles repo that contains the configs for the command line programs and editors you use. These days if you do that, pretty much everything else but the operating system will come right along.

Honestly the wording around that feature is pretty insulting. ↩︎

The Mystery Of Docker And The Disk-Eating Cow

Tue, 28 Mar 2017 10:40:00 -0400

Yesterday morning I was innocently minding my own business, downloading some files, when I noticed that nearly all the disk space on my 256GB work laptop had been consumed. This seemed rather unlikely to me, given that I’d only had the laptop for about a year and I didn’t store anything other than code and work documents on it. Text files just don’t take up that much space most of the time. So I decided to make a purchase I’d considered for a while, bought Daisy Disk, and began investigating.

The first thing I noticed was a huge amount of the disk space (about half) was taken up by the ~/Library/Containers folder. That folder contained my email history and also data on my Docker containers. Docker functions as a lightweight VM, and essentially holds copies of virtualized operating systems and file systems inside each docker container and image, so it made sense to me that it could be taking up a lot of space, though >120GB still seemed wrong for my paltry 4 containers. So my first step was to delete all of the containers and images on my laptop. That cleared about 20GB of space but still left my drive looking like this:

At this point, I was annoyed. 103.5 GB was taken up by some folder called Docker.qcow2 and Daisy Disk wouldn’t show me what was inside. So I drilled in on the file system and immediately found out that Docker.qcow2 was not a directory like I’d assumed due to size, but a single 100+GB file. At that point I decided to poll my teammates to see if I was the only one dealing with this:

So between the 5 of us, we each had “cow files” taking up between 23 and 103GB of disk space. Some Googling revealed a github thread that showed this is a so far unsolved issue with Docker For Mac. Summary: qcow2 files are a format for saving disk images. Docker For Mac’s implementation works well for writing and updating images, but doesn’t automatically free up disk space when a container or image is deleted. So as you use and delete containers over time this file gradually grows. I have a habit of deleting and recreating containers when I’m trying to debug something, which explains why I had a much larger file than others. There isn’t currently a true fix for this issue, but you can delete the file. You’ll lose all your containers and images, but when you recreate them the file will be small again. There also is supposed to be some automatic limiting of the file. The current cap is 64GB, but there is work on making it configurable. Unfortunately the current cap is not respected if you had previously used docker-machine to control docker on your machine, which is how my file exceeded the cap. I’m also unclear on what happens when the file hits the cap. I get the impression that things stop working and you need to delete everything anyway (you just avoid running out of disk space on the host machine).

Fortunately it was no problem for me to delete everything, and so I was able to clear things out, at which point my disk looked a lot happier:

Lessons Learned

I had 3 takeaways from this interesting adventure.

If you’re using Docker For Mac, keep an eye on your disk space. If you’re able to occasionally delete and recreate your containers without data loss, consider occasionally doing that and deleting the cow file. If you can’t do that, be careful how many containers you add and delete, and make sure you manage your disk space well.
Daisy Disk is awesome and highly recommended. It’s an example of a rare breed: the beautiful system utility. The visualizations it shows are both pretty and useful; it made diagnosing this issue a breeze.
One more thing I learned from my coworkers slack yesterday: ls takes an -h argument that shows file sizes in KBs/MBs/GBs instead of all in bytes. This is super helpful when examining large files. Compare the 2 lists of files from my Downloads folder in the image below. The normal form is very nice for comparing 2 files side by side and seeing which one is bigger, but the second form is much more helpful when you want to get an idea of exactly how big something is, or communicate it to others. Most of us don’t think about file sizes in terms of bytes anymore.

Running Jest Tests Before Each Git Commit

Sun, 26 Feb 2017 19:00:00 -0400

My main work project makes heavy use of Jest to test our JavaScript code. For a while now I’ve wanted to set up a way to run tests every time I run a commit. I knew that git provides hooks that allow scripting actions to occur before or after any commit or push, and in fact we were already using a pre-commit hook script to lint our code with ESLint. But it was non-obvious how to make that work well with Jest testing. I eventually figured out a setup that worked, and the found a better way to do both the Jest and ESLint testing. Since it took me a while to work through, I thought I’d share it here and save the rest of you some time.

What didn’t work

A naive approach to this problem would be to set up a pre-commit hook that simply ran jest to run all tests. The problem is that running our full test suite currently takes between 10 and 20 seconds to run all tests and that time is increasing as we grow our test suite. Adding that overhead to every commit would cost my team a lot of time, and would be especially inefficient since the repo contains plenty non-JavaScript code that doesn’t require tests to be run when updated.

For our ESLint hook, we queried git to get a list of staged files, and then ran eslint against each one of them individually, displaying a pass/fail message. That looked something like this:

STAGED_FILES=$(git diff --cached --name-only --diff-filter=ACM | grep "js$")
if [[ "$STAGED_FILES" = "" ]]; then
    exit 0
fi
PASS=true
for FILE in $STAGED_FILES
do
eslint –quiet “$FILE”
if [[ "$?" == 0 ]]; then
    echo "\t\033[32mESLint Passed: $FILE\033[0m"
else
    echo "\t\033[41mESLint Failed: $FILE\033[0m"
    PASS=false
fi

done

This works great for ESLint, but doesn’t work out of the box for Jest, because I don’t want Jest to run the files that changed, I want them to run any tests that changed AND any tests that might have been broken because of that.

Jest has a wonderful command line flag jest --onlyChanged/jest -o that runs only the tests related to files that have been changed according to git. It is extremely helpful since it is smart enough to read the dependency structure for the project and run all tests that might be changed from updating a source file. It also has a --lastCommit option that does the same thing for files that were in the previous commit. Unfortunately, these options aren’t helpful at the point of committing, since onlyChanged does not look at files that have been staged for commit, and we haven’t actually made a commit yet for lastCommit to read.

What Did Work

Fortunately Jest has a lower level command that uses the same logic as onlyChanged and lastCommit. --findRelatedTests is a flag that tells Jest to run any tests related to the files passed to it instead of trying to run those files as tests as it would normally do.

This is a perfect fit for a pre-commit hook. I was able to integrate it into my existing script like this:

jest --bail --findRelatedTests $STAGED_FILES
if [[ "$?" == 0 ]]; then
    echo "\t\033[32mJest Tests Passed\033[0m"
else
    echo "\t\033[41mJest Tests Failed\033[0m"
    PASS=false
fi

$STAGED_FILES is re-used from the eslint portion of the script, and is just a space delimited list of files that are being committed. The --bail option simply stops running tests as soon as one has failed. Including that is optional, you won’t see all the tests that have failed, but the failure will happen faster and you’ll be able to decide how to proceed, including possibly running the full test script on your own.

Those lines (along with some sanity checks for the existence of Jest and error handling when PASS is set to false) are enough to get a workable commit hook going, but they’re not ideal. pre-commit hooks aren’t persisted by git, so each user has to install the hook script individually, and any updates aren’t shared automatically. Plus I’m inefficiently passing all staged files to eslint and Jest regardless of whether they’re JavaScript that those tools are actually meant to work on. My ESLint code was also written before ESLint developed robust --fix capabilities, and doesn’t try to fix the errors it is capable of fixing. Finally, while this is just poor coding and not an inherent limitation of my other method, I’m using globally installed versions of Jest and ESLint instead of scoping them to my project.

Making it better

Fortunately I’d discovered a better solution the other day while working on something else. There is an npm package for making this process easier, lint-staged. Lint staged abstracts away the boilerplate of getting the staged files, and makes it easy to run local node executables against specific sets of files. I was able to replace my whole pre-commit script and address all of the problems mentioned above with only a few lines in my package.json:

{
  "scripts": {
    "lint-staged": "lint-staged",
  },
  "lint-staged": {
    "*.js": [
      "eslint --fix",
      "git add",
      "jest --bail --findRelatedTests"
    ]
  },
  "devDependencies": {
    "eslint": "3.16.1",
    "jest": "19.0.1",
    "lint-staged": "3.3.1",
  },
}

lint-staged now handles making sure that I’m only running checks against JavaScript files, lets eslint autofix and stage any files that it can fix, and uses the local versions of jest and eslint. It also adds a nice pretty interface for the results:

lint-staged docs recommend using an npm based hooks manager like pre-commit for running the scripts, but due to some quirks in my setup (node_modules is symlinked, and I sometimes want to run these commands inside a docker container), I found it easier to maintain a custom pre-commit script and just replace the previous logic with npm run lint-staged. Either way, all of the actual logic is now checked into the repository and shared between all users. The only manual step is adding the call to a pre-commit hook or (if you’re using a helper lib) running npm install.

More Resources

This maybe merits a future post on its own, but if you’re interested in automating away your code style maintenance look at combining prettier (and possibly eslint-plugin-prettier) with the pre-commit hooks from above. Prettier will guarantee that your code follows a consistent style, and because it handles maximum line lengths, is much more robust than ESLint on its own. Using its ESLint plugin within my editor and on pre-commit hooks has pretty much eliminated me manually fixing code style problems.

Giving Context To Best Practices

Sat, 18 Feb 2017 19:30:00 -0400

I’ve noticed a frustrating trend in online discussions about web development where developers take their personal experience with a tool or practice and promote it as a universal truth. Some of this is the nature of being human: we all see the world through our own lens. But the best technical observations exist within a specific context.

Compare

Everybody should be writing code using static types, the extra syntax is worth it because of all the bugs it prevents.

If you’re working in a large code base with many developers, static typing is extremely useful, because the overhead from writing extra code and compiler errors is more than made up for by reducing bugs and clearly expressing intent when developers edit code they didn’t write.

The first quote presents a tradeoff and declares a universal preference for one direction in the tradeoff. The second quote adds more details about the context in which the practice has been beneficial, and explains what about those circumstances (developers editing code they didn’t write) affects the tradeoff decision.

These type of distinctions are often lost in online discussions about development best practices. I have no interest in naming and shaming, but specifically I’ve seen this in discussions around JavaScript performance, testing, frameworks vs other frameworks vs VanillaJS, responsive design, progressive enhancement, and build pipelines. For this post, I’m going to try to lay out a list of some of the context that I’ve found helpful to take into account when giving or interpreting best practice advice.

Context to keep in mind when discussing and interpreting best practices

Beginner vs Veteran

I’ll start with a distinction that folks seem better about making. Sometimes recommendations that make sense for beginners don’t work as well for more experienced developers and vice versa. For instance it might make sense to advise a more experienced developer to use and heavily configure a build system like Webpack for a new project. But beginners already have a lot to learn with each new project and might benefit from a simpler system; either a simple project without a build step, or a “batteries included” system like Create React App or Ember CLI. Making these distinctions is important for helping developers who may feel overwhelmed by all the concepts they need to learn when getting started with web development.

Big Team vs Small team vs Individual

Another dynamic that matters when expressing best practices is the size of a team working on a project. If a project is being developed by a large team, or several teams, or is meant to be used as a library by many people, techniques like automated documentation, static type checking, and clear style guidelines may be essential. But for a personal project or a professional project run by a 1 person team, those same techniques may provide a lot of overhead for minimal benefit. Most teams will fall between those extremes and will have to weigh the tradeoffs in each case.

Personal Project vs Internal tool vs B2B Products vs Consumer Products

The audience of a project is also important. Some projects we build for ourselves. Some may be internal for a company, or for a group that we’re a part of. Other projects are sold professionally to businesses or offered freely to consumers on the web. There are different goals and expectations for each of these groups, that may impact which practices and techniques are important. For instance, nobody cares if a personal web application you’ve made to organize your todos crashes in Internet Explorer or doesn’t have alt-text for images. But browser compatibility and accessibility might be extremely important for a consumer web app aimed at a broad population. Similarly, it might be ok for your internal web tool to have un-minified JavaScript and large un-optimized CSS and JavaScript bundles, since it will always be served locally over a fast connection. But it could severely hurt the user experience to do the same things for a high traffic public site.

Web page vs Web app

Distinguishing between web pages¹ and web apps² would short circuit a large number of internet arguments before they started. Specifically arguments about progressive enhancement, whether we use too much JavaScript and whether a site is “acceptable” if it doesn’t work without JavaScript. Many accessibility and performance proponents push very strongly for less JavaScript and sites that work without JavaScript. And that is a good thing… for the sites that can function without JavaScript. Web apps like Trello, Google Docs, or Postman just don’t make sense without JavaScript. But saying that we should use less JavaScript isn’t bad advice, it’s just bad advice when given or taken without context.

Existing project vs Greenfield project

Some choices are only easy to make at the start of a project. It is possible to change the language or framework a project is written in midstream, but it is often extremely risky and painful. So asking a developer working on a 5 year old JavaScript project with thousands of lines of code why they didn’t write it in Elm or ClojureScript is not helpful. But suggesting that somebody write a new project in Elm may be just fine.

Short term project vs Long term project

Some software projects are meant to be written, completed, and never seen again. For instance a promotional site for an event. Some projects, like financial software, are meant to last for decades. This is yet another distinction than can impact best practice discussions. When working on a short term project, speed of delivery and initial quality tend to matter a lot. For long term projects it may be better to prioritize maintainability, security and ease of deployment.

Web page: sites that are mostly content focused with some possible interactivity layered on top ↩︎
Web app: sites that are primarily an interactive experience for the user and behave more like traditional desktop or mobile applications ↩︎

Evaluating Web Apps With Lighthouse

Mon, 13 Feb 2017 23:30:00 -0400

Last year Google coined the term “Progressive Web App” as a way of describing the type of sites and applications that they think are the future of the web. Progressive Web Apps are sites that load quickly, are always responsive to user input, work as well as possible offline, and integrate with native platforms like native apps. While this site is not an “app” per say, I do want it to be as fast and responsive as possible, and I also wanted to better understand what Google is pulling under the PWA umbrella. Fortunately, Google has built a tool Lighthouse to help developers profile and improve their web applications, comparing them to a set of PWA benchmarks. So I took this site through Lighthouse, and worked against their suggestions with the goal of learning more about PWAs and improving the experience of using benmccormick.org. I’m passing along my experience here.

Getting Started

I began by downloading the Lighthouse extension in the Chrome Web Store. Lighthouse can be installed either as a chrome extension or a node command line tool. Both produce the same HTML report, but I chose the chrome extension for the convenience of keeping everything in the browser. When I ran the initial report, I got the following result:

48 out of 100 didn’t seem like a bad starting place. Scrolling down it looked like I was mainly dinged on a few main areas:

I had no offline support
I did’t have any special setup for being included on a users homescreen. (I did have platform appropriate icons, but no app configuration)
I had some possible inefficiencies in the assets I loaded.

Native app support isn’t a huge priority for me, and the assets issue seemed small, so I decided to investigate the offline support issue first.

Offline

I based my service worker implementation off of Simple Offline Site a demo Service Worker repo created by Chris Coyier for an article on CSS Tricks. This was perfect since it’s default behavior was what I wanted: cache everything, and check for updates from the server everytime we return from cache so we’re never out of date for long. You can see what I did specifically in my github repo for this site. After adding the service worker, my score moved up to 63/100.

Finally, Service Workers only work on HTTPS connections, but most links to my site are not https currently. So to take full advantage (and to address another metric I was flagged on), I used a CloudFlare page rule to redirect all visting traffic to use https. That bumped me up to 69/100.

Native Support

As I said previously, native platform support isn’t a major priority for this blog, but I was curious about the process. Getting native support turned out to be very straightforward. I had already collected icons for the various platforms, and created a simple manifest when I originally created a favicon for my site. But the manifest file didn’t have everything and it wasn’t being deployed properly to my site. Fixing that and making it available at /manifest.json, fixed all of the problems related to native platform support.

My manifest file now looks like this:

{
	"name": "benmccormick.org",
  "short_name": "benmccormick",
	"icons": [
		{
			"src": "\/android-chrome-192x192.png?v=yyxgnp97qG",
			"sizes": "192x192",
			"type": "image\/png"
		},
		{
			"src": "\/android-chrome-384x384.png?v=yyxgnp97qG",
			"sizes": "384x384",
			"type": "image\/png"
		}
	],
	"theme_color": "#ffffff",
	"background_color": "#57A3E8",
	"display": "browser"
}

After correctly deploying that manifest file, my score moved all the way up to 100/100!

I clearly benefited from starting in a good spot. Gatsby is “fast by default” as a platform for building blogs, and I’ve done my best to avoid making it slow. But it was fun seeing what I could do to make the site a better experience for users with bad connectivity, and to learn more about what Google is trying to do with PWAs.

More Resources

Google has a bunch of resources around creating Progressive Web Apps. You can find them on their PWA landing page.
It was so easy to make my blog a PWA mostly because it is built on Gatsby. Worth looking at if you’re a web developer who blogs.

MobX: First Impressions

Mon, 09 Jan 2017 09:30:00 -0400

I spent some time around the holidays this year playing with MobX, a state management library for JavaScript. MobX is an unopinionated library that provides a layer over normal JavaScript data structures that allow other code to efficiently observe data changes and update based on what has changed. It’s an interesting tool for handling state in web applications, especially in existing projects that might need to update state handling iteratively. Here are some first impressions.

What is MobX?

MobX has 3 core concepts developers need to understand:

Observable state - MobX provides functions to make data observable. Observable data can be watched by other pieces of code which may efficiently update when the state changes. Observable data is primarily created in MobX using the observable function.
Derivations - Functions that watch observable data are called derivations. MobX has 2 primary types of derivations: computed values and reactions. Computed values update a value based on other data, while reactions produce side effects: updates to a UI, a network call, or a logging statement for example. MobX provides a computed function for defining computed values; in most cases reactions will likely be mostly defined using a framework specific helper library like mobx-react or ng2-mobx. MobX does provide some lower level libraries for reactions though, including autorun and when.
Actions - Code that updates observable state is known as an action. MobX has a formalized version of actions which can be defined using the action function, but it is also possible to modify state directly using any normal JavaScript code and maintain observable behavior.

In practice this looks something like this:

import React from 'react';
import ReactDOM from 'react-dom';
import {observable} from 'mobx';
import {observer} from 'mobx-react';

let digits = observable([1, 2]);

const DigitPrinter = observer(({digits}) => {
  return (<div>
    {digits.join(', ')}
  </div>);
});

ReactDOM.render( <DigitPrinter digits = {digits} />, document.getElementById('root'));

/* prints 1, 2 */


digits.push(3);

/* DOM re-renders to print 1, 2, 3 */

digits[2] = 4;

/* DOM re-renders to print 1, 2, 4 */

The component is able to passively observe changes in the data, and re-render when it changes. But MobX is actually even smarter than this example shows. Let’s look at another example, this time using a more complex data structure (a class).

import React from 'react';
import ReactDOM from 'react-dom';
import {extendObservable} from 'mobx';
import {observer} from 'mobx-react';
import moment from 'moment';

class Appointment {

  constructor() {
    extendObservable(this, {
      dueDate: '01-01-2017',
      title: 'Dinner with Joe'
      location: 'Chik-Fil-A',
      isToday: computed(function() {
        return moment(dueDate, 'MM-DD-YYYY').diff(moment.now(), 'days') === 0;
      }),
    });
  }

  toString() {
    return `${this.title} @ ${this.location}`;
  }
}


const TodayBox = observer(class TodayBox extends React.Component {
  render () {
    let {appointments} = this.props;
    return (<div>
      {appointments.filter(a => a.isToday).map(a => <span>a.toString()</span>)}     
    </div>);
  }
});

let appointment = new Appointment();

ReactDOM.render( <TodayBox appointments = {[appointment]} />, document.getElementById('root'));

/* prints an empty div */

appointment.title = 'Dinner with Bob';

/* Nothing relevant has changed. The component does not re-render */

appointment.dueDate = '01-09-2017';  

/*  assume that 01-09-2017 is "today" */

MobX has the “magical” ability to determine what changes actually affect the observer and not make unnecessary calls of reaction functions. In practice, that magic allows you to write less, more efficient code for updating your UI based on data.

How Does MobX work?

I have a learned skepticism of magical solutions like this. Generally I find that time-saving magic like this ends up costing time when it comes to maintenance, and explicit relationships between code saves time over less-boilerplate heavy code. Fortunately, while the libraries code itself is fairly complex, it isn’t too hard to understand the logic behind how MobX works.

On the Observer side, MobX uses ES5 setters and getters to proxy updates to observable data structure and listen in when data is updated. This allows the type of event listening that Backbone and other libraries provide, without requiring the user to go through special set or get methods to update an object’s properties. Most of the time you should be able to just be able to write code as you normally would, and MobX will make it work. There are some exceptions though that mean its important to actually understand how the library work. Getters and Setters only work when looking up an existing property on an object, so when using primitive values or adding new values to an existing object, some special syntax (the return of get and set methods) may be required. In Mobx primitive observables are referred to as boxed values, and objects that require new values over time can be handled by using observable Maps, which use the API of ES6 Maps.

On the derivative side, reactions and computed values are always defined as functions. MobX wraps these functions and is able to determine (through the method described above) what properties were accessed during each function run. It then only listens for changes to these properties. Thus it doesn’t matter if you theoretically could access thousands of observable properties in a function, if you wrap those references in an if statement to a single property, and that property returns false on first run, the derivative function will only listen for changes on that property before running again. When an observable is updated, all derivative code is run synchronously and atomically so there is no concern of getting into bad intermediate states.

Most of the “magic” is covered by the 2 paragraphs above, and once you understand the ideas behind them (see the resources at the bottom of this post for more in-depth explanations), it is fairly easy to understand the reasoning behind the cases where the magic fails.

Decorators

Developers who actually use MobX are probably questioning my examples by now, because I’ve been using a different syntax than the primary MobX documentation for showing my examples. So let’s talk about decorators. The preferred way (according to the documentation) to write MobX code is to use decorators to define observers and observables. Decorators are a proposed new JavaScript feature for declaratively adding extra behavior to classes, class properties and class methods. Using them with MobX, my above example would look like this ¹:


class Appointment {

  @observable dueDate = '01-01-2017'
  @observable title = 'Dinner with Joe'
  @observable location = 'Chik-Fil-A'

  @computed get isToday() {
    return moment(dueDate, 'MM-DD-YYYY').diff(moment.now(), 'days') === 0;
  }

  toString() {
    return `${this.title} @ ${this.location}`;
  }
}


@observer class TodayBox extends React.Component {
  render () {
    let {appointments} = this.props;
    return (<div>
      {appointments.filter(a => a.isToday).map(a => <span>a.toString()</span>)}     
    </div>);
  }
}

That obviously looks a lot cleaner than my example. So what’s the problem? Again, decorators are a proposed JavaScript feature. Currently they’re in Stage 2, which means roughly that they are very likely to eventually make it into the language, but may change in non-trivial ways before that happens. Currently decorators are implemented in TypeScript, as well as a 3rd party babel plugin, but as of January 2017 are not implemented in any browser engines, and are not implemented in any first party babel plugins due to spec instability. Decorators are a key part of Angular2, and are used by other frameworks including Aurelia and Ember, so I don’t think they’re going away. But I also don’t see them as ready for production use for teams that don’t have the capacity to dedicate the time to update existing code if/when the spec changes². If for one reason or other you’re comfortable with the risks however, decorators are a great way to clean up the MobX API, and I’m excited for the spec to get to the point where I can use them.

Comparison to Redux and setState

As a state management library that was built to be used with React, the obvious questions to ask about MobX are how it improves over React’s built in state handling, and how it compares to Redux, the current most popular state management solution for React. MobX takes a different approach to each.

It’s important to say at the start that MobX isn’t mutually exclusive with using setState. It’s possible to use them together, with MobX managing application data and setState handling individual component/UI state. But there is some advantages to using MobX for UI state as well. The biggest reason is that MobX is smarter than setState about when re-rendering is required. By default React Components re-render on any call to setState, regardless of whether the state change actually affects what is rendered or not. MobX’s creator wrote more about this on Medium.

Comparing MobX and Redux could be a whole article on its own, so I’m not going to go too deep here, but suffice it to say that MobX is more flexible than Redux, but as a result loses some of the benefits of Redux’s structured approach. Specifically, it is easy to convert a single object to become a MobX observable and use it in an existing React component. It can be easily inlined in an existing file and used for one component even. You also don’t have to deal with the ceremony of using immutable data. Redux is intended as a solution that will take over all your state, and doesn’t scale down nearly as well or mix with other solutions ³. However, Redux’s structure also has benefits. It allows easy testings of each component of an app individually, the creation of powerful tooling that will work with any Redux app, and reliable guarantees of how state will be updated. These things are mostly possible with MobX ⁴, but require more developer discipline, and/or can’t be written as easily as universal tools. Overgeneralizing, I would prefer Redux in a new application, but would look to MobX for improving data management in an existing application. MobX also has advantages in applications where you’re displaying a large amount of UI elements based on a relatively small amount of data, or have a lot of derived data (something like a spreadsheet), whereas Redux has benefits for applications that need to be extremely reliable and well tested.

Stray Thoughts

If you do want more discipline in MobX, make sure to use useStrict to require updates to use Actions
MobX has a nice set of devtools that are useful if you’re opting in to using actions, and using React with MobX.
If this isn’t clear yet, I definitely recommend hacking around with Mobx’s low level APIs to learn more about how it works before integrating it straight into a framework
~~Also note that MobX 3.0 is imminent, there is a release candidate that you can play with now.~~ Update: That didn’t take long. MobX 3.0 is out.

More Resources

The two most helpful resources I found for conceptually understanding MobX were in-depth articles by the library’s author Michael Westrate: Becoming fully reactive: an in-depth explanation of MobX on Medium and Concepts & Principles from the MobX documentation
This tweet thread and the linked article, are a good breakdown of the tradeoffs between MobX and Redux

Technically this also uses class properties as well, another proposal that works well with decorators ↩︎
It’s possible any spec changes will be trivially fixable with a code mod script, but right now there is more uncertainty than I am personally comfortable with. I’m grateful to people who are willing to take more risks here than me, they push the language forward. ↩︎
I’m talking here about what is encouraged and easy, not what is possible. It is of course possible to mix in Redux with other solutions. ↩︎
MobX has some nice devtools! ↩︎

Orthogonality and CSS in JS

Mon, 02 Jan 2017 20:00:00 -0400

One of the realities of the constant change in the web development world is that “best practices” are often hotly contested. One issue that the front end community doesn’t seem to have come to a consensus on yet is how tightly to bundle JavaScript, HTML and CSS in code. Should they be bundled and managed completely separately? Kept in different files and imported together into JavaScript component files? Or can we remove HTML and CSS files entirely and generate everything in JavaScript? This is a discussion that still comes up all the time in my twitter feed.

Many new frameworks that encourage a component based architecture (including React, Vue, and Angular2 ¹) provide affordances for including HTML and CSS directly in JavaScript, but those solutions are not yet used universally. Many people are still very happy keeping CSS bundled separately. I have no idea what the best solution is. But I am interested in how people talk about the problem. One major argument I have seen from people advocating both sides is that their preferred approach encourages good “separation of concerns”.

I'm moving from storing my CSS, JS and HTML in different files, to physically putting them on different drives. Keep those concerns separate
— Ben Lesh (@BenLesh) December 23, 2016

I’d like to take a deeper look at this and give some vocabulary that can hopefully help anyone working through discussions on how to manage CSS for components. I’ll focus specifically on CSS since frameworks tend to be more agnostic to how it is blended with JS than they are about HTML, and on components since that is the main context in which people are arguing for blending of css. For this piece, I’ll consider a component to be a set of HTML, CSS, and JavaScript that combine to form a reusable piece of UI ².

Orthogonality, Cohesion, and Coupling

The Pragmatic Programmer, one of the best books ever written on Software Engineering, uses the following three terms when talking about “Separation of Concerns”: orthogonality, cohesion, and coupling.

Orthogonality is the idea that modules should be written in a way that a change in one module should not require changes in any other module.

Two or more things are orthogonal if changes in one do not affect any of the others. In a well-designed system, the database code will be orthogonal to the user interface: you can change the interface without affecting the database, and swap databases without changing the interface.

— The Pragmatic Programmer

Cohesion is a measure of how well the internal contents of a module relate to each other. A cohesive module is one with a single well defined purpose, where all code in the module is related to that purpose. A less cohesive module might have multiple purposes, with pieces of code that are completely unrelated to each other. Robert C. Martin describes this as each module having “a single reason to change”.

Coupling is a measure of how dependent different modules are on the internal workings of other modules. In a loosely coupled system, any module can be completely rewritten as long as it exposes the same public interface, without any other modules needing to change. In a tightly coupled system, changing the internal details of one module may require changes in many other modules.

In the real world most systems are not purely orthogonal, and their modules are likely not 100% cohesive and uncoupled. But striving for these goals is a good approximation of what many people mean when they talk about separation of concerns.

Conways Law

The Pragmatic Programmer does touch on another application of orthogonality to software engineering:

Have you noticed how some project teams are efficient, with everyone knowing what to do and contributing fully, while the members of other teams are constantly bickering and don’t seem able to get out of each other’s way?

Often this is an orthogonality issue. When teams are organized with lots of overlap, members are confused about responsibilities. Every change needs a meeting of the entire team, because any one of them might be affected.

How do you organize teams into groups with well-defined responsibilities and minimal overlap? There’s no simple answer. It depends partly on the project and your analysis of the areas of potential change. It also depends on the people you have available. Our preference is to start by separating infrastructure from application. Each major infrastructure component (database, communications interface, middleware layer, and so on) gets its own subteam. Each obvious division of application functionality is similarly divided. Then we look at the people we have (or plan to have) and adjust the groupings accordingly.

You can get an informal measure of the orthogonality of a project team’s structure. Simply see how many people need to be involved in discussing each change that is requested. The larger the number, the less orthogonal the group. Clearly, an orthogonal team is more efficient.

— The Pragmatic Programmer

The idea is that teams work more efficiently when each can work in its own areas without having to be bogged down by using another group (or individuals) code, and only interacting with those modules through a well communicated interface. This is also known as Conway’s Law, which is often used disparagingly but still stands as a true observation about real life code.

organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations

— Conway’s Law

Although it usually isn’t phrased quite so explicitly, I believe Conway’s law is often related to what people mean when they discuss separation of concerns in front end development.

Boundaries and Interfaces between CSS and JavaScript

Let’s look at 2 examples of how CSS can be structured. We’ll use a “page view counter” as our example. The element will show the number of users who have viewed the page, and respond to click events by showing a modal with the most popular pages on the site. First, for a more traditional example, I’ll show a Marionette View, with separate CSS. For a more integrated example, I’ll use a React component.

Marionette

//view-counter.js

import Mn from 'backbone.marionette';
import template from './view-counter.hbs'
import { getPageViews, showModal } from '../util/page-views';

var ViewCounter = Mn.View.extend({

  template,

  className: 'page-view-counter',

  ui:  {
    'showPageViewsModal': '.show-modal-js',
  },

  events: {
    'click @ui.showPageViewsModal': 'showPageViewsModal',
  }

  templateContext() {
      return {
        pageViews: getPageViews(),
      };
  },

  showPageViewsModal() {
    showModal();
  }

});

{{!view-counter.hbs}}

<span class="page-view-counter__title"> Page Views: </span>
<span class="page-view-counter__counter show-modal-js"> {{pageViews}} </span>

//view-counter.css

.page-view-counter {
  display: flex;
}

.page-view-counter__title {
  font-weight: 700;
  padding: 3px;
}

.page-view-counter__counter {
  padding: 3px;
}

React

//view-counter.jsx
import React from 'react';
import { showModal } from '../util/page-views';

export const PageViewCounter = (props) => {
    return <div style = {{display: 'flex'}}>
      <span style = {{
        fontWeight: 700,
        padding: '3px',
      }}>
        Page Views:
      </span>
      <span style = {{padding: '3px'}}>{this.props.pageViews}</span>
    </div>
}

The React and Marionette examples have set different module boundaries. In the Marionette example, we have defined 3 modules, split by code type. Ignoring the leaky abstractions in the Marionette boundaries ³, we can say we have 3 modules with clear singular purpose (styling, behavior, structure) that use class names and templateContext as interfaces. The handlebars file exposes classes, which the CSS uses to style elements and the JavaScript code uses as attachment points for event handling. The JavaScript view passes data to the template through templateContext. In the React code we have defined a single module that exports a component as its only external interface. The module’s single purpose could be defined as “rendering a PageViewCounter”.

How do we define module boundaries?

If you’re excited to read which of the above examples is the correct module boundary definition, I’m sorry to disappoint you. It turns out that module boundaries are more of an art than a science. Let’s consider each of these examples by the criteria we laid out above.

The Marionette modules are cohesive. Each module is single purpose, with a clear reason why it might change. The React module is also cohesive, as it describes a single atomic component. However, it has more reasons it might change. We might change that module because of a change in the look and feel of the site, because of a change in the expected behavior of the click event, or because we’re changing the text inside the component.

The Marionette modules are not quite decoupled. While this CSS doesn’t nest selectors and we don’t have any explicit dependencies on the HTML structure, it is still written in a way that assumes .page-view-counter__title and .page-view-counter__counter will be direct children of .page-view-counter. So changing the “internal details” of the Handlebars file by adding an extra element around those children would break the CSS. While the modules are not completely decoupled from each other, they don’t rely on any private details of other modules or global styles and can be used together as a reusable component. The React module is similarly decoupled from the rest of the system, and as a single module faces no internal coupling issues.

Both components should be orthogonal from the rest of the system, even though the Marionette modules may be less orthogonal internally. The question of how they meet a Conway’s law style of orthogonality depends on a team. If your team has designers, and developers separately working on style/structure and behavior, the Marionette version may allow for more efficient division of labor, with communication centering on class based communication. If you instead have a group of polyglot front end developers who implement mocks from designers across all 3 areas, the React version will instead present a simpler implementation that maps better to your team, with the focus on interfaces across different components.

In the end decisions like this are an exercise in understanding context and preferences. What will make your team productive? You can accept the coupling of the first example in order to gain the benefits of small focused modules. Or you can take the larger scope of the React component in exchange for keeping all information relevant to a component in one place. Are you making single developers responsible for a set of components? Or are they responsible for behavior generally, with design handled by someone else? Make the decisions that work for your project.

More Resources

The Pragmatic Programmer is a great book. Much of the vocabulary in this post comes from its Chapter 8, but the whole book is worth a read and is highly recommended.
CSS Tricks has a good look at the pros and cons of CSS in JS. I focused on theory here, but this is much more hands on about the practical concerns around this debate.

This is a bit of a generalization. React provides an abstraction over HTML that replaces hand-written HTML, but doesn’t specify anything for CSS, CSS in JS solutions are simply popular in that community. Vue and Angular both allow CSS and HTML to share a file with JS, but CSS can still be handled separately. ↩︎
There are some distinctions about CSS in JS vs importing CSS into JS using webpack that I’m not really dealing with here. This is a post about how to think about these decisions moreso than the specific options for bundling CSS and JS ↩︎
The root HTML element in any Marionette component is always defined implicitly in JavaScript, and Handlebars is an expressive templating language that can handle more than structure. ↩︎

2016 Roundup

Thu, 29 Dec 2016 22:15:00 -0400

Thanks to all of you who have followed my blog this year! It was a busy year and I didn’t get as many posts out as years past, but you all have been as supportive as ever. I’m excited about the things I’ve been able to do with the new site this month, and am looking forward to a great 2017. But first, a look back.

I wrote 15 posts in 2016, down from my pace of 19 in 2015 and 25 in 2014. Some of that was an unusually busy work year, and some of the time I normally put into the site went to the redesdign. I also sat on a few articles that didn’t turn out that great when actually written out. But I made a good run at the end and hopefully will be writing more in 2017. The most read posts from this site in 2016 were:

The Most Interesting Atom Packages I’ve found so far - I started using Atom as my primary text editor late in 2015. This was my roundup of the coolest and most unique plugins I had found for Atom at that point.
Testing with Jest Snapshots - This was my review of Jest’s snapshot testing feature. I found it extremely useful for testing UI components. I also wrote about Jest again this month, transcribing a talk I gave at a local meetup on How Jest can save you time.
Reusable Code Patterns - This article was a high level look at the ways you can approach sharing code for 2 different use cases.

Honorable Mention

ES6 Patterns: Converting Callbacks to Promises - This article was technically written in 2015, but it was published during the last week of the year and thus wasn’t eligible to be included last year. However it has proven quite popular. This was my attempt at a straightforward explanation of how to convert a callback based API to a Promise based one.

Code I wrote in 2016

Most of the code I wrote this year was for my work at Windsor Circle. But I have spent some time getting this blog up and running, and anyone interested in what it takes to get a fully functioning blog running on Gatsby is welcome to take a look at the github repo.

Languages, Libraries, and Technologies I started using in 2016

Jest is a JavaScript testing library that I started using this year. It’s fantastic, and I’ve had the opportunity to write and speak about it a few times already.
Gatsbyjs now powers this blog. Gatsby is a ReactJS based static site generator, and it has been a pleasure to work with.
My team started using Docker this year. It’s been a mixed bag for me personally, but I love the idea of what it provides

Languages, Libraries, and Technologies I stopped using in 2016

Despite my affection for Marionette, I’m no longer actively using it other than in some legacy code for my work application

Languages and Libraries I used (more or less) every day in 2016

JavaScript (ES6), Python, React, PostgreSQL, Backbone, lodash

Languages and Libraries I want to try in 2017

RxJS, Flow, VueJS, Victory, Pandas and other python data analysis libraries, Rust

Blogs I started following in 2016

Nothing new this year,

Blogs I read every post from in 2016

Rands In Repose, Stratechery, Daring Fireball, Marco.org, rauchg.com, purposedworking.com

Podcasts I started listening to in 2016

NPR Politics - Approachable and fun while still being informative, one of the best ways to follow this years election.
How I Built This - This has been a great new podcast on entrepreneurship

Software I started using in 2016

Google Inbox + Airmail for email: I’m not sure I’m ever going to be happy with an email client, but I’m currently using the combo of Google Inbox on iOS and Airmail on MacOS. Post Mailbox, Inbox is my favorite iOS mail app. On desktop though I like to have universal inbox and tight integration with the rest of the operating system, so I use Airmail, which is less bad than the rest of the MacOS mail clients I’ve tried
Copied: Copied is a clipboard manager for MacOS (and sort of iOS). It’s been great to be able to keep multiple things in the clipboard at once, and be able to go back and get something if I blow it away. I’m not a power user. I haven’t really figured out a reason to use their lists features, or their iOS client, but it has been great for the limited things I use it for.
Homebrew Cask: I’ve used homebrew for a while. But being able to install GUI apps as easily as command line apps has been pretty amazing. If you are on OSX and don’t use homebrew to install pretty much everything, you’re missing out. Of course the real effect is that it makes me dread and hate the Mac App Store even more.
The New York Times App: Ok this is a bit of a cheat since this is more about content than software, but if you’re out of touch on what is going on in the world, there’s never been a better time to support a good source of news and reporting, whichever outlet is your preference.

Software I stopped using in 2016

Nothing I’m aware of, though I am currently checking out Bear and Annotate as potential replacements for Evernote and Skitch.

Software I used (nearly) every day in 2016

MacOS/iOS, Google Search, Chrome/Safari, Google Inbox/Airmail, Twitter/Tweetbot, Feedbin/Reeder, Instapaper, iMessage, Slack, Trello, 1Password, Atom/Vim, iTerm, Fish Shell, tmux, Bitbucket, Fantastical, Spotify, Evernote, Skitch, Dash

What are Higher Order Components?

Wed, 14 Dec 2016 10:30:00 -0400

If you spend much time at all reading through articles and documentation around React, you’re likely to run into the term “Higher Order component”, often abbreviated HoC. Since it is often used without context in React discussions, and is not a particularly intuitive term for many (most?) developers, I thought it was worth publishing a quick and simple explanation of what Higher Order Components are.

Higher Order Components are functions that take a React Component, and return a React Component. The “higher order” phrasing is borrowed from the term “higher order function”, a more widely used term in Mathematics and Computer Science that refers to a function that either takes one or more functions as an argument, returns a function, or does both. Since React Components can be thought of as functions that take data and return ui elements, the naming and usage make sense. Let’s look at an example of a higher order component.

const ExtraPropsComponent = (WrappedComponent, componentType) => {
  return class ExtraPropsChild extends React.Component {
    render() {
      let props = Object.assign({}, this.props, {
        importantFunction: () => 'foo',
        componentType,
      });
      return <WrappedComponent {...props}/>
    }
  }
}

This is a pretty basic HoC that takes a component and returns a new version of that component that will always be passed 2 extra props importantFunction and componentType. One is statically generated in this case, but the other is dynamically passed in when the component is generated. The ability to pass dynamic content in when defining the component is one of the key wins of HoCs in the real world. For instance, this is how Redux’s connect function works.

Note that this is different from a normal React Component that renders a subcomponent passed as an argument like this:

let componentType = '???';
class ExtraPropsComponent extends React.Component
  render() {
      let { ChildComponent } = this.props;
      let props = Object.assign({}, this.props, {
        importantFunction: () => 'foo',
        componentType,
      });
      delete props.ChildComponent;

      return <ChildComponent {...props} />;
  }
}

These 2 examples are roughly equivalent for this case, but accomplish things in different ways. The HoC is a function that takes a component and data when it is created and generates another component. The normal component example is a component that takes a component as a property and accesses a variable that is in its scope at creation time to generate the element tree that it renders. The HoC is more flexible as you could theoretically create multiple versions of a component using the single HoC.


const FancyButton = ExtraPropsComponent(Button, 'fancy');
const CrazyButton = ExtraPropsComponent(Button, 'crazy');
const BoringButton = ExtraPropsComponent(Button, 'boring');

For the equivalent functionality without using an HoC, you would have to define the whole component for each case.

Higher Order Components in the real world

I’ve already mentioned connect from react-redux as an example of a higher order component. Other popular libraries that use HoC as part of their APIs include:

Radium: a library for inline styles in React that uses a higher order component to transform in-line styles added to a component into a more robust-css like system
React Router: a routing library for React that uses withRouter a HoC that injects access to the router into components that it wraps
Relay: Facebook’s library for connecting to GraphQL APIs includes Relay.Container, a HoC that allows developers to declare the data requirements for a component

More Resources

This was intentionally a very high level look at Higher Order Components. If you want to go deeper, this medium article is a great deep dive into all the use cases they allow.
If you’re interested in using HoCs in your own code base, recompose is a utility library for making the syntax around defining and using them painless

Readable code: Know your audience

Mon, 12 Dec 2016 00:00:00 -0400

A coworker asked me last week if there was a better way to write the following python code:

self.custom_attributes = {
  x.split(':')[0]: x.split(':')[1]
  for x in a.split(',')
}

I semi-jokingly replied that there were 3 better ways:

For people who like comprehensions

self.custom_attributes = {
  key: value
  for [key, value] in [entry.split(':') for entry in a.split(',')]
}

For people who like functions

def build_obj(acc, val):
   [key, value] = val.split(':')
   acc[key] = value
   return acc
self.custom_attributes = reduce(build_obj, a.split(','), {})

For people who like other folks to be able to read their code

# create a hash of custom attributes
self.custom_attributes = {}
# get the entries from splitting the input string
entries = a.split(',')
# for each entry split the key value pairs and add them to the hash
for entry in entries:
   [key, value] = entry.split(':')
   self.custom_attributes[key] = value

The joke of course is that it doesn’t really matter what the author of the code considers the best way to write it. Like writing an essay or persuasive speech, it is much more important to consider the audience, and what style will work best for them. Most code will eventually be read by other people, even if that is simply a future version of the original author who has been stripped of context by time ¹.

If my coworker is confident that his code will be read in the future by confident Python programmers who understand object comprehensions, then he could go with the first example, with its heavy use of python idioms. If he thought it was more likely that somebody like me who is used to JavaScript idioms, he might consider the less idiomatic python from the second example that matched his teammates. If he didn’t know who would read the code in the future, he might go with the more explicit and difficult to misinterpret version 3, and possibly tuck it away within a named function so that the main code flow could remain uninterrupted.

self.custom_attributes = parse_entry_string(a)

# ... elsewhere ...

def parse_entry_string (entry_string):
    attributes = {}
    # get the entries from splitting the input string
    entries = a.split(',')
    # for each entry split the key value pairs and add them to the hash
    for entry in entries:
       [key, value] = entry.split(':')[0:2]
       attributes[key] = value
    return attributes

Of course once the functionality is separated out into a function, it matters a lot less how it is implemented since the only time somebody will need to read it is when they are interested in that specific functionality and can invest the time to understand the code. The well named function replaces the need for understanding everything when people browse through the code. This is extremely important in larger code bases, since data often flows through many lines of code. The more of it that can be usefully (and accurately) abstracted away for a reader, the easier the code base is to debug.

So quick tip version:

Think about the audience of the code you’re writing
Naming functions well limits the needs to make implementation details readable for everybody. It can also make your program flow easier to follow.

In my experience, it usually only takes about a week for me to look back at my old code and wonder what maniac wrote it and what he could have possibly been thinking ↩︎

Saving Time With Jest: Meetup Summary

Sat, 10 Dec 2016 01:15:00 -0400

This week I gave a talk at the brand new Triangle ReactJS meetup group on how Jest can save developers time. I was asked afterwards to share the content online for those who missed it. My talk was mostly demos and wasn’t recorded, so there unfortunately isn’t much to share, but I thought I could write out the basic ideas behind the talk, and share them here.

What is Jest

Jest is a JavaScript testing framework built by Facebook to address their internal problems testing JavaScript code. It was open-sourced in 2014, but unlike Facebook’s other recent high profile JavaScript OSS projects like React, Flux, Relay and GraphQL, it failed to develop much traction in the community and mostly stagnated for 2 years. That changed in early 2016 when 2 Facebook developers, Christoph Pojer and Dmitrii Abramov began working on improving the project. Over the past year they’ve brought massive improvements to the developer experience of using Jest.

At a more practical level Jest is a test runner with a built in assertion library and support for code coverage, babel transpilation, and module mocking. It began as a wrapper around Jasmine, an older JavaScript testing library, but has since replaced parts of Jasmine with replacement functionality, while retaining backwards compatibility.

How Can Jest Save Me Time?

Jest saves developers time in 4 main ways:

Setup - Jest is simple to configure
Writing Tests - Jest provides a method for writing simple UI tests, and makes it easy to port over existing tests
Running Tests - Jest runs tests in parallel, doesn’t require a browser, and only runs the tests that are affected by the current code changes
Fixing Tests - Jest gives great error messages, helps you find the failing code, and has other smaller UX features that make it easy to debug your failing tests quickly.

Tonight @_benmccormick speaks about using #jest with #reactjs for your testing needs @nestraleigh pic.twitter.com/CAV8wZUwFw
— Tracy Lee | ladyleet (@ladyleet) December 8, 2016

Setup

Setting up Jest is extremely straightforward. Here’s a Hello World React Project

//package.json
{
  "name": "simple_example",
  "version": "1.0.0",
  "description": "A minimal jest testing example",
  "scripts": {
    "test": "jest"
  },
  "dependencies": {
    "react": "^15.4.1"
  },
  "devDependencies": {
    "babel-jest": "^17.0.2",
    "babel-preset-es2015": "^6.18.0",
    "babel-preset-react": "^6.16.0",
    "jest": "^17.0.3",
    "react-test-renderer": "^15.4.1"
  }
}

In package.json we have 4 dependencies that are part of a normal React project setup, react, the 2 babel presets for using es2015 syntax, and react-test-renderer, which is a react addon for rendering react for testing, similar to how react-dom renders components for the browser. We then add jest and babel-jest. jest provides the test runner, assertions API, mocking capabilities, and cli. babel-jest is a drop-in dependency that lets Jest integrate with Babel. Neither require any additional configuration in the normal case. Lets say the rest of our project looks like this:

├── __tests__
│   └── simple-component.spec.js
├── package.json
└── simple-component.js

// simple-component.js
import React from 'react';

export const SimpleComponent = () => <div>
    Hello World
</div>;

// __tests__/simple-component.spec.js
import React from 'react';
import renderer from 'react-test-renderer';
import {SimpleComponent} from '../simple-component';

it('shows Hello World', () => {
  const component = renderer.create( <SimpleComponent/>);
  let componentJSON = component.toJSON();
  expect(componentJSON.children[0]).toBe('Hello World');
});

Given this project, we only have to run

yarn # or npm install
jest

and jest will automatically find our test file and run the test.

Writing Tests

Jest provides Snapshot Tests, which are a quick way of writing tests to catch changes in the rendered output of a UI component (or any other serializable content). I’ve written about Snapshot testing before, so I won’t go into depth here. But snapshot tests are significantly faster to write than traditional assertion based tests, and for the write type of code can provide greater or equal benefit. This is one area where technically Jest may not save you time. Instead it may take your team to a place where you’re actually willing to write tests for your UI components for the first time.

There are also a great set of codemods for Jest. jest-codemods can help you take your existing suite of tests from Mocha, Ava, or tape and translate it over to Jest automatically. Of course if you were using Jasmine previously, your tests can be migrated without changes. So Jest saves you time by not requiring you to rewrite any tests.

Running Tests

Jest runs tests fast by default. While I’ve seen from comments on my last Jest post and elsewhere that some people may have been able to optimize mocha or AVA to be fast enough that switching to Jest was a downgrade, for most people Jest is going to be faster than whatever they may have been using before. This is true because of performance reasons (tests run in parallel, and mock out timers to avoid unnecessary waiting), but also because of UX decisions. Jest is able to use git and node’s modules system to detect what files have changed since the last commit, and which tests are affected by that. Because of that, it is able to optionally run only the tests that might have changed on each run (jest -o). Since it is usually able to run tests fast as a result (since it only runs a few at a time), Jest also is able to have an awesome watch mode (jest --watch). In watch mode, tests rerun as files change, and there are options to target specific tests, run all tests, or fix snapshots as you go.

Fixing Tests

Jest provides 4 main conveniences for fixing tests

Error messages have clear formatted diffs, making it easy to know the difference between what was received and expected
Error messages have a clear traceback to the line that failed, either the assertion in the test file, or the exception in the source file
Even though tests run in parallel, Jest makes sure that any console output from the test is placed in line with the test output, making it easy to debug with quick console messages
When running Jest multiple times, Jest runs the tests that failed in the previous run prior to other tests, leading to a faster feedback loop even when not using watch mode

More Resources

Another speakup at Wednesday’s meetup has a Jest related blog post out this week. Nate Hunzaker blogged about using Jest with Nightmare.js for acceptance tests
If you live in the Triangle area (thats Raleigh/Durham, North Carolina to the rest of the world) make sure to check out the Triangle React meetup. The plan as I understand it is to do events once a month, alternating project nights and talks. I believe Tracy is looking for more speakers too, so hit her up if you’re interested. This time 4 of us spoke for 15-20 minutes, so it isn’t as much preparation as other events where you’re the only speaker.

A whole new site

Fri, 02 Dec 2016 12:00:00 -0400

After 2.5 years of stability, this week I moved benmccormick.org to a new platform and overhauled the look and feel. Although most readers have seen the site remain relatively stable, this is actually the 5th major revision of the site since it was first created, so this seems like a good time for a quick trip down memory lane.

I first started it as a Django app, wanting to challenge myself to build a blog. That was so short-lived that the way-back machine never even registered it, and I don’t have any evidence that it ever existed (thankfully). In December 2012, I moved the blog over to Octopress a static site generator based on Ruby. I started out using a very nice stock template:

and then shortly switched to a version that I mostly did myself based off of the default octopress theme. This was one of my first attempts to design anything from scratch, and was not particularly pretty:

I enjoyed Octopress, but spent a lot of time messing around with it (and struggling through the unfamiliar ruby ecosystem), and not a lot of time writing. So in the spring of 2014 I decided I wanted to switch to a blogging platform where I could write, rather than messing with code. So I moved my blog to Ghost, lightly modified a public theme, and began writing about Vim. It was around this time (2 years and 40 articles in) that I began to attract more readers, and also began to focus more on higher quality technical articles:

About a year after that I re-introduced some color to the blog after a prolonged absence. The black and white sidebar was traded for a colorful green, and a I began using a mix of blue and red for text emphasis. I also began to focus on making it more clear how to subscribe to the blog, and highlighted the more popular content:

Ghost was a great blogging platform, but this fall I decided it was time to move off of it. I was interested in trying some things that didn’t fit into the platform’s more structured approach, and wanted to move the blog to a place where it was no longer losing me money to run. So I decided to take another look at the static site generator landscape. I looked through the top 12 or so options on StaticGen and Gatsby quickly stood out. It uses concepts I’m extremely comfortable with (built with React and Webpack!), produces fast server-rendered SPA style sites, and dictates almost nothing about how the site is built.

The site is now built with Gatsby and hosted on Github pages. I’ve ditched the noisy green sidebar in favor of a simpler text-focused experience and added a small ad at the top of the page to offset some of the costs associated with the blog. I’m planning on iterating on the design (better organizing and exposing older content, making it simpler to subscribe to the blog, and making it easier for me to add rich content/code examples to posts without relying on 3rd party services), but for right now I’m excited to get back to focusing on writing.

Like all software, as this blog has evolved, I’m sure not all of the progress has been forward. If you see any issues with the new site, feedback and bugfixes are welcome. Feel free to hit me up on twitter, comment on this post, or open an issue on the site’s github repo. Thank you to everyone who has read and supported this site over the past several years.

More Resources

If you’re interested in Gatsby this post on Practical Gatsby.js by Scott Nonnenberg is excellent. Its not a how-to, but more of a series of examples of how he deviated from the base Gatsby blog (and why). I found it really helpful when brainstorming ideas for this site.

Testing with Jest Snapshots: First Impressions

Mon, 19 Sep 2016 08:41:00 -0400

Over the past 6 months my team at work has been slowly transitioning a large Backbone application to React. One of the main goals of the transition has been to build out unit tests as we transition components. As part of that we switched from using Karma to using Jest for managing our tests.

Our new setup with Jest has several advantages over our previous setup. All Jest tests run through node using a fake DOM implementation. We don’t need to start up a browser so tests are faster and less flaky ¹. Jest makes it possible to mock all modules by default ², which works well for us since our code base currently contains a large amount of code that is difficult to test for one reason or another. Jest provides great integration with Babel, which is helpful since all of our Backbone and React code uses ES6 that we transpile for the browser. Finally Jest provides a wonderful watch mode that runs only the tests that are relevant to the changes you’ve made and makes it easy to see the result.

About a month ago, Jest debuted a new feature: Snapshot testing. It works a bit differently than a normal unit test. Instead of executing some code and comparing the output against a value provided by a developer, the first time a test is run, the output passed into the test is saved to a “snapshot file”. Then when the test is run in the future, the output is compared to the snapshot file. If the output matches the file, the test passes, if the output is different then the file, the test fails and Jest prints a diff. We’ve been using Jest snapshots for a few weeks now and I wanted to share my initial impressions.

What is it like to use a snapshot test ?

The process for snapshot testing is a bit different than normal tests. Most snapshot tests look fairly simple. Something like this example from the Jest repository:

// Link.react.js
// Copyright 2004-present Facebook. All Rights Reserved.

import React from 'react';

const STATUS = {
  NORMAL: 'normal',
  HOVERED: 'hovered',
};

export default class Link extends React.Component {

  constructor() {
    super();

    this._onMouseEnter = this._onMouseEnter.bind(this);
    this._onMouseLeave = this._onMouseLeave.bind(this);

    this.state = {
      class: STATUS.NORMAL,
    };
  }

  _onMouseEnter() {
    this.setState({class: STATUS.HOVERED});
  }

  _onMouseLeave() {
    this.setState({class: STATUS.NORMAL});
  }

  render() {
    return (
      <a
        className={this.state.class}
        href={this.props.page || '#'}
        onMouseEnter={this._onMouseEnter}
        onMouseLeave={this._onMouseLeave}>
        {this.props.children}
      </a>
    );
  }

}

// Link.react-test.js  (partial)
// Copyright 2004-present Facebook. All Rights Reserved.
/* eslint-disable no-unused-vars */

'use strict'

import React from 'react';
import Link from '../Link.react';
import renderer from 'react-test-renderer';

it('renders correctly', () => {
  const tree = renderer.create(
    <Link page="http://www.facebook.com">Facebook</Link>
  ).toJSON();
  expect(tree).toMatchSnapshot();
});

The first time the test is run, a snapshot file is produced. In this case, running the previous test produces a snapshot file that looks like this:

// Link.react-test.js.snap  (partial)
exports[`test renders correctly 1`] = `
<a
  className="normal"
  href="http://www.facebook.com"
  onMouseEnter={[Function bound _onMouseEnter]}
  onMouseLeave={[Function bound _onMouseLeave]}>
  Facebook
</a>
`;

This gives us a baseline for what we expect the UI to look like. The snapshot is generated in a folder in your __tests__ directory, so that it can be checked into source control.

The next time a test is run, if nothing has changed, the test passes. However, if we change something (let’s say we add a class), the test fails and shows us a diff.

// updated link render method

render() {
  return (
    <a
      className={`link-item ${this.state.class}`}
      href={this.props.page || '#'}
      onMouseEnter={this._onMouseEnter}
      onMouseLeave={this._onMouseLeave}>
      {this.props.children}
    </a>
  );
}


# jest output

FAIL  __tests__/Link.react-test.js
 ✕ renders correctly (16ms)

● renders correctly

    Received value does not match the stored snapshot 1.

    - Snapshot
    + Received

      <a
-       className="normal"
+       className="link-item normal"
        href="http://www.facebook.com"
        onMouseEnter={[Function bound _onMouseEnter]}
        onMouseLeave={[Function bound _onMouseLeave]}>
        Facebook
      </a>

      at Object.<anonymous> (__tests__/Link.react-test.js:14:16)

We can then choose to accept this change by running jest -u to update the snapshot, or update our code to fix the regression. If we update the snapshot file, the test will start passing again.

Snapshot tests are a complement for conventional tests not a replacement

The first thing that became clear to me while using snapshot testing is that they’re not for everything. They are optimized for a different case than normal assertion-based tests.

Classic assertion based tests are perfect for testing clearly defined behavior that is expected to remain relatively stable.

Snapshot tests are great for testing less clearly defined behavior that may change often.

Snapshot testing is perfect for things that in the past would have raised concern for “overly brittle” tests that slow development teams down. The classic example of this (and the main focus of discussion around Jest snapshots so far) is UI testing.

UI components often change in small and trivial ways. Copy is changed, whitespace is added, a border color is modified. Generally this means that developers have to choose between detailed tests that catch any regression but require constant updates, or less detailed tests that focus on core behaviors but miss smaller regressions.

Snapshot tests provide a new way of approaching these problems in unit tests. Because they are easy to write and effortless to update, and provide a clear view of what changed after each update, whether it is major or minor.

This isn’t without any disadvantage. Snapshot tests preserve a starting point, but don’t give any indication of developer intent to future developers. Like any other test, if the initial snapshot passes with bugs, or doesn’t capture the full range of cases, the test won’t adequately cover those behaviors. But unlike traditional tests, snapshot tests don’t provide specific guidance for what the original developer expected beyond the “actual behavior”. If there are multiple sub-behaviors that interact to produce output, its not clear which of those are essential, and which are incidental. That’s not something you want for your critical application logic, especially complicated logic with many subtle cases. But for UI components and other application elements that are likely to change a lot over time, this is a good set of tradeoffs.

Snapshot tests are more useful with a healthy code review process

I’ve pointed out several times now that snapshot tests are easy to update. One of the downsides of the current design is that they can be almost too easy to update. When a snapshot test fails, you have the option of modifying the output so that the test passes again, or updating the snapshot to reflect the new output. Unfortunately, Jest currently provides no granularity for this updating. Running jest -u to update the snapshots will update all snapshots. So if a single change causes many snapshots to break, it can be easy to accidentally update all of them, and possibly difficult to work through the whole diff to find changes.

A robust code review process minimizes these problems. Because snapshots are checked into source, the diffs in the snapshots are checked into source control. Viewing diffs in a dedicated code review tool like Github, Bitbucket or Gitlab, it’s easier to see exactly what changed over many files than it might be when updating snapshots from the command line, and if you update a snapshot multiple times in a single branch, you can see the net effect on the snapshot through source control diffs better than you can through Jest.

Snapshot tests work well with auto-mocking

Auto-mocking has been one of Jest’s more controversial features. Prior to version 15, Jest mocked every module used in tests by default. To run the actual module code, it was necessary to manually unmock each module used or change Jest configurations. Though it is now disabled by default, auto-mocking actually works very well with snapshot tests. One problem with tests that rely on generating DOM in React is the fact that most components of any complexity are actually made up of smaller components, that often use external functions with complex logic to determine what to show. If we fully render everything, changing any of those pieces can break tests for many components, moving us away from the concept of a “unit test”. If we mock child components and helper functions and then test them individually though, we can focus our tests to only change based on the logic of the individual component being tested. This is incredibly easy with Jest, and especially helpful in large code bases that don’t have complete test coverage. Auto-mocking is considered an “advanced feature” of Jest, but it’s not hard to use. Most mocked modules produce something non-harmful by default, and when its necessary to override, its not difficult to produce something meaningful without getting too involved. Just add a file with the same name as the file to be mocked into a __mocks__ folder in that directory, and give the mocked module the same interface as the mocked component. You can either write it from scratch or use jest.genMockFromModule as a starting place.

Summary

I’m impressed by Jest snapshots. When combined with React, they’re providing my first non-frustrating experience unit testing UI components. I’m excited to use them more going forward, and will be interested to see how they evolve from here.

Extra: A few miscellaneous Jest Thoughts

The UI output from Jest 15 tests is the best I’ve ever seen from a testing framework. Aside from the nice diffs on snapshots, they do a good job explaining how output differed from expectations in assertion based tests, and communicating context for test failures, warnings and logging messages. They also give great guidance when making configuration changes:

@cpojer This is lovely, thank you! pic.twitter.com/CX1v11CAVc
— Ben McCormick (@_benmccormick) September 5, 2016

I’ve also been super impressed with Jest’s watch mode in Jest 15. --watch makes it easy to focus on a few files or run tests on all files, and stays up to date as you change code. If you haven’t worked with Jest in a while, the developer experience alone makes it worth trying out.
One potential future annoyance with snapshot tests: The snapshot format can’t be iterated on without breaking all current tests. Make sure not to mix functional changes with Jest updates!

We'll update the snapshot format sometimes in minor/major releases. When updating Jest, please update snapshots

See https://t.co/gPE63dMz8P
— Christoph Pojer (@cpojer) September 16, 2016

More Resources

Randy Coulman has a nice article on some of the reasons to be careful about embracing snapshot testing. He makes good points about the requirement of human involvement in verifying a change in time-crunched situations and also the potential value of snapshot tests for legacy code.
My team uses a mix of enzyme and snapshot tests for unit testing React Components. This repo combines them, allowing you to use enzyme to generate the snapshot output to diff against rather than the renderer provided by Jest. I personally haven’t felt the need to integrate these two tools, but if your team uses enzyme and wants consistency/fewer (used) dependencies, this could be a cool option.
Finally here’s another article on how Snapshot testing convinced one team to migrate to Jest from Tape

PhantomJS was a tremendous source of pain in our Karma setup. ↩︎
This used to be the default setting, with Jest 15 it can be configured to work that way. ↩︎

Quick Tip: Take advantage of lodash collections

Sun, 26 Jun 2016 17:27:55 -0400

One of the more frustrating things about using JavaScript in the browser is the number of different collection types that you need to deal with. When using native APIs and 3rd party libraries, it’s easy to encounter 3 or more different types of collections, including NodeLists, HTMLCollections, jQuery Objects, and plain arrays.

let a = document.querySelectorAll('div')
a instanceOf NodeList //true

let b = document.getElementsByTagName('div')
b instanceOf HTMLCollection //true

let c = $('div')
c instanceOf jQuery // true

let d = ['button1', 'button2', 'button3'].map(id => document.getElementById(id));
d instance of Array //true

This ends up mattering in practice because not every function that is available on Array is available on NodeList, HTMLCollections, and jQuery objects.

let ids = document.querySelectorAll('div').map(el => el.id)
// Uncaught TypeError: document.querySelectorAll(...).map is not a function

ids = document.getElementsByTagName('div').map(el => el.id)
// Uncaught TypeError: document.getElementsByTagName(...).map is not a function

let numDivs = $('div').reduce((count, el) => count + 1, 0);
// Uncaught TypeError: $(...).reduce is not a function

Since we can’t use native array methods on these alternate collections, we end up having to be very careful about knowing where we’ve retrieved data from, and how to convert those collections or how to work around their limitations.

It’s useful to know about these distinctions, but in practice dealing with them is a pain. One of the nice properties of lodash is that it abstracts over these incompatibilities.

let ids = _.map(document.querySelectorAll('div'), el => el.id);
// ['container', 'left-box', 'right-box']

ids = _.map(document.getElementssByTagName('div'), el => el.id);
// ['container', 'left-box', 'right-box']

let numDivs = _.reduce($('div'), (count, el) => count + 1, 0));
// 3

For methods that iterate over a collection lodash treats all array-like objects (essentially anything with a length property) the same and iterates over their numerical properties from 0 to length-1. This provides a nice consistency when working with DOM APIs and removes the mental overhead from managing their inconsistencies.

What are mutable and immutable data structures?

Sat, 04 Jun 2016 17:52:52 -0400

t Sometimes concepts and ideas slowly grow in a programming community, sometimes they seem to appear in a flash. For the first several years I wrote JavaScript, I don’t recall ever seeing anything written online about immutable data. Since React hit the scene in the last 2 years however, articles mentioning mutable and immutable data seem to have multiplied, as have libraries like Immutable.js and alternate front end languages like Elm that allow users to “use immutable data”. I’m not going to address those libraries, but thought it would be useful to throw out a quick primer on what immutable data actually is, how it differs from mutable data, and why anyone cares.

A mutable object is an object whose state can be modified after it is created. An immutable object is an object whose state cannot be modified after it is created. Examples of native JavaScript values that are immutable are numbers and strings. Examples of native JavaScript values that are mutable include objects, arrays, functions, classes, sets, and maps.

Implications of mutable objects

So why does this matter? Consider the following code examples:

let a = {
    foo: 'bar'
};

let b = a;

a.foo = 'test';

console.log(b.foo); // test
console.log(a === b) // true

let a = 'test';
let b = a;
a = a.substring(2);

console.log(a) //st
console.log(b) //test
console.log(a === b) //false

let a = ['foo', 'bar'];
let b = a;

a.push('baz')

console.log(b); // ['foo', 'bar', 'baz']
console.log(a === b) // true

let a = 1;
let b = a;
a++;

console.log(a) //2
console.log(b) //1
console.log(a === b) //false

What we see is that for mutable values, updating state applies across all references to that variable. So changing a value in one place, changes it for all references to that object. For the immutable data types, we have no way of changing the internal state of the data, so the reference always gets reassigned to a new object. The biggest implication of this is that for immutable data, equality is more reliable since we know that a value’s state won’t be changed out from under us.

Finally, its worth noting that it’s still possible to treat JavaScript objects as immutable. This can first be done through Object.freeze, which shallowly renders a JavaScript object immutable. But it can also be done with programmer discipline. If we want to rely on object’s being immutable, it’s possible to enforce that all object updates are done through something like Object.assign(a, {foo: 'bar'}) rather than a.foo = 'bar', and all array updates are done through functions that generate new arrays like Array.prototype.map, Array.prototype.filter, or Array.prototype.concat, rather than mutating methods like Array.prototype.push, Array.prototye.pop, or Array.prototype.sort. This is less reliable without language level constraints, but has become popular in the React ecosystem for dealing with data for folks who don’t want to introduce abstractions like Immutable.js.

Digging Into React: Choosing Component Styles

Sun, 01 May 2016 22:32:36 -0400

Over the past month I’ve been building my first production features using React. One of the first things I’ve noticed is the amount of choices that come into a new React project. A lot of this is about the ecosystem around React (Webpack vs Browserify? Redux vs Relay? Do I need CSS modules or Immutable.js?) but many of the choices are just part of the library, figuring out the best, or at least a consistent, way to do things.

One such choice is the syntax for defining components that you choose to use. In React, you can define components in at least 3 different styles: the React.createClass style, the ES6 class style, and the pure functional style ¹. Each has some advantages and disadvantages.

React.createClass

React.createClass is the original style for defining React components, and is still the preferred style in the React documentation.

Here’s what a theoretical SaveButton component looks like in this style. It is a simple component that listens for clicks and then uses a separate dispatcher object to send a request to save the data on the page. After saving the button will either show some specified saved text or “Saved”.

import React from ‘react’;
import {dispatcher} from ‘./lib/dispatcher’;

export const SaveButton = React.createClass({

    propTypes: {
        //text to show after the component is saved
        savedText: React.PropTypes.string.isRequired,
        //primary text to show on the button
        text: React.PropTypes.string.isRequired,
    },

    getDefaultProps() {
        return {
             savedText: ‘Saved’,
        };
    },

    getInitialState() {
        return { saved: false };
    },

    save() {
        dispatcher.sendMessage(‘saveData’);
        this.setState({saved: true});
    },

    render() {
        let {text, savedText} = this.props;
        return (<span className=“button” onClick={this.save}>
           {this.state.saved ? savedText : text}
        </span>);
    },
});

One of the best things about this style is its cohesive nature. Everything about the component is described as a single object passed to React.createClass. Since it is just an object that React processes, it is also able to accommodate more usage styles than ES6 classes and functional components. Specifically we can use mixins and add declarative properties like propTypes directly to the component definition.

The downside of the createClass approach is the custom nature of it. External tools can’t inspect a component declaration and know what functions and methods will be available on the final component without either some deep inspection of the React source code or pre-existing knowledge of React.createClass that somebody has to document and maintain.

ES6 Classes

Last March, with React v0.13, the React team introduced a second syntax for React components using ES6 classes. The constraints of the ES6 class definition change the component API in a few ways. With ES6 classes there is no way to add properties to the class property as part of the main definition, they need to be added afterwards ². Some niceties that are possible with React.createClass don’t work with ES6 classes as well. Mixins aren’t possible, and React will no longer “autobind” your functions to the component instance, so when passing class methods as callbacks in your components it is necessary to bind it to the current context, either by using arrow functions like <span onClick= {() => this.save()} > or by binding the function like this.save = this.save.bind(this).

import React from ‘react’;
import {dispatcher} from ‘./lib/dispatcher’;

export class SaveButton extends React.Component {

    constructor(props) {
        super(props);
        this.state = { saved: false };
    }

    save() {
        dispatcher.sendMessage(‘saveData’);
        this.setState({saved: true});
    },

    render() {
        let {text, savedText} = this.props;
        return (<span className="button" >
           {this.state.saved ? savedText : text}
        </span>);
    },
});

SaveButton.propTypes = {
    savedText: React.PropTypes.string.isRequired,
    text: React.PropTypes.string.isRequired,
};

SaveButton.defaultProps = {
    savedText: ‘Saved’,
};

While this approach makes some syntactic compromises and loses mixins, it has the advantage of standardization and tooling support. A good JavaScript tool can easily tell that a SaveButton should have render and save methods, as well as detecting the methods it inherits from React.Component. This is useful for things like auto-complete and linting, as well as (theoretically) run-time performance.

Pure Functional Components

Last fall with React 0.14, React added a 3rd component style. Functional components dispose of many of the secondary features of React, and focus on the render function. They are the least powerful of all the component syntaxes. In addition to losing mixins, pure functional components don’t have the life-cycle functions of the class based syntaxes and also don’t have any internal state. For these components, all state must be managed externally through something like Redux, or parent components that do have state.

import React from ‘react’;
import {dispatcher} from ‘./lib/dispatcher’;

export const SaveButton = ({text, savedText, isSaved, setSaved}) => {
    const save = () =>{
        dispatcher.sendMessage(‘saveData’);
        setSaved();
    },
    return (<span className=“button” onClick={save}>
       {isSaved ? savedText : text}
    </span>);
};

SaveButton.propTypes = {
    //text to show after the component is saved
    savedText: React.PropTypes.string.isRequired,
    //primary text to show on the button
    text: React.PropTypes.string.isRequired,
    // has the data already been saved?
    isSaved: React.PropTypes.bool.isRequired,
    // a function to update the application state and mark the page as saved
    setSaved: React.PropTypes.func.isRequired,
};

SaveButton.defaultProps = {
    savedText: ‘Saved’,
};

Despite being the least powerful of the 3 component styles, functional components do have some advantages. First of all, they’re very simple. It is easy to see what is going on with them, and easy for React to optimize for them. Like ES6 classes, they’re easy to understand for 3rd party tools since they’re “just functions”, and easy to explain to new developers for the same reason. They also work well with systems like Redux which already encourages pulling state out of components into global stores.

Choosing a component style

So given these 3 styles, how do you choose which style to use for a component? Balance a desire for consistency with a respect for the Principle of Least Power.

For consistency purposes, don’t use more than 2 of these styles within a single project. This mostly breaks down to “don’t use createClass and ES6 classes” together in the same project. There are significant differences in capability and complexity between the first 2 styles and functional components. But the class based styles are similar enough that using both adds complexity but fails to do much in terms of signaling intent. Switching between 2 different styles of communicating a concept should also indicate a meaningful difference in intent.

Of course the most consistent approach would be to always use createClass, since it is the most powerful and any component that can be implemented in the other 2 styles can be implemented using createClass. This is where the principle of least power comes in. Here is Tim Berners-Lee’s initial description of that principle:

Computer Science in the 1960s to 80s spent a lot of effort making languages which were as powerful as possible. Nowadays we have to appreciate the reasons for picking not the most powerful solution but the least powerful. The reason for this is that the less powerful the language, the more you can do with the data stored in that language. If you write it in a simple declarative from, anyone can write a program to analyze it in many ways. The Semantic Web is an attempt, largely, to map large quantities of existing data onto a common language so that the data can be analyzed in ways never dreamed of by its creators. If, for example, a web page with weather data has RDF describing that data, a user can retrieve it as a table, perhaps average it, plot it, deduce things from it in combination with other information. At the other end of the scale is the weather information portrayed by the cunning Java applet. While this might allow a very cool user interface, it cannot be analyzed at all. The search engine finding the page will have no idea of what the data is or what it is about. This the only way to find out what a Java applet means is to set it running in front of a person.

The less powerful react components styles are the more readable styles, both for humans and computers. That readability also limits the classes of errors and unexpected behaviors we can enact. So we should strive for simple components where we can, while balancing that with our desire for consistency.

For projects looking into this, there are a few simple questions that you can answer to decide which style to use.

Do you need to integrate with non-React libraries that modify the DOM?
Do you need to manage state within your components (IE are you not using an external state management library like Redux)?
Do you need to use mixins for anything in your project?

If you answered yes to any of these questions, then you need to include a class based component style in your options for your projects. If you answered yes to #3, then you will need to use React.createClass.

Once you’ve made those decisions at the project level, the decisions are easy for an individual component. If you answered no to all of the questions above (unlikely for a large project, but possible), you should always use functional components. If not, you can ask the same 3 questions for each new component. If the answer to all of the questions are no for a component, use a functional component, otherwise use the class based style you’ve chosen for your project.

More Resources

The React documentation is the best place to look at the benefits and tradeoffs of the different component styles
James Nelson wrote a similar guide last year focused on the decision tree of what style to use for a component. He dismisses the createClass style completely and has a similar but slightly different set of questions to ask.
This github thread makes the important distinction that functional components are not yet optimized in any special way, though they may be in the future.

Technically you can also use an ES5 class based style, but I’m going to ignore that. It is not a common idiom for React programs as far as I can tell. ↩︎
There is a proposal for adding static properties to ES6 classes, but it is still in early stages right now and is not guaranteed to end up as part of the ECMAScript spec. ↩︎

Ack Tips

Sat, 30 Apr 2016 11:48:25 -0400

Last summer I briefly considered creating a separate blog focused on command line tools. The idea is that the articles would be shorter and more practical than the longer pieces I write here. I ended up scrapping the idea, but had written a few posts on one of my favorite command line tools Ack, so I wanted to preserve them here, as a single, longer piece, separated into 3 “tips”

#1: Prefer ack or ag to grep

grep is one of the most well known unix utilties. If you’ve had any exposure to the command line at all, you’ve probably searched for something using grep. grep’s standalone use is searching for content inside of files, but with Unix’s piping philosophy it is useful as a general search and filtering tool. Originally created for Version 4 Unix 42 years ago, it has been a staple of *Nix operating systems ever since. If you’ve been using it for even half that time, I’m sure a blog post isn’t enough to convince you to stop. But for the rest of us, there’s an opportunity to do better.

ack and ag ¹ are similar projects with the same goal: providing a better experience than grep when searching directories of files, particularly for programmers. The key selling points:

Improved speed when searching through directories
Ignores version control and binary files by default when searching
Easy filtering by file-type
A cleaner API for searching directories, without adding complexity to the single file case

This is easiest to see by comparing commands.

Searching for “javascript” in a file

grep javascript foo.txt

ack javascript foo.txt

Searching every file in the current directory for “javascript”

grep -r javascript .

ack javascript

Searching every file in the current directory for “javascript”, excluding a .git directory

grep -r javascript . $(find . -type f | grep -v '\.git')

ack javascript

Searching every HTML file in the current directory for “javascript”, excluding a .git directory

grep -r javascript . $(find . -name '*.html' -or -name '*.htm'| grep -v '\.git')

ack --html javascript

Basically, as the complexity of what you’re trying to do increases in terms of multiple files, the overhead of using grep increases proportionately. Because ack and ag are optimized for common use cases, they’re able to scale much better.

#2: Use Ack to get familiar with a new codebase

ack is an incredibly useful tool when approaching a new codebase, especially if you’re focused on a specific feature. It allows you to quickly get a view of the codebase, and then zero in on the functionality you’re interested in. To see how this works, let’s use jQuery’s source repo as an example, and see what happens if we want to get a look at how it implements ajax.

❯ git clone git@github.com:jquery/jquery.git
❯ cd jquery

We can start by trying to get a general idea of the size of the codebase. Running ack -f to show all files in the directory will immediately overrun our terminal, so we can get a file count using wc instead

❯ ack -f |wc -l
    249

Ok thats a good amount of files. But how many of them are actually source files? We can filter down to just the javascript files.

❯ ack -f --js |wc -l
    155

But thats probably still a bit high, since not all of the JavaScript files are source files. A quick ls shows that we probably want to filter down to the src directory.

❯ ack -f --js src |wc -l
    92

Ok so now we’re down to 92 source files. Still a bit much, but its probably worth printing them with ack -f --js src at this point to get an idea of the directory structure. When we do that, we can see there’s an ajax folder, which is probably what we’re interested in. To make sure we’re not missing related files, we can continue on to searching for files with ajax in the file name directly.

❯ ack -g --js ajax src
    src/ajax/jsonp.js
    src/ajax/load.js
    src/ajax/parseJSON.js
    src/ajax/parseXML.js
    src/ajax/script.js
    src/ajax/var/location.js
    src/ajax/var/nonce.js
    src/ajax/var/rquery.js
    src/ajax/xhr.js
    src/ajax.js
    src/event/ajax.js

That gets us down to 11 files to look at. But we might be getting a bit greedy. Not every file related to ajax may have ajax in the file path. We can use the -l option of ack to display the list of files that contain ajax somewhere in their text.

❯ ack --js -l ajax src
    src/ajax/jsonp.js
    src/ajax/load.js
    src/ajax/script.js
    src/ajax/xhr.js
    src/ajax.js
    src/event/ajax.js
    src/jquery.js
    src/manipulation/_evalUrl.js
    src/serialize.js

Ok that adds 3 additional files with ajax references. But its not clear that they’re relevant to jQuery’s ajax implementation, it may just be a file reference or comment. Lets use ack to get a quick look at how ajax is referenced in those files. This part gets a bit more complicated. We can use ack’s -v option to invert our matching and find non-matching inputs, then use unix piping to filter our previous command.

❯ ack --js -l ajax src |ack -v ajax
    src/jquery.js
    src/manipulation/_evalUrl.js
    src/serialize.js

Now we can use ack’s -x argument to search the files piped in through standard input.

❯ ack --js -l ajax src |ack -v ajax|ack -x ajax

    src/jquery.js
    20:     "./ajax",
    21:     "./ajax/xhr",
    22:     "./ajax/script",
    23:     "./ajax/jsonp",
    24:     "./ajax/load",
    25:     "./event/ajax",

    src/manipulation/_evalUrl.js
    2:      "../ajax"
    6:      return jQuery.ajax({
    9:              // Make this explicit, since user can override this through ajaxSetup (#11264)

    src/serialize.js
    61:             traditional = jQuery.ajaxSettings && jQuery.ajaxSettings.traditional;

So we can see pretty quickly that these files are safe to ignore in terms of understanding how ajax works in jQuery. They’re just module loads, an actual call to the ajax method for other purposes, and settings serialization. It is useful to note that src/ajax is loaded into the main jQuery file first. Its probably worth looking at it as the starting point for our investigation. We’re well on our way to diving into the ajax functionality, able to focus on 11 of the 249 files in the repo immediately, already knowing how those files are loaded into the main library.

This was just a simple example. Whenever you’re spelunking into new code territory, consider using ack to get your bearings. Things go a lot faster when you know the lay of the land in advance.

#3: Take advantage of Ack’s configurability

One of the nice things about ack is the fact that it takes an .ackrc configuration file ². These files can be scoped at a directory level, a user level or at a root level, and mirror the options available on the executable itself. This allows you to extend and customize acks behavior in 3 primary ways.

a. Add new filetypes, and bundle existing ones

One of ack’s most useful features is the ability to add a --<filetype> flag to your searches to restrict your searches to a specific filetype. ack comes with 75 builtin types including almost all common languages. But those aren’t always good enough. To handle obscure languages, and not so obscure syntaxes like handlebars templates, ack lets you define new filetypes. You’re also able to modify existing filetypes to add additional file extensions to include under that filetype. So for instance my ackrc includes these lines:

--type-add
hbs=hbs
--type-add
js=js,es6

The first 2 lines allow me to specifically search handlebars files, which are where all of the html content for my work projects is stored. The next 2 lines include es6 files in my JavaScript searches, useful since I append JavaScript files that I intend to process with Babel with the .es6 extenstion. Customizing the js extension allows me to search all of my JavaScript together. You can also define richer types in ackrc based on file names using regular expressions or direct name matching.

b. Set default options

Any option that the ack executable takes can have a default set or changed in an ackrc file. This is mainly useful for the display option. By default ack groups matches from different files and gives the file information above each group of matches. It also shows all output in color. Any of those things can be disabled or changed in your ackrc file.

c. Exclude unwanted directories or files

Oftentimes there are a predictable group of files that you may want to ignore in searches. Perhaps you have a build directory that mirrors your source code, or documentation files mingled with your source. You can put --ignore-directory or --ignore-file. Ignore directory takes a directory name, while ignore-file takes matches in the same format as the custom filetypes mentioned above.

If you’re starting to use ack, don’t miss the chance to make your searches even more efficient by removing any rough edges with an .ackrc.

Installation Instructions

ack came first; ag was created as a faster version of ack. ack is written in perl, making it more portable, and has a few more configuration options. ag is written in C and is generally faster. Your choice may vary depending on your use case.
↩
This is also one of the main traits that separates ack from ag.
↩

Stability vs Decline

Wed, 09 Mar 2016 02:04:39 -0400

My post on the sad state of the Backbone ecosystem from this weekend was surprisingly popular. I got a lot of responses pointing out some things I got wrong, sharing my concerns, or pushing back on my main points. Objections to the post came in 2 primary forms, which I want to quickly address. The first concern is that the Backbone ecosystem doesn’t matter. The second concern is that the trend I’m identifying as decline and decay is actually stability.

Does Backbone’s ecosystem matter?

The first concern came in 2 main forms. The first (and more polite) form was from commenters who shared their experience and said that they’d found Backbone’s core to be enough when combined with their own code. The second group was a generalization of that, effectively saying that Backbone is a minimal framework, and real programmers can write their own code on top of that. I’m sympathetic to the first framing, much less so to the second.

One of the beauties of Backbone is that it can be used in a variety of ways. Its small and simple enough that you can pull it in and use only part of it (maybe just the models or event system for instance), but it has been scaled to be used for large web applications like Trello, Rdio, Backbone, Bitbucket and others. Some of those use cases quite frankly, don’t need anything else most of the time. If you’re writing a very simple single screen piece of interactive behavior with Backbone, you don’t need any special extra libraries for that.

I personally work on a large scale web application (~200 Backbone Views) that has been developed for over 3 years and will continue to be developed (hopefully) for a long time. In that situation, having well documented, tested and supported third party libraries for common tasks like rich tables, form validation, UI component libraries, and integrations with other tools is tremendously useful. As an experienced developer, I’m capable of writing all of these things myself, but doing so takes time, and the result is likely to be less stable, less well documented, and less well thought through than a popular open source solution. To be clear, nobody is obligated to write these things for me. I’m merely making a factual statement that the experience of writing a large web application in Backbone becomes more difficult as the community around the library shrinks.

There’s a second reason why down playing the importance of an ecosystem is harmful though. I’ve said that I am capable of writing the various components of an ecosystem myself. But that’s only true of me because I have years of experience writing code, and specifically of working with different libraries that gave me ideas on how those problems should be tackled. A community that says you should be able to do everything yourself without starting places and good examples is a community that is hostile to beginners. Let’s not let the JavaScript community be that.

Is Backbone just stable?

The second disagreement (and the most common theme on Hacker News) was that the current state of Backbone and its ecosystem is not sad at all. Backbone is stable, and thats something to be celebrated. In some cases this echoed what I actually said in the post. I specifically called out the core Backbone library as a good example of stable software:

I’m starting with Backbone itself to give an idea what a stable mature project graph looks like. Backbone isn’t getting the activity it used to, but there’s still a steady flow of bug fixes and documentation updates, and issues are being worked through.

Backbone is stable, and in a vacuum that is something to be happy about. Stability has not prevented it from having an active team of contributors who are continuing to refine it and make small improvements. This is great, and on it’s own would make me incredibly pleased as a user. The problem comes with the libraries around it. Excepting Marionette, which appears to be getting ready to launch their long awaited version 3, activity in the ecosystem has stagnated in a different way than Backbone core.

Unlike Backbone core, secondary libraries have important issues requested by several people being ignored. I’ve chosen to avoid calling out specific projects because I don’t think anyone has a responsibility to fix other people’s open source problems and don’t want to add to open source guilt. But it is not hard to find examples of this if you browse through popular Backbone packages on Github.

Unlike Backbone core, secondary libraries sometimes have no clear active maintainer who can be counted on to put out a release in the case of an emergency change being needed. This matters because Backbone and JavaScript libraries in general live on an unstable platform. Browsers change all the time. While they do their best to maintain backwards compatibility, browsers occasionally introduce bugs, and occasionally fix bugs that a library may have been relying on or working around. In addition to disaster scenarios like that, tools change and best practices change over time. When documentation stagnates and doesn’t keep up with changes, libraries become less useful and approachable over time. Most Backbone tutorials on the web, and the documentation for many libraries, assume that scripts will be loaded in directly and that variables will be referenced through namespacing. In a world where module loaders are becoming a default tool for larger JavaScript applications, thats a gap that makes libraries hard to learn, or teaches out of date practices to less experienced developers.

A healthy stable project ought to be able to check off 3 boxes.

It meets the goals of its authors
Major bugs are fixed as they’re reported.
It is maintained to the extent that as new issues come in, they are responded to and dealt with

That definition could mean that a library handles a single use cases, and there is rare activity because nobody ever encounters any problems. But for larger projects, this likely means refinements to fix small issues, documentation changes, and an occasional big bugfix over time. Its these things that make the difference between stable software and abandonware.

What does it matter in the end?

I didn’t write my original post to say that Backbone is dead. I wanted to highlight a way in which the experience of using it has declined over time. It is specifically a way that makes using Backbone harder for me. I’m grateful to the many people who have built Backbone and the ecosystem around it, and think it is still a very useful tool. This ecosystem issue does make me unlikely to recommend Backbone for folks who are building web applications “at scale”. I think Ember or possibly React are better choices for that. For small one off projects, and for content focused sites that need as little JavaScript as possible while still providing a rich experience, Backbone is a great choice.

The Sad State of the Backbone Ecosystem

Mon, 07 Mar 2016 01:06:40 -0400

For the past 2 years, ~90% of my coding time has been spent working on a large scale Backbone.js app. In February 2014, the Backbone community was in a fairly good place. The core library had reached 1.0 the previous year and had a strong team around it. Marionette had emerged as a de-facto standard for web applications ¹. There were Backbone specific plugins for many tasks, and generally gaps could be filled using the jQuery plugin ecosystem, or a framework agnostic library. At the time, Backbone would have been a very defensible choice as a library for starting a new web development, as a simpler and more stable alternative to AngularJS, the current “hot” framework.

Since then a few things have happened. First, React has taken off, while Angular and Ember have continued to grow and learn from React and each other. Angular and React have benefited from significant marketing and financial support for a core development team that have helped them grow quickly. Google Trends gives a pretty good idea of how that has turned out.

Update: I’ve changed the above graph from an original that used angular.js, backbone.js, ember.js, and react.js as the search terms. Only Backbone prefixes itself with a .js that way, and it undersold the other libraries, especially Angular. I believe that this chart more accurately reflects usage trends.

Secondly, Backbone’s author, Jeremy Ashkenas made a decision to call Backbone “finished” in terms of API and feature set after the 1.0 release. This has the advantage of leaving Backbone as by far the most stable major JavaScript framework, but hinders efforts to pull in lessons from other frameworks ².

Finally, whether as a result of the first 2 items or due to some other factor, the ecosystem around Backbone has crumbled. While Backbone’s core has stabilized, it still has a team of maintainers who are actively managing the project. That does not appear to be the case for many of the other libraries in the Backbone ecosystem. To investigate my perception of this, I decided to look at the contribution graphs of popular Backbone libraries to see the trends over the last 2 years. I built a list of libraries based on the projects that the bower registry returns for the backbone keyword, looking for projects with at least 400 github stars³. That lead to a list of 24 projects below.

To be clear, I’m not posting this as a criticism of anyone. Open source is a volunteer effort, and maintaining a project is hard and often doesn’t make sense when the author has moved on to other challenges. This is simply an honest look at where the Backbone ecosystem stands, with the goal of helping those of us who are building Backbone applications or considering using it to evaluate the ecosystem honestly.

1. Backbone

Open issues: 54

Issues closed in 2016: 50+

Last Release: 3 days ago (March 3rd 2016)

I’m starting with Backbone itself to give an idea what a stable mature project graph looks like. Backbone isn’t getting the activity it used to, but there’s still a steady flow of bug fixes and documentation updates, and issues are being worked through.

2. Marionette

Open Issues: 107

Issues closed in 2016: 30+

Last Release: November 22 2015

Of all the graphs here, this one is the nearest and dearest to me. I’ve contributed code to Marionette, and have written about it many times on this blog. Marionette still is seeing plenty of work, but as the contributors have cycled through, work on a long awaited version 3 with important improvements has sat on a branch for over a year. This is a great library, but it hasn’t been immune to the wider issues in the ecosystem.

3. Backbone.Relational

Open Issues: 122

Issues closed in 2016: 0

Last Release: August 19 2015

Backbone.Relational is another “stable” library. On its own, there’s nothing wrong with that, this is a focused library that doesn’t necessarily need a lot of new features. But this is the start of a pattern.

4. backbone-forms

Open Issues: 172

Issues closed in 2016: 14

Last Release: Jan 21, 2016

5. backgrid

Open Issues: 111

Issues closed in 2016: 3

Last Release: Jan 21 2014

This is the first project on this list that I wouldn’t consider to be actively maintained. Commits over the past 2 years have been primarily about keeping tooling and documentation up to date.

6. backbone.localStorage

Open Issues: 45

Issues closed in 2016: 1

Last Release: Jan 22 2015

7. backbone.stickit

Open Issues: 26

Issues closed in 2016: 1

Last Release: May 11 2015

8. backbone.paginator

Open Issues: 20

Issues closed in 2016: 7+

Last Release: Feb 3 2016

9. Backbone-validation

Open Issues: 79

Issues closed in 2016: 1

Last Release: May 8 2015

I have personal experience with the abandoned state of this library since my team has been working off a forked version of it for 6 months while waiting on maintainers to address a pull request. Like many others here, it appears to be a project that the core team moved on from and nobody else had the motivation or time to pick it up. While there’s nothing wrong with that, it makes it hard to rely on these projects.

10. Knockback

Open Issues: 4

Issues closed in 2016: 1

Last Release: Sep 28 2015

11. Exoskeleton

Open Issues: 21

Issues closed in 2016: 2

Last Release: August 4 2014

12. Backbone.dualstorage

Open Issues: 30

Issues closed in 2016: 0

Last Release: Feb 16 2015

13. Backbone-react-component

Open Issues: 8

Issues closed in 2016: 1

Last Release: Nov 6 2015

14. Backbone-offline

Open Issues: 30

Issues closed in 2016: 0

Last Release: Aug 21 2013

This project is actually documented as being deprecated and discontinued. There is talk in the README about building a successor library, but it appears to have never emerged.

15. React.Backbone

Open Issues: 11

Issues closed in 2016: 0

Last Release: Jan 5 2015

Interesting, and not encouraging, that the 2 newest libraries at this popularity level are both for helping Backbone users migrate to React, and neither appear to be very active.

16. Backbone.expoxy

Open Issues: 19

Issues closed in 2016: 1

Last Release: Nov 20 2014

17. backbone.iobind

Open Issues: 16

Issues closed in 2016: 1

Last Release: Dec 10 2014

18. backbone-associations

Open Issues: 16

Issues closed in 2016: 0

Last Release: Jun 28 2014

We’re really starting to get into the meat of the dead projects now. While there are always going to be older projects that were once popular and got a lot of stars and are now abandoned, that shouldn’t be this many of the top 20 projects

19. backbone-tastypie

Open Issues: 12

Issues closed in 2016: 0

Last Release: Oct 26 2013

This is a single use case library that presumably was abandoned after solving a problem, but its worth including to illustrate that the 400 star bar isn’t that hard to hit. A similar search for React projects on bower yields 46 projects with 400+ stars, and React is only ~3 years old.

20. backbone.syphon

Open Issues: 19

Issues closed in 2016: 1

Last Release: Oct 18 2015

21. backbone-nested

Open Issues: 45

Issues closed in 2016: 1

Last Release: Jul 27 2015

22. backbone.radio

Open Issues: 9

Issues closed in 2016: 0

Last Release: Sep 5 2015

Backbone.Radio (along with Backbone.syphon) is part of the marionette project. You can see that those projects at least have continued work during the past year, albeit at a significantly less active rate than during 2014

23. backbone-fetch-cache

Open Issues: 26

Issues closed in 2016: 2

Last Release: Feb 16 2014

24. backbone-query-parameters

Open Issues: 10

Issues closed in 2016: 0

Last Release: Sep 23 2014

So what does this look like in aggregate. Of the “top 24” Backbone projects, only 2 were started after January 2014, and one of those is a tool to migrate Backbone projects to React, the other was a rebrand of an existing tool. 18 of the top 24 libraries have not had a release in the last 6 months, 9 have not had a release since December 2014 or earlier. If we classify these libraries as “Actively Growing”, “Actively being developed”, “Stable and maintained”, “Flickering Signs of life”, “All but dead”⁴, and “Dead”, I would personally split that out like this:

Title	Libraries
Actively Growing
Actively Developed	Marionette
Stable + minimally maintained	Backbone, Backbone.Relational, backbone-forms, backbone.paginator, Backbone.syphon, Backbone.radio
Flickering Signs of life	back grid, knockback,
All but Dead	backbone.localstorage, backbone.stickit, backbone-validation, exoskeleton, backbone.dualstorage, backbone.react-component, react.backbone, backbone-nested, backbone-fetch-cache, backbone-query-parameters
Dead	backbone.offline, backbone.iobind, backbone.epoxy, backbone-associations, backbone-tastypie

It is not a pretty picture right now. If you’re part of the Backbone community, I’d love to hear from you on this. Should we be calling Backbone dead and looking to migrate ASAP? Do you think I’m overselling this problem? Are there libraries that I’m not considering right now that are thriving? Or is focusing on Backbone-specific libraries a mistake because most developers are pasting together multiple libraries that aren’t necessarily tightly tied.

When I attended BackboneConf later that year, about 75% of attendees raised their hands when asked if they had used Marionette ↩︎
Some ideas, like a better router or better templating systems, can be pulled in as 3rd party libraries just fine. But conceptual changes like 1 way data flow or easily composable components from React and the ES6 class model can’t easily be pulled into Backbone without changes to the core library. It’s debatable whether those ideas are useful for Backbone, but it is clear that they will not fit within the current state of the library. ↩︎
I used Bower rather than npm because it returns search results sorted by stars rather than trying to directly match the query. ↩︎
I don’t like calling somebody else’s project dead if its not labeled that way and has had recent issues activity ↩︎

Staying Productive

Mon, 25 Jan 2016 00:41:46 -0400

I’ve been thinking a lot lately about what makes me productive vs unproductive. A lot of the time Software Developers tend to focus on their tools and environments when they think about productivity. I’m certainly all into that, as you can see if you look back, through, my blog. But in my mind the most important factors in building productivity are non-software related.

Setting Goals

By far the most important thing I’ve found for being productive was setting clear goals for what I want to get done. Without goal posts to aim for, it’s hard to tell what you’ve really accomplished, and easy to get pulled off onto suboptimal tasks. Goals are best set with a wider context in mind. The act of setting and measuring goals tends to make us focus on those areas, so make sure your goals reflect your priorities. If you set all of your goals around work, it can be easier to justify neglecting your family. If you set a goal of writing every day, that might overwhelm time for exercising and keeping yourself healthy. Therefore its helpful to set goals in a holistic way that allows you to judge what you value in the big picture.

Mechanical / Creative Tasks

I saw a great comment on Hacker News the other day that differentiated between projects and chores. The idea is that when you’re feeling productive you should work on projects, when you’re not you should do chores. One of the biggest boosts to my productivity I’ve found was when I started categorizing work in my head as “mechanical” vs “creative” tasks. Mechanical tasks (ie chores) are things that require little mental energy. They’re well defined and possibly onerous. Examples include filing email, documenting code, code reviews, and low-thinking code tasks like reproducing a client issue or adding a small well-defined feature. Creative tasks for me tend to look like more involved code changes designing new features, or debugging client issues that aren’t obvious.

I’m not universally energetic or productive. Some days I can focus intently on a problem, other days I have the attention span of the dogs from UP!. When I’m able to focus, I intentionally put aside mechanical tasks, and leave them for times when I have less energy, continuous time or attention. Oftentimes I’ll even put off higher priority mechanical tasks (or vice versa) to match my mental level to the task, because I end up getting more done sooner in the long run.

Productive Environments are a thing

It’s easy to think of productivity as a purely internal thing. But the environment you’re in matters too. For creative software work that means getting large blocks of non-distracting time. That requires some intentionality. I’ve been blessed with a workplace where I’m given plenty of flexibility. That means that at times of high distraction-level (big projects, lots of meetings) I tend to actually do most of my work outside of the office, sometimes at weird times. I use my time at the office for mechanical tasks and meetings, and do the creative work elsewhere. That’s obviously suboptimal, and when I can I try to clear time during days to do creative work at the office and bunch the distractions. However it works out for you, the key is to (as much as possible) group your distractions and leave large un-interrupted periods for creative work.

Distractions can be self-inflicted as well as coming from others. When I want to get stuff done, I try to listen to familiar music (oftentimes looping a single album for a few days). I’ll also close Slack/Twitter/Email and other distracting apps. Note that the distinction between mechanical and creative is important here. When doing mechanical work, I’m fine having distractions around. It’s only when I’m trying to harness mental energy that I care.

Day to Day work adds up

I’m a magpie. I love shiny new tools, projects, and ideas. It’s very easy for me to get bored with a problem once I feel like I understand it. I get excited by pouring effort into learning something new, not so much the hard work to finish the last 20% of what I’ve started. What I’m slowly learning though is that it is my day-to-day work that makes an impact, not my crazy productivity binges. Making things that last requires day to day dedication. That also means maintaining focus. The easiest way to waste day to day hard work is to spend it on a different thing each day. There’s nothing wrong with trying new things. But real productivity requires setting some priorities and sticking with them for a while.

Team matters more than the project

Finally, individual productivity doesn’t happen in a vacuum. Who we work with can undermine or boost our results. I don’t believe that it’s a small difference either. I’d rather have a great team with no individually brilliant members than a bad team with one or two great programmers any day. One of Windsor Circle’s values is that teams survive longer than individuals. The same can be said of their results. Teams can magnify what is possible for a product. Any individual has limits on what he or she can create. Teams allow for those limits to be lifted by building on each others skills. At the same time, teams can limit each other by fighting, mis-communicating, distracting, or pursuing different goals. So all of the other productivities ideas here can be applied just as well to a team as they do to an individual. The best work comes from teams consistently working together towards a common goal over a long period of time in a constant and productive environment.

The Most Interesting Atom Packages I've found (so far)

Sun, 10 Jan 2016 22:25:49 -0400

I’ve been using Atom as my primary text editor for the last 3 months. I don’t completely love it. There are elements (startup speed, maturity of some core editing components, not being able to run it in a terminal) that I miss from my experiences with Vim and/or Sublime Text. I’m using Atom though because the core editing experience is now “good enough” and the way the editor is built allows for some very cool and unique experiences. Atom (like Visual Studio Code) is built on Electron, Github’s JavaScript environment for building desktop apps. Because it is built with web technologies and was designed from the beginning to be extensible, it is much easier for most developers to build rich visual extensions on top of Atom that provide unique UIs for editor functions than it is in Sublime Text, Vim, Emacs or an IDE. The following are some of the “most interesting” Atom packages I’ve found. Interesting in this case means that they really only could have been done as they are in Atom or a similarly flexible editor.

Pigments

At first use, Pigments seems like a pretty run of the mill color highlighter. There are similar projects for Sublime Text and Vim that look at color names in a CSS or LESS file and highlight them with the color they represent.

But Pigments does much more than that. In addition to highlighting text, Pigments searches your project and creates a directory of all the colors used in your projects. You can look through your palette and quickly go to the definition of any individual color. Because pigments tracks everything, it can highlight LESS/SCSS variables as well as plain color names, even if those variables were originally defined in a different file. Pigments also hooks into Atom’s autocomplete, and displays a preview of the color options when autofilling colors.

It is a rich way of managing colors for a whole project, the best way of seeing at a glance every color used in your CSS project that I’ve ever seen.

Expose

Expose is a file management tool modeled after Mac OSX’s expose feature. With it, you can instantly display all open files as small thumbnails, and switch quickly between them using the keyboard, which can be super helpful when tabs get out of control.

Autocomplete-plus

Autocomplete-plus started as a community package but has now become part of the core editor. It’s also probably the least unique idea I’m listing here. It’s still worth including on this list though, for 2 reasons. First, autocomplete plus has flexible visual context options, allowing autocomplete providers to show the type of the autocomplete option, a brief description, and potentially a more in depth description. So users can distinguish between (for instance) completions offering a snippet, a function, a keyword, or an import.

Autocomplete-plus is also a great example of the flexible nature of Atom. Because Atom itself is made up of packages, with a very small core package, it’s easy to replace key parts of the editor like the auto-complete package with a community version. And if that version ends up being better in the long run, it’s possible to pull that into the editors core, as happened with autocomplete-plus.

Todo-show

Todo-show is a cool little package that searches your project for comments containing TODO, FIXME, CHANGED, REVIEW and other configurable comment strings, and lists them as a table. It’s a simple idea that formalizes something that many developers do all the time.

Update: Twitter has pointed out this even crazier riff on this concept that I’d seen a while ago but had been unable to find when I dove back into Atom. Imdone is a trello-like experience in the editor for managing todos.

@_benmccormick @AtomEditor You might also like imdone-atom. It's a drag and drop TODO task board with issue tracking integration!
— imdone.io (@imdoneIO) January 11, 2016

Linter

Similar to autocomplete-plus, you may think you’ve seen Linter before. It’s similar to Syntastic for Vim and SublimeLinter for Sublime Text. But Atom’s flexibility allows it to be better than both of those. Like Syntastic and SublimeLinter, Linter integrates with many different linting providers to show errors and warnings that tools pick up on through static analysis in files. But rather than being limited to the line-by-line gutter warnings or showing collections of warnings through awkward use of UI elements meant for a different purpose, Linter shows you a live collection of errors at the bottom of the file and lets you easily switch back and forth between that and errors for all open project files.

git-time-machine

git-time-machine is the newest package on this list. It’s only a month old, but is quite possibly my favorite example of Atom’s potential. After setting it up, you can use a keyboard shortcut to see a bubble plot of that files git history, and then hover to see the commit messages, and click to see an old version of the file. It is the most accessible UI I’ve ever seen for working with git history.

Reusable Code Patterns

Fri, 08 Jan 2016 01:22:29 -0400

One of the most repeated mantras in all of software is the term “Don’t Repeat Yourself”. The phrase was first popularized by the book The Pragmatic Programmer, with it’s call for DRY software. At it’s core, DRY is all about finding a way to keep logic related to a particular function in one place, so that changes for new functionality can be limited in scope and have predictable consistent effects. Software developers are usually very good at knowing that repetition is bad, but I’ve seen that we can be less great at communicating about our options to avoid it, or discussing the tradeoffs between different approaches. This post is going to look at a specific situation that programmers encounter all the time, and give a framework for talking about how to handle it, focusing on positive options rather than simply what we want to avoid.

The high level situation I want to talk about is when a programmer encounters a problem for which they have an existing piece of code that mostly works as a solution, but doesn’t quite meet all the requirements. Some examples would include:

a screen that is required to show a user avatar, but at a different size than other avatars displayed in the interface (all of which use an existing avatar ui widget)
a program that already has an API to get friends data from a users Facebook account, and needs to add a second API to get similar data from their Twitter account
a program that needs a function that takes an integer or a string and displays it on the screen, when it already has a function that displays data but only accepts strings

Some of those are simple, some are more complex, and you might already be thinking about how you’d handle each case. The point isn’t so much the exact example so much as having a way to communicate what the options are. I’m going to lay out 6 different patterns for handling situations where you’re required to implement something similar but not exactly the same as something that already exists. I will not be endorsing a particular strategy as correct, since they all have their place. Instead I’ll be discussing the benefits and risks of each, and hopefully giving a good framework for thinking about and discussing these tradeoffs in the future.

Using the existing component as is

Pros

Least Effort
Encourages consistency and simplicity

Cons

Often not allowed
May be compromising user experience for developer expediency

The first option is the simplest. If you have a component that is similar to what is required but doesn’t quite match the requirements, change the requirements. This can make sense in situations like the avatar example above, when you have a UI element or control that you use everywhere in the system. It may make sense to use that same element in a new place for the benefit of consistency and user knowledge, rather than inventing a new UI element for the page. This obviously requires an environment where developer input is welcome in the design and requirements process.

Developers should be cautious with this though. If using an existing component removes unecessary complexity from the design and saves time, it’s a win-win. But if the design was a good one, and users lose out on functionality through developer “efficiency”, it can hurt the product. Even worse, decisions like that usually end up coming back round again, as the need that was ignored often will pop up in new situations.

Copy and Paste

Pros

Simple
Keeps existing code stable in the short term

Cons

Adds to maintenance burden
Decreases readability/understandability of the code base for other developers

This is the approach that says “To Hell with DRY”, and takes some existing code, creates a copy of it, and then partially modifies it to suit the current situation. This would be analogous to taking the User avatar code listed above, creating a copy of it and simply modifying it to be bigger. That might look like this in CSS:

.user-avatar {
    height: 30px;
    width: 30px;
    padding:5px;
    float: left;
    border-radius: 5px;
}

.user-avatar-big {
    height: 50px;
    width: 50px;
    padding:5px;
    float: left;
    border-radius: 10px;
}

Everything now works exactly the way we want it. The problem comes when somebody wants to change the floats on avatar elements, or add a border to all of them now. Instead of being able to make one change, they have to be aware of both avatar versions, and modify both.

Note that copy and paste isn’t always bad. Often there are very good reasons for not modifying existing code. It may be code that you don’t control (for instance forking an open source project that is not actively maintained to add a feature that you want), or code that is from another project (or from a legacy portion of the current project that is handled as a different code base). DRY is not crazy advice though, and if you’re operating within a single codebase you control, this is usually not the best option.

Classical Inheritance

Pros

Well understood by most programmers, especially those with an academic background
fits very well when you have multiple things that are variations of a type, with more similarities than differences

Cons

Often used as a silver bullet for all situations
Not suitable for situations where the objects are more different than similar, or when they are 2 different things at the core with shared behavior

If you ask a recent Computer Science graduate what the best way to solve the problem of similar behavior across objects is, 99% of them will immediately say inheritance. The “conventional wisdom” that’s taught in most introductory computer science courses is that Object Oriented Programming is the best way to structure programs, and that Object Oriented programming is primarily about inheritance. Both statements are shaky, but the prevalence of this mindset means that many developers come out of school and look at every problem they see as a nail to hit with their inheritance hammer (this is certainly something I was guilty of when I first graduated).

Inheritance isn’t bad. It’s definitely a tool you want in your toolbox, and shallow hierarchies of objects that inherit from a base class are a great pattern. 90% of “Frameworks” out there (Backbone, React, django, Rails, etc) use some form of inheritance as a way of structuring how users interact with the framework. But when it is your only tool, things get ugly quickly.

Inheritance is a bad choice when 2 objects are mostly different. Inheritance forces objects to share an API, and implies that objects can be used interchangeably. If the objects aren’t really interchangeable, this can lead to jumping through hoops to maintain a shared API, with a small base class being stretched between 2 or more verbose child classes. Interfaces especially tend to be bad places to use inheritance, as they tend to require small tweaks and differences when used in different places that aren’t friendly to being passed as configuration properties or over-ridden methods.

Configuration

Pros

Keeps everything in a single place
Declarative configuration can be easier to read and update than a bunch of logic

Cons

Can get out of hand fast as use cases increase
Often unclear what defaults should be, bad defaults make common use cases verbose

We can define separate behaviors for 2 similar things by using a single configurable object. This maximizes code re-use since everything is in one place, and provides an easy to read abstraction over the code for quick changes later. However, like inheritance it works much better for similar objects with small differences rather than different objects with shared behaviors, and can lead to “configuration creep” when the differences slowly grow, creating a monster configuration and spaghetti code that tries to handle all of the possible configuration options it could be passed.

A big design point with configuration based strategies is figuring out good defaults. If an object is designed so that the default case requires minimal to no configuration, and only true edge cases require a lot of configuration, configuration based objects and APIs can be very pleasant to deal with. But objects with bad or no defaults that require common cases to include extensive configuration eventually lead to bloated projects that are very difficult to debug.

Mixins

Pros

Share individual behaviors or attributes well

Cons

Can break down when the behavior or attribute requires significant knowledge about the object

In contrast to inheritance and configuration, mixins work well when objects are significantly different, but share similar behaviors. Mixins are the strategy of adding individual methods or properties to an object without implying any relationship between objects that use it. Mixins are great when used to add a standalone behavior to different objects. They avoid the problems of inheritance and don’t lock an object into a single API.

That said, it can often be hard to construct good mixins, since many behaviors require knowledge of the object that is performing the behavior, such as making assumptions about the structure of the objects data. This either requires the objects implementing the mixin to obey rules about how they’re structured, similar to inheritance, involves offloading significant complexity into the mixin to allow it to handle different formats (making it a potential source of bugs), or it risks naming conflicts if the mixin tries to implement the data storage itself.

Composition

Pros

Incredibly flexible for handling subtly different behaviors

Cons

Can ramp up complexity, making it difficult to see the whole system
May end up pushing code re-use problems downwards instead of solving them, leading to many small similar modules

Often when faced with 2 similar objects or functions, the best solution is to create more objects or functions. Composition is trendy these days. On the server side, you have people talking up micro-services and programs composed of small single-purpose modules. On the client, React is the hottest library around. The idea is appealing. Instead of having one big complicated “thing”, I can create a few smaller “things” that I can wrap my mind around, and then get them to work together. When this works well its great. React for example, makes testing UIs much easier, since you can unit test individual components in a reliable way. The problem is that it often moves complexity out of the components and into the architecture. The hard part becomes figuring out how components fit together, designing interfaces and deciding what should be going where. Although any individual piece of code may be comprehensible and simple, the system at large is harder to visualize and understand.

Like mixins, composition often works best when you have 2 or more fairly different “things” that share behavior. The composition mindset is to treat code like lego blocks that you can snap together in different configurations as needed. When done well, it allows for useful abstraction, letting developers create complex components out of meaningful sub-components without having to know all the details of the subcomponents. When done poorly, it can lead to a fractured project with many undocumented dependencies between components.

All Together Now

The great thing about these strategies is that none of them is the right answer. All of them have their place, and can even be combined. At my work we’ve developed code for describing workflows. The workflows are described using an object that takes a configuration and uses it to layout a series of widgets. Each widget inherits from a framework, and many of them use mixins to share behavior. Most of the widgets are composed of several smaller views, and as developers we have pushed back against designs several times in order to allow greater widget re-use. Each time we need to create a new automator we’ve started by copy and pasting some of the configuration boilerplate (which hopefully we’ll reduce going forward by providing better default behavior as we understand the requirements better). This is one feature of a large software product, and it uses all 6 reuse patterns I’ve described to perform its purpose.

In the end, what I hope you as a reader get out of this is not that one strategy is good or bad. Instead I want to provide a way of thinking about the choices around re-use, so that we as developers can talk intelligently about the options and tradeoffs and make the right choice for a given situation.

2015 Roundup

Thu, 31 Dec 2015 01:56:04 -0400

Thanks to all of you who have followed my blog this year. More people visited the blog this year than in its previous 2 years of existence combined. I’m excited to continue writing and starting conversations in 2016.

What Backbone Developers Can Learn From React - A look at lessons that Backbone developers can take from the React community.
Building Modular Web Apps With Backbone.Radio - A deep dive into Backbone.Radio, a messaging system for Backbone and Marionette applications.
ES5, ES6, ES2016, ES.Next: What’s going on with JavaScript versioning? - An explanation of the weird and confusing naming conventions for JavaScript versions
Why Backbone.js and ES6 Classes don’t mix - An examination of the problems that developers face trying to use ES6 classes and Backbone together. A followup article explored some potential solutions.

Code I wrote in 2015

Most of the code I wrote this year was for my work at Windsor Circle and on a React based side project that is not yet ready for prime time. But I did get

to release a small service library for Marionette developers, and made several contributions to the core Marionette library.

Languages and Libraries I started using in 2015

Babel has had by far the biggest impact on my code in 2015. Being able to write modern JavaScript without compatibility concerns has been a breath of fresh air
My team switched from Underscore to lodash at the end of 2014, and I’ve been using it a lot more this year. Like Babel, it changes the way I write code.
I don’t use it day to day, but I’ve loved playing with React and Redux. I think UI as a pure function of state with a defined set of user actions/mutations is the future.

Languages and Libraries I stopped using in 2015

With ES6 becoming a valid option, I’ve moved on from Coffeescript. It’s a beautiful, non-standard language that is never going to develop the tooling that is building up around the standard JavaScript language.
I haven’t stopped using jQuery, since Marionette uses it for event handling and ajax, but I’ve begun looking at direct jQuery-style DOM manipulation as a code smell
This mainly happened early in 2014, but 2015 was the first year I didn’t write a single line of Java. I haven’t missed it.

Languages and Libraries I used (more or less) every day in 2015

JavaScript (ES6), Python, Backbone, Marionette, lodash

Languages and Libraries I want to try in 2016

Elm or ClojureScript, Rust or Go, Flow, Ember, Pandas and other python data analysis libraries

Blogs I started following in 2015

2ality is a great blog by Axel Rauschmayer focused on the JavaScript language
rauchg.com is not updated often, but all 3 posts so far have been fantastic
raganwald.com is a great resource for functional programming fans who write JavaScript

Blogs I read every post from in 2015

Rands In Repose, Stratechery, Marco.org, rauchg.com, Little Big Details, Bull City Rising

Software I started using in 2015

I started using Atom this fall as my primary code editor. There are things about it I like less than Vim or Sublime Text, but it’s configurability and the fact that people have used it to make beautiful, intuitive UIs for tools that I use have sold me for now. I still use Vim on a regular basis when I want to stay in the terminal, and am keeping an eye on Neovim
Ghostery is a fantastic ad blocker that focuses on transparency, showing you the ad networks and trackers that are running on a page and allowing you to selectively block ad networks or sites, helpful for those of us who want to support sites we use and allow analytics but are overwhelmed by the slimy, slow state of the web.
Overcast is a great podcast app for iOS that has convinced me to start listening to podcasts. (I’ve enjoyed The Changelog, Exponent, and Serial).

Software I stopped using in 2015

Mailbox: RIP
OneNote: I made an effort to switch from Evernote to OneNote this summer, but it failed because it’s much harder to get stuff into OneNote for those of us who live in Apple-Land. Not enough integrations

Software I used (nearly) every day in 2015

OSX/iOS, Google Search, Chrome/Safari, Gmail, Twitter/Tweetbot, Feedbin/Reeder, Instapaper, iMessage, Slack, Trello, 1Password, Atom/Vim, iTerm, Fish Shell, tmux, Bitbucket, Fantastical

ES6 Patterns: Converting Callbacks to Promises

Tue, 29 Dec 2015 23:30:47 -0400

I've been writing code using the new features defined in the ECMAScript 2015 version of JavaScript (more commonly known as ES6) since January. Throughout the year I've seen myself start using a few new patterns that I think make my code better. I'm going to share them here with a few quick hitter articles. If you missed it, I wrote about clean higher order functions earlier this week. This post focuses on working with asynchronous code in ES6.

Background

One of the nicest new features of ES6 JavaScript is the standardization of Promises. Promises are a method for managing asynchronous code that serve as an alternative to the standard callback function syntax that has been the JavaScript standard for years. If you’re unfamiliar with them, a good example of a Promise based API is the new fetch api provided by browsers. fetch is a replacement for the older callback based XMLHttpRequest API. A quick example of an HTTP request with the 2 APIs provides a nice comparison of how Promises can lead to clearer code.

A simple get request with XMLHttpRequest looks like this:

function reqListener() {  
  var data = JSON.parse(this.responseText);  
  console.log(data);  
}

function reqError(err) {  
  console.log('Fetch Error :-S', err);  
}

var request = new XMLHttpRequest();  
request.onload = function() {
  var data = JSON.parse(this.responseText);  
  //do stuff with data
};  

request.onerror = function() {
    alert('There was a problem with the request');
}
request.open('get', '/api/foo/bar', true);  
request.send();

whereas with fetch we get this instead

fetch('/api/foo/bar').then(function(data) {
    return data.json();
}).then(function(jsonData) {
    //do stuff with the data
}).catch(function(e) {
    alert('There was a problem with the request');
})

This is admittedly a slightly unfair example due to XMLHttpRequest’s clunky object oriented API, but the key take away here is that promises allow for easier visualization of a program’s flow, as well as the ability to easily chain both synchronous and asynchronous operations together into a unified API.

Promises have been around for a little while in user-land. There are a bunch of Promise libraries out there that eventually standardized on a spec called Promises/A+. Promises/A+ compliant libraries include Q, Bluebird, and rsvp. There are also many older libraries that provide Promise-like capabilities but are not completely spec compatible, most notably jQuery deferreds. But with ES6 Promises are being standardized. Happily, since the implementation uses the standard that user-land libaries have agreed upon, the Promise spec is compatible with existing implementations, and existing code bases can remove their existing libraries in favor of the browser supplied version, or have code written to use the browser version interop cleanly with their existing code. Support for Promises now exists in the latest versions of all major browsers, but it never made it to Internet Explorer and is Edge only for Microsoft browsers. So most developers will still want to consider using a polyfill for the time being.

Converting callback-driven code to use Promises

If you believe that Promises are worthwhile, you’ll immediately encounter a problem in today’s JavaScript world. Many JavaScript APIs, including most standard browser APIs and older but still popular libraries like jQuery and Backbone are heavily callback driven. Rather than mixing 2 different styles of asynchronous code, wouldn’t it be nice if we could easily convert callback-based APIs to use Promises? It turns out that it’s not that hard. Let’s take the simplest example possible to start. setTimeout is a straightforward browser API that waits for a specified period of time and then executes a callback. A standard use looks like this:

function doStuff() {/*...*/}

setTimeout(doStuff, 300);

A Promise-based API for this function would likely look something like this code.

timeout(300).then(doStuff)

We can create an API like that using setTimeout. To do that, we’ll need a function timeout which takes a timeout variable and returns a Promise.

You can define A+ compliant Promises using the Promise constructor, which expects a single function as an argument. That function takes 2 arguments, a resolve function and a reject function. The wonderful thing is that under the covers these are just callback functions that the Promise api glosses over.

Since we already have an API that can handle callbacks, the implementation of our timeout function is pretty simple.

function timeout(delay) {
    return new Promise(function(resolve, reject) {
        setTimeout(resolve, delay);
    });
}

We don’t use the reject callback, since setTimeout doesn’t provide any hooks for an error state. So we pass resolve as the callback to setTimeout, and that is all we need. Now we have a great chainable setTimeout function that we could include in a Promise chain.

Moving on to a more complicated example, let’s take our XMLHttpRequest code from above and see if we can create a simplified version of the fetch API using XMLHttpRequest under the covers. In this case I’m going to use ES6 style arrow functions to reduce the boilerplate a bit.

const fetch = (url, options = {method:'get'}) => new Promise((resolve, reject) => {
    let request = new XMLHttpRequest();  
    request.onload = resolve
    request.onerror = reject;
    request.open(options.method, url, true);  
    request.send();
});

This is a simplified implementation that doesn’t come close to covering all the use cases of fetch, but it provides a great example of how simple it can be to transform a callback based api to a Promise based one, without having to rewrite the existing code¹.

For what it’s worth, it is equally easy to convert functions in the other direction. For instance a callback based implementation of fetch can be a one liner.

const callbackFetch = (url, options, succ, err) => fetch(url, options).then(succ).catch(err);

It’s useful to know that the various syntaxes for asynchronous code in JavaScript are effectively equivalent and interoperable. When designing APIs for your code, both for public libraries with external libraries and components that you use in an application, these patterns can be useful for providing a consistent API, even if you’re using code under the covers that presents it’s asynchronous code in a different way than your API.

More Resources

The free online version of Exploring ES6 by Axel Rauschmayer has a great chapter on ES6 Promises explaining the API and how they fit in with the A+ standard.
There was also a great post by David Catuhe on one of the MSDN blogs earlier this year highlighting how asynchronous code has evolved in JavaScript over the years, including a look at the new async/await patterns proposed for ES2016.

And even a full implementation of fetch using XMLHttpRequest is only ~300 lines
↩

ES6 Patterns: Clean Higher Order Functions

Mon, 30 Nov 2015 01:06:03 -0400

I've been writing code using the new features defined in the ECMAScript 2015 version of JavaScript (more commonly known as ES6) since January. Throughout the year I've seen myself start using a few new patterns that I think make my code better. I'm going to share them here with a few quick hitter articles, starting with a simpler way of writing higher order functions

A higher order function is a function that either

Takes another function as an argument or
Returns a function when called

Although many people have never heard of them, higher order functions are a key part of JavaScript development and something that most JavaScript developers will encounter every day, whether they know the term or not. jQuery, Underscore, and many other JavaScript libraries are built on higher order functions. Many of the core JavaScript APIs, like map, filter, forEach, and setTimeout are functions that take functions as arguments. If you’re not comfortable with these patterns, learning about them can help you understand the JavaScript ecosystem better. Hopefully this article will pique your interest.

For this pattern, I’ll be focused on the second class of higher-order functions, functions that return other functions. They can be useful in a variety of cases. As an example, let’s consider a blog commenting application with a list of users. Our task is to get a list of comments for each user. We can do that with the builtin map and filter functions and a simple helper function like this.

import { commentList } from './comments';

function getCommentFromUser(userId) {
    return {
        user: userId,
        comments:commentList.filter(function (comment) {
            return comment.user === userId
        },
    };
}

let userIds = [1, 4, 5],
    userComments = userIds.map(getCommentFromUser);

//do stuff with comments

We’ve now written some simple, working code. We take a list of user ids and return a list of users with the ids mapped to their comments. Everything is fine, and we can move forward to do things with the comments. If we don’t need to maintain this code base, we could probably stop here.

The problem is, while this example works for the simple case, it isn’t very flexible. We pull the comments directly from a single source that we’ve made available to all of the code in our module. We can’t add any additional filters on the comments used or pull comments from additional sources without rewriting our mapping logic. If we decide we only want comments that were written in the past year for instance, we’ll need to rewrite our function or mutate commentList in a way that could effect code in the rest of the file. That feels pretty sloppy, since mapping comments to users should be unrelated to determining what comments we’re including in our groupings. Fortunately we can solve this using a higher order function. If we rewrite the code so that getCommentFromUser takes a list of comments and returns a function that takes an id, we can then control exactly what comments are included in the grouping. For now we’ll leave our full list, but it could be rewritten later to be a filtered list or to pull comments from multiple sources, without re-writing this logic.

import { commentList } from './comments';

function getCommentFromUser (comments)
    return function (userId) {
        return {
            user: userId,
            comments: comments.filter(function (comment) {
                return comment.user === userId
            },
        };
    }
}

let userIds = [1, 4, 5],
    userComments = userIds.map(getCommentFromUser(commentList));

//do stuff with comments

Our code has gotten much more flexible, but at the cost of readability. Most developers without significant functional programming experience will have to look twice at the 4 levels of nesting and multiple return statements in our helper function definition to understand what is going on. We’ve added significant boilerplate in a way that muddies up the meaning of our function, when all we’re really trying to do is provide a way to make the comments list used in the function configurable. That is extremely “beginner-unfriendly”. At the same time using higher order functions like this is a useful pattern that serves a real need in the design of our application. Fortunately, ES6 gives us the opportunity to clean up our syntax.

ES6 provides “arrow functions”, a shorthand for functions that have 2 main differences from the JavaScript function keyword. First, they don’t bind a this value as a context, instead using the this of the parent scope. Secondly, they can be written inline, with no parentheses and an implicit return. So you can write a squaring function as const square = (x) => x * x;. The first property of these functions has many consequences that have been covered in depth elsewhere around the internet. But when writing non object oriented functions that don’t use this like our example above, it is mostly not relevant. In this case though, the cleaner syntax that arrow functions provide allow us to write a clear and clean higher order function to solve our use case. With arrow functions, our example from above now looks like this.

import { commentList } from './comments';

const getCommentFromUser = (comments) => (userId) => ({
    user: userId,
    comments: comments.filter( (comment) => comment.user === userId),
})

let userIds = [1, 4, 5],
    userComments = userIds.map(getCommentFromUser(commentList));

//do stuff with comments

I like to call the () => () => pattern a “double function”, and have found it popping up all over the code I write since I began using ES6, especially when I’m using map, filter, and reduce. Array transformations using those functions often require contextual information beyond the contents of the array. A naive approach to this would involve having the function reference variables in the outer scope directly, like we did with commentList above. More commonly that happens when a user inlines a function inside of a map call and references variables in the parent scope. I’ve found that it is much clearer and less error prone to use a higher order function that takes only the variables that are needed for the transformation in, and returns a new function that can be used for the transformation. This reduces the number of things that can effect the output of the function, allows for greater configurability without directly changing the transformation logic, and makes it clear to readers what values are expected to be used in the function. Double functions make this easy, and they’ve easily been one of my favorite new code patterns with ES6.

More Resources

If you’re interested in higher order functions and JavaScript, I know of no better reference than JavaScript Allongé. It’s a fantastic deep dive into the JavaScript language with a functional flavor, building understanding from the bottom up and providing many ideas for code recipes along the way. And it’s recently been rewritten to reflect the changes to the language that ES6 brought.
Here’s a nice deep dive on arrow functions at 2ality by Axel Rauschmayer. If you haven’t seen his blog, it’s a fantastic reference on ES6 and JavaScript in general

Productive JavaScript Development

Wed, 25 Nov 2015 10:43:46 -0400

I’ve been thinking a lot lately about how we as a JavaScript community can enable more productive JavaScript development. Most of the major JavaScript libraries were created or popularized to further a theory of how we can be more productive. jQuery emerged when the biggest productivity challenge for web developers was the incompatible implementations of JavaScript and browser APIs in different browsers. It largely solved that problem. Since then, Backbone advocates have pushed for greater productivity through libraries with minimal abstraction where users can understand the source code, Ember has created an ecosystem around the idea that “shared solutions” lead to greater productivity, Angular aimed to make developers (and designers) more productive by adding more power to concepts that were already familiar to them, and React is trying to improve productivity by simplifying the programming model around interfaces. Meanwhile, tangential to the big frameworks, there has been work to enable more productive development through transpile-to-JavaScript languages (CoffeeScript, TypeScript, Elm, ClojureScript etc), replacing JavaScript on the web period (Dart, WebAssembly, ASM.js), and improving the core JavaScript language while allowing developers to use new features early (ES6 and beyond, as well as Babel). There’s also work to improve tooling around JavaScript with projects like Tern.js, Time Machine/React Hot Loader, and continued improvements to Chrome and Firefox developer tools (along with Framework specific developer tools). So there’s no shortage of work being done in this area. But let’s pull out for a moment. What are the big picture things that affect productivity?

Most of the time productivity tools are focused on one of 6 areas. They make it easier to learn concepts, easier to write code, easier to read code, easier to architect applications, easier to maintain applications, or easier to debug applications. Here are some examples of each

Easy To Learn

Angular and Backbone both try to improve programmer productivity by being easy to learn, each in different ways. Angular puts much of its view logic inside HTML, a medium that many server-side developers and designers are already familiar with. Therefore it’s a reduced learning curve than something like React, where the UI tends to be more spread across files and is mixed with JavaScript logic ¹. Angular also provides extensive documentation, which helps programmers come up to speed faster.

Backbone on the other hand, attempts to be easy to learn in two ways. First, there just isn’t that much to learn. Backbone is a small library with only 6 core concepts (Models, Collections, Views, Events, and Router) each of which should be familiar to developers coming to it from other UI focused frameworks. That simplicity reduces the load on new users. At the same time Backbone strives to have libary source code that is both short and readable. It’s quite possible to read the code for the whole library in under an hour, and the core concepts are obvious when looking through the code.

Easy to Write

Ember.js aims to make it easy to write application code in at least 2 ways. First it provides tooling to help generate the boilerplate necessary for an ember application. You can give ember-cli a model name and it will generate the file for that model in the correct place. In addition Ember abstracts away many of the connections between classes so that you can load the correct files simply using naming conventions, resulting in less code.

The recent updates to the JavaScript language are about much more than just making it easier to write JavaScript, but several of the features are focused on providing cleaner syntax for common idioms. Arrow functions in particular allow for maintaining the outer scope of a function in a clean way, when previously it required jumping through hoops that were easy to mess up.

Redux and React Hot loader are an attempt to develop a programming idiom that allows for drastically reduced latency between making a change and seeing a result. A React app using redux and a hot loader can be updated instantly without losing state and cycle backwards and forward through state updates. Lower latency between writing code and seeing the results makes it much easier to iterate quickly towards good code.

Easy to Read

One of the biggest gains provided by React is an increase in the readability of code by combining all of the elements of a component (JavaScript, HTML and in some cases CSS) into a single file where the component can be read as a coherent whole without flipping through code.

CoffeeScript is an attempt to strip out the unnecessary syntax bloat of JavaScript and focus on the pure logic. Whether it succeeds is a totally different question, but it’s a great example of attempting to satisfy this side of the productivity problem by reducing the distractions in code.

Type annotation tools like TypeScript and Flow also aim to increase code readability, but not by clearing out bloat (annotations can actually make the code feel more crowded and less readable at first). Instead they add additional information to the code, making the structure and expectations of the code more apparent to a reader.

Easy to Architect

Ember really shines when it comes to making it easy to architect apps. By providing clear guidance on how to structure apps, it can increase productivity by reducing bike-shedding. In Ember every state of the UI is tied to a URL and each route is tied to controllers that display components. Communication between components happens with actions and data is managed by Ember Data. Thats a reasonable architecture thats general enough to work for most large web applications (Embers target).

Unfortunately examples of innovation in this area are much more sparse than the others here, probably because it’s hard to demonstrate in a todo app. There’s been a recent spate of interest in application architecture from all of the Flux implementations that have popped up for React apps. I’ve also seen interest in the architecture model of Elm. But most of the major MVC frameworks have much less community agreement on architecture than they do on smaller scoped issues.

Easy to Maintain

There are a lot of technologies around JavaScript that promise to make your code more maintainable. Module loaders like requireJS, browserify and webpack, linters like ESLint and JSHint, type checkers like TypeScript and Flow, and test runners like Mocha and Karma all provide tooling around maintainability.

At the framework Backbone addresses the idea of maintainability by keeping small and flexible enough to remain usable as requirements change. React deals with it by simplifying the programming model enough that its easier for new developers to see whats going on. And Ember deals with it by maintaining a disciplined process for continually improving the framework to “keep up with the web” without leaving developers behind.

Easy To Debug

In recent years a lot of work has gone into improving developer tools for debugging JavaScript. This has happened at 3 levels. First browser vendors, especially Google and Mozilla, have made a huge investment in improving the developer tools that ship with browsers. It’s now possible to identify many errors quickly that would have been mere guesswork before. Secondly, frameworks have worked on building their own developer tools. React and Ember both have incredibly powerful debugging tools with first class support from the framework team. Marionette (a framework for Backbone apps) has a similarly rich debugging tool, and Angular also has tools for debugging. Finally, linters and transpilers are also helping to catch some classes of errors at compile time rather than waiting to discover them at run-time.

Other than tools though, Frameworks can have some features that make them easier or harder to debug. React and Angular both make a big deal of being easy to write tests for. In Angular this is true because of their dependency injection system. In React this works out because React components are basically pure functions that take state and properties and return DOM. Pure functions are easy to test, especially compared to traditional UIs that are tied to the global mutable DOM. Backbone also contributes to easier debugging by presenting a relatively straightforward and readable source that developers can trace through without becoming baffled by “library mystery meat”.

Choices, Choices

When picking a toolset in JavaScript it’s helpful to think about all 6 of the above areas. Any library or framework that has gotten past the minimum threshold of solving a problem will claim to help you in at least one of these areas. But a useful comparison will run across all six. What do you think are the best tools for a truly productive workflow in the JavaScript community today?

Not saying that this is the best way to make things easy to learn, just that it is a goal.
↩

ES5, ES6, ES2016, ES.Next: What's going on with JavaScript versioning?

Sun, 13 Sep 2015 23:11:34 -0400

JavaScript has a strange naming history. For its initial release in 1995 as part of Netscape Navigator, Netscape labeled their new language LiveScript, before renaming it to JavaScript a year later, hoping to capitalize on Java’s popularity at the time (JavaScript has no actual relationship to Java). In 1996 Netscape submitted JavaScript to ECMA International for standardization. This eventually resulted in a new language standard, labeled ECMAScript. All major JavaScript implementations since have actually been implementations of the ECMAScript standard, but the term JavaScript has stuck for historical and marketing reasons ¹. In the real world ECMAScript is usually used to refer to the standard while JavaScript is used when talking about the language in practice.

This has mostly been trivia for JavaScript developers, because ECMAScript didn’t change much for the first 15 years of its existence, and real world implementations often differed significantly from the standard. After the initial version of ECMAScript, work on the language continued and two more versions were quickly published. But after ECMASCript 3 came out in 1999, there were no changes made to the official standard for a decade. Instead various browser vendors made their own custom extensions to the language, and web developers were left to try and support multiple APIs. Even after ECMAScript 5 was published in 2009, it took several years for wide browser support of the new spec, and most developers continued to write code in ECMAScript 3 style, without necessarily being aware of the standard.

Around 2012 things started to change. There was more of a push to stop supporting old Internet Explorer versions, and writing code in ECMAScript 5 (ES5) style became much more feasible. At the same time work was underway on a new ECMAScript standard, at which point it became much more common to start referring to JavaScript implementations in terms of their support for different ECMAScript standards. The new standard was originally named ES.Harmony, before eventually being referred to as ECMAScript 6th Edition (ES6). In 2015 TC39, the committee responsible for drafting the ECMAScript specifications, made the decision to move to a yearly model for defining new standards, where new features would be added as they were approved, rather than drafting complete planned out specs that would only be finalized when all features were ready. As a result ECMAScript 6th edition was renamed ECMAScript 2015 (ES2015) before it was published in June.

Currently there are several proposals for new features or syntax to be added to JavaScript. These include decorators, async-await, and static class properties. These are often refered to as ES7, ES2016, or ES.Next features, but should realistically be called proposals or possibilities, since the ECMAScript 2016 specification hasn’t been written yet, and might include all or none of those features. TC39 divides proposals into 4 stages. You can see the current state of various proposals on the TC39 Github repo.

So where does that leave us in terms of terminology? The following list might be helpful:

ECMAScript: A language standardized by ECMA International and overseen by the TC39 committee. This term is usually used to refer to the standard itself.
JavaScript: The commonly used name for implementations of the ECMAScript standard. This term isn’t tied to a particular version of the ECMAScript standard, and may be used to refer to implementations that implement all or part of any particular ECMASCript edition.
ECMAScript 5 (ES5): The 5th edition of ECMAScript, standardized in 2009. This standard has been implemented fairly completely in all modern browsers
ECMAScript 6 (ES6)/ ECMAScript 2015 (ES2015): The 6th edition of ECMAScript, standardized in 2015. This standard has been partially implemented in most modern browsers. To see the state of implementation by different browsers and tools, check out these compatibility tables.
ECMAScript 2016: The expected 7th edition of ECMAScript. This is scheduled to be released next summer. The details of what the spec will contain have not been finalized yet
ECMAScript Proposals: Proposed features or syntax that are being considered for future versions of the ECMAScript standard. These move through a process of five stages: Strawman, Proposal, Draft, Candidate and Finished.

Going forward in this blog, I’ll be referring to the recent ECMAScript version as ES6 (since that is how it is best known by most developers), next years spec as ES2016 (since that will be what it is called the whole way through its standardization process, unlike ES6/ES2015) and future language ideas that are not yet part of a draft or finalized spec as ECMAScript proposals or JavaScript proposals. I’ll do my best to point back to this post in any cases that might be confusing.

More Resources

If you’re wondering about the best way to keep up with JavaScript language changes, check out the resources in this post
TC39 has a github repo tracking all of their current open proposals.
If you aren’t familiar with ES6 yet, Babel has a great rundown of its features
If you want to go deeper with ES6 I’ve heard great things about 2 books on the subject: Exploring ES6 by Axel Rauschmayer and Understanding ECMAScript 6 by Nicholas Zakas². Axel’s blog 2ality is also a great ES6 resource.

And obviously because ECMAScript is an awful language name
↩
Note that I have not read either of these books yet, though I have read plenty of other content from both authors and consider them experts on the JavaScript language. So take that recommendation with an appropriate grain of salt.
↩

Lessons Backbone Developers Can Learn From React

Wed, 09 Sep 2015 07:19:43 -0400

Since I started programming professionally, I’ve always kept an informal list of technologies I want to check out. Things that I thought would be useful for my career, would expose me to new ideas, or just looked plain cool. I spent a chunk of my Labor Day weekend working through that list a bit by learning more about React, the JavaScript View library from Facebook. React is a fascinating piece of technology, and a strong ecosystem of tools and libraries is growing up around it. For developers starting brand new front-end projects in 2015, it ranks as one of the 2 main libraries I’d suggest looking into as a base, along with Ember. Most developers though are not starting brand new projects. We’re maintaining existing code, or starting a new project while trying to reuse components of an old one. Fortunately, React is about ideas as much as it is technology. For this piece, I’m going to go through the big ideas of React and look at 3 of them that developers working on other frameworks (and Backbone in particular) can learn from. There’s a lot more to React and its community than just these 3 ideas, so I’ll also include some extra resources at the bottom for those interested in learning more about it.

Idea 1: Interfaces should be a tree of composable components

React interfaces are constructed by combining many small “components”, each of which can be created by combining other smaller components. In the end a normal React interface will resemble a tree, with a top level component that encompasses the whole app and many smaller components nested inside of it. It’s a model that should be familiar; the browser DOM works the same way.

Building interfaces this way lets you reuse more code and also makes code easier to reason about. Because you’re composing your interfaces rather than using inheritance or monolithic page objects to build your pages, you can write common code for items like buttons, date-pickers and lists once and then re-use them all over place, even creating larger components like a dialog box out of smaller components like buttons, inputs and an overlay. And since components are small and focused, it’s much easier to dive into them and understand what is going on. Since react components are built to work in this standardized ways, you can be comfortable knowing that no other code is going to be changing the area of the UI controlled by that component.

Backbone doesn’t enforce a specific way to organize your UI code. You can create a single Backbone View for a whole page, have different non-nested views control different portions of a page, or use a nested scheme. For simple sites or apps, each of these can make sense. You probably don’t need a nested component tree for a simple content page with one or two pieces of interactive behavior. But for large rich apps, small composable Views (or other building blocks) can provide much more flexibility, allowing you to mix and match pieces of your application to build diverse pages without becoming overwhelmed with code.

Building this sort of tree system in Backbone is one of the main selling points of Marionette, the most popular of several libraries that add conventions on top of Backbone’s structure. Marionette provides collection Views and layout Views that let you build rich View trees that are very similar in structure to a React app, while maintaining compatibility with Backbone’s code and the Backbone communities conventions. It’s also possible to compose an applications interface with a mix of Backbone and other “components”; for instance using web components, React Components, or even jQuery UI widgets to represent individual pieces in an application and then tying together their layout using Backbone Views.

That’s not to say there are no problems with implementing this style in Backbone though. For one thing, when nesting one Backbone view inside of another, Backbone does not provide strong encapsulation of child views. Because Backbone Views act on HTML directly, when a parent view listens to an event or modifies HTML directly, it is possible for the view to listen to events on elements controlled by a child view and even modify those elements directly. That can create confusing side effects, since in a deeply nested view tree it means an event could lead to code being triggered in one of many different Views, and the state of that piece of DOM could be affected by many different areas of code at once. In Backbone these problems must mostly be solved with programmer discipline, a weight that adds to the difficulty of deeply nested UIs, but doesn’t prevent them.

Idea 2: Modern JavaScript leads to cleaner code

Using modern JavaScript is less a core idea of React, and more a value of its community. When researching React, almost every example of React code I found was written using ES6 style JavaScript code and a modern module system (commonJS or ES6 modules using either webpack or browserify). Many React developers are even pushing the boundaries of todays browsers and standards and experimenting with how their code could be improved by proposed ES7 features. I rarely see Backbone code examples using these styles. They’re usually in ES5 style or Coffeescript and use AMD modules or global namespaces to structure code. Some of this is a natural function of the hype cycle: most Backbone code examples were written 3-4 years ago when it was the hot new JavaScript framework, while most React examples are written now. But the net result is that most people who use Backbone today aren’t being exposed to these new styles and tools. Since things like JavaScript APIs and module loaders can be chosen separate from what framework you choose ¹, this is an opportunity to take advantage of innovation from other places without giving up an investment in Backbone.

Take the following code examples for instance. This is the same View written in 3 styles: ES5 with namespaces, ES6 and ES7. Compare the readability and usability in each case. The example is just a simple view that takes a template, logs a message when it is created and shows a different modal when 2 different buttons are clicked, with a callback function after the modal is closed.

//ES5 with namespaces
(function(App, Backbone, Modal, _) {

    App.ExampleView = Backbone.View.extend({

        template: App.templates['exampleview'],

        events: function() {
            'click .example-button': 'showSuccess',
            'click .example-button2': 'showError',
        },

        constructor: function() {
            Backbone.View.apply(this, [].slice.call(arguments));
            console.log('Created a Example View');
        },

        render: function() {
            this.$el.html(_.template(this.template)(this.model.attributes));
        },

        showSuccess: function() {
            this.showModal('You did it');
        },

        showError: function() {
            this.showModal('You failed', 'Error');
        },

        showModal: function(message, title) {
            if (typeof title === 'undefined') {
                title = 'Alert';
            }
            Modal.show(message, title, this.onModalClose.bind(this));
        },

        onModalClose: function() {
            console.log('re-rendering after modal closes to capture any changes');
            this.render();
        },
    })

})(App || {}, Backbone, Modal, _);

Obviously the scenario is a bit contrived here (we’d want to generalize any logging in the constructor in real life, and we would listen for model changes to re-render rather than just blindly doing it in a callback). But notice how many confusing things are going on that are completely incidental to what the code is doing. A JavaScript beginner would have a lot to work through in this example. Why is the whole file wrapped in a function? Where do Backbone and Modal come from? What’s going on with Backbone.View.apply(this, [].slice.call(arguments));? That’s even aside from the incidental complexity of having to know what order your files are loaded in when using this particular module style. We can do so much better.

//ES6
import {View} from Backbone;
import * as Modal from 'utils/modal';
import {template} from 'lodash';
import * as ExampleViewTemplate from 'templates/exampleview';

export const ExampleView = View.extend({

    template: ExampleViewTemplate,

    events() {
        'click .example-button': 'showSuccess',
        'click .example-button2': 'showError',
    },

    constructor(...args) {
        View.apply(this, args));
        console.log('Created a Example View');
    },

    render() {
        this.$el.html(template(this.template)(this.model.attributes));
    },

    showSuccess() {
        this.showModal('You did it');
    },

    showError() {
        this.showModal('You failed', 'Error');
    },

    showModal(message, title='Alert') {
        Modal.show(message, title, () => this.onModalClose());
    },

    onModalClose() {
        console.log('re-rendering after modal closes to capture any changes');
        this.render();
    },
});

ES6 allows us to clean up our original example a lot! We’re able to ditch the wrapping function, and instead pull our dependencies directly at the top of the file, with clear pointers to module paths so that new developers can easily go find the code we’re referencing. At a smaller level, we’ve cleaned up many of the annoyances from the original code. We no longer have to slice arguments; instead we can use the rest operator to collect all of the arguments as an array and pass them to the constructor directly. Similarly, we don’t need to explicitly check for undefined anymore in showModal since we can display default arguments. Finally, we can get rid of some function boilerplate, removing the function keyword completely for object methods and changing the bind(this) from onModalClose to use an ES6 lambda function. All of this is helpful, and represents the best of what is stable for production at the moment. But if we want to look ahead to the current proposed ES7 additions, we’ll be able to clean this code up even more.

//ES7
import {View} from Backbone;
import * as Modal from 'utils/modal';
import {template} from 'lodash';
import * as ExampleViewTemplate from 'templates/exampleview';

export class ExampleView extends View {

    template = ExampleViewTemplate;

    constructor(...args) {
        super(args);
        console.log('Created a Example View');
    }

    render() {
        this.$el.html(template(this.template)(this.model.attributes));
    }

    @on('click .example-button')
    showSuccess() {
        this.showModal('You did it');
    }

    @on('click .example-button2')
    showError() {
        this.showModal('You failed', 'Error');
    }

    showModal(message, title='Alert') {
        Modal.show(message, title, ::this.onModalClose);
    }

    onModalClose() {
        console.log('re-rendering after modal closes to capture any changes');
        this.render();
    }
}

ES7 allows us to clean up more boilerplate using classes ², static properties, and :: as a special shorthand for function binding. But it also allows us to start actually improving the interface of Backbone itself. The example above uses decorators to define extra behaviors that wrap the View’s methods. In this case decorators allow us to contextualize the events to function mapping for our View, improving on the default events hash that Backbone provides. There are many examples of similar convenient conventions that are being experimented with in the React community. Backbone developers can learn a lot here by looking through React code. Since React components are syntactically similar to Backbone Views (even though the underlying model is quite different), it’s easy to learn a lot of new JavaScript techniques that can be used in Backbone in the process.

Idea 3: Don’t use DOM as a source of state

The final big idea that Backbone developers can learn from React is its insistence on not using the DOM as a source of state. React encourages a programming model where in theory, any component can be re-rendered at any time based on a change in data. That means that any UI state must be captured in code and not in the UI. This is another approach that requires programmer discipline in Backbone. It means that instead of just typing out this.$('.main-content').addClass('highlighted'), you’ll want to set an contentIsHighlighted variable somewhere and then either re-render your view or do a structured update based on your state. That way, instead of having to read the DOM later to know the state of your application it is all copied in code. This leads to better testability, more predictable code and fewer edge cases when the structure of your HTML changes.

This idea was historically the primary benefit for Backbone apps over pure jQuery applications. But the truth is that while Backbone does a good job of pulling application data into code with models and collections, it makes it very tempting to encode UI state info in the DOM by exposing jQuery helpers and not providing a canonical way to store view state. Still, a little discipline goes a long way. By using a separate Backbone Model as a view-model or just storing your state as properties on the view object itself, you can pull your state out of the DOM and make it much easier to inspect and reason about when you’re debugging your code or trying to refactor.

You probably can’t take this idea to the full extent that React takes it using Backbone. The extreme end of the React philosophy is to use React components as stateless functions that simply take application data and ui state as arguments and return an HTML representation of the UI. This means that React interfaces can be re-rendered completely after any change without losing information. React supports this and makes it performant by using a “virtual DOM” to generate the new HTML that would result from a data or state change, compare it to the existing HTML, and then only make the changes that are required in the actual DOM ³. That works well in React since it has a clear concept of what is controlled by each component. As previously noted, Backbone does not strongly encapsulate its Views, which complicates doing the type of virtual DOM analysis that React manages. So it’s more practical to focus on pulling state out of Views and managing re-renders based on Backbone’s event system (the idiomatic Backbone approach). If you want to pursue the UI as pure functional programming paradigm, you’d probably do better moving off Backbone to a system designed for that like React, cycle.js, Om, or rxjs. But even if you can’t go all in on functional UI programming in your current code base, understanding the problems inherent with using DOM to manage state will help you write better Backbone apps.

React Resources Round-up

If you’re interested in learning more about React, many others have put together better resources than I’m capable of. Here’s a quick roundup of some of the resources I’ve found helpful.

Intro

If you want to get a big picture view of what React is about, I’d recommend starting with this 2013 conference video where a member of the React team, addresses some of the criticisms that React received early on and lays out the ideas behind the library. Some of the details have changed since, but the big picture view remains.
The React documentation is also exceptionally well-written and accessible. This blurb on thinking in react is a good place to start.

Talks

If you want to see some of the nice potential side effects that React can provide this talk by Dan Abramov, the creator of Redux, shows how easy it is to create developer tools that significantly improve a developers feedback loop and user experience while writing code.
The keynote from React Europe gives a good feel for the current state of the ecosystem around React.

Articles

Pure UI by Guillermo Rauch is the best explanation I’ve read of the benefits of the UI model that React encourages, though it is not specifically about React.
One of the biggest instinctive objections many developers have when they first see React code is the mixing of JavaScript and the HTML syntax of JSX inside a single file. Eric Elliot has a nice post on medium examining those objections and explaining why JSX makes sense.

Projects

Flux is one of the 2 main data management frameworks that Facebook and the React team recommend for use with React. The second newer one, Relay is a bit more crazy advanced and requires a very specific type of backend API to implement. If you’re interested in Flux, also make sure to check out Redux, an opinionated flux implementation focused pure functional programming concepts.
React Router is a router for React applications based on Ember’s router. It provides a nice declarative model for defining routes based on JSX, the JS language extension that Facebook created along with React
Babel isn’t a react specific technology, but its important to understand it if you want to digest most of the React examples out there, or build a React app yourself, since it is now recommended as the tool for writing React code in an idiomatic style.

Well at this point many frameworks are doubling down on "1st class experiences" with Babel (React/Ember) or Typescript (Angular2). But Backbone at least doesn't care what transpilers you use or don't, and the other libraries are all still usable with vanilla ES5 JavaScript.
↩
Yes I know that classes are an ES6 feature. But without decorators, using them with Backbone is clunky.
↩
In addition to other performance enhancements if an app is structured in a way that it can be determined exactly which data changed.
↩

Backbone and ES6 Classes Revisited

Mon, 06 Jul 2015 07:22:22 -0400

A few weeks ago I wrote an article explaining why the final spec for classes in the ES6/ES2015 version of JavaScript was not usable in Backbone applications. That post got a lot of great feedback, including this discussion with the Backbone core team. The gist of that discussion has been that there are several ways to make ES6 classes work with Backbone now, though they may be less clean than the current Backbone.extend syntax. But there’s hope for a really great experience: decorators, a proposed feature for the ES7/ES2016 spec, provide a much cleaner interface when working with Backbone and native classes.

Let’s start with a quick reminder on the problem that Backbone developers face when adapting ES6 classes. Both Backbone.extend and ES6 classes serve as syntactic sugar around traditional JavaScript inheritance methods. So the following code snippets are effectively equivalent:

let ExampleView = Backbone.View.extend({
    foo: function() {
        alert('foo')
    }
});

class ExampleView extends Backbone.View {
    foo() {
        alert('foo');
    }
}

and each corresponds roughly to this vanilla ES5 snippet

function ExampleView() {
    Backbone.View.call(this);
}

ExampleView.prototype = Object.create(Backbone.View);
ExampleView.prototype.constructor = Backbone.View;
ExampleView.prototype.foo = function() {
    alert('foo');
};

All of that works fine. The problem is that even though these abstractions are similar, they’re not quite the same. Specifically ES6 class syntax supports an opinionated subset of what you can accomplish with plain prototype inheritance. There’s good reason for this, since many developers don’t understand prototype inheritance and can shoot themselves in the foot. But ES6 class restrictions don’t mesh well with Backbone’s conventions. Specifically, Backbone relies on adding properties to a class' prototype before a class' constructor is invoked ¹. ES6 classes don’t provide any way to define properties on the prototype as part of the class definition. They also do not allow you to add instance properties to a constructor prior to calling super() to run a parent class' constructor. As a result, properties that are processed within Backbone objects' constructors like events, className, urlRoot and others can’t be set within an ES6 class definition without resorting to some less than beautiful syntax. There are however at least 3 different ways of dealing with this problem under the ES6 spec. I’ll take a look at each of them.

ES6 solutions

The 3 main solutions I’ve seen proposed for using the current ES6 class spec with Backbone classes are passing properties to the super function, creating everything as a method, and adding properties to the prototype outside of the class definition. I’ll go through those one by one, using a view from one of the Backbone TodoMVC implementations as an example.

Pass properties to the super constructor

The first solution is to define a constructor function for each Backbone object that has properties, and pass the properties directly to the constructor.

import * as $ from 'jquery';
import * as Backbone from 'backbone';
import * as _ from 'underscore';
import * as todosTemplate from 'text!templates/todos.html';
import * as Common from 'common';

class TodoView extends Backbone.View {

    constructor() {
        super({
            tagName:  'li',

            template: _.template(todosTemplate),

            events: {
                'click .toggle':    'toggleCompleted',
                'dblclick label':   'edit',
                'click .destroy':   'clear',
                'keypress .edit':   'updateOnEnter',
                'keydown .edit':    'revertOnEscape',
                'blur .edit':       'close'
            },
        });
    }

    initialize() {
        this.listenTo(this.model, 'change', this.render);
        this.listenTo(this.model, 'destroy', this.remove);
        this.listenTo(this.model, 'visible', this.toggleVisible);
    }

    render() {
        this.$el.html(this.template(this.model.toJSON()));
        this.$el.toggleClass('completed', this.model.get('completed'));

        this.toggleVisible();
        this.$input = this.$('.edit');
        return this;
    }

    toggleVisible() {
        this.$el.toggleClass('hidden',  this.isHidden());
    }

    isHidden() {
        let isCompleted = this.model.get('completed');
        return (// hidden cases only
            (!isCompleted && Common.TodoFilter === 'completed') ||
            (isCompleted && Common.TodoFilter === 'active')
        );
    }

    toggleCompleted() {
        this.model.toggle();
    }

    edit() {
        this.$el.addClass('editing');
        this.$input.focus();
    }

    close() {
        let value = this.$input.val();
        let trimmedValue = value.trim();

        if (trimmedValue) {
            this.model.save({ title: trimmedValue });

            if (value !== trimmedValue) {
                this.model.trigger('change');
            }
        } else {
            this.clear();
        }

        this.$el.removeClass('editing');
    }

    updateOnEnter(e) {
        if (e.keyCode === Common.ENTER_KEY) {
            this.close();
        }
    }

    revertOnEscape(e) {
        if (e.which === Common.ESCAPE_KEY) {
            this.$el.removeClass('editing');
            // Also reset the hidden input back to the original value.
            this.$input.val(this.model.get('title'));
        }
    }

    clear() {
        this.model.destroy();
    }
};

export default TodoView;

There are a few advantages to this approach. It keeps everything within the class definition, and lets Backbone itself manage attaching properties to the prototype, assuring compatibility with existing code. However it adds a constructor call to a class that otherwise wouldn’t need one, making it less obvious when a developer is actually modifying the constructor to do something important. It also isolates properties from methods, whereas Backbone philosophically has always treated them interchangeably. In fact that philosophy is the key to the next solution.

Treat everything like a method

Backbone evaluates all of its properties using Underscore’s _.result function ². _.result checks to see if an object property is a function. If it is then _.result will evaluate it and return the result. This allows Backbone to accept its properties as either an object or a function. Very handy. So one workaround to the constructor problem is to make everything a method, including properties. We can actually go one better than this, and use the get keyword to make our methods serve as getters for a property. This means they will be accessible as properties, but defined as methods, retaining compatibility with any existing references if you’re converting existing code.

class TodoView extends Backbone.View {

    get tagName() { return 'li'}

    get template() { return _.template(todosTemplate)}

    get events() {
        return {
            'click .toggle':    'toggleCompleted',
            'dblclick label':   'edit',
            'click .destroy':   'clear',
            'keypress .edit':   'updateOnEnter',
            'keydown .edit':    'revertOnEscape',
            'blur .edit':       'close'
        }
    }

    initialize() {
        this.listenTo(this.model, 'change', this.render);
        this.listenTo(this.model, 'destroy', this.remove);
        this.listenTo(this.model, 'visible', this.toggleVisible);
    }

    render() {
        this.$el.html(this.template(this.model.toJSON()));
        this.$el.toggleClass('completed', this.model.get('completed'));

        this.toggleVisible();
        this.$input = this.$('.edit');
        return this;
    }

    toggleVisible() {
        this.$el.toggleClass('hidden',  this.isHidden());
    }

    //... etc
};

export default TodoView;

This approach uses only standard class syntax, which is great. Getters and methods are highly idiomatic class structures, and work fine with Backbone right now. They also enforce the immutability of these properties, a design concern with the current Backbone property implementation ³. But lets be real, get tagName() { return 'li'} is a significantly less clean interface than the tagName: 'li' syntax you get with Backbone.extend.

Defining properties directly on the prototype

The final example is the most straightforward. Since there’s no special class syntax for adding properties to the prototype, we can just do so directly after we’ve defined the class.

class TodoView extends Backbone.View {

    initialize() {
        this.listenTo(this.model, 'change', this.render);
        this.listenTo(this.model, 'destroy', this.remove);
        this.listenTo(this.model, 'visible', this.toggleVisible);
    }

    render() {
        this.$el.html(this.template(this.model.toJSON()));
        this.$el.toggleClass('completed', this.model.get('completed'));

        this.toggleVisible();
        this.$input = this.$('.edit');
        return this;
    }

    toggleVisible() {
        this.$el.toggleClass('hidden',  this.isHidden());
    }

    //... etc
};

TodoView.prototype.tagName = 'li';
TodoView.prototype.template = _.template(todosTemplate);
TodoView.prototype.events = {
    'click .toggle':    'toggleCompleted',
    'dblclick label':   'edit',
    'click .destroy':   'clear',
    'keypress .edit':   'updateOnEnter',
    'keydown .edit':    'revertOnEscape',
    'blur .edit':       'close'
};

export default TodoView;

This approach is unambiguous, but its also removes the advantages of using abstracted syntax to begin with. Adding directly to the prototype takes properties out of the context of the class definition, making it harder to view the class as a holistic unit. Forcing properties to appear after the class definition also breaks with Backbone community conventions. Almost all examples of Backbone code that I’ve seen put properties at the top of Backbone class declarations, above any methods, since they’re usually important to read when trying to understand the purpose and role of an object. They’re then followed by methods, which often contain extensive logic that is less helpful in understanding a class' overall purpose.

Of these 3 solutions, I personally prefer using methods and getters. However I don’t see any of them as an improvement on the existing Backbone.extend syntax.

ES7 and Decorators

JavaScript is not staying stagnant however. The TC39 committee recently approved the final spec for ES6 and is aiming to release a new spec on a yearly basis going forward. One proposed feature for a future spec is JavaScript decorators. Decorators are a declarative way of adding extra behavior to a class or class method. They’re simple functions that take a class or method as an argument and then return an augmented version.

For instance here’s a simple example from the proposal of a decorator that adds an isTestable property to classes that it decorates.

@isTestable(true)
class MyClass { }

function isTestable(value) {
   return function decorator(target) {
      target.isTestable = value;
   }
}

Based on that example, it’s not hard to see how this is relevant to the syntax struggle we experienced above. Here’s a naive solution to our properties problem that utilizes a props decorator to add a Backbone-esque hash of properties to the prototype.

//decorators.js
export function props(value) {
    return function decorator(target) {
        _.extend(target.prototype, value);
    }
}

import * as $ from 'jquery';
import * as Backbone from 'backbone';
import * as _ from 'underscore';
import * as todosTemplate from 'text!templates/todos.html';
import * as Common from 'common';
import {props} from 'decorators'

@props({

    tagName:  'li',

    template: _.template(todosTemplate),

    events: {
        'click .toggle':    'toggleCompleted',
        'dblclick label':   'edit',
        'click .destroy':   'clear',
        'keypress .edit':   'updateOnEnter',
        'keydown .edit':    'revertOnEscape',
        'blur .edit':       'close'
    },
})
class TodoView extends Backbone.View {

    initialize() {
        this.listenTo(this.model, 'change', this.render);
        this.listenTo(this.model, 'destroy', this.remove);
        this.listenTo(this.model, 'visible', this.toggleVisible);
    }

    render() {
        this.$el.html(this.template(this.model.toJSON()));
        this.$el.toggleClass('completed', this.model.get('completed'));

        this.toggleVisible();
        this.$input = this.$('.edit');
        return this;
    }

    toggleVisible() {
        this.$el.toggleClass('hidden',  this.isHidden());
    }

    //... etc
};

export default TodoView;

That’s a big improvement over our ES6 solutions above. Our properties are added declaratively at the top of the class and are still clearly associated with the class. Everything lines up reasonably well with both Backbone and native idioms. This is also reminiscent of the approach that Angular 2 is taking with its interface, using decorators heavily for component configuration. It turns out though that we can actually do better.

I got a tweet from Steven Langbroek last week asking me to look at his idea for using decorators to provide a cool new syntax for event binding in Backbone. I thought it was awesome and it inspired me to think in a more fine grained way about using decorators with Backbone. How can we make the syntax better, as opposed to just replicating the current experience. Here’s what a more “fine grained” decorators approach might look like with our example.

//decorators.js
export function tagName(value) {
    return function decorator(target) {
        target.prototype.tagName = value;
    }
}

export function template(value) {
    return function decorator(target) {
        target.prototype.template = _.template(value);
    }
}

export function on(eventName){
  return function(target, name, descriptor){
    if(!target.events) {
        target.events = {};
    }
    if(_.isFunction(target.events)) {
        throw new Error('The on decorator is not compatible with an events method');
        return;
    }
    if(!eventName) {
        throw new Error('The on decorator requires an eventName argument');
    }
    target.events[eventName] = name;
    return descriptor;
  }
}

import * as $ from 'jquery';
import * as Backbone from 'backbone';
import * as _ from 'underscore';
import * as todosTemplate from 'text!templates/todos.html';
import * as Common from 'common';
import {tagName, template, on} from 'decorators';

@tagName('li')
@template(todosTemplate)
class TodoView extends Backbone.View {

    initialize() {
        this.listenTo(this.model, 'change', this.render);
        this.listenTo(this.model, 'destroy', this.remove);
        this.listenTo(this.model, 'visible', this.toggleVisible);
    }

    render() {
        this.$el.html(this.template(this.model.toJSON()));
        this.$el.toggleClass('completed', this.model.get('completed'));

        this.toggleVisible();
        this.$input = this.$('.edit');
        return this;
    }

    toggleVisible() {
        this.$el.toggleClass('hidden',  this.isHidden());
    }

    isHidden() {
        let isCompleted = this.model.get('completed');
        return (// hidden cases only
            (!isCompleted && Common.TodoFilter === 'completed') ||
            (isCompleted && Common.TodoFilter === 'active')
        );
    }

    @on('click .toggle')
    toggleCompleted() {
        this.model.toggle();
    }

    @on('dblclick label')
    edit() {
        this.$el.addClass('editing');
        this.$input.focus();
    }

    @on('blur .edit')
    close() {
        let value = this.$input.val();
        let trimmedValue = value.trim();

        if (trimmedValue) {
            this.model.save({ title: trimmedValue });

            if (value !== trimmedValue) {
                this.model.trigger('change');
            }
        } else {
            this.clear();
        }

        this.$el.removeClass('editing');
    }

    @on('keypress .edit')
    updateOnEnter(e) {
        if (e.keyCode === Common.ENTER_KEY) {
            this.close();
        }
    }

    @on('keydown .edit')
    revertOnEscape(e) {
        if (e.which === Common.ESCAPE_KEY) {
            this.$el.removeClass('editing');
            // Also reset the hidden input back to the original value.
            this.$input.val(this.model.get('title'));
        }
    }

    @on('click .destroy')
    clear() {
        this.model.destroy();
    }
};

export default TodoView;

Now we’re getting somewhere. Steven’s on decorator provides a really nice upgrade to Backbone’s event system. Event handlers are now in the context of the methods they refer to, making it much easier to get context as you scroll through a large view. Our tagName gets a nice declarative setter at the top of the view rather than being bundled into an object hash. And our template processing gets standardized, with the call to _.template abstracted away into the decorator. That last decorator would probably be project specific, but the other 2 could easily be incorporated into a general Backbone decorator library.

The main downside I see to this is one shared by a few of the other approaches. Specifically that declaring a property like tagName as a method or as a property requires different implementation styles. Backbone philosophically treats these scenarios as equivalent, and its not unusual to start with a static property for keys like className, url, or template and then refactor to use a function as code is reused in different situations. Artificially separating out those scenarios is not very Backbone-like and might discourage that type of refactoring.

Recommendations

If you want to use ES6 classes with Backbone in production today, you should probably make everything a method, and make liberal use of the getter functionality. That seems to be the most Backbone-like, with everything contained in a single class definition, and easy refactoring when a property begins to require logic around it. If you’re experimenting though, you should definitely consider playing with decorators and see how you can make Backbone’s interface better.

If you have ideas about this, feel free to contribute on the github issue

I completely screwed up explaining this in my first piece. I was under the mistaken impression that Backbone was adding properties like `className` and `events` to each instance rather than to the prototype. Which makes no sense in retrospect, since those properties need to be present before the Backbone.View is run. ↩
Well, almost all. iDAttribute is still catching up. ↩
Oh you didn't realize that adding an item to the event hash for your view would add it to every instance of that View? Whoops. ↩

Book Review: Talking with Tech Leads

Mon, 15 Jun 2015 07:30:44 -0400

Can we be honest for a moment? Figuring out a career path is hard for software developers. Talking to some people, you’ll get the impression that we face a pretty bleak binary choice. Behind door number #1 lies a relatively rapid path to a “senior developer” role, followed by stagnation the rest of your career as you reimplement the same solutions in new technologies for minimally increasing pay over time, until youth-friendly tech culture decides that you’re too old to be useful. Door #2 serves basically as an escape hatch, as you trade the opportunity to work with code for a path into management, where you receive a few more pay raises in return for your soul.

I’m not that cynical, and hopefully not that dramatic, but as a relatively young developer it’s been hard to figure out what I want from a career and harder still to know exactly what different options would look like if I got there. I know that I love technology; at the same time I want to have real influence on the things that I’m building, and in most places it’s the product and people challenges that ultimately determine whether software projects succeed or fail, rather than the pure technical ones. So what does it look like to build a career leading across these different areas? That’s the question that Talking with Tech Leads tries to answer.

Talking with Tech Leads is an eBook about what it’s like to pursue a middle path between the two sides of the false dichotomy above. It’s structured as a series of interviews with “Tech Leads”, which the author defines as a developer leading a development team. Each interview shares a series of questions, as well as an open space for interviewees to share more freeform thoughts and advice.

What should a Tech Lead focus on and why?

What has been your biggest challenge as a Tech Lead?

Any time-management tips?

How do you strike the right balance between writing code and dealing with other issues?

As you would expect from a series of nearly forty interviews with the same structure, there is plenty of repetition across answers. Themes emerge quickly: a need to look beyond code for solutions without losing sight of your technical skills, the importance of enabling your team members to succeed, and the need to bridge the communication gap between technical and non-technical participants in a business. At the same time it’s clear that many of these individuals view their roles drastically differently despite superficially similar job descriptions. Some focus on leading people, some on software architecture, and some on aligning business and technical objectives. What the interviews expose are a wide and varied set of niches that fall between the traditional “Senior Developer” and “Management” career paths.

I have no idea how this book would read to a seasoned tech lead. I expect that it would be affirming to see how others have approached similar challenges, but I’m not sure it does a good enough job pulling the various threads from the interview into practical insights to provide much tangible big picture advice for current tech leads. Instead it would likely be a collection of small useful ideas and nuggets.

For a developer like myself who is still figuring out what it looks like to have a career in this industry though, this book is a tapestry of ideas and possibilities. I’d recommend it to anyone who’s been underwhelmed by the extreme career paths that serve as the “path of least resistance” in some organizations, and want to become leaders without sacrificing their roots.

You can purchase Talking with Tech Leads as an e-book through Leanpub or as a paperback on Amazon

Mozilla: The state of Web Components

Sun, 14 Jun 2015 17:35:58 -0400

Over on Mozilla’s development blog, Wilson Page has a great summary of the current status of the Web Components spec:

Web Components have been on developers’ radars for quite some time now. They were first introduced by Alex Russell at Fronteers Conference 2011. The concept shook the community up and became the topic of many future talks and discussions.

In 2013 a Web Components-based framework called Polymer was released by Google to kick the tires of these new APIs, get community feedback and add some sugar and opinion.

By now, 4 years on, Web Components should be everywhere, but in reality Chrome is the only browser with ‘some version’ of Web Components. Even with polyfills it’s clear Web Components won’t be fully embraced by the community until the majority of browsers are on-board.

I’ve been using Custom Elements (the least controversial part of the Web Components spec) in production for a little less than a year now thanks to a small polyfill script. I know that many others are using a version of them through Polymer. It’s been sad to see the spec stagnate without browser support for much longer than might have been expected, but it seems there’s reason for hope.

Web Components have been in planning for over three years, but we’re optimistic the end is near. All major vendors are on board, enthusiastic, and investing significant time to help resolve the remaining issues.

It’s exciting to see public cooperation on this issue, something that was not very visible over the past few years as Chrome fired ahead with an implementation while others pursued other priorities. If you care about this spec at all, make sure to read the full article. It’s an interesting look at where the spec may be going as well as a nice peak behind the scenes of how web APIs become established.

Is Safari being left behind?

Wed, 10 Jun 2015 00:03:41 -0400

On Twitter yesterday, Alex Sexton surfaced Apple’s fact sheet on what’s new with Safari 9:

https://t.co/fVg9U9dEq4 Some ES6 and lots of unprefixed CSS (flexbox!), but I really want Intl, ServiceWorkers, PointerEvents, HTTP2, CSP2…
— Alex Sexton (@SlexAxton) June 8, 2015

Safari 9 is still in developer preview and won’t ship till September or so, so this all could change, but Apple’s changelog is underwhelming to say the least. The developer facing changes in Safari 9 consist of only standardized versions of CSS properties (that were already supported with a prefix), and a handful of ES6 features.

The following ECMAScript 6 content is now supported by Safari:

Classes

Computed Properties

Weak Set

Number Object

Octal and Binary Literals

Symbol Objects

Template Literals

- Safari Pre-Release notes

While it’s great to see ES6 support, the truth is that by the time Safari is released in the fall, Microsoft Edge will have been released, and Safari 9 will release as (at best) the 3rd most ES6-compliant browser among the 4 major desktop browsers. And due to its yearly schedule, it’s basically guaranteed to be in last by the end of the year. At the same time Safari has neglected other APIs like Internationalization, Pointer Events, Web Components, CSS Variables, Service Workers, or ASM.js optimizations, all of which are supported or in progress by each of the other browsers ¹. In addition, Safari’s developer tools, which were once best in class, have been slowly falling behind Chrome and Firefox, especially in more advanced performance features.

So does this mean that Safari is the next IE6? Not so much. We’ve come a long way since the IE6 era, and browsers are more standards compliant than ever. Tools like Babel make it easier for developers to participate in the future of the web without being restricted by slow moving browsers or legacy support concerns. But there’s always going to be some browser that serves as the “bottleneck” for what web developers are able to reasonably use in production. Currently that is older versions of Internet Explorer and old Android browsers. But as more and more users upgrade to Android phones using Chrome, and Microsoft makes strong pushes to move users to the evergreen Edge browser, it’s quite likely that Safari could take their place. It is now the only major browser being updated yearly, and with Microsoft’s recent re-emphasis on JavaScript and HTML development, Apple is the only major browser vendor who seems to view the web as a second class platform.

Safari is not going away. Its monopoly status on iOS more or less guarantees that, and on the desktop I’ve talked to plenty of people who like its simple UI, and others who feel that it’s faster or consumes less memory than Chrome and Firefox. But it will be a sad sight if it gets left behind. Here’s hoping that the final release contains more surprises, and that going forward Apple chooses to embrace the web browser as a platform worthy of the respect and effort its competitors have lavished on it.

Ok, CSS Variables, Service Workers and Web Components are "under consideration" by IE. ↩

How jQuery Works - An Introduction

Mon, 08 Jun 2015 07:51:25 -0400

Every developer has had the experience of using “magic” code. Not the “hacked into the NSA in 20 seconds” type of magic code you see in movies. This is the code created when you nervously typed an incomprehensible incantation that somebody else passed on to you, desperately hoping to receive the results you want, ignoring the sinking feeling in your stomach that you have no clue what to do if it fails. Almost all code feels like this when you’re pushing out to learn new things. But it doesn’t have to stay that way.

The primary goal of this blog has always been to help myself understand the tools I’m using, and to share that understanding with as many people as possible. My experience is that really understanding a library or framework requires diving into the source code and swimming around a bit. Which sounds great. But often the real world gets in the way. Real world source code is full of “special code”. Code that handles browser edge cases, performance optimizations that obscure meaning, indirection to allow shared solutions for common problems; all things that are completely necessary, and also likely to obscure the true structure of code. So while reading the source is the best way to understand a library, it often helps to have some guideposts to lead you through.

This post is the start of a new series diving into how jQuery (certainly a very magical library for many developers) works. I’ll pull snippets from the source code, but also try to lay out the big picture approaches the library takes to provide functionality that combines to form by far the best-known and most used JavaScript library out there ¹. For this introduction, the focus will be on the overall structure of the jQuery repository, how the code is laid out for development purposes, how it’s built into a single file for production use, and a quick look at the core jQuery object code. Future posts will cover how different features of the libary work, including deep dives into selectors, ajax, animation, and events.

So where to start?

When you look at the jQuery Github repo, one of the first things you’ll probably notice is that the source isn’t developed as a single file the way it is distributed. Instead it’s split up into many files separated by functionality. That makes it easier to find specific functions when you know what you’re looking for, but harder to know where to start when trying to understand the big picture. If you’re new to the library, it could be overwhelming.

Fortunately it’s not that hard to get the lay of the land. jQuery is broken into modules using the AMD format, and built using requireJS. So each file lists its dependencies, and it’s easy enough to trace back what code is referenced where. So we can start by looking at src/jquery.js, which serves simply as a base file to require all of the other dependencies, then make the jQuery namespace available globally, both as jQuery and the famous $.

From that file we can see the layout of the library. The core file is loaded and attached to the window, and all of the other “feature files” are also listed as dependencies to make sure they load. Each of those files lists core.js as a dependency, so it is loaded first and creates the main jQuery namespace, which other dependencies then edit as needed. If you’re interested in the process of how the package is built, you can take a read through build/tasks/build.js, where the dependencies are built using requirejs ².

A look at core.js

We can see from jquery.js that core.js is an important file. It exports the main jQuery namespace, and if we look around we’ll see that it is required by about half of the files in the source directory as a dependency. So what does it do? core.js is responsible for defining the jQuery namespace, as well as the prototype for jQuery objects. This is a great place to start before taking a more detailed look into the library’s features, so lets take a look.

define([
	"./var/arr",
	"./var/document",
	"./var/slice",
	"./var/concat",
	"./var/push",
	"./var/indexOf",
	"./var/class2type",
	"./var/toString",
	"./var/hasOwn",
	"./var/support"
], function( arr, document, slice, concat, push, indexOf, class2type, toString, hasOwn, support ) {

var
	version = "@VERSION",

	// Define a local copy of jQuery
	jQuery = function( selector, context ) {
		// The jQuery object is actually just the init constructor 'enhanced'
		// Need init if jQuery is called (just allow error to be thrown if not included)
		return new jQuery.fn.init( selector, context );
	},

There’s a lot going on in the first 20 or so lines. First we see a pattern that will repeat in future files. jQuery uses a lot of small utility functions throughout its code. Rather than attaching those to a single object, each one gets its own file in the var directory, and then is imported as a dependency as needed. These modules are typically quite small and often trivial. For instance the document module that is pulled in from var/document consists in its entirety of the following code:

define(function() {
	return window.document;
});

This is obviously a small shortcut, but it allows for consistent use of a pattern across the library. You’ll see small utility modules like this loaded in every significant file throughout the source code.

After the imports, this file sets the version (the @VERSION annotation is replaced with the version number as part of the build process), and then creates the base jQuery function. That base function proxies out to the init function if it is available, but more importantly serves as a namespace for the rest of the library. All future functionality gets extended off of this object or its prototype. After setting up a few constants, we dive right into that:

jQuery.fn = jQuery.prototype = {
	// The current version of jQuery being used
	jquery: version,

	constructor: jQuery,

	// The default length of a jQuery object is 0
	length: 0,

	toArray: function() {
		return slice.call( this );
	},

	// Get the Nth element in the matched element set OR
	// Get the whole matched element set as a clean array
	get: function( num ) {
		return num != null ?

			// Return just the one element from the set
			( num < 0 ? this[ num + this.length ] : this[ num ] ) :

			// Return all the elements in a clean array
			slice.call( this );
	},

	// Take an array of elements and push it onto the stack
	// (returning the new matched element set)
	pushStack: function( elems ) {

		// Build a new jQuery matched element set
		var ret = jQuery.merge( this.constructor(), elems );

		// Add the old object onto the stack (as a reference)
		ret.prevObject = this;

		// Return the newly-formed element set
		return ret;
	},

	// Execute a callback for every element in the matched set.
	each: function( callback ) {
		return jQuery.each( this, callback );
	},

	map: function( callback ) {
		return this.pushStack( jQuery.map(this, function( elem, i ) {
			return callback.call( elem, i, elem );
		}));
	},

	slice: function() {
		return this.pushStack( slice.apply( this, arguments ) );
	},

	first: function() {
		return this.eq( 0 );
	},

	last: function() {
		return this.eq( -1 );
	},

	eq: function( i ) {
		var len = this.length,
			j = +i + ( i < 0 ? len : 0 );
		return this.pushStack( j >= 0 && j < len ? [ this[j] ] : [] );
	},

	end: function() {
		return this.prevObject || this.constructor(null);
	},

	// For internal use only.
	// Behaves like an Array's method, not like a jQuery method.
	push: push,
	sort: arr.sort,
	splice: arr.splice
};

Here the code sets up the jQuery prototype, which is also made available as jQuery.fn. jQuery objects, what you get when you select a set of elements with jQuery, have access to all of the methods defined on this prototype. jQuery.fn is also the extension point for jQuery plugins, so that custom functions are available to jQuery objects. Most of the functions built here are pretty basic ones that mimic built in array functions for jQuery objects.

A note about jQuery objects

I’ve glossed over jQuery objects twice now, but we’ve seen enough already to get a deeper understanding of what they are. jQuery objects are the objects created when the jQuery init function is used as a constructor to create a new object ³. That sounds fancy, but in reality that’s what is happening every time you make a simple jQuery call like $('#foo') or $('div'). The jQuery namespace function is being called and returns a new object with jQuery.init as the constructor. Since init shares a prototype with the namespace function ⁴, these objects all have access to jQuery.prototype. The second post in this series will talk more about how the init function parses the various inputs that you can pass to the namespace function, but for now it’s best to focus on understanding that the namespace function and jQuery.fn are not magic, but are just a wrapper and alias for portions of a fairly normal prototypical inheritance example in JavaScript.

Extend

After the base functions get added to the prototype, a special function is created and added to both the prototype and the namespace function directly.


jQuery.extend = jQuery.fn.extend = function() {
	var options, name, src, copy, copyIsArray, clone,
		target = arguments[0] || {},
		i = 1,
		length = arguments.length,
		deep = false;

	// Handle a deep copy situation
	if ( typeof target === "boolean" ) {
		deep = target;

		// Skip the boolean and the target
		target = arguments[ i ] || {};
		i++;
	}

	// Handle case when target is a string or something (possible in deep copy)
	if ( typeof target !== "object" && !jQuery.isFunction(target) ) {
		target = {};
	}

	// Extend jQuery itself if only one argument is passed
	if ( i === length ) {
		target = this;
		i--;
	}

	for ( ; i < length; i++ ) {
		// Only deal with non-null/undefined values
		if ( (options = arguments[ i ]) != null ) {
			// Extend the base object
			for ( name in options ) {
				src = target[ name ];
				copy = options[ name ];

				// Prevent never-ending loop
				if ( target === copy ) {
					continue;
				}

				// Recurse if we're merging plain objects or arrays
				if ( deep && copy && ( jQuery.isPlainObject(copy) ||
					(copyIsArray = jQuery.isArray(copy)) ) ) {

					if ( copyIsArray ) {
						copyIsArray = false;
						clone = src && jQuery.isArray(src) ? src : [];

					} else {
						clone = src && jQuery.isPlainObject(src) ? src : {};
					}

					// Never move original objects, clone them
					target[ name ] = jQuery.extend( deep, clone, copy );

				// Don't bring in undefined values
				} else if ( copy !== undefined ) {
					target[ name ] = copy;
				}
			}
		}
	}

	// Return the modified object
	return target;
};

$.extend is jQuery’s way of merging properties from one object into another. It’s also a very handy shortcut for cloning an object (var clone = $.extend({},obj)). We’ll see an example usage shortly, but since it’s a common function, it’s worth digging into how it works for a second.

Looking at the code above, we can see that extend takes a variable number of arguments, the first of which can optionally be a boolean telling the function to “deep copy”. If only a single object is passed, the function acts as if the jQuery prototype was passed as the first object, and targets that, otherwise it targets the first non-boolean argument.

extend then loops over each argument after the target and for each one goes over each of its keys. For each object in order, properties are copied from the arguments to the target object. If deep copy is set, properties that are objects are merged by calling extend recursively. Otherwise, existing properties are overwritten. This continues left to right till all arguments have been merged.

The source code helpfully gives us an immediate example of how extend can be useful. jQuery makes several properties and helpful functions available directly off of the $ namespace. Several of them are added here using extend. You can see how the object notation flows much better than a long series of assignments with = would.

jQuery.extend({
	// Unique for each copy of jQuery on the page
	expando: "jQuery" + ( version + Math.random() ).replace( /\D/g, "" ),

	// Assume jQuery is ready without the ready module
	isReady: true,

	error: function( msg ) {
		throw new Error( msg );
	},

	noop: function() {},

	isFunction: function( obj ) {
		return jQuery.type(obj) === "function";
	},

	isArray: Array.isArray,

	isWindow: function( obj ) {
		return obj != null && obj === obj.window;
	},

	//... many more functions and properties here
});

After working through 2 more helper functions, the namespace object gets returned, wrapping up core.js. So 2 files and ~500 lines of source code in, we already have a pretty good idea of how jQuery is structured, both in terms of using AMD modules to separate out the source, and using a namespace function and its prototype as a central object to attach functionality.

More Resources

It’s no longer maintained with newer versions, but a few years ago Rob Flaherty created an annotated version of jQuery 1.6’s source. You can see that here
Along the same lines: John Resig, jQuery’s creator, recently went through his original jQuery library (a early ~1.0 version) and annotated it with both code commentary and historical notes. It’s a great look inside the history of the library.

You can see that on Google Trends, Bower stats, or Github stars
↩
jQuery's build process is actually super intersting, as they allow custom builds that optionally exclude portions of the library. If you get a chance make sure to look through build.js and note the function to remove requireJS boilerplate after the modules have been built.
↩
If you're unclear on how constructors and prototypes work in JavaScript, feel free to catch up with this post I wrote on Object Oriented Programming in JavaScript a few years back.
↩
Ok so we didn't have quite enough information to fully understand this yet, but you can see where the init function gets its prototye here.
↩

Moving Past RequireJS

Thu, 28 May 2015 10:10:00 -0400

RequireJS has been a hugely influential and important tool in the JavaScript world. It’s still used in many solid, well-written projects today. But as it has stagnated and competitors have grown up, it now suffers compared to the alternatives. It’s time for new projects to look past RequireJS when deciding how to structure their front-end projects.

What is RequireJS?

RequireJS is the most well known module loader for JavaScript. Module loaders are tools for specifying dependencies for JavaScript files and loading those files into a browser. They became necessary because JavaScript has not traditionally offered a built in way to specify what other code is needed to execute a file. Developers coming from server side languages like Java and Python are used to being able to import anything they need from another file, letting the language handle the details. Since that was not designed into JavaScript, the only alternative originally was having developers take care that all scripts were correctly loaded into a page together in the right order, then making references across files without the assurance that the other file existed.

Module loaders improve on that by allowing you to define dependencies for your JavaScript files, and assuring that those dependencies are loaded in the correct order so that the variables the code needs are there when referenced. These dependencies are specified using one of several “module formats”. There are currently 3 primary competing standards for JavaScript modules: AMD or Asynchronous Module Definition, CommonJS, and EcmaScript 6 Modules. I’ll discuss these more below, but each of them provide syntax for listing the files that a module requires as a dependency, and the object that the module wishes to import from those files.

Why not start a project with RequireJS?

RequireJS was the first JavaScript module loader to gain widespread popularity. It’s used in hundreds of successful production sites, and is widely referenced in JavaScript books, blog posts, and talks. I personally led a project to convert a large web application to use it for dependency management in my previous job, and still use it now in a current project. So why would I say not to use it for your project? There are two sets of issues with RequireJS: issues with the AMD module format, and increased competition in terms of the features expected from a module loader.

The problems with AMD

RequireJS requires developers to use AMD modules. That choice made a ton of sense in 2011 when RequireJS first launched. It makes less sense today. In 2011, the only popular alternative to AMD modules was the CommonJS standard, and AMD held a technical advantage over CommonJS. It was essentially a decision between a technological superiority and a cleaner API. In 2015, AMD is one of 3 realistic module syntax alternatives, and no longer holds a significant technical advantage, but it still has a less clear syntax and has begun to experience problems with network effects.

Syntax Comparison

AMD modules look like this:

define(['file1','file2'], function(Class1, Class2) {
    let obj = new Class1(),
        obj2 = new Class2();
    return obj.foo(obj2);
});

A define function is used to define a new module. It takes 2 arguments: an array of dependencies (specified as strings), and a callback function that is passed in the values exported by each dependency and run after all of the dependencies have been loaded. AMD also provides a require function that takes the same arguments, but serves as the initialization point of a program, loading its dependencies and executing them when it’s run.

Compare that to CommonJS and the ES6 Modules syntax:

//commonJS
let Class1 = require('file1'),
    Class2 = require('file2'),
    obj = newClass1(),
    obj2 = new Class2();

module.exports = obj.foo(obj2);

//ES6
import Class1 from 'file1';
import Class2 from 'file2';

let obj = newClass1(),
    obj2 = new Class2();

export default obj.foo(obj2);

There’s some obvious niceties there. CommonJS and ES6 modules don’t require you to wrap your code in an outer function, which serves to reduce boilerplate and also aligns the import code more closely with the expectations of developers who have used imports in server-side languages.

But there are also other subtler problems with the AMD syntax. Because you’re asked to specify your dependencies as an array of strings that then result in parameters to a callback function, adding or removing a dependency requires code to be changed in 2 places. ¹ This is no big deal in a simple example like above. But what about in this real world example:

define(['marionette',
        'radio',
        'underscore',
        'lib/analytics',
        'collections/widget',
        'collections/user',
        'models/user',
         'routers/widget_page_router',
        'views/widget_page/page'],

function(Mn, Radio, _, Analytics, WidgetCollection, UsersCollection,
    User, WidgetRouter, WidgetPage) {
        //module code here
});

If you were asked to remove a dependency here, it’s easy to imagine accidentally putting a parameter in the wrong position in the list, or removing an item from one list without the other, causing the parameters to be set to the wrong values. It’s a small issue, but the type of subtle source of bugs that can drive developers crazy.

Finally the mixed use of require and define for pulling in dependencies seems to be confusing to many developers. While it certainly isn’t a large learning curve (require is for initiating new code, define is for defining a module that can be pulled into a dependency tree started by a require), it’s an additional point of confusion that doesn’t exist in CommonJS, which simply has a “starting script” that is run and pulls in its dependencies.

Update: Commenters on Reddit have correctly pointed out that there’s a second usable AMD syntax that cleans up some of these issues. I didn’t include it originally since it’s not the “official” AMD syntax and I have not seen it widely used, but it is better and worth noting for users sticking with RequireJS/AMD

Technical Advantages

When RequireJS was first created, AMD had a few technical advantages over CommonJS solutions. AMD was set up to be asynchronous. The callback function syntax made it easy to handle dependency loading over a network. You made the define call, loaded the dependencies, and once all of the dependencies had been pulled over the network, ran the callback function. This also allowed some dependencies to be lazy-loaded after the initial page load, since that could be handled like any other ajax request with a callback attached. Since CommonJS was set up in a more synchronous fashion, it was less clear how to accomplish this, and early systems were less popular than requireJS.

AMD’s asynchronous nature remains a theoretical advantage, but in practice, modern module loaders have found ways to solve these problems for CommonJS. Webpack allows configuration options for lazy loading individual modules, and developers using Browserify have found workarounds to the problem. At this point there’s no real difference in the capabilities of AMD compared to other module formats.

Network Effects

While RequireJS and AMD were initially the most popular module system for the browser, CommonJS is built into Node.js and has always been the dominant module format on the server-side. This was initially no big deal, but Node has grown in popularity and also become more important to client-side developers due to the tooling ecosystem around it. Because of that, code that doesn’t play well with Node is limited in ways that CommonJS is not. One area that I’ve noticed this is unit testing. There’s a whole ecosystem of unit-testing libraries that allow you to test your modules on the command line using Node (Mocha, jasmine-node, Jest). This has nice advantages relative to running your tests in a browser. It’s easy to integrate into a build workflow, the test framework can run all tests in a directory without developers needing to manually add them to a test page, and there’s no extra browser window required. With AMD you can get part of the way there by making all of your tests asynchronous (painful) and importing requireJS or by using PhantomJS to run your browser tests on the command-line. But you can’t match the experience of full node compatibility.

In addition to the unit test example, there are many small node modules out there that can be usefully pulled into a web project. Libraries like Ampersand.js and Event-Emitter are quite useful in a browser context, but only play nicely if you’re using CommonJS. While it’s true that most libraries do support AMD, they tend to do so through a “Universal” module format that checks the environment to see what module format is being used and then uses appropriate format. I’m unfamiliar with any examples of major JavaScript libraries that support AMD module loaders but not CommonJS. So choosing AMD limits you in a way that choosing CommonJS does not.

This is not a disadvantage that’s likely to get fixed for AMD modules anytime soon. If these network effects shift at all, they’re only likely to do so in favor of ES6 modules, which have the appeal of being a “blessed” part of the JavaScript language. So in the short and long term, it’s fair to expect that using AMD modules will limit your compatibility with other tools and libraries compared to CommonJS.

What are the alternatives to RequireJS?

In 2015, RequireJS is one of 3 major options on the module loading scene, along with Browserify and Webpack. Browserify is an attempt to build a module loader on top of the NPM ecosystem and node modules. It uses CommonJS modules and integrates tightly with NPM. Webpack is an attempt to unify the modules landscape by supporting AMD, CommonJS and ES6 modules. It handles JavaScript, CSS and other assets, as well as preprocessors for each. RequireJS suffers in comparison to both of them, both in terms of features and workflow.

Browserify and Webpack both provide servers for development, allowing you to instantly integrate changes without a long build process or extra JavaScript files loaded into the browser. RequireJS doesn’t have a great solution for development workflow, requiring you to either do a full build every time, or load a copy of the require.js JavaScript file up to your browser where it loads files from the client. This has disadvantages both in terms of reliability (you’ll be running the files in a different way in development than on production, and there will be timing differences due to the need to load the files on the client), and configuration (you’ll need to have separate configurations for development and production that will have to be kept in sync somehow).

Browserify and Webpack also provide a clean syntax for preprocessing things like CoffeeScript or JSX files, using transforms or loaders (respectively). These transforms can be applied to a subset of files with configuration. These methods compare favorably to RequireJS’s plugin system, which require you to manually specify the plugins used each time you load a resource.

All of this adds up to a strong case for not starting new projects with RequireJS. It’s a good tool that has benefited many projects, but as other people have learned from its pain points and provided superior solutions, the time has come to move on.

Disclaimers

It should be noted here that this article is addressed primarily at developers starting new projects. If you’re working on an project that already uses RequireJS, you’ve probably already worked through the configuration pain, gotten used to the syntax, found a compatible unit testing library and developed a strategy for live development. If these aren’t things that are causing you pain, don’t switch. RequireJS is a stable tool that has stood the test of time. It’s not inherently flawed. There are simply better options out there these days for developers starting new projects.

More Resources

Axel Rauschmayer has a great piece up on his blog 2ality describing ECMAScript 6 modules and how they compare to CommonJS
This blog post is primarily an introduction post for Ampersand.js, a modular JS framework. But it also serves as a great look at the environment around nodejs and CommonJS modules, including a discussion of the advantages and disadvantages of using many small modules to structure applications.
Several users on reddit, have pointed out that systemjs is another competing module loader worth discussing. I didn’t include it because it’s less mature and wouldn’t be something I’d currently recommend for production apps, but it looks to be getting there quickly and is certainly another option to consider.

AMD does allow you to use an alternative syntax that is more similar to CommonJS that avoids this issue with "off by 1" errors. But since it's an alternative syntax, not the recommended one, I think it's fair to critique this as an issue. My other points apply equally to both forms. ↩

marionette-service: Service Objects for Marionette

Mon, 25 May 2015 10:02:00 -0400

Backbone.js is a great way to structure web applications. Its built in Models, Collections, Views and Routers are a very reasonable “minimum common ground” that pretty much any web project will be able to put to good use. But more sophisticated applications have needs that extend beyond these basic concepts. In addition to having some data that needs to be displayed, rich web apps often have resources and functionality that need to be shared across many different Views and Models. There’s no definitive way to implement this consistently, but a common pattern is to create “services” of some type that Views and Models can communicate with.

This pattern is easiest to show with an example. Let’s take a look at a simple logging implementation. We might start out just using simple console commands throughout our code like this:

function doFoo() {
  // Do a thing
  console.log('We did Foo');
}

That’s fine for a while, but what if we wanted to switch to a logging library like JSNLog or some 3rd party service? We’d have a huge find and replace job to work through. It’s better to centralize. Backbone.Radio is perfect for this purpose. We can use it to centralize our logic and create a logging module:

import * as Radio from 'backbone.radio'

let loggingChannel = Radio.channel('log');

loggingChannel.comply('log', function(message) {
    console.log(message);
});

loggingChannel.comply('logError', function(message) {
    console.error(message);
});

loggingChannel.comply('logWarning', function(message) {
    console.warn(message);
});

We could then log a message like this:

function doFoo() {
  // Do a thing
  loggingChannel.command('log', 'We did Foo');
}

There are 2 main use cases for this sort of service pattern: centralizing common tasks and UI actions like logging or displaying an alert, and requesting a resource without having to worry about where it came from. Both are easily possible using Backbone Radio, using code very similar to whats above.

Using Backbone Radio like this is a useful pattern, but it’s very ad hoc and imperative. Not very Backbone-like. What if we could follow the Marionette route of pulling a useful pattern into an object and allow it to be configured declaratively? We could instead create a “Service Object” like this:

import * as Mn from 'backbone.marionette'

let LoggingService = Mn.Service.extend({

    radioCommands: {
        'log log': 'logMessage',
        'log logError': 'logError',
        'log logWarning': 'logWarning',
    },

    logMessage: function(message) {
        console.log(message);
    },

    logError: function(message) {
        console.error(message);
    },

    logWarning: function(message) {
        console.warn(message);
    },
});

let logger = new LoggingService();

This is much more in the Backbone and Marionette style, with a declarative hash to delegate to different functions based on channels and the events passed. It makes it much easier to create “service objects” to make functionality available in a decoupled way.

Declarative radio handling will be coming to Marionette Core as part of Marionette version 3, but for now I’ve written a small library to make it available in the short term.

marionette-service provides the above Service Object to Marionette. It allows you to listen to Radio Events, Commands or Requests, and associate them with callback functions. It’s compatible with plans for Marionette v3, so all that will be required to remove it when upgrading will be a name change for service objects to use Marionette.Object rather than Marionette.Service. I’m excited to see how people use it.

Marionette-service is available to install both on npm, and bower. Or you can clone the repo directly from Github.

More Resources

If you’re new to Backbone.Radio you can learn more about it from my post, or from the official docs.
If you’re confused why somebody would want to separate out the implementation of a service through Radio rather than passing a service object directly, this is a nice piece by HB Stone on the power of loose coupling and the Mediator Pattern

Why Backbone.js and ES6 Classes Don't Mix

Mon, 06 Apr 2015 23:19:44 -0400

I’ve seen some confusion out there about how to use ES6 Classes with Backbone. In the original version of the ES6 class spec, this was actually a simple thing to do, though the syntax wasn’t great. With the finalized specification for classes and subclassing though, it’s no longer practical to try and use the two together. Here’s a short explanation of the issues, as well as a discussion on whether any of this matters.

What are ES6 Classes anyway?

ES6 ¹ is the newest version of JavaScript. It provides a variety of new features for JavaScript developers including native Promises, destructuring, modules and default function parameters. One of these new features is a native JavaScript Class implementation. It allows us to simplify class patterns that were formerly a bit verbose in JavaScript. Take this example of JavaScript inheritance from Javascript: The Good Parts:

var Mammal = function (name) {
    this.name = name;
};

Mammal.prototype.get_name = function () {
    return this.name;
};

Mammal.prototype.says = function () {
    return this.saying || '';
};

var Cat = function (name) {
    this.name = name;
    this.saying = 'meow';
}

Cat.prototype = new Mammal();

Cat.prototype.purr = function (n) {
    var i, s = '';
    for (i = 0; i < n; i += 1) {
        if (s) {
            s += '-';
        }
        s += 'r';
    }
    return s;
};

Cat.prototype.get_name = function () {
    return this.says() + ' ' + this.name + ' ' + this.says();
};

It’s a bit verbose. With ES6 classes we can instead write:

class Mammal {
    constructor() {
        this.name = name;
    }

    get_name() {
        return this.name;
    }

    says() {
        return this.saying || '';
    }
}

class Cat extends Mammal {
    constructor() {
        super();
        this.saying = 'meow';
    }

    purr(n) {
        var i, s = '';
        for (i = 0; i < n; i += 1) {
            if (s) {
                s += '-';
            }
            s += 'r';
        }
        return s;
    }

    get_name() {
        return this.says() + ' ' + this.name + ' ' + this.says();
    }
}

This allows us to clean things up a lot, removing some boilerplate and focusing the code on our intended functionality. ES6 classes work quite nicely for simple JavaScript classes like that. But what about Backbone objects like Models and Views?

We might think that we can just treat them as classes and the new syntax will “just work”, similar to how CoffeeScript classes function. So we’d take this example from the Backbone website

var DocumentRow = Backbone.View.extend({

  tagName: "li",

  className: "document-row",

  events: {
    "click .icon":          "open",
    "click .button.edit":   "openEditDialog",
    "click .button.delete": "destroy"
  },

  initialize: function() {
    this.listenTo(this.model, "change", this.render);
  },

  render: function() {
    //...
  }

});

and convert it to something like this:

class DocumentRow extends Backbone.View {

  tagName: "li"

  className: "document-row"

  events: {
    "click .icon":          "open",
    "click .button.edit":   "openEditDialog",
    "click .button.delete": "destroy"
  }

  initialize() {
    this.listenTo(this.model, "change", this.render);
  }

  render() {
    //...
  }
}

Unfortunately this errors out. So what’s the problem? It turns out that ES6 classes don’t support adding properties directly to the class instance, only functions/methods. This makes sense when you understand what is actually happening. With JavaScript inheritance, properties are generally meant to be set on an instance when its created, while methods are set on the prototype object and shared between every instance. If properties are added to the prototype directly they will also get shared between every instance, creating problems if the property is an object with mutable state like an array. You can see that in the following simple example:

var testClass = function(){};
testClass.prototype.foo = [];

var testA = new testClass();
var testB = new testClass();

console.log(testA.foo.length); //0
console.log(testB.foo.length); //0
testA.foo.push('x');
console.log(testA.foo.length); //1
console.log(testB.foo.length); //1

When we place a property on the prototype it can cause unexpected side effects. Instead most of the time, we want to use the class' constructor to add any properties to each new instance. ~~Since that is a bit unwieldy when adding many properties, Backbone abstracts away this difference with its extend function.~~ Edit: Backbone actually does add properties directly to the prototype, going against conventions a bit. But ES6 classes are focused on the prototype, and the code that is shared between instances. They make the opinionated default properties for instances are handled in the constructor. This maps very cleanly to the current semantics for existing JavaScript class creation, but is different than Backbone’s abstraction. So using ES6 classes for Backbone, we’ll need to give up this nicety.

class DocumentRow extends Backbone.View {

    constructor() {
        this.tagName =  "li";
        this.className = "document-row";
        this.events = {
            "click .icon":          "open",
            "click .button.edit":   "openEditDialog",
            "click .button.delete": "destroy"
        };
        super();
    }

    initialize() {
        this.listenTo(this.model, "change", this.render);
    }

    render() {
        //...
    }
}

Until recently, this was a valid way of defining a Backbone View with ES6 classes. It’s questionable whether it was a real gain, given the relative ugliness of the syntax for defining instance properties, but it did work. Unfortunately, in the final version of the ES6 spec for classes, this is no longer valid. The final spec requires super() to be called before this is referenced in a constructor for classes that are extending another class. So we have to change our constructor to look like this:

class DocumentRow extends Backbone.View {

    constructor() {
        super();
        this.tagName =  "li";
        this.className = "document-row";
        this.events = {
            "click .icon":          "open",
            "click .button.edit":   "openEditDialog",
            "click .button.delete": "destroy"
        };
    }
}

Unfortunately this doesn’t do what we expect. Backbone does significant initialization processing in its constructor, and most properties that Backbone expects to be defined on a Model or View need to be pulled in prior to the constructor executing in order to be used properly. So with the final ES6 spec we no longer have any way of using declarative properties on subclasses if we want them to be processed by the constructor.

So what are our options?

There are a few workarounds here, but they’re ugly. First, Backbone allows any of of its instance properties to also be defined as methods. So we can rewrite our View like this:

class DocumentRow extends Backbone.View {

    tagName() { return "li"; }

    className() { return "document-row";}

    events() {
        return {
            "click .icon":          "open",
            "click .button.edit":   "openEditDialog",
            "click .button.delete": "destroy"
        };
    }

    initialize() {
        this.listenTo(this.model, "change", this.render);
    }

    render() {
        //...
    }
}

We’re starting to get ridiculous here though. First, this isn’t a one to one conversion, so if we’re converting existing code, we could develop bugs from any code that directly referenced a View’s properties. And if that code modified those properties, its no longer as simple to modify them again. Of course if it’s a property that isn’t processed within the constructor like the url property for Models, we can simply leave it as is. But suddenly we’re requiring ourselves to know what properties are processed in the constructor or not, as well as wrapping and rewrapping properties in functions. We’ve added a lot of complexity, and it’s not totally clear what we’ve gained.

The second option is to simply re-run the Backbone View or Model constructor a second time after adding the initialization properties. This has a few issues though. First, it’s ugly. Nobody wants to do this:

class DocumentRow extends Backbone.View {

    constructor() {
        super();
        this.tagName =  "li";
        this.className = "document-row";
        this.events = {
            "click .icon":          "open",
            "click .button.edit":   "openEditDialog",
            "click .button.delete": "destroy"
        };
        Backbone.View.apply(this);
    }
}

But it also opens an opportunity for more subtle bugs. Constructors aren’t meant to be run twice. Doing so, will among other things cause the Views initialization function to run twice, with a different cid (Backbone’s internal unique id system) set on the View each time. That violates a ton of expectations, again for no clear gain. So the long and the short of it is that there’s no longer any viable way to use ES6 classes for Backbone objects without a clear loss in functionality and predictability.

So Does Any Of This Matter?

The idea of using ES6 classes for Backbone has some appeal. The “class” keyword is nice syntax and there’s an appeal to using more standardized code and fewer library extensions. Right now, many developers are also just curious about ES6 and how it fits into their workflow. But does it really matter that this doesn’t work? I think a case can be made either way, with 2 clear points in each sides favor. The case for this being a problem:

ES6 Classes are becoming the standard

By pretty much any standard, Backbone is currently one of the 4 most popular JavaScript frameworks/libraries for building web applications, along with Angular, Ember, and React. What do the Angular, Ember, and React teams all have in common? They’re each working to make sure that current or future versions of their frameworks use ES6 classes to define objects. As this style of code becomes ubiquitous, Backbone will start to look even more boilerplate heavy and non-standard. Developers will also start learning about ES6 classes as part of standard JavaScript training, requiring them to map their knowledge about prototypes and class syntax to Backbone’s concepts, rather than just using things they already understand as in other libraries.
More Native == Less Library code

This is more of a far-future advantage, but at some point ES6 classes will be usable everywhere without library support, allowing libraries based on it to include less code for handling this themselves.²

And the case that it doesn’t really matter:

Just because its standardized doesn’t make it reusable

While it’s true that ES6 classes are a standard, realistically Backbone code written using a class syntax isn’t going to be easier or harder to convert to equivalent code using React for example. Backbone’s way of using it’s standard functions and properties is a much bigger obstacle to code reuse than the particular way it defines classes. ES6 classes and Backbone objects are in the end just different syntactic sugar over the same prototypical inheritance concepts. A developer who understands the concepts will be able to use both, and code written in either style will not be easy to reuse outside the Backbone ecosystem.
Sometimes doing it yourself is just better

The truth is that sometimes there are advantages to writing your own APIs. Backbone’s class code makes the things that are important to using Backbone easy, without any of the trade offs discussed in this article. There’s something to be said for using the right interface for the job. There are trade offs for this, just as their are trade offs in other libraries choice to require or encourage a build step for their code. Both these choices improve the developer API at a cost (complexity, extra code, extra process, etc). Whether those trade offs are worth it is a judgment call.

It will be interesting to see how the lack of standardization here effects Backbone in the long run, if it does at all. Along with native Promises, ES6 now offers 2 major features that are similar to but incompatible with important Backbone features ³. Along with ES5’s get and set capabilities and ES7’s Object.Observe ⁴, a significant number of Backbone features that once required library code will someday soon be natively supported. Of course Backbone has never been a set of features as much as a way of writing programs. Since Backbone is essentially finished and there are no plans to significantly change the interface, the continued improvements to the standard language serve as both a validation of Backbone’s core concepts as well as a challenge to its future.

Update: There’s been a great discussion of this issue on Backbone’s Github issue tracker. Feel free to weigh in there if you’re interested in figuring this out.

More Resources

If you do want to use ES6 now, Babel is the best tool for making that happen. The Babel website also contains a great rundown of the various features that make up ES6.
Axel Rauschmayer has a great article describing the final spec for ES6 classes, including a description of the changes that made them incompatible with Backbone.
React Developers have noted the same issues with property initializers that Backbone users encounter. As part of version 0.13 of React, they’re supporting a special property initialization syntax for classes, which may eventually be standardized. There’s more info on that in this ESDiscuss thread. This standard is still being worked out but an experimental support version is available in Babel 5.0.0. Unfortunately that version defines class properties as being instantiated after the superclass constructor is run, so this doesn’t solve Backbone’s issues here.
If you were confused by the discussion of prototypes and the code examples of non-ES6 JavaScript classes, I wrote a quick explanation of Object Oriented Programming in Javascript back in 2012.

ECMAScript version 6, now officially ECMAScript 2015 ↩
This is admittedly a bigger advantage for Ember, Angular 2.0, and React, which all encourage using a build step to convert ES6/Typescript/JSX to ES5 JavaScript. That means they can all rely on ES6 class support now, since older browser support is handled through compilation.
Backbone doesn't require or encourage a transpilation step (although CoffeeScript works very nicely for this case), and so it's more important for it to have a top-notch API supported by all JavaScript engines that it is likely to be run on. It will be a long time before that is true of ES6 classes. ↩
Promises are similar to jQuery deferreds, which are technically not a Backbone feature, but are a key part of Backbone development for many developers because they're used with Backbone sync. ↩

Staying DRY with Marionette Behaviors

Mon, 23 Mar 2015 00:04:17 -0400

This is the sixth post in a series on Marionette.js. For more background on what Marionette is, check out the series page, Marionette Explained

Any successful project draws complaints, and Backbone.js is a successful project by pretty much any measurement. The biggest complaints I’ve heard leveled against Backbone as an MVC framework revolve around the boilerplate involved in writing Backbone apps. The claim goes that Backbone forces you to write the same event binding and workflow code over and over again. In general I’d claim that’s a feature of bad Backbone apps, and not something unavoidable. But it’s certainly true that Backbone doesn’t give you much guidance on how to reduce repetition and boilerplate in your code. Making it easy to reduce repetition turns out to be one of the main roles of Marionette. This post is a look at one of the tools that Marionette provides to help you simplify your applications: the Behavior class.

Marionette Behaviors are reusable chunks of UI logic that you can mix into your Views. When you have a common problem that you face across several Views, Behaviors can help you reduce boilerplate and repetition. We’ll take a look at a practical example first, and then we can get into the Behaviors API.

Let’s see how Behaviors can help us write better form validation code. When I’m validating forms, I use the backbone.validation library. It provides helpers for checking the validity of model properties and attaching error messages to inputs, but doesn’t force a specific workflow. However, I have a specific flow I usually want. A user should be able to enter data as they see fit to start, with no validation warnings until they’ve tried to submit the form. After they’ve initially submitted the form, warnings should appear next to any invalid fields, and they should update and disappear as the user fixes their input.

That process is pretty easy to implement with backbone.validation. I simply connect my Model to each form View I create, and in the Model I add some logic to re-validate each time the Model is changed, but only after the first time validation is triggered by a user submit. That works great in most cases. The problem comes when I need to re-render my view in the middle of this process. This is a pretty common occurrence if your form has sub-views that may appear based on the contents of other parts of the form. In that case, I have to make sure that the validation state persists across renders. That might look something like this. ¹

import * as Mn from 'marionette';
import * as Validation from 'backbone-validation';

var FormView = Mn.LayoutView.extend({

    template: '#form',

    ui: {
        submit: '.submit'
    },

    events: {
        'click @ui.submit': 'submitForm'
    },

    modelEvents: {
        'validated': 'setValidated',
    },

    onRender: function() {
        //any other post-render View code here

        Validation.bind(this);
        if(this.validated) {
            this.model.validate();
        }
    },

    setValidated: function() {
        this.validated = true;
    },

    submitForm: function() {
        //handle form submission
    }

});

So it’s not that much code to add the functionality we want. But when we need this code on multiple Views, adding and maintaining it becomes a pain quickly, creating the type of boilerplate that developers rage against. So how can Behaviors help? It turns out to be easy to pull the functionality from our example out into a separate Behavior that any View can implement.

That would look something like this:

import * as Mn from 'marionette';
import * as Validation from 'backbone-validation';

var ValidationBehavior = Mn.Behavior.extend({

    modelEvents: {
        'validated': 'setValidated',
    },

    onRender: function() {
        //Set up any other form related stuff here
        Validation.bind(this.view);
        if(this.hasBeenValidated) {
            this.view.model.validate();
        }
    },


    setValidated: function() {
        this.hasBeenValidated = true;
    },

});

export default ValidationBehavior;

You could then pull the Behavior back into the View like this:

import * as Mn from 'marionette';
import * as Validation from 'backbone-validation';
import * as ValidationBehavior from 'behaviors/validation';

var FormView = Mn.LayoutView.extend({

    template: '#form',

    ui: {
        submit: '.submit'
    },

    events: {
        'click @ui.submit': 'submitForm'
    },

    behaviors: {
        validation: {
            behaviorClass: ValidationBehavior
        }
    },

    submitForm: function() {
        //handle form submission
    }
});

We could do this for any number of Views that required validation. The same thing can be done for many other types of functionality, including key-binding and “warn before delete”.

Behaviors API

Behaviors have essentially the same API as Marionette Views, because they’re meant to be extracted out of existing Views. So the event hashes, ui elements, and life-cycle methods that you find on a Marionette View are all also available on Behaviors. The life cycle functions on a Behavior execute immediately after their corresponding functions on the Behavior’s associated View. Behaviors also get el, $el and $ properties that are proxied to the equivalent properties on the Behaviors view.

Behaviors are not just mixins though. Behavior properties are isolated from the Views they’re associated with, preventing naming collisions and also encouraging reusable, decoupled designs. Generally Behaviors don’t directly modify their associated Views or data. The communication flow is one way, with Behaviors accepting input from their parent View in the form of an options object that is attached to the Behavior when it is created. You can pass these options through using the behaviors hash on the View. For instance to pass an error message to the validation behavior above, you could define your behaviors hash like this:

    behaviors: {
        validation: {
            behaviorClass: ValidationBehavior,
            errorMessage: 'You did something wrong.'
        }
    },

A quick note on how Behaviors attach to Views

You can see in the example above that the general form for attaching Behaviors to a View is to include a key-value pair for each Behavior, where the value is an object that lays out the options for the Behavior. The interesting thing in the example above is that the name of the key doesn’t matter at all. It’s not used by the library in any way; instead Marionette picks out the correct behavior to use based on the behaviorClass property in the value. So why is the API structured like that? To understand, you need to know that the Behaviors API supports two separate scenarios for looking up the correct Behaviors for a View.

The original API Marionette included for looking up Behaviors requires you to override a behaviorsLookup function on Marionette to tell the library where to retrieve your Behaviors from. The keys of the behaviors hash on a view were passed to the lookup function, which then retrieved the Behavior classes from wherever the user chose to store them (probably a global namespace object).

That API works great for developers used to using namespaces to organize their code, but for developers using modern JavaScript module loaders, it introduces global state and indirection where it wasn’t needed. To handle this case, the behaviorClass key was added as an option for the value object, allowing developers to specify Behaviors as dependencies of the file they were defining a View in, and then attach the Behavior directly to the View.

That makes much more sense for module users, but leads to a clunky API where the keys aren’t used. Fortunately, in newer versions of Marionette you can get around this entirely, by defining your behaviors as an array on a view. So the example above would look like:

    behaviors: [{
        behaviorClass: ValidationBehavior,
        errorMessage: 'You did something wrong.'
    }]

This works due to an undocumented implementation detail in Marionette 2.4.1 ², but will be fully supported in future versions. In future versions you also will be able to pass a class directly as an array item if the Behavior is not taking any arguments, further cleaning up the API.

When are Behaviors useful?

Marionette’s documentation explains Behaviors very broadly, which makes sense for official documentation since it gives ideas without limiting how users think about them. But in my experience Behaviors are best used for three different use-cases.

This is the most obvious use for Behaviors, and what the documentation focuses on. It’s easy to set up common event based UI patterns like “warn on close” with behaviors. But they also can be used for more complicated event based UI logic. At Windsor Circle we use 2 behaviors to share common “drag and drop” code with a Droppable Behavior that can be implemented by a CollectionView and a Draggable Behavior for ItemViews. That makes it simple to make any CollectionView based list sortable with drag and drop, and keeps the code in a nice central place where any improvements will be shared across the code base.

In addition to event handling, Behaviors also give you access to the various life cycle methods of Marionette Views. That lets you split out common life cycle functionality and reduce code. An easy example of this would be a Behavior that fades a View in each time by adding a css class with an animation associated with it.

let FadeIn = Marionette.Behavior.extend({
    onBeforeShow: function() {
        this.$el.addClass('hidden fadein');
    },
    onShow: function() {
        this.$el.removeClass('hidden');
    }
});

which would work with a CSS snippet like this:

.fadein {
  transition: opacity 1s;  
  opacity: 1;
}

.fadein.hidden {
    opacity: 0;
}

Simplifying integration with libraries

This final use case is a special case of the first 2, but I think its worth mentioning in its own category. Many 3rd party Backbone or jQuery libraries require repetitive initialization code to use within a View. This code is often the same across all Views, or only requires minor tweaks. The perfect use case for a Behavior! The Validation code above is one example of this, but it’s useful for many plugins. For instance, if you use the Chosen jQuery plugin to create rich dropdown boxes, you could create a Behavior to automatically initialize select elements with Chosen, and optionally pass a class to restrict it to only initialize selects with a specific class.

let UseChosen = Marionette.Behavior.extend({
    onRender: function() {
        let className = this.options.className,
            chosenOptions = this.options.chosenOptions || {};
        if(className) {
            this.$('.${className}').chosen(chosenOptions);
        }
        else {
            this.$('select').chosen(chosenOptions);
        }
    }
});

Wrap-Up

Behaviors are a great resource for avoiding repetition in your Backbone code. If you have examples of good open-source Behaviors out there, please mention them in the comments. I’d also love to hear what use cases you’ve found for them, even if the code isn’t public to share.

More Resources

Atomic Object has a nice writeup on Marionette Behaviors, and a followup on testing them.
Behaviors seem to make the most sense to people when given examples of how you might use them. One great open-source example is this key-binding Behavior that makes it easy for you to add keyboard shortcuts to a View

Going forward on this blog I'm planning on using ES6 features in code examples where appropriate. If the code looks weird to you check out this nice summary of ES6 features.
↩
The latest version of Marionette at the time of writing.
↩

Guillermo Rauch on ECMAScript 6

Sun, 22 Feb 2015 17:38:46 -0400

ECMAScript 6, henceforth ES6 and formerly ES.next, is the latest version of the specification. As of August 2014 no new features are being discussed, but details and edge cases are still being sorted out. It’s expected to be completed and published mid-2015.

Adopting ES6 has simultaneously resulted in increased productivity (by making my code more succinct) and eliminated entire classes of bugs by addressing common JavaScript gotchas.

More importantly, however, it’s reaffirmed my belief in an evolutionary approach towards language and software design as opposed to clean-slate recreation.

http://rauchg.com/2015/ecmascript-6/

A really great look at the highlights of ES6 in practice. I’ve been working heavily with ES6 for the last month or so, and agree with him on the highlights. The standardized module syntax (which I transpile to AMD modules), Promises and destructuring particularly stand out as useful syntax that produces better code than what I had before. I also enjoy the => syntax, though I wish the language authors hadn’t combined a syntactic convenience (shorter syntax) with a semantic difference (fat arrow functions are bound to the current context by default, normal functions are not).

Also of note is his repeated oblique references to Babel (formerly 6to5). Since I wrote my post on Alternative JavaScript 2 months ago, Babel has clearly emerged as the best way to use JavaScript features that are not yet widely supported today. It integrates with a wide variety of tools, is reasonably performant speed-wise, and produces roughly 1-1 output, Coffeescript style. Highly recommended for anyone looking to get started using or learning about new JavaScript features.

Building Modular Web Apps With Backbone.Radio

Mon, 26 Jan 2015 08:00:00 -0400

This is the fifth post in a series on Marionette.js. For more background on what Marionette is, check out the series page, Marionette Explained

One of the great things about Backbone is how it encourages a modular approach to building UIs. Different portions of the pages are separated into individual Views, allowing you to simplify your reasoning about your code by focusing on bite-sized chunks at a time. Easier said than done though, right? In practice, it’s not hard to create Backbone applications with large views that have tight dependencies on each other. This usually isn’t an intentional decision, but part of a slow decay as developers work to connect the various aspects of their applications.

So far in this series I’ve talked about how Marionette makes it easier to create maintainable Views. Now I’m going to look at Backbone.Radio, Marionette’s tool for communication between application components.

What’s so hard about modular architecture?

There are many advantages to designing your web UI in a modular or component-based fashion. Your code tends to be easier to reason about and test, it’s easier to swap out one piece of code for another, and it greatly increases the chances that you’ll be able to reuse your code in other places. Unfortunately for all except the most basic applications, different components of a UI always have to be aware of the things that happen in other areas of the UI. It may be something simple like responding to a button click by showing a modal, or more complicated like updating multiple dashboard items when the user applies a filter, but these connections will exist. At this point you have options. You can begin having components reference each other directly for example, or you could try to capture state within a shared data object that each component watches. But there’s another solution that borrows from many years of development best practice on backend systems. You can use a message bus.

A simple Message Bus

A message bus is a software architecture model for communicating between two systems. The idea is that instead of referencing each other directly, system components are able to send messages through a shared mediator object. It’s quite straightforward to create a simple message bus using Backbone.

Backbone.Events is Backbone’s event object, which gets mixed into the various Backbone classes to provide the Backbone event functions (on, off, listenTo, trigger, etc). It’s easy and common to mix it into a separate object. That’s all you need to create a message bus: something like this:

var mediator = _.extend({},Backbone.Events);

Now Models and Views can listen to or trigger events on the mediator object, allowing them to communicate without having to be directly aware of each other. This is a great pattern, and quite useful as is, but there are some limitations. With a mediator object, all events are global. Any object can listen to any event, and you have to be careful to avoid naming collisions when using common terms. It’s also a single object for multiple types of communication.

The Backbone.Radio API

Backbone.Radio builds on this concept to provide a more powerful message bus. Instead of a single object, Radio provides multiple “channels” each of which can correspond to a page or functional section of your web app. Also, instead of a single events API, Radio provides three, Events, Commands, and Requests, each built for a different use case.

Events

Radio Events are the exact same as our simple message bus. When you create a channel with Radio, they extend Backbone.Events. That means you can call any Backbone.Events function on them, or pass them to the listenTo method of other Backbone objects. Events provide Radio’s publish-subscribe functionality. Multiple objects can subscribe to an event using on or listenTo. And any object can publish an event using trigger. For those of you unfamiliar with Backbone’s events, they have two features that you might not expect. First, unlike native browser events or jQuery events, Backbone events are synchronous; when you trigger an event, all callbacks that are registered on that event run immediately. Second, it’s easy to pass data with Backbone events. Unlike native browser events which tend to simply consist of an event and a target, any data you pass when you trigger an event gets passed directly to the callback function, making it a true messaging system.

Functions Provided - on, off, trigger, once, listenTo, listenToOnce, stopListening

Commands

Update January 2016 - During 2015, Commands were deprecated from Backbone.Radio as part of their 1.0.0 release. From a code level, they’re simply a request that doesn’t return a value, so feel free to keep any existing code, but change command to request and comply to reply

Radio Commands are similar to Events, with the distinction that only a single callback can handle or “comply” to a given command. Commands do not add any functionality on top of events; instead, they provide a semantic distinction and enforce simpler logic. When exampleChannel.command('something') is called, you know that it is requesting an action be performed, and you can be confident that there will be no side effects. Only one object will comply to the command. Commands are a maintenance-friendly tool; they make your code easier to follow by doing less and doing it more obviously.

Functions Provided - comply, stopComplying, command, complyOnce

Requests

Radio Requests share the one-to-one nature of Commands, but provide functionality not seen in Backbone.Events. You can use Requests to retrieve information from other objects. Each request can be serviced by a single response callback, with the request receiving the return value of the callback function. This is very useful for providing resources in a decoupled manner, where a View or other object doesn’t have to know the details of how an object is stored or retrieved.

Functions Provided - reply, stopReplying, request, replyOnce

Channels

Radio Channels are a method of segmenting out your messaging across functions. Instead of holding every event inside a global namespace, objects can interact with a Channel that’s limited to the functionality they need. This minimizes the chances of naming collisions across the app, and provides additional semantic guidance for future readers of the code. It’s much easier to trace through small, focused messaging networks than a large, global one when you’re trying to understand a complex behavior.

Backbone.Radio in practice

Radio’s API is fairly small and simple. Its power comes from the development patterns it enables. The following are examples from my own code of how Radio assists in building modular testable UI code.

Decouple your Views with Radio Events

Loose coupling is one of the most commonly cited design principles in Object Oriented Programming. In Backbone, loose coupling can be applied to mean that your Views should know as little about each other as possible. In my own work I implement this by allowing Views to know about and reference their own child Views, but do not allow them to directly reference any other Views. If Views need to communicate up or across a tree of Views, I use Radio.

Radio Events are great for alerting other parts of the application about a change of state. For instance, in an application where we have a main content area and a sidebar that provides context for the main area (for instance, a mail app like Gmail that shows contact info in the sidebar next to an open email), we could use Radio Events to keep our sidebar in sync with the main content. That might look something like this.

var Mn = require('backbone.marionette');
var Radio = require('backbone.radio');

var inboxChannel = Radio.channel('inbox');

var ContactView = Mn.ItemView.extend({

    template: '#contact-template',

    initialize: function() {
        this.listenTo(inboxChannel, 'show:email', this.showContact);
        this.listenTo(inboxChannel, 'show:inbox', this.showAd);
    },

    showContact: function(emailObject) {
        //show the contact for the emailObject
    },

    showAd: function() {
        //when we don't have a contact to show, show an ad instead
    }

});

module.exports = ContactView;

Note that our sidebar View doesn’t need to know anything about the inbox area. The show:email event could be fired by our inbox view, by a navigation View, or by our router on page load. The sidebar doesn’t know and doesn’t care.

It’s also important to note that the event represents a change in state external to our component. Something else changed (the state of the inbox), and the sidebar component is reacting to this. There might be many coponents that react to this state change, or just this one. Our View doesn’t know or care.

Centralize UI Logic with Commands

While Events are a way of notifying other components about a change in state, commands are useful when you want to cause a change in state without having to know the implementation. For instance, in an application that I work on, we have a centralized event-logging module that can take application events that we want logged and send them to a backend API and also to Google Analytics. We use commands for this. In practice the logging module looks a bit like this.

var Radio = require('backbone.radio');

function log(event) {
    // Send event to the backend API and Google Analytics
}

Radio.channel('appEvents').comply('log',log);

Then we can run Radio.channel('appEvents').command('log', event) anywhere we want to log an application event. We could use events for this, but there’s value in using commands. We’re explicitly saying that we want our application to log a specific piece of data, whereas a similar event would have the connotation that an event had occured, and anything that wanted to respond to it could. We can be more confident of what’s happening here.

One of the challenges of nested View architectures like Marionette or ReactJS is deciding how to pass data down to deeply nested View components. One approach is to pass all necessary data into the top level view and then pass it down along the View tree to whatever View needs it. That has the advantage of a clear data flow, but the disadvantage of cluttering up parent Views with data they don’t need and tightly coupling the parent Views to the implementation of their children. Another approach is simply making the data available globally, which has all the disadvantages that global data normally brings. Radio requests provide a nice solution to this problem. You can define “data providers” that are responsible for providing any data you need that is not passed directly down a view chain. These could be separate modules similar to Angular’s service objects. They could be other UI components like a datepicker. Or they could just be models and collections you define at the root of your application but don’t pass directly to your Views. In the same application as above, I have a set of task views that can have users assigned to them. Because these tasks are deeply nested in the UI, I don’t pass the collection of possible Users to assign down the View tree. Instead, I define that collection within my top level application object and then request it when the user opens up the user-picker View. I use jQuery deferreds to make sure that the collection has been populated before the picker appears. That all looks something like this:

//in app.js
var usersDeferred = (new UserCollection()).fetch();
resourceChannel.reply('userlist',function() {
    return usersDeferred;
});

//in task.js
var usersDeferred = resourceChannel.request('userlist');
usersDeferred.done(this.showUserPicker);

A Quick Note on Backbone.Radio and Marionette

Those of you who have used Marionette before may be a bit confused as to why this post is part of my Marionette article. As of version 2.3, the current shipping version of Marionette, Radio is not distributed as part of the library. Marionette instead ships with Backbone.Wreqr as its message bus implementation. Wreqr is a precursor to Radio with a significantly more confusing API. Radio provides a cleaner approach, as well as a more intuitive name. However, because Marionette follows semantic versioning, they’re waiting for 3.0 to make Radio the default.

In the meantime, though, it’s easy to replace Wreqr with Radio. You can simply make sure that you’re loading in Radio instead of Wreqr, and then implement the following shim:

(function(root, factory) {
  if (typeof define === 'function' && define.amd) {
    define(['backbone.marionette', 'backbone.radio', 'underscore'], factory);
  } else if (typeof exports !== 'undefined') {
    module.exports = factory(require('backbone.marionette'), require('backbone.radio'), require('underscore'));
  } else {
    factory(root.Backbone.Marionette, root.Backbone.Radio, root._);
  }
}(this, function(Marionette, Radio, _) {
  'use strict';

  Marionette.Application.prototype._initChannel = function () {
    this.channelName = _.result(this, 'channelName') || 'global';
    this.channel = _.result(this, 'channel') || Radio.channel(this.channelName);
  }
}));

It’s also worth pointing out that Radio has no direct dependency on Marionette, and can be used on any Backbone project. For example, it might be an interesting tool to implement a Flux-style dispatcher in an application using Backbone and React. All of the benefits Radio provides can help any application with decoupled View components.

Thanks to the Marionette Teams for the diagrams above, which come directly from the Backbone.Radio README.

More Resources

Although it’s focused on Node and backend use cases, this video is a great rundown of semantic messaging patterns.

Is Bower Useful?

Thu, 22 Jan 2015 00:52:06 -0400

I first heard about Bower about a year and half ago, and was immediately confused. Bower is branded as a package manager for the web. My immediate question: “Why create another package manager for JavaScript, when npm was just starting to become a standard?” I certainly wasn’t the only one thinking it.

"What's bower?" "A package manager, install it with npm." "What's npm?" "A package manager, you can install it with brew" "What's brew?" ...
— Stefan Baumgartner (@ddprrt) November 5, 2014

If you can use npm to install Bower, you can use it to install packages directly. And if you don’t like npm, is it that hard to just pull in your dependencies manually? Why use a package manager at all?

A year later, I now believe Bower is a useful and important part of the web development ecosystem, so in the interest of sharing knowledge and because I can’t resist the chance to flout Betteridge’s law of headlines, here’s my case for why Bower is useful.

Manual package management doesn’t scale

For quite a while I was content to manage my web dependencies manually. I would go to a project site, hit the big download button, move the files into my project and get on with my day. It was simple and required no setup or tools. I was happy. However I’m now managing a project that includes almost 30 external front end dependencies of some kind. At that scale it’s impossible to shrug off the pain points of the “Do It Yourself” approach.

The biggest problems come around keeping dependencies up to date. Simple solutions unravel quickly here. First, while it would be lovely if every front end library included a version number in their distributed file, that’s not always the case. So keeping track of versions reliably invariably means some sort of external documentation. Even when I knew the version, I would have to manually go to each libraries site to see if there was a newer version, then test it, and then either keep 2 copies of the library around or mess with git branches while I was testing to see if the new version caused any problems or regressions. That was fine with a few libraries, but it doesn’t scale well.

So my advice? Don’t try to manage your assets manually after your projects grow to a certain size. You’ll spend unnecessary time keeping them up to date, clutter your version control logs, and will have to actually document the versions somewhere anyway. So manual package management is out. But why not just use npm?

npm is the right answer to the wrong question

npm is a fantastic package manager for node development. Being able to immediately pull in a library and all of its dependencies then be confident that it will work with the module format you use is a fantastic thing, and npm has pulled it off well. So why not just use it for all of your web development package management needs?

The problem is that it’s impossible for tools to optimize for everything. npm has optimized for node development. That leads to problems when using it in other settings. First of all, many npm packages assume you’re using the commonJS module format. Node encourages small modules that may have many dependencies, linked with commonJS. It’s possible to use those modules on the front end of course if you use a tool like Browserify, but your choice of package manager should not lock you into a specific module format or build tool. Most major packages do distribute their code in a UMD format that can be loaded in multiple ways, but some do not, and when those packages have dependencies, we have another problem.

npm has a nested dependency tree, where each module also loads all of its dependencies individually. This is fine on the server, where you can put node_modules into your .gitignore file and effectively pretend those files are not there. When writing client-side code though, it’s important to be efficient with bandwidth. You can’t load multiple versions of a dependency if, for instance, you used multiple libraries that depend on Underscore or jQuery. That’s a problem if you’re using browserify, since you’ll need to find a way to normalize those dependencies to a single file. But nested dependencies are also annoying when using other problems, both due to the need to manually figure out what you need and avoid duplication as well as the awkwardness of making deep path references down several layers of node_modules folders.

Lastly, I’d encourage people considering npm for front-end package management to use a tool scoped for their needs. npm started as a package manager for node, and that is still its main use. There are plenty of packages that can be useful both on node and in the browser, but many things only make sense in one environment. For instance CSS libraries are not useful within node, and file system libraries won’t work in the browser. I’ll admit that this ship may have already sailed though. Many popular browser only libraries like Bootstrap and Angular are available on npm right now as a response to people who are using npm for everything¹.

So Why Bower?

The problems above explain the case for “yet another package manager”. So now we can talk about how Bower is better. Bower is focused on front-end development, provides a flat dependency structure, and doesn’t lock you in to a single way of doing things. Let me break that down.

Bower is focused on the front end. If you find a package registered on bower.io, you can be confident that it will useful on the front end, and won’t be using node-specific dependencies.

Bower has a flat dependency structure. If multiple libraries require a dependency, Bower will figure out the best version to load and place it in the top level of your bower_components directory.

Bower doesn’t lock you in. Nothing about Bower is specific to a certain module format or build tool. You can use it with RequireJS and Grunt, Webpack and Gulp, or by loading scripts directly into your HTML.

Bower addresses the pain points that come from manually managing your scripts or using npm. You get easy version tracking and updating, keep your git logs clean, and can be confident that it will work cleanly with your existing setup. If you’re starting a new project or updating your dependencies on an existing one, consider using Bower to make your life easier.

More Resources

Want to hear the case for using npm in your front end development? You can find it on their blog.
If you’re interested in Bower, it’s also worth checking out Yeoman, a project for quickly scaffolding out web projects using Bower and Grunt.

Something npm is clearly encouraging
↩

The Life of a Marionette View

Mon, 05 Jan 2015 00:01:45 -0400

This is the fourth post in a series on Marionette.js, a framework for developing apps with Backbone.js. For more background on what Marionette is, check out the series page: Marionette Explained Also note that this article was written about Marionette v2. The current latest version of Marionette is v3, and while most of the high level concepts remain the same, the specific syntax has changed in some cases.

When building Single Page Applications, it’s important to manage UI components throughout their existence on the page, making sure they’re properly created, attached, and destroyed. Marionette.js helps Backbone developers manage that by providing life cycle methods that developers can use when defining their Views. These life cycle functions fit nicely with the rest of Marionette’s features, and can help simplify UI development with Backbone.

Building scalable user interfaces is hard. It’s a challenge that sits at the intersection of fields requiring very different skill sets: Software Development, Graphic Design, User Experience and Information Architecture. Because interface code is concerned with both presentation and interaction logic and is exposed directly to user input, it can be difficult to reason about. Fortunately, as modern web developers, we have 30+ years of industry knowledge to fall back on. One lesson learned is the value of Object Oriented Programming concepts in UI design; specifically, the idea of breaking interfaces into a set of independent components, and giving each component a consistent “life cycle”, a set of events that occur between a component’s creation and destruction.

Having a clear life cycle for UI components makes a developer’s life easier in a few ways: It eases the process of reasoning about changes in a component over time, and how it interacts with other elements of the UI. It creates consistent hooks to add behaviors, which promotes readability and reduces boilerplate code. Finally, it smooths integrations with outside code that may need to be run when the component is in a specific state, without needing a bunch of conditional logic to detect when it should run. Marionette Views provide a life cycle for Backbone Views that help give all of these benefits.

The Marionette View life cycle is made up of a list of events and conventions. Events are fired at each signficant step in a View’s life cycle. Each event has a corresponding View method associated with it that Marionette automatically runs when the event is fired. These events and callbacks follow a consistent naming convention. An event that occurs before a moment in the life cycle is fired in the form before:<eventname> and the callback is called onBeforeEventName. An event that is fired after the same moment in the life cycle will have the form eventname and the callback will be called onEventName. By default the callback functions for each event do nothing, but developers can overwrite them to define behavior for a specific View instance. For instance, defining the callback functions for the render and before:render events on a new View would look like this.

var ExampleView = Mn.ItemView.extend({

    onRender: function() {
        // do stuff after the View is rendered
    },

    onBeforeRender: function() {
        // do stuff before the View is rendered
    }

});

Render is only one of the relevant moments in a View’s life cycle. This is the full list of relevant methods and events, with a brief explanation of how each one might be used.

Initialize

The first function in a View’s life cycle is initialize. initialize isn’t a Marionette-specific method. It’s part of Backbone, but it’s important to make sure you understand where it fits into the View’s life. Backbone provides two functions that are run when a View is instantiated, constructor and initialize. constructor is used for Backbone and Marionette’s internal setup, and shouldn’t normally be overridden. initialize is the suggested function for defining behavior on View setup. It’s a great place for setting up event listeners, and allows workings directly with the options passed to the View on creation ¹. Unlike the other methods on this list, initialize only provides a single method with no associated event.

onRender/onBeforeRender

The second step in the Marionette View life cycle is the render step, which exposes two events render and before:render, corresponding to the render and onBeforeRender callbacks. These events fire before and after the View template is rendered. In a Marionette View, rendering means that the View has generated a fragment of HTML, but it may or may not be visible or attached to the DOM. onRender is useful for any direct manipulation of the generated HTML that doesn’t require it to be visible or attached to the DOM.

onAttach/onBeforeAttach (Marionette 2.3+)

After rendering, the next set of events that might fire in a Marionette View’s life cycle is before:attach and attach. These events are fired whenever showing a View in a region causes it to be attached to the DOM. Because Views can be rendered independently of Regions, and some Regions may not be attached to the DOM when a View is shown, this event may not always be fired immediately (or at all). But if you’re using Regions to render all your Views, it can be very useful for setting up jQuery plugins or other operations that require HTML to be attached to the DOM before being acted on. A nice aspect of the attach events is that they will propagate down a tree of LayoutViews if you’ve set up a View hierarchy as described in my last post, so you can count on the event firing even if you build the View tree and then attach the top View to the DOM later.

onShow/onBeforeShow

Another Region-specific set of events, show and before:show fire before and after a View is shown within a region. These events are great for rendering child Views. The Marionette docs recommend using onBeforeShow specifically for rendering children, since it allows for only a single DOM paint when rendering nested Views, and prevents attach events from firing multiple times, which might happen if the child Views are attached to the parent after the parent View has already been shown.

onDomRefresh

The dom:refresh event is the last Region-specific event It’s triggered after an already shown View is rerendered. Like onAttach, this can be useful for managing UI plugins that manipulate the DOM, if they need to be reinitialized when the HTML tree is refreshed. It differs from onRender in that it will not run the first time the View is rendered, and you can count on the View always being attached to the document when it fires. So if an operation requires the View to be attached to the DOM, you can run any initial logic in onShow, and then put any refresher logic into an onDomRefresh callback. If you’re running Marionette 2.3 or later though, onDomRefresh is less useful because onAttach provides similar functionality in a more consistent way.

onBeforeAddChild/onBeforeRemoveChild/onAddChild/onRemoveChild (CollectionView Only)

CollectionViews get their own special life-cycle events to manage adding and removing Models to the Collection. CollectionViews automatically display a new ChildView when a Model is added to its Collection, and destroy the View when the Model is removed. They fire events before and after each of these changes, to allow you to customize the behavior around each. You could for instance hide specific Views if necessary. You could also modify them based on some logic, though that type of thing may be more appropriate within the child View’s code itself.

onDestroy/onBeforeDestroy

The final Marionette life-cycle event set is destroy and before:destroy. Appropriately, these events are fired when a View is destroyed and serve as a place to handle any custom cleanup. The most common use case for these functions is unbinding any events not covered by Marionette’s built-in event cleanup². It’s worth noting that you cannot use onBeforeDestroy to prevent the View being destroyed. The Marionette team’s position here is basically that you should not call destroy directly on any View you don’t want destroyed. If you’re swapping a View out of a region but don’t want it to be destroyed, Marionette provides a preventDestroy option to its show method that allows you to preserve the existing View.

Cogs In A Machine

The nice thing about Marionette’s View life cycles is the consistency it shares with the rest of Marionette. The life cycles are built to work with the Region and Layout system I described in my last post. If you lay out your Views using the system described in that post, you’ll be able to take advantage of all of the above functions consistently across your application. Regions themselves have similar life cycle methods, with callbacks that run before and after showing Views, swapping Views and emptying the region. Finally, Marionette Applications, the object that Marionette uses to initialize the app, also has a single set of life-cycle events, onStart and onBeforeStart that can be hooked into when setting up your application. This sense of cohesiveness in a set of components that share conventions makes Marionette easy to learn without obscuring what happens with the type of magic you see in other frameworks.

Take What You Need

Like everything else in Marionette and Backbone, life cycles are a pattern that you can choose to use or ignore. If you want to define your own way of managing a Views life cycle, it’s perfectly possible to do so. But the advantages provided by Marionette’s implementation are quite nice. It presents a simple mental model of what occurs to a View over time, fits in with the other patterns provided by Marionette, and doesn’t require any extra boilerplate when you choose not to use them. I personally use some of these methods a lot, and have never used a few of them at all. But it’s helpful to know what is available so that you can keep that simple model of how a View changes over time and choose the appropriate hook for any given View code. Code is scalable when you can take a few simple concepts and use them repeatedly to build up to something big. That’s what Marionette can do for you.

Marionette also exposes those options later using the view.getOption() method
↩
Marionette cleans up events created through its event hash objects, and any defined using listenTo. But any custom eventing stuff you do will need to be cleaned up directly
↩

Building Complex Layouts With Marionette.js

Mon, 22 Dec 2014 00:43:47 -0400

This is the third post in a series on Marionette.js. For more background on what Marionette is, check out the first post in the series, The Case For Marionette Also note that this article was written about Marionette v2. The current latest version of Marionette is v3, and while most of the high level concepts remain the same, the specific syntax has changed in some cases.

When building complex web applications, many of the first decisions you have to make revolve around the structure and layout of your application. Most applications will have some static portions like navigations or footers, and some dynamic portions holding content or page specific controls. A good layout manager will allow you to easily control these layouts, helping you keep structure consistent between pages and actions while swapping out content on changes in routes or data.

Backbone provides Views as a way of managing the display of a web app’s UI. But it doesn’t provide any guidance on how you should use them to construct complex layouts. Should you have one view per page and control the layout with templates and jQuery? Use one View for each section of the page? Or have a top level view that creates child views to manage different sections? All of these are possible with Backbone, and none of them are particularly encouraged or discouraged by the library design. In fact because Backbone doesn’t provide any consistent conventions for managing subviews, rendering content, or attaching rendered content to the DOM, even projects that take the same approach to managing layout might implement these patterns quite differently.

In my first post, I talked about how one of the strengths of Marionette is the way it helps remove many of the decisions that Backbone developers have to make for each project by providing components based on community-tested patterns. Layout is another area where Marionette can give you a standardized solution. Marionette does this by providing Regions and by making it easy to manage child Views.

Layout in Marionette

Regions are the primary tool Marionette provides for defining layouts. A Region is a defined portion of the page, held within a root DOM element. You can swap different Views in and out of Regions, or show one View and let it stay there permanently. Like everything in Backbone applications, Regions emit events, which allow you to design functionality that is triggered before or after a region’s content changes. Regions map very well to low fidelity layout wireframes like the one below, but they also can be used for organizing smaller components. They’re an organization tool for attaching and removing Views in a consistent way.

Regions really shine when it comes to child Views. When you’re setting up Views in a normal Backbone application, you have to be very careful to avoid memory leaks by replacing a view without properly destroying it, especially for long-running SPA style apps. If you’re properly cleaning up Views when they’re destroyed, another thing that Marionette can help with, Regions remove that pain by ensuring that a View is properly destroyed if replaced in a Region, removing boilerplate and the opportunity for subtle memory bugs. They also provide a clean central way to attach child Views, making it easier to maintain consistency across a project and promoting readability when moving into different code.

LayoutView

Regions are great, but for the most part you’re not going to want to use them directly. Instead, Marionette provides a special View class, LayoutView. LayoutViews allow you to build View tree hierarchies. You can have one top level View that controls the entire page, or a section of the page that your Marionette app lives in. That root view can then contain Regions which you use to show child Views. Some of those child Views may themselves be LayoutViews, and so on for as many levels as you like. This setup allows you to create modular View classes with few responsibilities that you can easily replace or update without having to write a lot of boilerplate code.

That’s the high level overview, but let’s dig into what the code looks like. To use regions in a LayoutView, we don’t have to directly create them. Instead we define a regions object that works similarly to the events object on normal Backbone Views. The regions object takes a Region name as a key and a jQuery selector as a value, and then create Region objects for each key-value pair and attach them to the View. If you need to do something more complicated you can create a regions function that returns an object with the key-value pairs that you would like. Here’s an example of what a LayoutView might look like in practice.

var Mn = require('backbone.marionette');
var Radio = require('backbone.radio');
var FooterView = require('views/footer');
var HeaderView = require('views/header');
var IndexView = require('views/index');

var RootView = Mn.LayoutView.extend({

  regions: {
    header: '#navbar',
    content: '.content-area',
    footer: 'footer'
  },

  initialize: function() {
    Radio.channel('root').comply('set:content',function(contentView) {
        this.getRegion('content').show(contentView);
    });
  },

  onBeforeShow: function() {
    this.getRegion('header').show(new HeaderView());
    this.getRegion('footer').show(new FooterView());
    this.getRegion('content').show(new IndexView());
  }

});

This is an example of a RootView that could be shown in a standalone Region and used as the top-level of a View hierarchy. There are a few things to note about this example. It defines 3 regions, a header and footer that render static Views, and a content area that originally contains an index View. The initialize function uses Backbone.Radio to listen for commands and can swap out the View held in the content region when commanded. Those views could be swapped based on users navigating to a new route, with the routes initializing the Views and passing the commands, or based on a data change of some type. We can also see that the child Views are shown within the onBeforeShow callback. I’ll be discussing View life-cycle methods more in my next post, but for now just know that onBeforeShow runs immediately before a View is shown in a region. You can use it when nesting your Views in order to make sure your Views are only attached to the DOM once, which prevents the browser from having to repaint the window multiple times.

It’s Turtles All The Way Down

The great thing about Marionette’s View system is that you can take the 3 Views I’ve gone over in my last 2 posts, ItemView, CollectionView, and LayoutView, and use them to represent pretty much any layout you’d like. The basic rules: LayoutViews are used to structure the page into regions and regions into subregions. ItemViews are used to bind data to a UI, and CollectionViews are used to iterate over a Collection when each item in a Collection has complex data or behaviors associated with it and should be bound to an individual View or View tree¹. This consistency makes it easy to learn how to structure Marionette apps, and makes it simple for experienced Marionette developers to jump into a new project quickly and get their bearings. That’s really the promise of Marionette in a nutshell: consistent patterns that allow for a better development experience.

More Resources

For more on nesting Marionette Views, you can check out this talk by Marionette core team member Sam Saccone. It’s a quick 10 minute video covering much of the same stuff I went through here.
I’ll talk more about this in a future post, but if you have a Marionette app and want to visualize your View hierarchy, Marionette Inspector is a great new tool for debugging Marionette apps, and it’s primary developer, Jason Lastner gave a great talk describing how to use it at BackboneConf this week.

Marionette also contains a 4th View type, CompositeView. But it's essentially a helper for the case where you want to display a CollectionView inside a region of a LayoutView, and isn't strictly necessary. Technically you can also use LayoutViews to replace ItemViews, but the semantic difference is useful for encouraging readability.
↩

Come Build With Me

Thu, 18 Dec 2014 10:00:00 -0400

One of the blessings of starting this blog has been the opportunity to answer questions, debate, and learn from the amazingly knowledgeable people who have read my posts. I’ve talked with so many developers who have incredible enthusiasm for software development and the web platform. If that’s you, I have a cool opportunity I want to share.

I try to stay on-topic on this blog and don’t talk about myself very much, but last winter I joined Windsor Circle, a tech startup in Durham North Carolina. We’re a software as a service company focused on helping marketers retain their customers. Since joining, it’s been a crazy ride. Last April, we were selected as the winner of Google’s “Tech Hub Demo Day”, a competition where Google brought in 10 leading startups from around the country and gave them the opportunity to pitch to an auditorium full of Silicon Valley venture capitalists.

This summer we followed that up by raising 6.5 million in Series B funding, and we’re now growing rapidly. Part of that growth is building out our front end development team, and we have a Front End Developer position that is open now.

I want to work with great people, so here’s my no BS pitch to any JavaScript developers out there.

Come live (stay?) in Durham NC, get great weather, big city resources for small city cost of living, and amazing food ¹.
Work with a small but growing development team that has a good mix of senior and less experienced developers.
Help build and shape an ambitious web app built with Backbone and Marionette.js on top of a Python backend
Work with a team that has a lot of fun while taking care of their work like professionals and places a genuine emphasis on the company’s values.
You can even get paid for all of this.
Finally, if you’re interested, we’re open to a mix of experience levels. The main thing we want to see is JavaScript and programming knowledge that matches up to where you are right now, and an enthusiasm for building things on the web.

In all seriousness, if this sounds interesting to you, feel free to get in touch by email or on Twitter. You can also see the full job description and apply directly on jobvite.

Durham's been getting national attention for its food scene for several years now.
↩

Marionette Explained: Connecting Data to Your Views

Wed, 10 Dec 2014 08:30:00 -0400

This is the second post in a series on Marionette.js. For more background on what Marionette is, check out the first post in the series, The Case For Marionette. Also note that this article was written about Marionette v2. The current latest version of Marionette is v3, and while most of the high level concepts remain the same, the specific syntax has changed in some cases.

One of the most interesting design choices of Backbone was the decision to not provide a default render function for Views. Backbone Views by default are an empty function that adds nothing to the DOM when rendered. Although a View can have a Model or Collection associated with it, there is no built in way to associate that with the generated View. When you contrast that to other modern frameworks like Ember, Angular and KnockoutJS, and their emphasis on using 2 way bindings to associate a data object to a rendered View, Backbone can feel a bit incomplete. The goal is to allow for a variety of rendering approaches, without making one of them a favored approach. But it leaves a situation where choosing a rendering procedure is one of the first steps in any new Backbone project. Marionette provides a structured approach for binding your data to your view layer, while still providing flexibility to introduce customization.

Rendering and Data-Binding in Marionette

Marionette.JS handles rendering in a way that is similar to the Model-View-ViewModel pattern that developers who have used KnockoutJS or Durandal will be familiar with ¹. Marionette implements its render cycle through 2 primary methods available in all of its base View classes. The first is serializeData. serializeData is a function that produces what is essentially a ViewModel in the sense of an MVVM application. It takes your data (a Backbone Model or Collection), and pulls it into a simplified form that a View can easily render. By default Marionette will simply call the Model or Collection’s toJSON function to serialize the Backbone object down into a simple JavaScript object. But it’s possible to override serializeData to provide more complicated logic. For instance here’s an example from the Marionette TodoMVC project that takes a collection of todos and calculates a few values to show the number of completed todos.

serializeData: function () {
    var active = this.collection.getActive().length;
    var total = this.collection.length;

    return {
        activeCount: active,
        totalCount: total,
        completedCount: total - active
    };
},

Once the serialized data/view model has been prepared, data is passed to the render function which starts by looking for a template to render. In Marionette, templates are defined by setting the template property on your View, or by setting a getTemplate method if you want to conditionally load your templates. The render function takes that template, passes it the serialized data, and renders it using a template engine. The default Marionette template engine is Underscore, which makes sense since it’s a Backbone dependency already. But it’s easy to switch to a different templating engine. Here’s the entire chunk of code I wrote to use Handlebar templates in Windsor Circle’s Marionette app.

//render templates with Handlebars instead of Underscore
Backbone.Marionette.Renderer.render = function(template, data){
    var renderer = Handlebars.compile(template);
    return renderer(data);
};

ItemView and CollectionView

Marionette provides several View classes that you can use as bases, but the core Views for representing data are ItemView and CollectionView. An ItemView takes a single data object, either a Model or Collection and passes it to the template to render. A CollectionView iterates over a Collection and renders a child View for each Model. You can either use the same View for every Collection, or mix and match Views based on the Model’s properties.

So for instance, views for a simple todo list might look something like this ²:

var TodoView= Marionette.ItemView.extend({
    tagName: 'li',
    template: '<span class="checkbox" {{checked}}></span><span class="text">{{ text}}</span><span class="delete"></span>',
    events: {
        'click .checkbox': 'toggleChecked',
        'click .delete' : 'deleteItem'
    },

    serializeData: function() {
        return {
            checked: this.model.get('checked') ? 'checked':'',
            text: this.model.get('text')
        };
    },

    toggleChecked: function() {
        //logic for checking the box
    },

    deleteItem : function () {
        //logic for deleting the todo
    }
});

var TodoListView = Marionette.CollectionView.extend({
    childView: TodoView,
    tagName: 'ul'

});

When the TodoListView is given a collection and rendered, it will display an unordered list of rendered TodoViews, with the data for each model bound to the template object associated with TodoView.

A brief note on 2 way data-binding

Marionette data-binding is one-way by default. Over the last few years the discussion around data-binding has gone from “cool there’s structured data-binding” when Backbone first came out, to “how can you ship without 2-way data-bindings?” when Knockout, Angular and Ember popularized the 2-way approach, to “1 way bindings keep things simple and they’re cool again” based on the popularity of React. I’m not going to go deep into that debate during this post, but the important takeaway is that Marionette generally prefers explicit bindings to implicit. So when model values change, it will be based on code that you’ve written, not implicit library code.

Wrap-Up

I kept things pretty simple in this post, but consistent data binding is a core part of what Marionette provides. Having a single documented system for attaching Model and Collection data to Views is a big win for keeping Backbone applications lean and readable. When using these conventions you can move from View to View or project to project and quickly pick up on the intent and structure of the code. Instead of trying to solve these problems for every View, you can rely on the experience of others and focus on writing code to achieve your application’s goals.

This was a short but important start to diving deeper into Marionette. I’ll talk more about layouts, view life-cycles, and other Marionette features in future posts, but at their heart, web applications are about exposing data in rich UIs. Without that core, your web applications won’t have much function at all. So make sure to understand and take advantage of this aspect of Marionette before you move on to anything else.

More Resources

If you absolutely must have 2 way data bindings in Backbone, Backbone.stickit is a good project to check out. At Windsor Circle we use it for binding model data to forms (2 way data-binding’s best use case). Rather than the template based approach most libaries use for 2 way data-binding, stickit allows you to configure bindings declaratively in your view code.

MVVM is normally associated with 2 ways bindings though, which Backbone and Marionette do not provide
↩
In practice Marionette developers would likely use a ui hash on the TodoView, pull the template out into a separate file, and might have the CollectionView be a CompositeView to provide structure around the list. But I'll dive into all of those things in more detail in future posts.
↩

The Case For Marionette.js

Mon, 01 Dec 2014 21:53:52 -0400

Building large web applications using Backbone.js can be hard. Backbone is a great tool, but it’s designed to be minimalist and useful in a wide variety of situations. As a result, you get less guidance and support from the tool as you scale up than you do from more opinionated frameworks like Angular and Ember. When a Backbone application grows, maintaining it requires adding structure, either through a custom set of conventions and components, or based on somebody elses framework. There are a lot of different Backbone frameworks out there, but I want to make the case for using Marionette.js.

I’ve been using Marionette for the past year, and it’s been incredibly valuable. It’s really helped ease a lot of the initial issues I found with scaling out Backbone. But there was one big issue with Marionette when I started with it. It was not simple to figure out how exactly I should be using it.

When I start to use a new piece of software, I’m always hoping to see 2 types of documentation. The first is the nitty-gritty stuff that most people associate with documentation. What are the APIs available, what can you pass to them, what components, features, or functions does it provide? The second type is what I think of as the “story telling” documentation. It overlaps a bit with marketing, but basically it’s the documentation that explains the value of the software, gives you ideas on how you might use it, and outlines the philosophy that lead to the tool. Without the first type of documentation you end up with an exciting idea that you may struggle to implement. Without the second type, you can end up with a lot of nice tools that you aren’t sure how to combine together.

Marionette is great, but it’s a tool that doesn’t have much of the story telling documentation I mentioned above. It gets off to a good start, with the following intro on its website:

Backbone.Marionette is a composite application library for Backbone.js that aims to simplify the construction of large scale JavaScript applications.

It is a Collection of common design and implementation patterns found in the applications that we have been building with Backbone, and includes pieces inspired by composite application architectures, event-driven architectures, messaging architectures, and more.

But the site fails to follow through on explaining the values or philosophy that went into building it. Instead, when you start with Marionette you’re left with a bunch of components that are well documented individually, without much guidance on how to put them together or use them. Any explanation of the value added is scoped to an individual component in the docs, and there’s no real indication of why these particular components are bundled together. I’m going to try and provide this type of high level explanation of the value that Marionette can bring, laying out the big picture in this piece and then digging into each area of value in future posts. To start, let’s look at the problems that Marionette is trying to solve.

Decisions, Decisions

Developing with Backbone is an exercise in decision making. Backbone provides you with a minimalist set of Models and Collections that essentially serve as light wrappers around JavaScript objects synced over Ajax. It provides you lightweight Views that associate an object with a DOM node and some data. It provides a router that associates URLs with function, and it provides helpers for managing events between all of these options. That leaves Backbone developers with many questions to answer.

How do you render Views? - By default, Backbone’s render method does nothing. To use it, you need to fill in your own rendering function. That could use a templating system like Underscore templates or Handlebars, jQuery manipulation of the DOM, or simple string inserts with .innerHTML(). You could use the same method for every View, or mix it up and use different methods for different Views.
How do you manage relationships between objects? - By default Backbone provides a way to manage sets of Models as a Collection, but it doesn’t have any built-in utilities for handling nested Models or Collections. And if you want to nest your Views you’re completely on your own. You can have a View manage it’s child Views, have a single object that manages all Views, or let each View manage itself.
How do your Views communicate between each other? - Views will often need to communicate with each other. If for instance one View needs to change the contents of another area of the page, it could do so directly through jQuery, could get a direct reference to a View managing that area and call a function on it, change a Model that another View listens to, adjust a URL that a router listens to, or fire an event that another View could respond to. Apps can use some combination of all of these methods.
How do you avoid repeating yourself? - If you’re not careful, Backbone can involve a lot of boilerplate. Taking the naive approach, you could end up writing rendering code, View management code and event management code over and over again in every View. If you try to get around that using inheritance, you can end up with brittle designs that require you to make calls down to a Views prototype when you want View specific code. Avoiding that type of repetition and the maintenance overhead it brings is a challenge.
How do you manage a View’s life-cycle?? - What code is responsible for rendering a View? Does it render itself on creation? Or is it the responsibility of the object creating it? Does it get attached to the DOM immediately on render? Or is that a separate step? When the View is removed from the DOM or deleted, how do you handle any cleanup that is needed?
How do you structure your application? - How do you get your app started? Do you have a central object that starts everything, or is it more distributed? If you do centralize, do you use the router to start things, or provide some other object for managing your code?
How do you prevent memory leaks? - If your application is a Single Page Application or it contains long lasting interactive sections, another issue that you may need to deal with is memory leaks. It can be easy to create “zombie Views” in Backbone if you’re not attentive to the need to unregister events attached to a View after you’re done with it.

That’s just a small sample of the type of decision making that you have to make for a Backbone project. Those questions signify flexibility, but they also represent mental overhead. If you’re like me, you see these common problems and think that you can get better results relying on a shared solution that leverages the experience of the community.

What does Marionette Give You?

Marionette is an attempt to provide this type of shared solution, capturing Backbone best practices as a set of components and design patterns. So what value does it provide? Marionette gives you:

A Standardized Rendering process - Marionette takes an opinionated stand on how Views should be rendered. Without any additional configuration, it will take a template that you specify with a View’s template property, compile it with Underscore’s templating function and pass it a model or collection. If you need to pass it other data, or want to use a different template library, Marionette provides hooks to customize that process in a DRY way.
A consistent View lifecycle - Marionette defines a consistent View life cycle where Views are initialized, rendered, shown, refreshed, and destroyed. Each of these events has events and callbacks associated it, and any common boilerplate associated with them is handled behind the scenes.
The ability to define and manage complex layouts - Marionette provides region objects that define portions of the DOM that can display and swap out Views. Combined with utilities to manage child views, you can easily create deeply nested View structures with Marionette while minimizing complexity.
A central event bus with semantic events to simplify communication between Views - Marionette includes Backbone.Wreqr or Backbone Radio as an event bus to allow communication between Views without explicitly coupling them.
Helpers to help you write DRY Code - In addition to centralizing the rendering and view management code, Marionette provides hooks to allow you to abstract away details of the DOM and events in your View code, and a mechanism to pull common ui operations out into separate reusable objects
Helpers to avoid “Zombie Views” and memory leaks - Marionette’s lifecycle includes an explicit destroy phase that cleans up many common sources of memory leaks, and provides a hook for you to take care of the rest
A central Application object to initialize your application - Using Marionette, you’re able to specify a set of initializers that run any code that needs to be executed before your application starts, providing a clear structure and starting point to your app.

That’s not the complete feature set, but it is the essential sales pitch. I’ll be digging deeper into each of these advantages over my next few posts, but the important thing to understand is that Marionette provides a framework for building Backbone apps that builds on established practices from the community. If you’re building a Backbone application and want to focus on the problems that are specific to your application, Marionette is a great way to move past common issues and focus on what’s unique to you.

More Resources

For a practical introduction to Marionette, you could do much worse than this talk from the recent Nodevember conference. Jeremy Fairbank, a member of the Marionette core team, explained how you can use Marionette to improve the architecture of your Backbone apps.
The resource that best helped me understand Marionette isn’t technically about Marionette at all. Derek Bailey’s book Building Backbone Plugins lays out the philosophy and thinking behind Marionette without directly referencing it, explaining the challenges that he faced in building a useful abstraction over Backbone. You can read my full review if you’re interested, but I highly recommend it for any Backbone developer as a way of better understanding how to structure your Backbone applications.

Alternative JavaScript

Sun, 23 Nov 2014 23:58:00 -0400

It’s become a bit of a cliche that JavaScript is the “machine code of the web”. But now that mobile browsers have conclusively killed Flash, Silverlight and other “plugin languages”, JavaScript has become the only language that can reasonably claim to be “write once, run anywhere”. That’s still a bit inflated though. More accurately, you can use a subset of modern JavaScript that is compatible with the set of browsers and JavaScript environments that you want to support and run it in those places. In practice this ends up being a version of JavaScript that matches the EcmaScript 5 spec ¹ for users who want to support modern browsers, or a very limited subset of the language that is somewhat similar to the ES3 spec for developers supporting old versions of Internet Explorer. Because of the strange dynamic that the user rather than the developer of JavaScript code controls the language runtime that code uses, the JavaScript language has been forced to evolve slowly, with very few developers using the latest language features at any given time. This, combined with a dislike of JavaScript from some developers has led to a proliferation of “compile to JavaScript” languages.

There’s a problem with talking about “compile to JS” languages though. That category groups 2 types of languages that are not especially similar. The first type are languages that are completely different than JavaScript, which are then compiled down into JavaScript so that they can run in a web browser. This is where the term “machine code of the web” really applies. From the developer’s point of view, the fact that the code is compiled to JS is an implementation detail, and the JavaScript that is generated is likely to be unreadable, without any clear relation to the structure of the original code. Examples of this type of “compile to JS” language include new languages created for this purpose like Dart and Haxe, as well as existing languages like Clojure and Java. The most extreme example of this has probably been ASM.js and emscripten, which together allow developers to compile C++ code to a subset of JavaScript that browsers can optimize for performance. This type of language is great for developers who have a strong preference in language style, or an existing codebase that they want to port to the web.

But there’s a second type of “compile to Javascript” language. These are languages that embrace the structure and semantics of JavaScript and focus on providing an alternative syntax that provides some benefit. Benefits can include better code readability, cleaner syntax for common operations, and support for modern JavaScript features that are not yet safe to use in common browsers without a compile step. These languages will generally compile down to readable ES5 JavaScript that can easily be matched up 1 to 1 with the original source code. They’re not intended to excuse you from learning JavaScript, instead they ’re meant to smooth over the development process as you build JavaScript applications. Examples of this second type of language include CoffeeScript, TypeScript, JSX, Sweet.js, and Traceur. In this post I’m going to take a look at the strengths and weaknesses of some of these alternative Javascript syntaxes, and thinking through when it’s a good idea to use them.

I’ll start by standing up for the much maligned vanilla ES5 JavaScript syntax. It has advantages. It’s supported in many environments these days, including most browsers that people actually still use. It has an incredible stable of tooling built up around it that allows for a smooth development experience. And you can use it without jumping through any compile step hoops, or having to mentally translate the production code back to the source. I’d argue that JavaScript sets a high bar for an alternate syntax to meet. Whether you agree or disagree with that, let’s at least admit that there is a bar and that the convenience of broad support and 1-1 mapping is an advantage that any alternative has to overcome.

So what type of advantages can alternatives offer? The different syntaxes I go through below all provide different strengths, but one mostly shared strength is that they allow you to start using some features of the newest version of JavaScript, EcmaScript 6. ES6 brings a bunch of new features that make common development patterns easier, and looks to be a great improvement to the language. By allowing you to take advantage of those features now, alternative syntaxes can speed up development and ease the transition into the new version of the language. To get an idea of the more specific benefits an alternate syntax can bring, I’m going to go through 4 of the most popular syntaxes and explain why somebody would want to use them.

CoffeeScript was the first alternative JavaScript syntax to gain popularity, and it is also one of the biggest breaks from the normal syntax. Rewriting a simple Backbone view from the TodoMVC project in CoffeeScript would look something like this.

class AppView extends Backbone.View
    el: '#todoapp'
    statsTemplate: _.template $('#stats-template').html()
    events:
        'keypress #new-todo': 'createOnEnter'
        'click #clear-completed': 'clearCompleted'
        'click #toggle-all': 'toggleAllComplete'

    initialize: () =>
        @allCheckbox = @$('#toggle-all')[0]
        @$input = @$('#new-todo')
        @$footer = @$('#footer')
        @$main = @$('#main')
        @$list = $('#todo-list')

        @listenTo(app.todos, 'add', @addOne)
        @listenTo(app.todos, 'reset', @addAll)
        @listenTo(app.todos, 'change:completed', @filterOne)
        @listenTo(app.todos, 'filter', @filterAll)
        @listenTo(app.todos, 'all', @render)
        app.todos.fetch({reset: true})

    render: () =>
        completed = app.todos.completed().length
        remaining = app.todos.remaining().length

        if app.todos.length
            @$main.show()
            @$footer.show()

            @$footer.html @statsTemplate
                completed: completed,
                remaining: remaining

            filter = app.TodoFilter or ''

            @$('#filters li a')
                .removeClass 'selected'
                .filter "[href=\"#/#{filter}\"]"
                .addClass 'selected'
        else
            @$main.hide()
            @$footer.hide()

        @allCheckbox.checked = !remaining

    addOne: (todo) =>
        view = new TodoView
            model: todo
        @$list.append view.render().el

    addAll: () =>
        @$list.html ''
        app.todos.each @addOne, @

   # ... and so on

As you can see, Coffeescript uses Ruby and Python style significant whitespace, which gives it a very different feel than JavaScript. But it retains the language’s structure, giving it a low learning curve. All of CoffeeScript’s built in structures compile down to ES3-compatible code, so anything you write in CoffeeScript will work in any modern JavaScript environment as long as you don’t use any incompatible APIs.

Coffeescript gives you plenty of niceties to help you write faster and more succinct code. In general it’s focused on taking common patterns and finding a short succinct way of representing them. It removes some of the boilerplate from JavaScript code by replacing function(){} and this. with () -> and @ respectively. It’s a small change, but giving these common patterns easily recognizable shorthand saves plenty of characters across a code base. These aren’t the only shorthands. For instance it provides destructuring and string interpolation capabilities, allowing you to do things like this.

a = "example"
b = "This is an #{a}"

[a, b] = [b,a]

# a ends up as "this is an example", b is "example"

It’s easy to see that the EcmaScript 6 spec has been greatly influenced by CoffeeScript. Features like classes, arrow functions, and splats/spreads are all parts of CoffeeScript that are going to be melded into the future of the language. So why use it instead of something like Traceur which allows us to write ES6 code now? The best answer I’ve seen was this short piece by Reg Brathwaite. He argues that CoffeeScript is essentially a tool for writing good JavaScript. The structure of the language encourages you to take advantage of JavaScript’s good parts, and the compilation process is able to encode many JavaScript best practices. CoffeeScript features don’t add new capabilities to JavaScript, instead they encode accepted design patterns.

In terms of tooling, CoffeeScript has been around a while, and has built up a good base of support in the JavaScript community. It’s the recommended JavaScript syntax for Ruby on Rails projects and has wide support in the NodeJS community. This community adoption means that many tools offer integrations for it. You can also find some Coffeescript specific tooling. For instance, there’s a CoffeeLint tool that provides linting similar to JSHint. CoffeeScript still doesn’t have the ecosystem that basic JavaScript enjoys in this area, but it has established itself enough that most common scenarios are addressed.

Why use Coffeescript: You like the transformed syntax, you want concise shorthand for common JavaScript design patterns, and you want to be able to write more readable code ²

TypeScript is a language created by Microsoft to provide static type checking for JavaScript. It allows you to annotate variables with types, then validates your code when it’s compiled into JavaScript. The benefits are pretty obvious. Type checking forces you to think about the values that will get passed into functions and help you catch bad logic at compile time rather than runtime. Static types also allow for smart intellisense-style completion. As a Microsoft creation, this is fully supported in Visual Studio, but it’s also available in other IDES like WebStorm, and there’s some support for making it work in text editors like Vim and Sublime Text as well.

Although TypeScript is primarily about types, it also provides some other niceties that fit with it’s theme of tools for building large JavaScript applications. It provides classes that matches JavaScript’s ES6 class syntax, and modules that can compile down to AMD or CommonJS format. You also can use ES6 lambda functions.

Unlike CoffeeScript, TypeScript is unlikely to shorten your code. Instead writing type definitions will likely lead to a larger code base, but one that’s easier to reason about in a given context.

The TypeScript Hello World program from their site doesn’t look too much different than the original Javascript. The important thing to note is the ability to use ES6 features as well as the type annotations on functions.


class Greeter {
    constructor(public greeting: string) { }
    greet() {
        return "<h1>" + this.greeting + "</h1>";
    }
};
var greeter = new Greeter("Hello, world!");
var str = greeter.greet();
document.body.innerHTML = str;

TypeScript was obviously built to be tool friendly, and if you’re using an IDE like Visual Studio or WebStorm, they’ll be able to take full advantage of that. But the tooling around the text editors used by most JavaScript developers is less advanced. There’s a nice community effort to build a TypeScript server that text editors can plug into for completion information and other IDE functions, similar to the way that TernJS works. But there’s none of the official support that Microsoft has built into Visual Studio, and the lack of attention shows. Still, the compiler itself fills many of the roles that linting tools play for normal JavaScript, and many developers interested in type safety are probably already using an IDE style editor that will provide full support.

Why Use TypeScript: You want static type checking as well as the option to use some ES6 syntax now. Also a great choice if you’re using an IDE and want better “intellisense” while writing code.

Sweet.js is a very different animal than the other syntaxes I’m covering here. Strictly speaking it’s not an alternative syntax at all. It’s a toolkit to building your own your own alternate syntax. You do that by defining macros, which the compiler then loads and uses to transform your code. For instance the example from their website shows how you can define ES6 style classes in your code.

// Define the class macro here...
macro class {

  rule {

    $className {
        constructor $cparams $cbody
        $($mname $mparams $mbody) ...
    }

  } => {

    function $className $cparams $cbody

    $($className.prototype.$mname
      = function $mname $mparams $mbody; ) ...

  }

}

// And now classes are in JavaScript!
class Person {
  constructor(name) {
    this.name = name;
  }

  say(msg) {
    console.log(this.name + " says: " + msg);
  }
}
var bob = new Person("Bob");
bob.say("Macros are sweet!");

Sweet.js basically removes limits in terms of what your JavaScript syntax looks like. If you can programatically define a syntax, you can use it. And because Sweet.js macros are hygienic you can have confidence that you can use them without side effects. This means that you can wipe away boilerplate, hiding it behind a new operator or keyword. The modular nature of macros is also a big plus. Rather than being locked into a syntax and set of capabilities ², using macros allows you to mix and match the syntax you need.

Relative to the other languages on this list though, Sweet.js lacks a lot of the benefits of a community standard. Any macros you write will only be documented to the extent you document them, and while you can run linting tools on the output, pretty much any other tooling out there, including basic stuff like syntax highlighting, is likely to be unable to parse the non-standard syntax created by macros.

Why use Sweet.js: You want your JavaScript syntax to work in a very specific way, and none of the existing options appeal to you, or you find yourself writing a lot of repetitive boilerplate that you can remove with a syntax change.

Traceur

The last alternate syntax that I’m going to be going in depth on is a bit different, because it’s technically just JavaScript syntax. But it’s a form of JavaScript syntax that’s not supported everywhere yet. Traceur is a project by Google that allows you to write code that’s valid in EcmaScript 6, and transpile it down into browser-compatible ES5 JavaScript. Effectively it’s tomorrow’s JavaScript a bit early. This has some obvious advantages. First, it will eventually obsolete the compile step as all of the features of ES6 will be supported in browsers someday, and the changing world of browser release policies seem likely to make that sooner than you might expect. Secondly, tools and other support around ES6 is shaping up nicely, and it’s guaranteed to get better, because this is just JavaScript. It’s the future of the web platform.

The case against using Traceur? Besides the repeated themes of weaker tooling and an additional build step, Traceur simply doesn’t offer as much as the other syntaxes mentioned here. It doesn’t significantly change the feel of the language like CoffeeScript, provide any clear new features like Typescript, or offer the flexibility of Sweet.js. In the end it’s “just Javascript”, arriving a little bit early. The question you need to answer is whether that’s enough to justify the compile step.

Why use Traceur: You like JavaScript but are frustrated by slow standards adoption. You want to use the newest features of the language now while still supporting a variety of platforms.

Update: Since I originally wrote this post, Babel (formally 6to5) has exploded in usage and popularity as a tool for using future JavaScript features today. There are very good reasons for this. It is much better documented than Traceur, supports more features, and generally aims to be a tool that people are using to build things, whereas Traceur has always felt like more of a tool for learning about the future of JavaScript. I would highly recommend Babel for anyone looking to use future JavaScript features in their web projects.

Other Options

I chose to highlight the 4 syntaxes above because they’re the most popular general purpose alternate syntaxes for JavaScript. But there are a few others worth mentioning here, for the sake of completeness if nothing else.

JSX is seemingly becoming quite popular as an extension to the JavaScript syntax, but I chose not to highlight it here because it is currently so tightly knit to React. While it is technically possible to use JSX for things other than React apps, in practice JSX has a limited enough scope ³ that there’s no real motivation to do so. If you’re building a React application though, JSX is certainly worth exploring, as is the next syntax.

Flow is a new JavaScript syntax created by Facebook that uses the same type annotations as TypeScript, but does so in a way that aims to provide better type inference as well as supporting JSX. I’m very interested to see where it goes, but for now it’s still early days as it was only announced last week.

AtScript also uses TypeScript’s type annotations, is mostly tied to a single framework, and in terms of public usage doesn’t really even exist yet ⁴. So why is it worth a mention? Because it is the syntax that AngularJS 2.0 is being developed in, and will be the first class “recommended syntax” for that framework when it is released next year. Since Angular is currently the most popular JavaScript MV* framework, that carries some weight and makes AtScript worth tracking, especially for Angular developers. But in today’s world it’s not a real contender to be used in a project.

Decisions

So how do you decide what JavaScript syntax to use? My recommendation is to start by assuming ES5 JavaScript as a default. Then examine any problems you’ve run into while using it. Does your team struggle with JavaScript gotchas? Has your project become large enough that a strong type system would provide more benefits than cost? Do you really need some domain specific syntax to clean up your code? Or are you happy and productive as is? At that point you can decide whether a different syntax could help you get around those problems, or if the hassle of compilation steps and overhead of a new syntax would just end up making things worse. You don’t want to use CoffeeScript or Traceur because they’re cool or different. You’ll want to use them if they solve your problems.

More Resources

I linked to Reg Brathwaite’s CoffeeScript is not a language worth learning above, but I want to reiterate that it’s worth a read for anyone who’s interested in CoffeeScript, or who wants to understand why people would be excited to use it.
Steve Sanderson’s vide on Architecting large Single Page Applications with Knockout.js covers a lot of stuff and it’s worth watching the whole thing. But if you skip to the 49:30 mark, he has a nice 10 minute discussion of TypeScript, his teams experience building a large application in it, and the pros and cons they found.
I’m admittedly a bit skeptical about the value of Sweet.JS, but if you want to hear the case for it from somebody who really believes in it, take a look at this essay: Stop Writing JavaScript Compilers! Make Macros Instead. It does a great job of laying out the case for composable macros as an alternative to “monolithic” compilers.
Finally, Axel Rauschmayer has a great roundup of the various languages looking to add type systems to JavaScript, diving pretty deep into each and clearly explaining the differences between TypeScript, Flow and AtScript.

EcmaScript is the "official" name for the JavaScript language as described in its specification. EcmaScript 5 was the most recent finalized spec, published in December 2009. In practice though, JavaScript environements are never strict implementations of a specific spec. Instead they gradually bring in features over time alongside the standards process.
↩
Yes readable is a "beauty is in the eye of the beholder thing". But readability is certainly a reason that many people are interested in CoffeeScript, and many of it's operators are designed as meaningful shorthand for operations that are more bloated in JavaScript. While that may present an initial readability barrier, in the long term it can make for a more to the point code base.
↩
For instance it's taken CoffeeScript a looong time to implement support for generators.
↩
Effectively all JSX does is allow for a convenient representation of the DOM within JavaScript. While this is very important for React, and may be useful in some other cases, it does not provide the level of general purpose value of any of the other syntaxes mentioned in this article.
↩
You can use it and get it to compile by passing an option to Traceur, but as far as I know nobody is actually doing so yet outside of the Angular 2.0 team.
↩

Underscore vs Lo-Dash

Wed, 12 Nov 2014 09:00:00 -0400

Last week was the time for my team’s quarterly “dependency update” where we go through all the third party libraries we’re using and keep up on any updates or maintenance that need to happen. That presented the opportunity for me to do a bit more research into an alternative for one of our libraries, Underscore. Underscore is a JavaScript utility library that provides helper functions for common idioms with a focus on functional programming. It’s a dependency for Backbone, the MVC library that our web app is based on, and was originally created by Jeremy Ashkenas, who also wrote Backbone. I love the functional style that Underscore subscribes to, and while the much improved browser support for ES5 functions like map, reduce and filter makes it less necessary than it used to be, Underscore still provides plenty of benefits.

For a while now though, I’ve been hearing about Lo-Dash. Lo-Dash began its life as a fork of Underscore, and still tracks Underscore’s API enough to allow it to serve as a drop-in replacement. But under the hood it’s been completely rewritten, and it’s also added a number of features and functions that Underscore does not provide. About a year ago I saw a spurt of articles recommending that Underscore users switch to Lo-Dash, and it came up again recently when Underscore’s decision to not follow semantic versioning caused problems for some NodeJS applications. The definitive Stack Overflow question on the topic seems to come down pretty solidly in favor of Lo-Dash. And the Lo-Dash documentation touted a prettier chaining syntax as a feature, something that caught my eye since I’ve always disliked Underscore’s chaining mechanism. As I did my research though I noticed that most of the comparisons are at least a year old. Both libraries have continued to evolve and improve since then, so I thought there’d be value in adding another writeup comparing the current iterations.

Why Do We Even Care?

Before I get into the weeds of comparisons, let me address why you’d want to use either of these libraries at all. Underscore and Lo-Dash provide cross-browser compatible functional programming utility methods. This post will be long enough without me trying to explain all the benefits of functional programming, but suffice it to say that if you’re manipulating and transforming data in any reasonably complex way, an Underscore style library will help you write performant readable code easier than you could with what Javascript natively provides. We’ll touch more on performance later, but both Underscore and Lo-Dash perform better than native browser functional methods, and for complex data manipulations they’re significantly more readable than a series of for loops. They let you write code that’s fast and reads like a description of what you intend to have happen.

How are Underscore and Lo-Dash similar?

As stated above, Underscore and Lo-Dash provide similar functionality. Lo-Dash’s API is a superset of Underscore’s. It provides everything that Underscore does, along with a few additional helper functions. Lo-Dash can generally serve as a drop-in replacement for Underscore with no changes; it works just fine with Backbone. So to some extent this question could be answered simply with a link to Underscore’s documentation.

How are Underscore and Lo-Dash different?

This question could really be rephrased as “What has Lo-Dash added?”. Lo-Dash answers that with a pretty extensive list on its site, but the claims really boils down to a few things:

Usability Improvements
Extra Functionality
Performance Gains

Let’s examine those one by one.

Usability

Lo-Dash makes a number of additions to Underscore that don’t add new capabilities, but increase usability for developers. These include

~~Built in Support for AMD Module Loaders~~
Short hand syntaxes for chaining
Custom Builds to only use what you need
Semantic versioning and 100% code coverage

Each of these are relatively small things that add up to time savings for developers. ~~Supporting AMD natively saves a configuration step, allowing you to use Lo-Dash on requirejs-based projects without having to specify it as a shim.~~ (Update: As of 1.6, Underscore also has native AMD support.) Lo-Dash also improves the syntax for chaining functions. Underscore’s chaining syntax has always felt a bit clunky to me.

function getLongNames(users) {
    return _.chain(users).pluck('name').filter(function(name){
      return  name.length > 10;
    }).value();
}

Lo-Dash cleans that up a bit by not requiring the chain functionality and instead making the Lo-Dash object a function, jQuery style:

function getLongNames(users) {
    return _(users).pluck('name').filter(function(name){
      return  name.length > 10;
    }).value();
}

It’s a small nicety that clears away some syntactical noise.

Lo-Dash also provides custom builds. By default Lo-Dash is significantly larger than Underscore. The full library weighs in at 33KB minified, double Underscore’s 16KB size¹. But you can negate that a bit by only pulling in the things you need. The “modern version” that drops compatibility for IE8 and other older browsers weighs in at 29KB minified (but not gzipped). If you want to drop the Lo-Dash extra functionality entirely, and only use it for performance and usability reasons, you can build an underscore compatible version and use only 20kb. You can also go even further, and for instance use only the functionality required by Backbone. That will get you down to 13kb, smaller than the original Underscore library.

Finally, Lo-Dash provides semantic versioning and 100% code coverage. To me these mostly serve as signals that this is a project interested in being disciplined about its engineering practices, which is a good sign. But the semantic versioning issue seems to be particularly important for those using a utility library in Node, as Underscore’s versioning practices have caused problems for some.

Extra Functionality

Lo-Dash includes several functions not included in Underscore. Some of these are simply syntactical shortcuts for common idioms. _.forOwn allows you to iterate over an objects own properties, allowing you to skip the common practice of including a guard function in a loop to avoid touching items on an objects prototype chain. Similarly _.create is a simplified syntax for Object.create and _.parseInt is a safe version of the built-in parseInt function which can have surprising behavior. Lo-Dash also provides helper function for deep cloning, merging and extensions, operations that are not available in most libraries. Specifically _.cloneDeep and _.merge serve as “deep” versions of Underscore’s _.clone and _.extend, and _.where accepts an argument to support deep comparisons. If you are routinely doing comparisons and copying of nested objects for equality, these functions will definitely have their place for you.

Performance

Lo-Dash has an impressive set of benchmarks that you can run on its site which highlight its speed advantage over Underscore on a variety of operations. This is great, but benchmarking against specific tests like this can lead to local optimizations that might not lead to real-world speed. I also don’t have much interest in many of the functions tested; I use them rarely and can handle a speed dip if necessary. I was mostly interested in how Underscore and Lo-Dash compared on their core functional methods. So I set up some jsPerf tests that you can run yourself for _.each, _.contains, _.map, _.reduce, and _.filter. Here’s a simplified version of the results, comparing Underscore and Lo-Dash performance in terms of their improvement over the speed of the browsers' native functions. Results in the chart are averaged across browsers.

Edit: There used to be a chart here. Unfortunately the 3rd party service I used to embed the chart lost my data. A good reminder to own your own content. At this point the exact performance data from 2014 would no longer be representative of the current state of things anyway. The tests above will provide a more accurate look at current performance.

These tests are not incredibly scientific, and the chart especially is oversimplified, but they’re meant to represent real world usage in an easily digestible format. Draw conclusions from them at your own risk. That said, there were a few interesting results that I think are reasonably reliable.

Both Underscore and Lo-Dash blow away the native functional methods.

With its latest versions Underscore has joined Lo-Dash in using loop based iteration to handle each, map, reduce and filter instead of proxying to the native code. Counterintuitively this ends up being much faster. Consistently across the board, Underscore and Lo-Dash beat native functions in terms of speed. A little research indicates that this is because native functions optimize for sparse arrays and have more weird corner cases that they handle. In any case, the performance difference is pretty startling across the board.

You can still manage to beat the libraries' performance by using simple for loops, which is what underscore and lo-dash use after they handle edge cases of their own. The tradeoff at that point becomes about readability and reliability vs speed, and I’d suggest that using loops is an optimization step in most cases where you’d want to use a utility library, rather than a good starting point.
Lo-Dash is definitely not slower than Underscore

It’s hard to know how significant some of the results here are, as there are a mix of differences which change across browsers. Again, it’s not a very scientific test. But Lo-Dash is never significantly slower than Underscore on pretty much any test (the worst seems to be a ~7% difference on Firefox for _.each()). And for several tests it came across as significantly faster (8x speed difference for contains on Firefox!). I expect many of these differences would prove to be insignificant or inaccurate on a more rigorous test, but it would be very surprising if the net result was that Underscore was faster for common operations.

Summary: What to use?

So to wrap it all up, if you need a utility library which one should you actually use? I’ll stop just short of unreserved recommendations for Lo-Dash that I saw when I researched this. I think Lo-Dash is the clear winner for NodeJS users. Its biggest weakness (file size) doesn’t matter on the server, and its use of semantic versioning matters a lot more for developers depending on npm for dependency management. So take the performance gains and extra capabilities and go with it. However, there are more tradeoffs on the client. Both Lo-Dash and Underscore are small, but optimizing page-weight still matters for many sites, especially on mobile devices. Underscore has learned from Lo-Dash and improved its performance enough that the differences are often insignificant. And while Lo-Dash’s extras are convenient, none of them seem essential unless you are comparing and cloning a lot of deeply nested objects. So I’d advise web developers to weigh the tradeoffs and pick the library that best meets their priorities.

(Update: There’s been plenty of discussion on this post today. Check out the comments below for a lot of helpful insight from John-David Dalton, Lo-Dash’s creator and an Underscore contributor. You can also check out the discussion on Reddit.)

More Resources

Say Hello to Lo-Dash, a blog post written by Lo-Dash’s core team after they released 1.0 is a great summary of the advantages of Lo-Dash. It’s a year and a half old now, but mostly holds up pretty well, with the only real difference being that Underscore has pulled in several of the features they mention here since then.
I mentioned this in the post, but Lo-Dash provides a much more thorough benchmarking page than what I provided on their site. They also provide a helpful roadmap to let users know about plans for Lo-Dash 3.0.

These numbers are minified but not gzipped, based on the file size when I download the minified versions from the project site. ↩

Vim Workflows: File Switching

Mon, 10 Nov 2014 00:06:52 -0400

Becoming productive in Vim is about more than just knowing all of the features available to you¹. It’s all about knowing which ones to use in which situations. In my series from this summer on learning Vim, I focused a lot on the different tools Vim provides for solving problems. Today I’ll show how I put some of those tools together to actually be more productive.

A good tool should allow you to focus on your problem while using it, without making you spend a lot of time thinking about extraneous things. File switching is an area in text editing that can really slow you down if you let it. The first file switching method that most Vim users is the simple :e command to open a file. Technically it’s the only command you need for file switching, since you can use it to open up any file you want and start editing it. But doing so requires you to essentially internalize the complete structure of your file system and type out full paths every time. On a large project with nested directories that becomes a significant mental load and use of time, even if you’re taking advantage of Vim’s tab completion features. And it’s a mental load that’s inessential to the task at hand. You need to recall the precise layout of your file structure every time only because the tool isn’t able to make things easier for you.

Fortunately Vim is able to make easier things easier for you when you use the right tool for the job. That means rather than using one file switching strategy to rule them all, we can combine several tools to reduce the mental load on us and focus on our code or writing. I’ll specifically look at a few common scenarios and my approach for each. This isn’t the only workflow you can use to address these scenarios, but hopefully it will inspire you to improve your own Vim workflow. So here are the different scenarios:

I want to move to a file with a name I know

When you’re moving to a file with a name that you know, it’s hard to beat the power of a fuzzy finding plugin like CtrlP or Unite. CtrlP allows me to type in the name of a file without remembering or typing a long directory path and instantly get a list of potential files. Usually the file I’m looking for is the top hit.

I want to move to a file, but I don’t know what it’s called

This scenario happens most often on projects that you’re unfamiliar with and just getting into. You’re looking for some functionality, but don’t know exactly what it’s called within the project and need to look through the file system for hints. In this case you need a directory browsing tool. Vim bundles netrw by default and it’s a good starting place. If you have issues with netrw, many people install NerdTree as an alternative file explorer. I personally don’t use NerdTree very often these days. Most of my work over the last year or so has been on one large project or smaller experimental projects and I haven’t needed it. But it’s been very useful when I’ve cloned down a github repository to look it over or make changes. It’s the best way to quickly get the lay of the land of a project within Vim, and to find a specific file before you really know what you’re looking for.

I want to move to a file that I already have open

A common scenario in Vim involves moving back and forth between 2 files, possibly a file and its corresponding test file. If you don’t want to use splits, that means you’ll want a quick command to move between open files. Fortunately Vim keeps a list of active buffers open. When I’m only flipping between 2 files I’ll usually use the :bprev and :bnext commands, which I’ve aliased to [b and ]b ². If I have more than a few files open though I’ll usually either use :b and type the name of the file, or just use ctrlp again, which doesn’t require me to remember if the file I wasn’t is actually already open or not.

I want to move back to the place I just was

“Rabbit Holing” happens to me all the time when I’m working on code. I start looking at one thing, then look at code related to that, and before I know it I’m 4 files deep following a chain of stuff I’m debugging. Oftentime I want to move back up that chain. I could of course remember what files I opened and use one of the methods above, but that requires me to internalize the memory of events. The command that maps best to my desire to move back up the chain is using Vim’s jumplist. Vim keeps track of every “jump” you make within a window or split, and allows you to move back up and down that jump list as you like, both within files and across them. The commands to move up and down the jump list are mapped to <c-o> and <c-i> by default. When I use them I don’t have to keep track of what files I’m moving across or even if I’m switching files at all. I can just investigate my issue and have confidence that I’ll be able to get back. Think of it as tying down a rope before you dive into a cave full of snakes.

I want to create a new file

Creating a new file means that we finally find a use for the basic :e command. Because you need to specify a full directory path anyway when you’re creating a file, it makes sense to use the basic Vim commands here. If you need to create a directory to put the file in, you can use the shell command with :!mkdir <path/to/dir>. While writing this piece I learned that ctrlp has the ability to create a new directory path and file in a single command, which is a nice touch. But you lose out on Vim’s autocompletion of paths if you’re creating a directory nested in existing directories. I find the builtins to be simplest here.

I want to edit a specific piece of code, but don’t know what file it’s in

Finally, possibly the most common scenario for me is when I know I have a particular piece of code I need to edit, but don’t have it mapped to a particular file in my head. Rather than forcing myself to recall the file, the easiest path is to let Vim take the information I know and help me out. Specifically I use Ag.vim for code search within projects. It is a wrapper around Ag, a fantastic code search program based on Ack. It allows me to enter a search term, and then pulls up all occurrences of that term within my project. One of it’s best features is filtering by file type, which allows me to type something like :Ag --less "\.widget" and search my LESS files for all rules effecting the class .widget.

Let Vim Work For You

The workflow I described is not the “one true way” to navigate files in Vim. Some people hate plugins and will prefer to use :e,:b, and netrw for everything. You can certainly do many of the things above with NerdTree or CtrlP if you prefer to simplify to one tool. And Unite is an attempt to pull almost all of these scenarios into one tool. The point is to be aware of the inessential work created in the different scenarios you face when coding and offload the work to Vim in a way that makes sense to you.

More Resources

Oil and Vinegar is a nice Vimcasts article on how to make NerdTree work well with multiple splits
I wrote a more introductory post this summer on the tools you can use to manage files in Vim. It talks in a bit more detail about some of the Vim features I mentioned above.

That's true of many things actually.
↩
I actually use unimpared.vim to provide consistent keybindings for these types of motions.
↩

Thoughts on Angular 2.0

Sat, 08 Nov 2014 15:24:09 -0400

Rob Eisenberg has a pretty incredible rundown of the current Angular 2.0 proposal and the thought process behind it on his blog.

Given Angular’s design origins in combination with the changing web and general computing landscape, it’s pretty clear that some changes are needed. In fact, without starting to address these issues now, Angular runs the risk of being obsolete within a year’s time. A framework that cannot work with Web Components, bogs down on mobile or continues to push its own module and class API against the standards, is not going to last long. The Angular team’s answer to these problems is a new version: Angular 2.0. It is essentially a re-imagining of AngularJS for the modern web, taking into account everything that has been learned over the last five years.

He describes in detail the various challenges that the Angular team has faced in keeping the current version modern, the justification for a drastic breaking change, and provides a fascinating look at the inner debates and thought process that produced the current controversial proposal. It’s long, but well worth reading for any Angular developer or anybody interested in the near future changes coming to the web platform. The trends he identifies here are going to effect every library and framework out there, and it will be interesting to see how they respond.

As for Angular’s specific response, it’s a bit hard not to see their approach as throwing their hands up in the air, giving up and building something new, but keeping the name. Rob (a recently added member of the core team) even calls them out on this to a certain extent.

Even though I haven’t got to the details yet, you can probably tell that AngularJS 2.0 is quite different from 1.x. One might ask if it’s even the same framework. I think that’s a good question. As I’ve mentioned earlier, I think the Angular team needs to shoulder the burden of providing a concrete timeline for 1.x support, a migration path to 2.0 and some guidance for companies making decisions today or who would like to plan in advance for 2.0. These are not particularly exciting tasks for the technically-minded Angular team, but I think they are necessary, helpful and respectful to the community.

If you haven’t yet seen the changes coming in Angular 2.0, the tl;dr version is that everything has changed. Syntax is different across the board, not just at a micro level but also at the structural level. Several of the key concepts that Angular developers have had to learn including controllers, scope, and 2 way binding are either definitely gone, still being debated, or changed beyond recognition. And the core concepts of modules, dependency injection and directives have all had major compatibility breaking syntax changes. Keeping the Angular name seems to be more a matter of the new project sharing a general philosophy and set of maintainers with the old project rather than a suggestion that this is simply a large iteration on an existing framework.

A few more assorted thoughts:

I really appreciated the listing of the challenges Angular is facing at the beginning. From my outsider perspective (I’m a web developer but have not spent a lot of time with Angular recently), it seems like they’ve nailed the first step of asking the right questions. Any framework that isn’t optimized for mobile browsers or doesn’t integrate with ES6 classes and modules is likely to look silly in a few years. And while web components are still not a sure thing, they’re rapidly becoming a standard for reusable code on the web.
Creating a custom JS preprocessor to support the framework you’re building is starting to look like a trend. And that is not a good thing. I was not a fan of React bundling JSX, and Angular 2.0’s use of AtScript seems to be equally unfortunate. In both cases the language is not required to use the framework, but instead is treated as a “First Class” language used as an escape hatch to smooth over rough patches in their API. At least JSX is a spec, even if it’s a very React centric one. AtScript appears to be a Google only thing at the moment, inspired by but incompatible with Typescript.
The next time I hear a developer claim that marketing is worthless, I’m going to bring up the last few weeks in the JavaScript framework world. Angular announced their 2.0 plans for the world in a presentation that many people saw only as a set of slides filled with tombstones. By focusing (at least on the slides) on the things being lost and the drastic changes to syntax, the online conversation focused on the compatibility problems with the release, rather than the problems it would solve. Now we finally have a detailed explanation of why these breaking changes are happening, but it’s in the form of an opinionated blog post that emphasizes the current uncertainty around the project. Angular seems to have found an awkward middle ground between involving the community from the beginning and releasing a polished product, where they’ve introduced a “not quite ready” concept to the world.

Contrast that to the EmberJS team, which has pretty transparently taken advantage of the marketing confusion around Angular in order to double down on a message of “stability without stagnation. Their Ember 2.0 RFC is clearly informed by the reception to Angular’s plans, especially the 2nd paragraph.

This is not a big-bang rewrite; we will continue development on the master branch, and roll out changes incrementally on the 1.x release train. The 2.0.0 release will simply remove features that have been deprecated between now and then. Our goal is that you can move your Ember app to 2.0 incrementally, one sprint at a time.

Ember and Angular are facing roughly the same challenges in terms of deprecating old syntax and adapting to changes in the web platform. Regardless of what you think of their different approaches, by clearly articulating their plans in a united way before they begin actually writing the code to accomplish them, Ember is winning the marketing battle here by a mile.
Related to all of the above, it’s hard to recommend starting a new Angular project to anyone right now. While the 1.x branch will continue to receive support going forward if for no other reason than the number of people currently using it, the issues that 2.0 is trying to solve are real and will make it harder for 1.0 to keep up with the rest of the web as it goes forward. This is not a Python 2/3 situation where Python 2 still remains a viable language to start a project with 6 years later. Because the web platform will continue shifting under the code, a language that doesn’t support modern idioms will quickly begin to fall behind alternatives. It’s very possible that Angular 2.0 will regain Angular 1.x’s current spot as the most popular MVC framework on the web, but until it’s released or the core team can show a clear migration path from 1.x to 2.0, Angular development is going to look like a dead end road.

A Quick Review Of Google Inbox

Mon, 03 Nov 2014 23:45:30 -0400

Google announced their new email product, Google Inbox last week, and I was able to get a Day 1 invitation from a friend at Google. I’ve been using it for my personal email account since then. A few people have asked me about it, so I thought I’d write up my impressions so far.

I think it’s important to start by making it clear what Inbox is, because it seems like there’s a lot of confusion out there. Inbox is an alternative view to your existing Gmail account. It is not a new webmail account, and it is not a generic mail client ¹. Changes that are made in Inbox are reflected in Gmail and vice versa. Inbox is a “fresh coat of paint” with some add-on features, not a “whole new thing”. You can access the Inbox interface on the web at inbox.google.com, or through apps for iOS and Android. This review is primarily of the web interface, since I prefer to deal with my work and personal email through a single app on my phone, and that’s not yet possible with Inbox.

Using Inbox for the first time was a little bit of deja vu. Google seems to have borrowed heavily from Mailbox, my current iPhone email app of choice. The interface is heavily simplified compared to the a traditional mail client like Gmail or Outlook. It relies heavily on the “To-Do” list metaphor that several modern clients like Mailbox and Dispatch have popularized. So instead of archiving or deleting messages, you mark them done, and if you aren’t ready to deal with a particular email yet, you can “pin it” in your inbox, or set a reminder to bring it back up later. Other than pinning² , this format maps pretty well to how I deal with email in general, and is almost exactly Mailbox’s workflow, so at least for me it felt natural. Like Mailbox, Inbox is opionionated software. It’s optimized for triaging and processing email quickly. If you spend more time composing emails than reading them, you’ll probably find Inbox lacking in terms of compose tools. But for the majority of us who are just trying to keep our heads above the email ocean, this type of streamlined workflow saves a lot of time.

Inbox’s biggest innovation is its “bundles” concept. Bundles are the next generation of Gmail’s label system, and roughly map to labels when moving back and forth between Gmail and Inbox. They’re a standardized interface for the way I used labels to organize things anyway.

In Gmail, I’ve set up a bunch of rules that apply a label to emails that meet a criteria, then have it skip the inbox. This has the effect of being able to show things like unread updates from a social network as numbers on the left side of the screen, without cluttering my inbox. I like that system in Gmail, but it has always been annoying that I can’t easily see those numbers anywhere except the Gmail web interface. On my phone, those labels/folders are hidden away unless I dig down for them. With Inbox, I can change that rule to just apply the label without moving to the inbox, and then make that label a “bundle”. When I do that, those emails show up as grouped within my inbox, taking up minimal space. It’s easy to skim past them, or to bulk archive them if necessary. Apparently I’m not the only one who’s hacked this feature in on their own before, because Inbox contains a category of labels in its sidebar marked as “skipped the inbox”, where it puts any labels set up as I’ve described. Inbox lets you convert these labels to bundles in one click, meaning you don’t actually have to go into Gmail and change the rules. This has been nice for my personal email, I expect it to be amazing for my work email if/when Google Apps starts supporting the Inbox interface. I get a lot of automated status/notification style emails for work, which I currently tend to auto-archive without reading. For the most part this is fine, but for messages that indicate warnings of some type, I’d love to be able to see the count of current warnings in my inbox, with the option to bulk-delete with a click.

Other than Bundles, Inbox’s other features can all clearly be traced back to Gmail or the new wave of to-do style mail clients. The compose window and email viewing screen will both feel familiar to anyone used to Gmail’s current interface, albeit a bit stripped down and simplified. The compose button has the nice touch of showing the last few people you’ve emailed as default options to receive your new message. Reminders is a Mailbox feature that I’ve never used much, but Inbox steps it up a bit by allowing you to enter your own reminders without an existing email (much better than sending yourself an email), and by integrating Google Now style intelligence, for instance including directions to the restaurant when it reminds you about a dinner. I still don’t expect to get much use out of it personally, since I like my todos to get out of the email client to a more permanent resting place, but for those who use the inbox as their todo manager, it looks pretty slick.

One more praise: Inbox’s interface is simple, but avoids the mistake Mailbox makes of going too far down that road. Mailbox doesn’t allow you to filter by label and de-emphasizes the “sent from” email, 2 attempts to simplify the interface that have caused me grief in the past. Inbox avoids both those pitfalls while still removing a lot of clutter from the crowded Gmail interface. My one complaint in this area is that chat received the short end of the stick in Inbox. It’s tucked up into the header bar by default, with no quick visual cue as to who is currently online. Since I began writing this post, I’ve discovered it’s possible to pin a contact list to the right side of the screen, which makes me quite happy. That’s not at all obvious or discoverable though, and the default is a huge step down from classic Gmail’s wonderful in-browser chat interface.

Of course the reason that Inbox can be simple is that it is “just Gmail”. It ignores a lot of complex configuration that you’d want in an email client, and in doing so simplifies the UI. There’s no rules editor for handling different types of mail beyond the built-in intelligent bundles, and no ability to handle things like email forwarding or aliases as far as I’ve seen. It offloads that complicated stuff to the classic Gmail interface. That fact allows Inbox to be simple and clear for less sophisticated users, while still allowing power users a backdoor to customize, but it does mean that Inbox is an addition to, not a replacement for Gmail, at least for now.

Inbox is an extremely impressive piece of UI design. It’s the first major app to feature Google’s new Material Design style, and if this is the type of imaginative rethinking that we’ll get from Google’s design teams going forward, I’m excited to see more. My only real complaints at this point are the lack of support for Google Apps, something I’m sure will be rectified in the future, and the fact that, like Mailbox, it’s design encourages users to treat the inbox as a long term todo list, rather than a triaging area. I don’t expect that second issue to be fixed, it’s ingrained right down to the name of the application. But since there isn’t currently a better option on that front, I will happily enjoy Inbox’s simplicity and nice design touches. It has already replaced the default Gmail interface for me as my desktop mail client, and I expect it to replace Mailbox on my phone as soon as it supports Google Apps. If Inbox sounds interesting to you I definitely recommend finding a friend with an invite and checking it out.

I've seen some speculation that Inbox might eventually support clients other than Gmail to allow Google to gather data on email outside their ecosystem, but I highly doubt it. Inbox is built on Gmail's proprietary concepts, and trying to pull things like reminders off without controlling the backend servers would be hacky at the very least.
↩
I don't want my inbox to be a long term to do list (which is what pinning seems to be for). The to do metaphor is more helpful as a triaging tool, to split email into "ignore", "handle now", and do later. If I'm ignoring or replying right away, I can do that and then mark them done. If it needs to be done later, I don't want it to sit in the inbox. I'll put it into a todo list app or my calendar instead and then archive it for reference if necessary.
↩

Setting Up Your Text Editor For JavaScript Development

Sun, 12 Oct 2014 21:05:51 -0400

The coding workflow for JavaScript developers gets better all the time. Over the last 10 years, the JavaScript ecosystem has built up a rich set of tools to make your life easier. Most of the buzz around tooling comes from browser devtools and the various build systems available for JavaScript projects, but you can also take advantage of improved tools within your code editor. While there are some great IDEs for JavaScript development out there like WebStorm and Visual Studio, most JavaScript developers seem to prefer lighter-weight text editors. So this post is going to be a quick look at the tools I use for more seamless JavaScript development in a text editor.

Basics

Before we get to anything JavaScript specific, there are a few important things to set up. First, I’m assuming that you’re using a modern, extensible text editor. Those include, but are not limited to: Sublime Text, Vim, Atom and Emacs. There is great support for all of the below tips in each of these editors¹.

In addition you should have

Reasonable default configurations²
A fast system for navigating between files³
Great code searching⁴

If you’re still working through those things, make sure to take the time to learn a bit more about your editor along with any JavaScript specific tools you use. There’s a ton of efficiency you can gain from having those 3 things in place.

Finally the first two tips below require node.JS and npm to be installed, as they rely on node libraries to function. If you’ve never installed those before, the process is pretty painless.

Once you have all of that in place, we can move on to other things.

Syntax Linting

JavaScript is a dynamic, weakly-typed language. It is also written by human beings. As a result, it can be easy to start running bad code without realizing that it has problems. Some types of problems you’ll find out about quickly. Bad syntax will halt program execution for instance. But others will linger, subtly effecting logic or just making the code hard to understand for future developers (including yourself).

JavaScript linters (also called syntax checkers) can provide some of the safety of a compiler, and also help enforce a consistent set of styles to help maintain readability. They are not magic. JavaScript is still a weakly typed language with some weird syntactical sinkholes. It’s relatively easy to make logic mistakes if you’re undisciplined. But linters are a great safety tool, especially for developers newer to JavaScript and its conventions. Run as part of a build process, they can prevent you shipping bad code and help standardize conventions across a team

While it’s a great idea to make linters part of your build process, you get even more power from integrating them with your editor. Almost all modern text editors include some sort of plugin for displaying syntax errors and warnings in the gutter to the left of the text. You can check out the integration pages for ESLint and jsHint to get a feel for the plugins available for your editor.

TernJS + Autocomplete

Linters are great for telling you what not to do, but can’t give you much guidance on what you actually are able to do. Fortunately we have other tools for that. Tern.js is a fantastic library that provides “intellisense” style autocompleting for JavaScript code, along with other IDE-like features including documentation links and refactoring support. It’s not perfect, since JavaScript’s nondeterministic syntax sometimes makes it hard to tell what properties a variable will actually have at runtime, but it takes a good shot at it and is able to go pretty far, especially for libraries where it can expose the full API to you within the editor. To get the most out of Tern, you’ll need to set up a Tern project definition, where you can specify any files that you always want loaded, and any plugins you want to use. The preloaded files is useful if you’re making global references to a namespace or library, and the plugins can give you support for module systems, so that you can handle references that are passed into a module from a different file in code using requireJS or Angular.

Snippets

JavaScript development, especially on the front end, can involve a lot of repetitive boilerplate code⁵. Whether it’s module boilerplate like AMD require/define wrappers, library boilerplate like directive definitions in Angular or a Model definition in Backbone, or our own favorite patterns that we repeat a lot, we end up writing a lot of code over and over again. Snippets are a great way to accomplish that. They were one of the key features of Textmate when it first became popular. Today they’re built in to Sublime Text and Atom, and there are popular libraries for both Vim and Emacs to simulate the same functionality.

Snippets allow you to define templates for common boilerplate code, allowing you to quickly insert repetitive content, including adding hooks for text that will vary across uses. For instance here’s a snippet I have defined for creating a new AMD module⁶.

#A new AMD define module
snippet dfn
define([$1],function($2) {
    $0  
});
endsnippet

This allows me to type dfn<tab> at any time to expand out the template. Initially my cursor will be in spot $1, and I can then enter text and tab to spot $2, with my final tab ending me at $0. Snippet libraries can also provide other functionality like functions to show the current time, filename, or other contextual information. Some can also mirror text so that you can enter a string once and have it appear throughout the template. It can be a great time saver when you’re chugging through the boilerplate code of your project.

More Resources

If you want to implement JavaScript syntax checking definitely check out ESLint or jsHint. They both offer a lot more options and flexibility than jsLint, the pioneer in this area. It’s pretty easy to find an integration plugin for most common editors through those websites or Google, but Syntastic for Vim and Sublime Linter for Sublime Text come especially recommended and have the advantage of supporting all 3 major JavaScript linters as well as syntax checking for multiple languages.
Tern’s website has great documentation and links to the editor specific implementations for different editors. If you’re a Vim user I’d also recommend taking a look at YouCompleteMe for an improved autocomplete interface that works well with Tern and UltiSnips.
For snippet libraries, you can take a look at UltiSnips for Vim and Yasnippet for Emacs. If you want an application agnostic snippet solution, Dash is a documentation management program for OSX that also has a rich snippet component you can use in any program the same way you’d use an editor specific solution.

Other than the lack of an official Tern plugin for Atom ↩
Sublime Text and Atom provide pretty reasonable defaults. Vim users could consider checking out my piece on configuring Vim. I don't know enough to advise for Emacs, you're on your own their, but google is your friend. ↩
This really deserves a post of its own, not a footnote, but in general my approach to this is fuzzy search, split panes and a way to retrace my movements. ↩
Sublime Text and Atom have reasonably nice built in search tools, but I'd recommend using a plugin that integrates with ack or ag, two very nice search tools. I don't see such a plugin for Atom on their repository site right now, but the other 3 certainly support it. ↩
Hopefully not too much obviously ↩
This is using the syntax for Vim's Ultisnips plugin. ↩

Custom Elements By Example

Thu, 28 Aug 2014 08:33:00 -0400

The best attribute of HTML is its ability to be simultaneously machine-readable and human-readable. I’d argue that it’s one of the biggest reasons for the web platform’s success. Anyone can “view page source” and have some understanding of what they’re seeing. Javascript based “Single Page Applications” have undermined this a bit. Now the original HTML of a page may just be an empty body tag, and even when you inspect the generated page, it will probably just be a sea of div tags.

That’s going to start changing in the near future. Custom Elements, part of the Web Components spec, are a way of bringing back the semantic web. They allow you to encapsulate HTML and Javascript functionality into “elements” which you can include in your HTML like any existing native element, with all their semantic benefits. The best part? They’re usable in production now.

So What does it look like to use custom elements?

Talking about this stuff is great, but this is an API for shippable code, not just an abstract spec. So let’s look at a simple example. In a past job, I made pretty extensive use of the jQuery UI widget library. jQuery UI is great, but I always found it a bit akward to work with in practice. When it’s used, it takes an existing HTML Element, usually a <div> and transforms it into a widget. That means that to understand the end result created by the widget code, you have to look in at least 2 places, the original HTML code that contained the transformed element, and the JavaScript code that contained the call. It creates mental overhead knowing that you always had to remember to widgetize the dom element when it’s rendered. It also means accepting that the final result of the widget might look significantly different than the original. For instance, here’s the before or after for the progress bar widget. I chose it for it’s simplicity, but you can see that the final result is barely recognizable from the source.

display">

This seems like the perfect opportunity to use a custom element. If we can encapsulate all that behavior, we’ll be able to ensure that the progress bar is always rendered when it appears, and we can hide any ugly transformed DOM elements underneath a shiny semantic element. Sounds great! So let’s see how this works.

To start, we need an object to serve as our progress bar element’s prototype. It should inherit from the base HTMLElement class. We can create the prototype object like this:

var ProgressBar  = Object.create(HTMLElement.prototype);

Now that we have our Prototype element, we can register it as a custom element using the document.registerElement API. For now this is only natively available in Chrome, but it’s quite reasonable to polyfill. registerElement accepts 2 arguments: the name of the element¹, and an optional second argument to allow you to specify a prototype for your element. So let’s set up a progress-bar element:

document.registerElement('progress-bar',{
  prototype:ProgressBar
});

Our ProgressBar object will now serve as the prototype for the progress-bar element. That means we can extend it and give it extra functionality which will then be available on every instance of that element. Besides allowing us to attach arbitrary properties or functionality to an element, this lets us hook into the “lifecycle callbacks” of HTML Elements, which you can see in this chart from HTML5Rocks:

Callback name	Called when
createdCallback	an instance of the element is created
attachedCallback	an instance was inserted into the document
detachedCallback	an instance was removed from the document
attributeChangedCallback(attrName, oldVal, newVal)	an attribute was added, removed, or updated

Let’s start by looking at createdCallback. Since this gets run when the element is first created, this is a great time to run any initialization code that doesn’t require the element to be present in the DOM yet. Since jQuery UI’s progress bar code doesn’t require the element to be attached to the DOM, we can do all of our initialization here. That could look something like this:

ProgressBar.createdCallback = function() {
  var bar = $('<div id="bar">').progressbar({
    value:+this.getAttribute('value') || 0
  });
  $(this).html(bar);
};

So using jQuery, we create a new div element and immediately transform it into a progressbar object. Then we attach it inside our custom element container. Pretty straightforward.

Note that we set the value of our progress bar from the value attribute of our element. HTML attributes provide a very convenient “public API” for Custom Elements. You have complete access to an elements attributes inside these callbacks, allowing for easy declarative APIs. In the case of our simple progress bar, value is the only attribute we care about, and other attributes are ignored.

But what if the value of our element changes? Attributes aren’t a one time only API. If we want to react to changes in the attribute, we can use the aptly named attributeChangedCallback. That callback fires everytime an attribute changes, allowing us to catch the change and respond appropriately. A more sophisticated element might contain a registry of callbacks for different elements, but for this example we can focus on changes to value.

ProgressBar.attributeChangedCallback = function(attrName, oldVal, newVal) {
  var $bar;
  if(attrName === 'value') {
    $bar = $(this.getElementById('bar'))
    $bar.progressbar('value',+newVal);
  }
}

And that’s it! We now have a working progress-bar element, that we can put anywhere in our HTML with no extra js configuration.

See the Pen Custom Elements Progressbar Example by Ben McCormick (@_benmccormick) on CodePen.

Of course there is plenty more we could do. A more complicated element might require some initialization to be moved to the attachedCallback, and might require better cleanup using the detachedCallback. For communication with the rest of our app that didn’t fit well into the “changing attributes” model, we might need to implement an eventing system or global registry for accessing data. But those are all additions on this basic, useful building block.

If you’re interested in trying out custom elements, there’s never been a better time to get started. It’s easy to try out proof of concepts in Chrome, and there’s a light-weight polyfill with support back to IE9 for those who want to start using them in production. If this is something that excites you, there’s no reason not to start using this tool in your code today.

More Resources

HTML5Rocks has an awesome detailed look at Custom Elements that covers all of the ground we looked at here, and more. If you want to dive deeply into this feature, it’s a great read to get started.
Telerik posted another great piece on this topic a few weeks ago, which focuses more on the case for Custom Elements being production ready now
If you’re interested in using more than just Custom Elements, The Polymer Project by Google is an ambitious library built around the Web Component spec. They attempt to polyfill the whole spec, then wrap convenience functions around the low level elements. It’s an interesting, ambitious project, though not yet ready for production.

The Web Components spec specifies that valid names should be in the form of 2 words separated by a hyphen in order to avoid conflicts with existing and future "official" HTML elements. ↩

The Debugging Toolbox

Tue, 19 Aug 2014 07:35:00 -0400

Do you know a Problem Solver? They’re the ones who can take a crazy, vague bug report from a client and come up with a fix in an hour. The ones who everybody else asks for help when the project deadline is creeping up. The ones who get shipped out to the client site when the field team is out of their depth. In school they were the person you called at 1AM the night before your project was due, hoping for a miracle.

I’m guessing you can think of somebody. Problem solving is a valuable gift, something that stands out. But the truth is, all software developers should be problem solvers. It’s a key part of our jobs. We just happen to call it debugging.

Debugging is hard. As developers, we’re faced with vague problem descriptions, massive code bases, and constant pressure to move our projects forward. But you can do some great things if you’re disciplined about getting better at debugging.

I’ve built up my own debugging process over the years. It’s nothing more exciting than Problem Solving 101, but it’s a productive way of working that lets me get stuff fixed quickly and keep issues from happening again. It helps me fix things even when I’m not qualified to do so, and keeps maintenance work from overwhelming my ability to get things done. If that sounds useful to you, you should probably keep reading. Here’s my process.

Step 1: Define The Problem

The need to define the problem seems like common sense, but it often gets skipped. The first thing you need before fixing a problem is a clear definition of what the problem looks like. That means a detailed look at 2 things:

What is the current behavior?
What should the behavior be?

The second question often gets neglected. At Windsor Circle, we have a system that receives product records from another system. The data it was receiving occasionally contained blank product titles, which was not considered valid data. I was asked to ensure that the second system would not pass invalid data. I could of course have just decided to not pass any records that had blank product data. Or I could have filled them with a "None" string, calculated a value from other attributes, or a thousand other approaches. Since each of these would have removed the current bad behavior, I might have been able to slip them by for a while. Instead I asked what the correct behavior should be, which led to a longer discussion and an eventual decision that we would need to handle a small set of empty titles after all. At that point I was able to actually deal with the problem.

Step 2: Reproduce The Problem

After you define the problem, the next step is to reproduce it. Do not skip this step. If you can’t define and consistently reproduce a situation where the problem occurs, it will be impossible to know whether you’ve solved it. If the issue involves a lot of specific circumstances and moving parts, or only occurs sporadically, this may be the hardest part of the whole process. The things you want to keep in mind:

What are the circumstances under which this occurs? Always? Only on certain browsers/OSs/hardware? Is it a timing issue? A timezone issue? All of the above?
Is there a series of actions that I can do that makes this occur every single time?
Am I confident that I know all of the situations that cause this to occur?

Answering #1 is the first step. If you can say yes to #2 you’re in a fantastic position to proceed. If you can say yes to #3, you need to get a lot more suspicious and cynical. But the more confident you can be about each of these, the better equipped you’ll be going forward.

Step 3: Narrow the Scope

Now that we’ve established our problem and hopefully found a consistent path to reproduce it, we’re ready to start rooting out our problem.

The first step is to narrow the scope. In a normal software environment, you’re likely to be dealing with a large amount of your own code, as well as 3rd party libraries and APIs, some sort of environment, and unpredictable user behavior. That’s a lot of uncertainty and information to process. I’ve seen developers who really try to understand the whole system before even thinking about the problem. But in many cases that’s unrealistic. Instead, you should be trying to get rid of as much of that information as possible by narrowing the scope of where the bug might exist. There are 4 main ways of narrowing scope.

Prove That The Problem Is Unique To A Component

If you want to eliminate a system component from consideration, one good way is by proving that the issue is not unique to it. This works particularly well for platform issues. For instance, you might prove that something is not a browser-specific problem by showing that the issue occurs in all browsers. This can give you some firm ground to stand on. If you do expose a platform-specific issue, that can be good news too, as they tend to be better documented.

Follow The Flow

If your problem is a data issue (and many issues are), you can also narrow the scope by following the flow of data through your application. There are 2 main ways to do this. One is by starting at the beginning of the flow, where the data initially comes into the system, and follow it through, looking to see where things go wrong. You could also start at the end with the incorrect output and try to trace back and see where it came from.

A good debugging tool is a great asset here. There may be times when you’re forced to work without one though (for example, working in an environment you don’t control or a domain-specific language without adequate tooling). In that case you can get by with log statements, or even changing the code to output a result early. Regardless, what you want is clear evidence that data was good at a certain point, or already bad at another. This allows you to eliminate that code from consideration and focus in on the code before it.

Follow The History

If you know this problem was introduced relatively recently, you can approach it by figuring out what changed. Version control systems in particular provide great tools for this type of analysis. If you know that the change was introduced within a set period of time, that provides an additional dimension from which you can narrow the scope and filter ideas.

Step 4: Make An Educated Guess

Now it’s time to make a guess at where the problem may be. It’s important for this to be an educated guess. You don’t want to check code at random. Your starting point should come from your past experience with similar issues, or be the result of research into the problem. I can’t overstate how important research can be. It’s always worth starting with a quick Google search, since it might save you hours of time looking on your own. Once you’ve narrowed the scope a bit, you can revisit your research, looking specifically for known issues with a specific component for instance. Your guesses also can be a follow-up to new information from Step #5. Despite the ordering I give here, debugging is never a completely linear process. You need to have all of these tools in your toolbox. Experience will help you know which to use.

Step 5: Understand The Behavior

If you’ve managed to isolate the unwanted behavior a bit and have a guess where the problem might be, now’s the time to read more deeply into the code. What is actually happening in there? Does the logic of the code match your understanding of its purpose? Do you see any red flags or danger signs?

I once interviewed a candidate for a programming job who told me he didn’t read code very much while debugging. Instead he made changes to observe the new behavior and kept fiddling until the output worked the way he wanted. While that might work for aligning text on a Wordpress site, you simply can’t get away with that in a complex application. Reading and understanding code quickly is a skill, but it’s one you must develop if you want to be a productive developer.

Step 6: Repeat Steps 3-5 until you have a thesis

If you’ve narrowed your scope and read some code, you might have an idea about what you think the problem is and how to solve it. If you do, then proceed to step #6. If not, take what you’ve learned and use it to loop back to step #3, to reduce the scope even more.

Step 7: Test your Thesis

If you think you have a handle on the problem, now’s the time to test it. There are a few ways to do that. One obvious way is to apply a fix and see if the problem goes away. But if the fix would be expensive in time, money or complexity or if you’re afraid of treating the symptoms rather than the disease, you can also look into creating a simpler test case.

Earlier this month I encountered an issue where a handlebars template was failing to render correctly occasionally. After digging in and narrowing the scope, I determined that the issue occurred only on Chrome, and seemed to happen randomly after a few renders. I was able to create a very simple test case that reproduced the problem in a jsFiddle. I didn’t use any of my own data. Instead I just created the simplest test that I expected to fail. And it did. That gave me confidence that the issue was in the Handlebars-Chrome interaction, and I was able to find a workaround by looking through their issues page on Github¹. That type of iteration on your test cases also provides useful reference material in case the issue reoccurs.

Step 8: Repeat 3-7 until you can no longer reproduce

If your thesis turns out to be wrong, take your new knowledge and go back to step #3. It’s important to understand that this isn’t a rigidly structured process though. This also may be time to go back and double check steps 1 & 2. Do you really know what the right behavior is? Have you learned anything that might change that? If you couldn’t reproduce the issue after a change, are you sure that you’ve really fixed it? Or were there holes in your original path to reproduce it? A healthy amount of paranoia helps here.

Step 9: Document and future-proof

Once you’ve fixed the problem, or gotten to a point where you’re not going to continue for some reason, there’s one very important last step. Document what you’ve learned. If you made a change to the code that “looks wrong” but fixes the problem, leave a comment explaining it, and save it from future “code cleanups”. If you can’t reproduce the problem consistently and it still exists, document what you learned in a bug tracker so that somebody can come back to it later. If you isolated the problem to a 3rd party component, file a bug report and include your test case (simplified as much as possible). If you created a useful test case, automate it or add it as a reference somewhere. Exact processes will vary by team, but the key phrase here is “Those who don’t learn from the past are doomed to repeat it.” Are you going to bruise this bug or bury it? Make sure you can fix the problem quickly if it happens again. If you can do anything to keep it from coming back, do it.

The Takeaway

Real world engineering means dealing with code you don’t understand. That might be your coworkers' code, a 3rd party library, a buggy video driver, or your own 3 year old code. To do that effectively you need a process. That might be mine, or one of your own, it could be formal, or just built on habit. But make sure you have one. The ability to dive in quickly, understand what’s going on and create a fix will set you apart. It’s not that hard, but it takes discipline, creativity, and most of all practice. So go ahead and start fixing something. Maybe you’ll be the Problem Solver next time

Chrome 36.0.1985.143 seems to have resolved the issue. ↩

Component Based Development

Thu, 07 Aug 2014 07:30:00 -0400

If you look back at the history of JavaScript and the client-side web as a development platform, you can see that there were several critical turning points for the language. Javascript has evolved from a 10-day toy language to possibly the most used language in the world, but that didn’t happen overnight.

JavaScript’s growth started with its adoption as the lingua franca of the web when it shipped with Netscape and Internet Explorer in the 90s. The introduction of Ajax and asynchronous data exchange in the mid-aughts was another sea change, bringing the potential for truly responsive web applications like Gmail for the first time. jQuery’s introduction in 2006 dramatically reduced the pain involved in cross-browser development for rich applications. And the last 5 years have brought 2 more major changes with Node.JS and a raft of MVC frameworks. NodeJS is a server-side technology, but it has spurred a huge standardization and improvement in tooling for the web platform. That, along with great work by browser vendors on their developer tools, has led to drastically easier workflows for front end developers. At the same time, frameworks like BackboneJS, AngularJS and EmberJS have brought architecture concepts to the client-side and made a new type of “Single Page App” possible and easy to build.

Those 5 shifts, along with other significant events like Google’s creation of Chrome and the rise of mobile, have shaped the web development world as we see it today. Depending on who you listen to though, we may be on the cusp of another pivotal moment.

Component Based Development is a blanket term I’m using to describe a movement that has been growing over the past year in web development. The results of this movement have taken several forms. There’s an official W3C standard, Web Components, that will eventually be available in all browsers natively. That standard has polyfills, which allow developers to begin using Web Components now, along with convenience extensions. Several existing MVC frameworks have provided component features inspired by the Web Component spec. And there are alternative component implementations, which embrace the ideas behind component based development without embracing the specific implementation of Web Components.

So what is Component Based Development?

The heart of the component based development movement is a desire to provide greater code reusability and abstraction by bundling functionality into small composable components. JavaScript applications started as monoliths. The language doesn’t provide private attributes by default, and applications were initially mostly small, so everything lived in the global scope. As applications have become more complex, this has changed, first with namespacing, then with the module pattern, and again with module loaders like requireJS and Browserify. The features mapped out by the Web Components standard will build on this trend. Component based development in general is the natural conclusion of the march to modularity. Components are intended to be “black box” building materials. They have an API, but the developer doesn’t need to know about the internals, and ideally can’t take advantage of that knowledge regardless.

Talking about this type of abstraction is nothing new. Whether you’re using the module pattern or Backbone Views, it’s always been a best practice to keep internal implementations hidden. But web components in particular give this more teeth, by discouraging external CSS and Javascript from accessing the internal dom tree of the element. Other implementations can’t rely on that native split yet, but they similarly promote a declarative style that discourages tightly coupled micro-managing code.

Components also have the potential to provide greater readability. Compare Google Map’s current Hello World example, with a web component powered alternative:

<!DOCTYPE html>
<html>
  <head>
    <meta name="viewport" content="initial-scale=1.0, user-scalable=no" />
    <style type="text/css">
      html { height: 100% }
      body { height: 100%; margin: 0; padding: 0 }
      #map-canvas { height: 100% }
    </style>
    <script type="text/javascript"
      src="https://maps.googleapis.com/maps/api/js?key=API_KEY">
    </script>
    <script type="text/javascript">
      function initialize() {
        var mapOptions = {
          center: new google.maps.LatLng(-34.397, 150.644),
          zoom: 8
        };
        var map = new google.maps.Map(document.getElementById("map-canvas"),
            mapOptions);
      }
      google.maps.event.addDomListener(window, 'load', initialize);
    </script>
  </head>
  <body>
    <div id="map-canvas"/>
  </body>
</html>

<!DOCTYPE html>
<html>
  <head>
    <meta name="viewport" content="initial-scale=1.0, user-scalable=no" />
    <link rel="import" href="google-maps.html">
  </head>
  <body>
    <google-map></google-map>
  </body>
</html>

Quite a readability difference right? Of course all of that Javascript and CSS didn’t simply vanish. It still needs to exist somewhere, in this case within an imported HTML file. But it can be encapsulated and looked at only when necessary. The user who’s just glancing at your template can get a big picture view when she needs it, and then know where to dive in when she’s specifically interested in your mapping technology.

In the end this is just another way of managing complexity. There is unavoidable complexity associated with connecting to the Google Maps API and styling the map appropriately. But when that complexity sits inside an element, it reduces the cognitive load of looking at HTML. That allows you to focus on the semantic meanings of element and the document structure. This kind of clarity is much nicer than our current status quo of searching through a Javascript modified “div soup” to figure out how our generated DOM is structured. With current MVC frameworks the HTML may tell you nothing about the final DOM without going through several Javascript files. The transparency of components is a reclamation of one of the Web’s original central advantages: a semantic markup language that was both human and machine readable. The tension between the needs of web “documents” and “applications” has slowly eroded the Web’s declarative foundations, but component implementations do their best to embrace it.

So Why Should I Care?

Did that introduction excite you? Or are you sitting there yawning? You may think this is solving problems you don’t have. You also might be the guy still supporting IE8, who can’t even contemplate worrying about “Web Standards” if they aren’t even supported in IE11. If that’s you, it’s still worth following this movement, even if you’re not coding like this just yet on your current projects.

For one thing, the component based mentality is already having a big effect on the crop of frameworks that people are using now. EmberJS and Angular have supported component-like features for a while now. KnockoutJS (which supports back to IE6!) is about to release significant support for a component-based development strategy. And the latest JavaScript Framework darling, Facebook’s ReactJS, is a completely component based library that supports browsers back to IE8.

Component based development has also gained significant support in the community. I already mentioned the strong support from existing libraries, but there’s more to it than that. Facebook, Google, and Mozilla, three of the web’s foundational companies, have all released component based libraries. Google’s Polymer library in particular has been featured as the center of their web developer tooling lately. Web Components are already supported in Chrome and Firefox. Along with the community support and the trend towards auto-updating browsers, this is likely to be a relevant every-day technology sooner than you may think.

The Takeaway

2014 is an interesting time for web development. There’s a lot to be excited about, as the component paradigm is leading to a lot of interesting experiments that create new possibilities. Ideas like React’s Virtual DOM and Polymer’s completely declarative application structures are going to be tested and tried, and we’re likely to be better off in the end.

There’s also plenty to be skeptical about. The lack of any known support for Web Components in Safari or IE is troubling. While Polymer’s “everything is an element” philosophy is fascinating, I suspect that we’ll look back and see it as an example of taking a good idea too far. And like any technology, until it’s been proven in production, we can’t really know how these technologies will evolve.

For anyone invested in the Web Platform, it’s time to be informed. Regardless of where each individual spec goes, the ideas behind this movement are going to influence our professions for years to come. Over the next month or so, I’m going to be diving deep into the current state of components, taking a look at the various aspects of the Web Component spec and also looking at the various libraries that are providing their own take on component based development. I hope you’ll come along, and help grow the conversation about the future of the web platform.

More Resources

In The Web’s Declarative, Composable Future Addy Osmani lays out a manifesto for component based development, and Web Components in particular. If you’re struggling to understand why somebody would want Web Components, this is the piece to read.
TJ VanToll reminds us that Web Components Aren’t Ready For Production… Yet on Telerik’s Developer Blog. He does a good job of higlighting the browser support issues along with the difficulties of polyfilling this particular technology.
This Github discussion is an interesting look into the thought process of why a traditional MVC library would be interested in providing component features, and the value they bring.

Learning Vim in 2014: Search

Sun, 03 Aug 2014 21:16:20 -0400

Have you ever tried to summarize something that was just too big to explain? I’ve covered a lot of ground in my posts on Vim this past month, but it’s only scratched the surface of what Vim has to offer. The goal was to create a foundation that people could start with, and then let them build on their own. There’s not a conclusive way to say what somebody “needs to know” when learning Vim. Most people don’t (and shouldn’t) use every feature of the editor, they use the ones that make sense for them. So I’ve tried to cover the things that I know people have found useful, and the philosophy behind how Vim works, and then let you figure out what you’d like to use on your own. This post wraps up the series, but I wanted to highlight one last feature of Vim, its fantastic search capabilities.

Search

Vim has an extremely powerful built in search tool. To use it, you can type / at anytime in normal mode to start entering search terms. For instance /foo followed by enter searches for the next occurrence of foo in the document. You can then skip over matches with n or go backwards with N. If you wanted to start by searching backwards, you could have started with ?foo instead of /foo, in which case n and N would reverse behavior, with N moving forward in the document and n moving back. It’s easiest to think of n as “next match” and N as “previous match”, with the direction determined by the search operator, / or ?.

The search tool can be used for more than simple literal expressions, it also takes regular expressions. So /fo.\|bar will match foo, fox, and bar. You also can search for lines containing both foo and bar with the expression /.*foo\&.*bar. If you want to dig deeper into Vim’s regex language, I’d recommend typing in :h pattern to check out the excellent documentation.

Substitute

Search is great, but in many cases what we want to use it for is replacing or acting on each instance. Vim’s search and replace command looks something like :%s/foo/bar/g, which will replace all occurences of foo in the document with bar. Let’s break that down a bit. :s (substitute) is Vim’s search and replace command. By default it only works on the current line, so in order to get it to act on the whole document I prepended the % character, which sets the range to include everything. If you want a smaller range, you can select the text you want to act on in visual mode. Typing : will then preload the range into the command area. Substitute takes an expression in the form /<search expression>/<replace string>/<modifiers> where search expression defines the search, replace string is the text to substitute in, and modifiers are single letter arguments that change the behavior. By default only the first instance of an expression per line is substituted, so I added the g modifier to tell Vim to substitute all instances of the search expression on a line. Other useful modifiers are i, which makes the search case insensitive, and c which has Vim prompt for confirmation before substituting each instance. Confirmation is helpful if you want to change most but not all occurences of a phrase. One useful behavior of substitute to be aware of: If you leave the search expression blank, it reuses your last search. So if you’ve been playing around with a regex to get the right expression, once you get it right you can type :%s//<replacement>/g, and it will replace all instances, without forcing you to retype your complicated regex.

Finally, it’s worth noting that search plays nicely with Vim’s composable actions, and you can use it as a motion command with y, c, d or other actions. For instance d/foo will delete all text up to the next occurence of foo in the document. y?foo will yank all text backwards to the previous occurence of foo in the document.

Inline Searching

Along with the primary search and substitute commands, Vim has other specialty searching tools. One of these is its f and t commands. f<char> moves to a single character on the current line. While less powerful than /, this can be super useful because of its quick composability and guarantee of not moving out of the current scope. It’s really easy to type cf) to delete through the end of the parentheses. t works similarly, but is non-inclusive, so you can use ct) to delete everything up to and not including the parentheses. I remember it as “change find character” and “change to character” respectively. Like /, these commands support backwards and repeated search. F and T search backwards on the current line, and ; and , are the default commands for “next match” and “previous match” respectively.

Cross-file searching with vimgrep and grep

Searching within a file is nice, but what if you want to search in multiple files? Vim provides 2 commands for that. The first, :vimgrep, uses Vim’s internal regex engines to search across multiple files. You could for instance type :vimgrep /function/i ./*.js to search all Javascript files in the current directory for the “function” keyword. You’d then be able to loop through them using the :ln and :lp commands.

The second command, :grep, integrates with an external search application (grep by default), to do the searching. The advantage of the first is that it’s portable and will be available anywhere and everywhere that Vim is. But generally external tools are going to be faster than Vim’s internal search engine. Grep for instance is faster than vimgrep. I personally use the external tool ack which is faster than grep. Regardless though, Vim has support for whatever approach you choose to take.

Cross-file replace with the arglist

Up till now we’ve mostly been dealing with tasks that Vim has good answers for. If you want to do a search and replace across multiple files in Vim though, you end up in some of the more esoteric parts of Vim-land. In order to do a search and replace across multiple files in Vim, you need to first define a set of files to edit, and then run the replace command. Unfortunately there is no way in Vim to cleanly do this all in a single command. Instead you must pull your list of files from one of 3 places, the arglist, the buffer list, or the window list. The arglist is by default the files that you specified to open when you initially opened Vim, the buffer list is your list of currently opened buffers, and the window list is the list of files in currently open windows. Since you probably don’t want to have to open or view all the files that you want to run a search/replace on, and you may want to have files open or visible without running a replace on them, the arglist is the list of choice for replacements. Vim allows you to edit the contents of the arglist at any time using the :argadd and argdelete commands. So to replace foo with bar for all javascript files in the current directory you might type something like.

:argd * "Clear arglist
:arga ./*.js  "Add javascript files
:argdo %s/foo/bar/g "Substitute on each file

argdo is the command to execute a command across all files in the arglist. bufdo, tabdo, and windo work the same way for every “open buffer”, “active window in each tab”, and “window in the current tab” respectively. Personally I find this all to be a pain. A single command that let you specify an action and a range to run it on like :do '%s/foo/bar/g' ./*.js would make plenty of sense. But since that doesn’t exist, it’s worth at least knowing that the arglist is there, and being able to resort to it if necessary.

The Power Of Search

Most people start out using Vim knowing that they can move around with hjkl and switch between files with :e <filename> as they learn more they grow their toolbox. Search is a great tool for moving quickly around text in Vim, and for powerfully making big edits. Searching allows you to allow your brain to focus on the big picture, rather than remembering exactly where each piece of code or content is located. Like everything else in Vim, it’s a tool that you can combine with other tools to create powerful workflows.

More Resources

Vimcasts has a nice piece on the gn command, an alternative to n that lets you select the next search match in visual mode
Vim’s wiki also has a nice writeup of the :substitute command, with lots of examples and details.

Learning Vim in 2014: Copy and Paste the Vim Way

Sun, 27 Jul 2014 19:00:00 -0400

If you’ve been following my series on Vim, it should be clear now that Vim has a pretty clear philosophy of how text editing should work. It’s based on the Unix philosophy of small composable tools, and doesn’t necessarily match up with the conventions that other editors use for common commands. So it’s probably not surprising that Vim has its own way of handling copy and paste operations, and in fact doesn’t even use those terms. Vim’s copy and paste handling is minimalist, composable, and powerful, but most people take some time to adjust to it. I’m going to walk through the basics here, along with a few advanced features that are worth knowing about.

The primary Vim commands for copying something are y (yank) which acts like most people’s view of copy, and the delete/substitute commands d (delete),c (change), x (single character delete), and s (substitute), which unintuitively all act like the more common concept of “cut”. These can be composed with motion commands to select different regions.

Any Vim command that deletes text will also save a copy of that text unless you specifically tell it not too. The idiomatic Vim method of deleting without copying is "_d, which probably sounds bizarre. We’ll come back to that later, and you’ll see that it makes sense in the bigger scheme of Vim’s copy and paste system.

The basic Vim commands for paste are p (put) which pastes after of the current character/line and P which pastes before the current character/line. Vim auto detects whether you’ve yanked text on a line by line basis or a character by character basis, and pastes accordingly. Basically, if you select only full lines it will paste line by line and not break on the cursor position, if you select any partial lines it will paste at the character level, starting or ending at the current cursor location. Note that p also works in visual mode, where it will replace the current selection.

All of this leads to a very common annoyance for new Vim users. They’re editing a document and yank some text. Then they go to a new location, delete a few lines, and hit p to replace them with the copied text. Do you see the problem? The text that is “put” is not what they originally yanked, but instead it’s the contents of their last delete action. Obviously this can be worked around by changing the order of the actions, but it’s frustrating to users who are used to being able to easily delete without blowing away their paste action. Even more frustrating is when an experienced Vim user comes by and tells them the answer to their problem is to just type "0p instead. Which will in fact put the correct text. So what’s going on with the weird syntax for these basic action?

Registers

What we’re missing is a clear understanding of how Vim handles copy and paste operations. So let’s clarify. Unlike most modern systems, which have a clipboard that holds a single value, Vim can store the values of yank and delete commands in one of many registers. It has 26 normal registers, which correspond to the letters of the alphabet, and several “special” registers. One of those special registers is the default register, which is where yank and delete commands store their content by default. Other registers can be accessed with the "<char> prefix that we’ve already seen. So in the above example, the text from the initial example is moved into the default register, and then replaced there by the deleted text. But it remains in the 0 register, which always points to the last “yanked” text, ignoring text gained by deleting.

The 26 alphabetical registers serve as great “medium term” storage. You can use them to yank something that you want to have around for a while, and then put it in a few different places, even if you’re yanking and deleting other things in the meantime. These will even persist across sessions as long as you have the nocompatible option set in your vimrc file. However if you overwrite one, there’s no easy way to get it back.

One cool feature of registers is the built in ability to append to the end of them. If you want to add text to an existing register, for instance if you missed part of the text you wanted to yank, you can do so by capitalizing the register. So if you had deleted a line and put it in the a register with "add, but meant to include a second line, you could then delete the next line and append it to the register with "Add. "ap would then put the 2 lines back into the document.

Special Registers

In addition to the alphabetical registers, there are several special registers that are worth knowing about. I already mentioned the default register, which most Vim users know about, even if they don’t understand exactly how registers work. Other important registers are the clipboard register, the black hole register, and the numbered registers.

If you have to learn one special register, learn the clipboard register. One of the first things people notice about copy and paste in Vim is that it doesn’t interact nicely with other applications' copy and paste by default. If you yank a line of text in Vim, it’s not added to the system clipboard. If you copy some code from Stack Overflow and try to paste it in Vim with p, it won’t work ¹. But, since we know about registers, it makes sense that the default register might not be mapped to the clipboard, which we don’t want getting blasted away everytime we delete a character.

Vim isn’t completely disconnected from the system clipboard though. The clipboard register + is Vim’s proxy to your system clipboard. So "+y<motion> and "+p act like traditional copy and paste. Your version of Vim does have to be compiled with clipboard support in order to use the + register. You can check to see if you have clipboard support with :echo has('clipboard').² On OSX you can use MacVim to get clipboard support, since the default version of Vim shipped with OSX is not compiled with it. On other operating systems you’ll have to investigate the easiest way to install with clipboard support if your version doesn’t have it. It should be enabled for most modern mainstream distributions.

Another register to quickly note is the black hole register _. The black hole register, as you would expect, doesn’t retain what’s passed to it. So "_y and "_p are no-ops, and "_d is “true delete”, as opposed to the delete commands default “cut” like behavior. Of course since most people don’t use all 26 alphabetical registers, you can also achieve effective true delete by deleting to any unused register.

Finally, the number keys also have special register functionality. You can’t yank text directly to a numbered register. Instead the 0 register contains the last yanked text, and the 1 through 9 registers contain the last 9 deleted chunks of text. This feature is a cool idea, but unfortunately the implementation is inconsistent and a bit weird. For one thing the distinction between yanked and deleted text seems arbitrary, and since they act the same in other ways, puts an added cognitive load on the user to remember which one they used. Secondly, for whatever reason, deletions that span less than a line get special behavior. If you delete these to an alphabetical register, they’re saved to "1 just like any other delete. But if you delete them to the default register, they’re saved to "- and not put in "1. The logic is complicated enough that the numbered registers become tough to use in day to day tasks, and I’ll have to agree with Drew Neil in labeling them one of Vim’s “bad parts."

Macros

One last important thing to know about registers is that along with being used for copy and paste, they serve as a place to save macros, Vim’s reusable command language. You can save a macro by typing q<register key><commands>q, and the macro will be saved to the register. Like yank and delete, capitalizing the register name will let you append to the register instead of replacing it. So there’s a few things you should know about that. One, if you use macros, don’t save macros to the same registers that you use for copy and paste. If you use the y register a lot for the convenience of "yy<motion>, don’t use qy to save your macro unless you’re ok with it being blown away by your next yank. Two, the sharing of registers allows you to copy text to use as a macro. So if you for instance wanted to have a file with a list of common operations, it would be easy to go to that file, copy a line, and then execute it as a macro. This isn’t the first thing most people will want to do, but it illustrates the power and flexibility that come when you start combining Vim’s tools.

More Resources

For a more in depth look at Vim’s special registers, including a bunch I didn’t cover here, you can check out this great roundup
Vimcasts has a whole series of posts on copy and paste in Vim.

Of course some people would probably see that as a feature, not a bug ↩
Thanks to schweinschmeisser on Reddit for reminding me to add a way to check for support. ↩

One Day Left In Practical Vim Giveaway

Thu, 24 Jul 2014 19:55:45 -0400

My practical Vim giveaway ends tomorrow, Friday July 25th at midnight. If you haven’t subscribed or tweeted to win yet, now is the best time, so go for it!

Giveaway Details

As a thanks for all of the great interest and feedback I’ve gotten for my posts on Vim recently, I’m going to be giving away 2 copies of Drew Neil’s book, Practical Vim.

Practical Vim is the best resource on Vim I’ve found. Drew Neil is the creator of vimcasts, the popular online Vim site. And this book is his attempt to explain what makes Vim special. It’s bursting cover to cover with practical tips and tricks to help you become a more efficient writer and developer. Reading it helped me understand the philosophy behind Vim, and I would recommend it for anyone who uses Vim or thinks they might want to. There’s a reason I recommend it as the Vim resource on my reading list.

So how do you get a copy? There are a few ways

Follow @benmccormickorg on Twitter
Subscribe to my mailing list feed.
Tweet a link to one of my Learning Vim posts, with the hashtag #learningvim

You can accumulate up to 3 entries by doing each of the above (I won’t count multiple tweets for the 3rd). The giveaway will end July 25th, at which time I’ll pick 2 winners at random, and get in touch through Twitter or email. If you have questions or concerns, feel free to speak up in the comments. Otherwise, go subscribe and win a chance to get a great book for free!

Learning Vim in 2014: Getting More from Vim with Plugins

Mon, 21 Jul 2014 06:50:00 -0400

It’s impossible to design a tool that fits everyone’s workflow perfectly. No matter how many cases you hit, somebody will want to do something you don’t support, will find your solution for their problem confusing, or they’ll try to use your tool in a way you never imagined. Vim tries to deal with situations like that in 2 ways. First by making it easy to configure, and second by making it easy to extend. Extending Vim happens through plugins. Vim plugins allow you to enhance, change, or add to Vim’s existing behaviors. They’re a powerful tool, and a key part of what has helped Vim remain relevant for 23 years, even as the computing world has changed dramatically around it. They allow Vim to support languages that didn’t exist the last time its core was updated, handle niche use cases that would never get solutions in the main Vim distribution, and also allow for powerful new features that benefit all users.

Your key focus when it comes to Vim plugins should be identifying your needs and finding plugins to fit them. Don’t just toss every plugin you see recommended somewhere into your .vim/bundle folder. Plugins have costs; they slow down file loads, can fill up the key mapping space, and add complexity for users. But they’re also extremely powerful, and I recommend that everyone who controls their development environment think about using at least a few.

Plugin Management: Hello Vundle

Before you install anything, you’re going to need a strategy for managing your plugins. Plugin management for Vim has a bit of a troubled history. There’s never been an official plugin management solution, and unlike Sublime Text’s Package Control, there wasn’t a simple, easy to use solution that the community standardized. Instead, plugins were initially just placed into the .vim folder or elsewhere and then sourced in ~/.vimrc. To allow for download or installation, authors used a variety of compression formats, including a Vim specific Vimball format. Early efforts to standardize a process didn’t go particulary well.

That changed with the rise of Github and Tim Pope’s Pathogen. Pathogen was the first plugin manager to make it relatively simple to include new plugins, by encouraging a standard directory format and installation process (using git repositories). It began a standardization process that now means pretty much any modern Vim plugin can be installed with Pathogen without modification.

Despite Pathogen’s historical significance, in 2014 you should use Vundle, not Pathogen, to manage your plugins. Vundle is an easy to use plugin manager that is compatible with the Pathogen format. Vundle improves on Pathogen in several ways. It allows you to list the plugins you want to use in your vimrc file, then will pull them down and install them for you. So adding and removing plugins is just a configuration change, and it’s always easy to see which ones you’re currently using. It makes it easy to pull plugins from a variety of sources, but optimizes for Github, and Vim.org. Since roughly 99% of plugins that most users will need are available from one of those sources, this is extremely convenient. You can find full documentation for using Vundle in its Github repo, but trust me, it’s the best option out there for 99% of users.

What Can Plugins do for you?

Before you start installing plugins, it’s worth thinking about what you need from them. There are a few main purposes for using plugins. I’m including a few examples of each to get you started as you figure out which plugins make sense for you.

Let Vim Get Your Files In Order

File management plugins are some of the most important plugins you can use with Vim. Vim’s built-in file management is pretty weak in some ways compared to editors like Sublime Text and TextMate. Plugins can fill that gap. The plugins listed below show several approaches to file management. There’s plenty of overlap in functionality here, but they’re all worth trying to see which ones fit your style.

CtrlP is probably my most used Vim plugin. It provides fuzzy finding capabilities for Vim, to allow you to search for files in a similar way to Sublime Text’s “Go To Anything” bar.
Ack.vim and Ag.vim are plugins to connect Vim with Ack and Ag respectively. If you’re not using one of these tools for code search, you really should be. They’re much faster than grep or an IDE’s built in search. These plugins provide seamless integration for each within Vim, allowing you to easily jump to the code you’re searching on, opening new tabs or splits if you like.
Unite.vim is an interesting attempt to combine the functionality of plugins like ctrlp, ack.vim and others into a single “omnisearch” interface. I personally found it a bit unintuitive and poorly documented, but some people swear by it. If you are able to make sense of it, it’s certainly a great concept to pull in related functionality into a single interface.
The NERDTree is one of the most popular Vim plugins out there. It provides a more traditional side-drawer style navigation interface to complement Vim’s other navigation options. I personally prefer a “search over specifying” navigation style, so ctrlp and ack.vim work better for me. But I keep NERDTree around for looking at other people’s code in new projects, when I want to look around a directory to see what’s available and how it’s structured before diving in.
Projectionist.vim is something I’ve just begun to look into. It’s an interesting approach by Tim Pope to provide a more intuitive way to navigate structured projects. Rather than fuzzy finding or sidebars, Projectionist invites you to create categories of files that you can open files from (for instance model, view and controller files in a BackboneJS project). This gives you the simplicity of traditional Vim file path opens while removing some of the memorization and clutter from the process. Projectionist also allows you to specify alternate files for each file. That allows you to, for instance, jump quickly from a file to its unit test file.

Building On Vim’s Built In Language

Another interesting class of Vim plugins are plugins that extend Vim’s existing language concepts. In my past few posts I’ve looked at how Vim is intended to be used by combining composable commands with motions and text objects, then efficiently repeating that process when possible.

Surround.vim is another plugin by Tim Pope. It provides an extra command to use with Vim’s built in text objects. That command, ys allows you to surround a text object with some sort of enclosing tag. So ysiw' surrounds the current word with ', and ysip<div class="example"> surrounds the current paragraph with <div class="example">...</div> tags. There’s also a separate command to replace surrounding characters with different ones. It’s extremely useful, especially for HTML editing and quoting.
commentary.vim adds a “commenting” action, gc that can be used with existing motions. So gcc comments out the current line, gcG comments out the remainder of a file, and so on. This is yet another Tim Pope plugin. I promise I’m not trying to make up for recommending Vundle over Pathogen, the guy has just been prolific.
Repeat.vim is one final Tim Pope plugin for now. Repeat allows you to extend the . command to work with user created commands. It works with surround and commentary out of the box, but you can also set it up to work with your own commands. The . command is one of the most important Vim commands, so being able to use it in more situations is definitely a win.
vim-text-obj-user is an interesting project to make it easy to create plugins that define custom text objects which then can be used with Vim commands like any other text object. If that sounds scary to you, you can also take a look at the list of plugins already built with the project, and use any that will be useful to you.

Help Vim Look Good

One of the best uses for plugins is making Vim a bit easier on the eyes. We can do a lot to change Vim’s default look and feel. For instance the screenshot above shows what MacVim looks like with and without loading my ~/.vimrc file and plugins. If you’re a writer or developer, you may be looking at Vim for as much as 8-10 hours a day. So make it something you’re happy to look at.

Syntax highlighting plugins are one of the main things you can use to make your code easier to read. Vim packages syntax highlighting for many common languages, but for less mainstream languages you’ll need to download plugins to get syntax highlighting. These plugins often may contain other niceties for the language, like syntax checking or support for compiling from within Vim. For instance I use plugins for Less, CoffeeScript, and Markdown.
Colorschemes are also an important part of helping Vim look its best. Most distributions of Vim bundle several colorschemes, but the most popular ones come as plugins. If you’re somebody who needs to know that somebody else has sweated the details of the tools you use so that you don’t have to, I recommend Solarized, a color scheme based on the relationships between different colors and designed to echo the readability of words on paper. If you like to try different things and have more options, consider looking at Base16, a project to standardize color scheme creation for multiple editors. The advantage of both of these schemes is that they’re available for pretty much any editor or tool you’d want to use alongside or in place of Vim.
Thematic is a tool to allow you to manage your visual schemes. So if you like to use different color schemes or fonts in different situations, Thematic can make that easy. You can define a set of “themes”, and then switch between them whenever you feel like it. As somebody who uses Vim for both writing and programming, it’s nice to be able to use different themes for different situations. I currently use Solarized for code, and the Pencil theme for writing. Thematic and Pencil are two of a wide variety of plugins for writers that Reed Esau has written.
Airline is a plugin to pretty up your status bar. It creates the nice triangular effects in the image above, and integrates with other plugins to show helpful information in the status bar. It does require modified fonts to use though, so you’ll need to have pretty full control of the system you’re on to use this one.

Help Vim Understand Your Code

Vim is not an IDE. By default it doesn’t understand much about code, and very little about specific languages. It offloads that knowledge to the operating system, through things like :make and ctags, and to plugins. Here are a few plugins that can help make Vim a bit more “code aware”.

Syntastic is a syntax checking plugin for Vim that integrates with external syntax checkers to provide in-editor feedback on your code as you make changes. For instance, it can integrate with jshint, llvm or javac to provide immediate feedback on a file.
YouCompleteMe is a code completion engine for Vim that builds on Vim’s built in autocomplete features to offer “smart completion” for as many situations as possible. It can integrate with Clang, Jedi, Omnisharp, and Vim’s omnicompletion in order to bring smart autocomplete to a variety of languages.
Fugitive and Git-gutter are 2 plugins that bring Git integration into Vim. Fugitive provides a wrapper around the Git executable to allow you to easily manage your git actions from within Vim. GitGutter focuses on individual files and lets you see where you’ve made uncommited modifications in the current file. They complement each other well, and are both very useful for heavy Git users.
delimitMate is a simple plugin that removes a huge annoyance by auto inserting matching braces and brackets in a smart, intuitive way. Lack of auto brace insertion was a big mental hurdle for me coming from Sublime Text. DelimitMate makes things work the way I expect.
Language specific plugins are another plugin group worth investigating. There are way too many examples to list here, but one that I use and enjoy is TernJs, a code intelligence plugin for Javascript that you can use with something like YouCompleteMe to get the type of IDE style autocompletion that is very rare for Javascript code.

Help Vim Play Nice With Others

Some plugins exist to provide tighter integration between Vim and external programs. We’ve already touched on a few of those, like Ack.vim and Syntastic. Vim is intended to follow the Unix philosophy of doing one thing well and working well with other programs. Integration plugins allow for deeper versions of that.

Tmux Navigator is a must have plugin for Tmux users. It allows you to easily move between tmux and Vim splits using a single set of keyboard mappings. It lets Vim and tmux work together in a completely natural way, without the user having to keep track of where one ends and the other begins.
Dash.vim is an OSX only plugin that provides integration with Dash.app, a documentation manager app for OSX, making it easy to look up library method definitions from within Vim.

Grant Vim Superpowers

The last category of plugins are scripts that allow Vim to do things that are totally different than anything it can do on its own. These may be features pulled from other editors, or features inspired by Vim but implemented differently than the Vim core. These can be pretty amazing, but I’d be careful with them to start. You’ll want to learn how Vim works before piling a bunch of big extensions on top of it.

Ultisnips allows Vim to use Textmate like “snippets” that allow you to save canned templates of code or text, then insert and modify them whenever you’re ready. If this sounds interesting, I’d recommend checking out the recent Vimcast episodes where Drew explores Ultisnips.
EasyMotion is a plugin that introduces a new type of movement to Vim. It creates alternative versions of the Vim movements like f and w that allow you to find a character or word anywhere on the screen, and specify it by using a key character that temporarily appears on the screen in place of each possible target character. If that’s hard to follow, check out the demo.

More Resources

VimAwesome is a recent project to collect popular Vim plugins in a beautiful, searchable format. It lists some categories similar to what I have above and sorts plugins by popularity. It seems to pull rankings based on mentions in Github dotfiles, which is a creative and useful way of ranking plugins.
Vimcasts has a bunch of plugin related episodes, many of which touch on some of the plugins mentioned in this article
Plugins are great, but only have real value if you understand how Vim can make you productive. If you haven’t seen them yet, consider checking out the other posts in this series to learn how Vim works before loading up on plugins.

Learning Vim in 2014: Vim as Art

Wed, 16 Jul 2014 06:41:00 -0400

How do you feel about being efficient? If you’re reading this blog, I bet you like it. What if I told you your text editor could help you cut the amount of time you spend typing in half, with just a little reading and a bit of practice? For you Vim users, it’s very possible that you’re spending way more time typing than needed.

The software developers reading this have probably heard of the Don’t Repeat Yourself principle. Developers want to avoid duplicating work, and try to write each distinct meaningful chunk of code only once. DRY is extremely powerful when it comes to code, but it’s also a very useful principle in text editing. Much of the editing work we do is extremely repetitive. Vim can help you remove those inefficiencies.

If Vim’s composable commands are the language of Vim, then repeating commands are it’s art. For pretty much any thing you do in Vim, there are ways to automate and repeat it. These “repeating commands” are powerful tools, but it’s not always easy to know what to do with them. There aren’t simple rules you can learn for when to use repeating commands. Instead it takes practice, awareness, and experience.

All of Vim’s commands are designed to be repeatable and undoable. So if you find yourself typing something a lot, there is likely to be a good solution for making it easier and faster. I’m going to lay out the tools for doing that, along with a few hints for knowing when to use them.

Repetition Basics: The dot command

The most important Vim tool for repetition is the dot command. Pressing . at any time will repeat the last native command you used in Vim. How is this useful? Let’s say we’re faced with the following code

var widget = new Widget();
var widgets = new WidgetCollection();
widgets.push(widget);
if(widget.test()) {
    sendMessage('widget passed test');
}

If we want to rename the widget variable to be more informative, how could we go about it? We’ll ignore “language-wise” refactoring solutions, like ternjs, and look at the basic Vim options.

A simple find and replace causes problems. We can’t distinguish between the widget we want and other occurences. :%s/widget/testWidget/g results in plenty of unwanted changes.

var testWidget = new Widget();
var testWidgets = new WidgetCollection();
testWidgets.push(testWidget);
if(testWidget.test()) {
    sendMessage('testWidget passed test');
}

It might be possible to find a complex regex that only hit the variables we want, but thats suddenly a lot of thought for what should be a simple change. We could go one by one and knock out each word one by one. But that would be a lot of repeated typing of testWidget. So what can we do?

The dot command is a nice solution. If we start on the first occurence of widget, we can use ciw to replace the current word with new text. When we’ve added our new text and exit insert mode, we’ll have a command, which we can then repeat with .. After that we can move to the next occurence we want to change and hit . to replace it without retyping the variable name. If we accidentally use it in the wrong place, it’s easy to undo with u.

Of course it would still be a pain to navigate between each widget with hjkl. Fortunately, Vim has more commands that allow us to avoid unecessary work. If we start on one instance of widget, we can use * to search for all occurences of that word. We can then use n to move from occurence to occurence. If we overshoot and go too far, we can use N to go backwards. So if we start with * (shift+8 in the gif below) and then use ciw to replace the word, we can then use n and . to clean up the other widgets in only 5 keystrokes.

The Vim Way

If you’re used to another editor, I’m guessing you’re justifiably a bit skeptical at this point. After all, this example is something people do all the time, and many editors have tools to make it easy (multiple selections in Sublime Text and refactoring tools in language specific IDEs like Eclipse for instance). They may take even fewer keystrokes than Vim. And they don’t require you to learn 6 separate commands to use well. So what’s so special here?

Let’s have a quick refresher on the philosophy of Vim. Like the Unix ecosystem it came from, Vim provides small, composable tools that you can combine to create powerful actions. The power isn’t in a specific use case. Instead it comes from the range of possibilities that open up when we start mixing and matching the code. As with verbs, movements, and text objects, repeating commands and their corresponding reverse commands are tools we can add to our toolbox. Using the commands above and some of the other composable commands from my language post, we can do some cool things.

We can:

Search through json files and selectively remove lines that contain an attribute we don’t want to use (start on the attribute, *,dd, then use n and . repeatedly to select lines to delete)
Replace a long, hard to type function name throughout the document with another version without having to type out the function name (start on the function name, *,:%s//<newname>/g)
Fix poorly indented lines using the . command (>> or <<, . repeatedly, u to undo if you go too far. This can easily be used on groups of lines using visual mode too)

What do all of these have in common? They all use a command once, and a movement at most once, then allow for as much repetition as you like. In Practical Vim, Drew Neil encourages actively thinking about making your actions repeatable. In many scenarios, once you’ve made a change and movement once, you can then get by with a single keystroke for each future action and movement, with another keystroke available to cancel an action if you go too far. Here’s a partial list of repeating commands, with the corresponding cancel command to undo them.

Repeating Command	Purpose	Canceling Command
.	Repeat the last normal mode command	u
n	Repeat the last Search	N
;	Repeat the last character-wise search	,
//	Repeat the last search	N/A
@:	Repeat the last ex-mode command	N/A
@@	Repeat the last Macro used	u

None of these commands are all that useful on their own, but they’re each valuable things to learn as you build up your Vim toolbox. Focus on one or two to start, and learn to make edits so that you can easily repeat them. The art of Vim is learning to look for those opportunities, and crafting your commands so that they’re short, powerful, and repeatable.

More Resources

Some of you are probably confused that I wrote a whole post on repeating commands in Vim with only one side-reference to Macros. Macros are Vim’s DRY power tool, and different enough from the rest of these commands that I didn’t want to lump it all together. But for those interested, Drew Neil has a great screencast on how to build a good Vim macro.
On a similar note, Drew’s book Practical Vim is the best guide I’ve found on making good use of repetition in Vim. I learned plenty of the stuff in this article from that book. For those who don’t want to pay, I’m currently giving new subscribers a chance to win 2 copies of the book. You can also see more about repetition from Drew online at Vimcasts.
Tim Pope has published a plugin that allows you to extend the dot command to work on arbitrary user commands. It also works out of the box with many of his other plugins. You can see it on Github.

Learning Vim in 2014: Configuring Vim

Mon, 14 Jul 2014 06:50:00 -0400

Plenty of Vim articles you’ll find online start you off in the wrong place. Instead of teaching you how Vim works, they toss a bunch of scary looking code at you, tell you to shove it in a file, and claim that it will make Vim just like Textmate / Sublime / Emacs / Visual Studio. Except way cooler! People quickly find out that if what they want is to use their old editor, they should just use their old editor, and learn to use it as efficiently as possible.

While writing about Vim the last few weeks, I’ve avoided recommending any configuration changes, other than the quick sanity vimrc I recommended in the first article, because I think it’s important to learn what sets Vim apart before trying to configure it. Not because it’s good enough without configuration (honestly it’s not), but because once you understand the core, you can use configuration to build a great version of Vim instead of trying to hack together a Rube-Goldberg Textmate clone. But now that I’ve covered some of the basics (and if you haven’t yet, feel free to go ahead and start at the beginning), it’s time to look at Vim configuration.

Basic Configuration

Configuration for Vim starts with a vimrc file. It’s a text file located by default at ~/.vimrc on *nix systems, and $VIM\_vimrc on Windows. To begin configuration you need to open that file, creating it if it doesn’t exist. But before we go further, we should stop and figure out what we’re doing.

I said that configuration starts with the vimrc file, but the truth is that your vimrc is just a normal text file that Vim sources by default. The heart of Vim configuration is Vimscript, Vim’s custom scripting language. Vimscript is a full programming language, with variables and control structures. But for this article we’re going to focus on using it for configuration.

The basic commands that Vimscript uses for configuration are let, set, and map. let allows you to change a variable value with the syntax let <var-name>=<value>. set allows you to read or change an option value using set <option-name> for toggle options or set <option-name>=<option-value> for numeric or string-based options. map and its many variations allow you to create new keymappings by setting the output of one key combination to produce the value of another key combination. You can use variations on map to specify how a key sequence should behave in different vim modes.

Know what you’re configuring

Now that you know the basic commands, I could show you an example .vimrc, explain it a bit and then tell you to copy it, change the stuff you don’t like, and go on your way. The problem with that? It leaves your vimrc file as a “pile of junk”, that I understand and you don’t. If I did a good job, you might understand the purpose of individual lines, but they probably wouldn’t be the best fit for you, and you probably wouldn’t know exactly how to make it better. In the end you’ll keep tossing junk onto it until it becomes a confusing mess you want to avoid. So instead, I’m going to walk through the types of configurations you might want to consider, give a few examples, and let you build from there. So here are a few concepts to think about as you construct your vimrc file:

Adapt to your environment

Vi and Vim were originally developed in a low-resource terminal based environment. Many of their default settings and assumptions reflect that. Today though Vim is used in a wide variety of ways, from multimonitor Mac Pros with 16+GB of memory running multiple MacVim windows, to low memory remote servers running Vim in the terminal over ssh. If you know what your environment will look like, you can optimize Vim for your own use.

For instance, if you’re running in a high resource environment, (and pretty much any modern laptop or desktop these days qualifies as a high resource environment) you may want to enable some features that Vim disables by default, since those removing those features allow it to work well in very low memory situations. Some common choices:

syntax enable " Turn on syntax highlighting
set hidden " Leave hidden buffers open
set history=100 "by default Vim saves your last 8 commands.  We can handle more

On the other hand, if you know you’re going to be running remotely on low resource machines, you might choose to disable those options to minimize the load and increase speed. I’ll talk more about plugins in a later post, but they’re another thing that works much better in your local environment than on remote servers, for both performance and logistical reasons.

Add some visual niceties

If we’re honest, I think most people can agree Vim is pretty hard to look at out of the box. Very few visual affordances, no syntax highlighting or line numbers, and it’s style is dependent on your terminal color scheme or an ugly default gVim theme. But that’s pretty easy to fix. We can get syntax highlighting and line numbers with this snippet:

syntax enable
set number

Getting a decent color scheme will require you to download it as a plugin, but if you’ve done that or if you like one of the colorschemes that ship with your Vim distribution it’s easy to set up with colorscheme <theme-name>. Beyond that, there’s plenty more you can do to improve the visual feel of Vim. I’d look into customizing your status bar, relative vs absolute line numbers, and plugins like Airline and Vim-Thematic.

Set up keybindings

Many of Vim’s useful commands are either not given default keybindings, or are given hard to remember obtuse ones. Don’t want to spend your whole life typing source ~/.vimrc into each file everytime you add something to your configuration? I shortcut that to \rr.

map <leader>rr :source ~/.vimrc<CR>

You might be wondering what the notation is there. The <leader> key is a Vim convention for a key that comes before user created keystrokes. It’s meant to minimize conflicts, and allow plugins to provide default mappings while still allowing the user a bit of flexibility. The leader key can be set in your vimrc with the command let mapleader=<yourleaderkey>. The default is \, and I would recommend leaving it that way to start. You can change it later if you find it to be inconvenient, and it doesn’t cause any conflicts since its the default key. Common replacements like , overwrite useful functionality that may not be immediately obvious (, allows you to move backwards on a characterwise search for instance).

Configure plugins and file-type specific settings

I’ll touch more on plugins in a future post, but many of them allow extensive configuration, including setting keybindings, changing default behaviors, and enabling/disabling extra functionality. All of that happens here in the vimrc and should mostly use the let/set/map commands that we’ve already seen.

You can also set file-type specific rules. Rather than placing these in your main vimrc, these go into a special “file type plugins” directory, ~/.vim/ftplugin. Within that directory, you can create files like javascript.vim, python.vim or html.vim, and those files will be loaded after the vimrc when you open files of that type. Using those files, you can create completely separate profiles for each file type.

Discover what’s out there

I’ve intentionally avoided telling you what your vimrc should look like here. I’ve tried to show you the big picture of what you can do, and now I’ll leave it to you to figure out what pieces you need to be comfortable and productive. It will be different for everybody.

My advice? Now that you know what you’re looking for, go and take a look at other people’s configurations. Start with something like this vimrc boilerplate, and add the things you like.

It’s pretty easy to find good configurations online if you look. My vimrc file is on Github. It’s certainly not the best out there, but its relatively small and easy to follow. If you know developers you admire who use Vim, ask to see their vimrc files, or see if they have any fancy tricks to share. For those without those resources, Github’s dotfile page is a great place to look. It contains links to several very detailed “dotfile repos” from various developers. You don’t have to copy everything from these files (in fact you shouldn’t), but you can pull the things that look useful, get rid of things that don’t, and try things out as you slowly build up your own workspace, built to do the jobs that you need it to do.

More Resources

Github’s dotfile page provides some great pointers to dotfile repositories from experienced developers that can help you come up with cool configuration ideas
Janus is a great resource for users who’d prefer to avoid configuration and just start with a useable editor. Though if that’s you, you may find you’re better off just using Sublime Text

New Twitter Feed and Practical Vim Giveaway

Fri, 11 Jul 2014 07:12:00 -0400

I’ve got a few orders of site business to deal with today. First, there’s a new way to subscribe to the site.

New Twitter Feed

The new feed, @benmccormickorg, will be a Twitter “RSS Feed” with all of the posts from this site. It’s just another option for people who want to subscribe, along with RSS and my mailing list. You can also find my personal account at @_benmccormick.

Follow @benmccormickorg

Practical Vim Giveaway

To go along with that, and as a thanks for all of the great interest and feedback I’ve gotten for my posts on Vim recently, I’m going to be giving away 2 copies of Drew Neil’s book, Practical Vim.

So how do you get a copy? There are a few ways

Follow @benmccormickorg on Twitter
Subscribe to my mailing list feed.
Tweet a link to one of my Learning Vim posts, with the hashtag #learningvim

Book Review: Building Backbone Plugins

Wed, 09 Jul 2014 07:38:40 -0400

As the saying goes, judging a book by its cover is a dangerous business. A lot of books have titles that promise great things, and content that under-delivers and disappoints. Every once in a while, though, you find a gem that does the opposite. Building Backbone Plugins sounds like a niche book for library developers. But it actually is the best resource on Backbone.JS development I’ve found, a book that will benefit anyone who works with Backbone.

So what’s the fuss about? The book’s author, Derick Bailey, is the creator of Marionette.JS, a popular application framework for Backbone Apps. Building Backbone Plugins explores the problems that Derick tried to solve with Marionette and then explains the mindset behind Marionette’s solutions. It is not a book exclusively about Marionette though. The solutions in the book are example code that represents a simplified view of what Marionette does. Instead the book looks at the common problems any developer will encounter in a Backbone project, and provides a possible solution, while working hard to generalize the lessons and concepts as much as possible.

Building Backbone Plugins is divided into 4 parts. The first two look at Backbone Views and dealing with the DOM. The third looks at Models and Collections. The last part deals with application structure and events. These are followed by a series of appendices that read as stretched-out blog posts on a variety of related topics that didn’t quite fit into one of the 4 buckets that the rest of the book falls into.

In my mind, the first two parts are the heart of the book. They’re a step-by-step look at how you can remove repetitive code from your Backbone Views and structure your application to expose the developers intent in each file, rather than burying the purpose in boilerplate. I learned a lot about how I could improve my Backbone code, and these two sections actually inspired me to start integrating Marionette into a project at work. The application structure advice in the fourth part is also particularly strong, with quality information about events, and ideas on how to scale a Backbone application beyond TodoMVC sizes.

On the other hand, the appendices feel a bit tacked on, and seem to be full of stuff the author wanted to add but couldn’t figure out how to weave into the rest of the book. Since it’s legitimate content rather than simply greater detail on things that have been already covered, they don’t act as real appendices and leave the book feeling a bit unfocused at the end. When you combine that with some weaker content in part 3 (where the author moves away from Marionette into some other smaller examples), the book can feel a bit choppy and directionless. But the goldmine of knowledge and perspective in Parts 1, 2 and 4 more than make up for it.

I would not recommend BBP as a “first book” on Backbone. It assumes plenty of knowledge, and doesn’t attempt to teach Backbone basics. But for an intermediate level Backbone developer, or any Javascript developer who wants to grow their thinking about application structure, there is plenty of solid insight here. Meditations on handling nested views, cleaning up unused views, abstracting out repetitive boilerplate, and structuring genuinely large applications are all worth the price of the book. Finally, if you’re writing code using Marionette already, this book is a must-have. Marionette’s documentation does a great job of explaining the “what” and the “how” of Marionette. But this book explains the all-important “why” of a framework that is based on encoding Backbone best practices into a library. It will help you think like an experienced Backbone developer.

More Resources

Building Backbone Plugins - You can purchase the book on leanpub. (Note: Affiliate Link)
Developing Backbone Applications - This book by Addy Osmani is a great resource for Backbone Beginners who want a more basic introduction to Backbone concepts. It’s available as a free HTMl site on github, but you also can purchase it as a paperback or ebook.
Marionette.js - This framework serves as the inspiration for Derick’s book and is a great way to structure larger Backbone apps.

Learning Vim in 2014: Working with Files

Mon, 07 Jul 2014 07:10:37 -0400

As a software developer, you shouldn’t have to spend time thinking about how to get to the code you want to edit. One of the messiest parts of my transition to using Vim full time was its way of dealing with files. Coming to Vim after primarily using Eclipse and Sublime Text, it frustrated me that Vim doesn’t bundle a persistent file system viewer, and the built-in ways of opening and switching files always felt extremely painful.

At this point I appreciate the depth of Vim’s file management features. I’ve put together a system that works for me even better than more visual editors once did. Because it’s purely keyboard based, it allows me to move through my code much faster. That took some time though, and involves several plugins. But the first step was me understanding Vim’s built in options for dealing with files. This post will be looking at the most important structures Vim provides you for file management, with a quick peek at some of the more advanced features you can get through plugins.

The Basics: Opening a new file

One of the biggest obstacles to learning Vim is its lack of visual affordances. Unlike modern GUI based editors, there is no obvious way to do anything when you open a new instance of Vim in the terminal. Everything is done through keyboard commands, and while that ends up being more efficient for experienced users, new Vim users will find themselves looking up even basic commands routinely. So lets start with the basics.

The command to open a new file in Vim is :e <filename>. :e opens up a new buffer with the contents of the file inside. If the file doesn’t exist yet it opens up an empty buffer and will write to the file location you specify once you make changes and save. Buffers are Vim’s term for a “block of text stored in memory”. That text can be associated with an existing file or not, but there will be one buffer for each file you have open.

After you open a file and make changes, you can save the contents of the buffer back to the file with the write command :w. If the buffer is not yet associated with a file or you want to save to a different location, you can save to a specific file with :w <filename>. You may need to add a ! and use :w! <filename> if you’re overwriting an existing file.

This is the survival level knowledge for dealing with Vim files. Plenty of developers get by with just these commands, and its technically all you need. But Vim offers a lot more for those who dig a bit deeper.

Buffer Management

Moving beyond the basics, let’s talk some more about buffers. Vim handles open files a bit differently than other editors. Rather than leaving all open files visible as tabs, or only allowing you to have one file open at a time, Vim allows you to have multiple buffers open. Some of these may be visible while others are not. You can view a list of all open buffers at any time with :ls. This shows each open buffer, along with their buffer number. You can then switch to a specific buffer with the :b <buffer-number> command, or move in order along the list with the :bnext and :bprevious commands. (these can be shortened to :bn and :bp respectively).

While these commands are the fundamental Vim solutions for managing buffers, I’ve found that they don’t map well to my own way of thinking about files. I don’t want to care about the order of buffers, I just want to go to the file I’m thinking about, or maybe to the file I was just in before the current one. So while its important to understand Vim’s underlying buffer model, I wouldn’t necessarily recommend its builtin commands as your main file management strategy. There are more powerful options available.

Splits

One of the best parts of managing files in Vim is its splits. With Vim, you can split your current window into 2 windows at any time, and then resize and arrange them into any configuration you like. Its not unusual for me to have 6 files open at a given time, each with its own small split of the window.

You can open a new split with :sp <filename> or :vs <filename>, for horizontal and vertical splits respectively. There are keyword commands you can use to then resize the windows the way you want them, but to be honest this is the one Vim task I prefer to do with my mouse. A mouse gives me more precision without having to guess the number of columns I want or fiddle back and forth between 2 widths.

After you create some splits, you can switch back and forth between them with ctrl-w [h|j|k|l]. This is a bit clunky though, and it’s important for common operations to be efficient and easy. If you use splits heavily, I would personally recommend aliasing these commands to ctrl-h ctrl-j etc in your .vimrc using this snippet.

nnoremap <C-J> <C-W><C-J> "Ctrl-j to move down a split
nnoremap <C-K> <C-W><C-K> "Ctrl-k to move up a split
nnoremap <C-L> <C-W><C-L> "Ctrl-l to move	right a split
nnoremap <C-H> <C-W><C-H> "Ctrl-h to move left a split

The jumplist

Splits solve the problem of viewing multiple related files at a time, but we still haven’t seen a satisfactory solution for moving quickly between open and hidden files. The jumplist is one tool you can use for that.

The jumplist is one of those Vim features that can appear weird or even useless at first. Vim keeps track of every motion command and file switch you make as you’re editing files. Every time you “jump” from one place to another in a split, Vim adds an entry to the jumplist. While this may initially seem like a small thing, it becomes powerful when you’re switching files a lot, or moving around in a large file. Instead of having to remember your place, or worry about what file you were in, you can instead retrace your footsteps quickly using some quick key commands. Ctrl-o allows you to jump back to your last jump location. Repeating it multiple times allows you to quickly jump back to the last file or code chunk you were working on, without having to keep the details of where that code is in your head. You can then move back up the chain with ctrl-i. This turns out to be immensely powerful when you’re moving around in code quickly, debugging a problem in multiple files or flipping back and forth between 2 files. Instead of typing file names or remembering buffer numbers, you can just move up and down the existing path. It’s not the answer to everything, but like other Vim concepts, it’s a small focused tool that adds to the overall power of the editor without trying to do everything.

Plugins

So let’s be real, if you’re coming to Vim from something like Sublime Text or Atom, there’s a good chance all of this looks a bit arcane, scary, and inefficient. “Why would I want to type the full path to open a file when Sublime has fuzzy finding?” “How can I get a view of a project’s structure without a sidebar to show the directory tree?” Legitimate questions. The good news is that Vim has solutions. They’re just not baked into the Vim core. I’ll touch more on Vim configuration and plugins in later posts, but for now here’s a pointer to 3 helpful plugins that you can use to get Sublime-like file management.

CtrlP is a fuzzy finding file search similar to Sublime’s “Go to Anything” bar. It’s lightning fast and pretty configurable. I use it as my main way of opening new files. With it I only need to know part of the file name and don’t need to memorize my project’s directory structure.
The NERDTree is a “file navigation drawer” plugin that replicates the side file navigation that many editors have. I actually rarely use it, as fuzzy search always seems faster to me. But it can be useful coming into a project, when you’re trying to learn the project structure and see what’s available. NERDTree is immensely configurable, and also replaces Vim’s built in directory tools when installed.
Ack.vim is a code search plugin for Vim that allows you to search across your project for text expressions. It acts as a light wrapper around Ack or Ag, 2 great code search tools, and allows you to quickly jump to any occurrence of a search term in your project.

Between it’s core and its plugin ecosystem, Vim offers enough tools to allow you to craft your workflow anyway you want. File management is a key part of a good software development system, and it’s worth experimenting to get it right.

Start with the basics for long enough to understand them, and then start adding tools on top until you find a comfortable workflow. It will all be worth it when you’re able to seamlessly move to the code you want to work on without the mental overhead of figuring out how to get there.

More Resources

Seamlessly Navigate Vim & Tmux Splits - This is a must read for anyone who wants to use vim with tmux. It presents an easy system for treating Vim and Tmux splits as equals, and moving between them easily.
Using Tab Pages - One file management feature I didn’t cover, since it’s poorly named and a bit confusing to use, is Vim’s “tab” feature. This post on the Vim wiki gives a good overview of how you can use “tab pages” to have multiple views of your current workspace
Vimcasts: The edit command - Vimcasts in general is a great resource for anyone learning Vim, but this screenshot does a good job of covering the file opening basics mentioned above, with some suggestions on improving the builtin workflow

Learning Vim in 2014: Vim as Language

Wed, 02 Jul 2014 07:27:00 -0400

Wouldn’t it be nice if your text editor just did what you said instead of making you slowly and manually add and delete characters? Vim doesn’t speak English, but it has a language of its own, built out of composable commands, that is much more efficient than the simple movement and editing commands you’ll find in other editors. In my last post, I took an initial look at Vim as a language. I’m going to dive deeper into that here.

Vim Verbs: What can you do?

Vim’s “verbs” mostly fall into 2 main categories. Some of them act on a single character, and others act on a “motion” or “text object”. We’ll look at motions in a second, but lets start by looking at the verbs.

Single character verbs

So like I said, there are a few vim actions that act on a single character. They act as shortcuts for actions that you can also perform with motions, and allow you to save a few keystrokes.

Command	Action
x	Delete character under the cursor
r	Replace character under cursor with another character
s	Delete character under cursor and move to insert mode

These are great commands to know, and things that I use daily, but they act as a bit of an island. Let’s look at some verbs with more power.

Command	Action
d	Delete text specified by motion
c	Delete text specified by motion and go into insert mode
y	Yank (copy) text specified by motion

These aren’t the only 3 actions that you can use with motions, but they’re probably the most important. The first 2 are roughly the same as deleting or cutting in most instances, with the option to choose what mode you end in. The 3rd is Vim’s version of copying.

Motions

So how do we use these actions? You’ll notice that if you type any of the above characters into Vim by themselves, nothing happens. That’s because they’re expecting something to act on. If you’re a grammar nerd or remember your 8th grade english classes, you can think of these as transitive verbs that need to act on a direct object. These “direct objects” come in 2 forms, motions and text objects. Motions are the motion commands that you can use at any time to move around Vim. Some examples:

Command	Motion
$	Go to the end of the line
G	Go to the end of the file
f.	Go to the next occurrence of `.` on the current line

All of these commands work as motions on their own, but when you combine them with actions you can get powerful effects. So d$ means “delete to the end of the line”. cf) means “change through the next closing parentheses” and yG means copy everything through the end of the file. The list above is only a small subset of the motions available. I’d suggest learning 1 or 2 to start and using them when appropriate. As you get more comfortable with Vim you can continue to add more, and each one will unlock a set of commands for all of the actions you know.

The second type of “direct object” that Vim verbs can take is a text object. You can think of text objects as a “defined chunk of text.” Some examples include selecting words, html tag contents, or the contents of a function.

Command	Text Object
iw	Applies to everything in the current word
it	Applies to everything in current xml/html tag
i{	Applies to everything inside nearest curly brackets

So using text objects you can change the contents of a word with the easily memorizable ciw (change in word), or copy the contents of an html <a> tag with yit. It’s worth mentioning here that one of the most powerful text objects is a bit of an oddball. A really common thing people do is applying an action to the current line. Vi’s language makes this easy by allowing the shortcut for this text object to be a simple repeat of the action. So yy yanks the current line, dd deletes it, and cc changes it.

Piece by Piece

The key here is that learning Vim isn’t something you do in a weekend. It’s an iterative process. Since the actions, motions, and text objects build on themselves, you can start to grow your toolbox exponentially as time goes on. But you’ll want to start with a few commands that you find make you productive. I’ve personally found ciw, dd and ci( to be incredibly useful commands, and a great place to start as you learn to use these commands instead of the slower “mouse and select” methods that you may be used to. When we learn languages we all start with basic sentences, learn the grammar rules and grow from there. Vim is no different. Start small and add to your toolbox as you build great things with it.

More Resources

Your problem with Vim is that you don’t grok vi - A concise explanation of what makes vi/vim special, and one of the most popular answers ever on Stack Overflow.
Why Atom can’t replace Vim - A great piece on the big picture importance of composable commands.
Vim Text Objects: The Definitive Guide - This is a nice rundown of the different text objects available, including some that you can add using plugins.

Update

This post is currently on the front page of Hacker News. Feel free to join in the conversation there

Learning Vim in 2014: The Basics

Mon, 30 Jun 2014 07:28:00 -0400

Do you want to be efficient writing code? Using Vim can be a powerful help. Using it has let me move more effectively through the process of writing and editing code. But getting to where I am now was frustrating. Vim resources are scattered across the web, and contain a lot of different advice. It’s tough to figure out how to make Vim work for you. In my next few posts, I’m going to talk about the process of learning a 23 year old programming power tool, and what you need to know in 2014 to be productive with Vim. Today I’ll start with the basics, stuff that goes back all the way to the 1970s, and we’ll grow from there.

Why has Vim Survived?

Vim has been around for a while. Originally released in 1991 as an improvement on vi (which had been around since 1976), it has outlasted a generation of programmers. The software world has changed a lot since 1991. So why do people still use it?

Vim provides a system for editing text that is quite simply more powerful than its modern graphical based alternatives. It asks a lot of its users. To use Vim effectively you have to internalize the syntax for many commands that have better affordances on other editors. But you’re rewarded with a friction-free editing environment. Vi’s commands are fast, composable, and powerful. So let’s get started.

Step 0: Setup

The first thing you need to do is install a copy of Vim. If you’re on a Linux or Mac system, there is likely already a system copy, but it is probably not up to date. So use your system’s package manager to get the latest version (7.4 at the time of this writing). For Mac users I recommend using Homebrew to download MacVim. Windows users can download the latest executable on Vim.org.

The second thing you’ll need is a vimrc (the file vim uses for configuration). These files can become quite extensive, setting up many different options, configurations, and plugins. But we’re going to start simple. Download this minimal vimrc file and install it in the appropriate directory for your OS as instructed. If you already have a .vimrc file but don’t understand it, I’d recommend saving it elsewhere and using a minimal one for now. You can then build it out as you learn more.

Step 1: Learn the Syntax

Just like a programming language, the first thing to learn for Vim is the basic syntax. Vim has a built in tool to help you get comfortable called vimtutor. It’s a text file set up to help you learn to navigate Vim. Once vim is installed you can run it by calling vimtutor on unix-style systems. On Windows you can follow the instructions here. It will take you through a series of 7 lessons that will familiarize you with the basic commands.

Vim has rather non-standard commands, and it may be frustrating at first adapting to its different terminology and conventions. The payoff comes slowly, but it does come.

Modes

The most important thing to understand is that Vim is a modal editor. At any given time you’re in one of 6 modes. You will have different abilities and keystrokes available to you in each of these modes. For right now the key modes to understand are Normal Mode, Insert Mode, and Command line Mode.

Normal Mode is the default mode. It’s roughly equivalent to the state that other editors are in when you’re holding down the ctrl or cmd keys. Instead of entering text onto the screen, different keys trigger different commands. Initially this is a bit weird. Those of us coming from other editors or IDEs are used to being able to type text by default. The thought is that adding text is the primary task in a text editor. But Vim treats adding text as sitting on equal footing with editing, deleting and manipulating existing text, and starts us in a mode where we can quickly make any change. To steal an analogy from Drew Neil, normal mode is the opportunity to take our paintbrush off the canvas while we decide our next stroke.

Insert mode is the equivalent of most editor’s normal state. When you type a key in normal mode, it appears on the screen. You can enter insert mode by pressing i in visual mode, and exit it by hitting <esc>.

Command line mode lets you enter ex-commands, a command line language that complements normal modes shortcuts. You enter it by hitting : in normal mode. Important commands are :w to write (save) a file, and :q! to quit.

The above commands, and an understanding of the modes are enough for you to survive in Vim, and treat it as a sort of awkward Notepad clone. You can edit files, save them, and quit. Going through vimtutor will introduce you to many more commands, and a few more modes. But you can learn those at your own pace. Right now you want to see what taking time to learn this awkward unfamiliar syntax can buy you.

Step 2: Learn the Language

Vim’s key commands are different than you’ll find in other editors like Sublime Text. They’re not a series of standalone actions that you can do one after another to get what you want. Instead they form a language.

Vim statements are made up of actions and motions. The actions are what you’re trying to do, the motions are where you want to do them.

For example let’s take the d action. d deletes. You can delete a single character by typing dl. You can delete 2 characters by typing d2l. You can delete a whole line by typing dd, or delete inside a word by typing diw.

Once you learn the d action, you can use it with every motion you know. Similarly, when you learn a new motion, you can then use it with every action you know.

So if somebody shows me the y action and tells me that it yanks (copies) text, I’ll know I can yank a character with yl a line with yy and inside the current word with yiw. Each new Vim command is a tool in your toolbox, and since the tools build off of each other they become exponentially more valuable.

Step 3: Learn the Mindset

Vim can make you hyper-efficient. But to get there you need to know more than just the command combinations available. You have to shift your mindset. Vim commands aren’t just easy to learn, they’re designed to be repeatable. Bram Moolenar, Vim’s creator has listed out 7 habits of effective text editing. The second habit, don’t type it twice, is a pretty good summary of the “Vim Way” of doing things.

Vim provides a bunch of ways to avoid repetition. I’ll get into them in another post. For now though, let’s take a look at ., the dot command. The dot command repeats the last action you’ve taken. For instance, you can use ciw to change a word, hit <esc> to return to normal mode, then move to another word and type . to replace that word as well. You are able to avoid retyping the replacement word as well as the ciw command. You can run the whole replace with one keystroke, regardless of what words you’re replacing. Like other Vim commands, the dot command is small on its own, but gains power as you add more tools to your toolbox.

More Resources

If you enjoyed this article you’ll probably also enjoy

Practical Vim: This amazing book by Drew Neil, the creator of Vimcasts was the single best resource I found as I learned Vim. I highly recommend it for anyone who uses Vim.
Everyone Who Tried to Convince Me to use Vim was Wrong: This blog post is another take on how to start learning Vim, by Yehuda Katz, the prolific developer who’s a core member of the Rails, EmberJS, and jQuery teams.
Janus: Janus is another Yehuda Katz project. It’s a starting configuration for Vim meant to make it feel more comfortable for developers who want to come in and be productive immediately. I personally found that it didn’t work for me. The plugin list overwhelmed me and prevented me from learning the basics. But it may be a better fit for some people and is a great place to look to see configuration ideas and plugins that you can try out on your own.

Discovering Vim

Mon, 02 Jun 2014 18:00:00 -0400

I’ve now been using Vim as my primary text editor for the last 9 months. When I started, I intended to learn it well enough to use in environments that didn’t support Sublime Text. In the end though it ended up becoming my single purpose editor for all occasions. Writing code, blogging, diffs, and reading logs all go through vim. So why do I use Vim?

More Efficient Movement and Editing

This is the first thing you hear about from experienced Vimmers, and its an important one. Once you have your feet under you as a Vim user, you are just able to edit faster. This can be hard to understand before using Vim (or other keyboard based tools). Using a mouse doesn’t feel particularly inefficient, and when you start out, Vim’s keyboard mappings can feel arbitrary, unintuitive and slow. This is one of those times where a little work is needed to get the proper perspective. Once you’ve had time to learn the basics, even a mediocre typist like me ¹ can fly when editing by typing smarter not harder. This isn’t a tutorial on the basics of Vim, but the keys for me were understanding Vim was a language in its own right, and the biggest gains in Vim come when you create repeatable options to eliminate inefficiencies. These can be macros, but can also be as simple as using Vim’s easy control for repeating the last action². Vim is a huge toolbox of small tools that you can use together to create a powerful workstation.

Rich Plugin Ecosystem

Like many things in Vim, the value of its plugin system wasn’t immediately obvious. Unlike Sublime, which has a community consensus primary package manager and repository, Vim has multiple package managers that pull packages from a variety of locations³. So you need to do some research to figure out what you should be using⁵ and find the right plugins. But when you do, you’ll find that Vim has fantastic plugins for key tasks like fuzzy file search, syntax checking, and in-file search. It also has a rich depth of plugins for pretty much every language/framework/functionality in common usage, as well as some great plugins that extend Vim’s built in motion/editing language.

Long Term Security

This one is a judgement call, but from my point of view, Vim is the safest investment of my time out there. It has a rich open source community, one that has shown that its willing to try some big ideas to keep from getting stale. It’s had a great 20 years of usage and doesn’t appear to be in decline. Sublime Text meanwhile, regrettably seems to be in danger of heading down a similar path to Textmate, with a single developer who has done great work, but doesn’t appear to be interested in being involved in all of the different work that a software business requires. I don’t think open source is the only way to rely on software⁴, but for general purpose text editors, Vim seems the safest bet to still be around in 10 years.

My Brain is Broken Now

This is a warning as much as a reason for using Vim, but I’ve quickly found that I’m no longer able to use Sublime or other editors without being frustrated by the fact that they’re not Vim. Many editors, including Sublime, provide support for a large subset of Vim’s key bindings and modal controls. But they all fall down at random places. The power of vim doesn’t lie in one specific functionality, but the composable nature of their different options. When some of them aren’t there, all the others are weakened, and my trained brain falls out of high efficiency mode into slow plodding “think through every keystroke mode.” So don’t say I didn’t warn you. Vim will break your brain.

Those are the primary reasons I use Vim now. There are a few things I haven’t loved though.

Vim’s UI is uglier than Sublime Text.

I’m a front end developer, used to working with, and working to create, beautiful interfaces. Vim is not beautiful. It looks like exactly what it is, a text based program with a lineage reaching back to the 70s. It doesn’t handle margins or padding well, doesn’t have pretty autocomplete menus or visual alerts. If you use it in the terminal you may be severely restricted in your color options, and this situation is not something that is going to change anytime soon⁶. I still love Sublime Text’s UI. Nice margins, smooth scrolling, the ability to set different font and syntax highlighting options on a filetype basis are all nice gems that Vim just doesn’t offer. I’ve come to appreciate Vim’s minimalism, but would love to see if Neovim were to offer a way to beautify some of these things.

Vim is harder to learn than it should be.

I’ve seen some Vimmers object to this (its a power user tool, etc.) but the truth is, when pretty much every Vim tutorial starts by telling you that you should start by changing a bunch of the default settings, it becomes clear that learning Vim is harder than it should be. There’s no reason that basics like line numbers and syntax highlighting shouldn’t be turned on by default. Configurability is great, but if you start out in a user-hostile state, many people won’t take the time to find the power. There have been some efforts to change that. Neovim seems to be actively rethinking this stuff, and projects like Janus aim to help people get started with a richer initial setup. But the very existence of these reforms points to the fact that Vim shoots itself in the foot with defaults that may have made sense in the eighties, but appear archaic next to modern competitors.

For now I’m enjoying the Vim experience. Despite a few warts, I really do think its the best text editor you can use today, and I’m excited to see where it goes in the future.

I was never taught to type correctly growing up. Apparently this is unusual in my generation. Early last year I decided to ditch my hunt and peck 5 finger style for "the proper way." It was a great learning experience. I now muddle along at a mediocre 62 WPM according to online typing tests. ↩
I highly recommend Drew Neil's Practical Vim for anyone who wants to get serious about using Vim. It taught me about the power of repetition in Vim as well as how to formulate Vim's many disparate features into a coherent text editing philosophy. ↩
Yes there's an official repository, but have you been on Vim.org's repository? Its an ugly frustrating site that does nothing to highlightuseful plugins. It also often contains information that is less up-to-date than the plugins github README, and some important packages like ctrlp are not represented there.
↩
I'd have no problem investing time in IntelliJ or Visual Studio for instance if I felt like a heavy IDE like that was the way to go, and that they'd add productivity to justify their cost. They have established companies backing them that are able to do things like respond to customer feedback, provide product roadmaps, etc.
↩
Hint: Use Vundle
↩
Due to the terminal compatibility requirement ↩

Digging Into Knockout Builds

Mon, 06 Jan 2014 19:30:00 -0400

There’s a lot you can learn by looking through other people’s code. This is especially true when you’re looking at widely used open-source libraries, where you can see how people have solved real problems with code that has been battle-tested and debated.

In this spirit of code literacy, I spent some time this past weekend to look through the source code for KnockoutJS. Knockout is a MVVM library in Javascript designed to make 2 way declarative bindings easy. I’m wanted to write a bit about what I saw in Knockout’s structure and build process.

Intro

Some of the first decisions a developer has to make when starting a new javascript project revolve around project structure and build processes. They need to decide how they’ll handle things like testing and minification. For a library like Knockout, developers also need to figure out how to make sure that they don’t expose any internal logic to the global scope. This post takes a look at how Knockout deals with these challenges. You can follow along by getting the source off of Github.

Project Structure

Knockout is organized with a traditional src,spec,build structure, with the main src code in the src directory, Jasmine tests in the spec directory, and build-related files in the build directory. The src directory is fairly flat, with directories for bindings, templating and subscribables. The spec directory does not mirror it exactly. The spec files are organized around functional categories at the root of the source files, with a separate directory holding tests for the default bindings. The build direction contains several files related to the build process, which I’ll discuss below. In the root directory there are dotfiles for NPM, git, and TravisCI, a package.json file, and the gruntfile.

Grunt

Knockout’s build process is automated using Grunt, an automation tool built on NodeJS. Knockout’s gruntfile weighs in at a very readable 150 lines of code, and only defines one task. That default task looks like this:

//Grunt Default Task
grunt.registerTask('default', ['clean', 'checktrailingspaces', 'build', 'test']);

So when you build knockout you’re executing 4 steps.

Clean

This deletes the knockout-latest.debug.js and knockout-latest.min.js files from the build/output directory

Check Trailing Spaces

This checks source files for trailing spaces and throws an error if they exist, ending the build.

Build

More detail on this below.

Test

Test spawns a child process which calls 2 scripts: ‘spec/runner.phantom.js’ and ‘spec/runner.node.js’. The script runs asynchronously in the background, with the results being passed to standard out through a callback.

//Grunt Test Task
grunt.initConfig({
    //...
    test: {
        phantomjs: 'spec/runner.phantom.js',
        node: 'spec/runner.node.js'
    }
});

grunt.registerMultiTask('test', 'Run tests', function () {
    var done = this.async();
    grunt.util.spawn({ cmd: this.target, args: [this.data] },
        function (error, result, code) {
            if (code === 127 /*not found*/) {
                grunt.verbose.error(result.stderr);
                // ignore this error
                done(true);
            } else {
                grunt.log.writeln(result.stdout);
                if (error)
                    grunt.log.error(result.stderr);
                done(!error);
            }
        }
    );
});

Overall a few things stand out about Knockout’s gruntfile.

It’s written with no dependencies other than the base grunt-cli package, which is specifically called out in package.json to be run locally. This is in sharp contrast to most Grunt-based projects I’ve seen, which assume a global install of Grunt and then make heavy use of the Grunt plugin ecosystem.
The only code consistency check is flagging trailing whitespace. Rather than enforce styling with a tool like jsHint, the build process only protects the integrity of the diffs with the whitespace check. This is probably a consequence of the plugin-free build file set up. Overall the build philosophy seems to aim to keep things as simple as possible.

The Build Process

So what happens with the build command? When it’s executed the build command runs a task which creates 2 files, knockout-latest.debug.js and knockout-latest.min. Each of them contain the same content, with the debug version wrapped in an IIFE, and the minified version run through Google closure compiler.

The content is pulled together with the getCombinedSources function:

//Grunt Build Task
function getCombinedSources() {
    var fragments = grunt.config('fragments'),
        sourceFilenames = [
            fragments + 'extern-pre.js',
            fragments + 'amd-pre.js',
            getReferencedSources(fragments + 'source-references.js'),
            fragments + 'amd-post.js',
            fragments + 'extern-post.js'
        ],
        flattenedSourceFilenames = Array.prototype.concat.apply([], sourceFilenames),
        combinedSources = flattenedSourceFilenames.map(function(filename) {
            return grunt.file.read('./' + filename);
        }).join('');

    return combinedSources.replace('##VERSION##', grunt.config('pkg.version'));
}

So this takes each of the files returned by the getReferencedSources function, and wraps them with 2 more files on each side. Lets look at the wrapper files first.

extern-pre.js sets up an IIFE and defines several global values within the local scope. This is done in order to protect against any reuse of these names in the local scope where knockout is loaded.

//extern-pre.js
(function(undefined){
    // (0, eval)('this') is a robust way of getting a reference to the global object
    // For details, see http://stackoverflow.com/questions/14119988/return-this-0-evalthis/14120023#14120023
    var window = this || (0, eval)('this'),
        document = window['document'],
        navigator = window['navigator'],
        jQuery = window["jQuery"],
        JSON = window["JSON"];

amd-pre.js determines the type of module system in use (if any) and passes the correct object into the inner scope as koExports.

//amd-pre.js
(function(factory) {
  // Support three module loading scenarios
  if (typeof require === 'function' && typeof exports === 'object' && typeof module === 'object') {
    // [1] CommonJS/Node.js
    var target = module['exports'] || exports; //module.exports is for Node.js
    factory(target);
  } else if (typeof define === 'function' && define['amd']) {
    // [2] AMD anonymous module
    define(['exports'], factory);
  } else {
    // [3] No module loader (plain <script> tag) - put directly in global namespace
    factory(window['ko'] = {});
  }
})

The *-post.js files simply close these function statements.

So in the end we’re left with an IIFE that wraps all of the content and defines global variables within the scope, and then executes a second IIFE which determines the module format in use and executes the inner factory function accordingly. The factory function is made up of the various files in the src directory, concatenated together in the order specified in build/fragments/source-references.js.

Initializing the library

The first few files loaded into the inner function are short files designed to provide a base for the library.

namespace.js sets up the ko object that is referenced internally in the file.

//namespace.js
// Internally, all KO objects are attached to koExports (even the non-exported ones whose names will be minified by the closure compiler).
// In the future, the following "ko" variable may be made distinct from "koExports" so that private objects are not externally reachable.
var ko = typeof koExports !== 'undefined' ? koExports : {};

google-closure-compiler-utils.js defines two helper functions, exportSymbol and exportProperty which allow for greater minimization of the internal ko library while maintaining a consistent external api. These functions are used everywhere throughout the library for defining externally facing function and property names.

//google-closure-compiler-utils.js

// Google Closure Compiler helpers (used only to make the minified file smaller)
ko.exportSymbol = function(koPath, object) {
    var tokens = koPath.split(".");

    // In the future, "ko" may become distinct from "koExports" (so that non-exported objects are not reachable)
    // At that point, "target" would be set to: (typeof koExports !== "undefined" ? koExports : ko)
    var target = ko;

    for (var i = 0; i < tokens.length - 1; i++)
        target = target[tokens[i]];
    target[tokens[tokens.length - 1]] = object;
};
ko.exportProperty = function(owner, publicName, object) {
  owner[publicName] = object;
};

Finally version.js sets up a placeholder for ko.version, which is then replaced with the version number in Package.json as part of the grunt build task.

//version.js
ko.version = "##VERSION##";

ko.exportSymbol('version', ko.version);

Conclusion

There are a few interesting things we can note from Knockout’s build approach.

Using IIFE’s to isolate local scope is a practical and important technique for library design

If you’ve been counting, the debug version of the library is wrapped in 3 different levels of IIFEs before any code exposing external functionality is actually included. This allows Knockout to provide several layers of data protection and abstraction, making sure that it is referencing the correct global variables and preventing scope leaks.

Its relatively straightforward to support module formats even if your library doesn’t use them internally

Take a look again at amd-pre.js. Its 15 lines of code to support AMD loaders, CommonJS loaders and normal script tag loading. Knockout doesn’t use a modern “module solution” approach to code organization. Instead it uses a more traditional namespacing approach wrapped in IIFEs. But it still plays well with others nicely. It’s hard to see how this type of effort wouldn’t be worth it for library designers.

Meetings And Concurrency

Mon, 30 Dec 2013 07:00:00 -0400

Working with a large number of people on any significant project eventually involves meetings. You can delay it for a while, and you can try to replace them with all sorts of things, including email, IM, wikis, and guessing people’s intentions. Doing so is, of course, completely worthwhile. Eventually though, meetings are inevitable.

There are several different flavors of meetings. There are decision-making meetings, where a crowd of people gather together, either to argue, watch other people argue, or to carefully watch the hands of the clock make their way around its circumference. There are Powerpoint meetings, wherein an individual gathers his victims (sorry was that coworkers?) together and proceeds to read bullet points from a set of slides to them. There is also the conference call meeting, which has led many people to a deeper appreciation for the mute button on their phone. Rarest of all is the actual productive meeting, where information is exchanged, decisions are made, and life continues without irreparable harm.

So is there any hope for productive meetings? It’s important to realize that a meeting should be the last resort for most things. Meetings are blocking synchronous activities. Everyone gives up a set piece of time and that time is the same for everyone. The meeting length is limited by the attendee most willing to keep talking. Generally that time cannot be used for anything else by the people involved in the meeting.

Programmers have several options for handling long running blocking operations while coding. The first is the naive approach. The following coffeescript is an example:

meetingDone = false

# are we done yet?
until meetingDone
    if person1Ready() and person2Ready() and person3Ready()
        longRunningMeeting()

#Do whatever's next, plus anything that depends on the meeting
followUpOnMeeting()
backToWork()

This is how things often work when it comes to meetings, but that code wouldn’t last long in a professional software environment. Everything grinds to a halt until the meeting is done, which keeps the program from doing anything else or responding to user input.

A second option while coding is the multi-threaded approach. In coffeescript that looks something like this:


# Do meeting tasks in a worker thread
meeting = new Worker "meeting.js"

# When the meeting is done, act on a follow up message
meeting.addEventListener "message", (e) ->
    followUpOnMeeting(e.data)

meeting.postMessage "Start"

# get back to work right away
backToWork()

This is equivalent in the real world to limiting the people in a meeting. Rather than allowing a meeting to block all our resources, we put only some of our resources into the meeting and then pass the results of the meeting on to the rest of the team once the meeting is complete.

This is an important principle for meetings. Meetings should contain the minimum amount of people needed to accomplish their purpose. When meetings grow beyond that size (due to laziness, politics, process or confusion) they begin to become exponentially less productive.

Of course multi-threaded approaches can still suffer the same issues as single threaded programs if you try to do too much. If the worker thread in the example above looks like the code below, we will not have gained much.

# meeting.coffee
self.addEventListener "message", (e) ->
    somebodyHasMadeADecision = false

    # are we done yet?
    until somebodyHasMadeADecision
        if  person1Indecisive() and person2Scared() and person3Clueless()
            longRunningMeeting()

    self.postMessage "meeting is finally done"

No matter who is involved it’s important for meetings to have a clear purpose. Just like you wouldn’t put all of your program’s logic inside a single worker thread, the best meetings are simple, single purposed, and lightweight, based on clear, answerable questions.

There’s a third approach that programmers can take to long running processes. They can use asynchronous operations, as shown below using promise notation.


# define an email response function
readResponses = (message) ->
    respond = readEmail()
    if respond then respondToEmail()

sendEmail("importantTopic")
    # responses and respond if necessary
    .then(readResponses)
    # read more responses and respond if necessary
    .then(readResponses)
    # follow up when the responses are done
    .done followUpOnMeeting

# After email we can go back to work right away
backToWork();

This async approach doesn’t delegate different resources for different problems, instead we change how we’re thinking about long running operations. Rather than tying up resources to deal with them, we make them a separate operation that can be consumed and responded to when our program has available time. You can’t do that with meetings.

By contrast, email, documentation, and other “information artifacts” can be consumed asynchronously, at different paces and different times by different people. They don’t block off any particular time that might need to be devoted to other things. So Powerpoint meetings can become an email that contains a Powerpoint or wiki page, followed if necessary by clarifying questions which people can work through at their own pace. Decision meetings can become a back and forth discussion on a mailing list, that the interested parties can participate in as they have time and motivation. It removes blocking issues completely by removing the requirement of all participants being on the same time schedule.

So the takeaways from this coffee flavored reflection on meetings: Async operations are your friends. Don’t have meetings. If you do, keep them short, small, and focused. Friends don’t let friends read their powerpoint slides line by line.

A look at Ack

Mon, 25 Nov 2013 19:50:00 -0400

Do you spend a lot of time searching your projects for a specific piece of code? Frustrated by grep’s weird syntax or slow IDE search tools? Ack is a tool like grep designed for programmers. I use it for searching my source code to track down sections of code or look for occurences of a variable or reference. It’s a simple open-source tool that does one job well, and does it with blazing speed.

Ack performs a subset of grep’s functionality, optimizing for the programmer’s use case. In the simplest usage it will allow you to enter an expression, and will search files in the current directory/subdirectory for it, ignoring version control artifacts like .svn and .git files.

# search all files in current directory recursively
# to see if they contain "foobar"
ack foobar

Ack then spits out nicely formatted output with the full file paths and line numbers of each occurence. For instance here is the output of searching the directory containing my blog posts for references to Microsoft.

These are very simple examples. Because of ack’s sensible defaults and focused design, these basic commands cover about 80% of what I’m looking for from a code search tool. But for those who want to do a bit more, ack can be much more powerful. Because ack is written in perl, any perl-compatible regular expression can be used to search your files. Ack provides plenty of options for formatting the output and it’s also very easy to filter by filetype.

For instance, I have a small side project where I’m creating an event signup site with the front end code written in CoffeeScript. If I want to see which files I’ve referenced events in without necessarily seeing all the details, I can use the -l flag to only show file names.

Obviously this search brings up several types of files. Along with the CoffeeScript files I’m interested in, ack shows some html and css files, as well as a random server side file. It also shows the generated JavaScript files. I could get rid of these files by specifying a different root directory, but instead I’m going to show ack’s smart filtering by filetype. To search only coffeescript files, I just need to add the --coffee flag to see all CoffeeScript matches.

Its also easy to add multiple filetype flags in order to search several filetypes, and you can even define custom filetype flags with an .ackrc file. These also let you specify default options and set certain directories to be ignored on a global or per-project basis.

Other Options

Ack is hardly the only code search tool out there. It’s meant to improve on grep by focusing on the programmer’s use cases, but there are ~~many~~ some situations where grep is still a good option. Ack is situationally optimized for searching code, ~~and doesn’t work as well for filtering piped output~~. Grep is also installed on many remote servers where you won’t have access to ack. So ack makes a great complement to grep, moreso than a replacement.

Update: You can totally use Ack to filter piped output. Another reason to use grep gone.

In terms of direct competition, ag is an attempt to improve on ack, mostly in terms of speed. Ag uses various tricks to optimize speed performance for code search, using an ack compatible syntax. Since its written in C rather than perl, it is indeed significantly faster. I personally am more comfortable with ack. Its more mature, has better documentation, and has always been fast enough for me.

~~Ag was not updated to support ack 2’s filetype filtering, and generally seems to have focused on speed over all else.~~ It also has dependencies that I’ve found are a problem on old operating systems. Ack only depends on perl, and is thus very portable, even to Windows systems. Still, if you find ack to be slow on your code base, or just like knowing you’re doing everything as fast as possible, ag is a great alternative.

Update: As of version 0.22.0 ag supports ack’s filetype syntax, but does not allow you to define custom file types, a very useful ack feature.

Summary

If you’ve been using grep or slow GUI tools to search your source code, you can do better. Ack is fast, easy to use, and optimized for programmers. It’s also free, open-source code you can use without cost or licensing worries. If you want to go even faster, consider using ag. Just make sure you’re using a tool that was built for the work you’re doing.

Modern Dojo: Exploring declare

Thu, 14 Nov 2013 19:15:00 -0400

This is part 2 of my ongoing exploration of Dojo’s important modules and concepts. Specifically this post will be examining dojo/_base/declare, Dojo’s object oriented programming helper module.

What is dojo/_base/declare used for?

dojo/_base/declare (henceforth “declare”) is the Dojo module for dom assisting with Object Oriented Programming. It provides a function that allows for easy “classical style” inheritance, as well as mixins. It is fairly flexible, and there seem to be several ways to use it. I’m going to explore those here.

How do you use declare?

Dojo’s declare function takes up to 3 arguments. The first argument, which is optional, gives a fully qualified namespaced class name ¹. The second argument is a string or an array specifying a class or set of classes that the new class will inherit from. This argument can also be null. The final argument is an object with the properties we want the new class to contain. The full 3 argument syntax looks like the example below.

//declare with a className argument

//pull in the declare module
require(["dojo/_base/declare"], function(declare) {

  //we're going to create a class in the global myApp namespace
  declare("myApp.examples.Messenger",null, {
    message: "hello world",
    sendMessage: function() {
        console.log(message);
    }
  });
  //since our class was created globally, we can access it wherever
  //we want, by refering to the namespace property
  var messenger = new myApp.examples.Messenger();
  messenger.sendMessage();
});

This syntax works fine, but to me it seems fairly contrary to the spirit of Modern Dojo. Rather than keeping the class definition bound in a module, it is instead pushed out as a global, using namespacing techniques rather than AMD style modules for code organization. It seems to be solely a product of supporting legacy implementations and Dojo’s declarative syntax. As such I’m going to spend the rest of this piece focusing on the 2 argument syntax, where the above example would look like what we have below.

//declare without the className argument

require(["dojo/_base/declare"], function(declare) {

  var Messenger = declare(null, {
    message: "hello world",
    sendMessage: function() {
        console.log(message);
    }
  });
  //since our class was created locally, we can access it here
  var messenger = new Messenger();
  messenger.sendMessage();

  //If we need access our class in other places, we can return it
  //from a module and then load it as a dependency where needed
});

So what is this simple example actually doing? declare takes the object that you pass and adds it to the prototype chain of your class. It’s important to understand that properties on the prototype are shared between all objects based on that class. This is perfect for functions, as they only need to be created once. It also works well for primitive values, since if we rewrite them the values will be added to the object itself rather than the prototype. Putting properties on the prototype can lead to unexpected behavior when dealing with arrays and more complex objects though. So its important to understand how the prototype chain works. For example, the code below produces unexpected output since the messages array is shared between all Messengers.

require(["dojo/_base/declare"], function(declare) {

  var Messenger = declare(null, {
    messages: ["hello","world"],
    sendMessages: function() {
        console.log(this.messages);
    }
  });
  var messenger = new Messenger();
  var messenger2 = new Messenger();

  //change the messages array on messenger
  messenger.messages.push("extra value")
  //and the change has populated to all instances of Messages
  messenger2.sendMessage();
  //logs ["hello", "world", "extra value"]
});

Fortunately Dojo provides an easy way to get around this issue with the special constructor property. If you define the constructor property of your object as a function, dojo treats that property as the constructor for your class and runs it when instantiating your class. This allows you to have local properties, and only share properties that should be shared.

//Constructor example

require(["dojo/_base/declare"], function(declare) {

  var Messenger = declare(null, {
    constructor: function() {
      this.messages = ["hello","world"]
    },
    sendMessages: function() {
        console.log(this.messages);
    }
  });

  var messenger = new Messenger();
  var messenger2 = new Messenger();
  //change the messages array on messenger
  messenger.messages.push("extra value")
  messenger.sendMessage(); //logs ["hello","world","extra value"]
  //and the change has not populated to other instances of Messages
  messenger2.sendMessage(); //logs ["hello","world"]
});

Dojo also provides a special helper function to allow you to define arbitrary properties on a member of the class when an object is created. You can use code like this to accept object arguments that you can map to properties.

//safeMixin example

var TestObject = declare(null, {
  val:"default",
  constructor: function() {
    //make the constructor arguments a mixin
    declare.safeMixin(this,args);
  }
})

//then we can stick with the default value
var test = new TestObject();
//or come up with our own
var spicyTest = new TestObject({val:"spicy"});

alert(spicyTest.val) //"spicy"

Inheritance

declare also provides a flexible inheritance option. In the examples above, we created a class without any inheritance by passing null as the superClass argument. But we can also specify a class as a superclass. For instance to create a new messenger that alerts your message instead of logging them, you could inherit the existing Messenger class like the example below.

//Basic Inheritance

var AlertMessenger = declare(Messenger, {
    sendMessage: function() {
      alert(this.messages)
    }
  });
var alerter = new AlertMessenger();
alerter.sendMessage(); //alerts the message

In this example, AlertMessenger inherits the constructor property from Messenger, but overrides the sendMessage function. So we change the functionality in a classical OO way.

One nice feature of Dojo’s OO implementation is that it is easy to make calls up the prototype change even when we are overriding functions. So for instance if we wanted our AlertMessenger class to send an alert while still logging the message in the console, we could use this.inherited to make it happen.

//this.inherited Example

var AlertMessenger = declare(Messenger, {
    sendMessage: function() {
      alert(this.messages);
      //calls up the prototype chain to Messenger's sendMessage function
      this.inherited(arguments);
    }
  });
var alerter = new AlertMessenger();
alerter.sendMessage(); //alerts and logs the message

So we still have access to overriden functions if used judiciously. Also, each constructor will be executed along the prototype chain, so there is no need to use this.inherited in a constructor context.

Mixins

But we’re not limited to a single inheritance instance. We can still do more. We can also pass an array of classes to the className argument, allowing us to inherit from multiple types at once. The first argument is the base of the prototype chain, and its extended from there, with the additional class properties being mixed in.

//Multi-Inheritance Example

var Lion = declare( null, {
  head: "lion",
  body: "lion",
  legs: "lion"
});

var Bird = declare(null, {
  head: "bird",
  wings: "bird"
});

var Human = declare (null, {
  head: "human"
});

//Sphinx is declared with 3 mixins
//lion is first, then bird, then human
var Sphinx = declare([Lion,Bird,Human],{
  //head gets overriden by human last, wings body and legs are never overriden and stay with their first set values

  aboutMe: function() {
     alert("I have the head of a "+ this.head +
           ", the wings of a " + this.wings +
           ", and the body of a " + this.body);
  }

  });

var sphinx = new Sphinx();

sphinx.aboutMe(); //I have the head of a human, the wings of a bird, and the body of a lion

When should I use dojo/_base/declare

declare seems like a great choice for creating reusable objects, and allows for either a classical inheritance style or a more flexible mixin style. Its a bit of overkill for simple or one off objects, but for defining the structure of a complex model, it seems like a great fit.

Personally I’m going to avoid the className syntax unless I find a use for it that I currently don’t see. It seems to be a legacy option that is completely at odds with the rest of Dojo’s avoidance of global variables. The 2 argument syntax on the other hand, seems like a great choice for creating maintainable and reusable classes.

declare should be especially useful for programmers who want to implement a classical inheritance structure in JS. It makes the syntax quick and straightforward. The only gotcha I see is the need to be aware of how the prototype chain works under the covers so that there’s no being burned by unexpected sharing of changes.

More Resources

Modern Dojo: Exploring query

Wed, 13 Nov 2013 19:15:00 -0400

It appears I’m going to be spending a significant amount of time working with the Dojo toolkit for my next project at work, so I’ve spent some time over the last few days trying to familiarize myself with the library. Its quickly become apparent that there are a few challenges with learning Dojo at the pace I wanted. First, Dojo is built as a lightweight core with many small modules of functionality available as components. It’s effectively completely modularized in a way that no other framework really is (though jQuery appears to be moving in that direction as well). As such, learning it is a bit decentralized. Second, Dojo, which has existed since 2005, made a big transition back in 2011 to move to an AMD format for managing its modular components. This resulted in a significant change in basic syntax. Since this change happened after jQuery’s rise (which led to a significant drop in Dojo usage), it seems many examples on the web still use the old syntax, and I haven’t seen any print resources published since the change. As a result, the resources for learning modern Dojo seem somewhat piecemeal beyond the toolkit’s website.

That being the case, I’ve decided to explore Dojo’s primary modules on a case by case basis and document what I’ve learned here. Hopefully this can be a helpful modern reference to others who are learning. I’m going to start by exploring dojo/query, one of the primary Dojo modules, which provides jQuery selector style functions for Dojo users.

What is dojo/query for?

dojo/query is the Dojo module for dom selection. The core module provides basic CSS selection capability, with the ability to use CSS3 style selectors if needed. It can be extended using other modules to provide more advanced DOM traversal and manipulation functionality, effectively matching jQuery’s feature set in that area.

How do you use dojo/query?

To use dojo/query, you first need to require the module, using an AMD style require or define call. You can then call query (or whatever you’ve aliased it to), and pass it a CSS style selector.

//Basic query example

require(["dojo/query"], function(query){
    //select all elements with class special
    var elementList = query(".special");

    //you can also add an optional second parameter to search
    //under a single DOM element,rather than the whole DOM
    var parentNode = document.getElementById("parent");
    elementList = query(".children",parentNode);
});

The query function returns a dojo/NodeList object. This is a basic array-like wrapper object, similar to a jQuery object. By default it contains some basic array manipulation functions, but you can also extend it with plugin modules for dom manipulation, dom traversal, and other functions. In fact you can even write your own extensions to the object. These extensions are loaded in as additional AMD modules. For instance, we can load dojo/NodeList-dom to extend NodeList with some basic DOM functionality

//Extending NodeList

require(["dojo/query","dojo/NodeList-dom"], function(query){
    //add a btn class to all button elements
    query("button").addClass(btn);
});

In that simple example, the addClass method was added as an extension to the NodeList type by the NodeList-dom module. So adding that module brought extra functionality. Its relatively simple to add custom functionality in this way. An example from the Dojo docs looks like this:

//Dojo extension example

//definition (in myapp/NodeList-custom.js)
define(["dojo/_base/lang", "dojo/query", "dojo/NodeList-dom"], function(lang, query){
  lang.extend(query.NodeList, {
    makeRed: function(){
      this.style({ color: "red" });
      return this;
    }
  });
});

//usage (in another file)

require(["dojo/query", "myapp/NodeList-custom"], function(query){
  query(".greenText").makeRed();
});

This is great for modularizing common patterns and functionality, and is a core Dojo concept, not just a dojo/query feature.

How does dojo/query perform?

I was curious about how dojo/query performance stacked up to jQuery. So I set up some performance tests on jsPerf to take a look. I compared Dojo to jQuery across a few common and simple selection and manipulation operations. The results were informative.

You can see the results here for

The results varied a bit from platform to platform, but a few things stood out.

When selecting on IDs, don’t use dojo/query

For the basic ID selection tests, dojo/query was blown away by almost everything. It is very very slow to handle it. I was initially bothered by this, until I added the dojo/dom test. dojo/dom’s byID function is the “dojo way” of selecting an element by ID.

dom.byId appears to be a thin wrapper around the native API, with near native performance everywhere (and confusingly, better than native performance on IE11). dojo/query meanwhile, lags way behind, literally 100x slower than a native query on Chrome 31. You can still integrate with dojo/NodeList to get dojo/query’s chaining functionality

//Method chaining with search by ID

require(["dojo/query","dojo/dom", "dojo/NodeList-dom"], function(query,dom){

    var list = query.NodeList();
    list.push(dom.byId("menu"));

    list.addClass("sidebar").onclick(function(e) {
        alert ("this is the menu");
    })
});

For simple things operations, you can’t beat native selectors

On the basic tests, the native selectors blow away the library results, with little downside. When you’re simply trying to access an element and, for instance, change its inner HTML, native selectors will scale much better.

However, the situation begins to become more complicated when more complex queries are involved. My manipulation example is still a relatively simple example, but the trade-offs start becoming a lot more substantial very rapidly. For the manipulation example, native code requires significantly more code. The simplest method of doing things is not supported on IE9, and requires you to understand the distinction between an HTMLCollection and a normal Javascript array. A solution that supports IE9 and handles other edge cases involves even more code, and performs in the same range as Dojo and jQuery.

dojo/query seems to lag behind jQuery in performance, but its in the ballpark

dojo/query was consistently outperformed by jQuery in the testing that I did. But while the gap between native selectors and library versions is very wide, the dojo/jQuery divide seems to be on the order of a 2-4x increase in speed on basic operations. Thats significant, but hardly damning. If selector performance is your primary criteria for choosing a library, jQuery should be your pick. But Dojo is competitive enough here, that it doesn’t seem disqualified for use if you have other reasons for doing so.

When should I use dojo/query?

To sum it all up, it looks like the query library is a great choice for providing jQuery selector style functionality for Dojo users. It’s small, lightweight, and allows you to load only the things you need to get the job done. If you only are looking for this type of selector functionality, a custom built jQuery package might be a better fit. But if you’re already using Dojo, dojo/query will get the job done very well. Just don’t use it to search on id’s. Use dojo/dom or native selectors instead.

More Resources

BlendConf 2013: Takeaways From A Very Human Tech Conference

Tue, 10 Sep 2013 16:00:00 -0400

After 3 great days at Blendconf, my brain is pretty overloaded with cool ideas. One of the cool things about the conference was seeing certain themes emerge across different sessions. Here are three of my takeaways from a great weekend.

Modular Web Development

The biggest technical theme that I noticed in the different sessions I attended was a focus on modular design in JS and CSS. Shay Howe gave a great workshop where he encouraged us to view CSS at a component level rather than at a page level. That was followed by talks from Jina Bolton and John Long, who reinforced the patterns that Shay had advocated for, and also showed how they could be implemented with preprocessors like SASS. Vernon Kesner also gave a great talk on using requireJS for modular Javascript. That’s a relevant topic to me, since I’ve spent the last 2 months working to convert a large codebase to AMD modules, and Vernon did a great job explaining the motivation for modular JS.

Design and Development should not be separate processes

Design is how it works. – Steve Jobs

One of the real key points that multiple speakers hammered home was the importance of having tightly bound design and development. Cameron Moll talked about the importance of really understanding and being true to the things that you’re creating. Brandon Mathis argued convincingly that its important for designers to know their material.

Leave Things Better Than You Found Them

Aspire to change your world in some small but significant way –Cameron Moll

I feel like this final point in some ways sums up what BlendConf is all about. Its the result of one man thinking about how cool it would be to have a big-time tech conference in Charlotte and then going and making it happen, then going out and finding speakers who set a vision of a tech career that was about more than money. Carl Smith talked about his quest to find a more humane company structure. Ashe Dryden encouraged us to look for the people who are being excluded from tech culture and find ways to bring them in. Greg Baugues told us about how he’d worked through his bipolar disorder at least partially through the support of his coworkers and friends, while also showing that many people still don’t receive that support. Cameron Moll set the goal as “aspiring to change the world in a small but significant way.”

Blendconf was the first major tech conference I’ve attended. I wasn’t totally sure what to expect, but it defied whatever expectations I did have with it’s humanity. I’m excited to see that it is becoming an annual conference, and can’t wait to see where it goes from here.

A Great Summary From Twitter

The best part of @blendconf wasn’t the device policy or the talks, it was the uniformly awesome love and energy from everyone there.
— Tim Cheadle (@timcheadle) September 7, 2013

Searching for the perfect reading device: My Nexus 7 (2013) Review

Fri, 16 Aug 2013 08:00:00 -0400

Since Amazon announced its Kindle platform back in 2007, I have read books on a variety of different devices. I’ve owned and used two different Kindle devices and experimented with the rest. I’ve read on my smartphones, iPad, laptop, and traditional paper books. There are great things about each, but there are also aspects of all of them that drive me crazy.

When the new Nexus 7 was announced last month, it seemed like the type of device I was waiting for: powerful, light, and affordable. Most importantly, it included a screen that would be easy on the eyes.

The Problem with the Kindle

Up until now, I’ve mostly used Kindles for long form reading. Their e-ink screens are fantastic,the experience is seamless when reading Amazon content , and they’re light and easy to hold as a small paperback. The problem is, while I love reading, a lot of the content I was interested in, including blogs, the Bible, online news, and magazines, was difficult or impossible to access on a traditional Kindle. In addition, my Kindle often wasn’t able to cope with the diagrams and code snippets common in the more technical books I read.

As a result, my Kindle often ended up neglected, and I used my iPhone or laptop instead. Kindles are fantastic reading hardware, but their lack of integration with non Amazon store content cripples their usefulness. I think Amazon’s best play here would be to make a deal with a service like Pocket or Instapaper ¹ for deep integration that goes beyond the almost unusable “send document to Kindle” integration that Instapaper currently provides². That would go a long way to making it a viable multi-purpose reading device.

As for the Kindle Fire line, they serve as a step in the right direction, but have consistently been panned for laggy performance and an interface that is effectively a store front. They still might be a valid contender though, except for the growth of other options that don’t share those problems.

Why not an iPad?

The other serious contender for the title of “best reading device” is the iPad. Since it first came out in 2010, it has quickly become the most popular choice for multimedia consumption on a midsize (7-11 inch) screen. It has arguably the best hardware, and inarguably the best selection of third party apps and integrations. The question then becomes, “Which iPad?”

The “original iPad” line, now on its fourth generation, boasts a large ten inch screen- one of the nicest screens you can find on a mobile device. It also hosts three and a half years-worth of apps designed specifically for the iPad’s size and form. It is by almost all accounts the best multipurpose tablet that money can buy. As a reading device, though, it leaves much to be desired. At 1.44 lbs, it’s heavy and awkward to hold in one hand. It’s also expensive: $200-$400 greater than other reading devices. These factors can easily be overlooked relative to the versatility and utility of the iPad as a productivity and media-consumption device, but they make it difficult to recommend as a pure reader.

The iPad mini, on the other hand, is almost the perfect shape and size for a reader. It is the lightest high-end tablet on the market, and also feels like much more of a premium device than the Nexus or the Kindle when you hold one in your hand. It comes at a reasonable cost ($170 less than the “regular” iPad), and like the Retina iPad, it comes with the full support of the Apple ecosystem. The iPad mini is its older sibling’s only truly valid competitor for “best multipurpose tablet”, and a completely valid choice as a reading device at that.

On the other hand, the iPad mini also has a screen that is completely blown away by all of its competitors. Compared to any of the other devices mentioned here, its text looks pixelated and less crisp, as you can see in this comparison from Ars Technica’s Nexus 7 review.

For many uses this is not a big deal, but when reading longform text, I find that the display resolution really makes a difference in how long I’m able to read comfortably. When I looked at the Mini in my local Apple store, iBooks' text was significantly less crisp than it was on the other devices. Combined with the extra cost relative to a Kindle or Nexus 7, it becomes difficult to justify recommending an iPad Mini as a reading-focused device.

Enter the Nexus 7

The new Nexus 7 manages to avoid many of the tradeoffs hindering the other devices. It has a top of the line screen at a price point directly in between the Kindle Paperwhite and the iPad Mini. It also remains easy to hold, with a smaller (though slightly heavier) frame than the Mini. And since it is a full Android tablet, it comes equipped with a real web browser and integrations with all the most prominent reading services ³. That’s the Nexus 7’s promise, but is it able to live up to it?

A beautiful screen

The first thing you notice when you see a Nexus 7 is the screen. It’s a premium component set inside an otherwise solid but unremarkable slab. The icons on the home screen are crisp and detailed, and the text is clear. It’s a screen that practically begs you to read.

The only downside of the screen is that not every device has one yet. As a result, web sites and non-optimized apps can appear blurry and pixelated. The images and toolbars in my preferred Bible app, for instance, appear out of focus, although the text itself is still crisp and readable. This is the price of progress though, and something that Apple users have had to deal with each time a device has transitioned to Retina. Android’s fragmentation will make the transition slower, but its a problem that will be mitigated over time as more apps are targeted at newer devices with high resolutions.

Easy to hold

When I bought the Nexus 7, I was nervous about how the device would feel. Unlike iPads, which you can touch and handle in Apple stores, it’s difficult to know how a Google product will feel in your hands before you buy one. My local Best Buy had a single test unit, and it was anchored to the display table with a 6" plastic cord that weighed about as much as the device itself ⁴. I couldn’t find any other way to actually touch a Nexus 7 without purchasing, so I decided to purchase one before I went on a vacation and give it a week to prove itself.

In the end I was satisfied, if not quite delighted, by the devices feel. It feels sturdy in the hands, and definitely falls into the “book” size range. On the other hand, the Nexus 7 is both smaller and heavier than an iPad mini, giving it some unwanted heft. This seems like an unavoidable consequences of cramming a Retina-class screen into a 7" tablet while maintaining reasonable battery life. But it does mean that it can’t match the paperback feel of the Kindle or the “pleasant to hold” sensation of the iPad Mini. The Nexus 7 makes you think a bit more about it than those two devices, which is probably the biggest knock on it as a reader. It’s not too heavy to hold in one hand, but it may feel a bit awkward doing so. I usually end up using two hands if I’m sitting down to read, but will hold it with one hand if I’m carrying it around.

What about the apps?

The app selection was my second biggest concern when considering the Nexus 7. As a previous owner of one of the original Windows Phones, I know what it’s like to work with a platform that doesn’t get full support from developers. If you love the OS you can still have a good experience, but your choices will be limited and you may not be able to make some things work. Android tablets in 2013 aren’t quite where Windows Phone was in 2010, but there is a big drop in the quality and consistency of tablet apps from the iPad to an Android tablet. When it comes to reading, however, this seems to be a manageable issue.

Android has high quality tablet apps for book reading, RSS, and Read-It-Later services. The Kindle app is solid, though it’s missing some features from the iOS app like the X-ray option and the ability to change fonts. Press is as good as any RSS client I’ve tried on iOS or the desktop. Instapaper is surprisingly nice on Android, given Marco Arment’s well documented skepticism toward Android. There are other solid options for those using other services, including apps for Pocket, Nook, and Feedly.

The Verdict

Overall I’ve been incredibly pleased with the Nexus 7 so far. I think it makes the right tradeoffs for a reading device. It adds a little extra heft in exchange for a brilliant screen at an affordable cost, in a package that is still smaller than many paperbacks. It might not please somebody looking for a multipurpose media tablet: it’s still missing popular iOS apps like Amazon Instant Video and Tweetbot⁵ (and even when apps do exist they’re often not optimized for tablets). But if you don’t need that functionality, there is a lot here to love. As a multi-source reading device, the Nexus 7 is king.

Update June 2014: After almost a year with the Nexus 7, I stand by my analysis of the tradeoffs involved with different devices. But I’ve found that the ones the Nexus 7 make bother me more than I expected. It’s ended up feeling like an impractical device for me. Because it’s battery life doesn’t match something like the kindle, I have to worry about charging it. But because it doesn’t have the general utility of the iPad (or more importantly my iPhone), I end up neglecting it. The lack of a decent twitter app, and the lack of polish on most non-Google apps makes it a tough sell when faced with the option of doing something on the Nexus vs doing it on my phone or laptop. I still think the Nexus 7 is a great device. But it has not met my needs as well as I expected

Get The Nexus 7 (Amazon Affiliate Link)

Actually, an Amazon acquisition seems like the perfect out for Pocket, a popular service that completely lacks a business model. ↩
To be clear, it's unusable because Amazon makes it unusable. There is no way (as far as I can tell) to bulk-delete documents, or delete a document from the device after you finish it. Instead, you have to go to Amazon.com later and delete documents one at a time. It's left my Kindle library littered with old Instapaper articles that I have no interest in reading again. ↩
In my case that's Kindle, Instapaper, Feedbin and Pinboard, but it equally supports Nook, Pocket, Feedly, Delicious and most other popular services. Of course if all your books are in iBooks then you probably haven't even made it this far. ↩
Best Buy seems to go out of its way to cater to people who don't actually care if their devices are nice. They've gotten a little better with some of their featured electronic displays. There was a very nice "touch and play" section for Samsung devices. But it always seems like half the display devices are non-functional, and they're actively trying to push Windows machines with reasonable specs, cheap plastic casing and trackpads you have to hit with a hammer to get a response. ↩
Obviously there are plenty of twitter apps on Android, but I haven't been able to find one that is anywhere near as nice to use as Tweetbot. That's obviously a compliment to Tweetbot's development team, though it does seem to me that there may be some API limits that are hurting the Android apps. Otherwise, I can't understand why they seem to universally require 2 taps to follow a link. ↩
```
</li>
```

Somewhat Open

Thu, 15 Aug 2013 12:15:00 -0400

From Microsoft’s official “Microsoft on the Issues” Blog:

It seems to us that Google’s reasons for blocking our app are manufactured so that we can’t give our users the same experience Android and iPhone users are getting. The roadblocks Google has set up are impossible to overcome, and they know it.

It’s ok to have an open API, and it’s ok to not have an open API so that your competitors can’t benefit from your services, but claiming to have an open API and then shutting out users of a competitors platforms on a string of technicalities? That’s just a joke.

Book Review: User-Centered Design

Mon, 12 Aug 2013 11:15:00 -0400

User-Centered Design by Travis Lowdermilk is a solid overview of the principles and practices of designing a software project with a focus on your users. The book does a good job of laying out the basic tools and processes of user-centered design, like usability studies, surveys, and project plans. It also argues convincingly for the power of its central points: user focus, preparation, and a structured process.

If you’re looking for a deep dive into design theory, this is probably not a book you want. Rather than dig into abstract design issues, this book instead acts as a survey of the user-centered design landscape, and points you to resources to go deeper if a particular topic catches your interest. Lowdermilk also does a good job of pointing out examples of the different principles discussed, though for a stretch of the book he appears to be a bit fixated on a small selection of examples, particularly 53 Inc.’s popular iPad app Paper.

I’d recommend this book for developers who want to open their minds to design considerations and improve their ability to make usable products. It particularly seems suited for the engineer who knows technically how to implement features and write good code, but is still figuring out how to write applications that delight users. I’d especially recommend it for team leads or independent developers who are more likely to be able to meet users and be a part of managing requirements. I think all engineers could benefit from the books emphasis on the users point of view, but the real benefits will come to those who are able to make its suggestions part of the structure of their product workflows.

Buy User-Centered Design on Amazon (Affiliate Link)

This review was part of the O’Reilly Blogger Program. I am not compensated for this review, but did receive a free review copy of the book. The opinions expressed here are my own.

RSS Roundup

Sat, 29 Jun 2013 07:40:00 -0400

We’re at T-3 days till the Google Reader shutdown, and it seems like the “wait and see” crowd has made their initial decisions. As a large crop of companies and products have weighed in, a large and diverse selection of “Google Reader replacements” have formed. The groups behind them range from small indie developers (Feedbin, Go Reader) to fallen internet giants looking to reclaim glory (Digg, AOL). The readers themselves seem to fall into a few main categories:

Free cloud service aiming for scale
Free, paid, or ad-hoc self hosted option
Former Reader clients allowing standalone RSS or a proprietary sync service
Paid cloud hosted option

Lots of others have chimed in with reviews of each of these services over the last few days, but I think that its hard to recommend a specific service universally, since people have different things they’re looking for in those services. So I thought I’d take a look at the different categories and see why somebody would use them.

Free cloud based services

It seems like the most popular Reader replacement has been Feedly. Its one of several options that try to replicate what Reader provided while also maintaining its non-existent price tag. In doing so, they also inherit the inherent dangers of a product that adds more expense than revenue for each user they gain. Products like that can seem like an obvious choice for users in the short term, but the results are often less pleasant in the long term. Business models built around free products incentivize behaviors in ways that are unrelated to the customers' best interests. This can eventually cause privacy issues or unwanted restrictions. They also can go the way of Google Reader, a product that was killed off when the costs were no longer worth the benefits provided.

However, this group does best meet the expectations of an audience used to the free “available anywhere” nature of Google Reader, and includes several excellent products. Feedly clearly has become a major undertaking, and Digg Reader has an elegant minimalistic style. Each has its flaws, but also has a dedicated team behind it that is clearly listening to feedback and determined to succeed.

Self Hosted Options

There is a certain type of person whose immediate reaction to seeing Google Reader’s shutdown was to give up on external services and write their own. If you have tried that then you probably already have very strong opinions on this issue. If you didn’t, the good news is that several of these efforts have produced public open source options. There are also other options for self-hosting RSS, most notably Fever, a self hosted app that you can purchase for a 1 time $30 fee and then host yourself. There’s also Go Read, an open source app whose author is currently hosting it for free on App Engine (with ads soon to come), but also has open-sourced it and made it easy to run on App Engine’s free limits.

In the end, these options will appeal to those who want to control their experience, and especially those who want to tinker. But for most this will look like a lot of work and expense for unclear benefits.

Reader Clients

A huge ecosystem of apps was built up around Google Reader over the past several years. There are “Google Reader” apps on every major platform with different styles and features to appeal to a wide variety of users. Especially on mobile, most users accessed Reader through one of these apps, rather than through the site itself. Now that Google Reader has removed the sync backbone of their community, these apps have had the opportunity and challenge of reinventing themselves. Most have adapted one or more of the new services, but some are focusing on local RSS or are developing their own sync service. NetNewsWire in particular seems to be going it alone. This may make sense for somebody who’s dedicated to a certain reader. At the very least, the services supported by different apps are a big piece of the puzzle for those of us on the hunt for a new service.

Paid Sync Services

Finally, there is a market re-emerging for paid RSS sync services. Services like this began to emerge before Google Reader emerged as a dominant free alternative. Now that Reader is gone, the service has become economically viable again. The market appears to be settling on a cost of $18 to $25 a year for RSS hosting. Several services fall into that sweet spot, including Feedbin and Feed Wrangler.

What now?

For now, the most important step for Google Reader refugees is to download your subscription information from Google. Even if you’ve imported your information into a new service, it’s important to have a local copy, especially since several of the new services don’t support any export options.

After that I’d recommend trying a couple of the new services. They work differently, and reflect the different ways people use RSS. For me personally, I found Feedly’s UI to be frustrating, and am still totally confused that Digg’s otherwise excellent offering seems to provide no way to view only view unread items. I was also concerned by the lack of a business model for Feedly, and the fact that Digg Reader appears mostly dedicated to supporting Digg, rather than serving as a standalone business. I didn’t seriously consider Fever or any other self-hosted services. I want RSS to be a thought-free process for me, where I’m able to focus on the content and not the tool. I’m willing to pay for that.

In the end I’ve settled on Feedbin. I’m paying the $2/month rate, and have been quite satisfied so far. It has a clean and usable web interface, and works with my favorite iPhone client Reeder. It also makes it easy to quickly scan unread items, and move all items to read, which fits my personal workflow. Feedbin works for me. I hope you find what works for you.

Marco Arment doesn’t see a future for proprietary sync services
Dr Drang had a rough experience trying to do it himself, but now seems happy with Feed Wrangler
The Verge took a look at the more established options and recommends Feedly

How I Use Stack Overflow

Tue, 11 Jun 2013 17:19:00 -0400

The problem with self directed learning is that our ability to learn is often constrained by the very problem we are trying to solve, our limited knowledge of a field.

One of the first things I learned as I began to work in the software industry was that self driven learning is necessary for staying relevant. Especially when working on evolving platforms like the web and mobile, things change fast. The web frameworks of 2005 have little direct relevance to web development in 2013, and mobile development in its current form was a category that didn’t even exist in 2005. If you want to be at the top of your field, or even anywhere close, learning and growing independently of whatever your current task happens to be is incredibly important. But it can be hard to put into practice.

The problem with self directed learning is that our ability to learn is often constrained by the very problem we are trying to solve, our limited knowledge of a field. Some of these problems resolve themselves quite easily. A beginning Javascript programmer may not know that he needs to include a script in order to use the jQuery syntax he’s seen online, but he’ll discover it very quickly when his script fails to run. But it’s quite possible to go on programming for years without realizing that it’s bad practice to couple your view and model too tightly, or even without gaining a deep understanding of how a web application model actually works. We create our models of how programming works, and it’s easy to stay inside those closed doors. It can be easy to try and fit new languages into existing understandings, which is why you find people writing Java that looks like C and Javascript that looks like Java. We don’t grow because we don’t ask the right questions.

Enter Stack Overflow

The killer feature when asking questions is that developers often won’t answer within the scope of what you asked; they’ll help you ask the right questions instead

If the problem is that we don’t ask the right questions, a Q&A site may seem like a dubious solution. But Stack Overflow has been the best tool I’ve ever found for self-directed learning. It’s a crazy hybrid between a wiki, Q&A site, forum, and video game originally conceived by Jeff Atwood and Joel Spolsky back in 2008. With ambitions to become a comprehensive programming archive for the web, the site has become a critical resource for programmers. In fact it’s the 92nd most visited website in the world as of June 2013.

So how does StackOverflow enable self directed learning? The most obvious way is allowing open-ended questions to be asked, with answers crowd-sourced from a global community of developers. The killer feature when asking questions is that developers often won’t answer within the scope of what you asked; they’ll help you ask the right questions instead. If you ask how to change the backgrounds of a group of DOM items given a list of ids, you’re likely to get an answer (or 5) while also helpfully being informed of the existence of classes. Because multiple answers are allowed you may be exposed to multiple approaches. There is plenty to be learned this way. Stack Overflow also helps automate this process by suggesting related questions as you type your questions, based on keywords. The AI behind this is quite good, and usually causes me to abandon my question writing as I realize that my question has already been asked and answered.

Learning

Questions asked by other people that you can’t answer are the best part of Stack Overflow

Stack Overflow really began making a difference in my work and learning though when I moved past asking questions and also began contributing. At the end of 2012 I started making a serious effort to begin answering questions, both as a way to contribute back to the community, and as a way of learning about my primary language at work, Javascript. I was, and am, by no means a Javascript expert. But I have learned a lot through the following process:

Browsing regularly through new questions under the Javascript and Coffeescript tags.
Picking out interesting questions that haven’t been answered satisfactorily yet. If there are good answers that I can’t add to substantially, I upvote them. If there are harmfully misleading or unhelpful answers I may downvote them with a comment.
If I know the answer to the question, I answer it, with an attempt to cite some sources for further information. This happens sometimes, but much more often I don’t know the answer off the top of my head. This is where learning takes place.
If I don’t know the answer I’ll try to research it. This may involve some combination of:
1. Making a test with jsFiddle and learning about the behavior hands on.
2. Looking up the relevant spec on MDN, Github, or library documentation
3. Looking at related Stack Overflow questions to see if there are relevant resources or information there.
Once I have an answer I’ll return to the original question. Sometimes (usually in fact) somebody else will have beaten me to a response. If that happens, I evaluate whether I can add value. If I can, I answer. If not I upvote the best answer and move on.

This is where Stack Overflow really begins to show benefits. When I do the research to answer a question I usually learn many things along with whatever the specific question was. Questions asked by other people that you can’t answer are the best part of Stack Overflow. They are opportunities to not just expand your field of knowledge, but also to think about problems and solutions that you had never even contemplated. So go, find a question that you don’t understand, and see what it takes to answer it. You may just learn something.

Evergreen Browsers

Tue, 11 Jun 2013 14:45:00 -0400

Tom Dale on Evergreen Browsers:

If you’re like me, when you’re developing a new web application, you put features into mental buckets. There’s the “works in IE7″ bucket, the “works in IE8″ bucket, “(I think) works in IE9,” and of course, “works in MobileSafari.”

The one bucket I don’t have is the “works in Chrome” bucket. That’s too much mental overhead. Instead, if I want to test whether something works in Chrome, I just pop open a new JS Bin and try it out. I don’t worry about which version they’re on—I assume that by the time my code makes it to production, my users will be on more-or-less the same version as me.

What would the web platform look like if every browser with significant market share updated itself at the same pace—and lack of user intervention—as Chrome?

I’m excited for this to happen. Browser diversity is a good thing and worth the pain because it pushes things forward, but the advantages are lost when users aren’t moving forward along with the innovations that diversity and competition bring. Having an up to date browser shouldn’t be a user concern. Here’s to a web that moves at the pace of the leaders, rather than being paced by the least common denominator.

90% Done, Halfway There

Sun, 09 Jun 2013 14:45:00 -0400

There’s a running joke on my team at work when somebody asks about progress on another engineers bug.

“I’ve done 90% of the work, so I’m about halfway there!”

It’s really hard to estimate software development times. I would in fact say that from a project planning perspective, that may be THE hardest problem when planning a big project. Its certainly a question that engineers are asked all the time, and not one that is usually easy to answer even on an individual level.

Why do we care?

Estimating development time accurately is important both in the big picture and in day to day development work.

It allows you to set healthy and reasonable expectations
It allows for proper prioritization and planning of features
It gives a team flexibility to make changes early if the current schedule is untenable
It allows you on a personal level to properly prioritize and balance your time

Why Is This So Hard?

I’ve been thinking about this lately, and I’ve identified the factors that I’ve found contribute the most to the “90% done, halfway there” mindset.

Scale

The first factor is also the most inevitable. More code, bigger teams, and more complex requirements all make software estimation harder. When I work alone on a side project, I rarely have trouble estimating how long a particular feature will take to implement. This is true for a few reasons. First, and most importantly, I know and understand all of the code in the project. If I see code that looks out of place, I either remember, or can quickly work out, why it was put there. I also have a comfort level with all the tools and technologies in use because I chose them myself, and have either used them before or specifically wanted to learn them.

Working in a larger team however means greater uncertainty about the original intentions of existing code, less familiarity with the code base as a whole, and (on sufficiently large projects) possibly an acceptance that you cannot understand the whole code base. This unfamiliarity is usually mitigated by having engineers focus within a specific portion of the code. That is a great strategy, provided our second problem is not present.

Tight Coupling

When it comes to issues within the code base itself, I think the biggest obstacle to accurately estimating development time is tightly coupled software components. Anybody who’s ever read The Pragmatic Programmer can tell you that tight coupling is a BAD THING, but in practice it can be seductively easy to fall into. The problem is that tight coupling means that when making changes to a piece of code you don’t have to worry just about its internal function, but also about side effects. When combined with the first issue of scale, this means that changes to code you’re working on may have effects on other code that you don’t fully understand or anticipate. Too much of this can easily lead to

90% done, and now I have 4 more bugs to work on.

Defining Coding As The Task

The first 2 issues are to a certain extent unavoidable when working on a large existing code base like my teams. But bad estimates can also appear for reasons completely unrelated to the particular problem or code base. One issue that can be a problem for a programmer on a new team and also cause issues with programmer-manager communication is viewing coding as the task, without worrying about design, testing, documentation or code reviews. When the programmer says to his manager that something is a 3 line fix what he’s really saying is that the coding seems easy so the rest should be quick. If his initial solution has problems, he not only is missing the time to write a new one, but also the time for it to be discovered by test, formulating a new design, and any review or paperwork required for making the change.

Poor Communication

Finally the easiest way to be 90% done and only halfway there is to spend your first 90% writing the wrong code. When the specs aren’t clear and the programmer doesn’t understand the requirement (whether it comes from a customer, test, or a project manager) wasted time and poor estimates are pretty much a given. The key here is the same as the last point. There is more to software development than just writing code.

Improvement

So how can we do better? I’m still learning. But here is the main way that I’m trying to do to get better at this.

Tightening The Feedback Loop

One thing that can really improve the communication issues, and also make you aware of any tight coupling early on is tightening the feedback loop for your code. What is that? It means working to find out about errors and problems as soon as possible. Some ways to do this:

Automated Testing
Asserts or other constructs to test assumptions
Early prototypes of even small features to show requirement-setters

I’ve known for a while that there’s value in building a subset of a feature to make sure I’m on the right track before coding the rest. If a picture is worth 1000 words, working code is worth 1000 emails. When you build something, even if its only partial or simplistic, you immediately get a sense for the challenges involved, and you also get the chance to immediately find out whether you are on the same page as the requirement creators. For many people, especially non-programmers, its much easier to critique an existing system than to describe one ex nihilo.

Several of us are also working to introduce more automated testing on our team. We have a great team of manual testers, but there is great value in the immediate feedback that automated testing can bring, especially in revealing the type of unexpected side effects that tight coupling causes.

Summary

Software Development estimation is hard. There are a variety of challenges, some of them inherent to the practice or certain projects, some of which can be overcome or mitigated through experience and attention. In the end, accurate estimation is a skill worth cultivating, bringing value to both you and the team. Take the time, get it right.

Got some great tips for estimating software development times? Let me hear them on twitter @_benmccormick

Customer Culture Revisited

Sat, 08 Jun 2013 14:45:00 -0400

Marco Arment back in 2011:

For Windows 8 to succeed, especially for its tablets to compete well against the iPad, it will need to adopt a more Apple-like confidence to say “no” to its customers when they ask for everything under the sun. And, surprisingly, they might be doing exactly that.

One of the reasons Metro is interesting to people like me who usually ignore Microsoft is that it’s full of very un-Microsoft-like decisions, generally for the better.

The question isn’t whether Metro will be good: it probably will be. And that’s a huge accomplishment for Microsoft that they should be commended for.

But how will their customers react?

Will Metro be meaningfully adopted by PC users? Or will it be a layer that most users disable immediately or use briefly and then forget about, like Mac OS X’s Dashboard, in which case they’ll deride the Metro-only tablets as “useless” and keep using Windows like they always have?

Almost 2 years later, this is looking extremely prescient. I like what Microsoft was attempting to create with Windows 8, and love the Metro/Modern/Whatever style in general, but 8.1 seems to signal most of the effort since has been spent hedging on it instead of doubling down, and I can’t see how its going to get where it needs to go like that.

Revertible Observables in Knockout

Tue, 07 May 2013 16:05:00 -0400

Last week I was looking for a way to use KnockoutJS on a configuration menu that required the user to be able to cancel or accept their input after filling out the menu.

Initially I was just copying the initial data and refilling my data model manually on cancel. I wasn’t satisfied with that solution though. It didn’t seem like it would scale well if the menu got more complicated, and it lacked the elegance and frictionless nature of most knockout data management.

After some searching I found Ryan Niemeyer’s Protected Observable example, and it initially seemed like exactly what I wanted, a way to only persist changes to the view model if they were explicitly confirmed.

//knockout-protected.js
//https://gist.github.com/benmccormick/5537138

//wrapper to an observable that requires accept/cancel
ko.protectedObservable = function(initialValue) {
    //private variables
    var _actualValue = ko.observable(initialValue),
        _tempValue = initialValue;

    //computed observable that we will return
    var result = ko.computed({
        //always return the actual value
        read: function() {
           return _actualValue();
        },
        //stored in a temporary spot until commit
        write: function(newValue) {
             _tempValue = newValue;
        }
    });

    //if different, commit temp value
    result.commit = function() {
        if (_tempValue !== _actualValue()) {
             _actualValue(_tempValue);
        }
    };

    //force subscribers to take original
    result.reset = function() {
        _actualValue.valueHasMutated();
        _tempValue = _actualValue();   //reset temp value
    };

    return result;
};

There was a problem with this approach though. Ryan’s model saves the update to a temporary value and then moves it into an observable if the result is committed. That works great for simple models and mostly behaves well if the user cancels out of the edit screen without committing or resetting. But it breaks down if you have another computed observable that depends on the value of the protected observable. In my case I had an output that I wanted to show dynamically changing based on the input allowing experimentation. Here’s a simplified example using a protected observable:

As you can see, the tables field doesn’t update until you confirm the entry. Ideally though we’d like it to update as the user changes their guest number, so they can see the effect on cost and space used before they confirm a change in guests. We want to do this while still preserving the users abilities to cancel changes if they don’t like the results though. So how to we do this? We make a small change to the default value returned by the protected observable. Here’s what I’m calling a Revertible Observable:

/*
knockout-revertible.js
https://gist.github.com/benmccormick/5537115
*/

 //wrapper to an observable that requires accept/cancel
ko.revertibleObservable = function(initialValue) {
    //private variables
    var _actualValue = initialValue,
        _tempValue = ko.observable(initialValue);

    //computed observable that we will return
    var result = ko.computed({
        //always return the current value
        read: function() {
           return _tempValue();
        },
        //stored in a temporary spot until commit
        write: function(newValue) {
             _tempValue(newValue);
        }
    });

    //if different, commit temp value
    result.commit = function() {
        if (_tempValue() !== _actualValue) {
            _tempValue.valueHasMutated();
             _actualValue = _tempValue();
        }
    };

    //force subscribers to take original
    result.reset = function() {
        _tempValue(_actualValue);   //reset temp value
    };

    return result;
};

This preserves the ability to see the results of your changes in realtime, while also allowing you to easily revert with a simple cancel button and no explicit data tracking. Now our example can work as we’d like, with the customer getting immediate feedback on how their guest changes effect the total cost.

Note that this isn’t a silver bullet. An explicit confirmation is no longer required for the changes to be passed through, so its important to make sure that the user either confirms or resets the values after entering them. But it allows for instant feedback on changes while still supporting the ability to dump the changes if the user decides they don’t like the result. I think its a useful pattern for any situation where the user is entering data and you want to show them a preview of the outcome. I hope others will find it as useful as I have.

Update

I showed this to Ryan and he replied with a nice simplification of the concept.

@ben336 nice Ben. I have had to do something similar in the past. Here is a simplified version that I have used: jsfiddle.net/rniemeyer/SFCg…
— Ryan Niemeyer (@RPNiemeyer) May 8, 2013

//Ryan Niemeyer's simplified knockout-revertible.js
//https://jsfiddle.net/rniemeyer/SFCgr/ Fiddle
 ko.revertibleObservable = function(initialValue) {
    //private variables
    var result = ko.observable(initialValue);

    result.forEditing = ko.observable(initialValue);

    //if different, commit edit value
    result.commit = function() {
        var editValue = result.forEditing();

        if (editValue !== result()) {
            result(editValue);
        }
    };

    //force subscribers to take original
    result.reset = function() {
        result.forEditing(result());
    };

    return result;
};

Irreplaceable

Sat, 04 May 2013 13:00:00 -0400

If you are the average worker in the average job then you can be replaced. The reason I tell people this is because I want to help them realize that there are important things in life and then there are VERY important things in life. Make sure you are good in the areas of life that you aren’t replaceable (family) and then focus on the other areas (work). This is my view, but I think anyone with a family probably feels the same way. Telling someone they are replaceable is a great cure for arrogance.

I appreciate the work that my folks do and tell them that, but Microsoft will go on without them. Think about the last time someone left your team. How long did it take before they were a distant memory? Probably a day or so then the team moves on with their work. I’m not saying don’t care about work. I’m saying care about the VERY important things more. If your work/life balance is off and the VERY important things are threatened, then take a break and get some help.

At work you can be replaced in about 5 minutes, but you are effectively irreplaceable at home. Keep that in mind. I think about it every day.

My Thoughts on Work/Life Balance at Microsoft- Jon Gallant

Jon Gallant speaks the truth. Aside from any personal beliefs on which is more important or meaningful¹, its pretty clear that it would be much easier for your employer to find somebody to do your job than it would be for your spouse/children to find their way without you. We all have a desire to be significant and necessary I think, we just don’t always value it in the places we are.

I fully agree with him on that. Family will always be first for me. ↩

Book Review: Javascript Testing with Jasmine

Tue, 23 Apr 2013 16:30:00 -0400

I received a copy of Javascript Testing with Jasmine by Evan Hahn this week, and it seemed like perfect timing. The developers on my team at work have been discussing using Jasmine to automate the testing of Javascript code in our product, and I may be spending a significant amount of time with it soon. I was hoping to pick up on some tips and tricks, and maybe learn some things about testing. It ended up being a quick read with some useful tidbits that left me wanting more.

What is it?

Javascript Testing with Jasmine is a short little book about testing with Jasmine, a testing framework inspired by the Behavior Driven Development style. The book is short (only 52 pages), which is appropriate for its limited scope. It’s split into 7 chapters with each chapter comprised of a few pages of example and explanation:

What is Software testing?
Jasmine
Writing good tests
Matchers in depth
More Jasmine Features
Spies
Using Jasmine with other tools

What’s it like?

Unfortunately the book manages to simultaneously feel like its filling space in many spots, and like it missed opportunities to provide value in others. Chapters 4 and 5 for instance fail to say much that is not already in Jasmine’s “describe by showing”-style documentation. Chapter 1 seems to be searching for interesting things to say about testing, and failed to meaningfully explain the distinction between BDD and generic Test Driven Development. The section on Jasmine and Coffeescript in Chapter 7 could be succinctly summarized as “Yes you can use Jasmine with Coffeescript”. At the same time, the book misses opportunities to talk about things like writing testable code, or refactoring existing code to be meaningfully tested with Jasmine. I would have loved to see more about automating testing and fitting it into the development process, or running tests from the command line with phantomJS.

That’s not to say that there’s nothing good here. I found the explanation of spies to be very helpful, and the detailed documentation-like aspects would be more helpful to a Javascript beginner than Jasmine’s “show me the code” docs. All in all the book serves as a great reference guide for the library. I’m just not sure how much value it adds compared to a quick Google search.

Summary

In the end this feels like the type of book that made sense before Google took over the Internet. It provides all the basic how-tos and background information, but fails to provide any deep insight. If you’re starting a new Javascript BDD project and want a quick reference this may make sense as a quick reference guide, but for most people the information in here can be found just as easily with a little use of your favorite search engine and a trip to the Jasmine docs.

Where can I buy it?

You can get it from Amazon here: JavaScript Testing with Jasmine (Affiliate Link)

A New Look

Sat, 06 Apr 2013 13:00:00 -0400

I’m switching to a modified version of the excellent Object theme by gluttony for this blog. I like the cleaner content-focused look that it brings, and especially feel like it helps my code samples stand out. I hope you like it.

My site continues to be generated as static pages by octopress, and written using Sublime Text. Both are unreservedly recommended.

Creating a Build System For a Coffeescript Project with Ant

Sat, 23 Mar 2013 12:23:00 -0400

I’ve recently taken the time to develop an organized build system for a Coffeescript and SASS project that I’ve been working on in my free time. I learned a lot during the process, and I wanted to share my experience here. I used Ant for my build system, and worked in a variety of different libraries for minification, testing and documentation.

Getting Started

The first step is to install Ant. It comes preinstalled on OSX, is fairly straightforward to install on Linux, and is a bit more involved to install on Windows If you’re using my exact setup, you’ll also need NodeJS (for Coffeescript, UglifyJS,and Docco) and Ruby (for SASS) installed.

My Project Structure

I went for a structure with 4 directories.

src
build
test
docs

All of the source files are in the source directory, with Javascript and Coffeescript files in a scripts directory, and SASS styles in a styles folder. This particular app has only a single HTML file, which sits in the root of the source directory. The test folder contains a single HTML page, and a series of unit test files written in Coffeescript, in a structure that mirrors the src directory. The build and docs directories are initially empty.

The Basics

We start with a simple Ant setup. We’ll create an init task, a clean task, and debug and production tasks. The following code goes into our build.xml file, which is placed at the root of the project directory.

<project name="scratchcalc" default="debug">

    <loadproperties srcfile="build.properties" />

    <target name="debug" depends="clean, init">
        <echo>ScratchCalc Version ${version} (Debug Version)</echo>
    </target>

    <target name="production" depends="clean, init">
        <echo>ScratchCalc Version ${version}</echo>
    </target>

    <target name= "init">
        <mkdir dir="${build.dir}" />
    </target>

    <target name="clean">
        <delete dir="${build.dir}" />
    </target>

</project>

So for this project, we now have 4 tasks that we can call. To call the init task, all we need to do is run

Ant init

from the command line. Both our debug and production tasks mark clean and init as dependencies, so running them will remove and restore the build directory. Beyond that, our build doesn’t do much right now. Lets change that.

Copying

The simplest thing we can do with our Ant file is basic copying of files. So lets start with a task that just copies over our files from the src directory to the build directory.

<target name="copy" description="Copies files from src to build dirs">
    <echo>Copying JS, CSS & HTML Files</echo>
     <copy todir="${build.dir}">
        <fileset dir="${src.dir}" includes="**/*.HTML,**/*.css,**/*.js" />
     </copy>
</target>

This simply copies any HTML, CSS, or Javascript files in the src directory over to an equivalent position in the build directory. The segments in the ${} format are variables defined in our build.properties file. For my project, that looks something like this so far:

version = 0.0.1
copyright = Copyright 2012-2013 Ben McCormick.
src.dir = ./src
build.dir = ./build
test.dir = ./test
lib.dir = ${src.dir}/ext

This would be a great place to start for a project thats currently running without a build system. If you’re able to get your project to build like this, it will set you up to be able to start more interesting things going forward.

And we of course want to do more interesting things. This project uses Coffeescript and SASS, so we’ll need to do a bit more than just copy files. So lets copy those scripts as well, but convert them to Javascript and CSS while we do.

<target name="copy" description="Compiles Coffeescript and SASS files,copies the rest">
    <echo>Copying Coffeescript Files</echo>
     <copy todir="${build.dir}">
        <fileset dir="${src.dir}" includes="**/*.HTML,**/*.css,**/*.js" />
     </copy>
     <exec executable="coffee">
          <arg value="--compile"/>
          <arg value="--map" />
          <arg value="--output" />
          <arg value="${build.script.dir}" />
          <arg value="${src.script.dir}" />
     </exec>
     <mkdir dir="${build.styles.dir}" />
     <apply executable="sass" dest="${build.dir}" verbose="true" force="true" failonerror="true">
        <arg value="--unix-newlines" />
        <srcfile />
        <targetfile />
        <filelist dir ="${src.dir}" files="styles/scratch.sass" />
        <!--<fileset dir="${src.dir}" includes="**/*.sass"  />-->
        <mapper type="glob" from="*.sass" to="*.css"/>
    </apply>
</target>

So along with copying the raw files, we also now copy the Coffeescript and SASS files. For Coffeescript, we run the coffee executable against the entire src dir, and have it mirror the output to the build dir. For Sass, we only want to run the compile on a single file, which pulls all of the other files into it through import statements. So we use the filelist and mapper to specify the input and output files.

Concatenation

Mirroring your directory can be helpful for debugging, and is a good initial step if you’re converting to Ant from a locally run project. But we’d really like to limit the amount of requests we make when deploying our site. To do that, we’ll concatenate the js files together. For my project I create 2 files, one for my external dependencies, and one for my own local files. I may eventually reduce this to 1, but for now I like separating the 2 for debugging clarity.

<target name="concatenatejs" description="Concatenate All the JS together">
    <!-- Instead of concatenating the JS files directly, use coffee to preserve the source map -->
    <echo>Concatenating the JS</echo>
    <concat destfile="${build.libs}">
         <filelist dir ="${build.script.dir}" files="lib/jquery.js,lib/knockout.js,lib/BigDecimal.js" />

          <fileset dir="${build.script.dir}/lib" includes="**/*.js"/>
    </concat>
    <apply executable="coffee"  verbose="true" force="true" failonerror="true" parallel="true">
    <arg value="--compile" />
    <arg value="--map" />
    <arg value="--join" />
    <arg value="${build.script.output}" />
    <srcfile />
    <filelist dir ="${src.script.dir}" files="editor/toolkit.coffee,editor/markupGen.coffee,parser/eqTreeBuilder.coffee,parser/NumberValue.coffee,parser/eqScanner.coffee,parser/eqTokenizer.coffee,parser/tablePlaceHolder.coffee,parser/EQParser.coffee,parser/calcFramework.coffee,editor/editor.coffee" />
    </apply>
</target>

For the external libraries I use Ant’s built in concat task to concatenate the files together. For the Coffeescript files, I use the coffee compiler rather than concatenating the files that I copied. I was hoping to do this in order to preserve the source map, which allows for easier debugging. Unfortunately the compiler has trouble generating source maps for joined files right now. I’m hoping to fix that in the future, in the meantime I may be using a different approach, possibly using uglifyjs to concatenate the files and generate the maps.

Minify

Speaking of uglifyjs, I use that for minifying my Javascript.

<target name="minify" description="Minify the JS for Production">
    <echo>Minifying the JS For Production with UglifyJS</echo>
    <apply executable="uglifyjs" verbose="true" force="true" failonerror="true">
        <srcfile />
        <arg value="--output" />
        <arg value="${build.script.output}" />
        <filelist  files ="${build.script.output}"/>
    </apply>
</target>

I only require this for the production task, and leave the full source available for the debug task to preserve the readability of the generated JS.

Documentation

I use docco for my Coffeescript documentation. It converts comments into documentation and runs them through a Markdown parser to generate a clean HTML page with the code and comments intermingled.

<target name="documentation" description="Generate Docco Documentation for coffee files">
    <echo>Generating Documentation with Docco</echo>
    <apply executable="docco" verbose="true" force="true" failonerror="true">
        <srcfile />
        <fileset dir ="${src.script.dir}" >
            <include name="**/*.coffee"/>
        </fileset>
    </apply>
</target>

Testing

I use Jasmine for testing. Jasmine is a “Behavior Driven Development” style unit testing framework for Javascript. In an ideal world I would be setting up a build task to run on the server and display the results when I run the build. Unfortunately I wasn’t able to find a way to do that without generating additional dependencies. Its possible with NodeJS or RequireJS style modules and dependencies, and its also possible using phantomJS, but I didn’t want to require either of those for testing. So instead I have my build task generate an HTML page with the test results. I may add some scripting to open this page by default after running a production build in the future.

<target name="test" description="Generate the JS for testing with Jasmine">
  <!-- Ideally this would run a server side jasmine test, but that requires better dependency management than we have right now-->
  <echo>Generating the Test JS with jasmine</echo>
  <apply executable="coffee"  verbose="true" force="true" failonerror="true" parallel="true">
    <arg value="--compile" />
    <arg value="--join" />
    <arg value="${test.output}" />
    <srcfile />
    <fileset dir ="${test.dir}" >
      <include name="**/*.coffee"/>
    </fileset>
  </apply>
</target>

Summary

Adding a build system has been great for my project. Its allowed me to be a lot more organized and to do some cool things that are a pain by hand (unit testing, Coffeescript). I definitely recommend looking into doing this for any non-trivial Javascript/web project that you’re working on.

<project name="scratchcalc" default="debug">

  <loadproperties srcfile="build.properties" />

  <target name="debug" depends="clean, init, copy,concatenatejs">
    <echo>ScratchCalc Version ${version} (Debug Version)</echo>
  </target>

  <target name="production" depends="clean, init, copy,concatenatejs,minify,documentation,test">
    <echo>ScratchCalc Version ${version}</echo>
  </target>

  <target name="copy" description="Compiles Coffeescript and SASS files,copies the rest">
    <echo>Copying Coffeescript Files</echo>
     <copy todir="${build.dir}">
      <fileset dir="${src.dir}" includes="**/*.HTML,**/*.css,**/*.js" />
     </copy>
     <exec executable="coffee">
        <arg value="--compile"/>
        <arg value="--map" />
        <arg value="--output" />
        <arg value="${build.script.dir}" />
        <arg value="${src.script.dir}" />
     </exec>
     <mkdir dir="${build.styles.dir}" />
     <apply executable="sass" dest="${build.dir}" verbose="true" force="true" failonerror="true">
          <arg value="--unix-newlines" />
          <srcfile />
          <targetfile />
          <filelist dir ="${src.dir}" files="styles/scratch.sass" />
          <!--<fileset dir="${src.dir}" includes="**/*.sass"  />-->
          <mapper type="glob" from="*.sass" to="*.css"/>
      </apply>
  </target>

  <target name="concatenatejs" description="Concatenate All the JS together">
    <!-- Instead of concatenating the JS files directly, use coffee to preserve the source map -->
    <echo>Concatenating the JS</echo>
    <concat destfile="${build.libs}">
       <filelist dir ="${build.script.dir}" files="lib/jquery.js,lib/knockout.js,lib/BigDecimal.js" />

        <fileset dir="${build.script.dir}/lib" includes="**/*.js"/>
    </concat>
    <apply executable="coffee"  verbose="true" force="true" failonerror="true" parallel="true">
      <arg value="--compile" />
      <arg value="--map" />
      <arg value="--join" />
      <arg value="${build.script.output}" />
      <srcfile />
      <filelist dir ="${src.script.dir}" files="..." />
    </apply>
  </target>

  <target name="documentation" description="Generate Docco Documentation for coffee files">
    <echo>Generating Documentation with Docco</echo>
    <apply executable="docco" verbose="true" force="true" failonerror="true">
          <srcfile />
          <fileset dir ="${src.script.dir}" >
            <include name="**/*.coffee"/>
          </fileset>
      </apply>
  </target>

  <target name="test" description="Generate the JS for testing with Jasmine">
    <!-- Ideally this would run a server side jasmine test, but that requires better dependency management than we have right now-->
    <echo>Generating the Test JS with jasmine</echo>
    <apply executable="coffee"  verbose="true" force="true" failonerror="true" parallel="true">
      <arg value="--compile" />
      <arg value="--join" />
      <arg value="${test.output}" />
      <srcfile />
      <fileset dir ="${test.dir}" >
        <include name="**/*.coffee"/>
      </fileset>
    </apply>
  </target>

  <target name="minify" description="Minify the JS for Production">
    <echo>Minifying the JS For Production with UglifyJS</echo>
    <apply executable="uglifyjs" verbose="true" force="true" failonerror="true">
          <srcfile />
          <arg value="--output" />
          <arg value="${build.script.output}" />
          <filelist  files ="${build.script.output}"/>
      </apply>
  </target>

  <target name= "init">
    <mkdir dir="${build.dir}" />
  </target>

  <target name="clean">
    <delete dir="${build.dir}" />
  </target>

</project>

Update June 2014: Since writing this post, Javascript based build tools have continued to mature and grow. These days I’d recommend Grunt or Gulp for your front end build script needs, though Ant is still a fine choice if you’re integrating your builds with a Java project, or have lots of experience with it.

Coffeescript Is Great

Sat, 23 Feb 2013 19:50:00 -0400

This past week I decided to try and learn more about Coffeescript by converting a side project I’ve been working on to use Coffeescript and LESS. I really enjoyed the experience of working with Coffeescript and hope to use it more in the future. I thought I’d share a quick post about my experience, and outline what I see as the advantages and disadvantages.

What is Coffeescript?

Coffeescript is a programming language with a Ruby-like syntax that compiles transparently into readable, best-practice Javascript. To break that down:

It’s a language with Ruby-like syntax

Coffeescript features a short concise syntax with significant whitespace, short operators for many common actions, and syntax that encourages a pseudo-english style of coding. For instance if empty page then fill page else read page would be valid syntax for calling a function empty() on an object page and then calling the fill method on it if it was empty or the read method if it was not.

It compiles to readable best-practice Javascript

The coffeescript compiler generates Javascript that is easy to read and avoids common pitfalls like unexpected casting with the == operator.

It compiles transparently

The generated javascript also matches up in a 1-1 relationship with the coffeescript source, so it’s still easy to debug and understand what coffeescript is related to the associate Javascript. I’m also excited to try out the Coffeescript source maps for Chrome and bypass this issue completely by debugging directly on the Coffeescript, but haven’t had the opportunity to set this up yet.

What I like about Coffeescript

It’s super concise

I love writing short code. Brevity and readability aren’t exactly the same thing, but they often overlap, and it’s usually easier to debug code when there’s less of it, all things being equal. Coffeescript allows you to write less boilerplate. An easy example is checking whether an element exists. Compare the JS implementation below to the equivalent Coffeescript.

if(el !== null && typeof el ?== 'undefined'){
    //do something with el
}

if el?
  #do something with el

It doesn’t look like Java

This is a bit counterintuitive coming from a guy who learned programming through Java and has never written a line of Ruby, but I think Javascript’s “Java-like” conventions are usually detrimental. It has curly brace blocks, but not block level scope. It has constructors but not classical inheritance. It’s possible to write valid code that looks exactly like Java in Javascript, and not understand why the behavior is slightly different than you expect. Coffeescripts syntax feels less likely to lead you down a bad road.

It avoids the “bad parts” of Javascript

Javascript has some “features” that cause more problems than they’re worth, including the == operator, automatic semicolon insertion, and the with statement. Coffeescript either avoids these parts completely, or helps the programmer navigate them without having to think about it. So you can still write your while statements by dropping into normal Javascript code if you must, but Coffeescript won’t make it easy.

What I didn’t like so much

Debugging is slower

While it’s certainly much better than it could be due to the transparent compilation, debugging in coffeescript still means that you’re debugging a “product” of your code, not the code itself. That level of abstraction has a cost. That cost will vary depending on your experience, code style and tooling (certainly source maps have potential to minimize it), but it will exist, and it needs to be factored in to any language decisions.

Function Parameters without Parentheses

Coffeescript lets you write function parameters without parentheses. While this allows the cute sentence-like one liners that they demonstrate on their website, in practice I’ve found it to really hurt readability. It’s often not clear where the function begins and ends, especially if you’re nesting function calls. Coffeescript is also not consistent in how it applies these rules, because in some cases you DO have to use parentheses. When declaring a function you must use parentheses if there are 1 or more arguments. When calling a function you must use parentheses if there are no arguments. I’ve found that I struggle to read function calls if there are more than 1 parameters without parentheses. As a result I’ve used parentheses everywhere there is one or more parentheses, and for now on I’m planning on including them even for single param functions just to be consistent.

Conclusions

I’m very glad I tried Coffeescript. I think it definitely has its place as a concise syntax on top of Javascript that especially makes sense for larger projects where the investment in adjusting to it will pay off over time. I overall think it adds readability and maintainability from Javascript, though it is by no means perfect in either area. I’m excited to see what happens with this language going forward.

Simple Publish-Subscribe with jQuery

Wed, 13 Feb 2013 17:00:00 -0400

Here’s a cool pattern that I’ve been able to use in two of my projects lately:

//Publish Subscribe with jQuery

var exampleHandle = function(){
	//do stuff when topic is published
	...
}

function subscribe(topic,handle){
    $("#subscription").bind(topic,handle);
}

function publish(topic,params){
	$("#subscription").trigger(topic,params)
}

function unsubscribe(topic,handle){
    $("#subscription").unbind(topic,handle);
}

Publish-Subscribe is a pretty commonly used model for sending messages in software applications. Implementations can vary, from the very heavyweight setups of things like Java Message Service to much lighter methods. This is an example of a lightweight pubsub implementation for javascript.

Implementation Details

The method is based on jQuery’s trigger and bind functions. These allow you to listen for a custom event on an element, and manually trigger an event on an element. This provides the basic backbone for the simple subscription model.

If an application element wants to subscribe to a topic, they bind a handler to the “subscription element”. This can be a designated element on the page, or just the window element. You can also of course use different elements for different subscriptions. Then, when something publishes to that topic, the handler function will execute.

For publishing, a function can pass a topic and parameters to the publish function. This calls jQuery’s trigger to set off the event topic, passing along the parameters. These params are passed to the handle function.

If an element wants to cancel its subscription, they can pass the topic and handle function to the unsubscribe method. Note that the handle has to be the same function object that was used to originally subscribe, not a copy or similar function. You also can use jQuery’s unbind to cancel all subscriptions to a topic by only specifying the topic without the handle function.

I’ve found this to be a useful, lightweight way of creating loosely coupled messages across different components of my software systems.

How I Work: Refactoring

Sun, 03 Feb 2013 17:00:00 -0400

I recently took the time to re-examine a section of code that I’ve worked on as part of a team for the last year. Looking at it with fresh eyes I saw plenty of room for improvement. The issues weren’t the result of one bad checkin or poor decision. Instead they were the result of “death by a thousand cuts”, a series of small decisions made by different people that were individually justifiable but eventually resulted in brittle, hard to maintain code.

Over the last few days I’ve been taking some time to refactor the code. Doing so made me think about what questions I should be asking while refactoring code. Here are a few of the things I look for when I’m working to clean up code:

Does each function do only one thing?

This has been the hardest bad habit for me to break. I have a natural inclination when adding new functionality to try to toss it into a related function. This seems great as first, but its how you go from a function that “removes special characters from a string”, to one that “removes extra characters from a string”, to one that “processes strings”, and finally to “the string function” that takes a string and returns a new output determined by a glob of unreadable code that nobody understands and everyone is scared to touch.

Functions that perform only one purpose are easier to understand, easier to use, easier to test, and easier to port to new environments. When functions start to perform multiple purposes it becomes harder to maintain DRY principles. If one function performs actions A&B, another performs B&C, and a 3rd one performs A&C, you have to duplicate each action twice, adding code bloat and opening up the risk of inconsistent changes across methods.

Does each object/concept have a consistent naming convention throughout the system?

One source of frustration I’ve had when debugging the code for my project has been the fact that different developers have started different naming conventions for objects within the code. This results in a single object being called chartObj within function, chart within another, params in another place, and chartData in another instance. Sometimes these different references even pop up within the same function. This inconsistency makes it much harder for an unfamiliar developer to understand how the system functions. This again can lead to duplicating information, if a developer modifying a function or class doesn’t realize they already have the information they need.

What assumptions have I made?

One thing that can easily add confusion within a system is making undocumented assumptions. One common assumption thats made for javascript code is that it will always run in a browser, or have access to a specific library such as jQuery. While some of these assumptions are necessary, its worthwhile documenting them in some way, either in the comments or through code checks that through exceptions if the assumptions are violated.

This is important because requirements change. The particular code that I’m working with was originally intended to run only in the browser, without any thought of server-side use. When a requirement made it necessary to run it with PhantomJS, plenty of changes were needed. This isn’t necessarily problematic, because this particular change was not likely when the original code was made, and the adaptation was not hard. More problematic was the fact that developers not directly involved with the phantomJS work still were acting under the assumption that the code would run within a browser environment.

These three questions are obviously not the only things that can be wrong in code, but in my experience they’re especially treacherous because they can come in slowly and then make every future change harder.

Career Fairs: How To Not Get Hired (And How To Give Yourself A Chance)

Fri, 25 Jan 2013 15:30:00 -0400

It was 9:15 and I had just arrived at the gym where the career fair was set to start at 10:00. Company representatives were setting up their wares in the background and enjoying the provided bagels and coffee, chatting with each other before the rush set to begin at 10. Arriving at my company’s booth, I tossed my bag down and leaned over to fill out a name tag. As I stood up, I was greeted by a sheet of paper being shoved in my face. “Here is my resume!” Bob said as he transferred the paper from my face to my hands. I was then greeted with a 2 minute monologue explaining why Bob would be the perfect candidate to work for my company. Eventually I was able to get a word in edgewise, introduce myself, and ask him to come back in 45 minutes when the fair started. As I walked over to get my much needed bagel, I reflected to myself that Bob would really have to be super impressive to get an interview when he came back to our booth.

Career Fairs should not be hard. When recruiters come to a career fair, it is with the sole mission to find students who they can interview for jobs. When students come to career fairs, they do so to find a company willing to give them a job. But a lot of the time both sides leave disappointed. I received my first interview for my current company at a career fair, and have now worked at several as a recruiter. There is plenty that can be said about how recruiters can improve in how they reach out to students and other candidates. But in this post I’d like to point out a few simple things that students can do (or not do) to stand out from the crowd and give themselves their best chance for an interview.

Prepare

This is repeated a lot by career counselors, and I think its hard to understand from the student point of view sometimes. Students think of career fairs as a place to learn about companies. And thats valid to some extent. Certainly for younger students its a great way to see what options are out there. But as a recruiter the first thing I want to know about a student is “what type of position are you interested in” If they can’t answer that question, no matter how impressive they are, I would be hesitant to give them an interview.

Preparing for a career fair means knowing what you want, and having at least some idea who might be able to help you get it. There’s fairly little I can do to help a student who comes up to me and asks “So do you guys have any jobs that don’t involve technology?” which is actually probably the most common question we receive. There’s much more that we can say to help a student who comes to us and tells us that they’re interested in marketing, or environmental concerns, or really anything else specific. The second student lets us picture how they could fit in and help our organization, the first student just emphasizes how they don’t fit.

Never apologize for yourself

This may be the biggest problem that students have communicating with recruiters. Perhaps because of the perception of a loaded job market, students seem much more aware of how their resumes fall short compared to their peers rather than how they stand out. Unfortunately they tend to convey this to recruiters rather frequently. Bob’s early morning greeting certainly had its issues, but one thing that he did really well was emphasize what he would bring to our company.

Something that students fail to understand is that recruiters are usually looking for reasons to like an applicant. They will take a chance on a candidate with weaknesses on his resume if he or she is genuinely able to show how they can fit into a company. What is a lot harder to work with is a large assortment of candidates who all have good GPAs, some reasonable experience, and absolutely nothing to differentiate themselves from others. In an interview setting a student will probably have to explain any weaknesses on their resume. But a career fair is not an interview. Its an opportunity to make an impression. That happens when a student shows that they will bring something positive and useful to the company.

Take chances, and learn from your mistakes

One of the great things about career fairs is that you have the opportunity to instantly apply new lessons you’ve learned. Recruiters don’t have some sort of group social network that they’re keeping profiles of you on. If you go to one company’s booth and have a bad conversation, that doesn’t follow you around to the rest of your interactions with companies unless you let it. Instead you can learn from your mistakes and hopefully get more comfortable and confident as you get more used to how career fair interactions work.

Don’t try to cheat the process

Our early morning encounter was not the last I saw of Bob that day. He came back an hour later and I got to hear another, slightly toned down, explanation of why he would be perfect for my company. I asked him a few questions and as we wrapped up I told him we would take his resume and be in touch if we were able to offer him an interview. He asked if I would be able to sign him up for an interview right then. When I said that I could not, he then asked if he could talk to my coworker who was also recruiting for the same position. I politely told him that I couldn’t stop him, but that I’d prefer that he left him free to talk with other candidates, and that I’d be happy to answer any more questions he had. He said that it was fine and walked away. Five minutes later, I looked over to see my coworker being treated to Bob’s full explanation of why he would be a great candidate.

Assertiveness is in no way a bad thing. But being able to accept a no and learn from it is also important. Recruiters will generally be happy to tell a candidate how they could work to be a better candidate for their job, and some may even be willing to meet or exchange emails outside the fair setting to work with a candidate. If you don’t receive an interview, building a relationship can be an even better outcome of meeting with a recruiter at a career fair. But trying to subvert the system will pretty much guarantee that you end up with neither. Other examples of this include trying to badger a recruiter for information they’ve said they don’t know or can’t give, talking down other candidates, or (more obviously) lying about your qualifications.

Pay attention

I received this advice from a speaker at Carnegie Mellon before I went to the career fair that ended up leading to my first fulltime job. It turned out to be the best thing he could have said to me. When you’re in line to talk to a recruiter at a career fair, pay attention to the situation. What type of questions are they asking candidates? Are they mostly giving information, or trying to ask more interview style questions. Some companies will even ask technical questions at a career fair, and thats certainly something that you would want to know before you start talking to them.

In my case by listening I was able to learn that the recruiter I was interviewing was frustrated by the lack of Computer Science candidates he was meeting that day, and particularly frustrated that he was having to turn away qualified international students because he had a limit on how many candidates his company could sponsor for overseas work. When it was my turn to talk to him I was able to open by telling him that he could “accept or reject me on my own merits”, and then focused on telling him about my undergraduate computer science degree, with a lesser emphasis on my more management focused graduate program. As a result I was able to open a personal connection and present myself as an answer to his frustrations, whereas coming in unaware I may have set myself up as just another student in the wrong degree program who was less of a fit than the international students he was having to send away.

Don’t forget that recruiters are human

The first 5 points all point to this last one. Preparation is a way of showing recruiters that you value their time. When you present yourself as meeting a need rather than apologizing, you’re helping them solve their problem. You can take chances at career fairs because recruiters are not automatons blindly picking the best resumes, and relationships and personal interaction actually matter. You shouldn’t cheat the process, because in doing so you’re treating other human beings disrespectfully. And you should pay attention because when you don’t you’re forgetting that this is a human interaction, and missing out on much of what is involved in it.

Bob’s biggest mistake was forgetting that he was dealing with a human being. When he approached me in the morning it was with no awareness of my frame of mind or personal preferences. He was on a mission to get a job. But he severely sabotaged his efforts by overlooking the human factors of context, mood, and etiquette. If you can treat a recruiter as a human being, you’ll find that they are much more likely to be your ally in getting a job.

Unexpected Javascript: That Doesn't Do What You Think

Tue, 15 Jan 2013 19:30:00 -0400

When I started writing Javascript, I encountered a lot of stuff that was just plain weird. Stuff didn’t act the way I expected, syntax seemed misleading, and some behavior just seemed completely wrong. I have a better understanding now and have come to appreciate Javascript for the quirky but stable language that it is once you know the rules. But just for fun, here are some examples of assumptions that its easy to make when looking at Javascript for the first time. All of the following statements are wrong (at least in some cases). See if you know why. And feel free to contribute more in the comments!

Equality Problems

0.1+0.2 === .3 is true ¹
10000000000000001+20000000000000000 === 300000000000000001 is true ²
x === x is true for all values of x ³
"1.0e0" == {valueOf: function(){ return true;}}; is false ⁴
if x==y and x==z then z==y ⁵
parseInt('016') === 16 is true. ⁶

Environment Concerns

window will always be available as a global variable ⁷
Javascript will always be enabled on your web page ⁸
If it works in firebug it will work in the browser ⁹

Code Structure

You don’t need to add semicolons in javascript, they’re inserted automatically ¹⁰
I always add semicolons, so I don’t need to know about how they’re inserted ¹¹
The var keyword is unnecessary, variables are created just by referencing them. ¹²
return eval("x") is the same as var a = eval; return a("x"); ¹³
Functions are the only thing in Javascript that restrict scope. ¹⁴

Function Issues

I will always know what this is when the function is run ¹⁵
You can get the last argument of the function using arguments.pop() ¹⁶
The following will generate different names and alerts for each link: ¹⁷

function addLinks () {
  for (var i=0, link; i<5; i++) {
    link = document.createElement("a");
    link.innerHTML = "Link " + i;
    link.onclick = function () {
        alert(i);
    };
    document.body.appendChild(link);
  }
}

The following function will return 1: ¹⁸

(function f(){
    function f(){ return 1; }
    return f();
    function f(){ return 2; }
  })();

The if statement code can be removed here with no side effects: ¹⁹

function testf(){
  x=3;
  if(false){
    var x=5;
  }
}

Explanations here:

Actually comes to 0.30000000000000004 See here ↩︎
Loses the trailing 1 due to limits of floating point numbers ↩︎
It turns out that NaN === NaN is false ↩︎
Both values get cast to 1 by the == operator ↩︎
'' == ‘0’ is false, 0=='' is true, as is 0 ==‘0’ ↩︎
Prior to ECMAScript 5, strings with leading 0s were interpreted as Octal Notation. This is resolved in newer browsers, but IE8 and other older browsers still show this behavior. ↩︎
Window is a browser construct. It may not be available in all JS environments (for instance, it is not available in Node.js) Unless you know exactly where your code will be run, be careful ↩︎
Especially in government situations, security conscious users may disable javascript. Even if the user is able to enable JS, it may be disabled by default. You can prepare your site for this. ↩︎
This is not always accurate. Firebug executes in eval context, and that may lead to strange behavior . ↩︎
There are some cases when a JS parser may unexpectedly concatenate 2 lines you didn’t intend. This is particularly an issue when the next line begins with an arithmetic operator or a (, as is the case for immediately invoked function expressions. You can see an example with this fiddle ↩︎
You can still be burned by automatic semicolon insertion even if you don’t want to take advantage of it. The classic example is that Javascript will automatically insert a semicolon after return if you put the value to be returned on a new line, something that may happen often if you’re returning a new object and like to start your curly braces on a new line. ↩︎
This is more of a beginner assumption. Undeclared variables are automatically set as global variables and can cause plenty of problems if you don’t understand this. ↩︎
Strangely, Javascript differentiates between direct calls to eval and indirect calls. You can read more on this StackOverflow Question ↩︎
This is admittedly an edge case, but catch statements of try…catch blocks also restrict scope, and the with statement also modifies the scope, in a different way. with is not recommended ↩︎
Even methods of an object can be called by another object with Javascript’s call and apply functions. The only way to guarantee that the correct value of this is used is to use Javascript’s bind function ↩︎
arguments is not a real array. It is array-like, in that it has a length value, and has integer properties that it accepts. But you can’t run normal array arguments on it without converting it to an Array object. ↩︎
This is the classic example of improper closure behavior. I got this example from this article, where you can read more. ↩︎
Function declarations are hoisted to the top of the function. Therefore both versions of f are defined, and the second one overrules the first, before f is called and return ↩︎
More variable hoisting. In this case the var x declaration is hoisted to the top of the function, even though the assignment can never be executed. This has the effect of making the variable local to the function rather than global, which is potentially very significant. You can see the difference in this jsFiddle/ ↩︎

Explaining Javascript: Object Oriented Programming

Fri, 11 Jan 2013 23:05:00 -0400

I’m continuing this series on Javascript concepts for people new to the language by looking at object oriented programming. One of the more confusing concepts for new Javascript programmers, especially those used to the classical inheritance structures of Java or C++ is Javascript’s prototypical inheritance model. Because it is possible to write syntax for Javascript object creation that can look very similar to classical inheritance in Java, its easy to be misled and have false expectations. Its important to understand how Javascript’s object model works, so that you can adjust your expectations and write code that takes advantage of the language’s expressive power.

Creating a new object

Javascript provides multiple ways to create new objects. The simplest way is to just create an empty object with {}. The one I’ll be looking at today was modeled after Java syntax and gives the language a deceptively classical vibe:

//Creating Objects with new

    function Animal(){
        this.home = "earth";
        this.noise = "roar";
    }

    Animal.prototype.sound = function(){
        alert(this.noise);
    }

    var animalObject = new Animal();
    animalObject.sound(); // roar
    var home = animalObject.home //earth

This all looks very familiar and comfortable to a Java programmer. Animal() is the constructor of a class Animal, with a function sound() and animalObject is an instance of that class. And you can write code like that if you want. The problem is, that model is not enforced at all by javascript. Javascript doesn’t have any special “constructor functions” or classes¹. Any function can be used as a constructor

//Normal Function as constructor

    function normalFunc(){
        alert("This is a normal function")
    }
    var normal = new normalFunc();
    normal instanceof normalFunc  //true
    var x = {};
    x.y = normalFunc;

    var huh = new x.y();
    huh instance of normalFunc //true

So whats actually happening when you call new before a function? The new operator generates a new object, binds this to that object for the constructor function, calls that function and sets that function as the prototype of the new object. But what is a prototype?

Prototypes

Javascript inheritance is at its heart “prototypical inheritance”. This means that rather than having a concept of “classes” and “implementations” of those classes, javascript objects inherit their properties directly from other objects. So for the “classical” example above, what is actually happening is that a new object is being created, it is having some of its properties set by the constructor function, and then has the constructor’s prototype set as its prototype. This means that it has access to any properties on the “prototype chain” of the constructor, which in this case includes the sound method. All objects have acccess to Object.prototype, giving it access to object properties and methods like toString, hasOwnProperty and others.

The ES5 Way

The ECMAScript 5 specification defines a new way to create objects from existing objects with the Object.create() syntax. Object.create takes an existing object and sets it as the prototype object for a newly created object². This was added to make the syntax for setting the prototype of a new object easier and more clear.³ It also allows us to simulate classical inheritance:

//ES5 Classical Inheritance example

    var Cat = function(){
        Animal.call(this); //Call the Animal constructor to set local variables
        this.noise = "meow" //override the Animal sound
    }
    Cat.prototype = Object.create(Animal.prototype)

   var kitty = new Cat();

   kitty.sound() //meow
   alert(kitty.hasOwnProperty("sound")) //false
   alert(kitty instanceof Cat) //true
   alert(kitty instanceof Animal) //true

So as you can see, it is very possible to write Object-Oriented Javascript. The key is to understand that what’s happening behind the scenes in Javascript is different than Java or C++, even though the syntax may look similar.

Summary

Inheritance and object oriented programming in javascript are possible, but are more of a programmer defined construct rather than an inherent property of the language
Javascript supports prototypical inheritance, meaning that it inherits functions and properties from its prototype object. All objects inherit from Object.prototype.
ECMAScript 5 introduced support for Object.create, which provides an easier syntax for creating new objects and simulating inheritance.

Explaining Javascript

This is the second article in my Explaining Javascript series for introducing Javascript concepts to new JS developers.

In the next version of Javascript ES6 there will be a more formal concept of classes. You can read more here if interested ↩
With an optional second argument to set the local properties of the object from another object. Note that these properties will need to be Objects, not primitive values, or a TypeError will be thrown ↩
Thanks to Eric Elliot for pointing out that my original description of Object.Create was misleading. There are multiple ways to create and reuse object code in Javascript. I only touch on one of them in this article.
↩

Work on Ben McCormick

Coding is deciding

Vibe Coding

Tab Complete

Feature Development With Agents

Who decides what?

Retraining our Brains To Review LLM-augmented work

A Simple Path To Building Trust

Misses count more than makes

People can hold you to implicit expectations

People trust kindness

Setting Expectations: 4 key areas for performance management

3 Pillars For Effective Work

2022 Books

Decision Making

Productivity

Product Process

Soft Skills

Career

Miscellaneous

Going Forward

7 Months of Lifting At Home

Mental Models For Managing Change

Understanding the level of the change

Track Your Stability Over Time

More Thoughts On Time

Rhythms

Must-Dos

Opportunities

How I Think About Time

When Everything Falls Apart

Values Are How You Scale Culture

Strategies For Building Team Resilience

Good Meetings Are Designed Not Born

New Newsletter: Herding Lions

Thinking Fast & Slow: Lessons For Managers

Kustomer Is Hiring

Engineering Management: How To Give Feedback

More Engineering Management Posts

Engineering Management: How To Delegate

Practical Activities To Delegate Better

Why Delegate?

More Engineering Management Posts

Engineering Management: Handling Accountability

Accountability for a high performing empowered team

More Engineering Management Posts

Engineering Management: Choosing What To Work On

Stay Sane, Keep Your Integrity, and Remember That People Matter The Most

More Engineering Management Posts

Engineering Management: What do EMs do anyway?

It’s all about outcomes

More Engineering Management Posts

Driving Change Without Authority

Build the thing you want to exist

Make it easier

Make your preference a default

Identify allies and opponents

Show the future

Ask authentic questions

Work in “public”

Explain your work

Establish your expertise

Play Other Positions!

2020 Roundup

Most Read Post Published in 2020

Languages and Libraries I used a lot of in 2020

Tech I want to try in 2021

Blogs I started following in 2020

Blogs I read every post from in 2020

Software I used (nearly) every day in 2020

Best Technical Books I read in 2020

Best Technical Leadership Books I Read in 2020

Best Non-Technical Books I read in 2020

How Teams Go Faster

Simple Burnout Triage

What next?

Better than a wild guess

The Management Resources I Keep Coming Back To

Essays

Books