Built In Node

One of the most prominent companies that use Node.js is PayPal, they were one of the first majors to give Node a chance, to take a risk on it and make a difference. In the beginning, node was just another language among many others, but it stood out, it grew through the people who used it and at PayPal it saw true proeminence.

Sneaking in

Node was a language to play with, to explore, nothing big, nothing important. A few companies started using it for different prototypes, PayPal started on the same path. People wanted to play with it so they got it through the door, slowly building different prototypes on it. The prototypes were fun, but more importantly, they were practical. Through some experimentation they found that they could copy a lot of code from their existent Java Stack to the Node Stack and back.

Reasons for the change

Node solved a problem for PayPal, it allowed them to do more, faster. Their existent model couldn’t catch up with existent trends, starting with the introduction of full stack developers. The biggest problem at PayPal was the barrier between the server and the browser. Node was a great solution from the beginning because it allowed them to write applications in JavaScript both for the browser and the server.  The prototype experience with Node should great promise, so naturally the next step was production.

Getting Node into production

Replacing Java on such a large scale was a risk, but past experiences gave them a lot of confidence – one of these experience proved that Node is faster and more efficient than Java. PayPal had a team of 5 engineers working on the PayPal wallet app using Java when Bill Scott, the senior director of user interface design, assigned a two person team to work on the same project using Node. In two months the two person team caught up with the five person team. That’s fast! Scott’s conclusion was “You can do more work in a shorter amount of time with half the people,” and he was right. The results spoke for themselves. The node app was written in 33 % fewer lines of code and with 40% less files.

This app was built in Kraken.js on express, using the templating engine Dust and the Java software used an internal framework that was mostly based on Spring.  The first production tests showed great promise and also brought peace of mind, as the node app responded to requests about 45% faster than the Java app, a very visible difference for the user.

This was the beginning that eventually led to a transition to Node of PaPal’s consumer facing apps.

The data was there, the performance was there and Node took over Java in PayPal.

Tweet

Uber is an ambitious and innovating project, one that aims to make transportation as reliable as any other basic need in life, to make it accessible to everyone, everywhere. A tough task, a challenge! Uber needed a system that will keep running no matter what, and that’s why its engineers chose Node.js.

The Node.js Uber story starts actually at Joyent, the home of Node.  Curtis Chambers, one of the first employees at Uber and Amos Barreto, its Director of Engineering, way back when Uber was just a startup, went to Joyent for some node.js help. At Joyent they met Tom Croucher who is currently Uber Site Reliability Engineer. They wanted to use Node.js, who was still in its early stages for Uber, back in 2011. They took a huge change, it was risky to built and base almost an entire business on this new JavaScript backend thing that people kept talking about. Node.js was used for a few things, nothing very important, a bit here and there, and not at such a large scale.

Uber was one of the first big business that adopted Node at such a level, right from the start. Netflix and PayPal, both, big companies took the Node road, but they we’re already established companies when they made that change to Node.js from Java. Right now they’re running a older version of Node, 0.10. There are a few reasons why they went to Node and why node works so well for their business:

1 Node.js handles asynchronous I/O requests with a non-blocking, single-threaded event loop. It is particularly well-suited to distributed systems that make a lot of network requests.

“Node.js is particularly well-suited to writing systems that have all their state in memory”  Kris Kowal, Software Engineer at Uber.

2 Node.js—and JavaScript in general—s excellent for quick iteration; programs can be inspected and errors can be addressed on the fly without requiring a restart, so developers can publish and deploy new code constantly.

“One of the things that makes Node.js uniquely suited to running in production is that you can inspect and change a program without restarting it. So very few other languages offer that capability. Not a lot of people seem to know that ability exists, but indeed you can inspect and even change your program while it’s running without restarting it.”

Kris Kowal, Software Engineer at Uber.

3 The active open source community continuously optimizes the technology; it gets better, all the time, practically on its own.

“By building on Node.js’s actively-developed, open-source system, we get the benefit of lots of people making the software better”

Kris Kowal, Software Engineer at Uber.

The amazing thing is, that even though they’re such a huge company, they invest in the community, cherishing it, at the last node Interactive they made available the Node.js systems that make Uber work basically, available for the community, which was great. You can check out Tom Crouchers presentation here.

Besides this they also created three pieces of software to keep their matching system running all the time at the massive scale required: Ringpop, TChannel, and Hyberbahn.

We’re going to talk a bit more about Ringpop and the other Uber made Node.js products.

Ringpop is an open-source Node.js library that brings application-layer sharding to many of their dispatching platform. It has 3 core features:  a membership protocol, a consistent hash ring and request forwarding. Here are a few more details about it:

• Ringpop is an AP system, that trades consistency for availability, something very important for Uber

• Ringpop is an embeddable module that’s included in each Node process.

• Node instances gossip around a membership set. Once all the nodes agree who each other are, they can make lookup and forwarding decisions independently and efficiently.

• Ringpop  is very scalable, you can add more processes and get more things done.

• The gossip protocol is based on SWIM. A few improvements have been made to improve convergence time.

• A list of members that are up are gossiped around. As more nodes are added it is scalable. The ‘S’ in SWIM is for scalable and really does work. It has scaled to thousands of nodes so far.

• SWIM combines health checks with membership changes as part of the same protocol.

• In a ringpop system there are all these Node processes containing ringpop modules. They gossip around the current membership.

• Ringpop is built on Uber’s own RPC mechanism called TChannel.

• It’s a bidirectional request/response protocol that was inspired by Twitter’s Finagle.

• Uber is getting out of the HTTP and Json business. Everything is moving to Thrift over TChannel.

• Ringpop is doing all its gossip over TChannel based persistent connections. These same persistent connections are used to fanout or forward application traffic.  TChannel  is also used to talk between services.

Source;

TChannel is a networking multiplexing and framing protocol for RPC. It uses a request/response model with out-of-order responses, where slow requests at the head of the line will not block subsequent faster requests. It also creates a high performance forwarding path for the existent requests .

Hyberbahn is a service-to service discovery and routing system. It allows Uber to adapt in real time in order to ensure user requests get where the need to go to be resolved, no matter what’s happening in the system. It’s elastic, massively distributed, and fault tolerant.

Source;

Uber means transportation that’s always available. This is their core value and as such they needed a system that scales quickly, also taking into account how fat their business is growing, doubling in size almost every six months. Uber is now doing over two million RPCs per second at peak across the Node.js fleet. They also wants to add 1000 engineers to their team this year to be able to sustain their companies growth so don’t forget to follow them.

This quote from Kris Kowal, Software Engineer at Uber sums up the Uber Node.js story very well:

“Somewhere in the world, someone needs to get where they need to go, and without the flexibility and reliability that Node.js provides that someone might be left thirsty for another transportation option.”

Tweet

Laughter is the best medicine, at least that’s what they say. I’m not sure how true it is, you might want to visit a doctor if you’re changing colours faster than a chameleon; but if you’re down in the dumps and you want to turn that frown upside down – a few giggles or a serious dose of hysterical laughter is just the right stuff. So here’s 8 jokes and something extra to loosen you up a bit.

1 Oh snap

2 But is it as fun?

3 *Checking out shoes*

4 The oldie

5 Bugs, bugs everywhere

6 You don’t tell me what to do! Or where to go!

7 Ba-Dum Tshh

8 Keep them beards growing

Bonus feature:

Tweet

With only 7 candles on the cake you’d think Node doesn’t have much history to it, but actually a lot of things can happen in 7 years. 2009  was a different world, the U.S. had its first black President, Avatar was  released and Johanna Sigurdardottir became Iceland’s first female Prime Minister. And of course, Node.js had its initial release.

Let’s go through everything year by year.

2009

• An early preview of npm was released just a few months after Node

• Ryan Dahl had the first presentation on Node.js at JSConf

2010

• The initial release of npm (in January)

• The express framework was released

• Socked.io came to life

• Ryan Dahl had the first Google talk about Node

2011

• Npm 1.0 was released

• One of the first books for Node.js beginners was released: The Node.js Beginner book by Manuel Kiessling.

• 2 major companies started using Node.js: Linkedin and Uber

2012

• This year Ryan Dahl left the Node.js project and Isaac Schlueter took on the gauntlet of moving Node forward. At this point in time, the Node.js project was still under the Joyent umbrella, and Ryan remained an employee there.

• Hapi.js was released

2013

• The initial concept of the MEAN stack was introduced by Valeri Karpov, a MongoDB developer.

• Ghost, a node.js open source publishing platform was released

• Paypal adopted Node, so did ebay (they both moved from Java)

2014

• Node leadership changes hands again as TJ Fontaine takes over

• Netflix takes the Node way

• This was the year then the io.js project forked, the year Node divided. Io.js released several version in tandem with the original Node.js team. Opinions were divided and people went in different directions.

2015

• A house divided cannot stand on itself, so in June of 2015 the Node.js Foundation was created and the io.js project reunited with the original node project. The foundation has several big name in its members, companies that support the foundation and its endeavours. It still remains and independent entity that works for the promotion and development of Node.js

• 3 months after the Node Foundation was launched TJ Fontaine stepped down as the lead of the project

• In September 2015 the Node.js and io.js combined in a single codebase with the release of Node 4.0. This was the original version 1.0 planned before the merger.

• In 2015 a plan was outlined for constant and regular Node.js version updates. You can read more about it here.

• The first Node Interactive conference took place

2016

• Npm got to more than 200k users

• The first Node Interactive Europe conference happened

• Yarn was released

• Node.js 6 is the latest LTS version

What other major events of Node do you consider worth mentioning?

Also, have a look at some the companies and startups that are currently using Node.js

Tweet

Stack Overflow recently released the data on their yearly developer survey. 64.000 developers, from 213 countries completed the survey in January. 57% of them completed the entire survey while 24% completed enough questions to being considered as well. In total they had 51.392 usable responses from: North America (13.717) , UK and Ireland (4740), France (1740), Germany, Austria, Switzerland (5221). The rest of Europe brought in 10757 responses, while Asia, Pacific and Australia brought in 11528, Africa 1249 and South America and Central America had 2098 responses. 155 chose not to identify with a region.

This is the fifth edition of the survey and also the fifth time that JavaScript conquered the title as the most commonly used programming language. This can only be good news for the Node community. SQL takes second place and Java third. Another breakthrough this year came for Python, it overtook PHP for the first time since the survey is conducted.

36,625 responses

In the Frameworks, Libraries, and Other Technologies section Node.js is on top again, followed by Angular, another JS based framework.

The survey also included a Technology Popularity Over Time section where you see the popularity of lack thereof of a given technology for the past 5 years.  Node.js went from 8% in 2013 to 26% in 2017.

JavaScript went from 57% to 62%, Python from 22% to 32% while other technologies like PHP decreased from 35% to 28%, Java from 42% to 39% and C++, from 28% to 22%.

A very interesting section of their survey is Most Loved, Dreaded, and Wanted Languages, not just statistics and facts, but also preferences.

For the second year in a row Rust won the most loved programming language title and Visual Basic (6), won the most dreaded programming language award. For the most wanted section, Python takes the crown, jumping from fourth last year, while JavaScript takes second place.

For the record this section implies that developers that are using a certain technology, love it, dread it, or want to use it.

The same principle applies to the Most Loved, Dreaded, and Wanted Frameworks, Libraries and Other Technologies section.

React wins first place and Node takes the silves on the loved leaderboard, Cordova “wins” the most Dreaded award and, * drum rolls please * Node.js takes the top of the Most Wanted section!

Here are a few more interesting facts from the survey:

• Linux Desktop was the most loved platform.

• Sharepoint was the most dreaded.

• More developers wanted to work with Android this year than any other platform.

• JavaScript is also the most popular language for web developers, Sysadmin/Devops, Data Scientist/Engineers and second favorite for desktop developers (behind SQL)

• 53.3% of developers said remote options were a top priority.

• 72.6%  of respondents identify as web developers

• 88.6% of respondents were male

• 90% say they are at least partially self-taught.

Check out the full survey on Stack Overflow.

Tweet

Starting out as a developer can be daunting, learning a new technology either as a student or a already established programmer usually takes the road of blood, sweat and tears. But learning Node.js can be a lot easier with the help of the amazing community that was built around it.

Here are a few tips from node.js developers and active community members. We hope their tips will help and if you have other useful tricks to share just think about how they helped you and how much they can help others.

A 100% growth rate means that every day new people start to learn Node.js and the great thing about this technology is the support that exists within the community. So if you have other tips for people starting out on Node share them in the comment section.

Here are some tips for Node beginners:

Start simple, do Vanilla, learn to use Node.js itself before moving to modules. Become active in the community, connect and try to contribute, to give back as much and as little as you can.

Wyatt Preul

Product Engineer, Joyent

Take some time and look at what goes on “behind the scenes”, how do the modules work, who writes them and get to know these members of the community and see what you you can learn from them and what are their opinions.

Steven Faulkner

Director of Engineering, Bustle

Use a private npm registry, don’t check in your Node.js modules to Git, be careful of using Git URLs in your package JSON, they can lead to a lot of issues in time, but at the beginning are a nice shortcut.

Charlie Robbins,

Director of Engineering, GoDaddy

Lock your dependencies. He’s really putting his foot down here – just lock them. If you’re going to upgrade Node make sure you let your team know.

If you upgrade Node on your machine and something goes wrong, just remove rm-rf/node_modules and npm install it again. This one is hand on a very practical advice.

James Hush

Software Engineer, NFL

Make sure you handle errors when they occur and make sure you handle them accurately., crash the application when it’s in an unknown state and restart it. Create tests and test them, make sure you have automated tests. Node being a dynamically typed language, he urges you to have some strict data validation.

Paul Milham

Lead Developer, WildWorks

If you’re an experienced developer think about all the stuff you wished you knew, that could have saved you a lot of headaches and frustration. There’s a lot of them, right?

Well, write a few of them up, a lot of people will thank you for them.

The tips mentioned in this articles were recorded in the following video.

Tweet

Node.js versions support and calendar

Node.js V7 vas released in Beta recently, we covered this in an article and got a fair amount of interest in it. As suc,h we decided to cover Node.js versions a bit more broadly, to include more details about how and when are they released. We’re also going to talk about the cycles of support they go through. The Node.js Foundations releases new versions of Node very often (compared to other languages), every six months there is a new major release of Node. They actually have a system of release for them. There are basically two versions a year, one in April (an even numbered version, V6, V8, V10 etc) and one in October (an odd numbered version, V5 V7, V9).

You can check out the current calendar in the above picture.

Here are a few things you should know the release system:

When a new odd numbered version is released (October) the previous version (April, even version) moves to the Active Long Term Support Plan.

The Active Long Term Support Plan (LTS) lasts 18 months and covers every major version of Node.js. The LTS plan covers:

• Bug fixes;

• Security updates;

• Non-semver-major npm updates;

• Relevant documentation updates;

• Certain performance improvements where the risk of breaking existing applications is minimal;

• Changes that introduce large amount of code churn where the risk of breaking existing applications is low and where the change in question may significantly ease the ability to backport future changes due to the reduction in diff noise. (according to the Node.js Foundation)

After the Active LTS period ends each version will have a 12 month maintenance period, after which it will not receive any type of support. The maintenance period includes:

• only critical bugs,

• critical security fixes

• documentation updates

The release schedule was created in such a way that there won’t be more than two active LTS release at the same time. Basically each new version will be the current one for 6 months. When another version is released it becomes the current one and the previous one goes into LTS. After 6 more months the cycle is repeated.

If you don’t know which version to use the Node.js Foundation recommends to:

• Stay on or upgrade to Node.js v4.2.x if you need stability and have a complex production environment, e.g. you are a medium or large enterprise.

• Upgrade to Node.js v5.x if you have the ability to upgrade versions quickly and easily without distributing your environment and want to play with the latest features as they arrive.

This month, V7 will be the current version while Node.js V6 moved to its LTS period.

Tweet

You, as a registered member of the Node Foundation can run for the Individual membership director’s seat. That’s the good news, the bad news is if you wanted to do that now you’d be a bit disappointed as the nomination process closed on January 15th. There is always next year, as elections are held annually, so let’s get ahead of ourselves and explain the process and steps you need to take if you want to sit on the Node Foundation Board.

The Board of Directors is composed of 12 people, 9 representatives of corporate members, 1 representative of the Technical Steering Committee and 2 individual membership representatives, Feross Aboukhadijeh and Ashley Williams. The former’s seat is up for election. The term length for the individual membership representative is 2 years (they changed the rules this time –  Feross Aboukhadijeh  had a one year term – so that they can have a yearly election), but elections are held each year for one of the seats. This year’s election results will be announced by January 30th.

You can nominate yourself for this position, but first you need to be a registered member of the Foundation.

What are individual members and how can you be one

Individual members are contributors to the Node project, if you are part of any working group or subproject you get free membership. If you’re not a contributor you can become an individual member by paying a yearly $100 fee (as a student you get a 75% discount).

What does a individual membership director do?

As a individual membership director you have to attend board meetings, the Board meets every month, to discuss and approve resolutions regarding the Node Foundation. Several of these meeting take place in person so you need to take that into account as well. However, if needed, the board cover cost for travel and accommodations.

Your responsibility as an Individual Membership Director involves representing the community as a whole. Just like in any other democratic process they voted for you to represent their interest in the board.

Who elects the Individual Membership Director?

The community. Every registered member can cast a vote for the candidate she/he feels will represent the community best.

The current Board of Directors consists of:

• Chairperson: Danese Cooper, distinguished member of technical staff – open source at PayPal.

• Vice-Chairperson: Scott Hammond, chief executive officer at Joyent.

• Secretary: Dan Shaw, chief technology officer at NodeSource.

• TSC Director: Rod Vagg, chief node officer at NodeSource.

• Silver-level Director: Brian McCallister, chief technology officer of platforms at Groupon.

• Silver-level Director: Justin Beckwith, product manager at Google.

• Director: Todd Moore, vice president of open technology at IBM.

• Director: Colleen Evans, principal program manager at Microsoft.

• Director: Rich Sharples, senior director of product management at Red Hat.

• Director: Imad Sousou, vice president and general manager at Intel.

• Director: Ashley Williams, individual membership director.

• Director: Feross Aboukhadijeh, individual membership director.

Source: nodejs.org

On January 30th there will be a new member of the board, replacing  Feross Aboukhadijeh. Next year, Ashley Williams’s seat will be up for election, she can run again of course, but so can you. You should also note that in order to get elected you will have to “campaign” and show that you have the drive and experience for this position. Basically you have to show that you are a good candidate. Have a look at last year’s submissions by Feross Aboukhadijeh and Ashley Williams.

Tweet

As a Node developer npm is your best friend. Or it should be your best friend, but as a beginner it can be difficult to get around it. It’s a complex system, with plenty of features and it can be intimidating to get a handle of everything. By learning a few tips and tricks, a few shortcuts, you can make your life a lot easier. That being said, let’s dive in and have a look at a few npm tips and tricks.

1 Outdated dependencies

This is a common issues for any type of registry, but there is a simple fix to it.

By running npm outdated in your project you will receive the current version, the version that you need and the latest version.

2 Clean cache

If you want to clear things out to resolve issues with a package, this option is very helpful

Run npm clean

Npm install

3 Help command

Very basic command, but one that can be of great help. Hence the name.

Run: npm help or npm -1 which will give you more information related to subcommands

4 npm version

If you’re not sure which version of npm you’re using you can find out easily with this command

Run: npm -v

If you want/need to update it you can with npm install -g npm

5 Start something new

You have something in mind for a new project? Or a new module that you want to build? Great.

Run npm init.

These are just a few basic tips that can help you get around npm move easily. There are a lot more tips and tricks, if you have any suggestions that other developers might benefit from, feel free to list them in the comment section.

Tweet

If you’re thinking about Skynet, stop, this is a whole other company. Real, growing, successful and not determined to end mankind. Skycatch was founded in 2013 and has been using Node.js since the very beginning.

What do they do?

The company specialises in drones that make surveying properties a lot easier and cheaper. They build commercial drone data solutions for various fields, including construction, energy or mining. The drones (built by Skycatch) record all the information of the site you want to survey and then give you the data in an accessible form.

Where do they use Node.js?

The short answer is everywhere.

Node.js is a main technology for them, they use it for their drones and back end development and data processing.

Their main task is to look at certain sites and create a database with all the information that the drone records on that location.

For data processing they use a lot of microservices that work together. Node.js is in their own words the “glue” that keeps everything together. The drones are quite attractive, but the main point of the company is data processing. They have Node.js on the drones (Edison board) on their API site and also almost their entire backend is made in Node.js. The general advantage of being able to move very easily from the frontend to the backend is greatly appreciated at Skycatch.  

The fact that Node.js scales easily is also being taken advantaged of, as their work process is not limited to one section. Their developers have the freedom to move throughout the whole production line as needed and because it’s all JavaScript they can do that.

This is a whole other way to use drones in a practical and commercial way, as helper tools. Going beyond their perception of toys or weapons, this role brings more lucrative results and Skycatch is at the forefront of that.

Their drones can record and process images from a site and turn them into a 3D model in a few hours. The same task, done the old fashioned way, would take a week or more, depending on the size of the site.

Tweet