Node.js Design Patterns
Notes
Chapter 2: The Module System
Loading Phases
- Construction (or parsing): The interpreter identifies all imports and recursively loads the content of each module from their respective files.
- Instantiation: For each exported entity in every module, the interpreter creates a named reference in memory, but it does not assign it a value yet. References are created for all the
importandexportstatements to track the dependency relationships between them (liking). No JavaScript code is executed during this phase. - Evaluation: The Node.js interpreter executes the code so that all the previously instantiated entities can get an actual value. Now, running the code starting from the entry point is possible because all the blanks have been filled.
Really like this short summary of how to remember the various phasesWe could say that Phase 1 is about finding all the dots, Phase 2 connects those does creating paths, and finally Phase 3 walks through the paths in the right order
Modules that modify other modules
This technique, where a module modifies other modules or objects in the global scope, is known as monkey patching. Monkey patching refers to the practice of altering existing objects at runtime to change or extend their behavior, or to apply temporary fixes.
The Role of the TypeScript Compiler
- Module Loading: Will it load a TypeScript file or a pre-compiles JavaScript file?
- Module type and module resolution: What kind of module format does the target system expected. What module type is the loaded file using?
- Output transformation: How will the module syntax be transformed during the output process?
- Compatibility: Can the detected module types interact correctly based on teh syntax transformation?
Chapter 3: Callbacks and Events
setImmediate()gives callbacks lower priority thanprocess.nextTick()or eventsetTimeout(callback, 0). Callbacks deferred withprocess.nextTick()are called microtasks and they are executed just after the current operation completes, even before any other I/O event is fired. WithsetImmediate(), on the other hand, the execution is queued in an event loop that comes after all I/O events have been processed.
- Node.js emits a special event called
uncaughtExceptions
Observer Pattern
Reminder we have the following patterns that have been introduced
- Reactor Pattern: The main idea behind this pattern is to have a handler associated with each I/O operation. A handler in Node.js is represented by a
callbackfunction. - Callback Pattern: Functions triggered to handle the result of an operation.
- Observer Pattern: Defines an object (called subject) that can notify an observer (or listeners) when a change in state occurs.
The main difference from the Callback pattern is that the subject can notify multiple observers, while a traditional CPS (Continuation-Passing Style) callback will usually propagate its result to only one listener, the callback.
In traditional OOP, the Observer pattern requires interfaces, concrete classes, and a hierarchy… In Node.js the Observer pattern is already built into the core and available through the
EventEmitterclass. TheEventEmitterclass allows us to register one or more functions as listeners, which will be invoked when a particular event type is fired.
Design your own EventEmitter class
When subscribing to observables with a long life span, it is extremely important that we unsubscribe our listeners once they are no longer needed. This allows us to release the memory used by the objects in a listener’s scope and prevent memory leaks. Unreleased
EventEmitterlisteners are the main source of memory leaks in Node.js (and JavaScript in general).
Chapter 4: Asynchronous Control Flow Patterns with Callbacks
I really like the demonstration of how to solve a race condition in spider example. Spider concurrently downloads all links that it finds on a webpage and downloads it. This can link to race condition if it finds the same link on the page twice. While the function does check if the file is already downloaded it could check that, the event loop might switch to the next function which also checks it and then both functions are now going to download the file.
The key point to solving it is right here
all we need is a variable to mutually exclude
spider()tasks running on the same URL
So you can just add the url to a Set and have additional check to that this url isn’t being processed. Removing the downloaded url from the set after the file downloaded is good practice from having it grow indefinitely. The exists() will still catch other future calls to not download the file. In my head I like to think it as a way to indicate that this url is getting being processed.
This might be one of these best chapters I have read in the JS/TS/Node.js ecosystem. I’ll admit though, this callback way of programming is kicking my ass and I am really struggling. Will need to come back to this chapter and reread it few times for to settle. It’s hard for me to articulate what exactly it is I am struggling with. I think the two main things are,
- You never return a result you always call the callback with an error or the result you want to pass in. This makes it weird because the callback based function doesn’t usually have a return value but instead it’s the callback that has the value.
- Now this might be more of a recursion problem but I struggle with properly passing data from one callback to itself or another callback. It always seems you need multiple functions to accomplish “one true” function
Chapter 5: Asynchronous Control Flow Patterns with Promises and Async/Await
A promise is an object that represents the eventual result (or error) of an async operation… a
Promiseis pending when the async operation is not yet complete, it’s fulfilled when the operation successfully completes, and it’s rejected when the operation terminates with an error. Once aPromiseis either fulfilled or rejected, it’s considered settled.
Promises are executed in the microtask queue.
As a result of the adoption of the Promises/A+ standard, many
Promiseimplementations, including the native JavaScriptPromiseAPI, will consider any object with athen()method aPromise-like object, also called thenable.
The difference with
Promise.all()is thatPromise.allSettled()will always wait for eachPromiseto either fulfill or reject, instead of immediately rejecting when one of the promises reject.
Chapter 6: Streams
Use a
PassThroughstream when you need to provide a placeholder for data that will be read or written in the future.
The example they gave was you wanted to upload a file that took in a readableStream but you want to do some processing on the file stream before upload e.g. compress or encrypt data. What I didn’t understand about this was why not create the readableStream first then pipe the stream through encrypting the stream or compressing and then call upload at the end? Like why do you have to call the upload first with this placeholder?
- Every time you call
createReadStream()from thefsmodule, this will open a file descriptor every time a new stream is created, even before you start to read from those streams.
Design your own lazy Readable and Writeable streams using the PassThrough stream as a proxy until _read() mthod is invoked for the first time.