NodeJSBuffers & Binary Data

Buffers

A Buffer is Node's view onto a chunk of raw binary data — a fixed-length sequence of bytes living outside the V8 JavaScript heap. Long before JavaScript had typed arrays, Node needed a way to handle the bytes flowing from files, sockets, and processes; Buffer is that tool. Today it's a subclass of Uint8Array, so it's both a Node original and a first-class typed array. Understanding buffers is the key to streams, networking, and any binary protocol.

Why buffers exist

JavaScript strings are sequences of characters (UTF-16 code units) — great for text, useless for a PNG, a TCP packet, or a gzip stream. Those are sequences of bytes. A Buffer represents exactly that: a contiguous block of memory you can read and write one byte at a time, with no charset assumptions until you choose one.

Buffers are allocated off the V8 heap
Buffer memory is allocated outside V8's garbage-collected heap (via Node's internal allocator). This lets Node move large binary payloads around without bloating the JS heap or stalling GC — essential when piping gigabytes through a stream. The `Buffer` *object* is tracked by V8, but its backing bytes are not.
Creating buffers

JS
// From text (default UTF-8):
const a = Buffer.from('héllo')          // bytes of the UTF-8 encoding

// From a byte array:
const b = Buffer.from([0x48, 0x69])     // <Buffer 48 69>  → 'Hi'

// Zero-filled, fixed size (safe):
const c = Buffer.alloc(8)               // <Buffer 00 00 00 00 00 00 00 00>

// Uninitialized, faster, but contains OLD memory (unsafe):
const d = Buffer.allocUnsafe(8)         // <Buffer ?? ?? ...> — must overwrite!
`allocUnsafe` can leak old memory — fill it before use
`Buffer.allocUnsafe(n)` skips zeroing for speed, so the buffer starts with **whatever bytes were already in that memory** — possibly fragments of previous data (passwords, other requests). Only use it when you'll immediately overwrite every byte. When in doubt, use `Buffer.alloc(n)`, which zero-fills. Never expose an unfilled `allocUnsafe` buffer to user-visible output.
`new Buffer()` is deprecated and dangerous
The old constructor `new Buffer(size)` behaved like `allocUnsafe` (no zeroing) while `new Buffer(string)` did something else entirely — an overloaded, unsafe API that caused real security bugs. It's deprecated. Always use the explicit factory methods: `Buffer.from`, `Buffer.alloc`, `Buffer.allocUnsafe`.
Buffers and encodings

A buffer is just bytes. Turning bytes ↔ text always requires choosing an encoding. Get it wrong and you get garbled output or corrupted data:

JS
const buf = Buffer.from('hello', 'utf8')

buf.toString('utf8')     // 'hello'
buf.toString('hex')      // '68656c6c6f'
buf.toString('base64')   // 'aGVsbG8='

// Round-trip through base64:
const encoded = Buffer.from('data').toString('base64')   // 'ZGF0YQ=='
Buffer.from(encoded, 'base64').toString('utf8')          // 'data'

Encoding

Use for

utf8

Default; text (multi-byte aware)

ascii

Legacy 7-bit text (strips high bit)

hex

Two hex chars per byte — debugging, hashes

base64 / base64url

Binary-safe transport (email, URLs, JSON)

latin1 / binary

One byte per char, 0–255 — raw byte↔char

utf16le

UTF-16, little-endian

Length is in bytes, not characters
`.length` counts bytes — multi-byte characters count more than once
Because a buffer holds bytes, `buf.length` is the **byte count**, which differs from a string's character count whenever non-ASCII characters are present. `'héllo'` is 5 characters but 6 bytes in UTF-8 (the `é` takes two). Use `Buffer.byteLength(str, encoding)` to size a buffer for text, not `str.length`.

JS
'héllo'.length                      // 5  (characters)
Buffer.from('héllo').length          // 6  (UTF-8 bytes)
Buffer.byteLength('héllo', 'utf8')   // 6  (size it correctly)
5
6
6
Reading and writing bytes

Buffers are indexable like arrays, and offer typed read/write methods for multi-byte integers with explicit endianness — the bread and butter of binary protocols:

JS
const buf = Buffer.alloc(4)

buf[0] = 0xde            // index access, one byte (0–255)
buf.writeUInt16BE(0xbeef, 1)   // write 2 bytes big-endian at offset 1
console.log(buf)         // <Buffer de be ef 00>

buf.readUInt8(0)         // 222 (0xde)
buf.readUInt16BE(1)      // 48879 (0xbeef)
Endianness matters for multi-byte values
`BE` (big-endian) stores the most-significant byte first; `LE` (little-endian) least-significant first. A 16-bit value `0xBEEF` is `be ef` in BE but `ef be` in LE. Network protocols traditionally use big-endian ("network byte order"); x86 CPUs are little-endian. Always match the format you're parsing or you'll read scrambled numbers.
Slicing shares memory — beware
`buf.subarray()` returns a VIEW, not a copy
`subarray`/`slice` return a new `Buffer` that **points at the same underlying memory**. Mutating the slice mutates the original (and vice versa). This is fast and intentional, but surprising. When you need an independent copy, use `Buffer.from(buf)` or `Uint8Array.prototype.slice`. (Note: `Buffer.prototype.slice` is deprecated in favor of `subarray` precisely to make the sharing explicit.)

JS
const original = Buffer.from([1, 2, 3, 4])
const view = original.subarray(0, 2)   // shares memory
view[0] = 99
console.log(original)                  // <Buffer 63 02 03 04> — original changed!

const copy = Buffer.from(original.subarray(0, 2))   // independent
copy[0] = 0
console.log(original[0])               // 99 — unaffected
Combining and comparing

JS
const a = Buffer.from('foo')
const b = Buffer.from('bar')

Buffer.concat([a, b])           // <Buffer 66 6f 6f 62 61 72> → 'foobar'
Buffer.concat([a, b], 4)        // limit total length to 4 bytes

a.equals(Buffer.from('foo'))    // true — byte-for-byte comparison
Buffer.compare(a, b)            // 1 / 0 / -1 — for sorting
`Buffer.concat` allocates once
Pass `Buffer.concat` an array of buffers and it allocates a single result and copies each in. If you know the total size, pass it as the second argument so Node skips an internal length scan. For incremental assembly (like collecting stream chunks), push chunks into an array and `concat` once at the end — far cheaper than concatenating on every chunk.
Buffer is a Uint8Array

Every Buffer is a Uint8Array, so it works anywhere a typed array does — TextDecoder, crypto, fetch bodies, the Web Streams API. The reverse isn't automatic, but you can wrap a typed array's memory without copying:

JS
const u8 = new Uint8Array([72, 105])
const buf = Buffer.from(u8.buffer)        // shares the ArrayBuffer (no copy)

buf instanceof Uint8Array                 // true
new TextDecoder().decode(buf)             // 'Hi'  (Web-standard decoding)
Quick reference

Need

Use

Safe zero-filled buffer

Buffer.alloc(n)

Fast buffer (will overwrite)

Buffer.allocUnsafe(n)

From text

Buffer.from(str, encoding)

Bytes → text

buf.toString(encoding)

Correct text byte size

Buffer.byteLength(str)

Join buffers

Buffer.concat([...])

Independent copy of a slice

Buffer.from(buf.subarray(...))

Next
Buffers are the chunks that flow through streams — Node's model for processing data piece by piece: [Introduction to Streams](/nodejs/streams-intro).