How Computers Represent Data

Everything is bits

A computer's memory and storage are made of billions of tiny switches, each either off or on. We write that as 0 or 1 — a single bit. One bit is too small to be useful on its own, so we group them. Eight bits make a byte, and a byte can hold one of 2⁸ = 256 distinct patterns.

That's the whole trick: a byte is just a pattern of eight switches. What it means — a number, a letter, a pixel's color, part of a sound — depends entirely on the agreement (the encoding) we lay over it. The bits never change; only our interpretation does.

Why hexadecimal? Long strings of 0s and 1s are painful for humans to read. Hex (base-16, digits 0–9 then a–f) is a tidy shorthand because each hex digit maps to exactly four bits. So one byte is always exactly two hex digits — 0100 0001 becomes 0x41. You'll see hex everywhere: colors (#ff0000), memory addresses, and when you peek at a binary file.

Numbers: integers & floats

Computers store two broad kinds of numbers, and they behave very differently.

Integers (whole numbers) are stored directly in binary, in a fixed number of bytes. A common size is 32 bits, which can represent about 4.3 billion distinct values. Because the size is fixed, there's a ceiling — and when a calculation goes past it, the value overflows and wraps around instead of growing, the way a car odometer rolls back to zero. Python hides this for you (its integers grow without limit), but databases, Java, C, and most file formats use fixed-size integers, so overflow is a real bug you'll meet — a column typed INT silently maxes out, and the classic fix is to use a BIGINT (64-bit).

Floating-point numbers (floats) store fractions and very large or very small values. The catch: they store them in binary, and most everyday decimal fractions cannot be written exactly in binary — just as 1/3 can't be written exactly in decimal (0.3333…), the value 0.1 can't be written exactly in binary. So the computer keeps the closest value it can, and tiny rounding errors creep in.

# The most famous gotcha in computing
print(0.1 + 0.2)
print(0.1 + 0.2 == 0.3)

# Why? Ask Python to show more decimal places:
print(f"{0.1 + 0.2:.20f}")

This is harmless when you're plotting a chart, and a disaster when you're handling money. Add a few thousand prices stored as floats and the rounding errors accumulate into real cents that don't reconcile.

# Money the wrong way — floats
total = 0.0
for _ in range(10):
    total += 0.10        # ten dimes should be exactly $1.00
print("float total:", total)          # not exactly 1.0!

# Money the right way — integer cents, or Decimal
from decimal import Decimal
total = Decimal("0")
for _ in range(10):
    total += Decimal("0.10")
print("decimal total:", total)        # exactly 1.00

Text: ASCII, Unicode, UTF-8

If a byte is just a number, how does it become the letter A? Through a character encoding — a lookup table from numbers to characters. The history matters because the leftovers still bite you today.

• ASCII (1960s) assigned the numbers 0–127 to English letters, digits, and punctuation. 65 = A, 97 = a, 32 = space. Seven bits, 128 characters. Fine for English; useless for é, ñ, 日, or 😀.
• Unicode is the modern, universal answer: one giant catalog that gives every character in every language (plus emoji) a unique number called a code point, written like U+0041 for A. Unicode is the catalog of meanings, not a byte layout.
• UTF-8 is the rule for turning those code points into actual bytes. It's brilliant because it's backward-compatible: the original ASCII characters still take exactly one byte, while other characters take two, three, or four bytes. UTF-8 is now the default of the web and the format you should use everywhere.

This is where the crucial distinction lives: a string is a sequence of characters (a human idea); bytes are what's actually stored or sent. To save or transmit a string you must encode it to bytes; to read it back you decode. Watch:

s = "café"                 # a 4-character string
b = s.encode("utf-8")      # turn it into bytes
print(b)                   # the raw bytes
print(len(s), "chars,", len(b), "bytes")

# decode back to a string
print(b.decode("utf-8"))

# one accented character on its own:
print("é".encode("utf-8"))

So what is "mojibake"? That garbled text — café showing up as café — happens when one program writes bytes using one encoding and another program reads them using a different one. The bytes are perfectly fine; the reader chose the wrong table. UTF-8 produced \xc3\xa9 for é, but a reader assuming the old Windows "Latin-1" encoding sees those two bytes as two separate characters, Ã and ©. This is the single most common text bug in data work, and now you know the cure: agree on UTF-8 end to end, and state the encoding explicitly when you open files.

Text vs binary formats

When data leaves memory and lands in a file, it's stored in one of two broad styles, and the choice is a real engineering tradeoff.

• Text formats (CSV, JSON, XML, YAML) store everything as human-readable characters. You can open them in any editor and read them. The number 1000000 is stored as the seven characters 1 0 0 0 0 0 0.
• Binary formats (Parquet, Avro, Protocol Buffers, images, .zip) store data in compact, machine-oriented byte layouts. Open one in a text editor and you get gibberish. That same million might be stored in just four bytes.

Serialization & deserialization

Inside a running program, data lives as objects in memory — a Python dict, a list of rows. But memory is private and temporary. To save data to disk or send it over a network, you must flatten those objects into a stream of bytes. That flattening is serialization; rebuilding the objects from bytes is deserialization. Every time you write a file or call an API, this is happening.

import json

row = {"id": 1, "name": "Ana", "active": True}

# serialize: object -> JSON text (a string), then to bytes
text = json.dumps(row)
print(text)                       # '{"id": 1, "name": "Ana", "active": true}'

# deserialize: text -> object again
back = json.loads(text)
print(back["name"], type(back))   # Ana 

CSV and JSON are self-describing text: the field names or the comma positions tell you the structure, but every value arrives as text that you must guess a type for. That's flexible but slow and error-prone at scale (is "01" a number or a string? is the empty cell zero or null?).

Analytics tools therefore lean on schema-based binary formats like Avro and Parquet. They carry an explicit schema — a typed description of each column — so a number is stored as bytes that are a number, no parsing or guessing. Parquet goes further and is columnar: instead of storing row after row, it stores all of one column's values together. That makes values compress beautifully (similar things sit side by side) and lets a query read only the columns it needs. We'll dig into exactly why that's so fast in Chapter 07; for now, just hold the idea: typed, compact, columnar.

What a "row" really is

Let's tie the whole chapter together with one concrete record and watch it take three different shapes — same data, different bytes. Picture a single customer: id = 1, name = "Ana", balance = 19.99.

id,name,balance
1,Ana,19.99
2,Bo,250.00

In CSV, a row is literally a line of text: characters separated by commas, ended by a newline. Every value is text — 19.99 is the five characters 1 9 . 9 9, not a number, until something parses it. Cheap to read by eye, ambiguous about types.

{"id": 1, "name": "Ana", "balance": 19.99}
{"id": 2, "name": "Bo", "balance": 250.00}

In JSON, a row is an object with named fields. It repeats the keys (id, name, balance) on every record — readable and great for nested data, but wasteful and slow for millions of flat rows. Note balance is a JSON number here, so it inherits the float imprecision from earlier — a reason to send money as a string or integer cents.

# conceptual — Parquet is binary, not text. Internally it groups by COLUMN:
id      :  [ 1, 2 ]                # packed integers, one declared type
name    :  [ "Ana", "Bo" ]        # all names together (compress well)
balance :  [ 19.99, 250.00 ]      # decimals with a known type + scale
# plus a schema header describing each column's type.

In Parquet, there isn't really a "row" sitting in one place at all — the file stores each column's values together as typed, compressed bytes, with a schema header up top. To read one whole record the engine reassembles it from the columns; to sum the balance column it reads only that column and skips the rest. Same three facts about Ana, encoded three completely different ways — and which one you choose decides how big the file is and how fast your queries run.

✓ Check yourself

• Can you explain why 0.1 + 0.2 doesn't equal 0.3 — and what you'd use instead for money?
• What's the difference between a string and bytes, and why must you encode/decode between them?
• What causes garbled "mojibake" text, and what's the fix?
• Give one reason analytics pipelines prefer Parquet over CSV.

# 1) the float surprise
print(0.1 + 0.2)            # 0.30000000000000004
# WHY: 0.1 and 0.2 have no exact binary representation, so each is
# stored as a nearby approximation; the tiny errors add up. For money,
# use integer cents or Decimal, which count in exact base-10 units.

# 2) string -> UTF-8 bytes -> back
s = "piñata"
b = s.encode("utf-8")
print(s, "->", b)          # b'pi\xc3\xb1ata'
print(len(s), "chars,", len(b), "bytes")   # 6 chars, 7 bytes
# The ñ is non-ASCII, so it takes 2 bytes; the other 5 take 1 each = 7.

print(b.decode("utf-8"))   # 'piñata' — round-trip recovers the string

# Bonus: see mojibake happen — decode UTF-8 bytes as Latin-1
print(b.decode("latin-1")) # 'piñata'  <- the classic garble

Next

Now that you know what data is in bytes, we'll look at how to organize and process it efficiently — the data structures and algorithms that decide whether an operation takes a millisecond or an hour. → Data Structures & Algorithms for Data