DSCI 512 Lecture 4 : Stacks, Queues, and Graph Searches

Sky (Kehan) Sheng

Housekeeping 🏠

  • Cheatsheet 1 submission is open.
  • Section 004 lab today is at LIFE 2402
  • Quiz 1: Next Tuesday to Friday
    • Make sure you book a slot for all 4 quizzes.
  • The 512 quiz has both multiple choice and coding question.
    • Syllabus: lectures 1 to 4 (please review all the material in the lecture notes)
    • We will NOT test about Trees in the quiz, and will not discuss Trees in the lecture this week.

Sparse matrices, continued 🦦

Image generated by OpenAI GPT-image-2.5-sunburst

Build a matrix from triples: COO

COO (COOrdinate format, also called triplet format): for every non-zero, store its row, its column and its value.

How CSR stores a matrix in a compressed way

CSR (Compressed Sparse Row). The matrix does not need to be square: this one has 4 rows and 3 columns.

Sparse matrices: CSR in code

From CSR back to the matrix

Given only data, indices and indptr, can we get the matrix back?

Rebuild it in code

The same three steps per row, as a loop:

scipy does this for you: x.toarray().

Working with sparse matrices

Many NumPy operations work. Can we add up each row without building the dense matrix?

  • Row 0 holds 5 and 1, so its total should be 6.
  • Row 1 is empty, so its total is 0.
  • Row 2 holds 2 and 3, so its total should be 5.
  • Row 3 holds 4, so its total is 4.

Tricks in Python

  • axis = 1 VS axis = 0
    • axis says which direction to add up.
    • axis=1 collapses the columns, so you get one total per row.
    • axis=0 collapses the rows, so you get one total per column.
  • Use x[i, j] to access the element in row i and column j, not x[i][j].

Rows vs columns: CSR and CSC

CSR is stored row by row. CSC (Compressed Sparse Column) is the same idea, column by column.

The code below has 2 functions, one pulls out 200 rows, and the other pulls out 200 columns. We time how long it takes when we use CSR VS CSC.

G_10k = nx.fast_gnp_random_graph(10_000, 0.0025, seed=512)   # 10,000 users
csr = nx.adjacency_matrix(G_10k)
csc = csr.tocsc()          # convert once, before any loop

def get_rows(m, k=200):
    for i in range(k):
        m[[i], :]          # row i, all columns

def get_cols(m, k=200):
    for j in range(k):
        m[:, [j]]          # all rows, column j

times = {}
for name, m in [("CSR", csr), ("CSC", csc)]:
    times[name, "rows"] = timeit.timeit(lambda: get_rows(m), number=1)
    times[name, "columns"] = timeit.timeit(lambda: get_cols(m), number=1)
    print(f"{name}: 200 rows {times[name, 'rows'] * 1e3:6.1f} ms,  200 columns {times[name, 'columns'] * 1e3:6.1f} ms")
CSR: 200 rows    8.7 ms,  200 columns   68.3 ms
CSC: 200 rows   68.6 ms,  200 columns    6.1 ms

Each format is fast in its own direction

Which one should I use?

Your situation Use
Small matrix, or more than about 10% non-zeros dense matrix
Linear algebra calculation (e.g., full factorizations or eigendecompositions) dense matrix
Large matrix, with lots of zeros sparse matrix
Building it one entry at a time COO, then convert
Mostly reading rows (most of scikit-learn; matrix-vector multiplication) CSR
Mostly reading columns (extracting all values in certain features for a machine learning model) CSC
Look up edges or if two vertices are connected adjacency matrix
Graph search: visit each node’s neighbours adjacency list, or CSR

Density thresholds: MLX sparse, Performance guide

Sparse data is everywhere

  • Natural language processing: a document-term matrix, one row per document, one column per word. Each document uses only a tiny fraction of the vocabulary.
  • One-hot encoding: scikit-learn’s OneHotEncoder returns a sparse matrix by default.
  • Recommendation systems: a user-item matrix, one row per user, one column per item. Each user interacts with only a few items.

Screenshot: DSCI 571 lecture notes, Chapter 4 · Examples: AskPython, SciPy sparse

Word counts are sparse

Example adapted from AskPython, SciPy sparse

Stacks and queues

You are writing an Instagram post with a few photos. Which data structure we should use for the “Undo” button? Which one to use for the “Upload” button? 🤔

Image generated by OpenAI GPT-image-2.5-sunburst

Stack and queue

Stack: LIFO

Last In, First Out

Undo while you edit. Every edit goes on top, and undo takes off the most recent one.

Queue: FIFO

First In, First Out

The upload queue. Each photo joins the back, and the server takes the one that has waited longest.

Both add at the back. They differ only in which end you take from.

How a stack works: last in, first out

How a queue works: first in, first out

Stack in code

The top of the stack is the end of the list.

class Stack:
    def __init__(self):
        self.data = list()

    def push(self, item):
        self.data.append(item)    # add to the top

    def pop(self):
        return self.data.pop()    # take from the top

    def isEmpty(self):
        return len(self.data) == 0

    def __str__(self):
        return str(self.data)

append and pop() both work at the end of a list: \(O(1)\), nothing else moves.

Try it: a stack

Queue in code

The only change from Stack: take from the front of the list.

class Queue:
    def __init__(self):
        self.data = list()

    def push(self, item):
        self.data.append(item)    # join the back ("enqueue")

    def pop(self):
        return self.data.pop(0)   # leave from the front ("dequeue")

    def isEmpty(self):
        return len(self.data) == 0

    def __str__(self):
        return str(self.data)

Try it: a queue

⚠️ list.pop(0) is slow

So we use deque! collections.deque (“deck”, a double-ended queue) adds and removes at both ends in \(O(1)\): append to join the back, popleft to leave the front.

n = 100_000

def empty_list_queue(n):
    q = list(range(n))
    while q:
        q.pop(0)        # O(n) each time: shift everything left

def empty_deque(n):
    q = deque(range(n))
    while q:
        q.popleft()     # O(1) each time

t_list = timeit.timeit(lambda: empty_list_queue(n), number=1)
t_deque = timeit.timeit(lambda: empty_deque(n), number=1)
print(f"list.pop(0):     {t_list * 1e3:8.1f} ms")
print(f"deque.popleft(): {t_deque * 1e3:8.1f} ms   ({t_list / t_deque:,.0f}x faster)")
list.pop(0):        483.8 ms
deque.popleft():      4.8 ms   (100x faster)

Demonstration comparing list.pop(0) and deque.popleft()

A list keeps its items in slots 0, 1, 2, … with no gaps. What happens when the front one leaves?

Emptying a queue of 100,000 items

structure insert delete search use it when
stack (LIFO) \(O(1)\) push \(O(1)\) pop \(O(n)\) most recent first: undo, back button
queue (FIFO) \(O(1)\) push \(O(1)\) with deque \(O(n)\) oldest first: upload queues

Searching in a graph 🔍

How does LinkedIn know ada is your “3rd” level connection?

You search LinkedIn for a data scientist named ada. Next to her name: 3rd.

You have never met ada. LinkedIn has about 1 billion members.

How does LinkedIn know she is 3rd, and not 2nd or 4th?

Image generated by OpenAI GPT-image-2.5-sunburst · Member count: LinkedIn, About us

Breadth-first search: search in rings

BFS in code

def bfs_connected(g, start, goal, verbose=False):
    to_visit = deque([start])   # a queue holding just the start node
    visited = {start}           # nodes we have already put in the queue

    while to_visit:                    # while the queue is not empty
        node = to_visit.popleft()      # take the OLDEST node, from the front
        if verbose:
            print("At node", node)
        if node == goal:
            return True
        for neighbour in g.neighbors(node):
            if neighbour not in visited:
                visited.add(neighbour)
                to_visit.append(neighbour)   # join the back of the queue

    return False   # the queue ran out and we never reached goal

Try it: BFS

BFS finds shortest paths

  • BFS visits every 1st before any 2nd, every 2nd before any 3rd. So the first time it reaches a node, it used a shortest path.
  • That is how LinkedIn’s “3rd” works, and how nx.shortest_path_length works on an unweighted graph.

The algorithm

  1. Put the start node in a “to visit” list.
  2. Take a node out and visit it.
  3. Add its neighbours to the list, unless we have already seen them.
  4. Repeat until the list is empty.

The only real choice is which node to take out next:

“to visit” list is a… take out… search
queue (deque, popleft) the oldest node breadth-first search (BFS)
stack (list, pop) the newest node depth-first search (DFS)

iClicker: how fast is BFS?

A graph with \(V\) nodes and \(E\) edges, stored as an adjacency list. Time complexity of BFS in the worst case?

  • (A) \(O(V)\)
  • (B) \(O(E)\)
  • (C) \(O(V + E)\)
  • (D) \(O(V^2)\)
  • Answer: C. Each node goes in and out of the queue once: \(O(V)\). Looping over neighbours adds up to every degree, \(2E\): \(O(E)\).
  • Space: \(O(V)\). The queue and the visited set each hold at most \(V\) nodes.

Question: a list as the queue?

What if BFS used a plain list, with list.pop(0) instead of deque.popleft()?

  • pop(0) is \(O(V)\), called up to \(V\) times: \(O(V^2 + E)\).

Depth-first search: go deep first

DFS in code

def dfs_connected(g, start, goal, verbose=False):
    to_visit = [start]          # a list used as a stack
    visited = set()             # nodes we have already visited

    while to_visit:
        node = to_visit.pop()   # take the NEWEST node, from the top
        if node in visited:     # pushed twice and already visited: skip it
            continue
        visited.add(node)       # mark it visited when we take it out
        if verbose:
            print("At node", node)
        if node == goal:
            return True
        for neighbour in g.neighbors(node):
            if neighbour not in visited:
                to_visit.append(neighbour)   # push onto the top of the stack

    return False
The path DFS found to ada: [('sky', 'kelp'), ('kelp', 'mochi'), ('mochi', 'pixel'), ('pixel', 'ada')]

Why mark a node when we take it out?

❌ Mark when we push it

def dfs_connected(g, start, goal):
    to_visit = [start]
    visited = {start}          # S is marked right away

    while to_visit:
        node = to_visit.pop()
        if node == goal:
            return True
        for neighbour in g.neighbors(node):
            if neighbour not in visited:
                visited.add(neighbour)       # mark when pushed
                to_visit.append(neighbour)
    return False

✅ Mark when we take it out

def dfs_connected(g, start, goal):
    to_visit = [start]
    visited = set()            # empty at the start

    while to_visit:
        node = to_visit.pop()
        if node in visited:    # an old copy: skip it
            continue
        visited.add(node)      # mark when taken out
        if node == goal:
            return True
        for neighbour in g.neighbors(node):
            if neighbour not in visited:
                to_visit.append(neighbour)
    return False

BFS vs DFS

BFS DFS
“to visit” list queue (deque) stack (list)
order rings: closest nodes first one path as deep as it can, then back up
time \(O(V + E)\) \(O(V + E)\)
space (worst case) \(O(V)\) \(O(V + E)\)
shortest path (fewest edges)? yes no
good for shortest paths, degrees of separation random maze generation, file systems, finding cycles
networkx nx.bfs_edges, nx.shortest_path nx.dfs_edges
  • BFS holds a whole ring. A wide graph, like a social network, makes rings huge fast.
  • DFS holds the path it is on, plus the neighbours waiting along it. A dense graph, with many edges, pushes many copies.
  • Both can be used to check if two nodes are connected.

Question: connected components

How would you use BFS (or DFS) to find every connected component, like nx.connected_components?

  • Search from any node. Everything it reaches is one component.
  • Pick a node you have not visited yet, and search again. Repeat until every node is visited.
  • Each node and edge is still handled once: \(O(V + E)\).

Today’s takeaways

  • A sparse matrix stores only its non-zeros. Build it in COO, then convert once.
  • CSR keeps rows together (data, indices, indptr), so reading rows is fast. CSC does the same for columns.
  • A stack takes from the top (LIFO), a queue from the front (FIFO). Use deque for queues.
  • A graph search keeps a “to visit” list and a visited set, so it never loops around in a cycle.
  • BFS uses a queue and searches in rings: the first time it reaches a node, it used a shortest path.
  • DFS uses a stack and goes as deep as it can first. Both take \(O(V + E)\) time.

Questions?

(Bonus Slide) Question: trace it

Start at 0. Neighbours come out smallest first: node 0’s are [1, 2]. In what order do BFS and DFS visit the nodes?

  • BFS: 0, 1, 2, 3, 4, 5. One ring at a time.
  • DFS: 0, 2, 4, 5, 3, 1. It pushes 1 then 2, so 2 is on top. It dives 2 → 4 → 5 → 3 and only comes back for 1 at the end.

(Bonus Slide) iClicker: BFS or DFS?

You want the fewest number of flights from Vancouver to Taipei.

  • (A) BFS
  • (B) DFS
  • Answer: A. BFS checks every airport one flight away, then two flights away, and so on. The first time it reaches Taipei, it used the fewest flights.
  • DFS might fly Vancouver → Toronto → London → Dubai → … even when there is a direct flight.

(Bonus Slide) Question: the bottom of a stack

A stack holds \(n\) items. What is the time complexity of getting the bottom item, the first one that went in?

  • (A) \(O(1)\)
  • (B) \(O(n)\)
  • (C) \(O(\log n)\)
  • (D) \(O(n^2)\)
  • Answer: B. A stack only lets you take from the top, so you pop all \(n\) items to reach the bottom.

(Bonus Slide) Question: what is in the queue?

A queue holds [A, B, C, D], with A at the front. After these steps:

  1. push E
  2. push F
  3. pop one item
  4. push A
  • (A) [A, B, C, E, F]
  • (B) [A, F, E, B, C, D]
  • (C) [B, C, D, E, F, A]
  • (D) [B, C, D, E, F]
  • Answer: C. E and F join the back, pop removes A from the front, then A joins the back again.

(Bonus Slide) CSC: the same idea, column by column

(Bonus Slide) How CSC stores a matrix

CSC (Compressed Sparse Column): the same matrix, stored column by column.

(Bonus Slide) CSC in code

(Bonus Slide) From CSC back to the matrix

Given only data, indices and indptr from CSC, can we get the matrix back?

(Bonus Slide) Rebuild it in code: CSC

The same three steps per column, as a loop:

scipy does this for you too: y.toarray().

(Bonus Slide) PageRank

(Bonus Slide) Which page is most important?

BFS and DFS tell us what is connected. PageRank, the algorithm that made Google, asks which node matters most.

Web pages are nodes in a directed graph: an edge from A to B if A links to B. Simulate a random surfer:

  1. Start on a random page.
  2. With probability \(\alpha\), get bored and jump to a random page. Otherwise click a random link.
  3. Repeat many times. A page’s rank is how often the surfer landed on it.

(Bonus Slide) The random surfer in code

Compare the two lists: a page linked from a popular page can outrank one with more links. Who links to you matters more than how many.