Lab 8: Exploring a 400-Page Bulletin

1. Bulletin and corpus

Document
Bulletin of Duke Kunshan University Undergraduate Instruction 2021–2022 (July 2021 edition, 400 pages), from DKU Admissions. Accessed 27 September 2026.
Passages — build_corpus.py
1,419 raw passages → 1,281 after cleaning → 1,253 after removing 28 near-duplicates. Mean length 83 words (median 84). One passage is a paragraph, a list, a course-requirement table or one course description.
Structure
11 Parts (chapters) and 82 sections, taken from the PDF's own bookmarks; each passage also keeps its subsection, heading and page — lab8_bulletin_passages.csv.

2. Corpus overview

Passages and mean length by Part

Loading…

Top 20 TF-IDF terms (number = passages containing it)

Loading…

Part 10 (majors and course descriptions) holds 71% of all passages; the policy and service text of Parts 1–9 holds 28%.

Semantic topics

TopicPassagesTop TF-IDF terms
Method (Part C)

Embeddings: all-MiniLM-L6-v2 (sentence-transformers, 384 dimensions, normalised). Projection: UMAP, n_neighbors=15, min_dist=0.15, cosine metric, random_state=401. Clustering: k-means on the 384-d vectors (not on the 2-D map), k=8, random_state=401. Cosine silhouette stayed between 0.09 and 0.13 for k = 6–14, so k was chosen for readable topics and to stay inside the eight-colour palette. Labels: written by hand from each cluster's top TF-IDF terms and the three passages nearest its centroid (semantic_topics.py).

3. Semantic embedding map

Circle area = passage length
Loading the map…

Nearby points have similar meaning; the axes themselves mean nothing. Click a point for its passage, and its five nearest neighbours (by cosine similarity in 384-d) are ringed and joined by dashed lines.

4. Topic × Bulletin Section matrix

Share of the row
Loading the matrix…

Click a Part to open its sections. Click a cell to show only those passages on the map; a selected passage outlines its cells here. The search box also filters the matrix counts. (lab8_topic_section_matrix.csv)

5. Findings

1. What are the major semantic topics?

Five are academic fields: Politics, Society & History (293 passages), Math, Computing & Physics (220), Arts, Media & Literature (166), China, Chinese & Languages (126) and Life & Environmental Sciences (116). Three are rules and services.

2. Which topics appear across multiple formal sections?

Mission, Programs & Support spreads the widest: 56 sections in 10 of the 11 Parts. Enrollment (30 sections) and Credits (25 sections) each reach 6 Parts, and so does China, Chinese & Languages. The field topics barely leave Part 10: Life & Environmental Sciences sits entirely inside it and Politics 99%.

3. Which sections are the most semantically diverse?

Course Descriptions (7 of 8 topics) and Majors (6). Among policy sections, Degree Requirements in Part 3 has the most: 33 passages in 5 topics, because it discusses languages, credits, interdisciplinary majors and Signature Work together. Estimated Expenses follows (4 topics). Refund and the Academic Calendar each fall into one topic.

4. Which passages from different sections are similar?

The closest pair across Parts is a CR/NC rule in Grading (Part 6, p. 53) and a study-away rule (Part 8, p. 75). Both say Duke courses may be taken Satisfactory/Unsatisfactory. The 40-credit transfer limit appears in Transfer of Work (p. 39) and again under Study Away (0.92). The Independent Studies policy (p. 48) matches the INDSTU 391 course description (p. 304, 0.84).

5. Which passages look unusual for their section?

Admission and cost passages (Parts 4–5) land in China, Chinese & Languages because they compare Chinese and international students. The model keys on the word “Chinese”, not on language teaching.

6. Does a concept occupy one region or several?

registration matches 9 passages, 7 of them in Enrollment, Leave & Withdrawal: one region. graduation (32) stays in the three policy topics. credit matches 766 passages across all 8 topics, but mainly because every course title says “(4 credits)”. Credit rules themselves sit in Credits, Grades & Transfer. academic integrity appears in only 3 passages; the concept is thin in the text.

6. Design description

Each passage was embedded with all-MiniLM-L6-v2 into a 384-dimensional vector, and UMAP (15 neighbours, cosine) projected those vectors to 2-D for drawing only. Topics come from k-means (k = 8) on the full vectors. Each was named after its top TF-IDF terms and the three passages nearest its centroid, so TF-IDF explains clusters but does not form them.

On the map, position is meaning, fill colour is topic, and circle area is passage length, so long policy blocks stand out from short course lists. Eight hues cannot all stay distinct for colour-blind readers in a scatterplot, so every topic is also labelled at its centre. The matrix answers the question the map cannot: where in the bulletin each topic lives. Rows are the 11 Parts, which open into their 82 sections. Cell shade is the topic's share of the row and the count is printed, so small sections are comparable with Part 10.

The two views share one state. Clicking a matrix cell filters the map to those passages; selecting a point outlines its Part and section cells; the search box filters both at once, so a query like “credit” shows where it sits semantically and formally. The detail panel adds the full text and five nearest neighbours, flagging neighbours from another Part, which is how cross-section matches such as the duplicated study-away rules were found. A section-fit score (similarity to the section centroid) flags passages that are unusual for where they appear.

← Back to home