Data Mining Exercise 4

To clearly see the big picture of choosing "Distance/Similarity Measures" in Data Mining tasks.

1. Which data type is Cosine Similarity suitable for, and why?

Answer: Most suitable for "Document Data" or "High-dimensional Sparse Data", such as Text Mining data.

Reason (Why)

Because Cosine only cares about "Direction" or "Main Content", ignoring the Magnitude or length of the data.

Example: Article about "Data Mining"

• Document A: Written briefly on 1 page (has the word Data 5 times, Mining 5 times)
• Document B: Written in detail on 50 pages (has the word Data 500 times, Mining 500 times)

Comparison Result:

• If using Euclidean: Will see A and B as very different (Far distance because the word count is vastly different)
• If using Cosine: Will see A and B as "very similar" (Same angle) because they talk about the same topic and have the same word proportion.

2. Why is Jaccard suitable for Market Basket Data?

Answer: Because shopping basket data is "Asymmetric Binary Attributes".

Reason (Why)

There are tens of thousands of items in a supermarket, but one customer actually buys only 5-10 items (the rest are all 0).

Jaccard Index: Cares only about what they "have in common (1-1)" and what "only one of them has (1-0, 0-1)".

Key Point: Jaccard ignores items that both "did not buy" (0-0).

Comparison with Simple Matching

If not using Jaccard (e.g. using Simple Matching): The similarity value will be unrealistically high because we and our friend "both didn't buy a Ferrari", "both didn't buy a piano"... the list of things we didn't buy in common is in the tens of thousands, making it look like we have the exact same taste, even though we might have bought completely different things.

Conclusion: Jaccard cuts out the "Sparsity", focusing only on the actual items in the basket.

3. Calculate Euclidean and Manhattan distance from point (1,2,0,5) to (-2,6,3,-1)

Given: X = (1, 2, 0, 5) and Y = (-2, 6, 3, -1)

3.1 Euclidean Distance ($L_2$ Norm)

Formula: $\sqrt{\sum (x_i - y_i)^2}$

Calculation:

3.2 Manhattan Distance ($L_1$ Norm)

Formula: $\sum |x_i - y_i|$

Calculation:

4. Analyze Document Similarity

Problem Given

Terms: [data, mining, apple, banana, algorithm]

• Doc1 = (3, 2, 0, 0, 1) ← Focuses on Tech
• Doc2 = (4, 3, 0, 0, 2) ← Focuses on Tech (similar proportion to Doc1)
• Doc3 = (0, 0, 5, 4, 0) ← Focuses on Fruits

Measure Selection

Since it is Document Term Vector data (word frequency in documents), the most suitable measure is Cosine Similarity (based on reasons in question 1).

Analysis

Pair 1: Doc1 and Doc2

• Both have values in data, mining, algorithm positions, and are 0 in fruit positions.
• The direction of the vectors clearly goes in the same direction.
• Cosine(D1, D2) ≈ 1.0 (Almost 100% because word proportions are very similar)

Pair 2: Doc1 and Doc3

• Doc1 only has values at 1, 2, 5
• Doc3 only has values at 3, 4
• There is no position where both are greater than 0 at the same time (Dot Product = 0)
• (3×0) + (2×0) + (0×5) + (0×4) + (1×0) = 0
• Cosine(D1, D3) = 0 (Perpendicular, or completely different topics)

Conclusion