Data Mining Exercise 0
An overview of Data Mining (DM), its techniques, applications, and challenges.
Warning
This article is a work in progress and may contain incomplete information or inaccuracies. Please verify details from reliable sources.
Data Mining Exercise 0
1. 🎯 Data Selection
This is selecting the "raw materials" we will use. Not all data in a database is always necessary.
What to do:
- Extract data from a Database, Data Warehouse, or various files.
- Select only the parts relevant to our set problem.
💡 Example
If analyzing "coffee sales", we extract sales receipt data, but do not extract "employee car registration" data.
2. 🧹 Data Preprocessing
⚡ Important
This step is the most important (takes 60-80% of the work). It is the cleaning and preparation of raw materials for cooking.
Sub-steps:
🧽 Data Cleaning
Handle Noise (distorted data) and Missing Values (incomplete data).
🔗 Data Integration
Combine data from multiple sources (e.g., Sales Excel file + SQL Customer Database) into a single block.
🔄 Data Transformation/Normalization
Adjust data format, such as:
- Transform data to be in the range
0-1(Min-Max) - Convert text into numbers
🎯 Feature Selection/Reduction
Select only important columns. Discard unnecessary things to reduce data size.
3. ⛏️ Data Mining
This is the "cooking" step, or the core where we apply Algorithms.
What to do:
Choose a mathematical model suitable for the problem.
Techniques used:
-
📊 Classification: Predicting groups
- Example: Customer will churn/stay
-
🎨 Clustering: Grouping
- Example: Grouping customers by behavior
-
🔗 Association: Finding relationships
- Example: People who buy beer often buy diapers
4. 🔍 Pattern Evaluation / Postprocessing
After cooking, we must "taste" and "plate". Not every Pattern discovered will be useful.
Steps:
🧪 Filtering
Filter out redundant rules or rules we already know (Trivial).
📈 Visualization
Transform numerical results into:
- Graphs
- Diagrams (like Decision Tree Diagram)
- To make it easy to understand
💬 Interpretation
Translate mathematical meanings into business language.
5. 🎁 Knowledge Presentation
This is the "dish ready to serve".
What we get:
✨ Actionable Knowledge
It is not just data (Data) or information (Information), but it is Knowledge that can actually be used for decision-making (Actionable Insights).
💡 Example:
"We should run a discount promotion on baking flour on Friday evenings, because housewives often buy it together with fresh milk."
📝 Brief Summary to Remember
| Step | Description | Analogy | |------
---|----------|----------| | 1️⃣ Selection | Select data | Choose ingredients | | 2️⃣ Preprocessing | Prepare and clean | Wash/chop | | 3️⃣ Mining | Discover Pattern | Cook | | 4️⃣ Evaluation | Evaluate and filter | Taste/plate | | 5️⃣ Knowledge | Present knowledge | Delicious food 🍽️ |