User Analysis and Classification Based on Social Media Advertising Data Using the Random Forest Classification Model
Introduction
In today's digital advertising world, understanding user behavior is crucial to effectively target marketing efforts and increase conversions. Machine learning models, particularly classification methods, are a key tool for predicting user behavior.
In this paper, we will focus on building a classification model using Random Forest, one of the most robust and widely used algorithms for binary classification problems. This model is based on constructing an ensemble of decision trees, each trained on a random subsample of data and a subset of features. The final decision is made by voting among all trees, significantly reducing overfitting compared to a single tree.
Why Random Forest?
1) Robustness to Noise and Overfitting
By averaging multiple trees, Random Forest demonstrates good generalization ability even on small and noisy samples.
2) Interpretability
Despite its ensemble structure, the model allows for feature importance estimation, which is particularly valuable for user behavior analysis.
3) Flexibility
The algorithm works well with numerical and categorical data and does not require feature scaling, although scaling can be useful for consistency in visualizations.
In our study, we will use a real-world dataset containing information on users' age, gender, marital status, and estimated income. The goal is to predict whether a user will make a purchase after interacting with a social media ad. This is a typical binary classification problem, ideal for Random Forest.
Reviewing source data and importing libraries
To build the model, we use the dataset, which contains information about social media users and their advertising responses. Below is an example of the first lines of this CSV file:
Where:
1) Age - User's age
2) EstimatedSalary - Estimated income
3) Gender - Male/Female
4) Marital status - (1 - married, 0 - single)
5) Purchased Target variable - (1 - purchased, 0 - not purchased)
This dataset represents a classic binary classification problem, where the goal is to determine whether a user will buy a product based on their profile.
For analysis and model building, we will use the following popular Python libraries
Where:
1) Pandas is a library for analyzing and processing tabular data. It uses to load, filter, and transform DataFrames.
2) matplotlib.pyplot is the primary tool for plotting and visualizing data.
3) seaborn is a matplotlib add-on that provides more aesthetically pleasing and informative visualizations, especially for statistical plots.
4) train_test_split (from sklearn.model_selection) is a function for splitting data into training and test sets.
5) StandardScaler (from sklearn.preprocessing) is a tool for standardizing numerical features, converting them to a normal distribution with zero mean and unit standard deviation.
6) RandomForestClassifier (from sklearn.ensemble) is an implementation of a random forest model built on an ensemble of decision trees for classification problems.
7) classification_report, accuracy_score, confusion_matrix, roc_curve, auc (from sklearn.metrics) — metrics for assessing the quality of the model: accuracy, recall, f1-score, ROC curve, and area under the curve.
Data Processing and Scaling
Before training the model, it's necessary to prepare the data: convert categorical features into numerical format, split the data into training and test sets, and standardize the numerical variables. This is essential for the algorithm to function correctly, especially for models sensitive to feature scale (e.g., logistic regression, KNN, SVM). Although Random Forest doesn't require scaling, we'll still use it for consistency and potential compatibility with other models.
Explanation:
Machine learning doesn't work directly with text labels, so categorical features (e.g., "Male"/"Female") need to be encoded with numerical values. Here, we used label encoding—a simple replacement of categories with numbers.
Explanation:
In classification problems, it is important to separate independent variables (factors, features, X) from the dependent variable (y) - in our case, Purchased, which indicates whether the user made a purchase.
Explanation:
To objectively evaluate the model's performance, we split the data into a training set (for training the model) and a test set (for validation). The test_size=0.2 parameter means that 20% of the data will be used for testing, and random_state maintains randomness for reproducibility.
Explanation:
Scaling forces all numerical features to a uniform scale (mean 0, standard deviation 1), which is especially important when there are features with widely different units of measurement (e.g., age vs. salary). This helps prevent one feature from dominating the others.
NB: fit_transform() is applied only to the training data, while transform() is applied to the test data. This prevents data leakage from the test set to the training set.
Training the Random Forest model
In this step, we use the prepared data to train a Random Forest model to classify users based on features.
Model parameters:
n_estimators=100
Number of trees in the forest. The higher the number, the more stable the result (up to a certain limit).
max_depth=4
Limits tree depth to reduce overfitting.
min_samples_split=2, min_samples_leaf=1
Minimum number of samples for splitting and per leaf, respectively. These control tree complexity.
class_weight='balanced'
Accounts for class imbalances by automatically selecting weights.
random_state=42
Fixes the random number generator for reproducibility.
Explanation:
The .fit() method initiates the process of constructing trees using the training data. Each tree is build using a random subsample of features and data (bagging), which makes the forest robust and diverse.
Explanation:
The .predict() method is applied to previously unseen data to obtain the model's final predictions. This is the result we will compare with the true labels (y_test) to assess classification accuracy.
Model evaluation and results visualization
After training a Random Forest classification model, it is important to analyze its performance, interpret key metrics, identify significant features, and visually represent the model's behavior on test data.
Explanation:
Accuracy is the proportion of correctly predicted observations out of all observations. However, for imbalanced classes, this metric can be misleading.
The Classification Report includes:
Precision: the proportion of correct positive predictions.
Recall: the proportion of positive cases found out of all positive ones.
F1-score: the harmonic mean of precision and recall, especially useful for imbalanced classes.
Explanation:
Random Forest provides a built-in feature importance assessment—the ability to evaluate how much each feature influences the model's final decision. This allows you to identify key variables for the problem and, if necessary, optimize feature selection.
Explanation:
A confusion matrix is a table describing the performance of a classical model. It displays the number of:
True Positive (TP),
True Negative (TN),
False Positive (FP),
False Negative (FN) predictions.
It can be used to calculate precision, recall, F1-score, and other metrics.
Explanation:
Visually comparing the distribution of the Age and EstimatedSalary features allows us to determine whether classes differ in these features. This helps us understand which variables are discriminative (separating classes) and whether they should be used to train the model.
Explanation:
The ROC curve demonstrates how the model balances true positives and false positives at different thresholds.
The area under the curve (AUC) is an indicator of classification quality:
AUC = 0.5: random guessing
AUC = 1.0: perfect classifier
Interpretation of results
Precision = 0.9125
The model correctly predicted 91.25% of all cases. This is a high overall performance, especially when the classes are balanced. However, this alone is insufficient for an objective assessment, especially when working with imbalanced data.
Class 0 (Didn't buy the product):
Precision (Accuracy) = 0.96 — of all the cases the model predicted as class 0, 96% actually did not buy.
Recall (Recall) = 0.90 — the model identified 90% of all true representatives of class 0.
F1-score = 0.93 — an excellent balance between precision and recall.
Class 1 (Bought the product):
Precision = 0.84 — of all the cases the model predicted as having bought the product, 84% actually did.
Recall = 0.93 — the model is very good at detecting buyers (93%).
F1-score = 0.88 — slightly lower than in class 0, but still very respectable.
Average values:
Macro avg — simple average across classes: 0.90–0.92 for precision/recall/F1. This indicates balanced performance of the models in the upper classes.
Weighted average — takes into account the size of each class: all values ~0.91 — the model is not biased toward the majority.
Feature Importance
The graph shows the relative importance of each feature in user classification using the Random Forest model. It helps identify which variables contribute most to the prediction.
Conclusion:
Age and Estimated Salary turned out to be the most significant factors in predicting purchase decisions. Their combined importance accounts for over 95%, indicating a strong dependence of the decision on financial status and age category.
Gender and Marital Status showed extremely low significance, having virtually no impact on the final classification. This means that gender and marital status likely do not play a key role in purchasing decisions in this dataset.
Correlation matrix
Interpretation of results:
Feature Correlation with target variable Purchased
Age 0.62 → Strong positive relationship
EstimatedSalary 0.36 → Moderate positive relationship
Gender 0.06 → Almost no relationship
Marital status -0.05 → Almost no relationship, weak negative
Conclusions:
Age is the most important predictor—with increasing age, the likelihood of purchasing increases.
Salary also has an impact, but to a lesser extent.
Gender and marital status have little impact on the purchase decision—their influence can be considered statistically insignificant.
This is confirmed by the Feature Importance plot, where Age and EstimatedSalary are highlighted as the most significant.
Confusion Matrix
The model correctly classified 73 out of 80 examples.
Errors were concentrated primarily in False Positives (5 cases)—when the model predicted a purchase that never occurred.
There were fewer False Negatives (2 cases), indicating that the model is reasonably good at identifying buyers, which is important for identifying potential customers, for example.
This indicates balanced model performance and a good recall for the positive class (1), which is especially valuable when predicting rare but significant events (e.g., purchases).
Distribution of features into classes
Age
The left graph shows that:
Users under 30 generally do not make a purchase (marked in red).
Peak purchasing activity occurs between ages 40 and 60, where most users purchased the product (blue bars).
This indicates a nonlinear relationship between age and target action. Models such as Random Forest are good at handling such relationships thanks to decision trees that partition the data by threshold values.
Estimated Salary
Right graph:
At low salaries (up to 60,000), most users do not purchase the product.
Starting at ~90,000–100,000 and above, the proportion of buyers increases significantly (orange bars).
The highest proportion of purchases observed among users with an income of 120,000–150,000.
The distribution shows that salary is a significant predictor of purchase. The higher the income, the higher the likelihood of a target action. However, as with age, this relationship may not be linear, which also justifies the choice of the Random Forest model, which automatically accounts for such complex relationships.
Conclusion:
Both age and salary influence the likelihood of purchase, with salary having a more linear effect, while age has more of a threshold effect.
This confirms that the classification model is capable of making accurate predictions, as the features are truly informative.
Receiver Operating Characteristic Curve
On the graph:
Orange line – model
Dashed diagonal – random selection
The model curve is significantly above the diagonal, and the AUC is 0.97, indicating a near-perfect ability of the model to distinguish between classes.
Conclusion: A high AUC indicates that the model has an excellent balance between sensitivity (recall) and specificity (1 is the false positive rate).
Overall visualization conclusion
Visual analysis confirms the excellent model training:
Important features are identified and logical.
There are few classification errors.
The ROC curve demonstrates the model's high generalization ability.
Full Code
Final Conclusion
During the analysis and construction of a machine learning model for predicting user responses to social media advertising, a classification model was developed that demonstrated a high level of accuracy. The algorithm used demonstrated the following:
Model Accuracy: 91.25%
AUC ROC: 0.97 – demonstrating excellent discrimination between classes.
Low Error Rate: only 7 errors out of 80 predictions (FN + FP).
Feature importance analysis revealed that age and estimated income level are the main determinants of user behavior. Correlation analysis confirmed that the features are independent and suitable for joint use in the model.
Recommendations for applying the model in a real-world situation
1. Advertising campaign targeting
Use the model to predict a user's likelihood of response based on their age, salary, gender, and marital status. This will allow you to:
Reduce advertising costs through more precise targeting.
Increase CTR and conversion.
Create personalized offers.
2. Marketing budget optimization
Segment users by conversion likelihood and allocate budget to the most promising groups. This will ensure:
Increased ROI.
Efficient CPC (cost per click) management.
3. Integrate the model into A/B tests
Use model predictions as an additional metric when running A/B tests:
The group with the highest model speed should show the best results.
This will help verify the effectiveness of new creatives or approaches.
4. Regular retraining
The model must be retrained regularly (e.g., quarterly) to adapt to changes in user behavior and external factors (changing trends, prices, seasonality, etc.).
5. Ethics and privacy
When implementing the model in production, be sure to comply with personal data protection requirements (GDPR, Data Protection Act). Ensure that user data is processed anonymously and securely.
A wish to the readers
Data science isn't just about algorithms and metrics; it's also the art of seeing real stories behind the numbers, making more accurate decisions, and making the world a little more understandable. Let this model become more than just a tool, but a starting point for new experiments, ideas, and achievements. Keep exploring, ask bold questions, and don't be afraid to teach machines—after all, behind every line of code is a human being capable of changing the future.
Good luck with your projects, and until you discover new things!
