AI Treesa Binu Sebastian

Here we will give details of the project:

Title: The Acoustics of Feeling, Emotion Detection Through Speech Analysis Project

Objective: Speech is one of the most natural ways for humans to express themselves. It is completely obvious that feelings play a significant role in people’s decisions. My aim here is to analyze the acoustic characteristics of recorded audio data to identify underlying emotions in recorded speech. A technology like this might be useful in a variety of contexts, including interactive voice-based assistants, customer service, education, forensics and medical analysis.

Solution: The four key phases of the SER system are as follows:
– The first step is to gather a library of voice samples.
– The data would then be transformed into features. – The next stage is to figure out which characteristics were most important in distinguishing each emotion.
– These characteristics are then sent into a machine learning classifier, which recognizes them.

Data Sample: The data for this research was gathered from four different, publicly available data sources, as stated below.
– The Ryers on Audio-Visual Database of Emotional Speech and Song (RAVDESS): There are a total of 7356 files, recorded by 24 professional actors (12 female and 12 male) in a neutral north American dialect, portraying emotions such as calm, happy, sad, angry, fearful, surprise, and disgust.
– Toronto Emotional Speech Set (TESS): Two actresses, aged 26 and 64, recorded 2800 stimuli in total, indicating seven emotions: anger, disgust, fear, happiness, pleasant surprise, sad, and neutral.
– Surrey Audio-Visual Expressed Emotion (SAVEE): There are 480 total utterances, recorded by four British male actors aged 27 to 31, expressing six emotions: anger, disgust, fear, happiness, sadness and surprise. – Crowd-sourced Emotional Multimodal Actors Dataset (CREMA-D): Contains 7,442 clips featuring 91 actors and actresses of various ages and ethnicities exhibiting six emotions: happy, sad , angry, fear, disgust, and neutral.

Model: I experimented with different feature selection algorithms and observed that correlation-based feature selection gave the best results. They were all implemented in Python and made use of specialized libraries that provide optimization techniques and evaluation metrics. The results of a grid search to find the best parameters are listed below.
-Best for SVM: C=0.001, gamma=0.001, kernel=’poly’
-Best for RandomForestClassifier: {‘max_depth’: 7 , ‘max_features’: 0.5, ‘min_samples_leaf’: 1, ‘min_samples_split’: 2, ‘n_es timators’: 40}
-Best for DecisionTreeClassifier: {‘criterion’: ‘en tropy’, ‘max_depth’: 7, ‘max_features’: None, ‘min_samples_leaf’: 1, ‘min_samples_split’: 2}
-Best for MLPClassifier: {‘alpha’: 0.005, ‘batch _size’: 256, ‘hidden_layer_sizes’: (300,), ‘learning_rate’: ‘adaptive’, ‘ma x_iter’: 500}

I also explored the use of CNN’s:
– 1D CNN: My CNN network was made up of three blocks: a 1-dimensional convolution layer, an activation function (‘reLu’), a 1-dimensiona l pooling layer, and a dropout. As we are dealing with a multi-class problem, the three blocks were followed by two completely linked dense layers and a ‘SoftMax’ activation function:
– 2D-CNN: Tensorflow-Kera s was used to implement the CNN architecture. The neural network is initialized using a sequential method. Convolution2D is used to create the image-processing convolutional network. The pooling layers are added using the Max Pooling2D layer. Flatten is the function that transforms the pooled feature map into a single column that is then passed to the fully connected layer. Dense incorporates the fully connected layer into the neural network.

Conclusions: In terms of gender distribution, the number of female speakers was found to be somewhat higher than the number of male speakers, but the divergence was not significant enough to attract particular attention. After listening to a few sound files, I discovered that male and female emotions are expressed differently. Here are some of the findings:
– The loudness of Male’s Angry emotion is simply raised.
– The loudness of females’ happy, angry, and sad sounds is raised in volume.
– Female Disgust would add a throwing sound on the inside.

After the initial analysis I aimed at a more complex model-classifying different emotions based on gender data. The distribution of the samples was more balanced than the one I had before and I applied the same processing to the data as I have before and ran the exact same model. As expected, this classification seems relatively balanced, and with fairly good results as well. Th e Confusion matrix, showed that the vast majority of samples were classified correctly…..