data science and data mining

Taif University Department of Computer Science Master of Data Science Dr Huda Hakami 1 Master of Data Science Data Mining 501313-2 Assignment 1: Data Classification Implementing K-NN Algorithm Assessment information: Weighting 10% Deadline 28 Feb 2022 Submission Mode Electronic via blackboard Learning outcome assessed • Analyze methods of pre- processing, classifications, association and clustering. • Apply data mining in text, health and marketing Purpose of the assessment • This assignment assesses the understanding of the k-NN algorithm by implementing a binary k-NN for text classification. Marking criteria Marks for each question are indicated under the corresponding question. Coursework type Individual coursework 1. Objectives This assignment requires you to implement a sentiment classifier using k-nearest neighbour (kNN) algorithm using Python programming language. Note: NO credit will be given for implementing any other type of classification algorithms. More importantly, NO credit will be given also for using existing libraries instead of implementing it by yourself. However, you are allowed to use numpy, scipy or any other libraries that you might need to implement/evaluate the algorithm or to visualise the results (e.g., matplotlib). You must provide a README file describing how to run you code to re-produce the results. 2. Sentiment Classification Using k-NN In binary sentiment classification, the goal is to classify a given user review about a product as to whether the review expresses a positive or a negative sentiment about the product. We encounter such reviews in numerous online shopping sites such as Amazon or eBay. If we can automatically predict the sentiment of a review, then we can group reviews into positive and negative ones and read only a subset of all the reviews. Taif University Department of Computer Science Master of Data Science Dr Huda Hakami 2 3. Assignment Specifications Download the dataset Amazon book reviews from here1. Use the processed_acl.tar.gz folder for this task. The book reviews folder includes positive.reviews (1000 instances) and negative.reviews (1000 instances) for training, and unlabeled.reviews (4465 testing instances). Each review in each line of these data files is represented by unigram and bigram features with their counts (concatenated using underscores). Each line is represented in the following format: feature: …. feature: #label#: Use bag-of-features representation to represent each review by a vector in which its elements correspond to feature counts (i.e., frequency). Questions/Tasks (1) Write a program to load the train/test instances (positive/negative) from the train/test files. Represent each instances using the extracted features. (2) Implement a kNN classifier and measure the classification accuracy on the test instances. Classification accuracy is define as the percentage of the total number of correctly classified instances to the total number of test instances. Evaluate the performance of the classifier using other metrics such as precision, recall and F-score as shown in the figures below. (3) Vary the value of k and evaluate the performance of your kNN classifier. Plot the your results in a graph where the x-axis corresponds to the value of k and the y-axis corresponds to the classification accuracy. What trends can be observed from the graph? Briefly report your findings. (4) For measuring the similarity for computing the neighbourhood in your kNN classifier try different similarity/distance measures such as a) cosine similarity, b) Euclidean distance, and c) Manhattan distance. Compare the performance of the kNN classifier with these three measures. (you may use additional similarity/distance measures other than the (a), (b), and (c) listed above.) Briefly report your findings. (5) Using different sub-samples of positive vs. negative training instances, evaluate the robustness of the kNN classifier under unbalanced training datasets. Briefly report your findings. (6) Compared to bag-of-word features to represent the reviews, we want to try using word embeddings such that each review is represented as the average of the word embeddings of the unigram features that appear in the review. What do word embeddings mean? What are the most popular word embedding models, explain them briefly? What is the underlying hypothesis that is widely adopted to train word embeddings? (7) Download pre-trained Global Vectors GloVe (Pennington et al., 2014) 2 . Use the 50- dimensional word embeddings exist in the folder: glove.6B.zip. Each line corresponds to a representation of a single word. You need to build a function to load the word embeddings from the GloVe file. You can use Python dictionaries to store word embeddings. (8) Apply the same k-NN that you already build with the word embedding representations of the given reviews and compare the performance of the two different feature types. (9) Visualize the training reviews in a 2-dimensional space using PCA. 1 https://www.cs.jhu.edu/~mdredze/datasets/sentiment/ 2 https://nlp.stanford.edu/projects/glove/ Taif University Department of Computer Science Master of Data Science Dr Huda Hakami 3 Precision/Recall/F-score TP: True Positive TN: True Negative FP: False Positive FN: False Negative 4. Submission Instruction • Deadline for submitting the first assignment is 28 Feb 2022. • Submit a) the source code for all your programs, b) a README file (plain text) describing how to compile/run your code to produce the various results required by the assignment, and c) a PDF file providing the answers and graphs for the questions. Compress all of the above files into a single tar/zip file and specify the filename as studentid. It is extremely important that you provide all the files described above and not just the source code! • Submission is via the blackboard. Preamble: ● Install Python3. ● Install numpy and scipy libraries for linear algebra operations. ● Install scikit-learn (a machine learning library in Python). ● For figures (charts, histograms, etc) and visualisations, install matplotlib.