Cosine similarity is a measure of similarity between two vectors of an inner product space that measures the cosine of the angle between them.Note that these bounds apply for any number of dimensions, and cosine similarity is most commonly used in high-dimensional positive spaces
Here are two very short texts to compare:
Text 1: Julie loves me more than Linda loves me
Text 2: Jane likes me more than Julie loves me
We want to know how similar these texts are, purely in terms of word counts (and ignoring word order). We begin by making a list of the words from both texts:
me Julie loves Linda than more likes Jane
Now we count the number of times each of these words appears in each text:
me 2 2
Julie 1 1
likes 0 1
loves 2 1
Jane 0 1
Linda 1 0
than 1 1
more 1 1
We are not interested in the words themselves though. We are interested only in those two vertical vectors of counts. For instance, there are two instances of ‘me’ in each text. We are going to decide how close these two texts are to each other by calculating one function of those two vectors, namely the cosine of the angle between them.
The two vectors are, again:
a: [2, 1, 0, 2, 0, 1, 1, 1]
b: [2, 1, 1, 1, 1, 0, 1, 1]
The cosine of the angle between them is about 0.822.
These vectors are 8-dimensional. A virtue of using cosine similarity is clearly that it converts a question that is beyond human ability to visualise to one that can be. In this case you can think of this as the angle of about 35 degrees which is some ‘distance’ from zero or perfect agreement.
here is my c++ code implementing it
it takes input from a file.