Stop words are words that supposedly don't carry any sementic meaning., such as hasn't, your, i, being, who, not.
However, we can tell that simply removing these words may not be the right way, since words like not may carry meaning in a string.
Example
import nltk
nltk.download('stopwords')
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenizestop_words = set(stopwords.words('english')) # choosing the language
print(len(stop_words)) # seeing the total number of stop words
print(list(stop_words)[:20]) # show the first 20 stop words198
["hasn't", 'your', 'an', "haven't", "they're", 'were', 'ain', 'being', 'who', 'not', 'should', 'having', 'i', "it's", 'wasn', "shan't", 'that', 'once', 'himself', "isn't"]
text = "The movie was not as good as the book, but I still enjoyed it."
tokens = word_tokenize(text)
filtered = [w for w in tokens if w.lower() not in stop_words]
print(filtered) # notice how the word 'NOT' carries a strong sementic meaning
# but because its in the stop_words, it becomes omitted
# we therefore need to come up with something that is able to remove those immportant words in the stop_words['movie', 'good', 'book', ',', 'still', 'enjoyed', '.']
negations = {"not", "no", "nor"} # negations that contain sementic meaning, but are included in stop_words
custom_stop_words = stop_words - negations
filtered_new = [w for w in tokens if w.lower() not in custom_stop_words]
print(filtered_new)['movie', 'not', 'good', 'book', ',', 'still', 'enjoyed', '.']
A word removed here never reaches the matrix, so it never receives a weight in Term Frequency-Inverse Document Frequency (TF- IDF).