Improvement_freq_terms <- ame(find_freq_terms_fun(corpus_improvements)) I then pass my data frames (highlights and improvements) to the function I have just created to see if this method works: positive_freq_terms <- ame(find_freq_terms_fun(corpus_positive)) The return(ame(terms_group)) then forces R to return the results of the function. Next, we use the power of DPLYR to use arrange by the frequency descending and to add a mutated column to the data frame to calculate the proportion of that specific term over all terms. Then, a data frame is created of the terms in the function with the headings term and Frequency. The terms_grouped variable then slices the term matrix with the frequent terms, this is converted to a matrix, sum of each row are calculated i.e. These are the powerhouses of the function, as they highlight how many times a word has been used in a sentence across all the rows of text. Next, I use the findFreqTerms function to iterate from the first entry to the maximum number of rows in the matrix. The function uses as a single parameter the corpus that you need to pass in, then a variable is created to create the doc_term_mat which uses the tm TermDocumentMatrix. To clean the corpus objects I simply pass the original corpus objects back through this function to perform cleaning: corpus_positive %ĭata.frame(Term=freq_terms, Frequency =. Remove a custom vector of words to adjust for things like e.g., i.e., etc.Remove common English word (stop words).Change the underlying formatting of the text to UTF-8.Strip out whitespace between each text item, as the VectorSource has stripped out each word from each sentence in the data frame.The parameter here takes the corpus object previously created and uses the corpus passed to: Tm_map(removeWords, c("etc","ie", "eg", stopwords("english"))) Tm_map(content_transformer(function(x) iconv(x, to='UTF-8', sub='byte'))) %>% Function to create textual corpusĪs I want to replicate this for highlights and improvements – I have created a function that could be replicated with any text analysis to create what is known as a text corpus (see: ) this creates a series of documents, in our case sentences. The ws_highlights data frame uses the first column and the ws_improvements data frame uses the second. Installing and loading the required packages install_or_load_pack % If you can get over this – it is still really useful for text analysis. The driver for this package is the tm package and is still one of the main packages in R, but it assumes a non-tidy format. I think back to a post I put on the NHS-R community website, but never posted on my own site, as I have been doing some textual analysis recently and referenced this post again. # Remove the stop words such as to, and, or etc.Analysing the pre-conference workshop sentiments These are words such as to, and, or, but etc.įollowing is command set that achieves above objectives: # Change all the words to lowercaseĬorpus_clean <- tm_map(corpus, content_transformer(tolower))Ĭorpus_clean <- tm_map(corpus_clean, removeNumbers) tm_map(corpus, content_transformer(tolower)) Thus, with latest version of tm package, it is recommended to use following command for chaging to lower case. However, if you have recently installed tm package, using the tm_map command as mentioned earlier would throw error such as Error: inherits(doc, “TextDocument”) is not TRUE. In earlier versions of tm package, it was ok to use commands such as tm_map(corpus, tolower). Change the case of all words to lowercase.Following is cleaned as part of text cleaning activity:
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |