Quantitative Analysis of Textual Data
Overview
A fast, flexible, and comprehensive framework for quantitative text analysis in R. Provides functionality for corpus management, creating and manipulating tokens and n-grams, exploring keywords in context, forming and manipulating sparse matrices of documents by features and feature co-occurrences, analyzing keywords, computing feature similarities and distances, applying content dictionaries, applying supervised and unsupervised machine learning, visually representing text and text analyses, and more.
Bug fixes
- Fixed tests that failed under tm 0.7-20, which no longer attaches the **NLP** package, causing CRAN check errors on r-devel.
- Fixed a bug in
tokens()when the object's concatenator is an empty character. - Fixed a bug to create a redundant copy of
tokens_xptrwhendfmis constructed.
Changes and additions
- Verbose messages from
corpus,tokens,dfmandfcmmethods now summarise the object returned, in the form "Returning tokens of 5,394 documents (55,866 tokens, 1,000 types) from tokens_trim()", instead of comparing counts before and after the operation.fcm_select(),fcm_keep()andfcm_remove()now report on the fcm they return, not on an intermediate dfm. (#2016, #2518) - The summary line printed by
print()forcorpus,tokens,dfmandfcmobjects now has a common format, e.g. "tokens of 9 documents (6,855 tokens, 1,812 types)." It replaces "Corpus consisting of", "Tokens consisting of", "Document-feature matrix of:" and "Feature co-occurrence matrix of:", and adds the number of characters (corpus), tokens and types (tokens), occurrences (dfm), and co-occurrences and sparsity (fcm). Code or tests that match the old text will need updating. (#2518)
Show all changes in 4.5.1Hide the rest of the changes in 4.5.1
featfreq()anddocfreq()now work ontokensandtokens_xptrobjects, computing feature and document frequencies directly without first creating a dfm.docfreq()supports all weighting schemes for these objects, butthresholdis only supported for a dfm. (#2523)- Reduce memory usage for constructing a
dfmfrom a largetokensobject by excluding paddings from allocating memory whenremove_padding = TRUE. If the object is too large, raises a user-friendly error.
From NEWS.md in the release
Install
Health
ERRORtestsr-patched-linux-x86_64, r-release-windows-x86_64
r-patched-linux-x86_64
Running ‘spelling.R’ [0s/1s]
Running ‘testthat.R’ [101s/125s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> Sys.setenv("R_TESTS" = "")
> Sys.setenv("_R_CHECK_LENGTH_1_CONDITION_" = TRUE)
>
> library(testthat)
> library(quanteda)
Package version: 4.5.0
Unicode version: 17.0
ICU version: 78.3
Parallel computing: 3 of 32 threads used.
See https://quanteda.io for tutorials and examples.
>
> # for strong tests for Matrix deprecations
> options(Matrix.warnDeprecatedCoerce = 2)
>
> ops <- quanteda_options()
> quanteda_options(reset = TRUE)
> test_check("quanteda")
Saving _problems/test-corpus-203.R
Saving _problems/test-corpus-213.R
[ FAIL 2 | WARN 0 | SKIP 28 | PASS 3290 ]
══ Skipped tests (28) ══════════════════════════════════════════════════════════
• Behaviour changed - consider removing test (1): 'test-tokens.R:117:5'
• Different behaviour for convert() v. old as.data.frame.dfm() (1):
'test-as.dfm.R:134:5'
...[truncated]...
t2vec} is not installed (1): 'test-fcm.R:2:5'
• {tibble} is not installed (1): 'test-corpus.R:268:5'
• {tidytext} is not installed (1): 'test-as.dictionary.R:63:5'
• {topicmodels} is not installed (1): 'test-convert.R:174:5'
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test-corpus.R:203:5'): test corpus constructor works for tm objects ──
Error in `detach("package:NLP", unload = FALSE, force = TRUE)`: invalid 'name' argument
Backtrace:
▆
1. └─base::detach("package:NLP", unload = FALSE, force = TRUE) at test-corpus.R:203:5
── Error ('test-corpus.R:213:5'): test corpus constructor works for VCorpus with one document (#445) ──
Error in `detach("package:NLP", unload = FALSE, force = TRUE)`: invalid 'name' argument
Backtrace:
▆
1. └─base::detach("package:NLP", unload = FALSE, force = TRUE) at test-corpus.R:213:5
[ FAIL 2 | WARN 0 | SKIP 28 | PASS 3290 ]
Error:
! Test failures.
Execution haltedr-release-windows-x86_64
Running 'spelling.R' [0s]
Running 'testthat.R' [103s]
Running the tests in 'tests/testthat.R' failed.
Complete output:
> Sys.setenv("R_TESTS" = "")
> Sys.setenv("_R_CHECK_LENGTH_1_CONDITION_" = TRUE)
>
> library(testthat)
> library(quanteda)
Package version: 4.5.0
Unicode version: 15.1
ICU version: 74.1
Parallel computing: 2 of 48 threads used.
See https://quanteda.io for tutorials and examples.
>
> # for strong tests for Matrix deprecations
> options(Matrix.warnDeprecatedCoerce = 2)
>
> ops <- quanteda_options()
> quanteda_options(reset = TRUE)
> test_check("quanteda")
Saving _problems/test-corpus-203.R
Saving _problems/test-corpus-213.R
[ FAIL 2 | WARN 0 | SKIP 30 | PASS 3282 ]
══ Skipped tests (30) ══════════════════════════════════════════════════════════
• Behaviour changed - consider removing test (1): 'test-tokens.R:117:5'
• Different behaviour for convert() v. old as.data.frame.dfm() (1):
'test-as.dfm.R:134:5'
• On CRA
...[truncated]...
t2vec} is not installed (1): 'test-fcm.R:2:5'
• {tibble} is not installed (1): 'test-corpus.R:268:5'
• {tidytext} is not installed (1): 'test-as.dictionary.R:63:5'
• {topicmodels} is not installed (1): 'test-convert.R:174:5'
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test-corpus.R:203:5'): test corpus constructor works for tm objects ──
Error in `detach("package:NLP", unload = FALSE, force = TRUE)`: invalid 'name' argument
Backtrace:
▆
1. └─base::detach("package:NLP", unload = FALSE, force = TRUE) at test-corpus.R:203:5
── Error ('test-corpus.R:213:5'): test corpus constructor works for VCorpus with one document (#445) ──
Error in `detach("package:NLP", unload = FALSE, force = TRUE)`: invalid 'name' argument
Backtrace:
▆
1. └─base::detach("package:NLP", unload = FALSE, force = TRUE) at test-corpus.R:213:5
[ FAIL 2 | WARN 0 | SKIP 30 | PASS 3282 ]
Error:
! Test failures.
Execution halted- ERROR11 OK, 2 ERROR2026-10-01
- OK13 OK2026-08-12
- NOTE12 OK, 1 NOTE2026-07-11
- OK12 OK2026-04-25
- NOTE12 OK, 2 NOTE2026-04-22
Show 2 earlier changesHide 2 earlier changes
- ERROR11 OK, 2 NOTE, 1 ERROR2026-04-18
- NOTE12 OK, 2 NOTE2026-03-10
Documentation
- Help pages
- 163 (81 internal)includes a package overview page
- Examples that run
- 94%102 run in a plain check · 6 never run
- Exported functions with examples
- 87%
- Documented parameters
- 97%
- Return-value docs
- 75%
- Help pages citing references
- 6%
- Quick Start Guide HTML
Downloads
Repository
- PRs merged (all time)
- 92%
- Median days to close, 50 most recently updated closed issues
- 107
- Median days to close, 50 most recently updated closed PRs
- 4
Stars over time
Forks over time
Releases over time
Issues over time
PRs over time
Commits over time
11 development-tooling and community-health practices detected across 7 families in the upstream repository
Checks run against github.com/quanteda/quanteda on 2026-10-22.
The R CMD check workflow runs on Linux, macOS and Windows, and on R-devel
GitHub Pages site last deployed 2026-10-21
pkgdown site last built 2026-07-29, before release 4.5.1 (2026-10-21)
Show all practicesHide 3 practices
This repository shows signs of AI coding tools.
- Claude Codefirst seen 2026-01-18Seen in the repository: CLAUDE.mdNamed in .Rbuildignore: .claude, CLAUDE.mdNamed in .gitignore: .claudeCommits: at least 1 crediting it in the message
- Claude Opus 5.51 commit · Oct 2026
Dependencies
Installing quanteda adds 13 packages (10 with compiled code, 17 MB of source tarballs). Heaviest: stringi 11 MB, Rcpp 1.8 MB, jsonlite 1.0 MB.
Show the 48 most downloaded packagesHide the 48 most downloaded packages
Code & Tests
Datasets
People & History
Show 8 of 9 peopleHide 8 people
Pick two releases to compare their code metrics.
- 4.5.1Latest2026-10-21 · current release · diff ↗ ·
listed once in CRAN's queue
Seen in one reading of CRAN's incoming queue, at 2026-10-21 10:55 UTC, so it stayed less than the time between readings. - 4.5.02026-08-04 · diff ↗ ·
removed 1 export
dictionary_tokenizeadded 3 exports
tokenize_dictionary, tokens_match, tokens_recompile - RR 4.6.0 released · 2026-04-24
- 4.42026-04-06 · diff ↗ ·
added 5 exports
as.tensor, dictionary_tokenize, normalize_characters, tokenize_paragraph, tokens_annotate - 4.3.12025-07-10 · diff ↗
- 4.3.0
- RR 4.5.0 released · 2025-04-11
- 4.2.02025-01-08 · diff ↗
- 4.1.0
- 4.0.22024-04-24 · diff ↗
- RR 4.4.0 released · 2024-04-24
- 4.0.1
- 4.0.02024-04-04 · diff ↗ ·
removed 1 export
index_typesadded 5 exports
as.tokens_xptr, concat, concatenator, is.tokens_xptr, tokens_restore - 3.3.12023-05-18 · diff ↗
- RR 4.3.0 released · 2023-04-21
- 3.3.02023-04-07 · diff ↗ ·
removed 1 export
tokenize_wordadded 8 exports
breakrules_get, breakrules_reset, breakrules_set, segid, tokenize_custom, tokenize_word2, tokenize_word3, tokenize_word4
Show 63 earlier eventsHide 63 earlier events
- 3.2.42022-12-08 · diff ↗
- 3.2.32022-08-29 · diff ↗
- 3.2.22022-08-09 · diff ↗
- RR 4.2.0 released · 2022-04-22
- 3.2.12022-03-01 · diff ↗
- 3.2.02021-11-30 · diff ↗
- 3.1.0
- RR 4.1.0 released · 2021-05-18
- 3.0.02021-04-06 · diff ↗ ·
removed 33 exports
metacorpus<-, metadoc<-, View, as.DocumentTermMatrix, as.igraph, as.network, as.wfm, char_edit, char_tortl, char_trimsentences, corpus_trimsentences, dictionary_edit, list_edit, metacorpus, metadoc, nscrabble, nsyllable, textplot_keyness, textplot_network, textplot_wordcloud, textplot_xray, textstat_collocations, textstat_dist, textstat_entropy, textstat_frequency, textstat_keyness, textstat_lexdiv, textstat_proxy, textstat_readability, textstat_select and 3 moreadded 10 exports
check_character, check_double, check_integer, check_logical, corpus_group, index, is.index, object2fixed, object2id, tokens_group - 2.1.22020-09-23 · diff ↗
- 2.1.12020-07-27 · diff ↗
- 2.1.02020-07-05 · diff ↗ ·
added 7 exports
char_edit, char_keep, char_remove, char_select, dictionary_edit, list_edit, textstat_summary - RR 4.0.0 released · 2020-04-24
- 2.0.12020-03-18 · diff ↗
- 2.0.02020-02-26 · diff ↗ ·
removed 25 exports
settings<-, affinity, as.coefficients_textmodel, as.statistics_textmodel, as.summary.textmodel, coefficients.textmodel_ca, coefficients.textmodel_lsa, coefficients.textmodel_wordfish, is.corpuszip, nfeature, scrabble, settings, spacy_parse, textmodel_affinity, textmodel_ca, textmodel_lsa, textmodel_nb, textmodel_wordfish, textmodel_wordscores, textplot_influence, textplot_scale1d, textstat_dist_old, textstat_simil_old, tf, tfidfadded 10 exports
meta<-, as.fcm, docid, meta, tokenize_character, tokenize_fasterword, tokenize_fastestword, tokenize_sentence, tokenize_word, tokenize_word1 - 1.5.2
- 1.5.12019-07-30 · diff ↗
- 1.5.02019-07-04 · diff ↗
- RR 3.6.0 released · 2019-04-26
- 1.4.32019-04-01 · diff ↗
- 1.4.12019-02-26 · diff ↗
- 1.4.02019-01-30 · diff ↗ ·
added 5 exports
dfm_match, flatten_dictionary, textstat_entropy, tokens_chunk, tokens_split - 1.3.142018-11-19 · diff ↗
- 1.3.132018-11-01 · diff ↗ ·
added 4 exports
textstat_dist_old, textstat_proxy, textstat_simil_old, tokens_sample - 1.3.4
- 1.3.0
- RR 3.5.0 released · 2018-04-23
- 1.2.0
- 1.1.12018-03-07 · diff ↗
- 1.1.02018-03-05 · diff ↗ ·
added 3 exports
as.coefficients_textmodel, as.statistics_textmodel, as.summary.textmodel - 1.0.02018-01-28 · diff ↗ ·
removed 7 exports
collocations, dfm2ldaformat, quantedaformat2dtm, sequences, syllables, textmodel, textmodel_wordshoaladded 14 exports
affinity, as.network, coefficients.textmodel_ca, coefficients.textmodel_lsa, coefficients.textmodel_wordfish, dfm_replace, dfm_tfidf, nfeat, textmodel_affinity, textmodel_lsa, textplot_influence, textplot_network, textstat_select, tokens_subset - 0.99.222017-11-13 · diff ↗ ·
removed 25 exports
applyDictionary, as.tokenizedTexts, changeunits, compress, features, is.tokenizedTexts, joinTokens, ngrams, removeFeatures, segment, selectFeatures, selectFeaturesOLD, similarity, skipgrams, smoother, textfile, textmodel_NB, toLower, toUpper, tokenize, tokens_hash, trim, trimdfm, weight, wordstemadded 7 exports
dfm_keep, fcm_keep, spacy_parse, textmodel_nb, tokens_keep, tokens_replace, types - 0.99.122017-10-06 · diff ↗
- 0.99.9
- 0.992017-08-16 · diff ↗ ·
removed 4 exports
collocations2, is.sequences, ngrams.default, ngrams.tokenizedTextsadded 6 exports
as.dictionary, dfm_subset, is.phrase, phrase, textplot_keyness, textstat_frequency - 0.9.9-652017-05-26 · diff ↗ ·
removed 1 export
phrasetotokenadded 3 exports
char_trim, corpus_trim, dfm_group - RR 3.4.0 released · 2017-04-21
- 0.9.9-502017-04-20 · diff ↗ ·
removed 3 exports
as.kwic, kwic_old, sampleadded 8 exports
as.yaml, bootstrap_dfm, char_trimsentences, collocations2, corpus_trimsentences, is.sequences, quanteda_options, textstat_collocations - 0.9.9-242017-02-14 · diff ↗
- 0.9.9-222017-02-09 · diff ↗
- 0.9.9-17
- 0.9.9-32017-01-10 · diff ↗ ·
removed 5 exports
describeTexts, encoding, findSequences, lexdiv, readabilityadded 66 exports
View, as.corpus, as.kwic, as.tokens, char_ngrams, char_segment, char_tolower, char_toupper, char_wordstem, corpus_reshape, corpus_sample, corpus_segment, corpus_subset, corpuszip, dfm_compress, dfm_lookup, dfm_remove, dfm_sample, dfm_select, dfm_smooth, dfm_sort, dfm_tolower, dfm_toupper, dfm_trim, dfm_weight, dfm_wordstem, fcm, fcm_compress, fcm_remove, fcm_select and 36 more - 0.9.8.52016-10-31 · diff ↗
- 0.9.8.32016-10-16 · diff ↗
- 0.9.82016-07-28 · diff ↗
- 0.9.7-172016-06-21 · diff ↗
- 0.9.7-16
- 0.9.6-9
- 0.9.6-12016-05-09 · diff ↗ ·
removed 4 exports
cbind.dfm, df, phrasetotoken.corpus, weightingadded 2 exports
texts<-, compress - RR 3.3.0 released · 2016-05-03
- 0.9.4
- 0.9.2-0
- 0.9.0-1
- 0.8.6-0
- 0.8.4-2
- 0.8.2-1
- 0.8.2-02015-07-31 · diff ↗ ·
removed 2 exports
clean, cleanCadded 3 exports
applyDictionary, scrabble, selectFeatures - 0.8.0-42015-07-13 · diff ↗
- 0.8.0-32015-07-11 · diff ↗ ·
removed 6 exports
encoding<-, language<-, texts<-, language, tokenizeOnlyCppKB, tokenizeOnlyScanadded 4 exports
nsentence, ntype, readability, toLower - 0.7.2-12015-06-02 · diff ↗
- RR 3.2.0 released · 2015-04-16
- 0.7.22015-04-07
- RR 3.1.0 released · 2014-04-10
Package metadata
- First published
- 2015-04-07
- Total releases
- 66 / 11 yrs
- License
- GPL-3 OSI
- Minimum R
- ≥ 4.1.0
- Bundled data
- 248 KB / 5 files
- Download size
- 4.2 MB
Cite
Cite this package
Run in R for the authors' preferred citation:
citation("quanteda")The package provides a citation file that builds its entry when R runs it. Run citation() to see it.
Cite the R Observatory
For a number measured here: a download total, a coverage figure, an archival date.
From data release v2026-10-25, which the citation names so these numbers can be found later. More on citing and the projects behind them.