Tokenization, Parts of Speech Tagging, Lemmatization and Dependency Parsing with the 'UDPipe' 'NLP' Toolkit
Overview
This natural language processing toolkit provides language-agnostic 'tokenization', 'parts of speech tagging', 'lemmatization' and 'dependency parsing' of raw text. Next to text parsing, the package also allows you to train annotation models based on data of 'treebanks' in 'CoNLL-U' format as provided at https://universaldependencies.org/format.html. The techniques are explained in detail in the paper: 'Tokenizing, POS Tagging, Lemmatizing and Parsing UD 2.0 with UDPipe', available at doi:10.18653/v1/K17-3009. The toolkit also contains functionalities for commonly used data manipulations on texts which are enriched with the output of the parser. Namely functionalities and algorithms for collocations, token co-occurrence, document term matrix handling, term frequency inverse document frequency calculations, information retrieval metrics (Okapi BM25), handling of multi-word expressions, keyword detection (Rapid Automatic Keyword Extraction, noun phrase extraction, syntactical patterns) sentiment scoring and semantic similarity analysis.
- Fix on the declaration of persistent_unordered_map for C++20
From NEWS.md in the release
Install
Health
- OK13 OK2026-08-04
- NOTE12 OK, 1 NOTE2026-08-01
- OK13 OK2026-06-08
- ERROR12 OK, 1 ERROR2026-06-07
- OK12 OK2026-04-25
Show 3 earlier changesHide 3 earlier changes
- NOTE11 OK, 3 NOTE2026-04-22
- ERROR10 OK, 3 NOTE, 1 ERROR2026-04-18
- NOTE11 OK, 3 NOTE2026-03-10
Documentation
- Help pages
- 68
- Examples that run
- 98%65 run in a plain check · 1 never run
- Exported functions with examples
- 100%
- Documented parameters
- 96%
- Return-value docs
- 100%
- Help pages citing references
- 13%
- UDPipe Natural Language Processing - Basic Analytical Use Cases
- UDPipe Natural Language Processing - Model Building
- UDPipe Natural Language Processing - Parallel
- UDPipe Natural Language Processing - Text Annotation
- UDPipe Natural Language Processing - Topic Modelling Use Cases
- UDPipe Natural Language Processing - Try it out
- UDPipe Natural Language Processing - Universe
Downloads
Repository
- PRs merged (all time)
- 33%
- Median days to close, 50 most recently updated closed issues
- 2
- Median days to close, 50 most recently updated closed PRs
- 341
Stars over time
Forks over time
Releases over time
Issues over time
PRs over time
4 development-tooling and community-health practices detected across 3 families in the upstream repository
Checks run against github.com/bnosac/udpipe on 2026-10-11.
The R CMD check workflow runs on Linux and macOS, and on R-devel
GitHub Pages site last deployed 2020-12-10
Dependencies
Installing udpipe adds 2 packages (2 with compiled code, 7.5 MB of source tarballs). Heaviest: data.table 5.7 MB, Rcpp 1.8 MB.
Show all 20 packagesHide all 20 packages
Code & Tests
Datasets
People & History
Pick two releases to compare their code metrics.
- RR 4.6.0 released · 2026-04-24
- 0.8.16Latest
- 0.8.152025-11-28 · diff ↗
- 0.8.142025-11-26 · diff ↗
- 0.8.132025-11-26 · diff ↗
- 0.8.122025-09-04 · diff ↗
- RR 4.5.0 released · 2025-04-11
- RR 4.4.0 released · 2024-04-24
- RR 4.3.0 released · 2023-04-21
- 0.8.112023-01-06 · diff ↗
- 0.8.102022-11-10 · diff ↗
- RR 4.2.0 released · 2022-04-22
- 0.8.92022-03-24 · diff ↗
- 0.8.8
- 0.8.6
- RR 4.1.0 released · 2021-05-18
Show 22 earlier eventsHide 22 earlier events
- 0.8.5
- 0.8.4-12020-10-12 · diff ↗
- 0.8.42020-10-10 · diff ↗ ·
added 7 exports
as_fasttext, dtm_chisq, dtm_conform, dtm_svd_similarity, txt_count, txt_overlap, unlist_tokens - RR 4.0.0 released · 2020-04-24
- 0.8.3
- 0.8.2
- RR 3.6.0 released · 2019-04-26
- 0.8.1
- 0.8
- 0.7
- 0.6.12018-07-30 · diff ↗
- 0.6
- RR 3.5.0 released · 2018-04-23
- 0.5
- 0.42018-02-07 · diff ↗ ·
added 9 exports
as_conllu, as_cooccurrence, as_word2vec, document_term_frequencies_statistics, dtm_colsums, dtm_rowsums, keywords_collocation, keywords_phrases, keywords_rake - 0.3
- 0.2.22017-12-07 · diff ↗
- 0.2.1
- 0.22017-11-13 · diff ↗ ·
added 20 exports
collocation, cooccurrence, document_term_frequencies, document_term_matrix, dtm_cor, dtm_remove_lowfreq, dtm_remove_terms, dtm_remove_tfidf, dtm_reverse, dtm_tfidf, txt_collapse, txt_freq, txt_highlight, txt_next, txt_nextgram, txt_previous, txt_recode, txt_sample, txt_show, unique_identifier - 0.1.12017-09-13 · diff ↗
- 0.1.02017-09-01
- RR 3.4.0 released · 2017-04-21
Package metadata
- First published
- 2017-09-01
- Total releases
- 28 / 9 yrs
- License
- MPL-2.0 OSI
- Minimum R
- ≥ 2.10
- Bundled data
- 848 KB / 5 files
- Download size
- 3.1 MB
Cite
Cite this package
Run in R for the authors' preferred citation:
citation("udpipe")This package has no citation file, so citation() returns this entry built from DESCRIPTION.
Cite the R Observatory
For a number measured here: a download total, a coverage figure, an archival date.
From data release v2026-10-17, which the citation names so these numbers can be found later. More on citing and the projects behind them.