morphemepiece
Morpheme Tokenization
v1.2.3
·
Apr 16, 2022
·
Apache License (>= 2)
Description
Tokenize text into morphemes. The morphemepiece algorithm uses a lookup table to determine the morpheme breakdown of words, and falls back on a modified wordpiece tokenization algorithm for words not found in the lookup table.
Downloads
287
Last 30 days
14430th
836
Last 90 days
3.5K
Last year
Trend: +4.4% (30d vs prior 30d)
CRAN Check Status
14
NOTE
Show all 14 flavors
| Flavor | Status |
|---|---|
| r-devel-linux-x86_64-debian-clang | NOTE |
| r-devel-linux-x86_64-debian-gcc | NOTE |
| r-devel-linux-x86_64-fedora-clang | NOTE |
| r-devel-linux-x86_64-fedora-gcc | NOTE |
| r-devel-macos-arm64 | NOTE |
| r-devel-windows-x86_64 | NOTE |
| r-oldrel-macos-arm64 | NOTE |
| r-oldrel-macos-x86_64 | NOTE |
| r-oldrel-windows-x86_64 | NOTE |
| r-patched-linux-x86_64 | NOTE |
| r-release-linux-x86_64 | NOTE |
| r-release-macos-arm64 | NOTE |
| r-release-macos-x86_64 | NOTE |
| r-release-windows-x86_64 | NOTE |
Check details (14 non-OK)
NOTE
r-devel-linux-x86_64-debian-clang
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-devel-linux-x86_64-debian-gcc
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-devel-linux-x86_64-fedora-clang
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-devel-linux-x86_64-fedora-gcc
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-devel-macos-arm64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-devel-windows-x86_64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-oldrel-macos-arm64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-oldrel-macos-x86_64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-oldrel-windows-x86_64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-patched-linux-x86_64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-release-linux-x86_64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-release-macos-arm64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-release-macos-x86_64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-release-windows-x86_64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
Check History
NOTE 0 OK · 14 NOTE · 0 WARNING · 0 ERROR · 0 FAILURE Mar 10, 2026
NOTE
r-devel-linux-x86_64-debian-clang
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-devel-linux-x86_64-debian-gcc
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-devel-linux-x86_64-fedora-clang
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-devel-linux-x86_64-fedora-gcc
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-devel-macos-arm64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-devel-windows-x86_64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-patched-linux-x86_64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-release-linux-x86_64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-release-macos-arm64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-release-macos-x86_64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-release-windows-x86_64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-oldrel-macos-arm64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-oldrel-macos-x86_64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?
NOTE
r-oldrel-windows-x86_64
Rd files
checkRd: (-1) morphemepiece_cache_dir.Rd:17-19: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:20-21: Lost braces in \itemize; meant \describe ? checkRd: (-1) morphemepiece_cache_dir.Rd:22-24: Lost braces in \itemize; meant \describe ?