Async Japanese Tokenizer Native Plugin for React Native for iOS and Android
MIT License
Async Japanese tokenizer native plugin for React Native that works on iOS and Android. It tokenizes Japanese text in background thread so it won't block UI. It comes in handy for such as indexing a lot of texts for full-text search.
It utilizes CFStringTokenizer
to tokenize strings into words.
The implementation is based on TinySegmenter. It is lightweight word segmenter which works without dictionaries.
$ npm install react-native-japanese-tokenizer --save
$ react-native link react-native-japanese-tokenizer
Libraries
➜ Add Files to [your project's name]
node_modules
➜ react-native-japanese-tokenizer
and add RNJapaneseTokenizer.xcodeproj
libRNJapaneseTokenizer.a
to your project's Build Phases
➜ Link Binary With Libraries
Cmd+R
)android/app/src/main/java/[...]/MainActivity.java
import dog.craftz.japanese_tokenizer.RNJapaneseTokenizerPackage;
to the imports at the top of the filenew RNJapaneseTokenizerPackage()
to the list returned by the getPackages()
methodandroid/settings.gradle
:
include ':react-native-japanese-tokenizer'
project(':react-native-japanese-tokenizer').projectDir = new File(rootProject.projectDir, '../node_modules/react-native-japanese-tokenizer/android')
android/app/build.gradle
:
compile project(':react-native-japanese-tokenizer')
Using Promise
:
import Tokenizer from 'react-native-japanese-tokenizer'
var text = "週休七日で働きたい"
Tokenizer.tokenize(text).then(tokens => {
...
})
async/await
:
import Tokenizer from 'react-native-japanese-tokenizer'
var text = "週休七日で働きたい"
const tokens = await Tokenizer.tokenize(text)
Also check out the test to learn more.
MIT License. By Takuya Matsuyama (@craftzdog).
Note that this module is based on following libraries with respective license. Thank you for the fantastic works!