korean-wikipedia-corpus

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

Releases , tfds-korean ( ) .

wikiextractor , <doc> . kss .

 .
.
  .

, .

1 
1 - 1
1 - 2
1 - ...
1 - n

2 
2 - 1
2 - 2
2 - ...
2 - m

3 
3 - 1
3 - 2
3 - ...
3 - m

...

(, 1989 10 3 ~ ) KBO  NC  .
  KBO  KIA    .

 -  .
2008   2 4 8( 32) LG      .
 2  2010              .
    .
   4  34.2 2 1,  2.34.
  , ,    , ,            , ,         2008 2012   2 1 1  .

 - 2012 .
  2 1( 10)     ,  2011    .
 ,     2011      .
2 94   0.303, 16, 67, 29, 33 .
  4         .
   ,       3  .
  2012            .

...

1. https://ko.wikipedia.org/wiki/:_

```sh
docker run \
    --rm \
    -it \
    -w /app \
    -v `pwd`:/app \
    -v WIKI_FILE_PATH:/wiki.xml.bz2:ro \
    -e WIKI_FILE=/wiki.xml.bz2 \
    python:3.6 bash -c ./extract-text.sh
```

     `<doc></doc>`     .

```sh
docker run \
    --rm \
    -it \
    -w /app \
    -v `pwd`:/app \
    python:3.6 bash -c ./preprocess-text.sh
```