encoding-test-files

Encoding test files

APACHE-2.0 License

Stars
7
Committers
1
View Code on GitHub
Ecosystems: Python

Encoding test files

These are example test files for testing encoding issues.

They are generated by running generate.py with Python 2.7.

Assuming your browser can read this README as UTF-8, the content should be equivalent to:

première is first
première is slightly different
Кириллица is Cyrillic
𐐀 am Deseret

These are written into different encodings, where some characters might be replaced with ?. Only UTF8 and UTF16 is able to do all encodings.

TODO: Add BOM test files

Related Projects
python-percentcoding

Fast percent encoding and decoding for Python

07 Mar 2011 9
SublimeStringEncode

Converts characters from one "encoding" to another using a transformation (think HTML entities, n...

03 Jan 2012 149
top-1000

Top 1000 website URLs

09 Aug 2021 11
find-unicode-control

02 Nov 2021 24
charset_normalizer

Truly universal encoding detector in pure Python

02 Aug 2019 529
colab-convert

Convert .py files runnable in VSCode/Python or Atom/Hydrogen to jupyter/colab .ipynb notebooks an...

06 Apr 2022 116
text_renderer

Generate text images for training deep learning ocr model

28 May 2018 1,383
django-video-encoding

django-video-encoding helps to convert your videos into different formats and resolutions.

14 Mar 2017 115
validate-utf8

Python library and CLI for validating UTF-8 text

10 Mar 2019 3
chardet

Python character encoding detector

26 Jul 2012 2,087
_

_ allows you to write Python code using only _.

30 Oct 2023 3
steg

An exploration into Steganography with Python

05 Mar 2017 12
chardet

Rescue of Mark Pilgrim's chardet library

18 Jan 2012 13
pyutf8

Python extension for dealing with validation and cleanup of UTF-8 strings

13 Mar 2010 25
ftfy-web

Paste in some broken unicode text and FTFY will tell you how to fix it!

09 Jan 2018 67