T
START
get code_retour get lien_statut
check code_retour
read URL
F
code_retour=200
T
check encoding
ERR msg : code
get remote_encoding
F
T
F
T
encoding=""
get page_encoding
check encoding
F
encoding="utf-8"
get text
check language (file name)
check_encoding (iconv)
T
F
F
convertable
get_text
convert_text (iconv)
END
ERR msg : encoding
END
Dump-utf8
Page HTML
Dump-encodage
Dump-utf8
T
file="CN"
format/segment Chinese_text
format Latin_text
get context
get context_html
get frequency
get index
get Ngram
get ttr
token2line
Dump-utf8-formatted
Context-utf8
Context-html
concatenate dump concatenate context
END
fq
ttr
Whole dump-formatted Whole context
2-gram
Index
get context
get context_html
get_frequency
token2line
check language (file name)
F
file="EN"
lemmatize (tree-tagger en)
lemmatize (tree-tagger fr)
get index
get Ngram
get ttr
concatenate dump concatenate context
END
Dump-utf8-formatted
Context-utf8
Context-html
Index(lemmatized)
2-gram
Whole dump-formatted Whole context
ttr
fq
encodage
code_retour lien_statut
check language (file name)
T
file="CN"
format Latin_text
format/segment Chinese_text
get context
get context_html
get_frequency
token2line
F
check language (file name)
file="EN"
lemmatize (tree-tagger en)
lemmatize (tree-tagger fr)
get index
get Ngram
get ttr
concatenate dump concatenate context
END
2-gram
Whole dump-formatted Whole context
ttr
fq
Context-html
Context-utf8
Dump-utf8-formatted
get context
get context_html
get frequency
token2line
get index
get Ngram
get ttr
Dump-utf8-formatted
Context-utf8
Context-html
Index
2-gram
fq
concatenate dump concatenate context
Whole dump-formatted Whole context
END
ttr
Index (lemmatized)
T