mv: ‘./input-file.zip’ and ‘./input-file.zip’ are the same file Creating study carrel named subject-childrensStoriesGerman-gutenberg Initializing database Unzipping Archive: input-file.zip creating: ./tmp/input/input-file/ inflating: ./tmp/input/input-file/19915.txt inflating: ./tmp/input/input-file/30834.txt inflating: ./tmp/input/input-file/24593.txt inflating: ./tmp/input/input-file/24160.txt inflating: ./tmp/input/input-file/12116.txt inflating: ./tmp/input/input-file/8111.txt inflating: ./tmp/input/input-file/metadata.csv caution: excluded filename not matched: *MACOSX* === DIRECTORIES: ./tmp/input === DIRECTORY: ./tmp/input/input-file === metadata file: ./tmp/input/input-file/metadata.csv === found metadata file === updating bibliographic database Building study carrel named subject-childrensStoriesGerman-gutenberg FILE: cache/19915.txt OUTPUT: txt/19915.txt FILE: cache/30834.txt OUTPUT: txt/30834.txt FILE: cache/24593.txt OUTPUT: txt/24593.txt FILE: cache/24160.txt OUTPUT: txt/24160.txt FILE: cache/8111.txt OUTPUT: txt/8111.txt FILE: cache/12116.txt OUTPUT: txt/12116.txt === file2bib.sh === id: 24593 author: Hauff, Wilhelm title: The Oriental Story Book: A Collection of Tales date: pages: extension: .txt txt: ./txt/24593.txt cache: ./cache/24593.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 2 resourceName b'24593.txt' Traceback (most recent call last): File "/data-disk/reader-compute/reader-classic/bin/file2bib.py", line 107, in text = textacy.preprocessing.normalize.normalize_quotation_marks( text ) File "/data-disk/python/lib/python3.8/site-packages/textacy/preprocessing/normalize.py", line 32, in normalize_quotation_marks return text.translate(QUOTE_TRANSLATION_TABLE) AttributeError: 'NoneType' object has no attribute 'translate' === file2bib.sh === id: 24160 author: Schmid, Christoph von title: The Basket of Flowers date: pages: extension: .txt txt: ./txt/24160.txt cache: ./cache/24160.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 2 resourceName b'24160.txt' Traceback (most recent call last): File "/data-disk/reader-compute/reader-classic/bin/file2bib.py", line 107, in text = textacy.preprocessing.normalize.normalize_quotation_marks( text ) File "/data-disk/python/lib/python3.8/site-packages/textacy/preprocessing/normalize.py", line 32, in normalize_quotation_marks return text.translate(QUOTE_TRANSLATION_TABLE) AttributeError: 'NoneType' object has no attribute 'translate' 24593 txt/../ent/24593.ent 24593 txt/../pos/24593.pos 24160 txt/../wrd/24160.wrd Traceback (most recent call last): File "/data-disk/reader-compute/reader-classic/bin/txt2keywords.py", line 54, in for keyword, score in ( yake( doc, ngrams=NGRAMS, topn=TOPN ) ) : File "/data-disk/python/lib/python3.8/site-packages/textacy/ke/yake.py", line 96, in yake word_scores = _compute_word_scores(doc, word_occ_vals, word_freqs, stop_words) File "/data-disk/python/lib/python3.8/site-packages/textacy/ke/yake.py", line 205, in _compute_word_scores freq_baseline = statistics.mean(freqs_nsw) + statistics.stdev(freqs_nsw) File "/data-disk/python/lib/python3.8/statistics.py", line 315, in mean raise StatisticsError('mean requires at least one data point') statistics.StatisticsError: mean requires at least one data point 24593 txt/../wrd/24593.wrd Traceback (most recent call last): File "/data-disk/reader-compute/reader-classic/bin/txt2keywords.py", line 54, in for keyword, score in ( yake( doc, ngrams=NGRAMS, topn=TOPN ) ) : File "/data-disk/python/lib/python3.8/site-packages/textacy/ke/yake.py", line 96, in yake word_scores = _compute_word_scores(doc, word_occ_vals, word_freqs, stop_words) File "/data-disk/python/lib/python3.8/site-packages/textacy/ke/yake.py", line 205, in _compute_word_scores freq_baseline = statistics.mean(freqs_nsw) + statistics.stdev(freqs_nsw) File "/data-disk/python/lib/python3.8/statistics.py", line 315, in mean raise StatisticsError('mean requires at least one data point') statistics.StatisticsError: mean requires at least one data point 24160 txt/../pos/24160.pos 24160 txt/../ent/24160.ent 12116 txt/../pos/12116.pos 19915 txt/../wrd/19915.wrd 12116 txt/../wrd/12116.wrd 19915 txt/../ent/19915.ent 19915 txt/../pos/19915.pos 12116 txt/../ent/12116.ent === file2bib.sh === id: 19915 author: Hoffmann, Heinrich title: Slovenly Betsy date: pages: extension: .txt txt: ./txt/19915.txt cache: ./cache/19915.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:content_handler ToTextContentHandler X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 2 resourceName b'19915.txt' === file2bib.sh === id: 12116 author: Hoffmann, Heinrich title: Struwwelpeter: Merry Stories and Funny Pictures date: pages: extension: .txt txt: ./txt/12116.txt cache: ./cache/12116.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:content_handler ToTextContentHandler X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 3 resourceName b'12116.txt' 8111 txt/../wrd/8111.wrd 8111 txt/../pos/8111.pos 30834 txt/../wrd/30834.wrd 30834 txt/../pos/30834.pos 30834 txt/../ent/30834.ent 8111 txt/../ent/8111.ent === file2bib.sh === id: 8111 author: nan title: After Long Years and Other Stories date: pages: extension: .txt txt: ./txt/8111.txt cache: ./cache/8111.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:content_handler ToTextContentHandler X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 4 resourceName b'8111.txt' === file2bib.sh === id: 30834 author: Arndt, Margaret title: Fairy Tales from the German Forests date: pages: extension: .txt txt: ./txt/30834.txt cache: ./cache/30834.txt Content-Encoding UTF-8 Content-Type text/plain; charset=UTF-8 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:content_handler ToTextContentHandler X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 5 resourceName b'30834.txt' Done mapping. Reducing subject-childrensStoriesGerman-gutenberg === reduce.pl bib === id = 19915 author = Hoffmann, Heinrich title = Slovenly Betsy date = pages = extension = .txt mime = text/plain words = 2622 sentences = 334 flesch = 98 summary = Miss Betsy took no pains; Nice little boys and girls were there, THE DREADFUL STORY OF PAULINE AND THE MATCHES Mamma and Nurse went out one day, And kind Mamma and Nurse had told her, And Mamma often burns the same. And arms and hands, and eyes and nose; "Here, Charlotte," said Mamma one day. And soon upon the ground let fall While Charlotte hid her face and cried. Such was a kind mother's advice. "I pray you now, my little child," The little girls whom now you'll see The little girls whom now you'll see Soon as she saw her mother stand, And many a little gift obtained; Now Minnie was a pretty girl, Now to her little sister Bess This Minnie longed for (envious child), One day, upon some bee-hives near how good it looks!" she cries, Upon her little bed, "Fie!" said her mother, "don't you know, cache = ./cache/19915.txt txt = ./txt/19915.txt === reduce.pl bib === id = 30834 author = Arndt, Margaret title = Fairy Tales from the German Forests date = pages = extension = .txt mime = text/plain words = 55624 sentences = 3618 flesch = 90 summary = looked down the wood path, Hansi saw a troop of little men, such as you "Dear little men," she said, "may I have the tree? "O, Mr Dwarf, that's the wrong way home, I'm sure," said Hansi. "Dear little men, I think I really ought to be going home," said Hansi "My stones must be there still," said Karl as they came to the little "You little fairy," he said, and looked at her with "We'll sign anything you like, dear little men," said Norah, joyfully, "Mother," said Helmut that evening when she came to wish him good night, "Hush," said my Lord indignantly, "she looks more like a little angel," "Babette," said the Count sternly, "if you are a _good_ little girl, and "You look like two little wood-men yourselves," she said. "Well, it's like this," said the little man; "I'm a tree, and the tree's cache = ./cache/30834.txt txt = ./txt/30834.txt === reduce.pl bib === === reduce.pl bib === === reduce.pl bib === id = 12116 author = Hoffmann, Heinrich title = Struwwelpeter: Merry Stories and Funny Pictures date = pages = extension = .txt mime = text/plain words = 2448 sentences = 289 flesch = 106 summary = He caught the flies, poor little things, And whipped poor Tray till he was sore, Mamma and Nurse went out one day "Boys, leave the Black-a-moor alone! The Story of the Man that went out Shooting The little hare came, hop, hop, hop, He cries and screams and runs away; The Story of Little Suck-a-Thumb One day Mamma said "Conrad dear, To little boys who suck their thumbs; And caught out little Suck-a-Thumb. Mamma comes home: there Conrad stands, "Ah!" said Mamma, "I knew he'd come To naughty little Suck-a-Thumb." Next day, now look, the picture shows Look at him, now the fourth day's come! The Story of Johnny Head-in-Air "Look at little Johnny there, Little Johnny Head-In-Air!" Came a little dog one day; Headlong in poor Johnny fell. And, to tease poor Johnny, said "Silly little Johnny, look, All good little girls and boys No one heard his screams and cries; cache = ./cache/12116.txt txt = ./txt/12116.txt === reduce.pl bib === id = 8111 author = nan title = After Long Years and Other Stories date = pages = extension = .txt mime = text/plain words = 47188 sentences = 3193 flesch = 87 summary = Then Almira took his hand and said: "Antonio, how good you were to save "My dear husband," said his wife, "I wish you had come a few moments "Yes," said the old fisherman, "this is the boy who appeared to us, like "Well," said Hans, "I know a worthy man and I have come to bring him "You are looking for your mother," said she, "but little do we know last he said: "Oh, my dear parents, what great good you have always "Yes," said the father, "in the morning light, we found that our house me great pain," said the mother, "that every picture which your father "But," said he, "the old man must rest for three days before I can when the boy said: "Dear father, let us go to the top of the hill and One night Albert returned carrying the paper, and said: "The good cache = ./cache/8111.txt txt = ./txt/8111.txt Building ./etc/reader.txt 30834 8111 19915 8111 30834 24593 number of items: 6 sum of words: 107,882 average size in words: 26,970 average readability score: 95 nouns: mother; children; father; man; time; day; house; eyes; tree; boy; way; child; night; people; money; water; home; men; trees; room; joy; heart; door; work; head; hand; fairy; nothing; one; moment; light; morning; life; illustration; wife; hands; fire; son; years; family; village; things; days; boys; garden; daughter; picture; love; face; times verbs: was; had; said; is; have; were; be; do; are; been; see; did; come; know; saw; made; thought; has; found; came; seemed; stood; go; looked; let; heard; began; went; am; give; cried; think; get; take; took; find; ''s; make; say; tell; look; gave; felt; seen; put; called; brought; help; passed; knew adjectives: little; old; good; great; many; beautiful; poor; other; dear; last; young; more; own; such; long; few; much; red; fine; green; white; strange; dark; high; true; big; same; sure; large; wonderful; new; full; best; wild; able; first; deep; ready; rich; black; sweet; next; happy; several; real; open; nice; lovely; dead; pretty adverbs: not; so; then; now; up; out; very; never; again; there; n''t; here; too; just; soon; only; as; more; still; down; away; well; once; always; off; quite; all; on; ever; home; suddenly; back; far; much; often; most; even; in; really; together; however; perhaps; long; yet; over; almost; also; first; no; along pronouns: he; his; it; i; you; her; she; they; him; them; my; their; me; we; your; us; our; its; himself; herself; themselves; one; itself; yourself; myself; mine; yours; ourselves; ''s; theirs; ours; yourselves; musingly--"they; hers; bear-- proper nouns: _; mr.; babette; karl; antonio; count; hans; norah; god; david; trudel; emperor; mrs.; king; walter; hansi; george; daniel; acton; mr; hugo; christmas; alfred; rudolf; lottchen; easter; marie; hermann; countess; trent; richard; hare; richmond; albert; helmut; chapter; bush; amy; käthe; holle; berlow; old; ii; counselor; vosky; vollmar; philip; hut; castle; swift keywords: walter; trudel; trent; rudolf; norah; mrs.; mr.; mamma; lottchen; little; king; karl; johnny; illustration; hugo; hermann; hansi; hans; god; george; emperor; easter; david; daniel; count; christmas; betsy; babette; antonio; alfred; acton one topic; one dimension: said file(s): ./cache/30834.txt titles(s): Fairy Tales from the German Forests three topics; one dimension: said; said; alack file(s): ./cache/30834.txt, ./cache/8111.txt, titles(s): Fairy Tales from the German Forests | After Long Years and Other Stories | The Oriental Story Book: A Collection of Tales five topics; three dimensions: said little like; said father little; trough befell growled; trough befell growled; trough befell growled file(s): ./cache/30834.txt, ./cache/8111.txt, , , titles(s): Fairy Tales from the German Forests | After Long Years and Other Stories | The Oriental Story Book: A Collection of Tales | The Oriental Story Book: A Collection of Tales | The Oriental Story Book: A Collection of Tales Type: gutenberg title: subject-childrensStoriesGerman-gutenberg date: 2021-06-03 time: 14:06 username: emorgan patron: Eric Morgan email: emorgan@nd.edu input: facet_subject:"Children's stories, German" ==== make-pages.sh htm files ==== make-pages.sh complex files ==== make-pages.sh named enities ==== making bibliographics id: 30834 author: Arndt, Margaret title: Fairy Tales from the German Forests date: words: 55624.0 sentences: 3618.0 pages: flesch: 90.0 cache: ./cache/30834.txt txt: ./txt/30834.txt summary: looked down the wood path, Hansi saw a troop of little men, such as you "Dear little men," she said, "may I have the tree? "O, Mr Dwarf, that''s the wrong way home, I''m sure," said Hansi. "Dear little men, I think I really ought to be going home," said Hansi "My stones must be there still," said Karl as they came to the little "You little fairy," he said, and looked at her with "We''ll sign anything you like, dear little men," said Norah, joyfully, "Mother," said Helmut that evening when she came to wish him good night, "Hush," said my Lord indignantly, "she looks more like a little angel," "Babette," said the Count sternly, "if you are a _good_ little girl, and "You look like two little wood-men yourselves," she said. "Well, it''s like this," said the little man; "I''m a tree, and the tree''s id: 24593 author: Hauff, Wilhelm title: The Oriental Story Book: A Collection of Tales date: words: nan sentences: nan pages: flesch: nan cache: txt: summary: id: 19915 author: Hoffmann, Heinrich title: Slovenly Betsy date: words: 2622.0 sentences: 334.0 pages: flesch: 98.0 cache: ./cache/19915.txt txt: ./txt/19915.txt summary: Miss Betsy took no pains; Nice little boys and girls were there, THE DREADFUL STORY OF PAULINE AND THE MATCHES Mamma and Nurse went out one day, And kind Mamma and Nurse had told her, And Mamma often burns the same. And arms and hands, and eyes and nose; "Here, Charlotte," said Mamma one day. And soon upon the ground let fall While Charlotte hid her face and cried. Such was a kind mother''s advice. "I pray you now, my little child," The little girls whom now you''ll see The little girls whom now you''ll see Soon as she saw her mother stand, And many a little gift obtained; Now Minnie was a pretty girl, Now to her little sister Bess This Minnie longed for (envious child), One day, upon some bee-hives near how good it looks!" she cries, Upon her little bed, "Fie!" said her mother, "don''t you know, id: 12116 author: Hoffmann, Heinrich title: Struwwelpeter: Merry Stories and Funny Pictures date: words: 2448.0 sentences: 289.0 pages: flesch: 106.0 cache: ./cache/12116.txt txt: ./txt/12116.txt summary: He caught the flies, poor little things, And whipped poor Tray till he was sore, Mamma and Nurse went out one day "Boys, leave the Black-a-moor alone! The Story of the Man that went out Shooting The little hare came, hop, hop, hop, He cries and screams and runs away; The Story of Little Suck-a-Thumb One day Mamma said "Conrad dear, To little boys who suck their thumbs; And caught out little Suck-a-Thumb. Mamma comes home: there Conrad stands, "Ah!" said Mamma, "I knew he''d come To naughty little Suck-a-Thumb." Next day, now look, the picture shows Look at him, now the fourth day''s come! The Story of Johnny Head-in-Air "Look at little Johnny there, Little Johnny Head-In-Air!" Came a little dog one day; Headlong in poor Johnny fell. And, to tease poor Johnny, said "Silly little Johnny, look, All good little girls and boys No one heard his screams and cries; id: 24160 author: Schmid, Christoph von title: The Basket of Flowers date: words: nan sentences: nan pages: flesch: nan cache: txt: summary: id: 8111 author: nan title: After Long Years and Other Stories date: words: 47188.0 sentences: 3193.0 pages: flesch: 87.0 cache: ./cache/8111.txt txt: ./txt/8111.txt summary: Then Almira took his hand and said: "Antonio, how good you were to save "My dear husband," said his wife, "I wish you had come a few moments "Yes," said the old fisherman, "this is the boy who appeared to us, like "Well," said Hans, "I know a worthy man and I have come to bring him "You are looking for your mother," said she, "but little do we know last he said: "Oh, my dear parents, what great good you have always "Yes," said the father, "in the morning light, we found that our house me great pain," said the mother, "that every picture which your father "But," said he, "the old man must rest for three days before I can when the boy said: "Dear father, let us go to the top of the hill and One night Albert returned carrying the paper, and said: "The good ==== make-pages.sh questions ==== make-pages.sh search ==== make-pages.sh topic modeling corpus Zipping study carrel