mv: ‘./input-file.zip’ and ‘./input-file.zip’ are the same file Creating study carrel named subject-extinctCities-gutenberg Initializing database Unzipping Archive: input-file.zip creating: ./tmp/input/input-file/ inflating: ./tmp/input/input-file/24054.txt inflating: ./tmp/input/input-file/4711.txt inflating: ./tmp/input/input-file/499.txt inflating: ./tmp/input/input-file/50802.txt inflating: ./tmp/input/input-file/43204.txt inflating: ./tmp/input/input-file/63473.txt inflating: ./tmp/input/input-file/63392.txt inflating: ./tmp/input/input-file/metadata.csv caution: excluded filename not matched: *MACOSX* === DIRECTORIES: ./tmp/input === DIRECTORY: ./tmp/input/input-file === metadata file: ./tmp/input/input-file/metadata.csv === found metadata file === updating bibliographic database Building study carrel named subject-extinctCities-gutenberg FILE: cache/43204.txt OUTPUT: txt/43204.txt FILE: cache/4711.txt OUTPUT: txt/4711.txt FILE: cache/63473.txt OUTPUT: txt/63473.txt FILE: cache/24054.txt OUTPUT: txt/24054.txt FILE: cache/50802.txt OUTPUT: txt/50802.txt FILE: cache/499.txt OUTPUT: txt/499.txt FILE: cache/63392.txt OUTPUT: txt/63392.txt === file2bib.sh === id: 24054 author: Garrett, Randall title: Dead Giveaway date: pages: extension: .txt txt: ./txt/24054.txt cache: ./cache/24054.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 1 resourceName b'24054.txt' Traceback (most recent call last): File "/data-disk/reader-compute/reader-classic/bin/file2bib.py", line 107, in text = textacy.preprocessing.normalize.normalize_quotation_marks( text ) File "/data-disk/python/lib/python3.8/site-packages/textacy/preprocessing/normalize.py", line 32, in normalize_quotation_marks return text.translate(QUOTE_TRANSLATION_TABLE) AttributeError: 'NoneType' object has no attribute 'translate' 24054 txt/../pos/24054.pos 24054 txt/../ent/24054.ent 24054 txt/../wrd/24054.wrd Traceback (most recent call last): File "/data-disk/reader-compute/reader-classic/bin/txt2keywords.py", line 54, in for keyword, score in ( yake( doc, ngrams=NGRAMS, topn=TOPN ) ) : File "/data-disk/python/lib/python3.8/site-packages/textacy/ke/yake.py", line 96, in yake word_scores = _compute_word_scores(doc, word_occ_vals, word_freqs, stop_words) File "/data-disk/python/lib/python3.8/site-packages/textacy/ke/yake.py", line 205, in _compute_word_scores freq_baseline = statistics.mean(freqs_nsw) + statistics.stdev(freqs_nsw) File "/data-disk/python/lib/python3.8/statistics.py", line 315, in mean raise StatisticsError('mean requires at least one data point') statistics.StatisticsError: mean requires at least one data point 50802 txt/../wrd/50802.wrd 63473 txt/../pos/63473.pos 50802 txt/../pos/50802.pos 63473 txt/../wrd/63473.wrd 63392 txt/../wrd/63392.wrd 63392 txt/../pos/63392.pos 63473 txt/../ent/63473.ent 63392 txt/../ent/63392.ent === file2bib.sh === id: 50802 author: Doede, William R. title: A City Near Centaurus date: pages: extension: .txt txt: ./txt/50802.txt cache: ./cache/50802.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:content_handler ToTextContentHandler X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 3 resourceName b'50802.txt' 50802 txt/../ent/50802.ent === file2bib.sh === id: 63473 author: Hinckley, Lyman D. title: Dust Unto Dust date: pages: extension: .txt txt: ./txt/63473.txt cache: ./cache/63473.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:content_handler ToTextContentHandler X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 2 resourceName b'63473.txt' === file2bib.sh === id: 63392 author: Knight, Damon title: Doorway to Kal-Jmar date: pages: extension: .txt txt: ./txt/63392.txt cache: ./cache/63392.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:content_handler ToTextContentHandler X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 208 resourceName b'63392.txt' 499 txt/../wrd/499.wrd 499 txt/../pos/499.pos 4711 txt/../wrd/4711.wrd 4711 txt/../pos/4711.pos 43204 txt/../wrd/43204.wrd 43204 txt/../pos/43204.pos 499 txt/../ent/499.ent 4711 txt/../ent/4711.ent 43204 txt/../ent/43204.ent === file2bib.sh === id: 499 author: Appleton, Victor title: Tom Swift in the Land of Wonders; Or, The Underground Search for the Idol of Gold date: pages: extension: .txt txt: ./txt/499.txt cache: ./cache/499.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:content_handler ToTextContentHandler X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 3 resourceName b'499.txt' === file2bib.sh === id: 4711 author: Appleton, Victor title: Tom Swift in the City of Gold; Or, Marvelous Adventures Underground date: pages: extension: .txt txt: ./txt/4711.txt cache: ./cache/4711.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:content_handler ToTextContentHandler X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 3 resourceName b'4711.txt' === file2bib.sh === id: 43204 author: Young, Clarence title: The Motor Boys in Mexico; Or, The Secret of the Buried City date: pages: extension: .txt txt: ./txt/43204.txt cache: ./cache/43204.txt Content-Encoding ISO-8859-1 Content-Type text/plain; charset=ISO-8859-1 X-Parsed-By ['org.apache.tika.parser.DefaultParser', 'org.apache.tika.parser.csv.TextAndCSVParser'] X-TIKA:content_handler ToTextContentHandler X-TIKA:embedded_depth 0 X-TIKA:parse_time_millis 4 resourceName b'43204.txt' Done mapping. Reducing subject-extinctCities-gutenberg === reduce.pl bib === === reduce.pl bib === id = 63473 author = Hinckley, Lyman D. title = Dust Unto Dust date = pages = extension = .txt mime = text/plain words = 4936 sentences = 594 flesch = 91 summary = Martin led Wass, and the gangling, scarecrow-like Rodney, through the "Hey, Martin!" Rodney called through his helmet radio. Three feet from the edge of the city Martin stopped and stubbed at the Martin and Rodney started slowly down the wide metal "Martin!" Wass' voice came through the receivers in both their radios. Wass said, more quietly, "Remember that metal band? Martin clapped a hand on Rodney's shoulder again, starting him moving, "No go," Martin said to Wass. Walking with Rodney, Martin heard Wass' voice, flat and metallic Wass said coldly, "He's right, Martin." Wass stared down at the metal street, as if he could look through it. Martin adjusted his torch, began to lead the way down the metal ramp. After a time he said, "Rodney, Wass, it's dust, down there. Looking up, Martin saw Wass above Rodney. "All right, Wass," Martin said quietly, as Rodney released the rope and cache = ./cache/63473.txt txt = ./txt/63473.txt === reduce.pl bib === id = 43204 author = Young, Clarence title = The Motor Boys in Mexico; Or, The Secret of the Buried City date = pages = extension = .txt mime = text/plain words = 53106 sentences = 4579 flesch = 92 summary = "Run back to the machine and get one," said Jerry to Bob. "More like hanging two people with one rope," laughed Bob. But Ned's plan was voted a good one. When Jerry and Bob had pulled Professor Snodgrass up as far as possible, "Much obliged to you, boys," he said, coming over to Bob and Jerry. "I'm going to write home now and find if I can go," said Ned. Jerry and Bob said they would do the same, and soon three letters were "He got Jerry's money-belt," said Ned, "but----" "Good-night," said Ned and Jerry, as they locked their door, which had "It may be a whole school of fishes," said Bob. Just then there came a more violent agitation of the water, and the auto "We must get Bob, no matter what happens," said Jerry to Ned. professor, Ned and Jerry ran their machine over to where Noddy was. "And the other is Bob's," said Jerry. cache = ./cache/43204.txt txt = ./txt/43204.txt === reduce.pl bib === id = 499 author = Appleton, Victor title = Tom Swift in the Land of Wonders; Or, The Underground Search for the Idol of Gold date = pages = extension = .txt mime = text/plain words = 44647 sentences = 3491 flesch = 89 summary = "So, Tom Swift," went on the professor, "I come to you for help in this "Surely you don't mean it, Tom Swift," gasped Professor Bumper at "Nor did I ever hear Professor Beecher speak of Tom," said the "Tom how soon can we go?" asked Professor Bumper, as he began arranging while Mr. Damon went out with Tom and Ned to one of the shops to look "No," said Tom, with a look at Ned, for he did not want to take the big "Yes. You know Professor Bumper spoke of a rival--a man named Beecher "You are right, Tom," said Professor Bumper. Before Tom and Ned reached the place whence Professor Bumper had Professor Bumper was able to be about, and Tom and Ned the natives settled down, Tom, Ned, Mr. Damon and Professor Bumper Tom, Professor Bumper, Ned and Mr. Damon set cache = ./cache/499.txt txt = ./txt/499.txt === reduce.pl bib === id = 50802 author = Doede, William R. title = A City Near Centaurus date = pages = extension = .txt mime = text/plain words = 5104 sentences = 654 flesch = 95 summary = was doing in an old dead city so far from his people. "You never told us about this old dead city," Michaelson said, And even if they did, spirits care nothing for old cities half He walked away from the old man, heading for another building. Michaelson stood in the ancient street, tall, gaunt, feet planted wide, god to feel the spirits moving in the houses and walking in these old Michaelson was warming up to his subject, but Maota shook his head like Old Maota stood in the street with webbed feet planted far apart in Maota looked off toward the hills, old eyes filmed from years of sand Michaelson prepared himself to move if the old man's finger slid closer Old Maota read, Michaelson listened. But Maota was an old man. Michaelson did, and he saw the city beyond, as if he were looking "I don't like your thoughts," Maota said. cache = ./cache/50802.txt txt = ./txt/50802.txt === reduce.pl bib === id = 4711 author = Appleton, Victor title = Tom Swift in the City of Gold; Or, Marvelous Adventures Underground date = pages = extension = .txt mime = text/plain words = 45118 sentences = 3586 flesch = 93 summary = "This white man thought a great deal of the image," said Tom, again "Tom Swift, have you got that map?" suddenly cried Mr. Damon. "How--how big am dem gold images, Massa Tom?" asked Eradicate readiness, and when good-byes had been said to Mr. Swift, and Mrs. Baggert, Tom, Ned and Mr. Damon, followed by the faithful colored Tom and Ned went to several theatrical performances, and wanted Mr. Damon to go with them, but the odd man said he wanted to visit "Say!" began Ned in wondering tones, "Tom, doesn't that look like--" Tom found Ned and Mr. Damon, who were looking for him. The image was not in sight, though Tom and Ned and Mr. Damon looked eagerly around for it. Tom, Ned, Mr. Damon and Eradicate had to do most of the work. Mr. Damon and Eradicate went with Tom and Ned. It was no easy work cache = ./cache/4711.txt txt = ./txt/4711.txt === reduce.pl bib === id = 63392 author = Knight, Damon title = Doorway to Kal-Jmar date = pages = extension = .txt mime = text/plain words = 6740 sentences = 575 flesch = 90 summary = Syme murmured something and turned away, feeling the spaceman's eyes Syme looked at the man, nursing the tortured muscles of his arms. "Never mind," said Syme, taking his arm. "I wanta go to Kal-Jmar," said Tate. Harold Tate told him, and later, when Syme had taken him to his rooms, Syme turned the little sand car up a gentle rise towards the tortuous Syme nodded and moved the sand car up to the edge of the gully. "Most of these gullies peter out gradually," said Syme, "but if we want "Yeah," said Syme, and opened the door. "There are many secrets in Kal-Jmar," the Martian said, "among them a into cities like Kal-Jmar; our ancestors chose to adapt their bodies to "Kal-Jmar," said Syme. just stood there, looking dumbly from the gun to Syme's dark face. know, either, that Syme was not a Martian, or that he wanted a steak, cache = ./cache/63392.txt txt = ./txt/63392.txt Building ./etc/reader.txt 4711 499 43204 499 4711 63473 number of items: 7 sum of words: 159,651 average size in words: 26,608 average readability score: 91 nouns: city; professor; gold; time; man; way; place; one; boys; something; night; head; men; camp; friends; day; side; inventor; auto; stone; nothing; water; party; moment; air; thing; tunnel; naturalist; map; idol; voice; trip; eyes; image; hand; machine; days; temple; room; travelers; river; part; things; door; boy; money; chum; car; balloon; others verbs: was; had; is; be; have; were; said; do; ''s; are; did; get; asked; been; go; going; come; see; made; came; went; know; cried; take; think; ''m; ''re; got; make; look; saw; has; find; say; want; seemed; let; tell; looked; put; heard; found; ''ve; called; guess; bless; replied; told; took; buried adjectives: good; little; old; other; more; young; big; many; great; several; much; small; right; same; ancient; first; few; underground; long; sure; next; ready; new; better; golden; strange; last; dark; large; wild; white; such; own; low; glad; electric; afraid; mexican; whole; short; black; wonderful; possible; only; quick; dead; best; true; red; least adverbs: not; n''t; up; then; so; out; now; here; back; down; on; there; as; away; off; too; all; just; only; again; very; soon; in; more; never; well; over; still; much; once; even; later; perhaps; suddenly; along; far; quickly; about; right; maybe; enough; ever; ahead; yet; almost; first; better; slowly; forward; probably pronouns: it; he; i; they; his; you; we; him; them; their; my; us; me; our; your; she; its; himself; her; ''s; themselves; myself; itself; one; ourselves; ''em; yours; yourself; em; ye; ours; mine; herself; i''m; theirs; ob; yo; ya; then--"it proper nouns: tom; ned; mr.; jerry; damon; bob; professor; noddy; bumper; vasco; _; swift; exclaimed; beecher; martin; mexicans; eradicate; andy; mexican; rodney; mexico; boys; jacinto; indians; motor; syme; wass; tate; nestor; michaelson; new; maota; delazes; foger; dalsett; york; mary; bilette; snodgrass; rad; maximina; illingway; cried; chapter; yo; dat; fogers; series; nixon; jack keywords: ned; tom; swift; professor; mr.; damon; wass; vasco; tate; syme; snodgrass; rodney; noddy; motor; michaelson; mexico; mexicans; martin; maota; kal; jmar; jerry; jacinto; indians; illingway; honduras; foger; eradicate; don; delazes; dalsett; copan; bumper; boys; book; bob; bless; bilette; beecher; andy one topic; one dimension: tom file(s): titles(s): Dead Giveaway three topics; one dimension: tom; jerry; syme file(s): ./cache/4711.txt, ./cache/43204.txt, ./cache/63392.txt titles(s): Tom Swift in the City of Gold; Or, Marvelous Adventures Underground | The Motor Boys in Mexico; Or, The Secret of the Buried City | Doorway to Kal-Jmar five topics; three dimensions: jerry professor bob; tom professor ned; tom ned mr; syme said tate; impression bowl north file(s): ./cache/43204.txt, ./cache/499.txt, ./cache/4711.txt, ./cache/63392.txt, titles(s): The Motor Boys in Mexico; Or, The Secret of the Buried City | Tom Swift in the Land of Wonders; Or, The Underground Search for the Idol of Gold | Tom Swift in the City of Gold; Or, Marvelous Adventures Underground | Doorway to Kal-Jmar | Dead Giveaway Type: gutenberg title: subject-extinctCities-gutenberg date: 2021-06-06 time: 15:06 username: emorgan patron: Eric Morgan email: emorgan@nd.edu input: facet_subject:"Extinct cities" ==== make-pages.sh htm files ==== make-pages.sh complex files ==== make-pages.sh named enities ==== making bibliographics id: 4711 author: Appleton, Victor title: Tom Swift in the City of Gold; Or, Marvelous Adventures Underground date: words: 45118.0 sentences: 3586.0 pages: flesch: 93.0 cache: ./cache/4711.txt txt: ./txt/4711.txt summary: "This white man thought a great deal of the image," said Tom, again "Tom Swift, have you got that map?" suddenly cried Mr. Damon. "How--how big am dem gold images, Massa Tom?" asked Eradicate readiness, and when good-byes had been said to Mr. Swift, and Mrs. Baggert, Tom, Ned and Mr. Damon, followed by the faithful colored Tom and Ned went to several theatrical performances, and wanted Mr. Damon to go with them, but the odd man said he wanted to visit "Say!" began Ned in wondering tones, "Tom, doesn''t that look like--" Tom found Ned and Mr. Damon, who were looking for him. The image was not in sight, though Tom and Ned and Mr. Damon looked eagerly around for it. Tom, Ned, Mr. Damon and Eradicate had to do most of the work. Mr. Damon and Eradicate went with Tom and Ned. It was no easy work id: 499 author: Appleton, Victor title: Tom Swift in the Land of Wonders; Or, The Underground Search for the Idol of Gold date: words: 44647.0 sentences: 3491.0 pages: flesch: 89.0 cache: ./cache/499.txt txt: ./txt/499.txt summary: "So, Tom Swift," went on the professor, "I come to you for help in this "Surely you don''t mean it, Tom Swift," gasped Professor Bumper at "Nor did I ever hear Professor Beecher speak of Tom," said the "Tom how soon can we go?" asked Professor Bumper, as he began arranging while Mr. Damon went out with Tom and Ned to one of the shops to look "No," said Tom, with a look at Ned, for he did not want to take the big "Yes. You know Professor Bumper spoke of a rival--a man named Beecher "You are right, Tom," said Professor Bumper. Before Tom and Ned reached the place whence Professor Bumper had Professor Bumper was able to be about, and Tom and Ned the natives settled down, Tom, Ned, Mr. Damon and Professor Bumper Tom, Professor Bumper, Ned and Mr. Damon set id: 50802 author: Doede, William R. title: A City Near Centaurus date: words: 5104.0 sentences: 654.0 pages: flesch: 95.0 cache: ./cache/50802.txt txt: ./txt/50802.txt summary: was doing in an old dead city so far from his people. "You never told us about this old dead city," Michaelson said, And even if they did, spirits care nothing for old cities half He walked away from the old man, heading for another building. Michaelson stood in the ancient street, tall, gaunt, feet planted wide, god to feel the spirits moving in the houses and walking in these old Michaelson was warming up to his subject, but Maota shook his head like Old Maota stood in the street with webbed feet planted far apart in Maota looked off toward the hills, old eyes filmed from years of sand Michaelson prepared himself to move if the old man''s finger slid closer Old Maota read, Michaelson listened. But Maota was an old man. Michaelson did, and he saw the city beyond, as if he were looking "I don''t like your thoughts," Maota said. id: 24054 author: Garrett, Randall title: Dead Giveaway date: words: nan sentences: nan pages: flesch: nan cache: txt: summary: id: 63473 author: Hinckley, Lyman D. title: Dust Unto Dust date: words: 4936.0 sentences: 594.0 pages: flesch: 91.0 cache: ./cache/63473.txt txt: ./txt/63473.txt summary: Martin led Wass, and the gangling, scarecrow-like Rodney, through the "Hey, Martin!" Rodney called through his helmet radio. Three feet from the edge of the city Martin stopped and stubbed at the Martin and Rodney started slowly down the wide metal "Martin!" Wass'' voice came through the receivers in both their radios. Wass said, more quietly, "Remember that metal band? Martin clapped a hand on Rodney''s shoulder again, starting him moving, "No go," Martin said to Wass. Walking with Rodney, Martin heard Wass'' voice, flat and metallic Wass said coldly, "He''s right, Martin." Wass stared down at the metal street, as if he could look through it. Martin adjusted his torch, began to lead the way down the metal ramp. After a time he said, "Rodney, Wass, it''s dust, down there. Looking up, Martin saw Wass above Rodney. "All right, Wass," Martin said quietly, as Rodney released the rope and id: 63392 author: Knight, Damon title: Doorway to Kal-Jmar date: words: 6740.0 sentences: 575.0 pages: flesch: 90.0 cache: ./cache/63392.txt txt: ./txt/63392.txt summary: Syme murmured something and turned away, feeling the spaceman''s eyes Syme looked at the man, nursing the tortured muscles of his arms. "Never mind," said Syme, taking his arm. "I wanta go to Kal-Jmar," said Tate. Harold Tate told him, and later, when Syme had taken him to his rooms, Syme turned the little sand car up a gentle rise towards the tortuous Syme nodded and moved the sand car up to the edge of the gully. "Most of these gullies peter out gradually," said Syme, "but if we want "Yeah," said Syme, and opened the door. "There are many secrets in Kal-Jmar," the Martian said, "among them a into cities like Kal-Jmar; our ancestors chose to adapt their bodies to "Kal-Jmar," said Syme. just stood there, looking dumbly from the gun to Syme''s dark face. know, either, that Syme was not a Martian, or that he wanted a steak, id: 43204 author: Young, Clarence title: The Motor Boys in Mexico; Or, The Secret of the Buried City date: words: 53106.0 sentences: 4579.0 pages: flesch: 92.0 cache: ./cache/43204.txt txt: ./txt/43204.txt summary: "Run back to the machine and get one," said Jerry to Bob. "More like hanging two people with one rope," laughed Bob. But Ned''s plan was voted a good one. When Jerry and Bob had pulled Professor Snodgrass up as far as possible, "Much obliged to you, boys," he said, coming over to Bob and Jerry. "I''m going to write home now and find if I can go," said Ned. Jerry and Bob said they would do the same, and soon three letters were "He got Jerry''s money-belt," said Ned, "but----" "Good-night," said Ned and Jerry, as they locked their door, which had "It may be a whole school of fishes," said Bob. Just then there came a more violent agitation of the water, and the auto "We must get Bob, no matter what happens," said Jerry to Ned. professor, Ned and Jerry ran their machine over to where Noddy was. "And the other is Bob''s," said Jerry. ==== make-pages.sh questions ==== make-pages.sh search ==== make-pages.sh topic modeling corpus Zipping study carrel